I have two shell scripts .
(working one)
$ cat script_nas.sh
#!/bin/bash
for i in `cat nas_filers`
do echo $i
touch /mnt/config-backup/nas_backup/$i.auditlog.0.$(date '+%Y%m%d')
ssh -o ConnectTimeout=5 root#$i rdfile /etc/configs/config_saved > /mnt/config-backup/nas_backup/$i.auditlog.0.$(date '+%Y%m%d')
done
other
(not working one)
$ cat script_san.sh
#!/bin/bash
for i in `cat san_filers`
do echo $i
touch /mnt/config-backup/san_backup/$i.auditlog.0.$(date '+%Y%m%d')
ssh -o ConnectTimeout=5 root#$i rdfile /etc/configs/config_saved > /mnt/config-backup/san_backup/$i.auditlog.0.$(date '+%Y%m%d')
done
Cron entries are:
$ crontab -l
Filers config save script
0 0 * * * /mnt/config-backup/script_san.sh
0 0 * * * /mnt/config-backup/script_nas.sh
0 0 * * * /mnt/config-backup/delete_file
Script script_san.sh is not working.
Outputs are like
SAN backup directory
san_backup]# ls -lart alln01-na-exch01a.cisco.com.auditlog*
-rw-r--r-- 1 root root 210083 Mar 1 22:24 alln01-na-exch01a.auditlog.0.20150301
[root#XXXXX san_backup]# pwd
/mnt/config-backup/san_backup
NAS backup directory
nas_backup]# ls -lart rcdn9-25f-filer43b.cisco.com.auditlog*
-rw-r--r-- 1 root root 278730 Feb 26 00:06 rcdn9-25f-filer43b.cisco.com.auditlog.0.20150226
-rw-r--r-- 1 root root 281612 Feb 27 00:17 rcdn9-25f-filer43b.cisco.com.auditlog.0.20150227
-rw-r--r-- 1 root root 284105 Feb 28 00:02 rcdn9-25f-filer43b.cisco.com.auditlog.0.20150228
-rw-r--r-- 1 root root 284101 Mar 1 00:02 rcdn9-25f-filer43b.cisco.com.auditlog.0.20150301
[root#XXXXXXX nas_backup]#
From cron logs I can see that cron is executing both the script but output for script_san.sh is not coming.
From my experience, most of the times script is working manually but not from crontab is because login scripts were not running. Try to add something like source ~/.bash_profile in the begging of script or first line in cron file. Did you try (for debugging) to run the script with at command?
Related
I have this script ftp.sh in /amadeus folder:
#!/bin/bash
. /home/thierry/.bash_profile
# constantes
HOST=xxxx
LOGIN=xxxxx
PASSWORD=xxxxxx
PORT=21
ftp -i -n $HOST $PORT << END_SCRIPT
quote USER $LOGIN
quote PASS $PASSWORD
cd www
pwd
asc
mput *.json
quit
END_SCRIPT
With the same user this command works fine:
$ . /amadeus/ftp.sh
=> a json file is well putted to the remote server.
But with this entry in crontab:
* * * * * . /amadeus/ftp.sh 2>/tmp/2.log 1>/tmp/1.log
I have files 1.log and 2.log created every second but none json file in remote server.
tmp]$ ll
-rw-r--r-- 1 thierry thierry 36 Jan 27 16:00 1.log
-rw-r--r-- 1 thierry thierry 0 Jan 27 16:00 2.log
tmp]$ more 1.log
257 "/www" is your current location
tmp]$ more 2.log
tmp]$
Thanks a lot for yours answers!
Théo.
Before ftp command, I had to do a "cd" to the right folder /amadeus and now it works fine!
I have a scenario to automate the manual build update process via shell script on multiple VM nodes.
For the same, I am trying the below sample script to first ssh into the instance and then switch to root user to perform the further steps like copying the build to archives directory under /var and then proceed with the later steps.
Below is the sample script,
#!/bin/sh
publicKey='/path/to/publickey'
buildVersion='deb9.deb build'
buildPathToStore='/var/cache/apt/archives/'
pathToHomedir='/home'
script="whoami && pwd && ls -la && whoami && mv ${buildVersion} ${buildPathToStore} && find ${buildPathToStore} | grep deb9"
for var in "$#"
do
copyBuildPath="${publicKey} ${buildVersion} ${var}:/home/admin/"
echo "copy build ==>" ${copyBuildPath}
scp -r -i ${copyBuildPath}
ssh -i $publicKey -t $var "sudo su - & ${script}; " # This shall execute all commands as root
done
So the CLI stats for the above script are something like this
admin //this is the user check
/home/admin
total 48
drwxr-xr-x 6 admin admin 4096 Dec 6 00:28 .
drwxr-xr-x 6 root root 4096 Nov 17 14:07 ..
drwxr-xr-x 3 admin admin 4096 Nov 17 14:00 .ansible
drwx------ 2 admin admin 4096 Nov 23 18:26 .appdata
-rw------- 1 admin admin 5002 Dec 6 17:47 .bash_history
-rw-r--r-- 1 admin admin 220 May 16 2017 .bash_logout
-rw-r--r-- 1 admin admin 3506 Jun 14 2019 .bashrc
-rw-r--r-- 1 admin admin 675 May 16 2017 .profile
drwx------ 4 admin admin 4096 Nov 23 18:26 .registry
drwx------ 2 admin admin 4096 Jun 21 2019 .ssh
-rw-r--r-- 1 admin admin 0 Dec 6 19:42 testFile.txt
-rw------- 1 admin admin 2236 Jun 21 2019 .viminfo
admin
If I use sudo su -c and remove &
like:
ssh -i $publicKey -t $var "sudo su -c ${script}; "
Then for once whoami returns the user as root but the working directory still prints as /home/admin instead of /root
And the next set of commands are still accounted for admin user rather than the root. So the admin user do not have the privileges to move the build to archive directory and install the build.
Using & I want to ensure that the further steps are being done in the background.
Not sure how to proceed ahead with this. Good suggestions are most welcome right now :)
"sudo su - & ${script}; "
expands to:
sudo su - & whoami && pwd && ...
First sudo su - is run in the background. Then the command chain is executed.
sudo su -c ${script};
expands to:
sudo su -c whoami && pwd && ...
So first sudo su - whoami is executed, which runs whoami as root. Then if this command is successful, then pwd is executed. As normal user.
It is utterly hard to correctly pass commands to execute on remote site using ssh. It is increasingly hard to do it with sudo su - the command will be triple (or twice?) word splitted - one time by ssh, then by the shell, then by the shell run by sudo su.
If you do not need interactive communication, it's best to use a here document with -s shell option, something along (untested):
# DO NOT store commands to use in a variable.
# or if you do and you know what you are doing, properly quote it (printf "%q ") and run it via eval
script() {
set -euo pipefail
whoami
pwd
ls -la
whoami
mv "$buildVersion" "$buildPathToStore"
find "$buildPathToStore" | grep deb9
}
ssh ... "sudo bash -s" <<EOF
echo "Yay! anything here!"
echo "Note that here document delimiter is not quoted!"
$(
# safely import context to work with
# note how command substitution is executed on host side
declare -f script
# pass variables too!
declare -p buildVersion buildPathToStore buildPathToStore
)
script
EOF
When you use su alone it keeps you in your actual directory, if you use su - it simulates the root login.
You should write : su - root -c ${script};
When I run a bash file inside its directory in ubuntu there isn't any problem
But when I try to run it by its directory I get the error:
sh: 0: Can't open /directory/file.sh
I followed this steps:
$ cat web/test.sh
#!/bin/bash
touch /tmp/testweb
echo "File has been created succesfully"
$ ls -la web/test.sh
-rwxr--r-- 1 test test 88 Dec 5 05:58 web/test.sh
1- $ date
Sat 05 Dec 2020 06:00:14 AM UTC
2- $ /home/test-env/web/test.sh
File has been created succesfully
3- $ ls -la /tmp/test*
-rw-r--r-- 1 test test 0 Dec 5 06:00 /tmp/testweb
Able to run the file in ubuntu and debian as well
I'm trying to iterate over url entries in a file and use each file as an input for a crawler tool. It's result should be written to a file.
here is the gitlab-ci.yml file:
stages:
- test
test:
stage: test
tags:
- shell-docker
script:
- wget https://github.com/FaKeller/sireg/releases/download/v0.3.1/sireg-linux
- chmod 775 sireg-linux
- mkdir output
- ls -alF
- while read line; do
echo $line;
./sireg-linux exec --loader-sitemap-sitemap \"$line\" >> ./output/${line##*/}_out.txt;
done < sitemap-index
- ls -alF output
artifacts:
paths:
- output/*
expire_in: 1 hrs
and here is the sitemap-index file (only one entry):
http://example.com/sitemap.xml
both files are in the same directory. I expect a file sitemap.xml_out.txt to be written into the output folder(also the same directory). I am pretty sure the ./sireg-linux script does not execute because it usually takes few minutes to complete (tested locally).
the output of the stage looks like this:
2020-04-02 18:22:21 (4,26 MB/s) - »sireg-linux« saved [62566347/62566347]
$ chmod 775 sireg-linux
$ mkdir output
$ ls -alF
total 61128
drwxrwxr-x 4 gitlab-runner gitlab-runner 4096 Apr 2 18:22 ./
drwxrwxr-x 10 gitlab-runner gitlab-runner 4096 Apr 2 15:46 ../
drwxrwxr-x 5 gitlab-runner gitlab-runner 4096 Apr 2 18:22 .git/
-rw-rw-r-- 1 gitlab-runner gitlab-runner 512 Apr 2 18:22 .gitlab-ci.yml
drwxrwxr-x 2 gitlab-runner gitlab-runner 4096 Apr 2 18:22 output/
-rw-rw-r-- 1 gitlab-runner gitlab-runner 30 Apr 2 15:46 README.md
-rwxrwxr-x 1 gitlab-runner gitlab-runner 62566347 Nov 11 2017 sireg-linux*
-rw-rw-r-- 1 gitlab-runner gitlab-runner 55 Apr 2 18:08 sitemap-index
$ while read line; do echo $line; ./sireg-linux **exec** --loader-sitemap-sitemap \"$line\" >>
./output/${line##*/}_out.txt; done < sitemap-index
$ ls -alF output
total 8
drwxrwxr-x 2 gitlab-runner gitlab-runner 4096 Apr 2 18:22 ./
drwxrwxr-x 4 gitlab-runner gitlab-runner 4096 Apr 2 18:22 ../
Uploading artifacts...
Runtime platform arch=amd64 os=linux pid=23813 revision=1f513601 version=11.10.1
WARNING: output/*: no matching files
ERROR: No files to upload
Job succeeded
update
tried to move all steps into a separate script but that did not work either.
update 2
forgot to add exec in the command:
./sireg-linux exec --loader-sitemap-sitemap \"$line\" >>
./output/${line##*/}_out.txt;
unfortunately it didn't help.
what can I do to make it working?
Try changing ./sireg-linux --loader-sitemap-sitemap \"$line\" to ./sireg-linux exec --loader-sitemap-sitemap "$line". Hope this helps!
EDIT: Also, it looks like the script doesn't enter the while loop at all. Maybe the file sitemap-index is empty or it has only one line without a newline at the end?
EDIT 2: The back-slashes in the command line are wrong. corrected my answer
You can of course painfully debug multi-line commands in YAML.
You can even use YAML multi-line strings:
How do I break a string over multiple lines?
https://gitlab.com/snippets/1717579
But I would just wrap code into a shell script, store it in the same GitLab repo, and just call it in .gitlab-ci.yml.
This way you can run this script exactly the same way both locally and in CI, which is a best practice in Continuous Delivery.
- ./script.sh
I am trying to get alias's setup so that they print out the command, then run the command.
Ex:
> alias ls='ls -alh'
> ls
Running "ls -alh"
total 1.8G
drwxr-x--- 36 root root 4.0K Apr 23 09:44 ./
drwxr-xr-x 28 root root 4.0K Mar 6 17:24 ../
Is this possible? I was thinking of using a wrapper function, but I am unsure as to how one would acomplish this.
Thanks!
Just add an echo command in your alias before the actual command:
alias ls='echo "Running ls -alh"; ls -alh'
alias ls='echo "Running ls -alh" && ls -alh'
This runs two commands one after the other. The first command is echo "Running ls -alh", the && checks the return value of the echo command, if that's 0, then the command ls -alh is run. However, if for some reason there is a problem with the echo command and its return value is not 0 then the ls command won't be run.
The && command can come in very handy when writing scripts to run one command only when another is successful.