I'm writing a script that needs to pass a variable to the sftp batch. I have been able to get some commands working based on other documentation I've searched out, but can't quite get to what I need.
The end-goal is to work similar to a file test operator on a remote server:
( if [-f $a ] then:; else exit 0;)
Ultimately, I want the file to continue running the script if the file exists (:), or exit 0 if it does NOT exist (not exit 1). The remote machine is a Windows server, not Linux.
Here's what I have:
NOTE - the variable I'm trying to pass, $source_dir, changes based on the input parameter of the script that calls this function. This and the ls wildcard is the tricky part. I have been able to make it work when looking for a specific file, but not just "any" file.
${source_dir}= /this/directory/changes
RemoteCheck () {
/bin/echo "cd $source_dir" > someBatch.txt
/bin/echo "ls *" >> someBatch.txt
/usr/bin/sftp -b testBatch.txt -oPort=${sftp_port} ${sftp_ip}
exit_code=$?;
if [ $exit_code -eq 0 ]; then
:
else
exit 0
fi
There may be a better way to do this, but I have searched multiple forums and have not yet found a way to manipulate this.
Any help is appreciated, you gurus have always been very helpful!
You cannot test for existence of any file using just exit code of the OpenSSH sftp.
You can redirect the sftp output to a file and parse it to see if there are any files.
You can use shell echo command to delimit the listing from the rest of the output like:
!echo listing-start
ls
!echo listing-end
Related
I am quite new to bash (barely any experience at all) and I need some help with a bash script.
I am using docker-compose to create multiple containers - for this example let's say 2 containers. The 2nd container will execute a bash command, but before that, I need to check that the 1st container is operational and fully configured. Instead of using a sleep command I want to create a bash script that will be located in the 2nd container and once executed do the following:
Execute a command and log the console output in a file
Read that file and check if a String is present. The command that I will execute in the previous step will take a few seconds (5 - 10) seconds to complete and I need to read the file after it has finished executing. I suppose i can add sleep to make sure the command is finished executing or is there a better way to do this?
If the string is not present I want to execute the same command again until I find the String I am looking for
Once I find the string I am looking for I want to exit the loop and execute a different command
I found out how to do this in Java, but if I need to do this in a bash script.
The docker-containers have alpine as an operating system, but I updated the Dockerfile to install bash.
I tried this solution, but it does not work.
#!/bin/bash
[command to be executed] > allout.txt 2>&1
until
tail -n 0 -F /path/to/file | \
while read LINE
do
if echo "$LINE" | grep -q $string
then
echo -e "$string found in the console output"
fi
done
do
echo "String is not present. Executing command again"
sleep 5
[command to be executed] > allout.txt 2>&1
done
echo -e "String is found"
In your docker-compose file make use of depends_on option.
depends_on will take care of startup and shutdown sequence of your multiple containers.
But it does not check whether a container is ready before moving to another container startup. To handle this scenario check this out.
As described in this link,
You can use tools such as wait-for-it, dockerize, or sh-compatible wait-for. These are small wrapper scripts which you can include in your application’s image to poll a given host and port until it’s accepting TCP connections.
OR
Alternatively, write your own wrapper script to perform a more application-specific health check.
In case you don't want to make use of above tools then check this out. Here they use a combination of HEALTHCHECK and service_healthy condition as shown here. For complete example check this.
Just:
while :; do
# 1. Execute a command and log the console output in a file
command > output.log
# TODO: handle errors, etc.
# 2. Read that file and check if a String is present.
if grep -q "searched_string" output.log; then
# Once I find the string I am looking for I want to exit the loop
break;
fi
# 3. If the string is not present I want to execute the same command again until I find the String I am looking for
# add ex. sleep 0.1 for the loop to delay a little bit, not to use 100% cpu
done
# ...and execute a different command
different_command
You can timeout a command with timeout.
Notes:
colon is a utility that returns a zero exit status, much like true, I prefer while : instead of while true, they mean the same.
The code presented should work in any posix shell.
i have a Shell script doing the following two commands, connecting to a remote server and putting files via SFTP, lets called it "execute.sh"
sftp -b /usr/local/outbox/send.sh username#example.com
mv /usr/local/outbox/DD* /usr/local/outbox/completed/
Then in my "send.sh" i have following commands to be executed on the remote server.
cd ExampleFolder/outbox
put Files_*
bye
Now my problem is
If the first command "sftp -b" fails due to a remote connection error some network problem, it still moves the files into the "completed folder" which is incorrect, so i want some way to do the next command "mv" to be executed only if the first command to "sftp" is successfully connected.
Can we do this by enhancing this shell script ? or some work around ?
My Shell is Bash.
Simply insert && between the two commands:
sftp -b /usr/local/outbox/send.sh username#example.com && \
mv /usr/local/outbox/DD* /usr/local/outbox/completed/
If the first fails, the second one will not run.
Alternatively, you can check the exit code of the first command explicitly. The exit code of the last command is always saved in $?, and it is 0 if the command succeeded:
sftp -b /usr/local/outbox/send.sh username#example.com
if [ $? -eq 0 ]
then
mv /usr/local/outbox/DD* /usr/local/outbox/completed/
fi
If you really wanted to capture the output of the first command, you could run it in $(...) and store the value in a variable:
sftpOutput="$(sftp -b /usr/local/outbox/send.sh username#example.com)"
and then use this variable in further checks, e.g. match it against a pattern in the next if.
I am trying to login on one of the remote server(Box1) and trying to read one file on remote server(Box1).
That contain the another server(Box2) details, base upon that details I have to come back to the local server and ssh to another server(Box2) for some data crunching. and so on.....
ssh box1.com << EOF
if [[ ! -f /home/rakesh/tomar.log ]]
then
echo "LOG file not found"
else
echo " LOG file present"
export server_node1= `cat /home/rakesh/tomar.log`
fi
EOF
ssh box2.com << EOF
if [[ ! -f /home/rakesh/tomar.log ]]
then
echo "LOG file not found"
else
echo " LOG file present"
export server_node2= `cat /home/rakesh/tomar.log`
fi
EOF
but I am not getting value of "server_node1" and "server_node2" on local machine.
any help would be appreciated.
Just like bash -c 'export foo=bar' cannot declare a variable in the calling shell where you typed this, an ssh command cannot declare a variable in the calling shell. You will have to refactor so that the calling shell receives the information and knows what to do with it.
I agree with the comment that storing a log file in a variable is probably not a sane, or at least elegant, thing to do, but the easy way to do what you are attempting is to put the ssh inside the assignment.
server_node1=$(ssh box1.com cat tomar.log)
server_node2=$(ssh box2.com cat tomar.log)
A few notes and amplifications:
The remote shell will run in your home directory, so I took it out (on the assumption that /home/rt9419 is your home directory, obviously).
In case of an error in the cat command, the exit code of ssh will be the error code from cat, and the error message on standard error will be visible on your standard error, so the echo seemed quite superfluous. (If you want a custom message, variable=$(ssh whatever) || echo "Custom message" >&2 would do that. Note the redirection to standard error; it doesn't seem to matter here, but it's good form.)
If you really wanted to, you could run an arbitrarily complex command in the ssh; as outlined above, it didn't seem necessary here, but you could do assigment=$(ssh remote 'if [[ things ]]; then for variable in $(complex commands to drive a loop); do : etc etc; done; fi; more </dev/null; exit "$variable"') or whatever.
As further comments on your original attempt,
The backticks in the here document in your attempt would be evaluated by your local shell before the ssh command even ran. There are separate questions about how to fix that; see e.g. How have both local and remote variable inside an SSH command. but in short, unless you absolutely require the local shell to be able to modify the commands you send, probably put them in single quotes, like I did in the silly complex ssh example above.
The function of export is to make variables visible to child processes. There is no way to affect the environment of a parent process (short of having it cooperate and/or coordinate the change, as in the code above). As an example to illustrate the difference, if you set PERL5LIB to a directory with Perl libraries, but fail to export it, the Perl process you start will not see the variable; it is only visible to the current shell. When you export it, any Perl process you start as a child of this shell will also see this variable and the value you assigned. In other words, you export variables which are not private to the current shell (and don't export private ones; aside from making sure they are private, this saves the amount of memory which needs to be copied between processes), but that still only makes them visible to children, by the design of the U*x process architecture.
You should get back the file from box1and box2 with an scp:
scp box1.com:/home/rt9419/tomar.log ~/tomar1.log
#then you can cat!
export server_node1=`cat ~/tomar1.log`
idem with box2
scp box2.com:/home/rt9419/tomar.log ~/tomar2.log
#then you can cat!
export server_node2=`cat ~/tomar2.log`
There are several possibilities. In your case, you could on the remote system create a file (in bash syntax), containing the assignments of these variables, for example
echo "export server_node2='$(</home/rt9419/tomar.log)'" >>export_settings
(which makes me wonder why you want the whole content of your logfile be stored into a variable, but this is another question), then transfer this file to your host (for example with scp) and source it from within your bash script.
I want to connect to a remote host using telnet
there is no username/password verification
just
telnet remotehost
then I need to input some commands for initialization
and then I need to repeat the following commands:
cmd argument
argument is read from a local file, in this file there are many lines, each line is a argument
and after runing one "cmd argument", the remote host will output some results
it may output a line with string "OK"
or output many lines, one of which is with string "ERROR"
and I need to do something according to the results.
basically, the script is like:
initialization_cmd #some initial comands
while read line
do
cmd $line
#here the remote host will output results, how can I put the results into a variable?
# here I want to judge the results, like
if $results contain "OK";then
echo $line >>good_result_log
else
echo $line >> bad_result_log
fi
done < local_file
the good_result_log and bad_result_log are local files
is it possible or not? thanks!
This won't work as echo will output to the stdout of the tty and not to the stdin of the telnet process.
I would suggest writing an expect script for this task. Perhaps you could adapt something like this.
This question was asked in at least four different forums at the same time. Don't know what kind of points this kind of entrepreneurship earns, but here are links to answers:
linux forums
unix.com
superuser.com
I've only been writing actual .sh scripts since sometime this morning, and I'm a bit stuck. I'm trying to write a script to check to see if a process is running, and to start it if it isn't. (I plan to run this script once every 10 to 15 minutes with cron.)
Here's what I have so far:
#!/bin/bash
APPCHK=$(ps aux | grep -c "/usr/bin/rsync -rvz -e ssh /home/e-smith/files/ibays/drive-i/files/Warehouse\ Pics/organized_pics imgserv#192.168.0.140:~/webapps/pavlick_container/public/images
")
RUNSYNC=$(rsync -rvz -e ssh /home/e-smith/files/ibays/drive-i/files/Warehouse\ Pics/organized_pics imgserv#192.168.0.140:~/webapps/pavlick_container/public/images)
if [ $APPCHK < '2' ];
then
$RUNSYNC
fi
exit
Here's the error that I'm getting:
$ ./image_sync.sh
rsync: mkdir "/home/i/webapps/pavlick_container/public/images" failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(595) [Receiver=3.0.7]
rsync: connection unexpectedly closed (9 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7]
./image_sync.sh: line 8: 2: No such file or directory
TRTWF is that
rsync -rvz -e ssh /home/e-smith/files/ibays/drive-i/files/Warehouse\ Pics/organized_pics imgserv#192.168.0.140:~/webapps/pavlick_container/public/images
runs just fine from a terminal window.
What am I doing wrong?
Your grep call is wrong on two counts. The pattern shouldn't include a newline. To look for an exact string, use grep -F 'substring' or grep -xF 'exact whole line'.
Finding if a process is running with ps | grep is highly brittle. On most unices (at least Solaris, Linux and *BSD), use pgrep: pgrep -f 'PATTERN' returns true if there's a running process whose command line matches PATTERN.
Every program returns a status code, either 0 to indicate success or a number between 1 and 255 to indicate failure. In the shell, any command is a valid boolean expression; the status code 0 is treated as true and anything else as false.
$(…) means run the command inside the parentheses and capture its output. So rsync is executed as soon as the shell hits the definition of the RUNSYNC variable. To store a block of shell code, use a function (example below, although you don't actually need a function here, you could just write the code directly).
Your test [ $APPCHK < 2 ] should be [ $APPCHK -lt 2 ]: < means input redirection. (In bash, you can also write [[ foo < bar ]], but that's string comparison, not numeric comparison.)
~/ at the beginning of the remote rsync path is optional. Also, -e ssh is the default unless your version of rsync is really old.
exit at the end of the script is useless, the script will exit anyway.
Here's a script taking the above into account:
#!/bin/bash
run_rsync () {
rsync -rvz '/home/e-smith/files/ibays/drive-i/files/Warehouse Pics/organized_pics' \
imgserv#192.168.0.140:webapps/pavlick_container/public/images
}
process_pattern='/usr/bin/rsync -rvz /home/e-smith/files/ibays/drive-i/files/Warehouse Pics/organized_pics imgserv#192\.168\.0\.140:webapps/pavlick_container/public/images'
if pgrep -xF "$process_pattern"; then
run_rsync
fi
Looks like with your rsync command that some directory along this path is wrong: ~/webapps/pavlick_container/public/images
Have you checked on the server 192.168.0.140 in imgserv's home directory to see if "pavlick_container/public" exists? That's my guess.
You have a number of problems. First you are running the commands instead of putting the commands in variables. There is also a much easier way.
RUNSYNC="rsync -rvz -e ssh /home/e-smith/files/ibays/drive-i/files/Warehouse\ Pics/organized_pics imgserv#192.168.0.140:~/webapps/pavlick_container/public/images"
if ! pgrep -f "rsync.*organized_pics"; then $RUNSYNC; fi
First of all, the way of checking if the program is running is mostly wrong. This may or may not work. You should rely on some special file you create when your script starts, that it is deleted when your script ends. This will tell you if the script is running, just checking if this file exists.
Then, try to either put a \ before the ~ or to remove the ~/ completely. If cron is run as other user, the tilde will be substituted in the client for the user directory. It works for the command line because maybe the home directory of your user in both machines match, but not in the user the cron is running. A guess at this point, but again, try to remove the ~/ and see if it works.
If your real code is missing a closing dlb-quote on the grep target, you're going to get weird results from the get-go.
Also, ps aux will not list a complete command line result like you show (at least on all the the pss I have used).
You need to make it ps auxwww. Often you will see people add | grep -v grep | (you'll see why at some point). This can be reduced to changing your static search target slightly like "/usr/bin/rsync" to "/usr/bin/[r]sync ".
Other users are also helping with their comments. Using a flag file as #DiegoSevilla mentions is marginally deprecated. use a mkdir /tmp/MyWatcher_flagDir for your flag. Directory creation is an atomic activity (where as file creations are not), and this will eliminate any errors you might encounter from having 2 copies of you monitor try to make a flag file at the same time. Only one process will succeed in making or removing a flag dir.
I hope this helps.