I have a bash script that needs to connect to another server for parts of it's execution. I have tried many of the standard instructions and syntaxes for executing ssh commands, but with little progress.
On the remote server, I need to source a shell script that contains several env parameters for some software. One of these parameters are then used in a filepath to point to an executable, which contains a function ' -lprojects ' that can list the projects for the software on that server.
I have verified that running the commands on the server itself works multiple times. My issue is when I try to run the same commands over SSH. If I use the approach where I use the env variable for the filepath, it shows that the variable is null in the filepath, giving a file/directory not found error. If I hard-code the filepath to point to the executable, it gives me an error saying that the shell script is not sourced (which I assume it needs for other functions and apis for the executable to reveal it's -lprojects function)
Here is how the code looks like somewhat:
ssh remote.server 'source /filepath/remotescript.sh'
filelist=$(ssh remote.server $REMOTEVARIABLE'/bin/executable -lprojects')
echo ${filelist[#]}
for file in $filelist
do
echo $file
ssh SERVER2 awk 'something' /filepath/"$file"/somefile.txt | sed 'something' >> filepath/values.csv;
done
As you can see, I then also need to loop through the contents of the -lprojects output in the remote.server, do some awk and sed on the files to extract the wanted text (this works), but then I need to write that back to the client (local server) values.csv file. This is more generic, as there will be several servers I have to do this for, but all of them have to write to the same .csv file. For simplicity, you can just regard this as a one remote server case, since it is vital I get it working for at least one now in the beginning.
Note that I also tried something like:
ssh remote.server << EOF
'source /filepath/remotescript.sh'
filelist=$(ssh remote.server $REMOTEVARIABLE'/bin/executable -lprojects')
EOF
But with similar results. Also placing the single-quotes in the filelist both before and after the remotevariable, etc.
How do I go about properly doing this?
To access the environment variable, you must source the script that defines the environment within the same SSH call as the one where you are using it, otherwise, you're running your commands in two different shells which are unrelated:
filelist=$(ssh remote.server 'source /filepath/remotescript.sh; $REMOTEVARIABLE/bin/executable -lprojects')
Assuming executable outputs one file name per line, you can use readarray to achieve the effect :
readarray -t filelist < <(ssh remote.server '
source /filepath/remotescript.sh
$REMOTEVARIABLE/bin/executable -lprojects
'
)
echo ${filelist[#]}
for file in $filelist
do
echo $file
ssh SERVER2 awk 'something' /filepath/"$file"/somefile.txt | sed 'something' >> filepath/values.csv;
done
Related
I use a script on my personal server to setup a few things, and I'm a bit lazy.
So instead of doing every command by hand etc I got this script:
#!/bin/bash
sshservers=(IP IP IP IP)
for sshserver in "${sshservers[#]}"
do
ssh root#$sshserver 'Foo Bar'
done
echo "done"
exit
So it takes the servers from the variable in sshservers=.
But I want it to take the IP's from a file called "servers.txt".
Though when using
sshservers=$(cat servers.txt)
This doesn't work. I have to manually insert it directly in the script.
How can I use the variable from a text file?
Thanks in advance!
I am trying to login on one of the remote server(Box1) and trying to read one file on remote server(Box1).
That contain the another server(Box2) details, base upon that details I have to come back to the local server and ssh to another server(Box2) for some data crunching. and so on.....
ssh box1.com << EOF
if [[ ! -f /home/rakesh/tomar.log ]]
then
echo "LOG file not found"
else
echo " LOG file present"
export server_node1= `cat /home/rakesh/tomar.log`
fi
EOF
ssh box2.com << EOF
if [[ ! -f /home/rakesh/tomar.log ]]
then
echo "LOG file not found"
else
echo " LOG file present"
export server_node2= `cat /home/rakesh/tomar.log`
fi
EOF
but I am not getting value of "server_node1" and "server_node2" on local machine.
any help would be appreciated.
Just like bash -c 'export foo=bar' cannot declare a variable in the calling shell where you typed this, an ssh command cannot declare a variable in the calling shell. You will have to refactor so that the calling shell receives the information and knows what to do with it.
I agree with the comment that storing a log file in a variable is probably not a sane, or at least elegant, thing to do, but the easy way to do what you are attempting is to put the ssh inside the assignment.
server_node1=$(ssh box1.com cat tomar.log)
server_node2=$(ssh box2.com cat tomar.log)
A few notes and amplifications:
The remote shell will run in your home directory, so I took it out (on the assumption that /home/rt9419 is your home directory, obviously).
In case of an error in the cat command, the exit code of ssh will be the error code from cat, and the error message on standard error will be visible on your standard error, so the echo seemed quite superfluous. (If you want a custom message, variable=$(ssh whatever) || echo "Custom message" >&2 would do that. Note the redirection to standard error; it doesn't seem to matter here, but it's good form.)
If you really wanted to, you could run an arbitrarily complex command in the ssh; as outlined above, it didn't seem necessary here, but you could do assigment=$(ssh remote 'if [[ things ]]; then for variable in $(complex commands to drive a loop); do : etc etc; done; fi; more </dev/null; exit "$variable"') or whatever.
As further comments on your original attempt,
The backticks in the here document in your attempt would be evaluated by your local shell before the ssh command even ran. There are separate questions about how to fix that; see e.g. How have both local and remote variable inside an SSH command. but in short, unless you absolutely require the local shell to be able to modify the commands you send, probably put them in single quotes, like I did in the silly complex ssh example above.
The function of export is to make variables visible to child processes. There is no way to affect the environment of a parent process (short of having it cooperate and/or coordinate the change, as in the code above). As an example to illustrate the difference, if you set PERL5LIB to a directory with Perl libraries, but fail to export it, the Perl process you start will not see the variable; it is only visible to the current shell. When you export it, any Perl process you start as a child of this shell will also see this variable and the value you assigned. In other words, you export variables which are not private to the current shell (and don't export private ones; aside from making sure they are private, this saves the amount of memory which needs to be copied between processes), but that still only makes them visible to children, by the design of the U*x process architecture.
You should get back the file from box1and box2 with an scp:
scp box1.com:/home/rt9419/tomar.log ~/tomar1.log
#then you can cat!
export server_node1=`cat ~/tomar1.log`
idem with box2
scp box2.com:/home/rt9419/tomar.log ~/tomar2.log
#then you can cat!
export server_node2=`cat ~/tomar2.log`
There are several possibilities. In your case, you could on the remote system create a file (in bash syntax), containing the assignments of these variables, for example
echo "export server_node2='$(</home/rt9419/tomar.log)'" >>export_settings
(which makes me wonder why you want the whole content of your logfile be stored into a variable, but this is another question), then transfer this file to your host (for example with scp) and source it from within your bash script.
In one of my bash script I want to read and use the variable value from other script which is on remote machine.
How should I go ahead to resolve this. Any related info would be helpful.
Thanks in advance!
How about this (which is code I cannot currently test myself):
text=$(ssh yourname#yourmachine 'grep uploadRate= /root/yourscript')
It assumes that the value of the variable is contained in one line. The variable text now contains you variable assignment, presumably something like
uploadRate=1MB/s
There are several ways to convert the text/code into a real variable assignment in your current script, like evaluating the string or using grep. I would recommend
uploadRate=${text#*=}
to just remove the part up and including the =.
Edit: One more caveat to mention is that this only works if the original assignment does not contain variable references itself like in
uploadRate=1000*${kB}/s
ssh user#machine 'command'
will print the standard output of the remote command.
I would tell two ways at least:
1) You can simply redirect output to a file from remote server to your system with scp command...It would work for you.Then your script on your machine should read that file as an argument...
script on your machine:
read -t 50 -p "Waiting for argumet: " $1
It waits for output from remote machine,
Then you can
sshpass -p<password> scp user#host:/Path/to/file /path/to/script/
What you need to do:
You should tell the script from your machine, that the output from scp command is the argument($1)
2)Run script from your machine:
#!/bin/bash
script='
#Your commands
'
sshpass -p<password> ssh user#host $script
And you have also another ways to run script to do sth with remote machine.
I have a text file in which I have a list of servers. I'm trying to read the server one by one from the file, SSH in the server and execute ls to see the directory contents. My loop runs just once when I run the SSH command, however, for SCP it runs for all servers in the text file and exits, I want the loop to run till the end of text file for SSH. Following is my bash script, how can I make it run for all the servers in the text file while doing SSH?
#!/bin/bash
while read line
do
name=$line
ssh abc_def#$line "hostname; ls;"
# scp /home/zahaib/nodes/fpl_* abc_def#$line:/home/abc_def/
done < $1
I run the script as $ ./script.sh hostnames.txt
The problem with this code is that ssh starts reading data from stdin, which you intended for read line. You can tell ssh to read from something else instead, like /dev/null, to avoid eating all the other hostnames.
#!/bin/bash
while read line
do
ssh abc_def#"$line" "hostname; ls;" < /dev/null
done < "$1"
A little more direct is to use the -n flag, which tells ssh not to read from standard input.
Change your loop to a for loop:
for server in $(cat hostnames.txt); do
# do your stuff here
done
It's not parallel ssh but it works.
I open-sourced a command line tool called Overcast to make this sort of thing easier.
First you import your servers:
overcast import server.01 --ip=1.1.1.1 --ssh-key=/path/to/key
overcast import server.02 --ip=1.1.1.2 --ssh-key=/path/to/key
Once that's done you can run commands across them using wildcards, like so:
overcast run server.* hostname "ls -Al" ./scriptfile
overcast push server.* /home/zahaib/nodes/fpl_* /home/abc_def/
I would like to copy a file from a remote machine onto my local machine, up to the first line containing a certain pattern.
Scenario: update my local Bash profile with a part of the remote Bash profile, up to the point in which my admin has verified it.
Is there a better way (I guess there likely is!) than this quick "shell scripting" hack?
ssh tinosino#robottinosino-wifi cat /Users/tinosino/.profile | sed '/Verify this script further than this point/,$ d' > /home/tinosino/Desktop/tinosino_bash_profile.sh
Remote machine: robottinosino-wifi (OSX)
Sentinel line: Verify this script further than this point
I can use basic shell scripting, preferably in Bash as it's the default, or the most common diff/source-control bins..
The idea, you guess it, is to ultimately automate this process. Cron? Any idea as to how you would do this? The start of my Bash profile should come from the server, the "rest" is free for me to customise.
Prev failed attempts of mine:
using head
using process substituion <( ... )
using grep
using a local named pipe (this was fun: the named pipe needs a program generating its text though, executing something like the cat->sed line above)
Important note: what would be highly desirable is for the remote system not to go through the entire file, but to truncate the filter once it "sees" the sentinel line.. If pattern is in line #300 of 1,000,000,000.. just go over 300 lines.
The problem is that your sed command is structured to read through the entire file.
You can use sed -n '/Verify this script/q; p' to instead quit once the line is found:
ssh tinosino#robottinosino-wifi cat /Users/tinosino/.profile | sed -n '/Verify this script/q; p' > /home/tinosino/Desktop/tinosino_bash_profile.sh
Or without the useless use of cat, which doesn't make a significant difference in this case, but which will transfer less data if you want to remove multiple sections later:
ssh tinosino#robottinosino-wifi "sed -n '/Verify this script/q; p' /Users/tinosino/.profile" > /home/tinosino/Desktop/tinosino_bash_profile.sh
Just perform the filtering on the remote server.
ssh tinosino#robottinosino-wifi sed -n 'p;/Verify.../q' /Users/tinosino/.profile \
>>/home/tinosino/Desktop/tinosino_bash_profile.sh
The -n flag and the p and q commands together print only the lines up to, but not including, the first line that starts with "Verify...".