I want to run a remote program via ssh which requires a certain environment. Thus before executing the program I source a specific file building up the environment. If I'm logged onto the machine directly this is no problem but when I execute the command via ssh
#!/bin/bash
foo=`ssh user#host "source ~/script.sh; ~/run/program"`
I get an error that indicates that the script was not sourced correctly. Do you know what I have to do in order to get the script sourced and the program executed in the same session?
EDIT:
I'm exporting the LD_LIBRARY_PATH with the script and the executable is complaining that it cannot find the shared object file. The default shell is bash. 'Session' is definitive not the right wording. I meant 'terminal environment'.
This may not be the cleanest way, but if you invoke bash with the interactive option (-i) and send commands through the standard input, it should work.
In particular,
foo=`ssh user#host bash -i <<EOF
source ~/script.sh
~/run/program
EOF`
It would be much easier if you have a script program_in_env.sh that does exactly the two steps you want:
#!/bin/bash
source ~/script.sh
~/run/program
Then you would just need to call ssh user#host program_in_env.sh.
Good luck.
Thank you for all your time and help. I found the issue. The basic idea of how to execute the remote program was right from the beginning. When testing my case locally on the machine, the current working directory was different. For some reason the cwd is important when sourcing this bash script.
Related
If I log in to my remote Mac via ssh -p22 jenkins#192.168.2.220 and type docker, it finds the executable because it also finds the path /usr/local/bin if I check with echo $PATH. But if I do the same in a heredoc inside a file setup-mac.sh like
#!/bin/bash
ssh jenkins#192.168.2.220 '/bin/bash -s' << 'EOF'
"echo $PATH"
"bash run-docker.sh"
EOF
which I execute via shell and bash setup-mac.sh it does not find /usr/local/bin in PATH and consequently does not run docker, because the command is unknown.
On the remote Mac, there is a file run-docker.sh which is a bash file that calls docker commands, and it works if called locally.
To solve this issue, I've enabled PermitUserEnvironment on the mac in sshd_config, but this did not work. Though I only restarted ssh service and not the whole machine. Meanwhile I've changed all docker commands on the remote run-docker.sh script to an alias ${DOCKER} and I initialize it at the beginning of the script to DOCKER=/usr/local/bin/docker, but this is only a workaround.
I guess that the problem is occurring because /usr/local/bin is being added to the PATH by the 'jenkins' user's personal initialization file (~/.bashrc). That is run only by interactive shells, and the shell run by ssh ... '/bin/bash -s' ... is not interactive.
You could try forcing the shell to be interactive by invoking it with /bin/bash -i -s, but that is likely to cause other problems. (The shell may try and fail to set up job control. The value of PS1 may appear in outputs. ...)
In general, you can't rely on the PATH being set correctly for programs. See Setting the PATH in Scripts - Scripting OS X for a thorough analysis of the problem. Correct way to use Linux commands in bash script is also relevant, but doesn't have much information.
A simple and reliable way to fix the problem permanently is to set the required PATH explicitly at the start of run-docker.sh. For example:
export PATH=/bin:/usr/bin:/usr/local/bin
You may need to add other directories to the path if run-docker.sh runs programs that are in other locations.
Another solution to the problem is to use full paths to commands in code. However, that makes the code more difficult to read, more difficult to maintain, and more difficult to test. Setting a safe PATH is usually a better option.
I have a Perl program on server_B which uses Perl DBI and 5.010 and runs fine from the server_B terminal. I run it from a shell script which first prepares some arguments and then passes them to the Perl program, all works fine.
I need to run a shell script on server_A that will execute that script on server_B. This is because the Perl program creates several files that I want to SFTP back over to server_A. This is the script I'm running on server_A:
ssh server_B <<- EOF
perl/update.sh
EOF
There is some strange behavior which I'm trying to understand:
The script (update.sh) on server_B runs mysql, which is not installed on server_A (which is why I have to do this whole thing.) If I try to run it on server_B as-is, I can call mysql just like that. But when I run the above script (on server_A) to ssh into server_B and run that script, it doesn't recognize mysql unless I change the file (on server_B) to call the full path /opt/mysql/client/bin/mysql (even though that file is already on server_B with mysql installed) Does this mean server_B is picking up on the PATH variable from server_A instead of using my PATH variable from server_B? Is it trying to run my programs from server_A on the files on server_B? How and why??
If I make the change above it executes the script, but when it hits Perl it says
Perl v5.10.0- required - this is only v5.8.8
Again, 5.10 works fine on server_B but the version of Perl on server_A is 5.8.8.
So I got rid of use 5.010; because it actually wasn't necessary, but then I have a similar problem with my modules (DBI and DBD::mysql). I get:
Can't locate DBI.pm in #INC (#INC contains.. [my Perl PATH from server_A])
at perlfile.pl line 4
I was expecting the ssh heredoc call to update.sh (from server_A) to run exactly as update.sh does if I call it on server_B, but instead it seems like it's trying to use my programs from server_A on server_B, which I find weird. Can anyone help me understand why it's happening? I feel like I'm misunderstanding something fundamental about how ssh works.
server_A is AIX with ksh
server_B is AIX with bash
Edit - since some of you voted to the effect that I haven't done my research, here's what else I've tried. I didn't mention because I don't understand them fully, these are just guesses based on other SO posts & hunches. It'd be disingenuous if I gave the impression I knew what I was talking about.
If this is a duplicate, which question should I be looking at? If this is a "just read the manual situation", which one? What should I look for?
Read man ssh looking for clues related to environment variables, didn't find anything I understood
Tried running with -t
Tried running with -t -t
Did log in remotely with ssh and manually running it - this DOES work
Sourced my .bash_profile in the update script
Tried to re-assign PATH as the remote server's PATH when ssh
Tried using a different delimiter for the heredoc
Tried < instead of <<
Tried without the "-"
Edit 2 with Saigo's help below I determined that when in interactive ssh, if I echo $PATH I do get the target server's $PATH, but in a shell script I don't. That led me to this:
https://serverfault.com/questions/643333/different-bash-path-variables-when-using-ssh-script-vs-interactive-ssh
where I found out that scripted ssh doesn't call .bashrc, but interactive ssh does. So it looks like I was on the right track trying to source .bash_profile inside the scripted SSH heredoc, just need .bashrc not .bash_profile - however I don't have a .bashrc on the target server. I do have .profile but when I source that, I get an error stating it's for interactive bash sessions only. So now I'm just trying to find whatever file would contain my $PATH variable because it's apparently not .bashrc as there isn't one in there.
Edit 3 - tried hard-coding the PATH variable into a file and sourcing that and even then when I echo $PATH I get the origin server's PATH variable. It is reading the file in correctly, I also assigned another test variable and echoed that as part of the script. I tried sourcing /etc/profile and no luck.
I found a solution that works perfectly. I wasn't able to get it to work with ~/.bashrc, ~/.bash_profile, or ~/.ssh/rc but still not sure why it's not picking up my environment variables even with sourcing these.
Since it works when I manually ssh in and then run the commands one-by-one, I used these arguments to run ssh in a forced interactive login.
ssh server_B bash --login -i "~/perl/update.sh"
See these for more:
https://superuser.com/questions/564926/profile-is-not-loaded-when-using-ssh-ubuntu
https://unix.stackexchange.com/questions/46143/why-bash-unable-to-find-command-even-if-path-is-specified-properly
Hope this is useful for someone in the future. Thank you for your assistance Saigo.
I'm pretty decent with bash and UNIX commands, and the Terminal app, but I'm wondering, is there a way to make a "macro" (maybe that's the wrong word!) to automate some tasks?
For example, to get into my current project directory, I type:
$ cd ~/Documents/College/F13/CS362/lab3/os-lab-3
And then I immediately do
$ hg pull
(password)
And then
$ hg update
So is there a way to even automate that first step of the cd call? I've never tried anything like that before, not sure if what I'm going for is even possible.
Bonus: is there a way to have the macro enter my password when prompted as well? Security/privacy is not really an issue here, there's no thermonuclear codes hiding around in the repo.
The name generally used in this context (a macro in bash) is shell script. To automate the commands from your post you would need to create a file with a name like 'myscript.sh' with the following contents:
#!/bin/bash
cd ~/Documents/College/F13/CS362/lab3/os-lab-3
hg pull
hg update
The first line of the script is a hashbang. This is a special comment line which indicates that file should be executed using the indicated program. Bash in this case.
To run the script first ensure that it is executable by running the following command:
chmod +x myscript.sh
From there in terminal window just run the script by specifying the path to it:
./myscript.sh
Don't put passwords into shell scripts though! It is a bad habit. For authentication with Mercurial an ssh key pair is the best way to go. The ssh key can be loaded into OSX's keychain, so you won't have to type anything when you run the script. You'll also need to add the public portion of your ssh key pair to the list of authorized hosts on the remote Mercurial repository.
I tried activating a VirtualEnv through a shell script like the one below but it doesn't seem to work,
#!/bin/sh
source ~/.virtualenvs/pinax-env/bin/activate
I get the following error
$ sh virtualenv_activate.sh
virtualenv_activate.sh: 2: source: not found
but if I enter the same command on terminal it seems to work
$ source ~/.virtualenvs/pinax-env/bin/activate
(pinax-env)gautam#Aspirebuntu:$
So I changed the shell script to
#!/bin/bash
source ~/.virtualenvs/pinax-env/bin/activate
as suggested and used
$ bash virtualenv_activate.sh
gautam#Aspirebuntu:$
to run the script .
That doesn't throw an error but neither does that activate the virtual env
So any suggestion on how to solve this problem ?
PS : I am using Ubuntu 11.04
TLDR
Must run the .sh script with source instead of the script solely
source your-script.sh
and not
your-script.sh
Details
sh is not the same as bash (although some systems simply link sh to bash, so running sh actually runs bash). You can think of sh as a watered down version of bash. One thing that bash has that sh does not is the "source" command. This is why you're getting that error... source runs fine in your bash shell. But when you start your script using sh, you run the script in an shell in a subprocess. Since that script is running in sh, "source" is not found.
The solution is to run the script in bash instead. Change the first line to...
#!/bin/bash
Then run with...
./virtualenv_activate.sh
...or...
/bin/bash virtualenv_activate.sh
Edit:
If you want the activation of the virtualenv to change the shell that you call the script from, you need to use the "source" or "dot operator". This ensures that the script is run in the current shell (and therefore changes the current environment)...
source virtualenv_activate.sh
...or...
. virtualenv_activate.sh
As a side note, this is why virtualenv always says you need to use "source" to run it's activate script.
source is an builtin shell command in bash, and is not available in sh. If i remember correctly then virtual env does a lot of path and environment variables manipulation. Even running it as bash virtualenv_blah.sh wont work since this will simply create the environment inside the sub-shell.
Try . virtualenv_activate.sh or source virtualenv_activate.sh this basically gets the script to run in your current environment and all the environment variables modified by virtualenv's activate will be available.
HTH.
Edit: Here is a link that might help - http://ss64.com/bash/period.html
On Mac OS X your proposals seems not working.
I have done it this way. I'am not very happy with solution, but share it anyway here and hope, that maybe somebody will suggest the better one:
In activate.sh I have
echo 'source /Users/andi/.virtualenvs/data_science/bin/activate'
I give execution permissions by: chmod +x activate.sh
And I execute this way:
`./activate.sh`
Notice that there are paranthesis in form of ASCII code 96 = ` ( Grave accent )
For me best way work as below.
Create start-my-py-software.sh and pest below code
#!/bin/bash
source "/home/snippetbucket.com/source/AIML-Server-CloudPlatform/bin/activate"
python --version
python /home/snippetbucket.com/source/AIML-Server-CloudPlatform/main.py
Give file permission to run like below.
chmod +x start-my-py-software.sh
Now run like below
.start-my-py-software.sh
and that's it, start my python based server or any other code.
ubuntu #18.0
In my case, Ubuntu 16.04, the methods above didn't worked well or it needs much works.
I just made a link of 'activate' script file and copy it to home folder(or $PATH accessible folder) and renamed it simple one like 'actai'.
Then in a terminal, just call 'source actai'. It worked!
I just started using a Solaris 10 (Sparc) box where I telnet in and get confronted with a very unfriendly interface (compared to the standard bash shell I use in cygwin or linux) --- the arrow keys do not work as I expect them to. Being an NIS system, changing the shell is not as easy as using the "chsh" command. And setting the SHELL environment variable in ~/.login and ~/.profile is not working for me. So I'm thinking that I may need to write a script to determine if bash is running the script and starting bash if the answer is no. My first attempt, trying to invoke /bin/bash from ~/.profile seems to work but kind of doesn't feel right. Other suggestions? And how do I tell programmatically which shell is actually executing?
You can tell what shell is running with echo $0. For example:
$ echo $0
-bash
If you're changing shell you probably want to replace the current shell process rather than be a child of it, so use exec.
Also, you want to pass bash the -l flag so it acts as if it has been called as part of the login process.
So you'll want something like:
exec bash -l
You are probably running with ksh(1) on Solaris. You have several options, read the manpage for ksh and configure it or install another shell you're more familiar with like bash. I'd personnaly recommend zsh.