I need to run a shell command from a ruby application. I'm using system() but this also applies to backticks.
When running my command, I need to load a shell script first that sets up some things so I try something like this:
system("source my_script.sh && my_command")
On my Mac laptop this works as intended but on my ubuntu server I get:
sh: 1: source: not found
I was wondering about the "sh" in there since my shell should be a bash, so tried this:
system("echo $SHELL && source my_script.sh && my_command")
Which gives me:
/bin/bash
sh: 1: source: not found
So, it is using the right shell but for some reason, source does not work.
Why? And what can I do about it?
Update
As Sergio Tulentsev pointed out, Ruby does not necessarily use the shell that is set in $SHELL.
This gave me the actual shell that ruby was using:
system("ps -p $$ | tail -1 | awk '{print $NF}'")
sh
=> true
So, it's using sh. Can I somehow force it to use bash?
You need to try adding ./ in front of the file you want to source, that should work if the subshell is bash (check $SHELL).
irb(main):003:0> system("source ./test.sh && echo $TEST && cat test.sh")
test
export TEST=test
=> true
If $SHELL is sh, then you need to do . ./test.sh instead of source ./test.sh, as the source keyword is bash only.
Or you can make sure that you are using bash, by doing:
irb(main):007:0> system("/bin/bash -c 'source ./test.sh && echo $TEST && cat test.sh'")
test
export TEST=test
=> true
As others have pointed out, Ruby uses sh for its subshells. One way to make it use bash would be something like system("/bin/bash -c '...'") which leads to all kinds of escaping problems. Instead I decided to use Open3 to spawn a "real" process, run bash in it and pipe my commands into it. Works like a charm:
require "open3"
# using bash --login to ensure the same env as usual
Open3.popen3('/usr/bin/env bash --login') do |stdin, stdout, stderr, wait_thr|
pid = wait_thr[:pid]
stdin.puts("cd some_directory")
stdin.puts("source some_script")
stdin.puts("some_command")
# don't forget to close it again
stdin.puts("exit")
# for debug purposes
stdout.each_line do |line|
puts "STDOUT: " + line
end
stdin.close
stdout.close
stderr.close
end
This may seem like a little overkill but the control it allows over the child process is actually pretty nice.
Thanks everybody for your suggestions.
Related
I'm looking for how I might allow user input in a second command in a bash statement and I'm not sure how to go about it. I'd like to be able to provide a one-liner for someone to be able to install my application, but part of that application process requires asking some questions.
The current script setup looks like:
curl <url/to/bootstrap.sh> | bash
and then boostrap.sh does:
if [ $UID -ne 0 ]; then
echo "This script requires root to run. Restarting the script under root."
exec sudo $0 "$#"
exit $?
fi
git clone <url_to_repo> /usr/local/repo/
bash /usr/local/repo/.setup/install_system.sh
which in turn calls a python3 script that asks for input.
I know that the the curl in the first line is using stdin and so that might make what I'm asking impossible and that it has to be two lines to ever work:
wget <url/to/boostrap.sh>
bash bootstrap.sh
You can restructure your script to run this way:
bash -c "$(curl -s http://0.0.0.0//test.bash 2>/dev/null)"
foo
wololo:a
a
My test.bash is really just
#!/bin/bash
echo foo
python -c 'x = raw_input("wololo:");print(x)'`
To demonstrate that stdin can be read from in this way. Sure it creates a subshell to take care of curl but it allows you to keep reading from stdin as well.
I have a host on which I don't have sudo. Its been setup with ksh, I'm too used to bash and chsh doesn't work. So I put in a /bin/bash as the first line in the .profile on the system.
So the result is, when I login to this system, it automatically gets me into bash. However, when I exit the shell, not suprisingly I land up in ksh.
Any tricks to avoid this?
Use exec to replace the current process (shell) with the new process (shell).
I recommend two steps:
if [ $SHELL != /bin/bash ]
then SHELL=/bin/bash exec /bin/bash --login
fi
Or, you can compress that to:
[ $SHELL != /bin/bash ] && SHELL=/bin/bash exec /bin/bash --login
You can then put the rest of your Bash profile after this. Note that probably you don't put a shebang on the first line - that will confuse things. Also, while testing, make sure you have a second connection (window) open so that you can adjust problems. It is annoying to get locked out by an erroneous profile.
You may write a script named myexit like this:
kill -1 $(ps | sed 1d | awk '{print $1}')
It sends the signal hang up (SIGHUP) to process attached to this terminal.
And would not affect any process started up by nohup.
The default shell on the the system is csh but I want to write a script in bash. How do I write a script that will run bash and then convert back to csh at the end.
I tried this but it doesn't work:
bash
var=Hello
echo $var
csh
The command you are looking for is exit. When typing at the keyboard use exit instead of csh to get back to csh. When you enteredcsh, that just started a new csh session on top of the csh and bash sessions already running.
%bash
$ var=Hello
$ echo $var
Hello
$ exit
exit
%
As others have said, when using a script:
#! /bin/bash
var=Hello
echo $var
exit # You don't need exit; but it's okay here.
You don't need to change shells back again. When the script is run, it will be run by a sub-shell (which exits at the end of the script), and the parent shell is unchanged. So, as already suggested, the only thing you have to do is ensure the script is run by the correct shell, and the 'shebang' is the way to do that:
#!/bin/bash
var=Hello
echo $var
That's all it takes.
Define it using the sha bang
#!/bin/bash
at the starting of your file.
In a C program I can write argv[0] and the new name shows up in a ps listing.
How can I do this in bash?
You can do it when running a new program via exec -a <newname>.
Just for the record, even though it does not exactly answer the original poster's question, this is something trivial to do with zsh:
ARGV0=emacs nethack
I've had a chance to go through the source for bash and it does not look like there is any support for writing to argv[0].
I'm assuming you've got a shell script that you wish to execute such that the script process itself has a new argv[0]. For example (I've only tested this in bash, so i'm using that, but this may work elsewhere).
#!/bin/bash
echo "process $$ here, first arg was $1"
ps -p $$
The output will be something like this:
$ ./script arg1
process 70637 here, first arg was arg1
PID TTY TIME CMD
70637 ttys003 0:00.00 /bin/bash ./script arg1
So ps shows the shell, /bin/bash in this case. Now try your interactive shell's exec -a, but in a subshell so you don't blow away the interactive shell:
$ (exec -a MyScript ./script arg1)
process 70936 here, first arg was arg1
PID TTY TIME CMD
70936 ttys008 0:00.00 /bin/bash /path/to/script arg1
Woops, still showing /bin/bash. what happened? The exec -a probably did set argv[0], but then a new instance of bash started because the operating system read #!/bin/bash at the top of your script. Ok, what if we perform the exec'ing inside the script somehow? First, we need some way of detecting whether this is the "first" execution of the script, or the second, execed instance, otherwise the second instance will exec again, and on and on in an infinite loop. Next, we need the executable to not be a file with a #!/bin/bash line at the top, to prevent the OS from changing our desired argv[0]. Here's my attempt:
$ cat ./script
#!/bin/bash
__second_instance="__second_instance_$$"
[[ -z ${!__second_instance} ]] && {
declare -x "__second_instance_$$=true"
exec -a MyScript "$SHELL" "$0" "$#"
}
echo "process $$ here, first arg was $1"
ps -p $$
Thanks to this answer, I first test for the environment variable __second_instance_$$, based on the PID (which does not change through exec) so that it won't collide with other scripts using this technique. If it's empty, I assume this is the first instance, and I export that environment variable, then exec. But, importantly, I do not exec this script, but I exec the shell binary directly, with this script ($0) as an argument, passing along all the other arguments as well ($#). The environment variable is a bit of a hack.
Now the output is this:
$ ./script arg1
process 71143 here, first arg was arg1
PID TTY TIME CMD
71143 ttys008 0:00.01 MyScript ./script arg1
That's almost there. The argv[0] is MyScript like I want, but there's that extra arg ./script in there which is a consequence of executing the shell directly (rather than via the OS's #! processing). Unfortunately, I don't know how to get any better than this.
Update for Bash 5.0
Looks like Bash 5.0 adds support for writing to special variable BASH_ARGV0, so this should become far simpler to accomplish.
(see release announcement)
( exec -a foo bash -c 'echo $0' )
ps and others inspect two things, none of which is argv0: /proc/PID/comm (for the "process name") and /proc/PID/cmdline (for the command-line). Assigning to argv0 will not change what ps shows in the CMD column, but it will change what the process usually sees as its own name (in output messages, for example).
To change the CMD column, write to /proc/PID/comm:
echo -n mynewname >/proc/$$/comm; ps
You cannot write to or modify /proc/PID/cmdline in any way.
Process can set their own "title" by writing to the memory area in which argv & envp are located (note that this is different than setting BASH_ARGV0). This has the side effect of changing /proc/PID/cmdline as well, which is what some daemons do in order to prettify (hide?) their command lines. libbsd's setproctitle() does exactly that, but you cannot do that in Bash without support of external tools.
I will just add that this must be possible at runtime, at least in some environments. Assigning $0 in perl on linux does change what shows up in ps. I do not know how that is implemented, however. If I can find out, i'll update this.
edit:
Based on how perl does it, it is non-trivial. I doubt there is any bask built in way at runtime but don't know for sure. You can see how perl does sets the process name at runtime.
Copy the bash executable to a different name.
You can do this in the script itself...
cp /bin/bash ./new-name
PATH=$PATH:.
exec new-name $0
If you are trying to pretend you are not a shell script you can rename the script itself to something cool or even " " (a single space) so
exec new-name " "
Will execute bash your script and appears in the ps list as just new-name.
OK so calling a script " " is a very bad idea :)
Basically, to change the name
bash script
rename bash and rename the script.
If you are worried, as Mr McDoom. apparently is, about copying a binary to a new name (which is entirely safe) you could also create a symlink
ln -s /bin/bash ./MyFunkyName
./MyFunkyName
This way, the symlink is what appears in the ps list. (again use PATH=$PATH:. if you dont want the ./)
I do this in a script:
read direc <<< $(basename `pwd`)
and I get:
Syntax error: redirection unexpected
in an ubuntu machine
/bin/bash --version
GNU bash, version 4.0.33(1)-release (x86_64-pc-linux-gnu)
while I do not get this error in another suse machine:
/bin/bash --version
GNU bash, version 3.2.39(1)-release (x86_64-suse-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
Why the error?
Does your script reference /bin/bash or /bin/sh in its hash bang line? The default system shell in Ubuntu is dash, not bash, so if you have #!/bin/sh then your script will be using a different shell than you expect. Dash does not have the <<< redirection operator.
Make sure the shebang line is:
#!/bin/bash
or
#!/usr/bin/env bash
And run the script with:
$ ./script.sh
Do not run it with an explicit sh as that will ignore the shebang:
$ sh ./script.sh # Don't do this!
If you're using the following to run your script:
sudo sh ./script.sh
Then you'll want to use the following instead:
sudo bash ./script.sh
The reason for this is that Bash is not the default shell for Ubuntu. So, if you use "sh" then it will just use the default shell; which is actually Dash. This will happen regardless if you have #!/bin/bash at the top of your script. As a result, you will need to explicitly specify to use bash as shown above, and your script should run at expected.
Dash doesn't support redirects the same as Bash.
Docker:
I was getting this problem from my Dockerfile as I had:
RUN bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
However, according to this issue, it was solved:
The exec form makes it possible to avoid shell string munging, and
to RUN commands using a base image that does not contain /bin/sh.
Note
To use a different shell, other than /bin/sh, use the exec form
passing in the desired shell. For example,
RUN ["/bin/bash", "-c", "echo hello"]
Solution:
RUN ["/bin/bash", "-c", "bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)"]
Notice the quotes around each parameter.
You can get the output of that command and put it in a variable. then use heredoc. for example:
nc -l -p 80 <<< "tested like a charm";
can be written like:
nc -l -p 80 <<EOF
tested like a charm
EOF
and like this (this is what you want):
text="tested like a charm"
nc -l -p 80 <<EOF
$text
EOF
Practical example in busybox under docker container:
kasra#ubuntu:~$ docker run --rm -it busybox
/ # nc -l -p 80 <<< "tested like a charm";
sh: syntax error: unexpected redirection
/ # nc -l -p 80 <<EOL
> tested like a charm
> EOL
^Cpunt! => socket listening, no errors. ^Cpunt! is result of CTRL+C signal.
/ # text="tested like a charm"
/ # nc -l -p 80 <<EOF
> $text
> EOF
^Cpunt!
do it the simpler way,
direc=$(basename `pwd`)
Or use the shell
$ direc=${PWD##*/}
Another reason to the error may be if you are running a cron job that updates a subversion working copy and then has attempted to run a versioned script that was in a conflicted state after the update...
On my machine, if I run a script directly, the default is bash.
If I run it with sudo, the default is sh.
That’s why I was hitting this problem when I used sudo.
In my case error is because i have put ">>" twice
mongodump --db=$DB_NAME --collection=$col --out=$BACKUP_LOCATION/$DB_NAME-$BACKUP_DATE >> >> $LOG_PATH
i just correct it as
mongodump --db=$DB_NAME --collection=$col --out=$BACKUP_LOCATION/$DB_NAME-$BACKUP_DATE >> $LOG_PATH
Before running the script, you should check first line of the shell script for the interpreter.
Eg:
if scripts starts with /bin/bash , run the script using the below command
"bash script_name.sh"
if script starts with /bin/sh, run the script using the below command
"sh script_name.sh"
./sample.sh - This will detect the interpreter from the first line of the script and run.
Different Linux distributions having different shells as default.