I'm building a bootstrap for a github project and would like it to be a simple one-liner. The script requires a password input.
This works and stops the script to wait for an input:
curl -s https://raw.github.com/willfarrell/.vhosts/master/setup.sh -o setup.sh
bash setup.sh
This does not, and just skips over the input request:
curl -s https://raw.github.com/willfarrell/.vhosts/master/setup.sh | bash
setup.sh contains code is something like:
# code before
read -p "Password:" -s password
# code after
Is it possible to have a clean one-liner? If so, how might one do it?
Workaround:
Use three commands instead of piping output.
curl -s https://raw.github.com/willfarrell/.vhosts/master/setup.sh -o vhosts.sh && bash vhosts.sh && rm vhosts.sh
I had the same exact problem as the OP and was looking for an answer. This question was one of the first hits on Google for me and since it doesn't have a real answer yet, here's the command that I eventually stumbled upon which solved my need of using read in a remote script.
bash <(curl -s https://example.com/my-bash-script.sh)
With the pipe, the read reads from standard input (the pipe), but the shell already read all the standard input so there isn't anything for the read to read.
Related
I'm looking for how I might allow user input in a second command in a bash statement and I'm not sure how to go about it. I'd like to be able to provide a one-liner for someone to be able to install my application, but part of that application process requires asking some questions.
The current script setup looks like:
curl <url/to/bootstrap.sh> | bash
and then boostrap.sh does:
if [ $UID -ne 0 ]; then
echo "This script requires root to run. Restarting the script under root."
exec sudo $0 "$#"
exit $?
fi
git clone <url_to_repo> /usr/local/repo/
bash /usr/local/repo/.setup/install_system.sh
which in turn calls a python3 script that asks for input.
I know that the the curl in the first line is using stdin and so that might make what I'm asking impossible and that it has to be two lines to ever work:
wget <url/to/boostrap.sh>
bash bootstrap.sh
You can restructure your script to run this way:
bash -c "$(curl -s http://0.0.0.0//test.bash 2>/dev/null)"
foo
wololo:a
a
My test.bash is really just
#!/bin/bash
echo foo
python -c 'x = raw_input("wololo:");print(x)'`
To demonstrate that stdin can be read from in this way. Sure it creates a subshell to take care of curl but it allows you to keep reading from stdin as well.
Suppose that I have the following simple bash script which I want to submit to a batch server through SLURM:
#!/bin/bash
#SBATCH -o "outFile"$1".txt"
#SBATCH -e "errFile"$1".txt"
hostname
exit 0
In this script, I simply want to write the output of hostname on a textfile whose full name I control via the command-line, like so:
login-2:jobs$ sbatch -D `pwd` exampleJob.sh 1
Submitted batch job 203775
Unfortunately, it seems that my last command-line argument (1) is not parsed through sbatch, since the files created do not have the suffix I'm looking for and the string "$1" is interpreted literally:
login-2:jobs$ ls
errFile$1.txt exampleJob.sh outFile$1.txt
I've looked around places in SO and elsewhere, but I haven't had any luck. Essentially what I'm looking for is the equivalent of the -v switch of the qsub utility in Torque-enabled clusters.
Edit: As mentioned in the underlying comment thread, I solved my problem the hard way: instead of having one single script that would be submitted several times to the batch server, each with different command line arguments, I created a "master script" that simply echoed and redirected the same content onto different scripts, the content of each being changed by the command line parameter passed. Then I submitted all of those to my batch server through sbatch. However, this does not answer the original question, so I hesitate to add it as an answer to my question or mark this question solved.
I thought I'd offer some insight because I was also looking for the replacement to the -v option in qsub, which for sbatch can be accomplished using the --export option. I found a nice site here that shows a list of conversions from Torque to Slurm, and it made the transition much smoother.
You can specify the environment variable ahead of time in your bash script:
$ var_name='1'
$ sbatch -D `pwd` exampleJob.sh --export=var_name
Or define it directly within the sbatch command just like qsub allowed:
$ sbatch -D `pwd` exampleJob.sh --export=var_name='1'
Whether this works in the # preprocessors of exampleJob.sh is also another question, but I assume that it should give the same functionality found in Torque.
Using a wrapper is more convenient. I found this solution from this thread.
Basically the problem is that the SBATCH directives are seen as comments by the shell and therefore you can't use the passed arguments in them. Instead you can use a here document to feed in your bash script after the arguments are set accordingly.
In case of your question you can substitute the shell script file with this:
#!/bin/bash
sbatch <<EOT
#!/bin/bash
#SBATCH -o "outFile"$1".txt"
#SBATCH -e "errFile"$1".txt"
hostname
exit 0
EOT
And you run the shell script like this:
bash [script_name].sh [suffix]
And the outputs will be saved to outFile[suffix].txt and errFile[suffix].txt
If you pass your commands via the command line, you can actually bypass the issue of not being able to pass command line arguments in the batch script. So for instance, at the command line :
var1="my_error_file.txt"
var2="my_output_file.txt"
sbatch --error=$var1 --output=$var2 batch_script.sh
The lines starting with #SBATCH are not interpreted by bash but are replaced with code by sbatch.
The sbatch options do not support $1 vars (only %j and some others, replacing $1 by %1 will not work).
When you don't have different sbatch processes running in parallel, you could try
#!/bin/bash
touch outFile${1}.txt errFile${1}.txt
rm link_out.sbatch link_err.sbatch 2>/dev/null # remove links from previous runs
ln -s outFile${1}.txt link_out.sbatch
ln -s errFile${1}.txt link_err.sbatch
#SBATCH -o link_out.sbatch
#SBATCH -e link_err.sbatch
hostname
# I do not know about the background processing of sbatch, are the jobs still running
# at this point? When they are, you can not delete the temporary symlinks yet.
exit 0
Alternative:
As you said in a comment yourself, you could make a masterscript.
This script can contain lines like
cat exampleJob.sh.template | sed -e 's/File.txt/File'$1'.txt/' > exampleJob.sh
# I do not know, is the following needed with sbatch?
chmod +x exampleJob.sh
In your template the #SBATCH lines look like
#SBATCH -o "outFile.txt"
#SBATCH -e "errFile.txt"
This is an old question but I just stumbled into the same task and I think this solution is simpler:
Let's say I have the variable $OUT_PATH in the bash script launch_analysis.bash and I want to pass this variable to task_0_generate_features.sl which is my SLURM file to send the computation to a batch server. I would have the following in launch_analysis.bash:
`sbatch --export=OUT_PATH=$OUT_PATH task_0_generate_features.sl`
Which is directly accessible in task_0_generate_features.sl
In #Jason case we would have:
sbatch -D `pwd` --export=hostname=$hostname exampleJob.sh
Reference: Using Variables in SLURM Jobs
Something like this works for me and Torque
echo "$(pwd)/slurm.qsub 1" | qsub -S /bin/bash -N Slurm-TEST
slurm.qsub:
#!/bin/bash
hostname > outFile${1}.txt 2>errFile${1}.txt
exit 0
I'm new to Bash scripting, so please be gentle.
I'm connected to a Ubuntu server via SSH (PuTTY) and when I run this command, I expect the bash script that downloads and executes to allow user input and then echo that input. It seems to just write out the echo label for the input request and terminate.
wget -O - https://raw.github.com/aaronhancock/pub/master/bash/readtest.sh | bash
Any clue what I might be doing wrong?
UPDATE: This bash command does exactly what I wanted
bash <(wget -q -O - https://raw.github.com/aaronhancock/pub/master/bash/readtest.sh)
Jonathan already mentioned: bash takes its stdin from the pipe.
And therefore you cannot pipe the script into bash when you want to interactively input something. But you could use the process substitution feature of bash (assumed your login shell is a bash):
bash <(wget -O - https://raw.github.com/aaronhancock/pub/master/bash/readtest.sh)
Bash is taking stdin from the pipe, not from the terminal. So you can't pipe a script to bash and still use the "read" command for user input.
Notice that you have the same problem if you save the script to a local file and pipe it to bash:
less readtest.sh | bash
I found this also works and helps keep the data in the current scope.
eval "wget -q -O - https://raw.github.com/aaronhancock/pub/master/bash/readtest.sh"
Here is the offending part of my script:
read -d '' TEXT <<'EOF'
Some Multiline
text that
I would like
in
a
var
EOF
echo "$TEXT" > ~/some/file.txt
and the error:
read: 175: Illegal option -d
I use this read -d all over the place and it works fine. Not sure why its not happy now. I'm running the script on Ubuntu 10.10
Fixes? Workarounds?
If you run sh and then try that command, you get:
read: 1: Illegal option -d
If you do it while still in bash, it works fine.
I therefore deduce that your script is not running under bash.
Make sure that your script begins with the line:
#!/usr/bin/env bash
(or equivalent) so that the correct shell is running the script.
Alternatively, if you cannot do that (because the script is not a bash one), just be aware that -d is a bash feature and may not be available in other shells. In that case, you will need to find another way.
The -d option to read is a feature unique to bash, not part of the POSIX standard (which only specifies -r and -p options to read). When you run your script with sh on Ubuntu, it's getting run with dash, which is a POSIX shell, and not bash. If you want the script to run under bash then you should run it with bash, or give it a #!/bin/bash shebang. Otherwise, it should be expected to run under any POSIX sh.
This is what was trying to do:
$ wget -qO- www.example.com/script.sh | sh
which quietly downloads the script and prints it to stdout which is then piped to sh. This unfortunately doesn't quite work, failing to wait for user input at various points, aswell as a few syntax errors.
This is what actually works:
$ wget -qOscript www.example.com/script.sh && chmod +x ./script && ./script
But what's the difference?
I'm thinking maybe piping the file doesn't execute the file, but rather executes each line individually, but I'm new to this kind of thing so I don't know.
When you pipe to sh , stdin of that shell/script will be the pipe. Thus the script cannot take e.g. user input from the console. When you run the script normally, stdin is the console - where you can enter input.
You might try telling the shell to be interactive:
$ wget -qO- www.example.com/script.sh | sh -i
I had the same issue, and after tinkering and googling this is what worked for me.
wget -O - www.example.com/script.sh | sh