I am using a shell script in Jenkins that, at a certain point, uploads a file to a server using curl. I would like to see whatever output curl produces but also check whether it is the output I expect. If it isn't, then I want to set the shell error code to > 0 so that Jenkins knows the script failed.
I first tried using curl -f, but this causes the pipe to be cut as soon as the upload fails and the error output never gets to the client. Then I tried something like this:
curl ...params... | tee /dev/tty | \
xargs -I{} test "Expected output string" = '{}'
This works from a normal SSH shell but in the Jenkins console output I see:
tee: /dev/tty: No such device or address
I'm not sure why this is since I thought Jenkins was communicating with the slave using a normal SSH shell. In any case, the whole xargs + test thing strikes me as a bit of a hack.
Is there a way to accomplish this in Jenkins so that I can see the output and also test whether it matches a specific string?
When Jenkins communicates with slave via SSH, there is no terminal allocated, and so there is no /dev/tty device for that process.
Maybe you can send it to /dev/stderr instead? It will be a terminal in an interactive session and some log file in non-interactive session.
Have you thought about using the Publish over SSH Plugin instead of using curl? Might save you some headache.
If you just copy the file from master to slave there is also a plugin for that, copy to slave Plugin.
Cannot write any comments yet, so I had to post it as an answer.
Related
Writing a script to retrieve various environment parameters back from a list of servers. My script returns no value when ran but the same command returns the desired value outside of a script.
I have tried using a couple of variations to retrieve the same data. One of the commands fails because of restrictions placed on the accounts I have access to. The second command works but only if executed in an elevated mode.
This fails with access denied (pwdx is restricted)
dzdo pgrep -f /some/path | xargs pwdx
This works outside of a script but returns no value within a script
dzdo /bin/readlink -e /proc/"$(pgrep -f /some/path)"/cwd
When using "bash -x" to execute my scriipt, I see the "readlink" code is blank.
Ideally, I would like to return the PID and path of the process running as the "pgrep" command does. I can work with the path alone as returned by the "readlink" version returns. The end goal is to gather the information from several servers for audit purposes. (version, etc.)
Am I using the wrong syntax for the "readlink" command? I'm fairly new to coding bash scripts so I appreciate any guidance to help understand when to to what if I'm using a command in a script vs command line.
If pwdx is the restricted program, you need to run that with dzdo, not pgrep.
pgrep -f /some/path | dzdo xargs pwdx
I have a series of bash commands, some with interactive prompts, that I need run on a remote machine. I have to have them called in a certain order for different scenarios, so I've been trying to make a bash script to automate the process for me. However, it seems like every way to start an ssh session with a bash script results in the the redirection of stdin to whatever string or file was used to initiate the script in the first place.
Is there a way I can specify that a certain script be executed on a remote machine, but also forward stdin through ssh to the local machine to enable the user to interact with any prompts?
Here's a list of requirements I have to clarify what I'm trying to do.
Run a script on a remote machine.
Somewhere in the middle of that remote script be command that will prompt for input. Example: git commit will bring up vim.
If that command is git commit and it brings up vim, the user should be able to interact with vim as if it was running locally on their machine.
If that command prompts for a [y/n] response, the user should be able to input their answer.
After the user enters the necessary information—by quitting vim or pressing return on a prompt—the script should continue to run like normal.
My script will then terminate the ssh session. The end product is that commands were executed for the user without them needing to be aware that it was through a remote connection.
I've been testing various different methods with the following script that I want run on the remote machine.
#!/bin/bash
echo hello
vim
echo goodbye
exit
It's crucial that the user be able to use vim, and then, when the user finishes, "goodbye" should be printed to the screen and the remote session should be terminated.
I've tried uploading a temporary script to the remote machine and then running ssh user#host bash /tmp/myScript, but that seems to also take over stdin completely, rendering it impossible to let the user respond to prompts for user input. I've tried adding the -t and -T options (I'm not sure if they're different), but I still get the same result.
One commenter mentioned using expect, spawn, and interact, but I'm not sure how to use those tools together to get my desired behavior. It seems like interact will result in the user gaining control over stdin, but then there's no way to have it relinquished once the user quits vim in order to let my script continue execution.
Is my desired behavior even possible?
Ok, I think I've found my problem. I was creating a wrapper script for ssh that looked like this:
#!/bin/bash
tempScript="/tmp/myScript"
remote=user#host
commands=$(</dev/stdin)
cat <(echo "$commands") | ssh $remote "cat > $tempScript && chmod +x $tempScript" &&
ssh -t $remote $tempScript
errorCode=$?
ssh $remote << RM
if [[ -f $tempScript ]]; then
rm $tmpScript
fi
RM
exit $errorCode
It was there that I was redirecting stdin, not ssh. I should have mentioned this when I formulated my question. I read through that script over and over again, but I guess I just overlooked that one line. Removing that line totally fixed my problem.
Just to clarify, changing my script to the following totally fixed my problem.
#!/bin/bash
tempScript="/tmp/myScript"
remote=user#host
commands="$#"
cat <(echo "$commands") | ssh $remote "cat > $tempScript && chmod +x $tempScript" &&
ssh -t $remote $tempScript
errorCode=$?
ssh $remote << RM
if [[ -f $tempScript ]]; then
rm $tmpScript
fi
RM
exit $errorCode
Once I changed my wrapper script, my test script described in the question worked! I was able to print "hello" to the screen, vim appeared and I was able to use it like normal, and then once I quit vim "goodbye" was printed and the ssh client closed.
The commenters to the question were pointing me in the right direction the whole time. I'm sorry I only told part of my story.
I've searched for solutions to this problem several times in the past, however never finding a fully satisfactory one. Piping into ssh looses your interactivity. Two connects (scp/ssh) is slower, and your temporary file might be left lying around. And the whole script on the command line often ends up in escaping hell.
Recently I encountered that the command line buffer size is usually quite large (getconf ARG_MAX > 2MB where I looked). And this got me thinking about how I could use this and mitigate the escaping issue.
The result is:
ssh -t <host> /bin/bash "<(echo "$(cat my_script | base64 | tr -d "\n")" | base64 --decode)" <arg1> ...
or using a here document and cat:
ssh -t <host> /bin/bash $'<(cat<<_ | base64 --decode\n'$(cat my_script | base64)$'\n_\n)' <arg1> ...
I've expanded on this idea to produce a fully working BASH example script sshx that can run arbitrary scripts (not just BASH), where arguments can be local input files too, over ssh. See here.
I'm creating a script that I want people to run with
curl -sSL install.domain.com | bash
As RVM, oh-my-zsh and many others does,
However, I'm having issues because my script is interactive (it uses read and select, and the user is not being prompted, the script just skip those steps as is being executed with a |.
I've tried adding {} to my whole code.
I was thinking in ask the script to download himself again and put in tmp folder, and execute from there, not sure if that will work.
You can explicitly tell read to read from the terminal using
read var < /dev/tty
The solution i found is ask user to run it like:
bash <( curl -sSL install.domain.com )
This way script is passed as an argument and standard input remains untouched.
This is a non-problem, as users should not be executing code directly from the stream. The user should be downloading the code to a file first, then verifying that the script they receive has, for example, an MD5 hash that matches the hash you provide on your website. Only after confirming that the script they receive is the script you sent should they execute it.
$ curl -sSL install.domain.com > installer.sh
$ md5sum installer.bash # Does the output match the hash posted on install.domain.com?
$ bash installer.bash
I'm using Envoy to provision a remote server. Provisioning is done by pulling the bash script from a private repo and then execute it.
The bash script ask some confirmation like yes/no (using bash "read -p"): it works as expected when i'm connected to the remote server... the script wait for user input.
Instead Envoy seems to ignore any prompt. Is it an expected behavior?
Any workaround?
Yes, this is expected. There's nothing for read to read from so it doesn't.
You have a few options.
Rewrite your script to use a config file when there's no terminal to prompt from.
Use something like [ -t 0 ] to test if the standard input is a terminal and load a configuration file with defaults. The simplest way to do that is just have a file that contains appropriate variable assignments and just source it . defaults.sh or whatever. You don't even need the -t test if you source the defaults first since then anything the user inputs will over-ride the default value.
Rewrite your script to have sane defaults.
Rewrite whatever runs the script to provide your script input via pipeline/file via redirection (e.g. printf 'answer 1\nanswer 2\n' | ./script.sh or ./script.sh <answerfile).
I'm trying to create a job in Jenkins that will execute a simple shell script. However, it seems that Jenkins isn't actually doing anything with my script. No matter what value I put into the Execute Shell Command section, Jenkins always says that it passes. Even if I put in a bogus filename like "RandomBogusFilename.sh" it'll say the job was a success.
Can anyone point out what I'm doing wrong and how I can get Jenkins to actually use my shell script?
The shell script, the job config, and the console output are all shown below. I'm currently trying to do this on a Windows Server 2008 R2 Standard machine.
Thanks.
My .sh file
File Name: surveyToolRequest.sh
File Location: /jobs/Jeff Shell Script Test/workspace
Description:
Hit a web address and retrieve the HTTP Response. Then print out the HTTP Response.
#!/bin/bash
response_code=$(curl -s -o /dev/null -w "%{http_code}" http://SOME-WEBSITE.COM)
echo "The response code is " $response_code
My Jenkins Job Config
Jenkins Console Output
I played with this and found that it worked if I specified the path to the script. If the script is in your job's workspace directory,
./surveyToolRequest.sh
should work as Jenkins looks for files relative to the root of the workspace.
It's more common to just put the contents of the script file directory into the job configuration; that way you can see what the job is doing and you'll avoid problems like this one.
You should use run "Execute windows batch command" and not "Execute Shell"