I am relatively new to working in bash and one of the biggest pains with this script I have to run is that I get prompted for passwords repeatedly when running this script. I am unable to pass ssh keys or use any options except expect due to security restrictions but I am struggling to understand how to use expect.
Does Expect require a separate file from this script to call itself, it seems that way looking at tutorials but they seem rather complex and confusing for a new user. Also how do I input into my script that I want it to auto fill in any prompt that says Password: ? Also this script runs with 3 separate unique variables every time the script is called. How do I make sure that those are gathered but the password is still automatically filled?
Any assistance is greatly appreciated.
#!/bin/bash
zero=`echo $2`
TMPIP=`python bin/dgip.py $zero`
IP=`echo $TMPIP`
folder1=`echo $zero | cut -c 1-6`
folder2=`echo $zero`
mkdir $folder1
cd $folder1
mkdir $folder2
cd $folder2
scp $1#`echo $IP`:$3 .
Embedding expect code in an shell script is not too difficult. We have to be careful to get the quoting correct. You'll do something like this:
#!/usr/bin/env bash
user=$1
zero=$2
files=$3
IP=$(python bin/dgip.py "$zero")
mkdir -p "${zero:0:6}/$zero"
cd "${zero:0:6}/$zero"
export user IP files
expect <<<'END_EXPECT' # note the single quotes here!
set timeout -1
spawn scp $env(user)#$env(IP):$env(files) .
expect {assword:}
send "$env(my_password)\r"
expect eof
END_EXPECT
Before you run this, put your password into your shell's exported environment variables:
export my_password=abc123
bash script.sh joe zero bigfile1.tgz
bash script.sh joe zero bigfile2.tgz
...
Having said all that, public key authentication is much more secure. Use that, or get your sysadmins to enable it, if at all possible.
Related
My script look like this , but not getting the password out.
#!/usr/bin/bash
export -p curdir=`basename ${PWD%/*/*/}`
/usr/bin/expect <<EOF
spawn scp -v -i ... <local file> <remote file containing $curdir>
expect -re "Enter passphrase.+:"
send "<password>\n"
exit;
EOF
exit
Something is wrong, as the passsword is evidently not sent. Something wrong with the expect line, the send line, or ???
When this script is executed it is asking for the password.
Thanks for any help.
I made this way too hard. ftp does everything I needed to do. And there are lots of places on the internet that explain how to use it. One can define bash variables as part of a bash script, and the password is simply part of what you define. scp is great of one copy, but ftp as part of a script is really easy.
I'm writing a script that will scp a tar file from my local server to a remote host. Since the script generates the file through a pre-requisite process, the name is generated dynamically. My script needs to take the name of the file and pass it to scp for transfer.
#!/usr/bin/expect -f
spawn scp test.$(date +%y%m%d_%H%M).tar user#IP-ADDRESS:/destination/folder
set pass "password"
expect "password: "
send -- "$pass\r"
expect eof
I've tried setting the filename as a variable but keep seeing the same error:
can't read "(date +%y%m%d_%H%M)": no such variable
while executing "spawn scp test.$(date +%y%m%d_%H%M).tar user#IP-ADDRESS:/destination/folder"
$(date +%y%m%d_%H%M) is not a Tcl command. If you use expect, you have to learn Tcl. To get a formatted date in Tcl, use the clock command. Also, interpolation of the result from a command in Tcl is not done by $(....), but by [....]. You can find examples for this construct here.
Decided to go another route since the team was able to provision a new Artifactory repo for this binary and alike. However, to the advice provided here I was able to make a few discoveries which I used to fix my issues:
I also had a password with $ symbol and that also caused a world of issues.
#!/bin/bash
TEST=$(date +%y%m%d_%H%M)
/usr/bin/expect <<eof
set password {pas\$word}
spawn scp "$TEST" user#IP-ADDRESS:/destination/folder
expect "*password:"
send "$pasword\r"
expect eof
I have a series of bash commands, some with interactive prompts, that I need run on a remote machine. I have to have them called in a certain order for different scenarios, so I've been trying to make a bash script to automate the process for me. However, it seems like every way to start an ssh session with a bash script results in the the redirection of stdin to whatever string or file was used to initiate the script in the first place.
Is there a way I can specify that a certain script be executed on a remote machine, but also forward stdin through ssh to the local machine to enable the user to interact with any prompts?
Here's a list of requirements I have to clarify what I'm trying to do.
Run a script on a remote machine.
Somewhere in the middle of that remote script be command that will prompt for input. Example: git commit will bring up vim.
If that command is git commit and it brings up vim, the user should be able to interact with vim as if it was running locally on their machine.
If that command prompts for a [y/n] response, the user should be able to input their answer.
After the user enters the necessary information—by quitting vim or pressing return on a prompt—the script should continue to run like normal.
My script will then terminate the ssh session. The end product is that commands were executed for the user without them needing to be aware that it was through a remote connection.
I've been testing various different methods with the following script that I want run on the remote machine.
#!/bin/bash
echo hello
vim
echo goodbye
exit
It's crucial that the user be able to use vim, and then, when the user finishes, "goodbye" should be printed to the screen and the remote session should be terminated.
I've tried uploading a temporary script to the remote machine and then running ssh user#host bash /tmp/myScript, but that seems to also take over stdin completely, rendering it impossible to let the user respond to prompts for user input. I've tried adding the -t and -T options (I'm not sure if they're different), but I still get the same result.
One commenter mentioned using expect, spawn, and interact, but I'm not sure how to use those tools together to get my desired behavior. It seems like interact will result in the user gaining control over stdin, but then there's no way to have it relinquished once the user quits vim in order to let my script continue execution.
Is my desired behavior even possible?
Ok, I think I've found my problem. I was creating a wrapper script for ssh that looked like this:
#!/bin/bash
tempScript="/tmp/myScript"
remote=user#host
commands=$(</dev/stdin)
cat <(echo "$commands") | ssh $remote "cat > $tempScript && chmod +x $tempScript" &&
ssh -t $remote $tempScript
errorCode=$?
ssh $remote << RM
if [[ -f $tempScript ]]; then
rm $tmpScript
fi
RM
exit $errorCode
It was there that I was redirecting stdin, not ssh. I should have mentioned this when I formulated my question. I read through that script over and over again, but I guess I just overlooked that one line. Removing that line totally fixed my problem.
Just to clarify, changing my script to the following totally fixed my problem.
#!/bin/bash
tempScript="/tmp/myScript"
remote=user#host
commands="$#"
cat <(echo "$commands") | ssh $remote "cat > $tempScript && chmod +x $tempScript" &&
ssh -t $remote $tempScript
errorCode=$?
ssh $remote << RM
if [[ -f $tempScript ]]; then
rm $tmpScript
fi
RM
exit $errorCode
Once I changed my wrapper script, my test script described in the question worked! I was able to print "hello" to the screen, vim appeared and I was able to use it like normal, and then once I quit vim "goodbye" was printed and the ssh client closed.
The commenters to the question were pointing me in the right direction the whole time. I'm sorry I only told part of my story.
I've searched for solutions to this problem several times in the past, however never finding a fully satisfactory one. Piping into ssh looses your interactivity. Two connects (scp/ssh) is slower, and your temporary file might be left lying around. And the whole script on the command line often ends up in escaping hell.
Recently I encountered that the command line buffer size is usually quite large (getconf ARG_MAX > 2MB where I looked). And this got me thinking about how I could use this and mitigate the escaping issue.
The result is:
ssh -t <host> /bin/bash "<(echo "$(cat my_script | base64 | tr -d "\n")" | base64 --decode)" <arg1> ...
or using a here document and cat:
ssh -t <host> /bin/bash $'<(cat<<_ | base64 --decode\n'$(cat my_script | base64)$'\n_\n)' <arg1> ...
I've expanded on this idea to produce a fully working BASH example script sshx that can run arbitrary scripts (not just BASH), where arguments can be local input files too, over ssh. See here.
I've been looking for a way to log some more detailed information about the history of commands. My main purpose is to have a rough log of commands that were issued in order to build rough server timelines when debugging issues with our application. It is not for highly detailed auditing purposes. I came across this post which suggested an excellent way to modify PROMPT_COMMAND to augment the history log with additional information about each command. It suggests adding the following to the ~/.bashrc file:
export PROMPT_COMMAND='hpwd=$(history 1); hpwd="${hpwd# *[0-9]* }"; if [[ ${hpwd%% *} == "cd" ]]; then cwd=$OLDPWD; else cwd=$PWD; fi; hpwd="${hpwd% ### *} ### $cwd"; history -s "$hpwd"'
This works awesome, except that it only happens when the PS1 prompt is issued. Is there a way to enhance this to work with non-interactive shells (I think that's the correct term)?
For example, I would like:
ssh host "ls | grep home"
To create an entry for ls | grep home on host as well, but since this isn't done through a PS1 prompt the linked solution falls short.
I have looked into auditd a little. This is a great utility, but the level of detail was way more than I need. I could have parsed the logs pretty easily, but pipes, redirects, loops become a nightmare to rebuild sanely into something pretty like what history already reports.
A simple wrapper around ssh would seem like a straightforward way to achieve this.
shout () {
local host
host=$1
shift
ssh "$host" <<____HERE
echo "$#" >>\$HOME/.shout-history
bash -c "$#"
____HERE
}
Or if you want the wrapper to run locally,
shout () {
local host
host=$1
shift
echo "$#" >>$HOME/.shout-history
ssh "$host" "$#"
}
I called this shout in opposition to ssh which ought to be, you know, quiet. See also this. Of course, if you are admin, you could simply move /usr/bin/ssh to someplace obscure and force your users to run a /usr/local/bin/ssh with contents similar to the above. It's easy enough to bypass by a knowledgeable user, but if you're really draconian, there are ways to make it harder.
If you are the admin of the remote host, you could force all users to run /usr/local/bin/shout as their shell, for example, and populate it with something more or less similar.
#!/bin/bash
echo "$#" >>/home/root/im.in.ur.sh.reading.ur.seekrit.cmds.lol
exec /bin/bash -c "$#"
Just make sure the transcript file is world writable but not world readable.
I've a script that launches inside of itself a command with a parameter that is a secret. For example:
#!/bin/bash
command-name secret
While running the command I can read through ps -ef | grep command-name which is the secret.
Is there any way of hiding the secret in a way that through ps -ef, the command line parameter is obfuscated?
First, you can NOT hide command line arguments. They will still be visible to other users via ps aux and cat /proc/$YOUR_PROCESS_PID/cmdline at the time of launching the program (before the program has a chance to do run-time changes to arguments). Good news is that you can still have a secret by using alternatives:
Use standard input:
mySecret='hello-neo' printenv mySecret | myCommand
Use a dedicated file if you want to keep the secret detached from the main script (note that you'd be recommended to use full disc encryption and make sure the file has correct chmod permissions):
cat /my/secret | myCommand
Use environment variables (with caveats). If your program can read them, do this:
mySecret='hello-neo' myCommand
Use temporary file descriptor:
myCommand <( mySecret='hello-neo' printenv mySecret )
In the last case your program will be launched like myCommand /dev/fd/67, where the contents of /dev/fd/67 is your secret (hello-neo in this example).
In all of the above approaches, be wary of leaving the command in bash command history (~/.bash_history). You can avoid this by either running the command from a script (file), or by interactively prompting yourself for password each time:
read -s secret
s=$secret printenv s | myCommand # approach 2
myCommand <( s=$secret printenv s ) # approach 3
secret=$secret myCommand # approach 4
export secret && myCommand # another variation of approach 4
If the secret doesn't change between executions, use a special configuration file, ".appsecrets". Set the permissions of the file to be read-only by owner. Inside the file set an environment variable to the secret. The file needs to be in the home directory of the user running the command.
#!/bin/bash
#filename: .appsecrets
set SECRET=polkalover
Load the config file so the environment variable gets set.
. ~/.appsecrets
What I've seen done:
1)
echo $SECRET | command
works if the command prompts for the password from stdin AND if 'echo' is a builtin of your shell. We were using Korn.
2)
password=$ENV{"SECRET"};
works if you have control of the code (e.g. in perl or C++)
3)
. ./.app.config #sets the environment variables
isql -host [host] -user [user] -password <<SECRET
${SQLPASSWORD}
SECRET
works if the command can accept the secret from std-in. One limitation is that the <<string has to be the last argument given to the command. This might be troublesome if there is a non-optional arg that has to appear after -password
The benefit of this approach is you can arrange it so the secret can be hidden in production. Use the same filename in production but it will be in the home directory of the account that runs the command in production. You can then lock down access to the secret like you would access to the root account. Only certain people can 'su' to the prod account to view or maintain the secret while developers can still run the program because they use their own '.appsecret' file in their home directory.
You can use this approach to store secured information for any number of applications, as long as they use different environment variable names for their secrets.
(WRONG WAY)
One old method I saw the DBAs use was to set SYBASE to "/opt/././././././././././././././././././././././././././././././././././sybase/bin". So their commandlines were so long the ps truncated it. But in linux I think you might be able to sniff out the full commandline from /proc.
I saw it on another post. This is the easiest way under Linux.
This modifies the memory part of command line that all other programs see.
strncpy(argv[1], "randomtrash", strlen(argv[1]));
You can also change the name of the process, but only when read from the command line. Programs like top will show the real process name:
strncpy(argv[0], "New process name", strlen(argv[0]));
Don't forget to copy maximum strlen(argv[0]) bytes because probably there's no more space allocated.
I think that arguments can only be found in the portion of the memory that we modify so I think that this works like a charm. If someone knows something accurate about this, please comment.
VasyaNovikov note: The password can still be intercepted after the program has invoked but before it started doing the changes you described.
The only way to conceal your secret argument from ps is not to provide the secret as an argument. One way of doing that is to place the secret in a file, and to redirect file descriptor 3 to read the file, and then remove the file:
echo secret > x.$$
command 3<x.$$
rm -f x.$$
It isn't entirely clear that this is a safe way to save the secret; the echo command is a shell built-in, so it shouldn't appear in the 'ps' output (and any appearance would be fleeting). Once upon a very long time ago, echo was not a built-in - indeed, on MacOS X, there is still a /bin/echo even though it is a built-in to all shells.
Of course, this assumes you have the source to command and can modify it to read the secret from a pre-opened file descriptor instead of from the command line argument. If you can't modify the command, you are completely stuck - the 'ps' listing will show the information.
Another trick you could pull if you're the command owner: you could capture the argument (secret), write it to a pipe or file (which is immediately unlinked) for yourself, and then re-exec the command without the secret argument; the second invocation knows that since the secret is absent, it should look wherever the first invocation hid the secret. The second invocation (minus secret) is what appears in the 'ps' output after the minuscule interval it takes to deal with hiding the secret. Not as good as having the secret channel set up from the beginning. But these are indicative of the lengths to which you have to go.
Zapping an argument from inside the program - overwriting with zeroes, for example - does not hide the argument from 'ps'.
The expect library was created partially for these kind of things, so you can still provide a password / other sensitive information to a process without having to pass it as an argument. Assuming that when 'secret' isn't given the program asks for it of course.
There's no easy way. Take a look at this question I asked a while ago:
Hide arguments from ps
Is command your own program? You could try encrypting the secret and have the command decrypt it before use.
You can use LD_PRELOAD to have a library manipulate the command line arguments of some binary within the process of that binary itself, where ps does not pick it up. See this answer of mine on Server Fault for details.
Per the following article:
https://www.cyberciti.biz/faq/linux-hide-processes-from-other-users/
you can configure the OS to hide / separate the processes from each other with the hidepid mount option for the /proc, requires Linux kernel 3.2+.
may be you can do like this:
#include <boost/algorithm/string/predicate.hpp>
void hide(int argc, char** argv, std::string const & arg){
for(char** current = argv; current != argv+ argc ;++current){
if(boost::algorithm::starts_with(*current, "--"+arg)){
bzero(*current, strlen(*current));
}
}
}
int main(int argc, char** argv){
hide(argc, argv, "password");
}
Here is one way to hide a secret in an environment variable from ps:
#!/bin/bash
read -s -p "Enter your secret: " secret
umask 077 # nobody but the user can read the file x.$$
echo "export ES_PASSWORD=$secret" > x.$$
. x.$$ && your_awesome_command
rm -f x.$$ # Use shred, wipe or srm to securely delete the file
In the ps output you will see something like this:
$ps -ef | grep your_awesome_command
root 23134 1 0 20:55 pts/1 00:00:00 . x.$$ && your_awesome_command
Elastalert and Logstash are examples of services that can access passwords via environment variables.
If the script is intended to run manually, the best way is to read it in from STDIN
#!/bin/bash
read -s -p "Enter your secret: " secret
command "$secret"
I always store sensitive data in files that I don't put in git and use the secrets like this:
$(cat path/to/secret)