I am trying to write a shell script that will, given source file, target directory, and hosts file, scp a file to multiple hosts. I have gotten the script to work however there are a few issues but the file gets there. The only problem is the network I am working on requires a password for every remote. I need to do this to 150 plus remotes and need this to run automatically, possibly with a logging feature. If any one could help me out it would be greatly appreciated. Here's what I got so far...
# This is a script to copy files from one host to a group of hosts
# There are three variables accepted via commandline
# $1 = first parameter (/source_path/source_filename)
# $2 = second parameter (/target_directory/)
# $3 = third paramter (file that contains list of hosts)
SOURCEFILE=$1
TARGETDIR=$2
HOSTFILE=$3
if [ -f $SOURCEFILE ]
then
printf "File found, preparing to transfer\n"
while read server
do
scp -p $SOURCEFILE ${server}:$TARGETDIR
done < $HOSTFILE
else
printf "File \"$SOURCEFILE\" not found\n"
exit 1
fi
exit 0
SSH public-key authentication is a good solution here.
In short, run:
ssh-keygen -t dsa
To generate your public/private key pair.
Then, add ~/.ssh/id_dsa.pub to remote:~/.ssh/authorized_keys
Now, ssh will not ask for a password, since it will do a public-key authentication challenge instead.
I wrote an article here that describes this in some detail:
http://matt.might.net/articles/ssh-hacks/
You should not have to modify your script for this to work.
Related
I have the following script. My requirement is if I give the username and IP address the script should check whether the host server has passwordless connectivity with the given server.
script.sh
#!/bin/sh
echo "Enter username"
read user
echo "Enter IP address"
read ip
echo "Enter condition"
read conditon
if [ $condition == "test" ]
then
ssh -o 'PreferredAuthentications=publickey' $user#$ip "echo"
fi
The above script checks for passwordless connectivity and exits. I want the script to return "success" if the connection is there or "No connection" if there is no passwordless connection between the hosts.
Thanks.
You can give an extra option to ssh that sets it in BatchMode
BatchMode
If set to yes, passphrase/password querying will be disabled. This option is useful in scripts and other batch jobs where no user is present to supply the password. The argument must be yes or no (the default).
source: man sshconfig
The following line will return the requested error code
ssh -oBatchMode=yes user#server echo > /dev/null 2>&1
This will return 0 if it can log in and will return a non-zero value (255) if it fails to login.
So you could do something like:
ssh -oBathMode=yes "${user}#${server}" echo > /dev/null 2>&1 && echo "Success" || echo "No Connection"
Or you could even let the server answer Success as
ssh -oBathMode=yes "${user}#${server}" echo Success 2>/dev/null || echo "No Connection"
Prerequisites
Access to command line/terminal window
User with sudo or root privileges
A local server & a remote server
SSH access to a remote server via command line/terminal window
Before You Start
Check for existing SSH Keys, using the below command
ls -al ~/.ssh/id_*.pub
If the output tells you there are no such files, move on to the next step, which shows you how to generate SSH keys.
Step 1: Generate SSH Key Pair
ssh-keygen -t rsa -b 4096 -C "your_email#domain.com"
Next, type in the path where you want to store the keys or hit
Enter to accept the default path.
It also asks you to set a passphrase, this is to ensure more secure connection. Note that, the Passphrase may be interrupted when you set up automated processes. Else, one can just press Enter to skip this step.
The output shows you the identification & about where is your public key stored along with the key fingerprints.
Verify your newly created SSH key pair, using the below command.
ls -al ~/.ssh/id_*.pub
Step 2: Uploading the Public Key to Remote Server
There are 2 options, that one can adopt:
Using ssh-copy-id Command
Using cat Command
Using ssh-copy-id command
ssh-copy-ide [remote_username]#[server_ip_address]
Using cat Command
ssh [remote_username]#[server_ip_address] mkdir -p .ssh
Type in the password for remote server, and then cat to view the contents of the stored Public Key
cat .ssh/id_rsa.pub | ssh [remote_username]#[server_ip_address] 'cat >> .ssh/authorized_keys'
Step 3: Log in to Server Without Password
Final step is to check whether the setup works fine or not.
ssh [remote_username]#[server_ip_address]
Troubleshooting, if found any errors
If you are still prompted for a password after going through all the steps, start by editing file permissions on the remote server.
Set permissions 700 for the .ssh directory.
Set permissions 640 for the .ssh/authorized_keys directory.
Edit file permissions using this command:
ssh [remote_username]#[server_ip_address] "chmod 700 .ssh; chmod 640 .ssh/authorized_keys"
References:
(https://stackoverflow.com/a/52744882/18154805)
These three lines of code require authentication twice. I don't yet have password-less authentication set up on this server. In fact, these lines of code are to copy my public key to the server and concatenate it with the existing file.
How can I re-write this process with a single ssh command that requires authentication only once?
scp ~/local.txt user#server.com:~/remote.txt
ssh -l user user#server.com
cat ~/remote.txt >> ~/otherRemote.txt
I've looked into the following possibilities:
command sed
operator ||
operator &&
shared session: Can I use an existing SSH connection and execute SCP over that tunnel without re-authenticating?
I also considered placing local.txt at an openly accessible location, for example, with a public dropbox link. Then if cat could accept this as an input, the scp line wouldn't be necessary. But this would also require an additional step and wouldn't work in cases where local.txt cannot be made public.
Other references:
Using a variable's value as password for scp, ssh etc. instead of prompting for user input every time
https://superuser.com/questions/400714/how-to-remotely-write-to-a-file-using-ssh
You can redirect the content to the remote, and then use commands on the remote to do something with it. Like this:
ssh user#server.com 'cat >> otherRemote.txt' < ~/local.txt
The remote cat command will receive as its input the content of ~/local.txt, passed to the ssh command by input redirection.
Btw, as #Barmar pointed out, specifying the username with both -l user and user# was also redundant in your example.
I have around 30 servers that have a single server's SSH keys in authorized_keys.
I want to write a program which connects one by one to these boxes and does two things.
[1] srm's various directories with the -R flag
[2] leaves a txt document in the root directory
I know this would be possible through bash but I don't have the experience to write something like this.
Can anyone help me out? The servers run on the sshd port.
A for loop + ssh user#host -p port command can get it done
#!/bin/bash
while read server; do
#execute commands on remote server
ssh user#$server ifconfig
#copy local file over
scp /path/to/file user#$server:/path/on/remote_server/
done < servers.txt
I've created a bash script to migrate sites and databases from one server to another: Algorithm:
Parse .pgpass file to create individual dumps for all the specified Postgres db's.
Upload said dumps to another server via rsync.
Upload a bunch of folders related to each db to the other server, also via rsync.
Since databases and folders have the same name, the script can predict the location of the folders if it knows the db name. The problem I'm facing is that the loop is only executing once (only the first line of .pgpass is being completed).
This is my script, to be run in the source server:
#!/bin/bash
# Read each line of the input file, parse args separated by semicolon (:)
while IFS=: read host port db user pswd ; do
# Create the dump. No need to enter the password as we're using .pgpass
pg_dump -U $user -h $host -f "$db.sql" $db
# Create a dir in the destination server to copy the files into
ssh user#destination.server mkdir -p webapps/$db/static/media
# Copy the dump to the destination server
rsync -azhr $db.sql user#destination:/home/user
# Copy the website files and folders to the destination server
rsync -azhr --exclude "*.thumbnails*" webapps/$db/static/media/ user#destination.server:/home/user/webapps/$db/static/media
# At this point I expect the script to continue to the next line, but if exits at the first line
done < $1
This is .pgpass, the file to parse:
localhost:*:db_name1:db_user1:db_pass1
localhost:*:db_name3:db_user2:db_pass2
localhost:*:db_name3:db_user3:db_pass3
# Many more...
And this is how I'm calling it:
./my_script.sh .pgpass
At this point everything works. The first dump is created, and it is transferred to the destination server along with the related files and folders. The problem is the script finishes there, and won't parse the other lines of .pgpass. I've commented out all lines related to rsync (so the script only creates the dumps), and it works correctly, executing once for each line in the script. How can I get the script to not exit after executing rsync?
BTW, I'm using key based ssh auth to connect the servers, so the script is completely prompt-less.
Let's ask shellcheck:
$ shellcheck yourscript
In yourscript line 4:
while IFS=: read host port db user pswd ; do
^-- SC2095: ssh may swallow stdin, preventing this loop from working properly.
In yourscript line 8:
ssh user#destination.server mkdir -p webapps/$db/static/media
^-- SC2095: Add < /dev/null to prevent ssh from swallowing stdin.
And there you go.
I can connect to a server via SSH using the -i option to specify the private key:
ssh -i ~/.ssh/id_dsa user#hostname
I am creating a script that takes the id_dsa text from the database but I am not sure how I can give that string to SSH. I would need something like:
ssh --option $STRING user#hostname
Where $STRING contains the value of id_dsa. I need to know the --option if there is one.
Try the following:
echo $KEY | ssh -i /dev/stdin username#host command
The key doesn't appear from a PS statement, but because stdin is redirected it's only useful for single commands or tunnels.
There is no such switch - as it would leak sensitive information. If there were, anyone could get your private key by doing a simple ps command.
EDIT: (because of theg added details in comment)
You really should store the key in to a temporary file. Make sure you set the permissions correctly before writing to the file, if you do not use command like mktemp to create the temporary file.
Make sure you run the broker (or agent in case of OpenSSH) process and load the key using <whatever command you use to fetch it form the database> | ssh-add -
Passing cryptokey as a string is not advisable but for the sake of the question, I would say I came across the same situation where I need to pass key as a string in a script. I could use key stored in a file too but the nature of the script is to make it very flexible, containing everything in itself was a requirement. so I used to assign variable and pass it and echo it as follows :
#!/bin/bash
KEY="${ YOUR SSH KEY HERE INSIDE }"
echo "${KEY}" | ssh -q -i /dev/stdin username#IP 'hostnamectl'
exit 0
Notes:
-q suppress all warnings
By the way , the catch here in above script, since we are using echo it will print the ssh key which is again not recommended , to hide that you can use grep to grep some anything which will not be printed for sure but still stdin will have the value from the echo. So the final cmd can be modified as follows :
#!/bin/bash
KEY="${ YOUR SSH KEY HERE INSIDE }"
echo "${KEY}" | grep -qw "less" | ssh -q -i /dev/stdin username#IP 'hostnamectl'
exit 0
This worked for me.
I was looking at the same problem. Adding private key content to ssh command via stdin did not work for me. I found out that its possible to add the private key file contents to ssh-agent using the command ssh-add. This will let you ssh into the remote host without explicitly specifying the identity file. My particular usecase was that I didn't want to store the SSH key in cleartext on my machine and was dynamically getting it from a secrets vault. This answer is mostly a collection of other answers on StackOverflow.
ssh-agent is a program to hold private keys used for public key
authentication. Through use of environment variables the agent can
be located and automatically used for authentication when logging
in to other machines using ssh
Source
This is what I have done.
First start the ssh-agent.
You can start it from your terminal by simply executing ssh-agent.
OPTIONAL: If you'd like to make sure ssh-agent is running on every login, you can add something like the following to your shell config.
This is what I have added to my ~/.bashrc file.
# set SSH_AUTH_SOCK env var to a fixed value
export SSH_AUTH_SOCK=~/.ssh/ssh-agent.sock
# test whether $SSH_AUTH_SOCK is valid
ssh-add -l 2>/dev/null >/dev/null
# if not valid, then start ssh-agent using $SSH_AUTH_SOCK
[ $? -ge 2 ] && ssh-agent -a "$SSH_AUTH_SOCK" >/dev/null
Source
(This particular snippet also makes sure new ssh-agent processes are not getting created when there's one already running.)
Now you have the ssh-agent running.
Since we're interested in loading SSH key as a string, I'll assume a scenario where private key contents has already been loaded in to a variable, $SSH_PRIVATE_KEY.
I can now add this Key contents to the ssh-agent by executing the following command.
ssh-add - <<< "${SSH_PRIVATE_KEY}"
This can just be added to the bashrc file as well.
You can confirm that your key has been added by listing all keys by executing ssh-agent -l. Aaand you're done now.
Try connecting to the remote host and you don't need a private key file.
ssh username#hostname
This does come with extra security risks. These are some I could think of:
Adding the private key to the ssh-agent will let any process on the machine access the key to authenticate remote hosts without explicitly providing any information.
Since the goal is to load Private key as a string, it will either be stored in a variable or the contents embedded directly in the command. This might make the key available in command history, the shell variable and other places.