I had to automate my postgre database backup. As instructed by my software vendor I am trying to use pg_dump.exe (see below) file to take a backup but that prompts me for password.
.\pg_dump.exe -h localhost -p 4432 -U postgres -v -b -F t -f "C:\Backup\Backup.tar" Repo
So googled and found that as per "https://www.postgresql.org/docs/9.6/libpq-pgpass.html" I can create a pgpass.conf file within 'C:\Users\User1\AppData\Roaming\postgresql\pgpass.conf" which I did.
Then I tried to pass data of pgpass.conf file to env variable before executing my pg_dump command. But it is not working. Still I am getting prompt to enter password. This is the content of pgpass.conf file: *:*:*:postgres:password
Below is the code I am trying in PowerShell,
$Env:PGPASSFILE="C:\Users\User1\AppData\Roaming\postgresql\pgpass.conf"
cd "C:\Program Files\Qlik\Sense\Repository\PostgreSQL\9.6\bin"
.\pg_dump.exe -h localhost -p 4432 -U postgres -v -b -F t -f "C:\Backup\Backup.tar" Repo
Why am I still being asked for password?
When I type following code $Env:AppData I get following response "C:\Users\User1\AppData\Roaming"
Everywhere there are guidance on how to use it in UNIX or command prompt but not in powershell. Any help is appreciated. Also if you could direct me how to secure this password file then it will be great.
With password prompt I cannot automate it with windows task scheduler.
I suspect you have a suitable solution, however, as a quick (and not secure) workaround via the command prompt, you can use the variable PGPASSWORD to hold the password then run the backup script.
A sample might be something like:
SET PGPASSWORD=password
cd "C:\Program Files\Qlik\Sense\Repository\PostgreSQL\9.6\bin" pg_dump.exe -h localhost -p 4432 -U postgres -b -F t -f "d:\qs_backup\QSR_backup.tar" QSR
Rod
I have yet to get the damned thing to work yet, but I did find this:
-w
--no-password Never issue a password prompt. If the server requires password authentication and a password is not available by other means
such as a .pgpass file, the connection attempt will fail. This option
can be useful in batch jobs and scripts where no user is present to
enter a password.
I don't see a -w parameter in your call to pg_dump
I used pg_hba file to allow connection "trust" this is riskier method but I had to get things done ASAP. Thank you for your time and effort
Using Bash scripting, I'm trying to upload the content of variable into an FTP server.
The variable is $HASHED and it contains some hashed password
echo $HASHED
The output of the above command is: M0eSl8NR40wH
I need to do the following:
Create a time/date stamped file in the FTP server (i.e. PASSWORD_18_02_2014)
The file needs to have the same content of the $HASHED value (i.e. the PASSWORD_18_02_2014 needs to have M0eSl8NR40wH inside it).
Trying Curl, I couldn't get it working using the following:
UPLOAD="curl -T $HASHED ftp://192.168.0.1/passwords/ --user username:password"
$UPLOAD
Your help is very much appreciated.
Something like this might help you (tested on Linux Mint 13):
#!/bin/bash
FILENAME=PASSWORD_`date +%d_%m_%Y`
echo $HASHED >$FILENAME
ftp -n your_ftp_site <<EOF
user your_user_name_on_the_ftp_server
put $FILENAME
EOF
rm $FILENAME
A few caveats:
You have to export HASHED, e.g. when you set it, set it like this: export HASHED=M0eSl8NR40wH
The above assumes you will be running this from a terminal and can type in your password when prompted
You may have to add some cd commands after the line that starts "user" and before the line that starts "put", depending on where you want to put the file on your ftp server
Don't forget to make the script executable:
chmod u+x your_command_script_name
You can code the password after the user name on the line that starts "user", but this leaves a big risk that someone can discover your password on the ftp server. At the very least, make the bash command script readable only by you:
chmod 700 your_command_script_name
Please try this:
HASHED="M0eSl8NR40wH"
echo "$HASHED" | curl --silent --show-error --upload-file \
-ftp://192.168.0.1/passwords/$(date +PASSWORD_%d_%m_%Y) --user username:password
Where:
--silent : prevents the progress bar
--show-error : shows errors if any
--upload-file - : get file from stdin
The target name is indicated as part of the URL
I've got problem, I've setup plink to create a connection to a BlueCoat device, retrieve the full configuration and redirect the output to a file.
The problem is, when I try it from the script, the output of plink is displayed on screen and not redirected to the file, but if I use the same exact command interactively, it works!
I've checked the file rights, etc. they all seem to be ok.
The way I use it is:
/usr/bin/plink -4 -batch -ssh -l <user> -pw <password> -m /tmp/bluecoat.backup <hostname> > output.txt
Any clues?
Kind regards,
Chris
I am writing my first shell script ever and trying to figure out how to transform this command:
sftp -o IdentityFile=/home/test/test/id_dsa test#test.test.com < sftp_put.txt;
into an equivalent command where I connect to a ftp server. The key difference is that I will be logging into this server via a username and password not my ssh credentials. Note I am trying to upload two files.
Again any help would be more than appreciated!
You can use .netrc for this:
$ cat > .netrc
machine your.machine.ip.address
login your.login.name
^D
$ ftp your.machine.ip.address < ftp_cmds.txt
Which would prompt you for a password. If you're okay with it, you can save the password (clear text) in .netrc to skip this prompt. See man netrc for more details.
ftp -v -n <<EOF > ${LOG_FTP} 2>&1
open ${IP_ADDRESS_SERVER}
user ${FTPUSER} ${FTPPASS}
cd ${REMOTE_DIR}
put sftp_put.txt
EOF
I need to transfer a log file to a remote host using sftp from a Linux host. I have been provided credentials for the same from my operations group. However, since I don't have control over other host, I cannot generate and share RSA keys with the other host.
So is there a way to run the sftp command (with the username/password provided) from inside the Bash script through a cron job?
I found a similar Stack Overflow question, Specify password to sftp in a Bash script, but there was no satisfactory answer to my problem.
You have a few options other than using public key authentication:
Use keychain
Use sshpass (less secured but probably that meets your requirement)
Use expect (least secured and more coding needed)
If you decide to give sshpass a chance here is a working script snippet to do so:
export SSHPASS=your-password-here
sshpass -e sftp -oBatchMode=no -b - sftp-user#remote-host << !
cd incoming
put your-log-file.log
bye
!
Another way would be to use lftp:
lftp sftp://user:password#host -e "put local-file.name; bye"
The disadvantage of this method is that other users on the computer can read the password from tools like ps and that the password can become part of your shell history.
A more secure alternative which is available since LFTP 4.5.0 is setting the LFTP_PASSWORD environment variable and executing lftp with --env-password. Here's a full example:
export LFTP_PASSWORD="just_an_example"
lftp --env-password sftp://user#host -e "put local-file.name; bye"
# Destroy password after use
export LFTP_PASSWORD=""
LFTP also includes a cool mirroring feature (can include delete after confirmed transfer --Remove-source-files):
lftp -e 'mirror -R /local/log/path/ /remote/path/' --env-password -u user sftp.foo.com
EXPECT is a great program to use.
On Ubuntu install it with:
sudo apt-get install expect
On a CentOS Machine install it with:
yum install expect
Lets say you want to make a connection to a sftp server and then upload a local file from your local machine to the remote sftp server
#!/usr/bin/expect
spawn sftp username#hostname.com
expect "password:"
send "yourpasswordhere\n"
expect "sftp>"
send "cd logdirectory\n"
expect "sftp>"
send "put /var/log/file.log\n"
expect "sftp>"
send "exit\n"
interact
This opens a sftp connection with your password to the server.
Then it goes to the directory where you want to upload your file, in this case "logdirectory"
This uploads a log file from the local directory found at /var/log/ with the files name being file.log to the "logdirectory" on the remote server
You can use lftp interactively in a shell script so the password not saved in .bash_history or similar by doing the following:
vi test_script.sh
Add the following to your file:
#!/bin/sh
HOST=<yourhostname>
USER=<someusername>
PASSWD=<yourpasswd>
cd <base directory for your put file>
lftp<<END_SCRIPT
open sftp://$HOST
user $USER $PASSWD
put local-file.name
bye
END_SCRIPT
And write/quit the vi editor after you edit the host, user, pass, and directory for your put file typing :wq .Then make your script executable chmod +x test_script.sh and execute it ./test_script.sh.
I was recently asked to switch over from ftp to sftp, in order to secure the file transmission between servers. We are using Tectia SSH package, which has an option --password to pass the password on the command line.
example : sftp --password="password" "userid"#"servername"
Batch example :
(
echo "
ascii
cd pub
lcd dir_name
put filename
close
quit
"
) | sftp --password="password" "userid"#"servername"
I thought I should share this information, since I was looking at various websites, before running the help command (sftp -h), and was i surprised to see the password option.
You can override by enabling Password less authentication. But you should install keys (pub, priv) before going for that.
Execute the following commands at local server.
Local $> ssh-keygen -t rsa
Press ENTER for all options prompted. No values need to be typed.
Local $> cd .ssh
Local $> scp .ssh/id_rsa.pub user#targetmachine:
Prompts for pwd$> ENTERPASSWORD
Connect to remote server using the following command
Local $> ssh user#targetmachine
Prompts for pwd$> ENTERPASSWORD
Execute the following commands at remote server
Remote $> mkdir .ssh
Remote $> chmod 700 .ssh
Remote $> cat id_rsa.pub >> .ssh/authorized_keys
Remote $> chmod 600 .ssh/authorized_keys
Remote $> exit
Execute the following command at local server to test password-less authentication.
It should be connected without password.
$> ssh user#targetmachine
The easiest way I found to accomplish this, without installing any third-party library like Expect, SSHPASS...etc, is by using a combination of CURL, and SFTP. Those two are almost in every Linux machine.
This is the command you should execute, after changing the values.
curl -k "sftp://SERVER_IP:SERVER_PORT/FULL_PATH_OF_THE_FILE" --user "SERVER_USER:SERVER_PASSOWRD" -o "THE_NAME_OF_THE_FILE_AFTER_DOWNLOADING_IT"
Example:
curl -k "sftp://10.10.10.10:77/home/admin/test.txt" --user "admin:123456" -o "test.txt"
Explanation:
We are connecting to the server 10.10.10.10:77 using the username admin and password 123456, to move the file /home/admin/test.txt from that server to the server you are using currently to execute the above command.
Combine sshpass with a locked-down credentials file and, in practice, it's as secure as anything - if you've got root on the box to read the credentials file, all bets are off anyway.
Bash program to wait for sftp to ask for a password then send it along:
#!/bin/bash
expect -c "
spawn sftp username#your_host
expect \"Password\"
send \"your_password_here\r\"
interact "
You may need to install expect, change the wording of 'Password' to lowercase 'p' to match what your prompt receives. The problems here is that it exposes your password in plain text in the file as well as in the command history. Which nearly defeats the purpose of having a password in the first place.
You can use sshpass for it. Below are the steps
Install sshpass For Ubuntu - sudo apt-get install sshpass
Add the Remote IP to your known-host file if it is first time
For Ubuntu -> ssh user#IP -> enter 'yes'
give a combined command of scp and sshpass for it.
Below is a sample code for war coping to remote tomcat
sshpass -p '#Password_For_remote_machine' scp /home/ubuntu/latest_build/abc.war #user##RemoteIP:/var/lib/tomcat7/webapps
You can use a Python script with scp and os library to make a system call.
ssh-keygen -t rsa -b 2048 (local machine)
ssh-copy-id user#remote_server_address
create a Python script like:
import os
cmd = 'scp user#remote_server_address:remote_file_path local_file_path'
os.system(cmd)
create a rule in crontab to automate your script
done
A few people have mentioned sshpass but not many clear coding examples...
This is how we are doing it with bash scripts for rsync backups:
sshpass -p "${RSYNC_PASSWORD}" sftp "${RSYNC_USER}"#"${RSYNC_REMOTE_HOST}"
Keep in mind you will have to sudo apt install sshpass before this works properly.