This question already has answers here:
Passing arguments to an interactive program non-interactively
(5 answers)
How do I specify a password to 'psql' non-interactively?
(11 answers)
Closed 5 months ago.
I have a bash script that runs a few psql commands. Every time a command is ran it prompts for user password. I'd like to make the script input the password automatically so I'd only need to run the script and not need to input anything.
I am aware that I can do this for psql
PGPASSWORD=root psql -h localhost -d $db_name -U root -c
but I'd like to know how to automate input in general/
You can export the PGPASSWORD as below:
export PGPASSWORD=root
psql -h localhost -d $db_name -U root -c
This question already has answers here:
Running shell command that has nested quotes via ssh
(3 answers)
How can I escape an arbitrary string for use as a command line argument in Bash?
(8 answers)
Closed 1 year ago.
Normally I can run a command to run a sql script via the commandline in the following manner in my server:
[ec2-user#ip-XX-XX-XX-XXX ~]$ sudo mysql -h BLAHBLAHBLAH.us-east-1.rds.amazonaws.com -u user -p'aaaaa:b>c[d{e]ff=|ggggggggg^$*' adi_chf_db < ./test.sql
So I wanted to make this simple and have a bash script run it for me:
#!/bin/bash
sql_cmd_to_run="sudo mysql -h BLAHBLAHBLAH.us-east-1.rds.amazonaws.com -u user -p\'aaaaa:b>c[d{e]ff=|ggggggggg^$*\' test_database < ./test.sql"
ssh -t test_server "${sql_cmd_to_run}"
My result is the following:
bash: ggggggggg^': command not found
ERROR 1045 (28000): Access denied for user 'user'#'XX.XX.XX.XXX' (using password: YES)
Connection to XX.XX.XX.XXX closed.
I also understand that there are some special characters in bash so I tried the following as well (by putting \ before the special characters):
#!/bin/bash
sql_cmd_to_run="sudo mysql -h BLAHBLAHBLAH.us-east-1.rds.amazonaws.com -u user -p\'aaaaa:b\>c\[d\{e\]ff=\|ggggggggg\^$\*\' test_database < ./test.sql"
ssh -t test_server "${sql_cmd_to_run}"
Which the output is:
ERROR 1045 (28000): Access denied for user 'user'#'XX.XX.XX.XXX' (using password: YES)
Connection to XX.XX.XX.XXX closed.
(I've obscured some of the values, obviously for some security reasons.)
Don't try to do shell quoting by hand: Let the shell do it for you.
So, if you have a working local command:
sudo mysql -h BLAHBLAHBLAH.us-east-1.rds.amazonaws.com -u user -p'aaaaa:b>c[d{e]ff=|ggggggggg^$*' adi_chf_db < ./test.sql
...then encapsulate it in a function, by adding a mycmd() { line before and a } line after:
mycmd() {
sudo mysql -h BLAHBLAHBLAH.us-east-1.rds.amazonaws.com -u user -p'aaaaa:b>c[d{e]ff=|ggggggggg^$*' adi_chf_db < ./test.sql
}
...and tell the shell to serialize that function into your ssh session:
ssh test_server "$(declare -f mycmd); mycmd"
This question already has answers here:
How to use SSH to run a local shell script on a remote machine?
(22 answers)
Closed 5 years ago.
I try to run a local script on multiple remote servers as root. I don't have su to root on those but just can run root commands using sudo. So far I tried:
for host in $(cat hosts_list); do ssh -tt $host "echo mypassword | sudo bash -s" < ./myscript.sh
And in myscript.sh there is something like:
echo "test test123" >> /etc/tests
exit 0
But it looks like not working and won't change the file. What is the proper way to run this script as root and without typing password separately for each host?
Ok, then why do you "echo mypassword" ?
Can't you add your SSH account to the sudoers file with NOPASSWD ?
From man sudoers:
authenticate If set, users must authenticate themselves via a password (or other means
of authentication) before they may run commands. This default may be
overridden via the PASSWD and NOPASSWD tags. This flag is on by default.
This question already has answers here:
How to run the sftp command with a password from Bash script?
(12 answers)
Closed 8 years ago.
Is it possible to pass SFTP USER/PASS to a server in an automated script that will log in and retrieve a file?
I know that KEY PAIRS are the recommended method but assume thats not possible in this case.
In the simplest case you use a key based authorization so you don't need to enter any credentials.
For doing that create a key:
ssh-keygen -t rsa
And copy it to the target system:
ssh-copy-id -i ~/.ssh/id_rsa.pub user#remote-system
Now you can login to the system without a password.
If your problem is the missing ssh-copy-id command try this here:
cat ~/.ssh/*.pub | ssh user#remote-system 'umask 077; cat >>.ssh/authorized_keys'
I am trying to automate a set of procedures that create TEMPLATE databases.
I have a set of files (file1, file2, ... fileN), each of which contains a set of pgsql commands required for creating a TEMPLATE database.
The contents of the file (createdbtemplate1.sql) looks roughly like this:
CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8';
\c mytemplate1
CREATE TABLE first_table (
--- fields here ..
);
-- Add C language extension + functions
\i db_funcs.sql
I want to be able to write a shell script that will execute the commands in the file, so that I can write a script like this:
# run commands to create TEMPLATE db mytemplate1
# ./groksqlcommands.sh createdbtemplate1.sql
for dbname in foo foofoo foobar barbar
do
# Need to simply create a database based on an existing template in this script
psql CREATE DATABASE $dbname TEMPLATE mytemplate1
done
Any suggestions on how to do this? (As you may have guessed, I'm a shell scripting newbie.)
Edit
To clarify the question further, I want to know:
How to write groksqlcommands.sh (a bash script that will run a set of pgsql cmds from file)
How to create a database based on an existing template at the command line
First off, do not mix psql meta-commands and SQL commands. These are separate sets of commands. There are tricks to combine those (using the psql meta-commands \o and \\ and piping strings to psql in the shell), but that gets confusing quickly.
Make your files contain only SQL commands.
Do not include the CREATE DATABASE statement in the SQL files. Create the db separately, you have multiple files you want to execute in the same template db.
Assuming you are operating as OS user postgres and use the DB role postgres as (default) Postgres superuser, all databases are in the same DB cluster on the default port 5432 and the role postgres has password-less access due to an IDENT setting in pg_hba.conf - a default setup.
psql postgres -c "CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8'
TEMPLATE template0"
I based the new template database on the default system template database template0. Basics in the manual here.
Your questions
How to (...) run a set of pgsql cmds from file
Try:
psql mytemplate1 -f file
Example script file for batch of files in a directory:
#! /bin/sh
for file in /path/to/files/*; do
psql mytemplate1 -f "$file"
done
The command option -f makes psql execute SQL commands in a file.
How to create a database based on an existing template at the command line
psql -c 'CREATE DATABASE my_db TEMPLATE mytemplate1'
The command option -c makes psql execute a single SQL command string. Can be multiple commands, terminated by ; - will be executed in one transaction and only the result of the last command returned.
Read about psql command options in the manual.
If you don't provide a database to connect to, psql will connect to the default maintenance database named "postgres". In the second answer it is irrelevant which database we connect to.
you can echo your commands to the psql input:
for dbname in foo foofoo foobar barbar
do
echo """
CREATE DATABASE $dbname TEMPLATE mytemplate1
""" | psql
done
If you're willing to go the extra mile, you'll probably have more success with sqlalchemy. It'll allow you to build scripts with python instead of bash, which is easier and has better control.
As requested in the comments: https://github.com/srathbun/sqlCmd
Store your sql scripts under a root dir
Use dev,tst,prd parametrized dbs
Use find to run all your pgsql scripts as shown here
Exit on errors
Or just git clone the whole tool from here
For that use case where you have to do it....
Here is a script I've used for importing JSON into PostgreSQL (WSL Ubuntu), which basically requires that you mix psql meta commands and SQL in the same command line. Note use of the somewhat obscure script command, which allocates a pseudo-tty:
$ more update.sh
#!/bin/bash
wget <filename>.json
echo '\set content `cat $(ls -t <redacted>.json.* | head -1)` \\ delete from <rable>; insert into <table> values(:'"'content'); refresh materialized view <view>; " | PGPASSWORD=<passwd> psql -h <host> -U <user> -d <database>
$