I have a CentOS linux machine that has postgres DB installed on.
I connected to this machine via ssh and I ran the next command in order to print a certain table.
sudo -u postgres bash -c "psql -d db -c \"SELECT *
FROM accounts;\"" 2>/dev/null
this table contains 800 rows BUT the output of this command only prints 38 rows.
Can anyone tell me how I can fix it?
The solution was to change the end of the command from 2>/dev/null to >/dev/null
Related
The command which on line 4 on the script below seems to have an issue, intellij says
which is non-standard. Use builtin 'command -v' instead
Since which psql seems like it is not working it automatically affects line 12 and 13.
While investigating i removed line 4 then the script executed line 6 to 10 which succefully created a docker file(pg-docker) however i also need the schema.sql (line 12) and data.sql (13) to be executed. Is there an alternative command for which command(line 4)
Below is my bash Script:
#!/usr/bin/env bash
set -euo pipefail
which psql > /dev/null || (echo "Please ensure that postgres client is in your PATH" && exit 1)
mkdir -p $HOME/docker/volumes/postgres
rm -rf $HOME/docker/volumes/postgres/data
docker run --rm --name pg-docker -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=dev -d -p 432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql postgres
sleep 3
export PGPASSWORD=postgres
psql -U postgres -d dev -h localhost -f schema.sql
psql -U postgres -d dev -h localhost -f data.sql
I get the below on the problems on Intellij
line 4 complains about which command
line 6,7 and 9 complains about $HOME
line 11 complains about PGPASSWORD
which is used to find and show the full path of a command (in your script it is only used to make sure the command psql is there).
IntelliJ or probably better the defined linter for (bash) scripts suggest not to rely on an separate whichcommand but just use the builtin bash-function command -v so the line 4 would read
command -v psql > /dev/null || (echo "Please ensure that postgres client is in your PATH" && exit 1)
That said - it's most likely not your real problem. You need the PostgreSQL Client psql installed and in your PATH variable to run the commands in line 12 and 13. Exactly that's what you're checking in line 4 - regardless of using which or command -v.
How to install the psql command depends on your OS.
I want to export a random entry of my database into a file with the command
SELECT * FROM my_table ORDER BY RANDOM() LIMIT 1 \g /path/file;
This query works if I enter it in my db terminal, but I want to us this query with a bash script but then I get the error: syntax error at or near "\g"
My bash script looks like this:
PGPASSWORD=*** psql -U user -d db_name -h localhost -p port -t -c "SELECT * FROM my_table ORDER BY RANDOM() LIMIT 1 \g /path/file"
Bash is interpreting the string and trying to interpolate it. Probably, escaping the backslash will solve your problem.
PGPASSWORD=*** psql -U user -d db_name -h localhost -p port -t -c "SELECT * FROM my_table ORDER BY RANDOM() LIMIT 1 \\g /path/file"
A SQL statement terminated by \g is not supported by the -c command switch. Per documentation of -c:
-c command
...
command must be either a command string that is completely parsable by the server (i.e., it contains no psql-specific features), or a single backslash command. Thus you cannot mix SQL
and psql meta-commands with this option
To redirect the results to a file, there are several options:
shell redirection: psql [other options] -Atc 'SELECT...' >/path/to/data.txt
-A is to switch to unaligned mode (no space fillers to align columns).
put the SQL part in a heredoc text instead of the command line:
psql [options] <<EOF
SELECT ... \g /path/to/file
EOF
This form has the advantage that multiline statements or multiple statements are supported directly.
\copy of the query. Be aware that COPY to a FILE is different: it creates the file on the server with the permissions of postgres and requires being a database superuser. COPY TO STDOUT works too but is not better than SELECT concerning the redirection.
I found a solution for my script, and now it works.
#!/bin/bash
RANDOM_NUMBER=0
while true
do
for i in `seq 1`
do
RANDOM_NUMBER=$(($RANDOM % 100000))
echo $RANDOM_NUMBER
PGPASSWORD=*** psql -U user_name -d db_name -h localhost -p PORT -c
"INSERT INTO numbers (number) VALUES ('$RANDOM_NUMBER');"
done
sleep 10
for i in `seq 1`
do
PGPASSWORD=*** psql -U user_name -d db_name -h localhost -p PORT -c
"DELETE FROM numbers WHERE id = (SELECT id FROM numbers ORDER BY RANDOM() LIMIT 1);"
done
done
I'm having an issue with a simple script here. Just can't find documented help that resolves my issue.
Here is my script.
#!/bin/bash
$VCOPS_BASE/cassandra/apache-cassandra-2.1.8/bin/cqlsh --ssl --cqlshrc $VCOPS_BASE/user/conf/cassandra/cqlshrc
-e "cql_statement;"
I left out the cql for simplicity's sake, but everytime I run my file from the command line I simply enter the cql shell.
--execute and echo don't work either and I'm really not sure why I would need to save the cql statement to another file.
Any help would be appreciated.
That's because -e is a cqlsh option, not a bash command that stands on its own. Therefore, it needs to be on the same line as your cqlsh command.
#!/bin/bash
$VCOPS_BASE/cassandra/apache-cassandra-2.1.8/bin/cqlsh --ssl --cqlshrc $VCOPS_BASE/user/conf/cassandra/cqlshrc -e "cql_statement;"
I tested this out with a simpler version:
aploetz#dockingBay94:~/scripts$ cat getEmail.sh
#!/bin/bash
cqlsh -u cassandra -p cassandra -e "SELECT * FROm stackoverflow.users_by_email WHERe email='mreynolds#serenity.com';"
aploetz#dockingBay94:~/scripts$ ./getEmail.sh
email | id | username
------------------------+--------------------------------------+----------
mreynolds#serenity.com | d8e57eb4-c837-4bd7-9fd7-855497861faf | Mal
(1 rows)
aploetz#dockingBay94:~/scripts$
Hi I want to automate my workflow of creating Postgres dump and downloading it. I want to do it from my local machine, for now I figured it out so far in two seperate commands:
sshpass -p "FuckinHardPass" ssh andi#1.2.3.4 "pg_dump -U andi andi_some_db -f /home/andi/PSQL_DUMPS/andi_some_db.sql"
sshpass -p "FuckinHardPass" scp -r andi#1.2.3.4:/home/andi/PSQL_DUMPS/andi_some_db.sql .
how can I join it in one command using pipes etc?
Thanks James Hightower for hint, using your answer I complete it in one command:
sshpass -p "FuckinHardPass" ssh andi#1.2.3.4 "pg_dump -U andi andi_some_db" > andi_some_db.sql
As pg_dump defaults to stdout as it's output file, and ssh displays the command's stdout on it's own stdout, you could do something like:
ssh andi#1.2.3.4 'pg_dump -U andi andi_some_db' > andi_some_db.sql
which would save the output from the command on your local disk as andi_some_db.sql
Depending on the size of your dump and the speed of your connection, you could perhaps benefit from pre-compressing your output:
ssh andi#1.2.3.4 'pg_dump -U andi andi_some_db | gzip' > andi_some_db.sql.gz
And so on.
I tried this but it does not work.
#!/bin/bash
TABLENAMES="user_stats"
ssh -t railsapps#xxx.xxx.xxx.xx -p xxx bash -c "'
for TABLENAME in $TABLENAMES
do
psql -d mydb -P format=unaligned -P tuples_only -P fieldsep=\, -c "SELECT * FROM $TABLENAME" > /tmp/$TABLENAME
done
'"
You should use the Postgres COPY command, and stuff them all in to a single sql file, that you then run with psql. Then you can take that script and either feed it to cron on Mac OS X, or launchd to run the script periodically.