How to pass timestamp from bash to psql - bash

I am having problem with passing timestamp parameter to psql. In $since variable I can have any string formatted according to SQL standard and I pass this value to sql like this:
First I check if $since is in correct format (if it fails it won't continue):
1) psql --command "SELECT ($since)::TIMESTAMPTZ;"
Second I use the value in my function (it takes timestamptz as input parameter):
2) cmd="SELECT myfunc($since);"
psql --command "$cmd" $DBNAME
Works: if since="NOW() - INTERVAL '5 months'"
Does not work: if since="2017-10-23 10:42:48" (it fails on LINE 1: SELECT (2017-10-23 10:42:48)::TIMESTAMPTZ; error)
I tried to escape the $since string somehow with ', ", \ characters, but after many combinations both in bash and sql I gave up.
What is the correct way to escape in such case?

If you need to cast a string to TIMESTAMPTZ, so you need to enclose the value of $since in ' ', either when creating the variable
since="'2017-10-23 10:42:48'"
or when passing it to psql:
since="2017-10-23 10:42:48"
psql --command "SELECT '$since'::TIMESTAMPTZ ;"
If you need to pass either a string or an expression like NOW() - INTERVAL '1 day' you would better decide about the quoting when assigning the value to the variable, so:
$ since="'2017-10-23 10:42:48'"
$ psql postgres --command "SELECT $since::TIMESTAMPTZ ;"
timestamptz
------------------------
2017-10-23 10:42:48+02
(1 row)
$ since="(NOW() - INTERVAL '1 day')"
$ psql postgres --command "SELECT $since::TIMESTAMPTZ ;"
timestamptz
-------------------------------
2018-03-24 08:49:24.577356+01
(1 row)

Related

Passing parameter in a BigQuery Script

I want to pass argument to a BigQuery script in shell, here is the example of script I wrote
#!/bin/bash
bq query --use_legacy_sql=false --destination_table=abc --append 'select * from `xyz.INFORMATION_SCHEMA.VIEWS` union all Select * from `def.VIEWS`) where table_name = "$1"'
when I run this script and pass the argument, I do not get any errors but no row is appended to the table. whereas when i specify the table_name as rty that row is appended to the table. What am I missing here?
When you run the script you'll get a prompt like:
Waiting on <BIGQUERY_JOB_ID> ... (0s) Current status: DONE
You can inspect the job in many ways, including the bqtool:
bq show -j --format=prettyjson <BIGQUERY_JOB_ID>
If you have jq installed (sudo apt install jq) you can get just the translated query with:
bq show -j --format=prettyjson <BIGQUERY_JOB_ID> | jq '.configuration.query.query'
which will get you something similar to:
select * from xyz.INFORMATION_SCHEMA.VIEWS where table_name = \"$1\"
As you can see the variable is not correctly escaped so no table matches the WHERE filter. To avoid this you can enclose the query in double quotes and the variable in single ones like this:
#!/bin/bash
bq query \
--use_legacy_sql=false \
--destination_table=xyz.abc \
--append \
"select * from xyz.INFORMATION_SCHEMA.VIEWS where table_name='$1'"
You can get the INFORMATION_SCHEMA.VIEWS: command not found error if using back-ticks. You can omit or escape them with a backslash:
"select * from \`xyz\`.INFORMATION_SCHEMA.VIEWS where table_name='$1'"

Error when trying to insert values in Postgres query inside Bash script

I'm executing the query as follows:
ssh user#XX.XX.1XX.XX "PGPASSWORD=myPassword psql -U psqlUser -h XX.XX.XX.XX -p 5432 -d myDB -c
'INSERT INTO table(\"CPU_IDLE_TIME\",\"TOTAL_SIZE\",\"USED_SIZE\",\"USED_STORAGE_P\") VALUES ($idlecputime,$totalSize,$usedSize,$usedStoragePercentage)';"
I obtain the values previous to this query doing snmpwalks. In order for the query to work the values have to be surrounded by single quotes (' '). I tried putting single quotes around the variable but everytime I get an error because the query is already surroundes by " ' ' ". I can't seem to find the configuration of quotes, or scaping quotes to make it work.
The variables are of type var char, integer and float.
One of the errores I get:
ERROR: syntax error at or near ","
Thanks in advance for your help.
Use printf to format the string for you
ssh user#XX.XX.1XX.XX "PGPASSWORD=myPassword psql -U psqlUser -h XX.XX.XX.XX -p 5432 -d myDB -c ""$(printf 'INSERT INTO table("CPU_IDLE_TIME","TOTAL_SIZE","USED_SIZE","USED_STORAGE_P") VALUES (%d, %d, %d, %d);' $idlecputime $totalSize $usedSize $usedStoragePercentage)"
The $( ) construct executes printf in a subshell.

Find if a PostgreSQL database exists with bash

I have a bash function where I check if a PostgreSQL database already exists.
I capture the output. If database exist PostgreSQL returns the database name as response.
function is_database() {
local database=$1
local output=$(sudo -u postgres psql -c "SELECT datname FROM pg_catalog.pg_database WHERE datname=\"$database\";")
if [[ $output = *"${1}"* ]]
then
return 0
else
return 1
fi
}
is_database test
I get the following error:
column "test" does not exist
I am not searching for a table, but a database.
Use single quotes for string literals:
sudo -u postgres psql \
-c "SELECT datname FROM pg_catalog.pg_database WHERE datname='$database'"
Your code as it is won't work for database names like has spaces or has'quotes.

MonetDB doesn't recognize function names given to mclient via command line

I am trying to export a few columns from a table as encoded integer.
Basically I want to use a bash script to pass the SQL command to mclient as command line argument. My bash script looks like:
#!/bin/bash
dir=`pwd`
for col in occupation native_country martial_status race sex
do
mclient -d adult -s \
"create function encode${col}(s varchar(200)) returns int begin return (select code from ${col}_dict where ${col}=s); end;"
mclient -d adult -s \
"COPY (select encode${col}($col) from adult) INTO '${dir}/${col}.txt' NULL AS '0'"
mclient -d adult -s \
"drop function encode${col}"
done
In each iteration, I want to create a SQL function on the fly. Then use the function to encode an attribute and export it to a text file. And lastly drop the function.
However, the output strangely contains some monster characters,
as if it can't recognize the function name.
If I remove the second mclient command, the other operations are successful.
operation successful
Function 'X�..X�' not defined
operation successful
operation successful
Function '��"X�.X�' not defined
operation successful
operation successful
Function ' X�.PX�' not defined

Removing leading \n while assaigning to a variable

I have a DB2 query in a shell script which return an integer value, but I am unable to store it in a variable.
temp1= echo db2 -x "select max(id) from work.work_tb"
I am getting this output when I run it, sh -x test.sh
db2 -x select max(id) from work.work_tb
echo 50
temp1=
50
So for some reason $temp1 is unable to get the value, I think its because the db2 query is returning value prefixed with \n. How do I get rid of the newline char and get the value to temp1?
No, that's not why.
temp1=`db2 -x "select max(id) from work.work_tb"`
emp1=$(echo db2 -x "select max(id) from work.work_tb")
or using backticks
emp1=`echo db2 -x "select max(id) from work.work_tb"`
In general, to remove newlines, you can pass it to tools like tr/sed etc
... | tr -d "\n"

Resources