I have a DB2 query in a shell script which return an integer value, but I am unable to store it in a variable.
temp1= echo db2 -x "select max(id) from work.work_tb"
I am getting this output when I run it, sh -x test.sh
db2 -x select max(id) from work.work_tb
echo 50
temp1=
50
So for some reason $temp1 is unable to get the value, I think its because the db2 query is returning value prefixed with \n. How do I get rid of the newline char and get the value to temp1?
No, that's not why.
temp1=`db2 -x "select max(id) from work.work_tb"`
emp1=$(echo db2 -x "select max(id) from work.work_tb")
or using backticks
emp1=`echo db2 -x "select max(id) from work.work_tb"`
In general, to remove newlines, you can pass it to tools like tr/sed etc
... | tr -d "\n"
Related
I want to pass argument to a BigQuery script in shell, here is the example of script I wrote
#!/bin/bash
bq query --use_legacy_sql=false --destination_table=abc --append 'select * from `xyz.INFORMATION_SCHEMA.VIEWS` union all Select * from `def.VIEWS`) where table_name = "$1"'
when I run this script and pass the argument, I do not get any errors but no row is appended to the table. whereas when i specify the table_name as rty that row is appended to the table. What am I missing here?
When you run the script you'll get a prompt like:
Waiting on <BIGQUERY_JOB_ID> ... (0s) Current status: DONE
You can inspect the job in many ways, including the bqtool:
bq show -j --format=prettyjson <BIGQUERY_JOB_ID>
If you have jq installed (sudo apt install jq) you can get just the translated query with:
bq show -j --format=prettyjson <BIGQUERY_JOB_ID> | jq '.configuration.query.query'
which will get you something similar to:
select * from xyz.INFORMATION_SCHEMA.VIEWS where table_name = \"$1\"
As you can see the variable is not correctly escaped so no table matches the WHERE filter. To avoid this you can enclose the query in double quotes and the variable in single ones like this:
#!/bin/bash
bq query \
--use_legacy_sql=false \
--destination_table=xyz.abc \
--append \
"select * from xyz.INFORMATION_SCHEMA.VIEWS where table_name='$1'"
You can get the INFORMATION_SCHEMA.VIEWS: command not found error if using back-ticks. You can omit or escape them with a backslash:
"select * from \`xyz\`.INFORMATION_SCHEMA.VIEWS where table_name='$1'"
I'm executing the query as follows:
ssh user#XX.XX.1XX.XX "PGPASSWORD=myPassword psql -U psqlUser -h XX.XX.XX.XX -p 5432 -d myDB -c
'INSERT INTO table(\"CPU_IDLE_TIME\",\"TOTAL_SIZE\",\"USED_SIZE\",\"USED_STORAGE_P\") VALUES ($idlecputime,$totalSize,$usedSize,$usedStoragePercentage)';"
I obtain the values previous to this query doing snmpwalks. In order for the query to work the values have to be surrounded by single quotes (' '). I tried putting single quotes around the variable but everytime I get an error because the query is already surroundes by " ' ' ". I can't seem to find the configuration of quotes, or scaping quotes to make it work.
The variables are of type var char, integer and float.
One of the errores I get:
ERROR: syntax error at or near ","
Thanks in advance for your help.
Use printf to format the string for you
ssh user#XX.XX.1XX.XX "PGPASSWORD=myPassword psql -U psqlUser -h XX.XX.XX.XX -p 5432 -d myDB -c ""$(printf 'INSERT INTO table("CPU_IDLE_TIME","TOTAL_SIZE","USED_SIZE","USED_STORAGE_P") VALUES (%d, %d, %d, %d);' $idlecputime $totalSize $usedSize $usedStoragePercentage)"
The $( ) construct executes printf in a subshell.
I am having problem with passing timestamp parameter to psql. In $since variable I can have any string formatted according to SQL standard and I pass this value to sql like this:
First I check if $since is in correct format (if it fails it won't continue):
1) psql --command "SELECT ($since)::TIMESTAMPTZ;"
Second I use the value in my function (it takes timestamptz as input parameter):
2) cmd="SELECT myfunc($since);"
psql --command "$cmd" $DBNAME
Works: if since="NOW() - INTERVAL '5 months'"
Does not work: if since="2017-10-23 10:42:48" (it fails on LINE 1: SELECT (2017-10-23 10:42:48)::TIMESTAMPTZ; error)
I tried to escape the $since string somehow with ', ", \ characters, but after many combinations both in bash and sql I gave up.
What is the correct way to escape in such case?
If you need to cast a string to TIMESTAMPTZ, so you need to enclose the value of $since in ' ', either when creating the variable
since="'2017-10-23 10:42:48'"
or when passing it to psql:
since="2017-10-23 10:42:48"
psql --command "SELECT '$since'::TIMESTAMPTZ ;"
If you need to pass either a string or an expression like NOW() - INTERVAL '1 day' you would better decide about the quoting when assigning the value to the variable, so:
$ since="'2017-10-23 10:42:48'"
$ psql postgres --command "SELECT $since::TIMESTAMPTZ ;"
timestamptz
------------------------
2017-10-23 10:42:48+02
(1 row)
$ since="(NOW() - INTERVAL '1 day')"
$ psql postgres --command "SELECT $since::TIMESTAMPTZ ;"
timestamptz
-------------------------------
2018-03-24 08:49:24.577356+01
(1 row)
I wrote a simple bash script to that takes the results from a query and appends them to an existing table. My script executes but the data doesn't seem to make it to the destination table. Any idea what i might be doing wrong? is it possible that I can't use a partition ($) as a destination?
Thank you so much for your help.
#!/bin/bash
bq query \
--destination_table=logs.p_activity_428001$20170803 \
--append_table <<EOF
SELECT
*
FROM log.p_activity_428001
where _PARTITIONTIME = TIMESTAMP('2017-08-03')
EOF
You need to escape the dollar sign; bash is expanding the positional parameter $20170803, which is empty unless you provide 20,170,803 arguments to the script. A single backslash will suffice:
#!/bin/bash
bq query \
--destination_table=logs.p_activity_428001\$20170803 \
--append_table <<EOF
SELECT
*
FROM log.p_activity_428001
where _PARTITIONTIME = TIMESTAMP('2017-08-03')
EOF
although single-quoting the whole table name may be more readable:
#!/bin/bash
bq query \
--destination_table='logs.p_activity_428001$20170803' \
--append_table <<EOF
SELECT
*
FROM log.p_activity_428001
where _PARTITIONTIME = TIMESTAMP('2017-08-03')
EOF
I have a file test.sh in which i have a query which gives me the count of total records in the table, as shown below:
Query:
echo db2 -x "select count(*) from testable”
Now I want to assign the whatever value it returns after executing it to the variable as totalRecords using the Unix Shell Script.
#!/bin/bash
cnt=`db2 -x "select count(*) from syscat.tables" `
echo "Counter is: ${cnt}"
Counter is: 474