I need to update values in a column when the row matches a certain WHERE clause, using the content of a text file.
The content of the file is javascript code and as such it may contain single quotes, double quotes, slashes and backslashes - out of the top of my mind, it could contain other special characters.
The content of the file cannot be modified.
This has to be done via psql, since the update is automated using bash scripts.
Using the following command - where scriptName is a previously declared bash variable -
psql -U postgres db<<EOF
\set script $(cat $scriptName.js))
UPDATE table SET scriptColumn=:script WHERE nameColumn='$scriptName';
EOF
returns the following error
ERROR: syntax error at or near "{"
LINE 1: ...{//...
^
I would like to treat the content of the file $scriptName.js as plain text, and avoid any interpretation of it.
You should quote the variable:
UPDATE table SET scriptColumn=:'script' WHERE ...
That causes the contents of the variable to be properly escaped as a string literal.
I found a solution to my problem, even though I don't know why it works.
I leave it here in the hope it might be useful to someone else, or that someone more knowledgeable than me will be able to explain why it works now.
In short, setting the variable as a psql parameter did the trick:
psql -U postgres db -v script="$(cat $scriptName.js)"<<EOF
UPDATE table SET scriptColumn=:'script' WHERE nameColumn='$scriptName'
EOF
Not sure how this differs from
psql -U postgres db <<EOF
\set script "$(cat $scriptName.js)"
UPDATE table SET scriptColumn=:'script' WHERE nameColumn='$scriptName'
EOF
which I tried previously and returns the following error:
unterminated quoted string
ERROR: syntax error at or near "//"
LINE 1: // dummy text blahblah
Thanks to everybody who helped!
This question already has answers here:
Expansion of variables inside single quotes in a command in Bash
(8 answers)
Closed 4 years ago.
I have the following command which does not work:
sqlite3 my_db.sqlite "SELECT name FROM sqlite_master WHERE type = 'table';" | for i in $(cat) ; do sqlite3 my_db.sqlite 'SELECT * FROM "${i}"'; done
To explain it quickly: the first part below is supposed to retrieve the table names from a sqlite file that I have:
sqlite3 my_db.sqlite "SELECT name FROM sqlite_master WHERE type = 'table';"
And this part is supposed to display the entire content of each table recursivelyin the stdout:
for i in $(cat) ; do sqlite3 my_db.sqlite 'SELECT * FROM "${i}"'; done
The problem is that I have no idea how I am supposed to pass i to the sqlite command. I tried with "${i}" but obviously it is interpreted as a classic string to find a matching table name, and just return Error: no such table: ${i}
How should I pass i ?
Thank you in advance for your help.
You must use double quotes around the query to allow bash to recognize the variable. This implies that you have to correctly escape the double quotes that are part of the SQL statement:
... | for i ... ; do sqlite3 my_db.sqlite "SELECT * FROM \"${i}\""; done
This question already has answers here:
Bash syntax error: unexpected end of file
(21 answers)
Closed 6 years ago.
I have prepared a shell script which takes 6 char(s) from a file and insert a particular record using sql insert query within a while loop(as the number of records in file is huge).
Upon running the script I am receiving below error -
./TestScriptFor.sh: line 40: syntax error: unexpected end of file
#!/bin/bash
varSIAREQUESTID=$1
varCINAME=$2
#varWORDLENGTH=$3
#DB connection parameter
DBUSER='#USERNAME#'
DBUSERPASSWORD='#PWD#'
DB='oracle'
MYDB='#CONNECTIONSTRING#'
OLDIFS=$IFS
# while loop
while IFS= read -r -n6 LOCATIONID
do
#insert into database
sqlplus -s ${DBUSER}/${DBUSERPASSWORD}#${MYDB} <<EOT
set linesize 32767
set feedback off
set heading off
insert into nbn_sia_locationids (siarequestid,locationid,ciname) values('$varSIAREQUESTID','$LOCATIONID','$CINAME');
exit
EOT
echo "$LOCATIONID"
# display one character at a time
done < TestData.csv
IFS=$OLDIFS
Thanks for your responses.
I did below two steps to resolved the issue -
1. dos2unix TestData.csv - although I had created this file in UNIX itself but did this as well.
2. a new line at the end of the file as suggested by #cdark
regards,
dhananjay
I've a concern which can be categorized in 2 ways:
My requirement is of passing argument from shell script to hive script.
OR
within one shell script I should include variable's value in hive statement.
I'll explain with an example for both:
1) Passing argument from shell script to hiveQL->
My test Hive QL:
select count(*) from demodb.demo_table limit ${hiveconf:num}
My test shell script:
cnt=1
sh -c 'hive -hiveconf num=$cnt -f countTable.hql'
So basically I want to include the value of 'cnt' in the HQL, which is not happening in this case. I get the error as:
FAILED: ParseException line 2:0 mismatched input '<EOF>' expecting Number near 'limit' in limit clause
I'm sure the error means that the variable's value isn't getting passed on.
2) Passing argument directly within the shell script->
cnt=1
hive -e 'select count(*) from demodb.demo_table limit $cnt'
In both the above cases, I couldn't pass the argument value. Any ideas??
PS: I know the query seems absurd of including the 'limit' in count but I have rephrased the problem I actually have. The requirement remains intact of passing the argument.
Any ideas, anyone?
Thanks in advance.
Set the variable this way:
#!/bin/bash
cnt=3
echo "Executing the hive query - starts"
hive -hiveconf num=$cnt -e ' set num; select * from demodb.demo_table limit ${hiveconf:num}'
echo "Executing the hive query - ends"
This works, if put in a file named hivetest.sh, then invoked with sh hivetest.sh:
cnt=2
hive -e "select * from demodb.demo_table limit $cnt"
You are using single quotes instead of double.
Using double quotes for OPTION #1 also works fine.
hadoop#osboxes:~$ export val=2;
hadoop#osboxes:~$ hive -e "select * from bms.bms1 where max_seq=$val";
or
vi test.sh
#########
export val=2
hive -e "select * from bms.bms1 where max_seq=$val";
#####################
Try this
cnt=1
hive -hiveconf number=$cnt select * from demodb.demo_table limit ${hiveconf:number}
I'm probably missing something very basic. I have a web script that tells a shell script to update some database records. These records are for statistics, and this way the web script does not have to wait while the database records are updated. However, I can't actually make the shell script work running it from commandline. Here is the code that I'm trying:
perl async_sql.pl 'UPDATE some_table set i = i + 1 WHERE (n in (\'328430\',\'334969\',\'330179\',\'335290\',\'335285\',\'335284\',\'335264\',\'335145\',\'335146\',\'335147\',\'335148\',\'335149\',\'335230\',\'335201\',\'335198\',\'335196\',\'335167\',\'335151\',\'335152\',\'335143\',\'334969\',\'334972\',\'334977\',\'334978\',\'334979\',\'334980\',\'334982\',\'334983\',\'334984\',\'334934\',\'334947\',\'334948\',\'334950\',\'334992\',\'335014\',\'335026\',\'335030\',\'335032\',\'334864\',\'334862\',\'334861\',\'334858\',\'334855\',\'334852\',\'334850\',\'334849\',\'334848\',\'334847\',\'334844\',\'334842\'))'
Bash tells me:
-bash: syntax error near unexpected token `)'
What am I missing?
It is not possible to escape single quote in single quotes. Use " " instead
perl async_sql.pl "UPDATE some_table set i = i + 1 WHERE (n in ('328430','334969','330179','335290','335285','335284','335264','335145','335146','335147','335148','335149','335230','335201','335198','335196','335167','335151','335152','335143','334969','334972','334977','334978','334979','334980','334982','334983','334984','334934','334947','334948','334950','334992','335014','335026','335030','335032','334864','334862','334861','334858','334855','334852','334850','334849','334848','334847','334844','334842'))"
Also, there are other ways to deal with this problem:
echo "quote'test"
echo 'quote'"'"'test'
echo 'quote'\''test'
echo $'quote\'test'
All these should print quote'test as a single parameter. Which means that another great solution for your problem is:
perl async_sql.pl $'UPDATE some_table set i = i + 1 WHERE (n in (\'328430\',\'334969\',\'330179\',\'335290\',\'335285\',\'335284\',\'335264\',\'335145\',\'335146\',\'335147\',\'335148\',\'335149\',\'335230\',\'335201\',\'335198\',\'335196\',\'335167\',\'335151\',\'335152\',\'335143\',\'334969\',\'334972\',\'334977\',\'334978\',\'334979\',\'334980\',\'334982\',\'334983\',\'334984\',\'334934\',\'334947\',\'334948\',\'334950\',\'334992\',\'335014\',\'335026\',\'335030\',\'335032\',\'334864\',\'334862\',\'334861\',\'334858\',\'334855\',\'334852\',\'334850\',\'334849\',\'334848\',\'334847\',\'334844\',\'334842\'))'
I have only placed a dollar sign $ right before the parameter. That's it. A dollar sign before single quotes turns on ANSI-C Quoting
bash: syntax error near unexpected token `('
this type of error can be solve by the turn on extended globbing by git bash command is "shopt -s extglob"