i have a delete statement delete * from table_name;. Currently shell expands it to list all existing files in current directory. How can i escape it, so that the string that is passed to sqlplus is indeed "delete * from table_name". I tried \* '*' and \\* and none of them work.
The exact script is
#!/bin/bash
table_name = $1
delete_sql= "delete * from $table_name;"
echo $delete_sql > abc.sql
How about
echo "delete * from table_name" | sqlplus
or
echo "delete * from table_name" > mystatements.txt
sqlplus #mystatements.txt
On a side note, you don't need to specify * in a delete statement - all fields in the matching rows are deleted.
You just need to quote the variable (and fix your spacing around the = sign):
#!/bin/bash
table_name=$1
delete_sql="delete * from $table_name;"
echo "$delete_sql" > abc.sql
Related
I use an HPC cluster. The compute nodes can't have access to internet, only the frontal.
So I want to wrap all the commands that need to access internet in order to execute them on the frontal.
ex: for wget
#!/bin/bash
ssh frontal /bin/wget "$#"
-> works fine
I have to wrap this bq (google BigQuery) command:
bq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '2016%' AND mgrs_tile == '32ULU' ORDER BY sensing_time ASC LIMIT 1000;"
I managed to requote the command and to launch it successfully on CLI:
ssh frontal '~/downloads_and_builds/builds/google-cloud-sdk/bin/bq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '"'"'2016%'"'"' AND mgrs_tile == '"'"'32ULU'"'"' ORDER BY sensing_time ASC LIMIT 1000;"'
Now I want to write a wrapper named bq able to get the parameters and launch this command through ssh ... here is what i have tried :
#!/bin/bash
set -eu
# all parameters in an array
args=("$#")
# unset globing (there's a * in the SELECT clause)
set -f
# managing inner quotes
arg2=`echo "${args[2]}" | perl -pe 's/'\''/'\''"'\''"'\''/g'`
# put back double quotes (") suppressed by bash
args="${args[0]} ${args[1]} \"${arg2}\""
# build command with parameters
cmd="~/downloads_and_builds/builds/google-cloud-sdk/bin/bq $args"
echo ""
echo "command without external quotes"
echo "$cmd"
echo ""
echo "testing it ..."
ssh hpc-login1 "$cmd"
echo ""
# wrapping command between simple quotes (like on the CLI)
cmd="'"'~/downloads_and_builds/builds/google-cloud-sdk/bin/bq '"$args""'"
echo "commande with external quotes"
echo "$cmd"
echo ""
echo "testing it ..."
ssh hpc-login1 $cmd
echo "done"
Here is the output of this script:
$ bq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '2016%' AND mgrs_tile == '32ULU' ORDER BY sensing_time ASC LIMIT 1000;"
command without external quotes
~/downloads_and_builds/builds/google-cloud-sdk/bin/bq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '"'"'2016%'"'"' AND mgrs_tile == '"'"'32ULU'"'"' ORDER BY sensing_time ASC LIMIT 1000;"
testing it ...
Waiting on bqjob_r102b0c22cdd77c2d_000001629b8391a3_1 ... (0s) Current status: DONE
commande with external quotes
'~/downloads_and_builds/builds/google-cloud-sdk/bin/bq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '"'"'2016%'"'"' AND mgrs_tile == '"'"'32ULU'"'"' ORDER BY sensing_time ASC LIMIT 1000;"'
testing it ...
bash: ~/downloads_and_builds/builds/google-cloud-sdk/bin/bq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '2016%' AND mgrs_tile == '32ULU' ORDER BY sensing_time ASC LIMIT 1000;": Aucun fichier ou dossier de ce type (in english: no file or directory of this kind)
As you can see, I managed to get a correct command string, just like the one which works on CLI, but it doesn't work in my script:
The first attempt succeeded but gives no output (I have tried to redirect it in a file: the file were created but is empty)
In the second attempt (with external simple quotes, just like the CLI command that worked), bash take the quoted arg as a block and don't find the command ...
Has somebody an idea on how to launch a complex command (with quotes, wildcards ...) like this one through ssh using a wrapper script ?
(ie. one wrapper named foo able to replace a foo command and execute it correctly through ssh with the arguments provided)
ssh has the same semantics as eval: all arguments are concatenated with spaces and then evaluated as a shell command.
You can have it work with execve semantics (like sudo) by having a wrapper escape the arguments:
remotebq() {
ssh yourhost "~/downloads_and_builds/builds/google-cloud-sdk/bin/bq $(printf '%q ' "$#")"
}
This quotes thoroughly and consistently, so you no longer have to worry about adding additional escaping. It'll run exactly what you tell it (as long as your remote shell is bash):
remotebq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '2016%' AND mgrs_tile == '32ULU' ORDER BY sensing_time ASC LIMIT 1000;"
However, the downside to running exactly what you tell it is that now you need to know exactly what you want to run.
For example, you can no longer pass '~/foo' as an argument because this is not a valid file: ~ is a shell feature and not a directory name, and when it's correctly escaped it will not be replaced by your home directory.
The basic way to do this, using shell here doc :
#!/bin/bash
ssh -t server<<'EOF'
bq --format=json query "SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '2016%' AND mgrs_tile == '32ULU' ORDER BY sensing_time ASC LIMIT 1000;"
command2
command3
...
EOF
I see you are already using Perl so...
use Net::OpenSSH;
my $query = q(SELECT * FROM [bigquery-public-data:cloud_storage_geo_index.sentinel_2_index] WHERE sensing_time LIKE '2016%' AND mgrs_tile == '32ULU' ORDER BY sensing_time ASC LIMIT 1000;);
my $ssh = Net::OpenSSH->new($host);
$ssh->system('bq', '--format=json', 'query', $query)
or die $ssh->error;
Net::OpenSSH would take care of quoting everything.
I have a strange problem - possibly I'm just going blind. I have this short script, which replaces the string #qry# in the here-document with a select statement in a file and then pipes it to mysql:
#!/bin/bash
if [[ "$1" == "-h" ]]
then
echo "sqljob [sqlfile] [procnm] [host] [database] [config file]"
echo " sqlfile: text file containing an SQL statement"
echo " procnm: name that will given to the new, stored procedure"
echo " host: hostname of IP address of the database server"
echo " database: the procedure will be created here"
echo " config file: default configuration file with username and password"
exit
fi
infile=$1
procnm=$2
hn=$3
pn=$4
db=$5
mycfg=$6
{
set -o noglob
sed -e "s/#qry#/$(echo $(cat $infile))/g" <<!
drop procedure if exists $procnm;
delete from jobs where jobname="$procnm";
insert into jobs
set
notes="SQL job $procnm",
jobname="$procnm",
parm_tmpl='int';
delimiter //
create procedure $procnm(vqid int)
begin
call joblogmsg(vqid,0,"$procnm","","Executing #qry#");
drop table if exists ${procnm}_res;
create table ${procnm}_res as
#qry#
end//
delimiter ;
!
} | mysql --defaults-file=$mycfg -h $hn -P $pn $db
However, when the select contains *, it expands to whatever is in the directory even though I use noglob. However, it works from the command line:
$ set -o noglob
$ ls *
What am I doing wrong?
Edit
Block Comments in a Shell Script has been suggested as a duplicate, but as you will notice, I need to expand ${procnm} in the here-doc; I just need to avoid the same happening to select *.
I suspect it is because the construct echo (cat). The echo command gets the * from the cat command and the shell in which it runs expands it. In that shell set noglob is not active.
Try leaving the echo away: /$(cat $infile)/, in the end that is the data you need; then there is no extra glob expansion by a shell.
I wrote a simple bash script to that takes the results from a query and appends them to an existing table. My script executes but the data doesn't seem to make it to the destination table. Any idea what i might be doing wrong? is it possible that I can't use a partition ($) as a destination?
Thank you so much for your help.
#!/bin/bash
bq query \
--destination_table=logs.p_activity_428001$20170803 \
--append_table <<EOF
SELECT
*
FROM log.p_activity_428001
where _PARTITIONTIME = TIMESTAMP('2017-08-03')
EOF
You need to escape the dollar sign; bash is expanding the positional parameter $20170803, which is empty unless you provide 20,170,803 arguments to the script. A single backslash will suffice:
#!/bin/bash
bq query \
--destination_table=logs.p_activity_428001\$20170803 \
--append_table <<EOF
SELECT
*
FROM log.p_activity_428001
where _PARTITIONTIME = TIMESTAMP('2017-08-03')
EOF
although single-quoting the whole table name may be more readable:
#!/bin/bash
bq query \
--destination_table='logs.p_activity_428001$20170803' \
--append_table <<EOF
SELECT
*
FROM log.p_activity_428001
where _PARTITIONTIME = TIMESTAMP('2017-08-03')
EOF
I have a bash script like this:
#!/bin/bash
pavadinimas=$1
pavadinimas2=${pavadinimas::-4}
echo "#!/bin/sh
mysql -uUSER -pPASSWORD -DDatabase -e 'UPDATE boom SET count = count + 1 WHERE Failo_vardas="$pavadinimas"';
vlc -f --play-and-exit /var/www/html/uploads/$pavadinimas" > /var/www/html/script/"$pavadinimas2.sh"
And I'm having problem with this line:
mysql -uUSER -pPASSWORD -DDatabase -e 'UPDATE boom SET count = count + 1 WHERE Failo_vardas="$pavadinimas"';
As you see I want to add the variable to quotes, but It comes out without It. I tried a lot of combinations to solve this out, but I failed. Lack of experience :/
Script result:
#!/bin/sh
mysql -uUSER -pPASSWORD -DDatabase -e 'UPDATE boom SET count = parodymai + 1 WHERE Failo_vardas=name.mp4';
vlc -f --play-and-exit /var/www/html/uploads/gaidys.mp4
I want to echo the variable in quotes like this:
mysql -uUSER -pPASSWORD -DDatabase -e 'UPDATE boom SET count = count + 1 WHERE Failo_vardas="name.mp4"';
You are really close. You just have to escape the quotes that you want to use.
e.g. WHERE Failo_vardas=\"$pavadinimas\"
You have to leave the single quoting or your variable won't be evaluated.
So insert a single quote after the double quote, put your variable to evaluate, and re-insert a quote after your variable. Where the single-quoting ends, your env. variable will be evaluated instead of being treated literally.
Demo:
$ pavadinimas=name.mp4
$ echo 'UPDATE boom SET count = count + 1 WHERE Failo_vardas="'$pavadinimas'"';
result:
UPDATE boom SET count = count + 1 WHERE Failo_vardas="name.mp4"
I was wondering how I could pass * in my variable in a bash for loop. I want to run multiple select queries on a db and I thought I'd use a bash for loop like so:
for query in "select * from filtertype limit 10;" "select * from filter limit 10;"; do echo $query; echo $query | mysql -uroot -ppassword database; done
The trouble is the * keeps on getting expanded like so:
[Bash]$ for query in "select * from filtertype limit 10;" "select * from filter limit 10;"; do echo $query; echo $query | mysql -uroot -ppassword database; done
select acceptance_test bin build-dist.xml build-path-definitions.xml build.properties build.xml classes config core-def database dependencies dev.properties dist docs dummyFile_pdffile.pdf eclipse_classes findbugsExcludeFilter.xml generated getRevision.sh local.build.properties log nohup.out plugins-defs plugins-framework pmc-api-client pmd_rules.xml port_helper.sh raw_data-pigunit-input-overridden.txt raw_event_data-pigunit-input-overridden.txt raw-pigunit-input-overridden.txt rpmbuild src testArea unit_test web from filtertype limit 10;
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'build-dist.xml build-path-definitions.xml build.properties build.xml classes con' at line 1
select acceptance_test bin build-dist.xml build-path-definitions.xml build.properties build.xml classes config core-def database dependencies dev.properties dist docs dummyFile_pdffile.pdf eclipse_classes findbugsExcludeFilter.xml generated getRevision.sh local.build.properties log nohup.out plugins-defs plugins-framework pmc-api-client pmd_rules.xml port_helper.sh raw_data-pigunit-input-overridden.txt raw_event_data-pigunit-input-overridden.txt raw-pigunit-input-overridden.txt rpmbuild src testArea unit_test web from filter limit 10;
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'build-dist.xml build-path-definitions.xml build.properties build.xml classes con' at line 1
[Bash]$
When I tried single quotes around the loop variables I got the same error and when I single and double quoted the stars I got:
select \* from filtertype limit 10;
ERROR at line 1: Unknown command '\*'.
select \* from filter limit 10;
ERROR at line 1: Unknown command '\*'.
So the question is - is it possible to pass a string variable with a '*' in it into a bash for loop?
A
The problem is that your variable references, $query, are unquoted, and thus subject to shell expansions (in this case, * expands to the names of files and directories in the current directory, a process called pathname expansion).
Thus, simply double-quote your variable references to protect them from expansions:
echo "$query"
I found the answer. Hope it's at least useful to somebody else. The solution is to quote the variable when it getting called:
for query in "select * from filtertype limit 10;" "select * from filter limit 10;"; do echo "$query"; echo "$query" |mysql -uroot -ppassword database; done
That works
A