Executing in bash
sqlite3 database.db 'select * from databases'
gives me good output, but unfortunately when I would like to create a query from a variable sqlite3 doesn't want to cooperate.
For example:
cmd="select * from databases"
cmd1="sqlite3 database.db"
cmd2="'select * from databases'"
echo $($cmd1 \'$cmd\')
echo $($cmd1 '$cmd')
echo $($cmd1 $cmd)
echo $($cmd1 $cmd2)
echo `$cmd1 $cmd2`
None of above works. I would like to have a function which would execute just query like "select * from databases".
Your trouble is word splitting. When you run this:
sqlite3 database.db $sql
The shell splits $sql to words. But sqlite3 expects the SQL command as a single parameter. To prevent word splitting, you must enclose $sql in double-quotes:
sqlite3 database.db "$sql"
If you want to store the sqlite3 database.db in a variable called cmd,
you can, and write:
$cmd "$sql"
The shell will split $cmd to words, and in this case you want that.
You want the shell to see the sqlite3 command, and another word database.db as the first argument. That's way we don't enclose $cmd in double-quotes.
Let's try this approach:
cmd1="sqlite3 database.db"
cmd2="select * from databases"
$cmd1 "$cmd2"
I've just tried - it works for me:
Related
I'm trying to write a systemd service file without resorting to using an external script.
I need to query an sqlite database and write the contents to a file. But my query uses double quotes, I need to wrap the query in single quotes and since systemd doesn't use a shell, I need to manually use one. So how do I accomplish this?
ExecStart=sh -c 'sqlite3 dbfile.db 'SELECT "The db value is: "||value FROM table' > output.log'
I have tried escaping the inner single quotes, but for some reason that doesn't work.
Try this:
ExecStart=sh -c 'sqlite3 dbfile.db '\''SELECT "The db value is: "||value FROM table'\'' > output.log'
I used to use mysql and double quotes work as well. You can also give it a shot:
ExecStart=sh -c 'sqlite3 dbfile.db "SELECT \"The db value is: \"||value FROM table" > output.log'
This question already has answers here:
Expansion of variables inside single quotes in a command in Bash
(8 answers)
Closed 4 years ago.
I have the following command which does not work:
sqlite3 my_db.sqlite "SELECT name FROM sqlite_master WHERE type = 'table';" | for i in $(cat) ; do sqlite3 my_db.sqlite 'SELECT * FROM "${i}"'; done
To explain it quickly: the first part below is supposed to retrieve the table names from a sqlite file that I have:
sqlite3 my_db.sqlite "SELECT name FROM sqlite_master WHERE type = 'table';"
And this part is supposed to display the entire content of each table recursivelyin the stdout:
for i in $(cat) ; do sqlite3 my_db.sqlite 'SELECT * FROM "${i}"'; done
The problem is that I have no idea how I am supposed to pass i to the sqlite command. I tried with "${i}" but obviously it is interpreted as a classic string to find a matching table name, and just return Error: no such table: ${i}
How should I pass i ?
Thank you in advance for your help.
You must use double quotes around the query to allow bash to recognize the variable. This implies that you have to correctly escape the double quotes that are part of the SQL statement:
... | for i ... ; do sqlite3 my_db.sqlite "SELECT * FROM \"${i}\""; done
I've a concern which can be categorized in 2 ways:
My requirement is of passing argument from shell script to hive script.
OR
within one shell script I should include variable's value in hive statement.
I'll explain with an example for both:
1) Passing argument from shell script to hiveQL->
My test Hive QL:
select count(*) from demodb.demo_table limit ${hiveconf:num}
My test shell script:
cnt=1
sh -c 'hive -hiveconf num=$cnt -f countTable.hql'
So basically I want to include the value of 'cnt' in the HQL, which is not happening in this case. I get the error as:
FAILED: ParseException line 2:0 mismatched input '<EOF>' expecting Number near 'limit' in limit clause
I'm sure the error means that the variable's value isn't getting passed on.
2) Passing argument directly within the shell script->
cnt=1
hive -e 'select count(*) from demodb.demo_table limit $cnt'
In both the above cases, I couldn't pass the argument value. Any ideas??
PS: I know the query seems absurd of including the 'limit' in count but I have rephrased the problem I actually have. The requirement remains intact of passing the argument.
Any ideas, anyone?
Thanks in advance.
Set the variable this way:
#!/bin/bash
cnt=3
echo "Executing the hive query - starts"
hive -hiveconf num=$cnt -e ' set num; select * from demodb.demo_table limit ${hiveconf:num}'
echo "Executing the hive query - ends"
This works, if put in a file named hivetest.sh, then invoked with sh hivetest.sh:
cnt=2
hive -e "select * from demodb.demo_table limit $cnt"
You are using single quotes instead of double.
Using double quotes for OPTION #1 also works fine.
hadoop#osboxes:~$ export val=2;
hadoop#osboxes:~$ hive -e "select * from bms.bms1 where max_seq=$val";
or
vi test.sh
#########
export val=2
hive -e "select * from bms.bms1 where max_seq=$val";
#####################
Try this
cnt=1
hive -hiveconf number=$cnt select * from demodb.demo_table limit ${hiveconf:number}
I'm trying to write a script which lists a directory and creates an SQL script to insert these directories, problem is I only want to insert new directories, here is what I have so far:
#If file doesn't exist add the search path test
if [ ! -e /home/aydin/movies.sql ]
then
echo "SET SEARCH_PATH TO noti_test;" >> /home/aydin/movies.sql;
fi
cd /media/htpc/
for i in *
do
#for each directory escape any single quotes
movie=$(echo $i | sed "s:':\\\':g" )
#build sql insert string
insertString="INSERT INTO movies (movie) VALUES (E'$movie');";
#if sql string exists in file already
if grep -Fxq "$insertString" /home/aydin/movies.sql
then
#comment out string
sed -i "s/$insertString/--$insertString/g" /home/aydin/movies.sql
else
#add sql string
echo $insertString >> /home/aydin/movies.sql;
fi
done;
#execute script
psql -U "aydin.hassan" -d "aydin_1.0" -f /home/aydin/movies.sql;
It seems to work apart from one thing, the script doesn't recognise entries with single quotes in them, so upon running the script again with no new dirs, this is what the file looks like:
--INSERT INTO movies (movie) VALUES (E'007, Moonraker (1979)');
--INSERT INTO movies (movie) VALUES (E'007, Octopussy (1983)');
INSERT INTO movies (movie) VALUES (E'007, On Her Majesty\'s Secret Service (1969)');
I'm open to suggestions on a better way to do this also, my process seems pretty elongated and inefficient :)
Script looks generally good to me. Consider the revised version (untested):
#! /bin/bash
#If file doesn't exist add the search path test
if [ ! -e /home/aydin/movies.sql ]
then
echo 'SET search_path=noti_test;' > /home/aydin/movies.sql;
fi
cd /media/htpc/
for i in *
do
#build sql insert string - single quotes work fine inside dollar-quoting
insertString="INSERT INTO movies (movie) SELECT \$x\$$movie\$x\$
WHERE NOT EXISTS (SELECT 1 FROM movies WHERE movie = \$x\$$movie\$x\$);"
#no need for grep. SQL is self-contained.
echo $insertString >> /home/aydin/movies.sql
done
#execute script
psql -U "aydin.hassan" -d "aydin_1.0" -f /home/aydin/movies.sql;
To start a new file, use > instead of >>
Use single quotes ' for string constants without variables to expand
Use PostgreSQL dollar-quoting so you don't have to worry about single-quotes in the strings. You'll have to escape the $ character in the shell to remove its special meaning in the shell.
Use an "impossible" string for the dollar-quote, so it cannot appear in the string. If you don't have one, you can test for the quote-string and alter it in the unlikely case it should be matched, to be absolutely sure.
Use SELECT .. WHERE NOT EXISTS for the INSERT to automatically prevent already existing entries to be re-inserted. This prevents duplicate entries in the table completely - not just among the new entries.
An index on movies.movie (possibly, but not necessarily UNIQUE) would speed up the INSERTs.
Why bother with grep and sed and not just let the database detect duplicates?
Add a unique index on movie and create a new (temporary) insert script on each run and then execute it with autocommit (default) or with the -v ON_ERROR_ROLLBACK=1 option of psql. To get a full insert script of your movie database dump it with the --column-inserts option of pg_dump.
Hope this helps.
There's utility daemon called incron, which will fire your script whenever some file is written in watched directory. It uses kernel events, no loops - Linux only.
In its config (full file path):
/media/htpc IN_CLOSE_WRITE /home/aydin/added.sh $#/$#
Then simplest adder.sh script without any param check:
#!/bin/bash
cat <<-EOsql | psql -U "aydin.hassan" -d "aydin_1.0"
INSERT INTO movies (movie) VALUES (E'$1');
EOsql
You can have thousands of files in one directory and no issue as you can face with your original script.
Let's imagine I want to save the text of various bash script in my database with sqlite3.
If my script do this
VARIABLE=$(cat "bashScript.sh")
sqlite3 mydb.db "CREATE TABLE data (myBash BLOB)"
sqlite3 mydb.db "INSERT INTO data VALUES (${VARIABLE})"
Because the bash script will have all the special character, this will not work. How can I do this?
You need to quote the value in the Insert statement, and you'll want to protect quotes in the value itself: untested:
sql_value=$(sed 's/'\''/&&/g' <<< "$VARIABLE")
sqlite3 mydb.db "insert into data values ('$sql_value')"