I have following in bash script:
sqlite3 database.db "INSERT INTO tbl (col1,col2) VALUES ('\"$line\"','\"$stdout\"');"
,where $line variable isn't going to make any trouble, but my $stdout
variable is output from whois $line, so it contains many chars, one of them
being ' which breaks the sqlite3 syntax.
Is there some way I could quote-escape that variable, so that it passes well
in this query?
EDIT
Sorry, I found my solution here, https://stackoverflow.com/a/14340973/2434479.
Should I delete this question?
The standard SQL escape sequence for simple quote ' is the double simple quote ''.
Using bash pattern substitution, you may perform the necessary replacements:
sh$ DSQ="''"
sh$ stdout="I'm in a big'trouble"
sh$ echo "${stdout//\'/$DSQ}"
I''m in a big''trouble
In your specific case:
DSQ="''"
sqlite3 database.db \
"INSERT INTO tbl (col1,col2) VALUES ('$line','${stdout//\'/$DSQ}');"
Related
My question is a simple one. I am looking to simply format my bash code into something a little more readable for other users .. While this works:
mysql --login-path=main-data -N -e "SELECT section_id FROM db.table WHERE contractor_id = '1'" | while read contractor_info_id; do
#......//
done
I don't understand why the backslash doesn't work with a input output statement separated by | .. IE
mysql --login-path=main-data -N -e "SELECT section_id\
FROM db.table\
WHERE contractor_id = '1'" | while read contractor_info_id; do
#......//
done
It generates a fatal error Syntax error: "done" unexpected
Why does multi-line formatting work on a single output command such as:
long_arg="my very long string\
which does not fit\
on the screen"
But not on an output | input command ?
The reason the escaped line ending does not work in your mysql use case is that mysql does not require an escaped line ending. The text between the two double quotes is treated as the string, and escaped characters are no interpreted. (eg: \t would not put in a tab). You can se this in action with these examples:
$ echo "Hello from line one
> and hello from line two"
Hello from line one
and hello from line two
$ echo Hello from line one \
> and also from line one
Hello from line one and also from line one
TL;DR: Within double-quotes, the slash and CR is treated as part of the string, not interpreted as an escape character.
Backslashes not needed inside quotes, and SQL isn't usually picky about extra whitespace anyway.
I don't have mysql installed here, but try this and let me know if it doesn't behave...
mysql --login-path=main-data -N -e "
SELECT section_id
FROM db.table
WHERE contractor_id = '1'
" | while read contractor_info_id
do echo $contractor_info_id
done
mysql "..." | while read contractor_info_id; do
#......//
done
Syntax error: "done" unexpected
This error is coming from bash, not from mysql, so we can rule out anything inside the double quotes. It doesn't matter what you're doing with backslashes there.
Do you actually have anything in place of #......//? You need at least one command inside the loop, otherwise you'll get this "done" unexpected error. Try :, a no-op command. That'll at least get rid of the syntax error.
mysql "..." | while read contractor_info_id; do
:
done
"SELECT section_id\
FROM db.table\
WHERE contractor_id = '1'"
The backslashes won't cause bash any heartburn but they do lead to invalid SQL. Once the shell deletes the backslashes and newlines you're left with:
"SELECT section_idFROM db.tableWHERE contractor_id = '1'"
To fix that you can either add spaces, or just leave out the backslashes. MySQL doesn't care if there are newlines, nor does bash, so you can stick with what you wrote originally.
"SELECT section_id
FROM db.table
WHERE contractor_id = '1'"
The reason for this happening to me was a unique issue I was having with my IDE .. It was inserting an actual character for a line break rather than just a physical line-break itself. The problem wasn't the syntax I was attempting, rather the inserted characters on save. Thanks to all who thoughtfully answered my question. My original un-escaped syntax was correct to begin with.
I have bash script which calls another script(some_script). some_script expects some input from the user. I have used printf statement for this purpose.
But the problem is the variable value is not being accepted by the target script. I think this is because '\' is being taken as an escape character in the script
The statement somewhat looks like this
printf 'yes\n$var1\n$var2\n$var3' | some_script
If i directly replace the variable with values it runs perfectly but i want the script to take the values from the variables. How do i achieve this?
There is a difference between " and '. Try
printf "yes\n$var1\n$var2\n$var3" | some_script
because with ' the variables won't get substituted.
Yes, \ is a character that has to be escaped.
Use \\n.
For more details we would need more details on how your script works.
I'm using the psql command \copy and I would like to pass a variable to it from the shell (for table name) like I've done when scripting queries. I've read in the documentation that:
The syntax of the command is similar to that of the SQL COPY command. Note that, because of this, special parsing rules apply to the \copy command. In particular, the variable substitution rules and backslash escapes do not apply.
This seems quite definitive, however I'm wondering if anyone knows of a workaroud?
You could use shell variable substitution with heredoc syntax. Example:
#!/bin/sh
tablename=foo
psql -d test <<EOF
\copy $tablename FROM '/path/to/file'
EOF
You can pass the variable from the shell to psql with -v psql_var="$shell_var" commandline parameter (or access it directly with a shell escape `echo "$shell_var"` after exporting it). Then, you can build the \copy meta command in another meta command (locally with \set or server side with \gset). Example:
#!/bin/sh
tablename=foo
psql -d test -v tbl="$tablename" <<\EOF
\set cmd '\\copy ' :tbl ' FROM ''/path/to/file'''
\echo :cmd
:cmd
EOF
I wrote a bash script to insert values to sqlite database. The command is as follow
sqlite3 ${db_name} "insert into ${table_name} (${column1},${column2}) values ('$f1','$f2');"
This command works fine until f1 variable contains a single quote
# e.g f1="I'm just kidding"
# the command reported error
Error: near "m": syntax error
May someone please show me how can we escape the single quote inside the variable?
Any recommendations are appreciated. Thanks!
To escape a single quote for SQL, you double it (https://www.sqlite.org/faq.html#q14):
$ f1="I'm just kidding"
$ echo "${f1//\'/''}"
I''m just kidding
$ f2="no single quotes"
$ echo "${f2//\'/''}"
no single quotes
So
sqlite3 ${db_name} "insert into ${table_name} (${column1},${column2}) values ('${f1//\'/''}','${f2//\'/''}');"
from bash you can use ${varname//x/y} to replace all instances of x with y in the varname variable.
sqlite3 ${db_name} "insert into ${table_name} (${column1},${column2}) values ('${f1//\'/\'}','${f2//\'/\'}');"
will replace any ' with \' though #ignacioVazquez-Abrams has the best answer as php perl python all have modules to help sanitise input.
You could use \
f1="I\'m just kidding"
I've just written a bash script that takes some info from the mysql database and reads it line by line, extracting tab-separated columns into separate variables, something like this:
oldifs=$IFS
result=result.txt
$mysql -e "SELECT id,foo,bar,baz FROM $db.$table" -u $user --password=$pass -h $server > $result
cat $result | grep -e ^[0-9].*$ | while IFS=$'\t' read id foo bar baz
do
# some code
done
IFS=$oldifs
Now, while this works OK and I'm satisfied with the result (especially since I'm going to move the query t oanother script and let cron regenerate the result.txt file contents once a week or so, since I'm dealing with a table that changes maybe once or twice a year), I'm curious about the possibility of putting the query's result in a variable instead of a file.
I have noticed that in order to echo out backslash-excaped characters, I need to tell the command explicitly to interpret such characters as special chars:
echo -e "some\tstring\n"
But, being a bash noob that I am, I have no idea how to place the backslash escaped characters (the tabs and newlines from the query) inside a variable and just work with it the same way I'm working with the external file (just changing the cat with echo -e). I tried this:
result=`$mysql -e "SELECT id,foo,bar,baz FROM $db.$table" -u $user --password=$pass -h $server`
but the backslash escaped characters are converted into spaces this way :(. How can I make it work?
To get the output of a command, use $(...). To avoid wordsplitting and other bash processing you will need to quote. Single quotes ('$(...)') will not work as the quoting is too strong.
Note that once the output is in your variable, you will probably need to (double) quote it wherever you use it if you need to preserve anything that's in $IFS.
$ listing="$(ls -l)"
$ echo "$listing"
Could you try to set double quotes around $result - thus echo -e "$result"?
% awk '/^[0-9]/ { print $2, $3, $4, $5 }' <<SQL | set -- -
> $("${mysql}" -e "SELECT id,foo,bar,baz FROM $db.$table" -u $user --password=$pass -h $server)
> SQL
% printf '%s\t' "${#}"
<id> <foo> <bar> <baz>
You might get some use out of this. The heredoc should obviate any escaping issues, awk will separate on tabs by default, and set accepts the input as a builtin argv array. printf isn't necessary, but it's better than echo - especially when working with escape characters.
You could also use read as you did above - but to better handle backslashes use the -r argument if you go that route. The above method would work best as a function and you could then iterate over your variables with shift and similar.
-Mike