I'm using the psql command \copy and I would like to pass a variable to it from the shell (for table name) like I've done when scripting queries. I've read in the documentation that:
The syntax of the command is similar to that of the SQL COPY command. Note that, because of this, special parsing rules apply to the \copy command. In particular, the variable substitution rules and backslash escapes do not apply.
This seems quite definitive, however I'm wondering if anyone knows of a workaroud?
You could use shell variable substitution with heredoc syntax. Example:
#!/bin/sh
tablename=foo
psql -d test <<EOF
\copy $tablename FROM '/path/to/file'
EOF
You can pass the variable from the shell to psql with -v psql_var="$shell_var" commandline parameter (or access it directly with a shell escape `echo "$shell_var"` after exporting it). Then, you can build the \copy meta command in another meta command (locally with \set or server side with \gset). Example:
#!/bin/sh
tablename=foo
psql -d test -v tbl="$tablename" <<\EOF
\set cmd '\\copy ' :tbl ' FROM ''/path/to/file'''
\echo :cmd
:cmd
EOF
Related
I am trying to create and use variables inside heredoc like this,
#!bin/bash
sudo su - postgres <<EOF
IP="XYZ"
echo "$IP"
EOF
This doesn't work right and I get a blank line as echo.
But if I use quotes around EOF like this,
#!bin/bash
sudo su - postgres <<"EOF"
IP="XYZ"
echo "$IP"
EOF
It works. Can someone please explain this? According to what I read in man the behaviour should be opposite.
The shell evaluates the unquoted here document and performs variable interpolation before passing it to the command (in your case, sudo). Because IP is not a defined variable in the parent shell, it gets expanded to an empty string.
With quotes, you prevent variable interpolation by the parent shell, and so the shell run by sudo sees and expands the variable.
I need to switch to oracle user to change permissions to tnsnames.ora file. I am passing this file path as argument but looks like somewhere the syntax is wrong. Appreciate help in fixing this issue.
Belows is the peice of my script.
#!/bin/bash
sudo su - oracle <<-"EOF"
chmod 777 "$1"
EOF
It is failing by giving the following error:
/home/itsh->./dothis.sh /home/oracle/orasys/11.2.0.2/network/admin/tnsnames.ora
chmod: cannot access `': No such file or directory
If you're in any way concerned about security, the right thing to do is not to change your quoting, but to keep it as it is and use bash -s to pass your arguments to the shell running as the oracle user directly:
#!/bin/bash
sudo -u oracle bash -s "$#" <<-'EOF'
chmod 777 "$1"
EOF
...or, if you must use sudo su - oracle (which I'd argue is bad practice, and best avoided):
#!/bin/bash
printf -v sudo_cmd '%q ' bash -s "$#"
sudo su - oracle -c "$sudo_cmd" <<-'EOF'
chmod 777 "$1"
EOF
With either of these practices, your inner shell runs the $1 expansion itself -- and the data on the command line isn't substituted into, and parsed as, code.
The operation of a here document is specified in the POSIX spec where it says:
If any character in word is quoted, the delimiter shall be formed by performing quote removal on word, and the here-document lines shall not be expanded. Otherwise, the delimiter shall be the word itself.
If no characters in word are quoted, all lines of the here-document shall be expanded for parameter expansion, command substitution, and arithmetic expansion. In this case, the <backslash> in the input behaves as the <backslash> inside double-quotes (see Double-Quotes). However, the double-quote character ( '"' ) shall not be treated specially within a here-document, except when the double-quote appears within "$()", "``", or "${}".
So by using <<-"EOF" (instead of <<-EOF) as your here document marker you are explicitly telling the shell not to expand any variables (from the shell context) in the here document contents.
This is often what you want when you are using a heredoc for a shell snippet but in your case this is exactly the opposite of what you appear to be looking for.
I cannot believe I've spent 1.5 hours on something as trivial as this. I'm writing a very simple shell script which greps a file, stores the output in a variable, and echos the variable to STDOUT.
I have checked the grep command with the regex on the command line, and it works fine. But for some reason, the grep command doesn't work inside the shell script.
Here is the shell script I wrote up:
#!/bin/bash
tt=grep 'test' $1
echo $tt
I ran this with the following command: ./myScript.sh testingFile. It just prints an empty line.
I have already used chmod and made the script executable.
I have checked that the PATH variable has /bin in it.
Verified that echo $SHELL gives /bin/bash
In my desperation, I have tried all combinations of:
tt=grep 'test' "$1"
echo ${tt}
Not using the command line argument at all, and hardcoding the name of the file tt=grep 'test' testingFile
I found this: grep fails inside bash script but works on command line, and even used dos2unix to remove any possible carriage returns.
Also, when I try to use any of the grep options, like: tt=grep -oE 'test' testingFile, I get an error saying: ./out.sh: line 3: -oE: command not found.
This is crazy.
You need to use command substitution:
#!/usr/bin/env bash
test=$(grep 'foo' "$1")
echo "$test"
Command substitution allows the output of a command to replace the command itself. Command substitution occurs when a command is enclosed like this:
$(command)
or like this using backticks:
`command`
Bash performs the expansion by executing COMMAND and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.
The $() version is usually preferred because it allows nesting:
$(command $(command))
For more information read the command substitution section in man bash.
I wrote a bash script to insert values to sqlite database. The command is as follow
sqlite3 ${db_name} "insert into ${table_name} (${column1},${column2}) values ('$f1','$f2');"
This command works fine until f1 variable contains a single quote
# e.g f1="I'm just kidding"
# the command reported error
Error: near "m": syntax error
May someone please show me how can we escape the single quote inside the variable?
Any recommendations are appreciated. Thanks!
To escape a single quote for SQL, you double it (https://www.sqlite.org/faq.html#q14):
$ f1="I'm just kidding"
$ echo "${f1//\'/''}"
I''m just kidding
$ f2="no single quotes"
$ echo "${f2//\'/''}"
no single quotes
So
sqlite3 ${db_name} "insert into ${table_name} (${column1},${column2}) values ('${f1//\'/''}','${f2//\'/''}');"
from bash you can use ${varname//x/y} to replace all instances of x with y in the varname variable.
sqlite3 ${db_name} "insert into ${table_name} (${column1},${column2}) values ('${f1//\'/\'}','${f2//\'/\'}');"
will replace any ' with \' though #ignacioVazquez-Abrams has the best answer as php perl python all have modules to help sanitise input.
You could use \
f1="I\'m just kidding"
I'm having trouble using a single quote in a command executed from within a shell script.
In my script I execute an rdesktop command that should have -u '' (<- 2 single quotes) as a parameter.
However, no matter how I try to escape the quotes it is not passed correctly.
If I just echo $command the output is fine, if I execute it weird output is created
This is the part of the script that doesn't work:
command="rdesktop -u "\'\'" $server"
`$command`
I also tried executing it directly:
`rdesktop -u "\'\'" $server`
I would appreciate any help since I read quite a few tutorials on escaping characters in shell scripts and didn't find the solution..
EDIT:
interestingly enough, if I just use
command=rdesktop -u '' $server
and echo it, the output is fine
however, if I execute it with
$command
it fails...
If your shell is bash or ksh or zsh, it's much safer and easier to build up a command with an array rather than a string:
command=( rdesktop -u '' $server )
and execute it like this
"${command[#]}"
I can't imagine the remote server needs to see a username named literally '' (i.e. 2 single quotes) -- it probably wants just an empty string.
When you invoke rdesktop -u '' $server, you are invoking rdesktop with 3 arguments, the second of which is the empty string. It's not clear to me why you are trying to invoke it in backticks, but you can get what you want using your first definition of command and:
eval $command
Note that if you do not completely control the contents of $server, then this is a big security risk. (For example, if server is the string ; cmd, then invoking eval will execute cmd)
First, if you type
`rdesktop something`
then that means to invoke rdesktop, take the output, and run that output as a command. To “execute it directly,” simply type
rdesktop something
Now for the quotes. If you execute
rdesktop -u '' $server
then rdesktop will not see the quotes. They are removed as part of the shell parsing the command. To get rdesktop to see an argument '', use
rdesktop -u "''" $server
There's no need really to add any further escaping, since single quotes are not special inside a double-quoted string.
EDIT: When all you want to do with the command in a variable is to execute it, note that this stripping of quotes only happens once:
cmd="rdesktop -u '' $server"
$cmd