I'm trying to read CSV file and writing the same into the table, CSV file was located in my local machine(client). I used /copy command and achieved the same. Here I have hardcoded my filepath in sql script. I want to parameterised my csv file path.
Based on my analysis /copy not supported :variable substitution, but not sure
I believe we can achieve this using shell variables, but I tried the same, It's not working as expected.
Following are my sample scripts
command:
psql -U postgres -h localhost testdb -a -f '/tmp/psql.sql' -v path='"/tmp/userData.csv"'
psql script:
\copy test_user_table('username','dob') from :path DELIMITER ',' CSV HEADER;
I executing this commands from shell and I'm getting no such a file not found exception. But same script is working with hardcoded path.
Anyone able to advise me on this.
Reference :
Variable substitution in psql \copy
https://www.postgresql.org/docs/devel/app-psql.html
I am new to Bash. So far your problem is way hard for me.
I can do it in one shell script. Maybe later I can make it to two scripts.
The follow is a simple one script file.
#!bin/bash
p=\'"/mnt/c/Users/JIAN HE/Desktop/test.csv"\'
c="copy emp from ${p}"
a=${c}
echo $a
psql -U postgres -d postgres -c "${a}"
Related
I need to execute a bash script containing SQL, so I am using a script to add custom configurations to a Postgres Docker container, according to the docs here:
https://github.com/docker-library/docs/tree/master/postgres#how-to-extend-this-image
But I don't know what EOSQL means. Here is an example of my script taken from the docs above:
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER docker;
CREATE DATABASE docker;
GRANT ALL PRIVILEGES ON DATABASE docker TO docker;
CREATE EXTENSION $MY_EXTENSION;
EOSQL
So, what is EOSQL? I cannot seem to find much information about this command or keyword.
EOSQL is a limit string for a Here Document block. The limit string signifies the start and end of a text block to the bash interpreter (or any POSIXy shell). The limit string can be any text that doesn't appear in your block, EOF is common in examples.
Variable substitution will work as normal in a here document:
#!/usr/bin/env bash
cat <<-EOF
a
$MY_EXTENSION
b
EOF
echo "script continues" > /dev/null
Then running that with the MY_EXTENSION variable set:
$ MY_EXTENSION=something ./test.sh
a
something
b
In Docker you will need ENV MY_EXTENSION=something in your Dockerfile or docker run -e MY_EXTENSION=something <image> on the command line for the environment to be setup.
Leading tabs
The <<-EOSQL that starts this heredoc includes a - to ignore the leading tab character on any lines of the heredoc.
Using <<EOSQL instead would leave the leading tabs in the output.
I have a shell script with a psql command to update a Redshift table. It's worked several times in the original spot in the shell script, but when I copied the command into another line in the script, I got an error saying psql: command not found.
Here's some pseudocode of where the psql command has and has not worked:
#start
if [ -f filename.txt ];
then echo 'first area';
psql #has not worked here
else psql ; #has successfully worked here
fi;
psql # has not worked here
#end
I'd just like pointers or suggestions to figure out the behavior of this. Is there any plausible explanation for this to work only in that else statement?
I try to execute an PSQL from shell, and the thing is - it returns the error "command not found". I have a shell script in which there're lines:
ID3=`more DATA/Id3.txt`
psql -h localhost test test -Atc "SELECT id, reference, timestamp FROM restricted WHERE id='`$ID3`'"
In the Id3.txt there's only the ID. When the psql command is written and executed direct through prompt - there's no problem at all and correct value is returned. When executed with a .sh file - error "command not found" is brought up. I have no clue why. Maybe anyone have a idea?
In your script try to add which psql to see whether you can find the executable
Run below command on your console: whereis psql
And then replace psql inyour script with the output of above command. This
I'm looking to create a script than can find a specific date/time based on a field in a database
Currently i do it manually...
# psql gttvdb -U postgres
gttvdb=# select patch_date from version_history where version ='1.1.1';
What i need is to be able to reference that date as a variable in a bash script
#! /bin/sh
patchdate=$("select patch_date from version_history where version ='1.1.1';")
echo "last update was " $patchdate
Try this:
patchdate=` psql -t -q -c "select patch_date from version_history where version ='1.1.1'"`
Below is my shell script that I am trying to execute using PLINK on MachineB from MachineA(Windows Machine).
#!/bin/bash
export HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
hive -S -e 'SELECT count(*) from testingtable1' > attachment22.txt
I am using plink to execute the shell script like below,
C:\PLINK>plink uname#MachineB -m test.sh
Using keyboard-interactive authentication.
Password:
Using keyboard-interactive authentication.
Your Kerberos password will expire in 73 days.
And this is the below error I always get whenever I try to run like above.
sh: HIVE_OPTS= -hiveconf mapred.job.queue.name=hdmi-technology: is not
an identifier
Something wrong with my shell script? or some trailing spaces? I am not able to figure it out. I am running PLINK from windows machine
The sh: prefix on the error message indicates that the script is being executed by sh, not bash.
bash lets you combine setting a variable and exporting it into a single command:
export foo=bar
sh, or at least some older versions of it, require these two actions to be separated:
foo=bar ; export foo
A version of sh that doesn't recognize the export foo=bar syntax will interpret the string foo=bar as a variable name (and an illegal one, since it isn't an identifier).
Either arrange for the script to be executed by bash, or change this:
export HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
to this:
HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
export HIVE_OPTS
For that matter, since you're referring to $HIVE_OPTS at the very beginning of your script, it's almost certainly already exported, so you could just drop the export.
(You'll also need to avoid any other bash-specific features.)
So why is the system invoking the shell with sh? The #!/bin/bash syntax is specific to Unix-like systems. Windows generally decides how to execute a script based on the file extension; apparently your system is configured to invoke *.sh files using sh. (You could configure your system, using Folder Options, to invoke *.sh files using bash, but that might introduce other problems.)
I think the -m option to plink is for reading commands to execute on the remote machine from a local file. If my comment about line endings doesn't work, try
plink uname#MachineB test.sh
Make sure test.sh is executable by running
chmod +x test.sh
on MachineB.