I am a noob in running db2 commands in the unix environment, so I have been trying to connect to a db2 instance from a bash script. However I get errors here is what my script looks like:
#!/bin/bash
DB2="java com.ibm.db2.clp.db2"
$DB2 "connect to <db2 server here> user **** using ****"
I get a DSNC102I : The option "connect to <db2 server> user **** using ****" specified after the "db2" command is incorrect.
I do not know what to do from here.
Currenty I am able to run an sql script from the same bash script by using
$DB2 -tvf part3.sql where both connection details and sql queries are in the part3.sql file.
Why can't I achieve the same results by writing the sql commands themselves in the bash script.
PS:
I want this since I my bash script is required to accept any db2 instance/ schema to conduct queries as a parameter to the bash script
It looks like you're using the CLP under USS and it is interpreting the connect statement as an option flag instead of a command.
Putting the connect statement and the statements to run in a file at run time should do what you need - this script takes parameters for db2 server, username, and password so you can remove them from part3.sql:
#!/bin/bash
DB2="java com.ibm.db2.clp.db2"
echo "connect to $1 user $2 using $3;" > temp.sql
cat part3.sql >> temp.sql
$DB2 -tvf temp.sql
rm temp.sql
The connect statement is put into a temp file, then the contents of the part3.sql file are copied in and the file is run by the CLP. Finally the temp file is removed.
Related
I have read all other solutions and none adapts to my needs, I do not use Java, I do not have super user rights and I do not have API's installed in my server.
I have select rights on a remote PostgreSQL server and I want to run a query in it remotely and export its results into a .csv file in my local server.
Once I manage to establish the connection to the server I first have to define the DB, then the schema and then the table, fact that makes the following lines of code not work:
\copy schema.products TO '/home/localfolder/products.csv' CSV DELIMITER ','
copy (Select * From schema.products) To '/home/localfolder/products.csv' With CSV;
I have also tried the following bash command:
psql -d DB -c "select * from schema.products;" > /home/localfolder/products.csv
and logging it with the following result:
-bash: home/localfolder/products.csv: No such file or directory
I would really appreciate if someone can show a light on this.
Have you tried this? I do not have psql right now to test it.
echo “COPY (SELECT * from schema.products) TO STDOUT with CSV HEADER” | psql -o '/home/localfolder/products.csv'
Details:
-o filename Put all output into file filename. The path must be writable by the client.
echo builtin + piping (|) pass command to psql
Aftr a while a good colleague deviced this solution which worked perfectly for my needs, hope this can help someone.
'ssh -l user [remote host] -p [port] \'psql -c "copy (select * from schema.table_name') to STDOUT csv header" -d DB\' > /home/localfolder/products.csv'
Very similar to idobr's answer.
From http://www.postgresql.org/docs/current/static/sql-copy.html:
Files named in a COPY command are read or written directly by the server, not by the client application.
So, you'll always want to use psql's \copy meta command.
The following should do the trick:
\copy (SELECT * FROM schema.products) to 'products.csv' with csv
If the above doesn't work, we'll need an error/warning message to work with.
You mentioned that the server is remote, however you are connecting to a localhost. Add the -h [server here] or set the ENV variable
export PGHOST='[server here]'
The database name should be the last argument, and not with -d.
And finally that command should have not failed, my guess is that that directory does not exist. Either create it or try writing to tmp.
I would ask you to try the following command:
psql -h [server here] -c "copy (select * from schema.products) to STDOUT csv header" DB > /tmp/products.csv
I am using plink to execute the shell script on the remote MachineB. And shell script is there on the MachineA(Windows Box).
C:\PLINK>plink uname#MachineB -m test.sh
Using keyboard-interactive authentication.
Password:
Using keyboard-interactive authentication.
Your Kerberos password will expire in 73 days.
And that shell script generates the output in a text file(aa.txt) on MachineB and that gets stored in /export/home/uname/aa.txt
So my question is- Is there any way that I can copy the aa.txt file from MachineB to MachineA as soon the script has completed all its task using the ssh. Or we need to put all these things in Windows Batch file?
So Problem Statement is like this-
Execute the shell script on MachineB from MachineA.
Then wait for the shell script to complete its task, in my case it will write the output to a text file.
And after the shell script has completed all its task means it finished writing everything to a txt file, then copy that txt file to MachineA from MachineB
Any suggestions will be appreciated on how I can achieve the above scenario?
Update:-
So Suppose if this is the content in test.sh shell script file and also after adding pscp at the end of script, then it should be like this?
#!/bin/bash
export HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
hive -S -e 'SELECT count(*) from testingtable2' > aa.txt
pscp uname#MachineB:/export/home/uname/aa.txt c:\documents\foo.txt
So I am executing a hive query in the above script and whose output is getting stored in aa.txt file and as soon as the query is completed and output is stored in aa.txt file, it will go to fourth line of pscp which will transfer aa.txt file to my local windows machine inside documents folder. Am I right? This will be the whole process?
And if the above process is right as far as I understood, then I can simply go to windows cmd prompt, and do like below and it will do the exact same process. Right?
C:\PLINK>plink uname#MachineB -m test.sh
Using keyboard-interactive authentication.
Password:
Using keyboard-interactive authentication.
Your Kerberos password will expire in 73 days.
Updated Again:-
So I need to create a bat file, and suppose this is the below test.bat file, so content should be like this in that test.bat file-
plink uname#MachineB -m test.sh
pscp uname#MachineB:/export/home/uname/aa.txt c:\documents\foo.txt
You can use scp to download the file after execution. If you setup winsshd on your windows machine and append a copy command in the test.sh file:
scp /export/home/uname/aa.txt user#windowsmachine.com:/homedir
The file will be transferred after completion.
Similarly, you can use a windows scp client like pscp on your windows machine, you can pull the file from the linux machine:
pscp uname#MachineB:/export/home/uname/aa.txt c:\documents\foo.txt
So on windows machine, have a batch script:
plink uname#MachineB -m test.sh
pscp uname#MachineB:/export/home/uname/aa.txt c:\documents\foo.txt
Just to add on to what has already been said, I am running something very similar and use the following format (from a batch script):
plink -ssh user# -pw password -m yourScript.sh
pscp -sftp -pw password user#IP-Address:/path/to/remote/file C:\path\to\local\save\directory
That second command is on one line, but it might not render like that on here. Of course, replace IP-Address, user, and password with the appropriate values for your login :-)
Hope that helps!
EDIT - Sorry, just realized this EXACT answer was already given. Kudos!
I am just getting started with shell scripts to save me typing in the same commands over and over. This command is used to copy a database over to a slave server as part of setting up MySQL database replication.
It works when typed into the command prompt directly:
mysqldump --host=192.168.1.1 –uUSER –pPASSWORD --opt database_name | mysql --host=192.168.1.2 –uUSER –pPASSWORD -C database_name
USER, PASSWORD and database_name all are replaced with their actual values in the real script.
When I type this command into a scripts.sh file, give it the execute permission, and then run it with ./scripts.sh I get:
'RROR1102 (42000): Incorrect database name 'database_name
mysqldump: Got errno 32 on write
What could be causing this error? Do I need to modify the command somehow when it is contained in a shell script?
The variable your database name is in has a CR at the end. You may need to run your script through dos2unix, or use one of the solutions on this site for stripping CRs from data if you're getting the database name from an external source.
I am trying to automate a set of procedures that create TEMPLATE databases.
I have a set of files (file1, file2, ... fileN), each of which contains a set of pgsql commands required for creating a TEMPLATE database.
The contents of the file (createdbtemplate1.sql) looks roughly like this:
CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8';
\c mytemplate1
CREATE TABLE first_table (
--- fields here ..
);
-- Add C language extension + functions
\i db_funcs.sql
I want to be able to write a shell script that will execute the commands in the file, so that I can write a script like this:
# run commands to create TEMPLATE db mytemplate1
# ./groksqlcommands.sh createdbtemplate1.sql
for dbname in foo foofoo foobar barbar
do
# Need to simply create a database based on an existing template in this script
psql CREATE DATABASE $dbname TEMPLATE mytemplate1
done
Any suggestions on how to do this? (As you may have guessed, I'm a shell scripting newbie.)
Edit
To clarify the question further, I want to know:
How to write groksqlcommands.sh (a bash script that will run a set of pgsql cmds from file)
How to create a database based on an existing template at the command line
First off, do not mix psql meta-commands and SQL commands. These are separate sets of commands. There are tricks to combine those (using the psql meta-commands \o and \\ and piping strings to psql in the shell), but that gets confusing quickly.
Make your files contain only SQL commands.
Do not include the CREATE DATABASE statement in the SQL files. Create the db separately, you have multiple files you want to execute in the same template db.
Assuming you are operating as OS user postgres and use the DB role postgres as (default) Postgres superuser, all databases are in the same DB cluster on the default port 5432 and the role postgres has password-less access due to an IDENT setting in pg_hba.conf - a default setup.
psql postgres -c "CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8'
TEMPLATE template0"
I based the new template database on the default system template database template0. Basics in the manual here.
Your questions
How to (...) run a set of pgsql cmds from file
Try:
psql mytemplate1 -f file
Example script file for batch of files in a directory:
#! /bin/sh
for file in /path/to/files/*; do
psql mytemplate1 -f "$file"
done
The command option -f makes psql execute SQL commands in a file.
How to create a database based on an existing template at the command line
psql -c 'CREATE DATABASE my_db TEMPLATE mytemplate1'
The command option -c makes psql execute a single SQL command string. Can be multiple commands, terminated by ; - will be executed in one transaction and only the result of the last command returned.
Read about psql command options in the manual.
If you don't provide a database to connect to, psql will connect to the default maintenance database named "postgres". In the second answer it is irrelevant which database we connect to.
you can echo your commands to the psql input:
for dbname in foo foofoo foobar barbar
do
echo """
CREATE DATABASE $dbname TEMPLATE mytemplate1
""" | psql
done
If you're willing to go the extra mile, you'll probably have more success with sqlalchemy. It'll allow you to build scripts with python instead of bash, which is easier and has better control.
As requested in the comments: https://github.com/srathbun/sqlCmd
Store your sql scripts under a root dir
Use dev,tst,prd parametrized dbs
Use find to run all your pgsql scripts as shown here
Exit on errors
Or just git clone the whole tool from here
For that use case where you have to do it....
Here is a script I've used for importing JSON into PostgreSQL (WSL Ubuntu), which basically requires that you mix psql meta commands and SQL in the same command line. Note use of the somewhat obscure script command, which allocates a pseudo-tty:
$ more update.sh
#!/bin/bash
wget <filename>.json
echo '\set content `cat $(ls -t <redacted>.json.* | head -1)` \\ delete from <rable>; insert into <table> values(:'"'content'); refresh materialized view <view>; " | PGPASSWORD=<passwd> psql -h <host> -U <user> -d <database>
$
Is there any way I can run Sybase SQL commands from command prompt. I need to write a batch file which runs an SQL query on machine as a fix for a bug.
Use ISQL. See http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.ase_15.0.utility/html/utility/utility10.htm for the complete reference.
Specifically, you're looking for commands like this:
ISQL -S server -D database -U user -P password
This will launch ISQL with the indicated settings, and from there you can run SQL statements against the database.