Hi I am new to shell scripting and trying to insert data in a csv file which i generated to an oracle database table. my code is below
FilePath =/oracle/some path
Filename=test.csv
X='sqlplus -s username/password#dbname<< EOF
set heading off pages 0 feedback off
SELECT a.user_id,a.email_id FROM addresses a where
a.user_id in('ABCD','EFGH','IJKL','MNOP');
EXIT
EOF`
While read line
do
echo $line
sqlplus -s username/password#dbname<< EOF
insert into sometable (uid,email) values ($line);
commit;
exit
EOF
done<<
Can someone tell me what i amdoing wrong as data is not getting inserted.A sample working code will be deeply appreciated.
Related
I have an external hive table on top of a parquet file.
CREATE EXTERNAL TABLE parquet_test LIKE avro_test STORED AS PARQUET LOCATION 'hdfs://myParquetFilesPath';
I want to get the count of table using shell script.
I tried with following command
myVar =$(hive -S -e " select count(*) from parquet_test;")
echo $myVar
Added -S to run hive in silent mode still I get whole map reduce log and count in the myVar variable. How to get only count.
I don't have access to any of the configuration file to enable or disable the level of logging. Is there any other way?
Finally found a work around.
First flushed the query result into a file in HDFS then read answer from file.
The file only contains the result of the query.
(hive -S -e " INSERT OVERWRITE LOCAL DIRECTORY '/home/test/result/'
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' select count(*) from parquet_test;")
Then reading the file into a variable
Count var=$(hdfs dfs -tail /home/test/result/)
echo $var
Thank you
myVar=$(eval "hive -S -e 'select count(*) from parquet_test;' ")
echo $myVar
I have a simple script in bash that just return the count of a given table
The trow command through bash its like that
user>bash Bash_Script.bsh -T MyTableTthatAlreadyExists
After the conections parameters it just do that:
SQLSTRING="SELECT COUNT(*) FROM $SYBTAB;"
BATCH_ARGS=`sqlplus -S /nolog <<SQL | tail -1
connect $ORCL_USR/$ORCL_PWD#$ORCL_TNS;
alter session set nls_date_format='YYYYMMDD';
set pagesize 0 long 4096 linesize 32767 feed off head off;
$SQLSTRING
quit;
SQL`
echo "$BATCH_ARGS"
And it works. Yest it works, it returns the nummer of rows of my table.
The problem come when I create a new table MyNewTable
- The table exists in Oracle
so when I do in SQL Developer
select count(*) from MyNewTable;
return the correct nummer.
But when I throw again the unix command. It doesnt work, it doesnt return anything
user>bash Bash_Script.bsh -T MyNewTable --> return nothing
I wonder myself what I m missing, wat I m not taking account.
I thounk about Grant privileges but they both haven the same.
can anyone here help me ?
Thanks in advance, Enrique
Your script is only getting the last line of output from SQL*Plus
BATCH_ARGS=`sqlplus -S /nolog <<SQL | tail -1
If you remove the tail part:
BATCH_ARGS=`sqlplus -S /nolog <<SQL
then you'll see what is actually happening. With a table that exists, with that modification I see:
user>bash Bash_Script.bsh -T MyTableTthatAlreadyExists
Session altered.
1815
and with a table that does not exist I see:
user>bash Bash_Script.bsh -T MyNewTable
Session altered.
SELECT COUNT(*) FROM t43
*
ERROR at line 1:
ORA-00942: table or view does not exist
It seems you are seeing ORA-01031: insufficient privileges, so the user you are connecting as needs to have select privileges granted for the new table.
As you only want the actual count, if you swap the order of the alter and set clauses the Session altered. message will also be suppressed as feedback will be switched off by then:
BATCH_ARGS=`sqlplus -S /nolog <<SQL
connect $ORCL_USR/$ORCL_PWD#$ORCL_TNS;
set pagesize 0 long 4096 linesize 32767 feed off head off;
alter session set nls_date_format='YYYYMMDD';
$SQLSTRING
quit;
SQL`
and you'd then see, for an existing table, just:
user>bash Bash_Script.bsh -T MyTableTthatAlreadyExists
1815
making the tail unnecessary anyway.
We have a UNIX box hosting an Oracle DB. We need to delete records on a daily basis from a table in this DB. We are setting up a scheduled job outside of Oracle that will run a script daily to do this.
Could you please help me create a .sh script file to do the same? I have the username/pwd for the DB.
The query is: DELETE FROM AUDIT_LOG WHERE EVENT_DATE <= SYSTIMESTAMP - 1;
In KORN shell you can do this as below:
#!/bin/ksh
`sqlplus "<schema_name>/<password>" << EOF
set feedback off
set heading off
DELETE FROM AUDIT_LOG WHERE EVENT_DATE <= SYSTIMESTAMP - 1;
exit;
EOF`
What is the best way to pass multiple values from one variable into separate records in an oracle db?
I want to take the output from:
hddlist=`iostat -Dl|awk '{print ""$1"="$(NF)}'
This returns output like this:
hdisk36=0.8
hdisk37=0.8
hdisk38=0.8
hdisk40=5.5
hdisk52=4.9
I want to insert them into a database like so:
sqlplus -s /nolog <<EOF1
connect / as sysdba
set verify off
insert into my_table ##Single Record Here
EOF1
How can I systematically separate out the values so i can create individual records that look like this:
Disk Value
--------- -------
hdisk36 0.8
hdisk37 0.8
hdisk38 0.8
hdisk40 5.5
hdisk52 4.9
I originally tried a while loop with a counter but could not seem to get it to work. An exact solution would be nice but some directional advice would be just as helpful.
Loop and generate insert statements.
sql=$(iostat -Dl | awk '{print ""$1"="$(NF)}' | while IFS== read -r k v ; do
printf 'insert into mytable (k, v) values (%s, %s);\n' "$k" "$v"
done)
This output can be passed in some manner to sqlplus, perhaps like this
sqlplus -s /nolog <<EOF1
connect / as sysdba
set verify off
$sql
EOF1
Although, depending on the line format of iostat, it might be simpler to just omit awk and parse with read directly.
You can redirect the output to a file and then use an external table
It should look something like this:
CREATE TABLE hddlist_ext_table (
disk CHAR(16),
value CHAR(3)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER DEFAULT DIRECTORY tab_dir
ACCESS PARAMETERS (RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY '=')
LOCATION ('your_file_name'));
Then you can either use this table for your data or insert-select from it to your table;
insert into my_table
select disk, value from hddlist_ext_table;
You can insert multiple rows in a single SQL statement in Oracle like this
INSERT ALL
INTO mytable (column1, column2, column3) VALUES ('val1.1', 'val1.2', 'val1.3')
INTO mytable (column1, column2, column3) VALUES ('val2.1', 'val2.2', 'val2.3')
INTO mytable (column1, column2, column3) VALUES ('val3.1', 'val3.2', 'val3.3')
SELECT * FROM dual;
If you intend to run this script automatically at intervals to then see the results of each disk, you will probably need additional columns to hold the date and time.
You might also look at sqlldr as you can specify a control file telling it what your data contains and then this will load the data into a table. It is more suited to the purpose if you are loading lots of data than SQL Plus.
Can we have multiple SQL* PLUS connections in a shell script?
I have written a shell script to copy the data of tables from one database to another using COPY command of SQL* PLUS. I don't have the privilege to create Database Link so, I am using the COPY command.
I need to copy the data of around 50 tables. When the dataset is small, it runs and copies the data of all the tables. But when the dataset is huge, it gets stuck and I get session inactive message in the unix machine.
I thought of splitting the statements and wrote it as below: But I am getting the error "SP2-0042: unknown command "END1" - rest of line ignored." and "SP2-0042: unknown command "END" - rest of line ignored."
#!/bin/bash
export ORACLE_HOME=/ora00/app/oracle/product/9.2.0.8
export PATH=$PATH:$ORACLE_HOME/bin
args=$#
if [ $args == 1 ]
then
echo "Shell script started"
else
echo "Wrong number of arguments"
exit 1
fi
time_start=`date +%H%M%S`
echo $time_start
sqlplus -s srcUN/srcPwd#srcSID <<END1
COPY from srcUN/srcPwd#srcSID to dstUN/dstPwd#dstSID INSERT tab1 USING SELECT * FROM tab1 WHERE col1 = $1;
COPY from srcUN/srcPwd#srcSID to dstUN/dstPwd#dstSID INSERT tab2 USING SELECT * FROM tab2 WHERE col1 = $1;
END1
sqlplus -s srcUN/srcPwd#srcSID <<END2
COPY from srcUN/srcPwd#srcSID to dstUN/dstPwd#dstSID INSERT tab3 USING SELECT * FROM tab3 WHERE col1 = $1;
END2
#END
Could you please help me resolve this?
Thanks,
Savitha
The problem is that END1 and END2 are not recognized as the end of the input redirection because they have leading whitespace.
Remove all the whitespace on these two lines and it should work.