open sqlplus connection only once in while loop - oracle

I have 100 records. When I am running the code below, the SQL*Plus connection is opening many times utilizing 100% of CPU. Is there any way in which I could open SQL*Plus connection only once, i. e. outside while loop?
**#!/bin/bash
export ORACLE_HOME=/software/oracle/ora10204
export PATH=$PATH:$ORACLE_HOME/bin
INPUT_FILE='file.csv'
IFS=','
i=0
while read name id do
a[i]="$name"
b[i]="$id"
echo "${a[$i]} ${b[$i]}"
set serveroutput on;
sqlplus .../...#'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=...)(HOST=...)(PORT=...)))(CONNECT_DATA=(SID=...)))'<<EOF
insert into code_entry(inh_valu,edi_valu) values('${a[$i]}' , '${b[$i]}');
EOF
let i=$i+1
done < $INPUT_FILE**

You could create a SQL script, that you would invoke once in the end of your shell script with SQL*Plus. It would use INSERT ALL DML command, and you would just append the strings to it using your shell script and redirect operator (>>) within your existing loop.
After running your shell script your SQL script could look like this:
insert all
into code_entry (inh_valu, edi_valu) values (1, 'foo')
into code_entry (inh_valu, edi_valu) values (2, 'bar');
...
You would then just run it like this:
$ sqlplus scott/tiger #myscript.sql
A sample shell script could look like:
echo "insert all" > myscript.sql
INPUT_FILE='file.csv'
IFS=','
i=0
while read name id do
a[i]="$name"
b[i]="$id"
echo "into code_entry(...) values ('${a[$i]}', '${b[$i]}')" >> myscript.sql
let i=$i+1
done < $INPUT_FILE**
echo ";" >> myscript.sql
sqlplus .../...#'(DESCRIPTION=...)' #myscript.sql

Related

How to get plain formatted output in csv with shell script for oracle sql query

This is my shell script.
echo "Start";echo #/opt/apps/Tests/SQLDir/Test1.sql | sqlplus Db1/Db1#//maydomain:port/abc;echo "Finish";
echo "Start";echo #/opt/apps/Tests/SQLDir/Test2.sql | sqlplus Db1/Db1#//maydomain:port/abc;echo "Finish";
I have 30 .sql files like this, added in one .sh file which results 30 .csv files
Test1.sql has
SPOOL /opt/apps/Tests/OF/output1.csv REPLACE;
select name from username where id = 10 and Sname is not NULL and ROWNUM < = 50000;
Test2.sql has
SPOOL /opt/apps/Tests/OF/output2.csv REPLACE;
select strname,ctyname from addr where city = 'NYC' and ROWNUM < = 50000;
My expected OP in output1.csv is
name
Abc
xyz
pqr
My expected OP in output2.csv is
strname | ctyname
10-AP NYC
11-KP MCH
90-ZP SDK
right now I am getting weird o/p in csv
name
-------------------------------
Abc
xyz
pqr
name
-------------------------------
TYU
KLH
50000 rows selected.
SQL>
So is there any way to remove those additional lines [--------- and 50000 rows selected.] with shell script/sql code?
And while executing shell script all sql result rows are getting printed on screen. how to avoid that?
Thanks in advance.
Following SQL*Plus command should do the job:
set markup csv on delimiter ' ' quote off
set feedback off
I must say, your method of passing the name of a script to sqlplus is the strangest I've ever seen. The usual practice (given your names) would be:
sqlplus Db1/Db1#//maydomain:port/abc #/opt/apps/Tests/SQLDir/Test1.sql
I don't see where your 'echo Start' and 'echo finished' accomplish anything, since there is no clarifying info coming along with it.
It looks to me like what you want in your scripts would be
set echo off trimsp on head off pagesize 0
spool /opt/apps/Tests/OF/output2.csv replace
select strname,ctyname from addr where city = 'NYC' and ROWNUM < = 50000;
spool off
BTW, 'spool' is a sqlplus command - a directive to sqlplus itself, not a sql statement. As such, it does not need a semi-colon at the end.
-- edit
Example of using environment variables on sqlplus command line:
username=scott
userpw=tiger
server=myserver
port=1521
dbname=mydb
sqlplus $username/$userpw#//$server:$port/$dbname
Though I would question why you need to set them as variables.
And I prefer to use tnsnames rather than ezconnect.

Iterating through PL/SQL result in shell script

I am new to the shell scripting hence need help. I am trying to execute sql query against Oracle DB. Once sql query result is received, I need to iterate through the result (as it will return multiple rows and columns.).
Goal here is to invoke a REST api using curl for each record retrieved in db result. Input to REST api will be ROOT_PROC_ID column value.
Below is the shell script I have developed so far and sample output of the sql query.
#!/bin/bash
#Update below properties as per enviornment
export ENV=DEV;
export SERVERHOST=localhost
export SERVERPORT=9000
export SERVERUSER=admin
export SERVERPASSWORD=admin123
export DBHOST=localhost
export DBPORT=1537
export DBSID=ORCL
export DBUSER=SCOTT
export DBPASS=TIGER
export LOGDIR=/usr/app/$USER/data/logs/
#-------------------------------------------------------------------------
#----------- DO NOT EDIT AFTER THIS LINE ---------------------------------
#-------------------------------------------------------------------------
#create directory structure if not exists for storing log files
mkdir -p $LOGDIR/process_cancellation
mkdir -p $LOGDIR/process_cancellation/old
mkdir -p $LOGDIR/process_cancellation/halted
export old_proc_cancellation_logfile=$LOGDIR/process_cancellation/old/log_$(date "+%Y%m%d%H%M%S").log;
export halted_proc_cancellation_logfile=$LOGDIR/process_cancellation/halted/log_$(date "+%Y%m%d%H%M%S").log;
#execute sql query to fetch halted process data from database
echo
echo "Enviornment : $ENV"
echo
echo "Connecting to - $DBUSER/$DBPASS#$DBHOST:$DBPORT/$DBSID"
echo
echo "Retrieving halted process data logged before : $(date -d "15 days ago" +%d/%m/%Y) 20:00:00"
echo
sqlplus -s $DBUSER/$DBPASS#$DBHOST:$DBPORT/$DBSID << FIN_SQL > $halted_proc_cancellation_logfile
set head off
set line 1024
set pages 9999
SET SERVEROUTPUT ON;
SELECT ROOT_PROC_ID, PROC_ID, PROC_NAME, START_DATE, STATUS, ORDER_REF
FROM USER.PROC_STATUS
WHERE START_DATE<(SYSDATE - 15) AND (STATUS='proc_halted' OR STATUS='proc_failed')
ORDER BY START_DATE DESC;
SET SERVEROUTPUT OFF;
FIN_SQL
echo "Please check log file for more details : $(readlink -f $halted_proc_cancellation_logfile)"
exit
Sample SQL query output:
ROOT_PROC_ID PROC_ID PROC_NAME START_DATE STATUS ORDER_REF
pvm:0a123akpd pvm:0a123akkh FunctionalErrorProcess 28-NOV-19 01.24.35.115000000 PM pi_halted 2642277
pvm:0a122utrn pvm:0a122uun0 TechnicalErrorProcess 22-NOV-19 02.28.17.217000000 PM pi_halted 2642278
pvm:0a122utl2 pvm:0a122uu1t TechnicalErrorProcess 22-NOV-19 02.27.54.024000000 PM pi_halted 2642279
pvm:0a122utln pvm:0a122uu22 TechnicalErrorProcess 22-NOV-19 02.27.50.287000000 PM pi_halted 2642280
Assuming your sql query output is in output.txt:
awk 'NR!=1' output.txt | while read rootprocid undef
do
callApi $rootprocid
done
NR!=1 is to skip the 1st line which contains the header.
read rootprocid undef reading only the 1st column in rootprocid, rest goes to variable undef since it is not of interest.
callApi $rootprocid callAPI will be replaced with your actual api call.

Parameterizing to dynamically generate the extract file from Oracle table using Shell Script

I have a requirement where I need to parameterize to generate one extract file from multiple Oracle tables through the UNIX shell script.
Here is the script which I have written to generate one tab delimited file which will fetch all the data from EMPLOYEE table.
I need to parameterize the TABLE_NAME,OWNER_NAME,USERNAME,PASSWORD and HOST to generate from 12 more tables.
So, I would like to have only one SQL to dyngenerate the extract for 12 tables by passing these parameters values when executing the scripts.
Could you please give me show me how we can modify the below script and how to pass the parameter during the script execution.
Second Requirement is to generate the file incrementally based on a column for example, ETL_UPDATE_TS. can you please show me this also.
Sample Scripts
#!/usr/bin/ksh
TD=/mz/mz01/TgtFiles
MD=/mz/mz01/Scripts
#CAQH_Server=sftp.org
#UN=user
#PWD=password
#RD=Incoming
#RD=/home/
cd $TD
FILE="EMPLOYEE.TXT"
sqlplus -s scott/tiger#db <<EOF
SET PAGES 999
SET COLSEP " "
SET LINES 999
SET FEEDBACK OFF
SPOOL $FILE
SELECT * FROM EMP;
SPOOL OFF
EXIT
EOF
Handling your parameters in a similar way you did for $FILE variable and passing them as options to the script
#!/usr/bin/ksh
TD=/mz/mz01/TgtFiles
MD=/mz/mz01/Scripts
cd $TD
FILE="undefined"
TABLE="undefined"
while getopts :f:t: opt
do
case $opt in
f) FILE=${OPTARG} ;;
t) TABLE=${OPTARG} ;;
*) echo "invalid flag" ;;
esac
done
if [ "$TABLE" == "undefined" ]; then
echo "ERROR. TABLE is undefined, use -f option."
exit 1
fi
# More required variables checks here
# create more options to parameterize connection
sqlplus -s scott/tiger#db <<EOF
SET PAGES 999
SET COLSEP " "
SET LINES 999
SET FEEDBACK OFF
SPOOL $FILE
SELECT * FROM $TABLE;
SPOOL OFF
EXIT
EOF
An execute it as
my_script.sh -f "EMPLOYEE.TXT" -t "EMP"

How to fetch more than one column value from oracle select query to shell variable

I am trying to fetch a row with more than one column value to different shell variables. Infact I found that at a time all the column values can be stored to single shell variable. But how can I put those column values to seperate shell variables. Below is an example I am trying for time being
function sqlQuery {
sqlplus -S shiyas/********* <<'EOF'
set heading OFF termout ON trimout ON feedback OFF
set pagesize 0
SELECT name,open_mode from v$database;
EOF
}
OUTPUT="$( sqlQuery )"
echo $OUTPUT
Here I am getting the output as
ORCL READ WRITE
But my requirement is column values ORCL, READ WRITE should get assigned to different shell variable.
I tried the below of parsing.
echo "$OUTPUT" | while read name open_mode
but it was throwing unexpected end of file error.
-bash-3.2$ sh call_sql_col_val_1.sh
ORCL READ WRITE
call_sql_col_val_1.sh: line 18: syntax error: unexpected end of file
Please let me know what concept I can use to fetch a single row column values to different shell variables.
I do this via eval myself:
oracle#******:/*****> cat test.sh
#!/bin/bash
function sqlQuery {
sqlplus -S / as sysdba <<'EOF'
set heading OFF termout ON trimout ON feedback OFF
set pagesize 0
SELECT name,open_mode from v$database;
EOF
}
eval x=(`sqlQuery`)
NAME=${x[0]}
OPEN_MODE="${x[1]} ${x[2]}"
echo NAME IS $NAME
echo OPEN_MODE IS $OPEN_MODE
So we are running the same function you have above, passing it into x and running it through eval to handle the delimitation. Then you have an array and call call is as such: x[0] for the first item, for example.
Output is:
oracle#******:/******> sh test.sh
NAME IS ******
OPEN_MODE IS READ WRITE

Variable won't resolve inside PL-SQL statement

I am trying to pass a SQL command to the SQL*Plus command-line program (Oracle) in my KornShell (ksh) script, but my $MY_VAR variable seems to be failing to resolve. $MY_VAR is a string value.
sqlplus -s << EOF >> $LOG_FILE
$MY_SCHEMA
UPDATE my_table SET run_flag = 'I', start_time = to_char(sysdate, 'DD-Mon-YYYY HH24:MI:SS') WHERE (process_id = '$MY_VAR' AND run_flag != 'F');
COMMIT;
EOF
I can successfully echo out the $MY_VAR variable, and so I can see that the variable is populated, but it does not seem to be resolving when inserting the variable into the SQL command which I am providing as an argument to the SQL*Plus program. The log file for the script simply outputs:
0 rows updated. Commit complete.
The SQL seems to be valid as we can successfully execute the command in SQL Developer (albeit with a hardcoded value for $MY_VAR).
Any ideas on what I am missing here?
Thank you
Try this:
sql="$MY_SCHEMA
UPDATE my_table SET run_flag = 'I', start_time = to_char(sysdate, 'DD-Mon-YYYY HH24:MI:SS') WHERE (process_id = '$MY_VAR' AND run_flag != 'F');
COMMIT;"
print "$sql"
sqlplus -s <<<"$sql" >> $LOG_FILE
Does the sql look correct? If it does and zero rows are updated, your where clause must be selecting zero rows.
Your version of ksh may not have the <<< here-string syntax. In that case:
print "$sql" | sqlplus -s >> $LOG_FILE

Resources