I am new to the shell scripting hence need help. I am trying to execute sql query against Oracle DB. Once sql query result is received, I need to iterate through the result (as it will return multiple rows and columns.).
Goal here is to invoke a REST api using curl for each record retrieved in db result. Input to REST api will be ROOT_PROC_ID column value.
Below is the shell script I have developed so far and sample output of the sql query.
#!/bin/bash
#Update below properties as per enviornment
export ENV=DEV;
export SERVERHOST=localhost
export SERVERPORT=9000
export SERVERUSER=admin
export SERVERPASSWORD=admin123
export DBHOST=localhost
export DBPORT=1537
export DBSID=ORCL
export DBUSER=SCOTT
export DBPASS=TIGER
export LOGDIR=/usr/app/$USER/data/logs/
#-------------------------------------------------------------------------
#----------- DO NOT EDIT AFTER THIS LINE ---------------------------------
#-------------------------------------------------------------------------
#create directory structure if not exists for storing log files
mkdir -p $LOGDIR/process_cancellation
mkdir -p $LOGDIR/process_cancellation/old
mkdir -p $LOGDIR/process_cancellation/halted
export old_proc_cancellation_logfile=$LOGDIR/process_cancellation/old/log_$(date "+%Y%m%d%H%M%S").log;
export halted_proc_cancellation_logfile=$LOGDIR/process_cancellation/halted/log_$(date "+%Y%m%d%H%M%S").log;
#execute sql query to fetch halted process data from database
echo
echo "Enviornment : $ENV"
echo
echo "Connecting to - $DBUSER/$DBPASS#$DBHOST:$DBPORT/$DBSID"
echo
echo "Retrieving halted process data logged before : $(date -d "15 days ago" +%d/%m/%Y) 20:00:00"
echo
sqlplus -s $DBUSER/$DBPASS#$DBHOST:$DBPORT/$DBSID << FIN_SQL > $halted_proc_cancellation_logfile
set head off
set line 1024
set pages 9999
SET SERVEROUTPUT ON;
SELECT ROOT_PROC_ID, PROC_ID, PROC_NAME, START_DATE, STATUS, ORDER_REF
FROM USER.PROC_STATUS
WHERE START_DATE<(SYSDATE - 15) AND (STATUS='proc_halted' OR STATUS='proc_failed')
ORDER BY START_DATE DESC;
SET SERVEROUTPUT OFF;
FIN_SQL
echo "Please check log file for more details : $(readlink -f $halted_proc_cancellation_logfile)"
exit
Sample SQL query output:
ROOT_PROC_ID PROC_ID PROC_NAME START_DATE STATUS ORDER_REF
pvm:0a123akpd pvm:0a123akkh FunctionalErrorProcess 28-NOV-19 01.24.35.115000000 PM pi_halted 2642277
pvm:0a122utrn pvm:0a122uun0 TechnicalErrorProcess 22-NOV-19 02.28.17.217000000 PM pi_halted 2642278
pvm:0a122utl2 pvm:0a122uu1t TechnicalErrorProcess 22-NOV-19 02.27.54.024000000 PM pi_halted 2642279
pvm:0a122utln pvm:0a122uu22 TechnicalErrorProcess 22-NOV-19 02.27.50.287000000 PM pi_halted 2642280
Assuming your sql query output is in output.txt:
awk 'NR!=1' output.txt | while read rootprocid undef
do
callApi $rootprocid
done
NR!=1 is to skip the 1st line which contains the header.
read rootprocid undef reading only the 1st column in rootprocid, rest goes to variable undef since it is not of interest.
callApi $rootprocid callAPI will be replaced with your actual api call.
Related
I have a strange problem - possibly I'm just going blind. I have this short script, which replaces the string #qry# in the here-document with a select statement in a file and then pipes it to mysql:
#!/bin/bash
if [[ "$1" == "-h" ]]
then
echo "sqljob [sqlfile] [procnm] [host] [database] [config file]"
echo " sqlfile: text file containing an SQL statement"
echo " procnm: name that will given to the new, stored procedure"
echo " host: hostname of IP address of the database server"
echo " database: the procedure will be created here"
echo " config file: default configuration file with username and password"
exit
fi
infile=$1
procnm=$2
hn=$3
pn=$4
db=$5
mycfg=$6
{
set -o noglob
sed -e "s/#qry#/$(echo $(cat $infile))/g" <<!
drop procedure if exists $procnm;
delete from jobs where jobname="$procnm";
insert into jobs
set
notes="SQL job $procnm",
jobname="$procnm",
parm_tmpl='int';
delimiter //
create procedure $procnm(vqid int)
begin
call joblogmsg(vqid,0,"$procnm","","Executing #qry#");
drop table if exists ${procnm}_res;
create table ${procnm}_res as
#qry#
end//
delimiter ;
!
} | mysql --defaults-file=$mycfg -h $hn -P $pn $db
However, when the select contains *, it expands to whatever is in the directory even though I use noglob. However, it works from the command line:
$ set -o noglob
$ ls *
What am I doing wrong?
Edit
Block Comments in a Shell Script has been suggested as a duplicate, but as you will notice, I need to expand ${procnm} in the here-doc; I just need to avoid the same happening to select *.
I suspect it is because the construct echo (cat). The echo command gets the * from the cat command and the shell in which it runs expands it. In that shell set noglob is not active.
Try leaving the echo away: /$(cat $infile)/, in the end that is the data you need; then there is no extra glob expansion by a shell.
I have a requirement where I need to parameterize to generate one extract file from multiple Oracle tables through the UNIX shell script.
Here is the script which I have written to generate one tab delimited file which will fetch all the data from EMPLOYEE table.
I need to parameterize the TABLE_NAME,OWNER_NAME,USERNAME,PASSWORD and HOST to generate from 12 more tables.
So, I would like to have only one SQL to dyngenerate the extract for 12 tables by passing these parameters values when executing the scripts.
Could you please give me show me how we can modify the below script and how to pass the parameter during the script execution.
Second Requirement is to generate the file incrementally based on a column for example, ETL_UPDATE_TS. can you please show me this also.
Sample Scripts
#!/usr/bin/ksh
TD=/mz/mz01/TgtFiles
MD=/mz/mz01/Scripts
#CAQH_Server=sftp.org
#UN=user
#PWD=password
#RD=Incoming
#RD=/home/
cd $TD
FILE="EMPLOYEE.TXT"
sqlplus -s scott/tiger#db <<EOF
SET PAGES 999
SET COLSEP " "
SET LINES 999
SET FEEDBACK OFF
SPOOL $FILE
SELECT * FROM EMP;
SPOOL OFF
EXIT
EOF
Handling your parameters in a similar way you did for $FILE variable and passing them as options to the script
#!/usr/bin/ksh
TD=/mz/mz01/TgtFiles
MD=/mz/mz01/Scripts
cd $TD
FILE="undefined"
TABLE="undefined"
while getopts :f:t: opt
do
case $opt in
f) FILE=${OPTARG} ;;
t) TABLE=${OPTARG} ;;
*) echo "invalid flag" ;;
esac
done
if [ "$TABLE" == "undefined" ]; then
echo "ERROR. TABLE is undefined, use -f option."
exit 1
fi
# More required variables checks here
# create more options to parameterize connection
sqlplus -s scott/tiger#db <<EOF
SET PAGES 999
SET COLSEP " "
SET LINES 999
SET FEEDBACK OFF
SPOOL $FILE
SELECT * FROM $TABLE;
SPOOL OFF
EXIT
EOF
An execute it as
my_script.sh -f "EMPLOYEE.TXT" -t "EMP"
I need to develop a shell script that updates some records on a oracle database, this programs read an input .txt file and then updates the record when some conditions are met, but for some reason this records were not updated, it seems that there's a problem with a variable that the plsql reads in order to update the records.
This is the shell script code
## #!/bin/sh
. /usr/local/bin/oracle.profile.prod
. /usr/local/bin/bscs.profile.prod
hostname=`uname -n`
basedato=BSCS1REP
if [[ $basedato = "BSCS2PROD" && $hostname = "comp35" ]];then
export Y/Shell
export Y/Sql
export Y/Log
export Y/Cfg
. Y.MM.txt
elif [ $basedato = "BSCS1REP" and $hostname = "comp44" ]; then
export X/Shell
export X/Sql
export X/Log
export X/Cfg
. X/.MM.txt
else
echo "ERROR, Servidor no valido para la ejecucion de este programa"
echo "Base de Datos = "$basedato
echo "Maquina = "$hostname
exit 1
fi
FECHA=$(date "+%Y%m%d_%H_%M_%S")
cat $RUTA_CFG/MSISDN.txt | while read elemento
do
set $elemento
elemento=$1
echo "El registro es $elemento "insertado""
sqlplus -S $USER/$PASS#$basedato #$RUTA_PLS/Num_Act.sql $elemento > $RUTA_LOG/test.txt
done
rm -f $RUTA_CFG/Act_num.ctl
echo "DONE"
As you can see this script reads an input data, is the following:
123456789
Just this simple string.
Now this is the invoked Pl/sql
ALTER SESSION SET OPTIMIZER_GOAL = CHOOSE;
ALTER SESSION SET NLS_LANGUAGE = AMERICAN;
ALTER SESSION SET NLS_TERRITORY = AMERICA;
SET serveroutput ON SIZE 1000000
SET term ON
WHENEVER SQLERROR CONTINUE
DECLARE
elemento constant varchar2(30) := '$1';
BEGIN
update directory_number dn
set dn.dn_status = 'a'
where dn.dn_num = ('$1')
and dn.dn_status in ('d','t','Z','f','r');
DBMS_OUTPUT.PUT_LINE (elemento);
END;
/
EXIT;
I added a log file to see what's going on, and it returned the following:
Session altered.
Session altered.
Session altered.
$1
PL/SQL procedure successfully completed.
The variable stands $1, and it should be the input data number.
Could be a sintax error ?, a variable declaration ?
When I put aside the variables and put directly the valor of the desire number in the plsql it updated the record with no problems.
Any help with be appreciated, thank so much for your time !
In your PL/SQL code try replacing $1 with &1.
See this link for an example and description.
I am trying to fetch a row with more than one column value to different shell variables. Infact I found that at a time all the column values can be stored to single shell variable. But how can I put those column values to seperate shell variables. Below is an example I am trying for time being
function sqlQuery {
sqlplus -S shiyas/********* <<'EOF'
set heading OFF termout ON trimout ON feedback OFF
set pagesize 0
SELECT name,open_mode from v$database;
EOF
}
OUTPUT="$( sqlQuery )"
echo $OUTPUT
Here I am getting the output as
ORCL READ WRITE
But my requirement is column values ORCL, READ WRITE should get assigned to different shell variable.
I tried the below of parsing.
echo "$OUTPUT" | while read name open_mode
but it was throwing unexpected end of file error.
-bash-3.2$ sh call_sql_col_val_1.sh
ORCL READ WRITE
call_sql_col_val_1.sh: line 18: syntax error: unexpected end of file
Please let me know what concept I can use to fetch a single row column values to different shell variables.
I do this via eval myself:
oracle#******:/*****> cat test.sh
#!/bin/bash
function sqlQuery {
sqlplus -S / as sysdba <<'EOF'
set heading OFF termout ON trimout ON feedback OFF
set pagesize 0
SELECT name,open_mode from v$database;
EOF
}
eval x=(`sqlQuery`)
NAME=${x[0]}
OPEN_MODE="${x[1]} ${x[2]}"
echo NAME IS $NAME
echo OPEN_MODE IS $OPEN_MODE
So we are running the same function you have above, passing it into x and running it through eval to handle the delimitation. Then you have an array and call call is as such: x[0] for the first item, for example.
Output is:
oracle#******:/******> sh test.sh
NAME IS ******
OPEN_MODE IS READ WRITE
I have 100 records. When I am running the code below, the SQL*Plus connection is opening many times utilizing 100% of CPU. Is there any way in which I could open SQL*Plus connection only once, i. e. outside while loop?
**#!/bin/bash
export ORACLE_HOME=/software/oracle/ora10204
export PATH=$PATH:$ORACLE_HOME/bin
INPUT_FILE='file.csv'
IFS=','
i=0
while read name id do
a[i]="$name"
b[i]="$id"
echo "${a[$i]} ${b[$i]}"
set serveroutput on;
sqlplus .../...#'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=...)(HOST=...)(PORT=...)))(CONNECT_DATA=(SID=...)))'<<EOF
insert into code_entry(inh_valu,edi_valu) values('${a[$i]}' , '${b[$i]}');
EOF
let i=$i+1
done < $INPUT_FILE**
You could create a SQL script, that you would invoke once in the end of your shell script with SQL*Plus. It would use INSERT ALL DML command, and you would just append the strings to it using your shell script and redirect operator (>>) within your existing loop.
After running your shell script your SQL script could look like this:
insert all
into code_entry (inh_valu, edi_valu) values (1, 'foo')
into code_entry (inh_valu, edi_valu) values (2, 'bar');
...
You would then just run it like this:
$ sqlplus scott/tiger #myscript.sql
A sample shell script could look like:
echo "insert all" > myscript.sql
INPUT_FILE='file.csv'
IFS=','
i=0
while read name id do
a[i]="$name"
b[i]="$id"
echo "into code_entry(...) values ('${a[$i]}', '${b[$i]}')" >> myscript.sql
let i=$i+1
done < $INPUT_FILE**
echo ";" >> myscript.sql
sqlplus .../...#'(DESCRIPTION=...)' #myscript.sql