I have 2 script
script 1 : demo_details.txt
script 2 : demo.sh
Script 1 : Path : /demo/d/demo_details.txt and contain below details
export CON_DB_TECY=Username/Password#host:port/Servicename -> `abc/abc#local:123/orabc`
Script 2 : Path : /demo/d/demo.sh and contain below code
. /demo/d/demo_details.txt
sqlplus -s -S << EOF
$CON_DB_TECY
select * from dual;
exit;
EOF
When i run above script 2 using -> sh x demo.sh
It prints details of demo_details.txt => CON_DB_TECY=abc/abc#local:123/orabc
which is connection details that i want to secure and
should not be displayed when i run script using sh -x demo.sh
you can put all connection and sql in demo_details.txt and redirect it
demo_details.txt
Username/Password.....
select * from dual;
exit;
then
demo.sh
sqlplus -s -S <demo_details.txt
Related
I am trying to execute a shell script on a windows node using Jenkins.
The bash script uses sort -u flag in one of the steps to filter out unique elements from an existing array
list_unique=($(echo "${list[#]}" | tr ' ' '\n' | sort -u | tr '\n' ' '))
Note - shebang used in the script is #!/bin/bash
On calling the script from command prompt as - bash test.sh $arg1
I got the following error -
-uThe system cannot find the file specified.
I understand the issue was that with the above call, sort.exe was being used from command prompt and not the Unix sort command. To get around this I changed the path variable in Windows System variables and moved \cygwin\bin ahead of \Windows\System32
This fixed the issue and the above call gave me the expected results.
However, When the same script is called on this node using Jenkins, I get the same error again
-uThe system cannot find the file specified.
Jenkins stage calling the script
stage("Run Test") {
options {
timeout(time: 5, unit: 'MINUTES')
}
steps {
script {
if(fileExists("${Test_dir}")){
dir("${Test_dir}"){
if(fileExists("test.sh")){
def command = 'bash test.sh ${env.arg1}'
env.output = sh(returnStdout: true , script : "${command}").trim()
if (env.output == "Invalid"){
def err_msg = "Error Found."
sh "echo -n '" + err_msg + " ' > ${ERR_MSG_FILE}"
error(err_msg)
}
sh "echo Running tests for ${env.output}"
}
}
}
}
}
}
Kindly Help
I am new to the shell scripting hence need help. I am trying to execute sql query against Oracle DB. Once sql query result is received, I need to iterate through the result (as it will return multiple rows and columns.).
Goal here is to invoke a REST api using curl for each record retrieved in db result. Input to REST api will be ROOT_PROC_ID column value.
Below is the shell script I have developed so far and sample output of the sql query.
#!/bin/bash
#Update below properties as per enviornment
export ENV=DEV;
export SERVERHOST=localhost
export SERVERPORT=9000
export SERVERUSER=admin
export SERVERPASSWORD=admin123
export DBHOST=localhost
export DBPORT=1537
export DBSID=ORCL
export DBUSER=SCOTT
export DBPASS=TIGER
export LOGDIR=/usr/app/$USER/data/logs/
#-------------------------------------------------------------------------
#----------- DO NOT EDIT AFTER THIS LINE ---------------------------------
#-------------------------------------------------------------------------
#create directory structure if not exists for storing log files
mkdir -p $LOGDIR/process_cancellation
mkdir -p $LOGDIR/process_cancellation/old
mkdir -p $LOGDIR/process_cancellation/halted
export old_proc_cancellation_logfile=$LOGDIR/process_cancellation/old/log_$(date "+%Y%m%d%H%M%S").log;
export halted_proc_cancellation_logfile=$LOGDIR/process_cancellation/halted/log_$(date "+%Y%m%d%H%M%S").log;
#execute sql query to fetch halted process data from database
echo
echo "Enviornment : $ENV"
echo
echo "Connecting to - $DBUSER/$DBPASS#$DBHOST:$DBPORT/$DBSID"
echo
echo "Retrieving halted process data logged before : $(date -d "15 days ago" +%d/%m/%Y) 20:00:00"
echo
sqlplus -s $DBUSER/$DBPASS#$DBHOST:$DBPORT/$DBSID << FIN_SQL > $halted_proc_cancellation_logfile
set head off
set line 1024
set pages 9999
SET SERVEROUTPUT ON;
SELECT ROOT_PROC_ID, PROC_ID, PROC_NAME, START_DATE, STATUS, ORDER_REF
FROM USER.PROC_STATUS
WHERE START_DATE<(SYSDATE - 15) AND (STATUS='proc_halted' OR STATUS='proc_failed')
ORDER BY START_DATE DESC;
SET SERVEROUTPUT OFF;
FIN_SQL
echo "Please check log file for more details : $(readlink -f $halted_proc_cancellation_logfile)"
exit
Sample SQL query output:
ROOT_PROC_ID PROC_ID PROC_NAME START_DATE STATUS ORDER_REF
pvm:0a123akpd pvm:0a123akkh FunctionalErrorProcess 28-NOV-19 01.24.35.115000000 PM pi_halted 2642277
pvm:0a122utrn pvm:0a122uun0 TechnicalErrorProcess 22-NOV-19 02.28.17.217000000 PM pi_halted 2642278
pvm:0a122utl2 pvm:0a122uu1t TechnicalErrorProcess 22-NOV-19 02.27.54.024000000 PM pi_halted 2642279
pvm:0a122utln pvm:0a122uu22 TechnicalErrorProcess 22-NOV-19 02.27.50.287000000 PM pi_halted 2642280
Assuming your sql query output is in output.txt:
awk 'NR!=1' output.txt | while read rootprocid undef
do
callApi $rootprocid
done
NR!=1 is to skip the 1st line which contains the header.
read rootprocid undef reading only the 1st column in rootprocid, rest goes to variable undef since it is not of interest.
callApi $rootprocid callAPI will be replaced with your actual api call.
I have a requirement where I need to parameterize to generate one extract file from multiple Oracle tables through the UNIX shell script.
Here is the script which I have written to generate one tab delimited file which will fetch all the data from EMPLOYEE table.
I need to parameterize the TABLE_NAME,OWNER_NAME,USERNAME,PASSWORD and HOST to generate from 12 more tables.
So, I would like to have only one SQL to dyngenerate the extract for 12 tables by passing these parameters values when executing the scripts.
Could you please give me show me how we can modify the below script and how to pass the parameter during the script execution.
Second Requirement is to generate the file incrementally based on a column for example, ETL_UPDATE_TS. can you please show me this also.
Sample Scripts
#!/usr/bin/ksh
TD=/mz/mz01/TgtFiles
MD=/mz/mz01/Scripts
#CAQH_Server=sftp.org
#UN=user
#PWD=password
#RD=Incoming
#RD=/home/
cd $TD
FILE="EMPLOYEE.TXT"
sqlplus -s scott/tiger#db <<EOF
SET PAGES 999
SET COLSEP " "
SET LINES 999
SET FEEDBACK OFF
SPOOL $FILE
SELECT * FROM EMP;
SPOOL OFF
EXIT
EOF
Handling your parameters in a similar way you did for $FILE variable and passing them as options to the script
#!/usr/bin/ksh
TD=/mz/mz01/TgtFiles
MD=/mz/mz01/Scripts
cd $TD
FILE="undefined"
TABLE="undefined"
while getopts :f:t: opt
do
case $opt in
f) FILE=${OPTARG} ;;
t) TABLE=${OPTARG} ;;
*) echo "invalid flag" ;;
esac
done
if [ "$TABLE" == "undefined" ]; then
echo "ERROR. TABLE is undefined, use -f option."
exit 1
fi
# More required variables checks here
# create more options to parameterize connection
sqlplus -s scott/tiger#db <<EOF
SET PAGES 999
SET COLSEP " "
SET LINES 999
SET FEEDBACK OFF
SPOOL $FILE
SELECT * FROM $TABLE;
SPOOL OFF
EXIT
EOF
An execute it as
my_script.sh -f "EMPLOYEE.TXT" -t "EMP"
The normal and formal here doc in bash script is as below.
clear-mysql(){
mysqlword="xxxxx"
port="22"
ip="11.22.33.44"
tmpdb="xxxx"
ssh -p $port root#{$ip} "mysql -u root -p$mysqlword <<EOF
use ${tmpdb};
UPDATE xx SET yy = replace(post_content, 'domain', '"http://$ip/wp"');
EOF"
}
And you can use another right format(a tab and <<- )in this kind of bash.
clear-mysql(){
mysqlword="xxxxx"
port="22"
ip="11.22.33.44"
tmpdb="xxxx"
ssh -p $port root#{$ip} "mysql -u root -p$mysqlword <<-EOF
use ${tmpdb};
UPDATE xx SET yy = replace(post_content, 'domain', '"http://$ip/wp"');
EOF"
}
Now i write a wrong format (two tabs and <<-) for this simple bash snippet as below.
clear-mysql(){
mysqlword="xxxxx"
port="22"
ip="11.22.33.44"
tmpdb="xxxx"
ssh -p $port root#{$ip} "mysql -u root -p$mysqlword <<-EOF
use ${tmpdb};
UPDATE xx SET yy = replace(post_content, 'domain', '"http://$ip/wp"');
EOF"
}
Copy and paste it into terminal.
I knew it is a wrong format and i knew the right here string format in bash.
What confused me is that why two tabs and <<-EOF and EOF result in listing all files in current directory?
Yes ,two tabs triggered auto completion,
[root#localhost tmp]#
Display all 1137 possibilities? (y or n)
Tab and tab will display all 1137 ,instead of listing files in current directory.
I am trying to fetch a row with more than one column value to different shell variables. Infact I found that at a time all the column values can be stored to single shell variable. But how can I put those column values to seperate shell variables. Below is an example I am trying for time being
function sqlQuery {
sqlplus -S shiyas/********* <<'EOF'
set heading OFF termout ON trimout ON feedback OFF
set pagesize 0
SELECT name,open_mode from v$database;
EOF
}
OUTPUT="$( sqlQuery )"
echo $OUTPUT
Here I am getting the output as
ORCL READ WRITE
But my requirement is column values ORCL, READ WRITE should get assigned to different shell variable.
I tried the below of parsing.
echo "$OUTPUT" | while read name open_mode
but it was throwing unexpected end of file error.
-bash-3.2$ sh call_sql_col_val_1.sh
ORCL READ WRITE
call_sql_col_val_1.sh: line 18: syntax error: unexpected end of file
Please let me know what concept I can use to fetch a single row column values to different shell variables.
I do this via eval myself:
oracle#******:/*****> cat test.sh
#!/bin/bash
function sqlQuery {
sqlplus -S / as sysdba <<'EOF'
set heading OFF termout ON trimout ON feedback OFF
set pagesize 0
SELECT name,open_mode from v$database;
EOF
}
eval x=(`sqlQuery`)
NAME=${x[0]}
OPEN_MODE="${x[1]} ${x[2]}"
echo NAME IS $NAME
echo OPEN_MODE IS $OPEN_MODE
So we are running the same function you have above, passing it into x and running it through eval to handle the delimitation. Then you have an array and call call is as such: x[0] for the first item, for example.
Output is:
oracle#******:/******> sh test.sh
NAME IS ******
OPEN_MODE IS READ WRITE