BTEQ Teradata passing query variable to shell to script - shell

I am new to Teradata.I am having a problem with my BTEQ program. The BTEQ job passes a variable shell script. The purpose of the program to check for any null values in the summary table that is 2 months delay. Here is an example of the table (the purpose of the program: when The Summary table is missing two months, for example, Oct and Nov then flag=Y. flag=Y means YES that there are null values in the Summary table for the month of Oct and Nov. flag=N means there are no null values):
dt_load maxmth summary
12-SEP Sep Sep
13 SEP Sep
29-SEP Sep
2-Oct Oct
3-Oct Oct
1-Nov Nov
2-Nov Nov
user="usr1"
pass="pass1"
SRC_DB="emp"
SOURCE_TABLE="summarytb"
FLAG =$(query_td)
td_query () { bteq << EOF |grep '^>' |sed -e "s/^>//"
${HOST}/${USER},${PASSWORD}
DATABASE $SRC_DB;
.set width 1000;
.set titledashes off;
SELECT COUNT(*)
FROM ${SRC_DB}.${SOURCE_TABLE};
( SELECT
dt_load,
summary ,
maxmth,
/*check for null 2 months back*/
MDIFF(summary, 2, maxmth, dt_load) AS diff1
FROM $SOURCE_TABLE
Group by dt_load) AS a
FROM
/*if summary field is null than insert 2*/
( SELECT *,
COALESCE(a.diff1,2) AS diff (Select FROM a) AS b
FROM
/* if diff= 2 than flag =Y else flag=N*/
( SELECT *,
CASE
When b.diff = 2 then ‘Y ‘ ELSE ‘N’
End ) AS flag (SELECT c.diff FROM b) c
FROM
/*pass flag variable to shell script*/
SELECT '>'||' c.flag';
$1
.LOGOFF;
.QUIT;
.EXIT
EOF
}
echo $FLAG
if [ $FLAG='Y' ] then echo Run file1;
exit 1
fi
IF [ $FLAG='N' ] then echo run file 2;
exit 1
fi

Related

Bash script not recognizing existing file

I have to write a simple bash script, which does the compilation of multiple sql scripts which can contain recursive references to the other sql scripts with Oracle SQLPlus convention, so in the end the result will be only one SQL file, containing all statements from all (possibly also recursively) referenced subscripts.
I have created a simple script called oracle-script-compressor.sh with following bash code:
#!/bin/bash
#
#
function err () {
echo $* > "/dev/stderr"
}
function replace_subscripts_placeholders() {
local PROCESSING_FILENAME=$1
echo "-- processing file $PROCESSING_FILENAME"
while read script_line; do
local script_line_NO_LEAD_SPACE="$(echo -e "${script_line}" | sed -e 's/^[[:space:]]*//')"
if [[ $script_line_NO_LEAD_SPACE == \#\#* ]] ; then
local file_name="./${script_line_NO_LEAD_SPACE#\#\#}"
echo "-- found reference to file $file_name"
echo "-- starting with processing, pwd=$PWD"
# for debug purposes:
ls -la
# but this returns always false:
if [ -f "$file_name" ]; then
# so this part is never started:
. replace_subscripts_placeholders $file_name
else
err_msg="WARNING: Could not find the referenced file $file_name"
echo "-- $err_msg"
err $err_msg
fi
else
echo "$script_line"
fi
done < $PROCESSING_FILENAME
}
if test -z "$1" ; then
err "Usage: oracle-script-compressor.sh {file_name_to_process} [> output_file]"
err " Be aware, if the referenced files within {file_name_to_process} are not specified by absolute paths, you have to start the script from corresponding directory."
err " If the part [> output_file] is omitted, then this script writes to standard output"
err " If the part [>> output_file] is used, then the result will be appended to output_file it it exists before the processing"
else
if [ -f "$1" ]; then
replace_subscripts_placeholders $1
else
echo "file $1 does not exist"
fi
fi
and I am stuck on interesting problem - it seems, inside of the function replace_subscripts_placeholders() the file check
if [ -f "$file_name" ]; then
. replace_subscripts_placeholders $file_name
else
err_msg="WARNING: Could not find the referenced file $file_name"
echo "-- $err_msg"
err $err_msg
fi
never goes to the recursion call and even, if I remove the if statement and really call the function in recursion with the correct referenced file name, which exists, it is still not recognized to be found and not passed into the loop over all lines of the referenced file in the recursive call (then the error file not found comes and the loop cannot be executed).
I have added the debug messages into script like mentioned above in the script, but still I am unable to find, why the hell should bash not find the file, if it is in the same directory. The scripts are placed in
user#mycomputer:/tmp/test$ ls -la
celkem 52
drwxr-xr-x 2 user user 4096 zář 14 21:47 .
drwxrwxrwt 38 root root 36864 zář 14 21:48 ..
-rw-r--r-- 1 user user 51 zář 14 21:45 a.sql
-rw-r--r-- 1 user user 51 zář 14 21:45 b.sql
-rw-r--r-- 1 user user 590 zář 14 21:46 start.sql
user#mycomputer:/tmp/test$
and the content of the file start.sql looks like this:
spool output__UpdSch.log
whenever sqlerror exit sql.sqlcode
--
--
PROMPT a.sql - starting
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as START_TIMESTAMP from dual;
##a.sql
PROMPT a.sql - finished
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as FINISH_TIMESTAMP from dual;
--
--
PROMPT b.sql - starting
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as START_TIMESTAMP from dual;
##b.sql
PROMPT b.sql - finished
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as FINISH_TIMESTAMP from dual;
--
--
spool off
and if I execute the script, it seems it decoded the filenames correctly, but there is still the problem in bash - the testing of the file existence returns always false:
user#mycomputer:/tmp/test$ ~/tmp/oracle-script-compressor.sh start.sql
-- processing file start.sql
spool output__UpdSch.log
whenever sqlerror exit sql.sqlcode
--
--
PROMPT a.sql - starting
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as START_TIMESTAMP from dual;
-- found reference to file ./a.sql
-- starting with processing, pwd=/tmp/test
celkem 52
drwxr-xr-x 2 user user 4096 zář 14 21:47 .
drwxrwxrwt 38 root root 36864 zář 14 21:48 ..
-rw-r--r-- 1 user user 51 zář 14 21:45 a.sql
-rw-r--r-- 1 user user 51 zář 14 21:45 b.sql
-rw-r--r-- 1 user user 590 zář 14 21:46 start.sql
-- WARNING: Could not find the referenced file ./a.sql
WARNING: Could not find the referenced file ./a.sql
PROMPT a.sql - finished
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as FINISH_TIMESTAMP from dual;
--
--
PROMPT b.sql - starting
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as START_TIMESTAMP from dual;
-- found reference to file ./b.sql
-- starting with processing, pwd=/tmp/test
celkem 52
drwxr-xr-x 2 user user 4096 zář 14 21:47 .
drwxrwxrwt 38 root root 36864 zář 14 21:48 ..
-rw-r--r-- 1 user user 51 zář 14 21:45 a.sql
-rw-r--r-- 1 user user 51 zář 14 21:45 b.sql
-rw-r--r-- 1 user user 590 zář 14 21:46 start.sql
-- WARNING: Could not find the referenced file ./b.sql
WARNING: Could not find the referenced file ./b.sql
PROMPT b.sql - finished
select to_char(current_timestamp,'YYYY-MM-DD HH24:MI:SS.FF3') as FINISH_TIMESTAMP from dual;
--
--
spool off
user#mycomputer:/tmp/test$
The script has read access to files, everything is in the same folder, where I start the script, It also does not matter, if I reference the file with the current directory prefix "./" or not, it is just never found. Interesting is, the check during the start of the script passes correctly, the only problem is the searching within the function... I have also tried to call the function with preceded "." and without it, it makes no difference... There are also no trailing spaces in the referenced file names... It also makes no difference if I declare the variables inside of the function as local or not.
So I have no idea, what could be the problem, maybe I just looked too long on it and cannot see something - seems it has to be something trivial, like the function is started in some different current directory - (but pwd and ls shows the directory is always correct...???) - Any help or pointers will be appreciated.
Thank you for the comments, it brought me to the solution. I found out, this problem was related to the fact, the test sql files were created on Windows, so they had in the end of line always the <CR><LF> characters. This leads to the fact in the original bash script I posted, the script line
while read script_line
puts into the variable not only the given file name from the line, preceded by the characters ##
##a.sql
but also the <CR> character - in the variable is then the value
##a.sql<CR>
what was the cause the file a.sql etc. could never be found. Of course the character is invisible, so therefore it has not been shown on any debug echoing I made here - I had to put the content of $file_name between some another characters and then i could see it in the echoed test... I made also some other corrections and the final working script looks in the following way, if somebody needs to join referenced SQL scripts into one, this does it:
#!/bin/bash
#
#
function err () {
echo $* > "/dev/stderr"
}
function replace_subscripts_placeholders() {
local PROCESSING_FILENAME=$1
echo "-- processing file $PROCESSING_FILENAME"
while read script_line || [ -n "$script_line" ]; do
local script_line_NO_LEAD_SPACE="$(echo -e "${script_line}" | sed -e 's/^[[:space:]]*//' | sed -e 's/^M$//')"
if [[ $script_line_NO_LEAD_SPACE == \#\#* ]] ; then
local file_name="${script_line_NO_LEAD_SPACE#\#\#}"
echo "-- found reference to file $file_name"
if [ -f "$file_name" ]; then
replace_subscripts_placeholders $file_name
else
err_msg="WARNING: Could not find the referenced file $file_name"
echo "-- $err_msg"
err $err_msg
fi
else
echo "$script_line"
fi
done < $PROCESSING_FILENAME
}
if test -z "$1" ; then
err "Usage: oracle-script-compressor.sh {file_name_to_process} [> output_file]"
err " Be aware, if the referenced files within {file_name_to_process} are not specified by absolute paths, you have to start the script from corresponding directory."
err " If the part [> output_file] is omitted, then this script writes to standard output"
err " If the part [>> output_file] is used, then the result will be appended to output_file it it exists before the processing"
else
if [ -f "$1" ]; then
replace_subscripts_placeholders $1
else
echo "file $1 does not exist"
fi
fi

Execute sqlplus sessions with jobs in powershell

I want to execute some sessions with various different scripts into a Oracle database. I want to do it with open jobs simultaneous but i think that i have problems with parameter pass. The $item_actual appears is null when start-job try to use.
$lista_scripts = #("insert","update","select")
For ($i=0; $i -le 2; $i++)
{
$item_actual=$lista_scripts[$i]
Write-Host "Ejecutandose $item_actual"
start-job {date;cat "$item_actual";cat $item_actual | sqlplus xxx/xxx#"(description=(address=(protocol=TCP)(host=xxx)(port=xxx))(connect_data=(SERVICE_NAME=xxx)))"; date;}
}
Thanks
$username = "scott"
$password = "tiger"
$connect_string ="test-ecdu"
$log_file = "c:\powershell_script\test\log.txt"
$lista_scripts = #("select 1 ,sysdate from dual ;","select 2,sysdate from dual;","select 3,sysdate from dual;")
For ($i=0; $i -le 2; $i++)
{
$item_actual=$lista_scripts[$i]
Write-Host "Ejecutandose $item_actual"
start-job -ArgumentList $username, $password,$connect_string,$item_actual, $log_file -ScriptBlock {param($username_, $password_,$connect_string_ , $item_actual_, $log_file_)
date >>$log_file_;
Write-output "RUN $item_actual_" >>$log_file_ ;
$sqlOutput= $item_actual_ | sqlplus -s $username_/$password_#$connect_string_;
Write-output $sqlOutput >>$log_file_
date >>$log_file_;
}
}
c:\powershell_script\test\log.txt
6 июля 2018 г. 13:20:45
RUN select 1 ,sysdate from dual ;
1 SYSDATE
---------- --------
1 06.07.18
Elapsed: 00:00:00.00
6 июля 2018 г. 13:20:45
6 июля 2018 г. 13:20:45
RUN select 2,sysdate from dual;
2 SYSDATE
---------- --------
2 06.07.18
Elapsed: 00:00:00.00
6 июля 2018 г. 13:20:45
6 июля 2018 г. 13:20:46
RUN select 3,sysdate from dual;
3 SYSDATE
---------- --------
3 06.07.18
Elapsed: 00:00:00.00
6 июля 2018 г. 13:20:46

Script runs in standalone but does not work in crontab

Here is the script which works well when executed in standalone but fails when scheduled as a cronjob,
#!/bin/bash
# TARGET TABLE COUNT FOR Server
#
CURRENTTARGETREF=$(sqlplus -s $DB_USER/$DB_PASS << END
set pagesize 0 feedback off verify off heading off echo off;
SELECT CURRENTTARGETREF FROM PARAMETER
exit;
END
)
TABLENAME="TARGET$CURRENTTARGETREF"
TARGETCOUNT=$(sqlplus -s $DB_USER/$DB_PASS << END
set pagesize 0 feedback off verify off heading off echo off;
SELECT COUNT(*) FROM $TABLENAME
exit;
END
)
echo "Current Target table count in server: $TARGETCOUNT"
#
# COUNTER TABLE COUNT FOR server
#
S1=`echo "SELECT COUNT(*) FROM COUNTER WHERE SERVERID=1;" | sqlplus $DB_USER/$DB_PASS | sed -n '/COUNT(\*)/{n;n;p}'`
S2=`echo "SELECT COUNT(*) FROM COUNTER WHERE SERVERID=2;" | sqlplus $DB_USER/$DB_PASS | sed -n '/COUNT(\*)/{n;n;p}'`
#
echo "Current Counter table count with SERVERID '1' in server:$S1"
echo "Current Counter table count with SERVERID '2' in server:$S2"
#
below is the cronjob for this script
31 01 * * * DS=$(date +\%Y-\%m-\%d); /path/Databasecount.sh >> /path/test.out.$DS.txt 2>&1
And i wanted to send the output of the script to mail and could you someone please help me on this.

Crond execute shell many times

I have the following timey.cpp code in RedHat 2.6.32 x86_64:
using namespace std ;
int main()
{
while( 1 ){
char x[64]={0} ;
strcpy( x,"1234567890") ;
std::string s = x ;
std::cout << "(" << x << ")" << std::endl ;
struct timeval localtimex ;
long secs,usecs ;
gettimeofday(&localtimex,0x00) ;
secs = localtimex.tv_sec ;
usecs = localtimex.tv_usec ;
//long mills = (time.tv_sec * 1000) + (time.tv_usec / 1000 ) ;
printf("secs=(%d),usecs=(%d)\n",secs,usecs) ;
sleep( 1 ) ;
} //while
}
in /home/informix/test, compiled by g++ --std=c++0x timey.cpp -o timey.exe,
and the shell timey.sh:
source /etc/bashrc
nohup /home/informix/test/timey.exe &
Then I run timey.sh by:
/home/informix/test/timey.sh
and take a look if timey.exe runs by:
ps -ef | grep timey
It seems timey.exe runs as expected:
informix 41340 1 0 10:32 pts/10 00:00:00 /home/informix/test/timey.exe
What confuses me is that I add this shell to crontab:
38 10 * * 1-5 informix /home/informix/test/timey.sh
and restart crond:
/etc/init.d/crond restart
What surprises me is I see 4 copies of timey.exe running:
ps -ef | grep timey
informix 41498 1 0 10:38 ? 00:00:00 /home/informix/test/timey.exe
informix 41499 1 0 10:38 ? 00:00:00 /home/informix/test/timey.exe
informix 41529 1 0 10:38 ? 00:00:00 /home/informix/test/timey.exe
informix 41561 1 0 10:38 ? 00:00:00 /home/informix/test/timey.exe
What did I do wrong so that 4 copies of timey.exe are running?
I see the following in /var/log/cron:
Jul 2 10:38:01 localhost CROND[41440]: (informix) CMD (/home/informix/test/timey.sh )
Jul 2 10:38:01 localhost CROND[41439]: (informix) CMD (/home/informix/test/timey.sh )
Jul 2 10:38:01 localhost CROND[41491]: (informix) CMD (/home/informix/test/timey.sh )
Jul 2 10:38:01 localhost CROND[41533]: (informix) CMD (/home/informix/test/timey.sh )
It seems that crond really ran timey.sh 4 times, but why?
Also:
In RedHat 2.6.32-279.el6.x86_64, it works!
In RedHat 2.6.32-358.el6.x86_64, it does not work!

Check if value within column exists, if not create the column

I need to check if one of the columns in my db contains specific value. If it doesn't I want to create that row with folowing values:
#!/bin/bash
#
MODEL=$1
if true (SELECT * FROM table.STATISTICS
WHERE MODEL = '$MODEL' )
do this (INSERT INTO table.STATISTICS('$MODEL',0,SYSDATE,0,SYSDATE,0); )
You could use a merge for this, run through SQL*Plus as a 'heredoc', so you don't have to do a separate count operation; the merge will do that for you effectively:
#!/bin/bash
MODEL=$1
sqlplus -s /nolog <<!EOF
connect user/pass
merge into statistics s
using (select '${MODEL}' as model, 0 as num1, sysdate as date1,
0 as num2, sysdate as date2 from dual) t
on (s.model = t.model)
when not matched then
insert (s.model, s.num1, s.date1, s.num2, s.date2)
values (t.model, t.num1, t.date1, t.num2, t.date2);
!EOF
But using your real column names, obviously. It's better to list them explicitly even for a plain insert.
get_count () {
sqlplus -s username/pass <<!
set heading off
set feedback off
set pages 0
select count(model) from statistics
where model='$MODEL';
!
}
count=$(get_count $1)
if [ "${count:-0}" -eq 0 ]; then
echo "its zero"
sqlplus -S username/pass << EOF
whenever sqlerror exit 1;
set echo on
set verify off
INSERT INTO table.STATISTICS VALUES('$MODEL',0,SYSDATE,0,SYSDATE,0);
exit;
EOF
fi

Resources