Saving a single log file from multiples sh scripts - shell

I have 2 sh scripts that saves log files accordingly, but and I want to create another sh to call them.
They are using a property file that contains 3 log functions
my first sh(bteq.sh) creates a table in my Teradata
#!/bin/bash
source "/valdc_procs/properties/general_config_file"
exec_logs "/valdc_procs/logs/" "BTEQ_VALDC_PROCS"
echo "INFO : Starting table creation"
function exec_btqe (){
bteq<<EOF 2>&1
.logmech LDAP
.LOGON ${ipServerPR}/${userNamePR},${passwordPR};
.SET TITLEDASHES OFF;
.SET WIDTH 2000
.SET SEPARATOR ';'
SELECT 1
FROM DBC.TABLES
WHERE DatabaseName = ''
AND TableName = ''
AND TableKind = 'T';
.IF ACTIVITYCOUNT = 0 THEN .GOTO CreateNewTable;
DROP TABLE ;
.LABEL CreateNewTable;
CREATE MULTISET TABLE xpto AS (
SELECT
...
)
WITH DATA;
.LOGOFF;
.QUIT;
EOF
}
exec_btqe
#it gets the rc of the last command if it's an error
rc_result "Error creating table " "/valdc_procs/logs/" "BTEQ_VALDC_PROCS"
#it logs the process when there were no errors
log_output "/valdc_procs/logs/" "BTEQ_VALDC_PROCS"
exit $rc
my second sh(tdch.sh) export that table to a file
#!/bin/bash
source "/valdc_procs/properties/general_config_file"
exec_logs "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
hadoop jar $TDCH_JAR com.teradata.connector.common.tool.ConnectorImportTool \
-libjars $LIB_JARS \
-Dmapred.job.queue.name=default \
-Dtez.queue.name=default \
-Dmapred.job.name=TDCH \
-classname com.teradata.jdbc.TeraDriver \
-url jdbc:teradata://$ipServer/logmech=ldap,database=$database,charset=UTF16 \
-jobtype hdfs \
-fileformat textfile \
-separator ',' \
-enclosedby '"' \
-targettable ${targetTable} \
-username ${userName} \
-password ${password} \
-sourcequery "select * from ${database}.${targetTable}" \
-nummappers 1 \
-sourcefieldnames "" \
-targetpaths ${targetPaths}
#it gets the rc of the last command if it's an error
rc_result "Error exporting the file " "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
echo "INFO : Moving file from HDFS to the FileSystem"
hdfs dfs -copyToLocal ${targetPaths}/"part-m-00000" ${targetFileSystemPath}/ARQ_VALDC_PROCS_OPBK_$TIMESTAMP
rc_result "Error moving the file " "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
echo "INFO : File Moved Arquivo movido com sucesso"
#it logs the process when there were no errors
log_output "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
exit $rc
This log process is working fine when I run each sh individually, but now I want to have another sh calling them.
But I'm not sure how can I get the rc of their commands(bteq.sh and tdch.sh) and save in a single log file

Not 100% certain I understand your need, but that might solve your question:
#!/bin/bash
logfile="/tmp/somefile.txt"
bteq.sh
status_bteq=$?
tdch.sh
status_tdch=$?
echo "Status bteq: $status_bteq" >"$logfile"
echo "Status tdch: $status_tdch" >>"$logfile"
If you want just the return codes, raw without anyother text, you could do:
#!/bin/bash
logfile="/tmp/somefile.txt"
bteq.sh
echo "$?" >"$logfile"
tdch.sh
echo "$?" >>"$logfile"

Related

LiquiBase CLI BASH script not outputting full CLI with ECHO

Hey all I am new to the world of BASHing :)
I am trying to put together this CLI line of code by doing the following:
echo "------"
echo '--url=$URL '\
'--username=$UN '\
'--password=$PW '\
'--referenceUrl=$RURL '\
'--referenceUsername=$RUN '\
'--referencePassword=$RPW '\
'--changeSetAuthor=TD '\
'--diffTypes=tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints '\
'$LIQUI_ACTION'
echo "------"
And the output I am getting is this:
------
--url=$URL --username=$UN
/jenkins/liquibase-3.6.2/liquibase: line 141: --password=$PW : command not found
I can tell right off that its not placing the varibles' value within the cli. Second, I'm not sure why its saying command not found when I am only outputting text? I am just doing the above to make sure all of it is indeed the correct data in the $ and also that it's spaced between each command correctly.
So any help about the above would be great!
UPDATE FOR #RavinderSingh13:
code used:
echo "------"
echo "--url=$URL \
--username=$UN \
--password=$PW \
--referenceUrl=$RURL \
--referenceUsername=$RUN \
--referencePassword=$RPW \
--changeSetAuthor=TD \
--diffTypes=tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints \
$LIQUI_ACTION"
echo "------"

GhostScript auto pagenumbering

I want to export one certain page from a pdf document to an image and automatically fill the page number in the file name. When I run the following code:
gs \
-sDEVICE=jpeg \
-o outfile-%03.jpeg \
-dFirstPage=12 \
-dLastPage=12 \
wo.pdf
I get: outfile-001.jpeg instead of outfile-012.jpeg.
I've wrote a bash script for the job:
function extract_nth_page(){
printf -v j "outfile-%05g.png" $1
echo $j
gs -q -dNOPAUSE -sDEVICE=png16m -r600 -dFirstPage=$1 -dLastPage=$1 -sOutputFile=$j $2 -c quit
return 0
}
# Extracts page number 42 from myFile.pdf to outfile-00042.png
extract_nth_page 42 myFile.pdf

Cron job - exported file emailed is blank

I am receiving the email from the following CRON job but with a blank file attached. I have checked the /home/XXXXXX/public_html/tmp/ directory and a good tqhsa_hikashop_order.txt file is created in that /tmp/ folder but the one attached to the email that I receive is empty.
mysql --database=XXXXXXXX_jml_6UCHadruwR8veju3 -u XXXXXXXX_8eh6Nu --password=fRa6r7wRerEtuzud -B -e "SELECT order_number, order_created, order_invoice_number, order_full_price, order_discount_code, order_discount_price, order_payment_method, order_payment_price FROM tqhsa_hikashop_order;" > /home/XXXXXXXX/public_html/tmp/tqhsa_hikashop_order.txt | mail -s "Daily Discount Table Dump" -a /home/XXXXXXXX/public_html/tmp/tqhsa_hikashop_order.txt luke#example.com
Any ideas why the good text file is not being attached?
You need to ensure that your first command finishes before running the second, so that the file exists on disk.
So essentially replace the | with a &&:
mysql \
--database=XXXXXXXX_jml_6UCHadruwR8veju3 \
-u XXXXXXXX_8eh6Nu \
--password=fRa6r7wRerEtuzud \
-B \
-e "SELECT order_number, order_created, order_invoice_number, order_full_price, order_discount_code, order_discount_price, order_payment_method, order_payment_price FROM tqhsa_hikashop_order;" > /home/XXXXXXXX/public_html/tmp/tqhsa_hikashop_order.txt \
&& \
mail \
-s "Daily Discount Table Dump" \
-a /home/XXXXXXXX/public_html/tmp/tqhsa_hikashop_order.txt luke#example.com
To illustrate using an example:
myuser#myshell:~ $ echo "testing" > testy.txt | cat testy.txt
cat: testy.txt: No such file or directory
myuser#myshell:~ $ echo "testing" > testy.txt && cat testy.txt
testing

Issue on concatenating files shellscipt

Sorry, I'm from Brazil and my english is not fluent.
I wanna concatenate 20 files using a shellscript through cat command. However when I run it from a file, all content of files are showed on the screen.
When I run it directly from terminal, works perfectly.
That's my code above:
#!/usr/bin/ksh
set -x -a
. /PROD/INCLUDE/include.prod
DATE=`date +'%Y%m%d%H%M%S'`
FINAL_NAME=$1
# check if all paremeters are passed
if [ -z $FINAL_NAME ]; then
echo "Please pass the final name as parameter"
exit 1
fi
# concatenate files
cat $DIRFILE/AI6LM760_AI6_CF2_SLOTP01* $DIRFILE/AI6LM761_AI6_CF2_SLOTP02* $DIRFILE/AI6LM763_AI6_CF2_SLOTP04* \
$DIRFILE/AI6LM764_AI6_CF2_SLOTP05* $DIRFILE/AI6LM765_AI6_CF2_SLOTP06* $DIRFILE/AI6LM766_AI6_CF2_SLOTP07* \
$DIRFILE/AI6LM767_AI6_CF2_SLOTP08* $DIRFILE/AI6LM768_AI6_CF2_SLOTP09* $DIRFILE/AI6LM769_AI6_CF2_SLOTP10* \
$DIRFILE/AI6LM770_AI6_CF2_SLOTP11* $DIRFILE/AI6LM771_AI6_CF2_SLOTP12* $DIRFILE/AI6LM772_AI6_CF2_SLOTP13* \
$DIRFILE/AI6LM773_AI6_CF2_SLOTP14* $DIRFILE/AI6LM774_AI6_CF2_SLOTP15* $DIRFILE/AI6LM775_AI6_CF2_SLOTP16* \
$DIRFILE/AI6LM776_AI6_CF2_SLOTP17* $DIRFILE/AI6LM777_AI6_CF2_SLOTP18* $DIRFILE/AI6LM778_AI6_CF2_SLOTP19* \
$DIRFILE/AI6LM779_AI6_CF2_SLOTP20* > CF2_FINAL_TEMP
mv $DIRFILE/CF2_FINAL_TEMP $DIRFILE/$FINAL_NAME
I solved the problem putting the cat block inside a function, and redirecting stdout to the final file.
Ex:
concatenate()

badly placed ()'s when creating oracle database link in tcsh

#my code
echo \
'create database link remotec101 \
connect to "os_user" \
identified by "password" \
using ' \
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP) \
(HOST=c101) \
(PORT=1521)) \
(CONNECT_DATA=(SID=XE)))';'|sqlplus
I tried to run some sql in this way and it worked. But when creating database link I got error, saying badly placed ()'s
This code is in tcsh.
Please help me.
Thanks
The parentheses are not quoted, so they're treated as shell metacharacters.
This:
echo \
'create database link remotec101 \
connect to "os_user" \
identified by "password" \
using \
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP) \
(HOST=c101) \
(PORT=1521)) \
(CONNECT_DATA=(SID=XE)));' | sqlplus
will feed the following to the sqlplus command:
create database link remotec101
connect to "os_user"
identified by "password"
using
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)
(HOST=c101)
(PORT=1521))
(CONNECT_DATA=(SID=XE)));
But a "here document" is probably cleaner:
sqlplus <<'EOF'
create database link remotec101
connect to "os_user"
identified by "password"
using
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)
(HOST=c101)
(PORT=1521))
(CONNECT_DATA=(SID=XE)));
'EOF'
If you want the last 4 lines to become a single line of input to sqlplus, I think you'll need to put them all on one line in your script. Or you might find it easier to use the printf command to organize your output, for example:
printf '%s\n%s\n%s\n%s\n%s %s %s %s\n' \
'create database link remotec101' \
'connect to "os_user"' \
'identified by "password"' \
'using' \
'(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)' \
'(HOST=c101)' \
'(PORT=1521))' \
'(CONNECT_DATA=(SID=XE)));' | sqlplus
This prints the last 4 lines as a single line. You can adjust the format string as needed.
I figured it out. You have to utilize the fact that tcsh supports both quote types and echo allows multiple arguments:
echo 'create database link remotec101 \
connect to "os_user" \
identified by "password" \
using ' "'" ' ( DESCRIPTION= ( ADDRESS= ( PROTOCOL=TCP )
( HOST=c101 )
( PORT=1521 ) )
( CONNECT_DATA= ( SID=XE ) ) ) ' "';"

Resources