issue with awk and variables from previous line - bash

Code typically speaks better than thousand bytes. So did this demo code that does not obey my logic. Typically I trust compilers and programs, so I think I have a typo or something somewhere.
Please, could you point it out to me?
copy & paste code below into mydemo.sh file, and run it using command sh ./mydemo.sh
in my ref system this prints out:
mytux:~# ./mydemo.sh
This should spit out:
{USER1NAME} {Firstname Surname} {user1name}
{USER2NAME} {Secondname Surname} {user2name}
{USER3NAME} {Thirdname Surname} {user3name}
---- but it spits out:
{USER1NAME} {somestring: user1name}, {user1name}
{USER2NAME} {somestring: user2name}, {user2name}
{USER3NAME} {somestring: user3name}, {user3name}
----
Why so and how to fix it?
and here's the code:
#!/bin/sh
echo "# Firstname Surname, Company">usernames.txt
echo "somestring: user1name">>usernames.txt
echo "# Secondname Surname, Company">>usernames.txt
echo "somestring: user2name">>usernames.txt
echo "# Thirdname Surname, Company">>usernames.txt
echo "somestring: user3name">>usernames.txt
echo "This should spit out:"
echo "{USER1NAME} {Firstname Surname}, {user1name}"
echo "{USER2NAME} {Secondname Surname}, {user2name}"
echo "{USER3NAME} {Thirdname Surname}, {user3name}"
echo "---- but it spits out:"
echo "{USER1NAME} {somestring: user1name}, {user1name}"
echo "{USER2NAME} {somestring: user2name}, {user2name}"
echo "{USER3NAME} {somestring: user3name}, {user3name}"
echo "---- See:"
cat usernames.txt|awk \
'BEGIN { $mylink="";} \
{ \
if(match($0,"^#")!=0) \
{ \
split($0, a, ","); \
$mylink=$a[1]; \
} \
else \
{ \
if(length($mylink)>0) \
{ \
print "{" toupper($2) "} {" $mylink "}, {" $2 "}"; \
} \
$mylink=""; \
} \
}'
echo "----"
echo "Why so and how to fix it?"

Is this what you're trying to achieve? It would be easier to post the input text as is.
$ awk -F"[, ]" -v OFS="}, {" '
/^#/{n=$2" "$3;next}
{print "{" toupper($2), n, $2"}"}
' usernames.txt
{USER1NAME}, {Firstname Surname}, {user1name}
{USER2NAME}, {Secondname Surname}, {user2name}
{USER3NAME}, {Thirdname Surname}, {user3name}

Related

Saving a single log file from multiples sh scripts

I have 2 sh scripts that saves log files accordingly, but and I want to create another sh to call them.
They are using a property file that contains 3 log functions
my first sh(bteq.sh) creates a table in my Teradata
#!/bin/bash
source "/valdc_procs/properties/general_config_file"
exec_logs "/valdc_procs/logs/" "BTEQ_VALDC_PROCS"
echo "INFO : Starting table creation"
function exec_btqe (){
bteq<<EOF 2>&1
.logmech LDAP
.LOGON ${ipServerPR}/${userNamePR},${passwordPR};
.SET TITLEDASHES OFF;
.SET WIDTH 2000
.SET SEPARATOR ';'
SELECT 1
FROM DBC.TABLES
WHERE DatabaseName = ''
AND TableName = ''
AND TableKind = 'T';
.IF ACTIVITYCOUNT = 0 THEN .GOTO CreateNewTable;
DROP TABLE ;
.LABEL CreateNewTable;
CREATE MULTISET TABLE xpto AS (
SELECT
...
)
WITH DATA;
.LOGOFF;
.QUIT;
EOF
}
exec_btqe
#it gets the rc of the last command if it's an error
rc_result "Error creating table " "/valdc_procs/logs/" "BTEQ_VALDC_PROCS"
#it logs the process when there were no errors
log_output "/valdc_procs/logs/" "BTEQ_VALDC_PROCS"
exit $rc
my second sh(tdch.sh) export that table to a file
#!/bin/bash
source "/valdc_procs/properties/general_config_file"
exec_logs "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
hadoop jar $TDCH_JAR com.teradata.connector.common.tool.ConnectorImportTool \
-libjars $LIB_JARS \
-Dmapred.job.queue.name=default \
-Dtez.queue.name=default \
-Dmapred.job.name=TDCH \
-classname com.teradata.jdbc.TeraDriver \
-url jdbc:teradata://$ipServer/logmech=ldap,database=$database,charset=UTF16 \
-jobtype hdfs \
-fileformat textfile \
-separator ',' \
-enclosedby '"' \
-targettable ${targetTable} \
-username ${userName} \
-password ${password} \
-sourcequery "select * from ${database}.${targetTable}" \
-nummappers 1 \
-sourcefieldnames "" \
-targetpaths ${targetPaths}
#it gets the rc of the last command if it's an error
rc_result "Error exporting the file " "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
echo "INFO : Moving file from HDFS to the FileSystem"
hdfs dfs -copyToLocal ${targetPaths}/"part-m-00000" ${targetFileSystemPath}/ARQ_VALDC_PROCS_OPBK_$TIMESTAMP
rc_result "Error moving the file " "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
echo "INFO : File Moved Arquivo movido com sucesso"
#it logs the process when there were no errors
log_output "/valdc_procs/logs/" "TDCH_VALDC_PROCS"
exit $rc
This log process is working fine when I run each sh individually, but now I want to have another sh calling them.
But I'm not sure how can I get the rc of their commands(bteq.sh and tdch.sh) and save in a single log file
Not 100% certain I understand your need, but that might solve your question:
#!/bin/bash
logfile="/tmp/somefile.txt"
bteq.sh
status_bteq=$?
tdch.sh
status_tdch=$?
echo "Status bteq: $status_bteq" >"$logfile"
echo "Status tdch: $status_tdch" >>"$logfile"
If you want just the return codes, raw without anyother text, you could do:
#!/bin/bash
logfile="/tmp/somefile.txt"
bteq.sh
echo "$?" >"$logfile"
tdch.sh
echo "$?" >>"$logfile"

bash script for seismic date

I have the following command and the original work 100 %,
the original
suchw key1=gstat key3=gelev c=-1 b=-0.00245187 key2=gy a=11803.7 d=3.2 < cmp.su > cmp_geophon_statik.su &
#suchw key1=sstat key3=selev c=-1 b=-0.00245187 key2=sy a=11803.7 d=3.2 < cmp_geophon_statik.su > cmp_statik.su &
#sustatic hdrs=1 sign=1 < cmp_statik.su > cmp_mit_statik.su &
#cat cmp_mit_statik.su | suwind key=ep min=1195 max=1197 | supsimage perc=95 title='der siemogram nach staitch' > evince www.pdf &
not: with last commend I tried to export the result to pdf file but it look like not work exactly just i would
but i wanted to write it like bash script(like 'trt.su') and i wrot them so,
#!/bin/bash
cat cmp.su \
| suchw key1=gstat key3=gelev c=-1 b=-0.00211187t key2=gy a=11803.7 d=3.2 \
| > cmp_geophon_statik.su &&
exit
#cat cmp_geophon_statik.su \
#| suchw key1=sstat key3=selev c=-1 b=$geschwindigkeit key2=sy a=11803.7 d=3.2 \
#| >> cmp_statik.su
#exit
#cat cmp_statik.su \
#| sustatic hdrs=1 sign=1 \
#| >> cmp_mit_statik.su
#exit
cat cmp_mit_statik.su | suwind key=ep min=913 max=913 | suxwigb clip=10
but they didn't worked.
can you help me?

LiquiBase CLI BASH script not outputting full CLI with ECHO

Hey all I am new to the world of BASHing :)
I am trying to put together this CLI line of code by doing the following:
echo "------"
echo '--url=$URL '\
'--username=$UN '\
'--password=$PW '\
'--referenceUrl=$RURL '\
'--referenceUsername=$RUN '\
'--referencePassword=$RPW '\
'--changeSetAuthor=TD '\
'--diffTypes=tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints '\
'$LIQUI_ACTION'
echo "------"
And the output I am getting is this:
------
--url=$URL --username=$UN
/jenkins/liquibase-3.6.2/liquibase: line 141: --password=$PW : command not found
I can tell right off that its not placing the varibles' value within the cli. Second, I'm not sure why its saying command not found when I am only outputting text? I am just doing the above to make sure all of it is indeed the correct data in the $ and also that it's spaced between each command correctly.
So any help about the above would be great!
UPDATE FOR #RavinderSingh13:
code used:
echo "------"
echo "--url=$URL \
--username=$UN \
--password=$PW \
--referenceUrl=$RURL \
--referenceUsername=$RUN \
--referencePassword=$RPW \
--changeSetAuthor=TD \
--diffTypes=tables, views, columns, indexes, foreignkeys, primarykeys, uniqueconstraints \
$LIQUI_ACTION"
echo "------"

Insert bcrypt hash into specific line in file using bash

I need to update an application configuration by inserting bcrypt hash into file config.yml at specific lines.
echo -e "$Enter password for user1"
read -p ": " user1_pass
echo -e "$Enter password for user2"
read -p ": " user2_pass
user1_hash=$(htpasswd -bnBC 10 "" $user1_pass | tr -d ':\n')
user2_hash=$(htpasswd -bnBC 10 "" $user2_pass | tr -d ':\n')
$user1_hash should be placed inline 2 and $user2_hash inline 7.
config.yml
user1:
hash: "$2y$12$shEKzuVfogdZFbbraSqhwOOh96hfxe1NzLQbpmHJvgDUeRfRrkf3a"
reserved: "true"
roles: "user"
user2:
hash: "$2y$12$Fkc5GAp9Za5caIfHjBgNQ.jNEss0SJfCLTlm9EhAcjzPVy.kLriBa"
reserved: "true"
roles: "user"
What is the best approach to do that using bash?
You can edit the file using Ruby :
#!/usr/bin/ruby
require 'yaml'
obj = YAML.load_file('config.yml')
bcrypt_hash='$2y$12$shEKzuVfogdZFbbraSqhwOOh96hfxe1NzLQbpmHJvgDUeRfRrkf3a'
obj['user1']['hash'] = bcrypt_hash
obj['user2']['hash'] = bcrypt_hash
puts YAML.dump(obj)
This will output :
... More yml content
user1:
hash: "$2y$12$shEKzuVfogdZFbbraSqhwOOh96hfxe1NzLQbpmHJvgDUeRfRrkf3a"
user2:
hash: "$2y$12$shEKzuVfogdZFbbraSqhwOOh96hfxe1NzLQbpmHJvgDUeRfRrkf3a"
... More yml content
Hope it helps!
Use the keyword echo to write into the file
echo -e "$Enter password for user1"
read -p ": " user1_pass
echo -e "$Enter password for user2"
read -p ": " user2_pass
user1_hash=$(htpasswd -bnBC 10 "" $user1_pass | tr -d ':\n')
user2_hash=$(htpasswd -bnBC 10 "" $user2_pass | tr -d ':\n')
echo "user1:" > config.yml
echo " hash: " + $user1_hash >> config.yml
echo "user2:" >> config.yml
echo " hash: " + $user2_hash >> config.yml
the contents of config.yml file being
user1:
hash: + $2y$10$7S0fC4wTqAfm9ytJ5BquC.3KITsqLoqPXHyj3mzgXdvw10TRIybni
user2:
hash: + $2y$10$jcYMrOsdIzwR3AlSONNfCuc.B5AoGVV4i31KSsx0PLlpn17issJfe

How to send a mail with special characters as subject by using sendmail command in unix shell script

In my script i want to send a mail with some subject, some text as body along with csv file as attachement
My problem is subject has special characters in portuguese language
like this
Subject: Relatório de utilização do QRCODE
i am using sendmail command to send mail because i need to change sender name(not email id)
I tried this :
Subject=Relatório de utilização do QRCODE
mnth=$(date '+%m/%Y' --date="1 month ago")
echo 'mês:'$mnth>>mailBody.html
echo 'contagem de registros:'11090>>mailBody.html
cat mailBody.html>out.mail
echo "$mnth"
uuencode QR_Log.csv QR_Report_$fname.csv >> out.mail
sendmail -F "xyzname" "$subject" -f "receiver#abc.com" <out.mail
echo "mail sent"
when i run the above script i am getting message like this :
Syntax error in mailbox address "Relat??rio.de.utiliza????o.do.QRCODE"
(non-printable character)
mail sent
How can i achieve this please Help me...
Help is very much appreciated. I'll just wait
Thanks in advance
I wrote a shell script like this and I got a valid title. Try to rewrite the code for sending mail like MIME:
#!/bin/bash
echo 'To: you#domain.net'>>test.html
echo 'From: Some User <user#domain.net>'>>test.html
echo 'Subject: Relatório de utilização do QRCODE'>>test.html
echo 'MIME-Version: 1.0'>>test.html
echo 'Content-Type: text/html; charset="utf-8"'>>test.html
echo 'Content-Disposition: inline'>>test.html
echo ''>>test.html
echo '<span style="color:red;">Your message goes here</span>'>>test.html
sendmail -i -t < test.html
rm test.html
Let me know if this helped :)
Below is my old answer...
Not a linux guy but this may help you. First you must encode the subject to base64. For example:
echo 'your subject' | openssl base64
Let's say you've put the encoded string into $subject variable. Next you set the subject like this when sending email:
"=?UTF-8?B?$subject?="
Basically try to put =?UTF-8?B? before the base64-encoded subject and ?= after it without spaces.
As I said I'm not too much of a linux guy but you'll manage :)
Let me know if it helped.
rfc2045 - (5) (Soft Line Breaks) The Quoted-Printable encoding REQUIRES that encoded lines be no more than 76 characters long.
For bash shell script code:
#!/bin/bash
subject_encoder(){
echo -n "$1" | xxd -ps -c3 |awk -Wposix 'BEGIN{
BASE64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
printf " =?UTF-8?B?"; bli=8
}
function encodeblock (strin){
b1=sprintf("%d","0x" substr(strin,1,2))
b2=sprintf("%d","0x" substr(strin,3,2))
b3=sprintf("%d","0x" substr(strin,5,2))
o=substr(BASE64,b1/4 + 1,1) substr(BASE64,(b1%4)*16 + b2/16 + 1,1)
len=length(strin)
if(len>1) o=o substr(BASE64,(b2%16)*4 + b3/64 + 1,1); else o=o"="
if(len>2) o=o substr(BASE64,b3%64 +1 ,1); else o=o"="
return o
}{
bs=encodeblock($0)
bl=length(bs)
if((bl+bli)>64){
printf "?=\n =?UTF-8?B?"
bli=bl
}
printf bs
bli+=bl
}END{
printf "?=\n"
}'
}
SUBJECT="Relatório de utilização"
SUBJECT=`subject_encoder "${SUBJECT}"`
echo '<html>test</html>'| mail -a "Subject:${SUBJECT}" -a "MIME-Version: 1.0" -a "Content-Type: text/html; charset=UTF-8" you#domain.net

Resources