Using sed instead of rev - shell

I am running a shell build step in Jenkins (Windows), but, it doesn't seem to recognize the rev command. The command I am trying to run is:
awk '/Connections/||/Endpoint/||/connections/||/endpoint/||/EndPointURI/ {print $config}' "$config" | rev | cut -c 8- | rev >temp11
I have used sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' to replace each instance of rev.
I have the following:
awk '/Connections/||/Endpoint/||/connections/||/endpoint/||/EndPointURI/ {print $config}' "$config" | sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' | cut -c 8- | sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' >temp11
Using this method, I get the following error in the Jenkins output:
+ awk '/Connections/||/Endpoint/||/connections/||/endpoint/||/EndPointURI/ {print $config}' 'C:\Program Files (x86)\Jenkins/workspace/DeploymentFiles/BUILD.xml'
+ sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//'
+ cut -c 8-
sed: -e expression #1, char 9: unknown command: `\'
+ sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//'
sed: -e expression #1, char 9: unknown command: `\'
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Not sure how to fix this or make it work.
Full script:
touch "${JENKINS_HOME}"/workspace/DeploymentFiles/UAT.properties
config=${JENKINS_HOME}/workspace/DeploymentFiles/BUILD.xml
properties=${JENKINS_HOME}/workspace/DeploymentFiles/UAT.properties
a='tibco.clientVar.'
awk '/Connections/||/Endpoint/||/connections/||/endpoint/||/EndPointURI/ {print $config}' "$config" | sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' | cut -c 8- | sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' >temp11
awk '/Connections/||/Endpoint/||/connections/||/endpoint/||/EndPointURI/ {getline; print $config}' "$config" |sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' | cut -c 9- | sed '/\n!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' >temp22
sed -e s/'<name>'//g -i temp11
sed -i '/^$/d' temp11
sed -e s/'<value>'//g -i temp22
sed -i '/^$/d' temp22
while read -r line
do
echo "${a}$line"
done <temp11 >temp3
sed -i '/^$/d' temp3
paste -d'=' temp3 temp22 > final
sed '/^tibco.clientVar.<checkpoint>/d' final > final.new && mv final.new final
mv final ${properties}

Related

Why does my awk redirection not work?

Im trying to redirect my output to replace the contents of my file but if I do this it doesn't change my output at all
#!/bin/bash
ssh_config_path="$HOME/.ssh/config"
temp_ssh_config_path="$HOME/.ssh/config_temporary"
new_primary_username=$1
curr_primary_username=`awk '/^Host github\.com$/,/#Username/{print $2}' $ssh_config_path | tail -1`
new_user_name=`awk "/^Host github-$new_primary_username$/,/#Name/{print $2}" $ssh_config_path | tail -1 | sed 's/#Name //' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'`
new_user_email=`awk "/^Host github-$new_primary_username$/,/#Email/{print $2}" $ssh_config_path | tail -1 | sed 's/#Email //' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'`
echo "Switching from $curr_primary_username to $new_primary_username"
echo "Setting name to $new_user_name"
echo "Setting email to $new_user_email"
awk "
!x{x=sub(/github-$new_primary_username/,\"github.com\")}
!y{y=sub(/github\.com/,\"github-$curr_primary_username\")}
1" $ssh_config_path > temp_ssh_config_path && mv temp_ssh_config_path ssh_config_path
but if I do this I get the correct output on my terminal screen
#!/bin/bash
ssh_config_path="$HOME/.ssh/config"
temp_ssh_config_path="$HOME/.ssh/config_temporary"
new_primary_username=$1
curr_primary_username=`awk '/^Host github\.com$/,/#Username/{print $2}' $ssh_config_path | tail -1`
new_user_name=`awk "/^Host github-$new_primary_username$/,/#Name/{print $2}" $ssh_config_path | tail -1 | sed 's/#Name //' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'`
new_user_email=`awk "/^Host github-$new_primary_username$/,/#Email/{print $2}" $ssh_config_path | tail -1 | sed 's/#Email //' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//'`
echo "Switching from $curr_primary_username to $new_primary_username"
echo "Setting name to $new_user_name"
echo "Setting email to $new_user_email"
awk "
!x{x=sub(/github-$new_primary_username/,\"github.com\")}
!y{y=sub(/github\.com/,\"github-$curr_primary_username\")}
1" $ssh_config_path
It's disappointing how far you've veered from the answers you were given but in any case here's the correct syntax for your script (untested since you didn't provide any sample input/output):
#!/bin/bash
ssh_config_path="$HOME/.ssh/config"
temp_ssh_config_path="$HOME/.ssh/config_temporary"
new_primary_username="$1"
curr_primary_username=$(awk 'f&&/#Username/{print $2; exit} /^Host github\.com$/{f=1}' "$ssh_config_path")
new_user_name=$(awk -v npu="$new_primary_username" 'f&&/#Name/{print $2; exit} $0~"^Host github-"npu"$"{f=1}' "$ssh_config_path")
new_user_email=$(awk -v npu="$new_primary_username" 'f&&/#Email/{print $2; exit} $0~"^Host github-"npu"$"{f=1}' "$ssh_config_path")
echo "Switching from $curr_primary_username to $new_primary_username"
echo "Setting name to $new_user_name"
echo "Setting email to $new_user_email"
awk -v npu="$new_primary_username" -v cpu="$curr_primary_username" '
!x{x=sub("github-"npu,"github.com")}
!y{y=sub(/github\.com/,"github-"cpu)}
1' "$ssh_config_path" > temp_ssh_config_path && mv temp_ssh_config_path "$ssh_config_path"
By doing that I noticed that your last statement was:
mv temp_ssh_config_path ssh_config_path
when you probably meant:
mv temp_ssh_config_path "$ssh_config_path"
and that would have caused a problem with your expected output file being empty.
The whole thing should, of course, have been written as just 1 simple awk script.

AWK variable input

I've got the following bash code:
md5sum -c checksum.md5 2>&1 | grep FAILED | awk '{print $1}' | sed 's/:$// > /tmp/check.tmp
awk '{system("wget http://example.com/"$1"")}' /tmp/check.tmp
How can I use awk without a temp file?
Something like
files=`md5sum -c checksum.md5 2>&1 | grep FAILED | awk '{print $1}' | sed 's/:$//`
awk '{system("wget http://example.com/"$1"")}' $files
You can simplify the whole command to this:
md5sum -c checksum.md5 2>&1 |\
awk -F'[:/]' '/FAILED/{system("wget http://example.com/"$(NF-1))}'
wget has a switch -i that can come in handy:
md5sum -c checksum.md5 2>&1 | \
sed -n '/FAILED$/ { s/: FAILED$//; s!^!http://example.com/!; p; }' | \
wget -i
Like this:
awk '{system("wget http://example.com/"$1"")}' <<< $files

Grep Spellchecker

I am trying to write a simple shell script that takes a text file as input and checks all non-punctuated words against a dictionary (english.txt). It should return all non-matching (misspelled) words. I am using grep but it does not seem to successfully match all the lines in english.txt. I have included my code below.
#!/bin/bash
cat $1 |
tr ' \t' '\n\n' |
sed -e "/'/d" |
tr -d '[:punct:]' |
tr -cd '[:alpha:]\n' |
sed -e "/^$/d" |
grep -v -i -w -f english.txt

Bash script builds correct $cmd but fails to execute complex stream

This short script scrapes some log files daily to create a simple extract. It works from the command line and when I echo the $cmd and copy/paste, it also works. But it will breaks when I try to execute from the script itself.
I know this is a nightmare of patterns that I could probably improve, but am I missing something simple to just execute this correctly?
#!/bin/bash
priorday=$(date --date yesterday +"%Y-%m-%d")
outputfile="/home/CCHCS/da14/$priorday""_PROD_message_processing_times.txt"
cmd="grep 'Processed inbound' /home/rules/care/logs/RootLog* | cut -f5,6,12,16,18 -d\" \" | grep '^"$priorday"' | sed 's/\,/\./' | sed 's/ /\t/g' | sed -r 's/([0-9]+\-[0-9]+\-[0-9]+)\t/\1 /' | sed 's/ / /g' | sort >$outputfile"
printf "command to execute:\n"
echo $cmd
printf "\n"
$cmd
ouput:
./make_log_extract.sh command to execute: grep 'Processed inbound' /home/rules/care/logs/RootLog.log /home/rules/care/logs/RootLog.log.1
/home/rules/care/logs/RootLog.log.10
/home/rules/care/logs/RootLog.log.11
/home/rules/care/logs/RootLog.log.12
/home/rules/care/logs/RootLog.log.2
/home/rules/care/logs/RootLog.log.3
/home/rules/care/logs/RootLog.log.4
/home/rules/care/logs/RootLog.log.5
/home/rules/care/logs/RootLog.log.6
/home/rules/care/logs/RootLog.log.7
/home/rules/care/logs/RootLog.log.8
/home/rules/care/logs/RootLog.log.9 | cut -f5,6,12,16,18 -d" " | grep
'^2014-01-30' | sed 's/\,/./' | sed 's/ /\t/g' | sed -r
's/([0-9]+-[0-9]+-[0-9]+)\t/\1 /' | sed 's/ / /g' | sort
/home/CCHCS/da14/2014-01-30_PROD_message_processing_times.txt
grep: 5,6,12,16,18: No such file or directory
As grebneke comments, do not store the command and then execute it.
What you can do is to execute it but firstly print it: Bash: Print each command before executing?
priorday=$(date --date yesterday +"%Y-%m-%d")
outputfile="/home/CCHCS/da14/$priorday""_PROD_message_processing_times.txt"
set -o xtrace # <-- set printing mode "on"
grep 'Processed inbound' /home/rules/care/logs/RootLog* | cut -f5,6,12,16,18 -d\" \" | grep '^"$priorday"' | sed 's/\,/\./' | sed 's/ /\t/g' | sed -r 's/([0-9]+\-[0-9]+\-[0-9]+)\t/\1 /' | sed 's/ / /g' | sort >$outputfile"
set +o xtrace # <-- revert to normal

Python - how to fix this line typeerror: not enough arguments for format string

From command line its working but when i include in the Python getting this
Terminal:
$ amixer -c2 | grep "Simple mixer control 'Mic',0" -A 5 | grep "Mono: " | sed -e 's/Capture /\n/g' | tail -1 | awk '{print $2}' | sed -e 's/%]//g' | sed -e 's/\[//g'
88
Python:
tmp = "2"
a = """amixer -c%s | grep "Simple mixer control 'Mic',0" -A 5 | grep "Mono: " | sed -e 's/Capture /\n/g' | tail -1 | awk '{print $2}' | sed -e 's/%]//g' | sed -e 's/\[//g'""" % tmp
print "Reply " + a
a = os.popen(a).read()
print a
Error:
Running: /var/tmp/p/test.py (Sun Dec 8 20:58:07 2013)
Traceback (most recent call last):
File "/var/tmp/p/test.py", line 2, in <module>
a = """amixer -c%s | grep "Simple mixer control 'Mic',0" -A 5 | grep "Mono: " | sed -e 's/Capture /\n/g' | tail -1 | awk '{print $2}' | sed -e 's/%]//g' | sed -e 's/\[//g'""" % tmp
TypeError: not enough arguments for format string
Execution Successful!
You see that thing in one of the sed:
... | sed -e 's/%]//g' | ...
Change that bit to this:
... | sed -e 's/%%]//g' | ...
This is how you escape a % in strings in python.

Resources