Inside C:\cygwin64\home\Administrator I have 3 files
script.sh
links_da_scaricare.txt
cookies.txt
To download from dplay I use successfully if I use this command, videos are downloaded
yt-dlp --referer "https:www.discoveryplus.com/" --add-header "Cookie:gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA#AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect" --batch-file links_da_scaricare.txt
ok, I try to download using Cygwin and I use this command
bash -x script.sh
But I get this error
$ bash -x script.sh
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno could not be read
+ IFS=
+ read -r url
++ echo https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
++ sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1'
sed: -e expression #1, char 52: unterminated `s' command
+ folder_name=
+ echo 'folder_name: '
folder_name:
+ mkdir -p /cygdrive/c/cygwin64/home/Administrator/
+ echo 'The script is running and creating folder: '
+ yt-dlp --referer https://www.discoveryplus.com/ --cookies /cygdrive/c/cygwin64/home/Administrator/cookies.txt -a https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas -o '/cygdrive/c/cygwin64/home/Administrator//%(title)s.%(ext)s'
ERROR: batch file https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas could not be read
+ IFS=
+ read -r url
script.sh have this code
#!/bin/bash
while IFS= read -r url; do
folder_name=$(echo "$url" | sed 'shttps://www.discoveryplus.com/it/video/\(.*\)/.*~\1')
echo "folder_name: $folder_name"
mkdir -p "/cygdrive/c/cygwin64/home/Administrator/$folder_name"
echo "The script is running and creating folder: $folder_name" > ~/script.log
yt-dlp --referer "https://www.discoveryplus.com/" --cookies "/cygdrive/c/cygwin64/home/Administrator/cookies.txt" -a "$url" -o "/cygdrive/c/cygwin64/home/Administrator/$folder_name/%(title)s.%(ext)s"
done < /cygdrive/c/cygwin64/home/Administrator/links_da_scaricare.txt
links_da_scaricare.txt have these 3 lines - I use notepad to copy paste these links
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello
cookies.txt is this - is the same that I use in powershell command so it should download because with powershell I can do it but with bash i have problems, I don't know why
gcl_au=1.1.192985240.1664694131; _scid=62382352-4832-438c-a74d-30699989ca96; _fbp=fb.1.1664694351012.468612256; __telemetric.v=1599968982.1664694367.81679; __zlcmid=1CFlGmqJqevtvtz; st=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJVU0VSSUQ6ZHBsYXk6MzEzNzY0NDYtOTkwMC00ODVlLWE3YjYtYTkyYzc2MDJkZWE4IiwianRpIjoidG9rZW4tNTA5ZTdiZWYtOTBmMC00ZmVjLWJlNWMtNzJhZmNiZThlYzZmIiwiYW5vbnltb3VzIjpmYWxzZSwiaWF0IjoxNjY0Njk0MzkzfQ.kH5QmwI2CH0-baBgJMGVoalTHbMcwOtH8Go_G2mEz6c; aam_uuid=50034166116354787000738543341305359891; _tt_enable_cookie=1; _ttp=45eb20c2-a55e-4740-be63-01b5e4166bd3; _hjSessionUser_2533509=eyJpZCI6IjM5NDAxNDA3LTllZDQtNWI1ZC1iYjY4LTRiNDlhOWIzMzZhOSIsImNyZWF0ZWQiOjE2NjQ2OTQzNjcxNDQsImV4aXN0aW5nIjp0cnVlfQ==; AMCV_9AE0F0145936E3790A495CAA#AdobeOrg=-1124106680|MCIDTS|19268|MCMID|50189272745869278580790010530934545485|MCAAMLH-1665300361|6|MCAAMB-1665300361|RKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y|MCOPTOUT-1664702762s|NONE|MCSYNCSOP|411-19275|vVersion|5.2.0; __telemetric.s=1.1664980917.1664982717; _hjSession_2533509=eyJpZCI6ImVhNmZmNTA3LTM1ZGYtNDBlZC1iMWFjLTA0MzRlYzZlYjJjNCIsImNyZWF0ZWQiOjE2NjQ5ODA5MTc5MTEsImluU2FtcGxlIjpmYWxzZX0=; _hjAbsoluteSessionInProgress=0; _clck=ji7lnf|1|f5g|0; fs_uid=#161CV1#6518423911501824:5976792460267520:::#/1696517470; _uetsid=2596f2d044bd11ed80c355f240f55627; _uetvid=c8269d50422011ed952c2140169cc91d; ABTastySession=mrasn=&sen=9&lp=https%3A%2F%2Fwww.discoveryplus.com%2Fit; ABTasty=uid=4hnc3e5beewkxmkz&fst=1664694364438&pst=1664813235740&cst=1664981468233&ns=4&pvt=21&pvis=10&th=; _clsk=6lrbo9|1664981584916|8|0|j.clarity.ms/collect
In other words script.sh should be do this
--> read urls from links_da_scaricare.txt and extract only these strings
citta-mortale
il-boss-delle-cerimonie
--> make folders with that names
--> download
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-1-strada-per-linferno
https://www.discoveryplus.com/it/video/citta-mortale/stagione-1-episodio-2-non-in-kansas
in this folder
citta-mortale
--> download
https://www.discoveryplus.com/it/video/il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello
--> in this folder
il-boss-delle-cerimonie
Script creates the first folder but doesn't download anything inside it
You are looking to download files into a directory named according to the 6th field delimited by "/" in the url specification (between slash #5 and #6).
That is accomplished as
folder_name=$( echo "${url}" | cut -f6 -d/ )
Regarding sed, you must specify a delimiter character which does not conflict with any possible characters that might be found in the text string that you are trying to edit.
For URLs that always have "/", you cannot use "/" as that delimiter. You must choose something else. I tend to use either the "+" or the "|" depending on circumstances.
So to strip the prefix from the URL string, you would use
echo "${url}" | sed 's+https://www.discoveryplus.com/it/video/++'
to obtain
citta-mortale/stagione-1-episodio-1-strada-per-linferno
citta-mortale/stagione-1-episodio-2-non-in-kansas
il-boss-delle-cerimonie/stagione-5-il-quarantesimo-compleanno-di-nello
You would make a second, similar, usage of sed to strip the tail end after the "/".
OR ... you can again use cut in this manner
id_string=$( echo "${url}" | cut -f7 -d/ )
to obtain
stagione-1-episodio-1-strada-per-linferno
stagione-1-episodio-2-non-in-kansas
stagione-5-il-quarantesimo-compleanno-di-nello
Related
I have a text file:
org.jitsi.videobridge.xmpp.user.shard-1.HOSTNAME=localhost
org.jitsi.videobridge.xmpp.user.shard-1.DOMAIN=auth.jc.name.com
org.jitsi.videobridge.xmpp.user.shard-1.USERNAME=name
org.jitsi.videobridge.xmpp.user.shard-1.PASSWORD=Hfr*7462
org.jitsi.videobridge.xmpp.user.shard-1.MUC_JIDS=JvbBredjoy#internal.auth.jc.name.com
org.jitsi.videobridge.xmpp.user.shard-1.MUC_NICKNAME=7896aee5-fgre-4b02-4569-0bcc75ed1d0d
I created a bash script:
#!/bin/bash
DPATH="/etc/jitsi/videobridge/sip-communicator.properties"
k=$(grep -o 'shard-1' $DPATH) # shard ends by a number#
i=$(grep -o 'shard-1' $DPATH | cut -c7)
m=$((i+1))
n="shard-$m"
sed -i "s|${k}|${n}|g" $DPATH
But I get error:
/home/scripts# ./shard_number
./shard_number: line 5: 1
1
1
1
1
1: syntax error in expression (error token is "1
1
1
1
1")
sed: -e expression #1, char 9: unterminated `s' command
Could you please help to solve this issue? Thank you.
If you call your script with bash -x /path/to/your/script or add set -x somewhere at the start of your script (after the #!shebang, but before the commands you want to debug), you will see that your grep commands return not a single 'shard-1' but rather one 'shard-1' per line :
++ grep -o shard-1 /etc/jitsi/videobridge/sip-communicator.properties
+ k='shard-1
shard-1
shard-1
shard-1
shard-1
shard-1'
Once cut, that gives the 1\n1\n1\n1\n1\n string that is mentionned in your error output as an invalid token for the $(( ... )) expression, which also breaks the syntax of your sed substitution :
++ cut -c7
++ grep -o shard-1 /etc/jitsi/videobridge/sip-communicator.properties
+ i='1
1
1
1
1
1'
Make that string a single number (for instance piping your grep into sort -u to unicize all the shards found) and your script will work just fine :
#!/bin/bash
DPATH="/etc/jitsi/videobridge/sip-communicator.properties"
k=$(grep -o 'shard-1' $DPATH | sort -u) # shard ends by a number#
i=$(grep -o 'shard-1' $DPATH | sort -u | cut -c7)
m=$((i+1))
n="shard-$m"
sed -i "s|${k}|${n}|g" $DPATH
You can try it here. Also check this test if you want to see your initial script debugged.
I've trace log generated by strace command like on running PHP by:
sudo strace -e sendto -fp $(pgrep -n php) -o strace.log
And the output looks like:
11208 sendto(4, "set 29170397297_-cache-schema 85 0 127240\r\n\257\202\v\0?\0\0\0\2\27\10stdClass\24\7\21\3cid\21\6schema\21\4d\37ata\25\n\247\21\5block\24\6\21\6fields\24\f\21\3bid\24\2\5\21\4type 0\37erial\21\10not null\5\21\6module\24\4\16\7\21\7va\37rchar\21\6length\6#\16\t\5\21\7default\r\21\5de\2lta#\5\16\v\16\f\6 \35\7\16\r\21\0010\21\5t \207C\30#6\2\16\r\r n\4tatus#0\4\21\3int/\7\6\0\21\4size \222\finy\21\6weight\24\3 ;\0\22\300 \6\6region#8\340\5P\5custom\27\300,\17\16\23\16\24\21\nvisibility\340\t\34\7\5pages\24\2 \205\3\4tex#\206 \261\1it \365\0\5\240\0\377y\10\r\21\ftransl!N\2ble %\1ca!a\340\3Q\0\1n\31\vprimary key\24\1\6\0\16\6\21\vunique#\21\ts\24\1\21\3tmd\24\3 \31\0\20 2\v\n\6\2\16\16\21\7index \210\10\1\21\4list\24\5\240\36\0\21 \36\10\26\6\3\16\25\6\4\16\n \1\6\4\21\4name \7\0\na\317\2_ro\252\0\5!$\0\n \3\341\2\23\0\16\340\0\16A\214\2\21\3r!\354# \v\22\21\10unsigned\5#\332\0\36\213\0\n \213\0\16 l\6%\16!\24\1\16%\271\0%#p\5\16#\16$\21\f\200l\241b#n\2\4\16\6M\2\10\16&#E\4\21\4bod\201_\5\32\16\t\4\16\23B\\\2g\16\34 \30\3info .\0\7a\255\0\200#q!L\5\6forma\201\332B/!d\2\4\16\37 y\0*y\0 \225a;\240\201\2'\21\van\0_\207\200\2\5\16\1\340\0U =#U\1\16\3#\222 \212\2lob#O\n\23\16)\21\6expire#\30\342\0\26\7\21\7create\241\17< \25\0\n\203\1\"\177\0dY\0\22 \305\5\5small\240!a\32\0.\230\0.\240\240\0\1\240\240\3,\21\vb S\2kpo\"\313\2s\24\6!\220\2\t\21\2\241q\0\10 ?\4\21\tno \213\6ort\5\21\fm\";\3ine_A\313\232\241\3\2\5\16#\340\4\16!\345\340\0U\223\340\0'AC\4sourc\202\202\340\3\27\0\v\200\27\0_C\326\340\0074\1\16\21_\240\363\2\1\16\25\340\3\16\r\0\21\vmultipliers\31\0- \223\1\21\t\341\0\30B-\0\1!\10\0003a\253\0005\v\0005ac \327Dz\"\364 \20\0\10 \6\0 #\333\r\0165\16\36\0163\21\nidenti$x\nr\0166\21\vadmin_ce\10\21\5label\21\f\244H\6 hook\21\23\240\r\0_\340\1\375\fs\21\3api\24\4\21\5own F\0062\16C\16B\21\17 H\5imum_v \260$\25\7\6\1\21\17curr m\340\1\22!\242\0002\"\305\0022\21\20\340\1N\5_groupa\247\2\6\0163\352\0\10 \352\2\0164\5 \325C%\341\0P\341\5\220\1\0162aQA\26\4\16:\5\21\17\201\321\1 c\"$\5back\21#\340\7b\0_\200!\340\3\311\1\16\7C\340\0a!\312\1\no \300#\240!&}\241\237\0\0\242e\341\4n\5\16;\24\10\16< \7\2=\21\35\340\1m\0\320\0 \342\3XAz\v\16>\16G\16?\16#\16A\21\30\341\tT\201\5\1\21\22\200\243\0 B0\6 string#o\4toolsbD\1\16C \260\0D!D\4C\16L\16E!P\0F \3\201T\16G\21\21ckeditor_set%\266\0gE\323\0\5%Q\0# 4#\345!)\"w#\372\1\21\10\340\0!\0\1 \31\0\32\240\334\4#\16\n\21\10\300D \r\2O\21\25\300\r\6_input_\244+\340\16V\1\16+ \31\340\4h X\0\2!;\0# \245\0+ \247\0Q T\7R\21\26comme#/\0_%\266\2cko W\3pane ;\4\5\24\10\21\7#\v\0_\243\257\301\231\1\21\4F\35 !\340\1\22F\323\0021\21\10\"\311'B\0e#\223A\254&f`\346\"~\6\vcollap&q%\227\340\6\35\2\0\21\t\240\35\344\1a\3009\0\0#\212\300.\0001\200L$\247\1enFl\344\0\216\300,\0\1G\5\3view\340\0002\300\177 \372\0\1 K\0T!"..., 8196, MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 8196
It sounds like these are represented by ordinary C escape codes.
I've tried to decode them in shell by printf like:
while read line; do printf "%s" "$line"; done < <(cat strace.log | head -n2)
but it failed (looks like it doesn't make any sense):
11208 sendto(4, "set 29170397297_-cache-schema 85 0 127240rn257202v0?00022710stdClass247213cid216schema214d37ata25n247215block246216fields24f213bid2425214type 037erial2110not null5216module244167217va37rchar216length6#16t5217defaultr215de2lta#516v16f6 35716r210010215t 207C30#6216rr n4tatus#04213int/760214size 222finy216weight243 ;022300 66region#83405P5custom27300,171623162421nvisibility340t3475pages242 20534tex#206 2611it 365052400377y10r21ftransl!N2ble %1ca!a3403Q01n31vprimary key2416016621vunique#21ts241213tmd243 31020 2vn621616217index 210101214list24524036021 3610266316256416n 164214name 70na3172_ro25205!$0n 3341223016340016A2142213r!354# v222110unsigned5#3320362130n 213016 l6%16!24116%2710%#p516#16$21f200l241b#n24166M21016&#E4214bod201_53216t41623B\2g1634 303info .07a2550200#q!L56forma201332B/!d241637 y0*y0 225a;2402012'21van0_207200251613400U =#U1163#222 2122lob#On2316)216expire#303420267217create24117< 250n2031"1770dY022 30555small240!a320.`2300.240240012402403,21vb S2kpo"3132s246!2202t212241q010...
Is there any better way to parse the output of strace command to see plain strings passed to recvfrom/sendto?
Ideally it is possible to print printable characters including new lines (\r\n), but cut-off NULLs and other non-printable characters?
The problem why read doesn't work, because shell is already escaping the characters, so the string is doubled escaped, therefore \r\n is printed as rn.
To ignore escaping of characters by shell, you can use read -r which allow backslashes to escape any characters (so they're treated literally). Here is example:
while read -r line; do printf "%b\n" "$line"; done < strace.log | strings
Since it's a binary data, above example also includes strings command to display only printable strings.
Strace also support printing all strings in hex when -x is specified, but it'll work the same.
Here is the version to parse strace output in real-time:
while read -r line;
do printf "%b\n" "$line" | strings
done < <(sudo strace -e recvfrom,sendto -s 1000 -fp $(pgrep -n php) 2>/dev/stdout)
Further more strings, can be replaced by more specific filter using grep, to get only what is inside double quotes:
grep -o '".\+[^"]"' | grep -o '[^"]\+[^"]'
however this may still print binary formats.
To avoid that, lets simplify the whole process, so lets define the following formatter alias:
alias format-strace='grep --line-buffered -o '\''".\+[^"]"'\'' | grep --line-buffered -o '\''[^"]*[^"]'\'' | while read -r line; do printf "%b" $line; done | tr "\r\n" "\275\276" | tr -d "[:cntrl:]" | tr "\275\276" "\r\n"'
where:
grep -o '".\+[^"]"' - select double-quoted string with quotes
grep -o '[^"]*[^"]' - select text within the double quotes
while read -r line - store each line into $line and do some action (help read)
printf "%b" $line - print line by expanding backslash escape sequences
tr "\r\n" "\275\276" - temporarily replace \r\n into \275\276
tr -d "[:cntrl:]" - remove all control characters
tr "\275\276" "\r\n" - restore new line endings
then the complete example to trace some command (e.g. php) can look like:
strace -e trace=read,write,recvfrom,sendto -s 1000 -fp $(pgrep -n php) 2>&1 | format-strace
Check for similar example: How to view the output of a running process in another bash session? at Unix.SE
In my directory I have plenty of *.yml files named like:
work-arran.yml
work-cap.yml
work-exposed.yml
work-humax.yml
work-instruc.yml
work-kiln.yml
work-lex.yml
work-merc.yml
and also directories with same names but using underscores instead of dashes:
work_cap
work_exposed
work_humax
work_instruc
work_kiln
work_lex
work_merc
I want to put yaml record grid_pool: dir_name_here into each respective
*.yml file automatically, where dir_name_here would be the name of each
file's respective directory.
Tried with this, and while it would work it appended desired line into
underscored version of *.yml file instead of exsisting dashed.
How to change that last fn.yml so it replaces again _ for -?
ls *.yml | sed 's/-/_/g' | sed 's/.yml//g' | xargs -n1 -I fn bash -c "echo ' grid_pool: fn' >> fn.yml"
Use parameter expansion with substitution:
for file in *.yml ; do
dirname=${file/-/_}
echo " grid_pool: ${dirname%.yml}" >> "$file"
done
You can use the same method with xargs but I fear it'll be slower, as it starts a new shell for each file:
... | xargs -I fn bash -c 's=fn; echo " grid_pool: fn" >> ${s/_/-}.yml'
Im trying to make a script that takes a .txt file containing lines
like:
davda103:David:Davidsson:800104-1234:TNCCC_1:TDDB46 TDDB80:
and then sort them etc. Thats just the background my problem lies here:
#!/bin/sh -x
cat $1 |
while read a
do
testsak = `echo $a | cut -f 1 -d :`; <---**
echo $testsak;
done
Where the arrow is, when I try to run this code I get some kind of weird error.
+ read a
+ cut -f+ echo 1 -d :davda103:David:Davidsson:800104-1234:TNCCC_1:TDDB46
TDDB80:
+ testsak = davda103
scriptTest.sh: testsak: Det går inte att hitta
+ echo
(I have my linux in swedish because school -.-) Anyways that error just says that it cant find... something. Any ideas what could be causing my problem?
You have extra spaces around the assignment operator, remove them:
testsak=`echo $a | cut -f 1 -d :`; <---**
The spaces around the equal sign
testsak = `echo $a | cut -f 1 -d :`; <---**
causes bash to interpret this as a command testak with arguments = and the result of the command substitution. Removing the spaces will fix the immediate error.
A much more efficient way to extract the value from a is to let read do it (and use input redirection instead of cat):
while IFS=: read testak the_rest; do
echo $testak
done < "$1"
I have the following log file which contains lines like this
1345447800561|FINE|blah#13|txReq
1345447800561|FINE|blah#13|Req
1345447800561|FINE|blah#13|rxReq
1345447800561|FINE|blah#14|txReq
1345447800561|FINE|blah#15|Req
I am trying extract the first field from each line and depending on whether it belongs to blah#13 or blah#14, blah#15 i am creating the corresponding files using the following script, which seems quite in-efficient in terms of the number of temp files creates. Any suggestions on how I can optimize it ?
cat newLog | grep -i "org.arl.unet.maca.blah#13" >> maca13
cat newLog | grep -i "org.arl.unet.maca.blah#14" >> maca14
cat newLog | grep -i "org.arl.unet.maca.blah#15" >> maca15
cat maca10 | grep -i "txReq" >> maca10TxFrameNtf_temp
exec<blah10TxFrameNtf_temp
while read line
do
echo $line | cut -d '|' -f 1 >>maca10TxFrameNtf
done
cat maca10 | grep -i "Req" >> maca10RxFrameNtf_temp
while read line
do
echo $line | cut -d '|' -f 1 >>maca10TxFrameNtf
done
rm -rf *_temp
Something like this ?
for m in org.arl.unet.maca.blah#13 org.arl.unet.maca.blah#14 org.arl.unet.maca.blah#15
do
grep -i "$m" newLog | grep "txReq" | cut -d' ' -f1 > log.$m
done
I've found it useful at times to use ex instead of grep/sed to modify text files in place without using temps ... saves the trouble of worrying about uniqueness and writability to the temp file and its directory etc. Plus it just seemed cleaner.
In ksh I would use a code block with the edit commands and just pipe that into ex ...
{
# Any edit command that would work at the colon prompt of a vi editor will work
# This one was just a text substitution that would replace all contents of the line
# at line number ${NUMBER} with the word DATABASE ... which strangely enough was
# necessary at one time lol
# The wq is the "write/quit" command as you would enter it at the vi colon prompt
# which are essentially ex commands.
print "${NUMBER}s/.*/DATABASE/"
print "wq"
} | ex filename > /dev/null 2>&1