I trying to get a value from a command into a var
and then print it our using printf.
Problem: i got the value in the var but with printf it does not appear
or is cut off.
INFO: In my script im calling redis-cli info memory
and to check whats wrong i tried a call on vmstat -s.
Working vmstat test:
format="%-16s | %-16s"
container_name="some_name"
used_memory=$(vmstat -s | sed -n "s/^\(.*\) K used memory.*$/\1/p")
row=$(printf "${format}" "${container_name}" "${used_memory}")
echo "${row}"
Output: some_name | 11841548
The actual script that is not working:
format="%-50s | %-16s"
container_name="totally_secret_container_name_1"
used_memory=$(docker exec -it "${container_name}" redis-cli info memory | sed -n "s/^used_memory_human:\(.*\)$/\1/p")
row=$(printf "${format}" "${container_name}" "${used_memory}")
echo "${row}"
Output: ecret_container_name_1 | 1.08M
Weird is than when i set the format to format="%-50s | %-1s"
then it works - the container name (left value) gets printed correctly.
What happen here?
How can i fix this?
Thanks for your time!
You need to remove the \r characters in the output that are causing it to go back to the beginning of the line and overwrite.
used_memory=$(docker exec -it "${container_name}" redis-cli info memory | sed -n "s/^used_memory_human:\(.*\)$/\1/p")
used_memory=${used_memory//$'\r'/}
row=$(printf "${format}" "${container_name}" "${used_memory}")
This uses the bash ${variable//old/new} replacement operator.
Related
i want to write a new script but i'm absolutely n00b in bash scripting
for example i want that script do kubectl get cs ( check cluster health ) and stdout of kubectl get cs is here:
administrator#rgv:~$ kubectl get cs
NAME STATUS MESSAGE ERROR
etcd-0 Healthy {"health": "true"}
scheduler Healthy ok
controller-manager Healthy ok
How I'm to parse the STATUS column?
If three nodes are the in cluster and the cluster is healthy I want to print just "cluster healthy.". If one node is down print "cluster is to die."
You could use a tool called awk.
kubectl get cs | awk 'NR>1 && $2=="Healthy"{count++}
END{if(count==3){print "All healthy"}
else{
print "Cluster(s) dead"
}
}'
Simplistic approach:
if kubectl get cs | tail -n +2 | grep -vq Healthy ; then
echo "It's dead, Jim"
else
echo "All is fine in the realm!"
fi
tail -n +2 skips the first line, and the grep looks for lines that don't contain the string Healthy. That is, even one line without it will trigger the warning.
This, of course ignores the columns completely, so it will fail if a dead member can contain that same string elsewhere in the line.
In your example the STATUS field has a Healthy status.
If you want to parse it as to find out which line contains the string you can use grep:
For your example it would be: kubectl get cs | tr -s ' ' | cut -f2 -d ' ' | grep Healthy which would show you all the lines containing a healthy string in the second column which is STATUS.
To check if cluster is down you can use: kubectl get cs | tr -s ' ' | cut -f2 -d ' ' | grep Down.
Note: tr -s ' ' squeezes the spaces and then you pipe-it to cut it with cut -f2 -d ' ' which will give you whole the column. There is space between the two single quotes ('') thank you for noticing #ilkkachu.
If you want to use it in a script you can do create a file using:
vim cluster_check.sh
#!/bin/bash
if kubectl get cs | tr -s ' ' | cut -f2 -d ' ' | grep -q 'Healthy';
then
echo 'cluster healthy!'
else
echo 'cluster is to die!'
fi
Then exit vim with ESC :wq
Change the execution flag for chmod +x cluster_check.sh.
Then you can execute it via: ./cluster_check.sh
For the command:
cat <file_name>.asc | head -1
Output:
10/30/2006,19:41:58,1.4871,1,E
I wanted to append commas to this line to maintain field consistency.
So I tried various combinations like issuing,
Command : echo cat <file_name>.asc | head -1,,,
Output : ,,,30/2006,19:41:58,1.4871,1,E
Command : echo cat <file_name>.asc | head -1,
Output : ,0/30/2006,19:41:58,1.4871,1,E
Command : echo cat <file_name>.asc | head -1",,,"
Output : ,,,30/2006,19:41:58,1.4871,1,E
Command : chk=$(echo cat <file_name>.asc | head -1)
echo ${chk},,,
Output : ,,,30/2006,19:41:58,1.4871,1,E
But my expected output is very simple,
10/30/2006,19:41:58,1.4871,1,E,,,
Actually my logic is to open a file with CAT and do a while operation on each line to check the data and append comma wherever needed and write the output to another file.
If I understood correctly, you want do this conversion:
10/30/2006,19:41:58,1.4871,1,E --> 10/30/2006,19:41:58,1.4871,1,E,,,
If so, then you should use sed util:
cat <file_name>.asc | sed -e 's/$/,,,/' > output_file
echo `head -n 1 cat.asc`,,,
If you need this in a variable, it goes like this:
X=`head -n 1 cat.asc`,,,
This answers the question which you asked. However, you also mention that you want to process a whole file and do some adjustment for each line. In this case, I don't see how this approach will help....
I've trace log generated by strace command like on running PHP by:
sudo strace -e sendto -fp $(pgrep -n php) -o strace.log
And the output looks like:
11208 sendto(4, "set 29170397297_-cache-schema 85 0 127240\r\n\257\202\v\0?\0\0\0\2\27\10stdClass\24\7\21\3cid\21\6schema\21\4d\37ata\25\n\247\21\5block\24\6\21\6fields\24\f\21\3bid\24\2\5\21\4type 0\37erial\21\10not null\5\21\6module\24\4\16\7\21\7va\37rchar\21\6length\6#\16\t\5\21\7default\r\21\5de\2lta#\5\16\v\16\f\6 \35\7\16\r\21\0010\21\5t \207C\30#6\2\16\r\r n\4tatus#0\4\21\3int/\7\6\0\21\4size \222\finy\21\6weight\24\3 ;\0\22\300 \6\6region#8\340\5P\5custom\27\300,\17\16\23\16\24\21\nvisibility\340\t\34\7\5pages\24\2 \205\3\4tex#\206 \261\1it \365\0\5\240\0\377y\10\r\21\ftransl!N\2ble %\1ca!a\340\3Q\0\1n\31\vprimary key\24\1\6\0\16\6\21\vunique#\21\ts\24\1\21\3tmd\24\3 \31\0\20 2\v\n\6\2\16\16\21\7index \210\10\1\21\4list\24\5\240\36\0\21 \36\10\26\6\3\16\25\6\4\16\n \1\6\4\21\4name \7\0\na\317\2_ro\252\0\5!$\0\n \3\341\2\23\0\16\340\0\16A\214\2\21\3r!\354# \v\22\21\10unsigned\5#\332\0\36\213\0\n \213\0\16 l\6%\16!\24\1\16%\271\0%#p\5\16#\16$\21\f\200l\241b#n\2\4\16\6M\2\10\16&#E\4\21\4bod\201_\5\32\16\t\4\16\23B\\\2g\16\34 \30\3info .\0\7a\255\0\200#q!L\5\6forma\201\332B/!d\2\4\16\37 y\0*y\0 \225a;\240\201\2'\21\van\0_\207\200\2\5\16\1\340\0U =#U\1\16\3#\222 \212\2lob#O\n\23\16)\21\6expire#\30\342\0\26\7\21\7create\241\17< \25\0\n\203\1\"\177\0dY\0\22 \305\5\5small\240!a\32\0.\230\0.\240\240\0\1\240\240\3,\21\vb S\2kpo\"\313\2s\24\6!\220\2\t\21\2\241q\0\10 ?\4\21\tno \213\6ort\5\21\fm\";\3ine_A\313\232\241\3\2\5\16#\340\4\16!\345\340\0U\223\340\0'AC\4sourc\202\202\340\3\27\0\v\200\27\0_C\326\340\0074\1\16\21_\240\363\2\1\16\25\340\3\16\r\0\21\vmultipliers\31\0- \223\1\21\t\341\0\30B-\0\1!\10\0003a\253\0005\v\0005ac \327Dz\"\364 \20\0\10 \6\0 #\333\r\0165\16\36\0163\21\nidenti$x\nr\0166\21\vadmin_ce\10\21\5label\21\f\244H\6 hook\21\23\240\r\0_\340\1\375\fs\21\3api\24\4\21\5own F\0062\16C\16B\21\17 H\5imum_v \260$\25\7\6\1\21\17curr m\340\1\22!\242\0002\"\305\0022\21\20\340\1N\5_groupa\247\2\6\0163\352\0\10 \352\2\0164\5 \325C%\341\0P\341\5\220\1\0162aQA\26\4\16:\5\21\17\201\321\1 c\"$\5back\21#\340\7b\0_\200!\340\3\311\1\16\7C\340\0a!\312\1\no \300#\240!&}\241\237\0\0\242e\341\4n\5\16;\24\10\16< \7\2=\21\35\340\1m\0\320\0 \342\3XAz\v\16>\16G\16?\16#\16A\21\30\341\tT\201\5\1\21\22\200\243\0 B0\6 string#o\4toolsbD\1\16C \260\0D!D\4C\16L\16E!P\0F \3\201T\16G\21\21ckeditor_set%\266\0gE\323\0\5%Q\0# 4#\345!)\"w#\372\1\21\10\340\0!\0\1 \31\0\32\240\334\4#\16\n\21\10\300D \r\2O\21\25\300\r\6_input_\244+\340\16V\1\16+ \31\340\4h X\0\2!;\0# \245\0+ \247\0Q T\7R\21\26comme#/\0_%\266\2cko W\3pane ;\4\5\24\10\21\7#\v\0_\243\257\301\231\1\21\4F\35 !\340\1\22F\323\0021\21\10\"\311'B\0e#\223A\254&f`\346\"~\6\vcollap&q%\227\340\6\35\2\0\21\t\240\35\344\1a\3009\0\0#\212\300.\0001\200L$\247\1enFl\344\0\216\300,\0\1G\5\3view\340\0002\300\177 \372\0\1 K\0T!"..., 8196, MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 8196
It sounds like these are represented by ordinary C escape codes.
I've tried to decode them in shell by printf like:
while read line; do printf "%s" "$line"; done < <(cat strace.log | head -n2)
but it failed (looks like it doesn't make any sense):
11208 sendto(4, "set 29170397297_-cache-schema 85 0 127240rn257202v0?00022710stdClass247213cid216schema214d37ata25n247215block246216fields24f213bid2425214type 037erial2110not null5216module244167217va37rchar216length6#16t5217defaultr215de2lta#516v16f6 35716r210010215t 207C30#6216rr n4tatus#04213int/760214size 222finy216weight243 ;022300 66region#83405P5custom27300,171623162421nvisibility340t3475pages242 20534tex#206 2611it 365052400377y10r21ftransl!N2ble %1ca!a3403Q01n31vprimary key2416016621vunique#21ts241213tmd243 31020 2vn621616217index 210101214list24524036021 3610266316256416n 164214name 70na3172_ro25205!$0n 3341223016340016A2142213r!354# v222110unsigned5#3320362130n 213016 l6%16!24116%2710%#p516#16$21f200l241b#n24166M21016&#E4214bod201_53216t41623B\2g1634 303info .07a2550200#q!L56forma201332B/!d241637 y0*y0 225a;2402012'21van0_207200251613400U =#U1163#222 2122lob#On2316)216expire#303420267217create24117< 250n2031"1770dY022 30555small240!a320.`2300.240240012402403,21vb S2kpo"3132s246!2202t212241q010...
Is there any better way to parse the output of strace command to see plain strings passed to recvfrom/sendto?
Ideally it is possible to print printable characters including new lines (\r\n), but cut-off NULLs and other non-printable characters?
The problem why read doesn't work, because shell is already escaping the characters, so the string is doubled escaped, therefore \r\n is printed as rn.
To ignore escaping of characters by shell, you can use read -r which allow backslashes to escape any characters (so they're treated literally). Here is example:
while read -r line; do printf "%b\n" "$line"; done < strace.log | strings
Since it's a binary data, above example also includes strings command to display only printable strings.
Strace also support printing all strings in hex when -x is specified, but it'll work the same.
Here is the version to parse strace output in real-time:
while read -r line;
do printf "%b\n" "$line" | strings
done < <(sudo strace -e recvfrom,sendto -s 1000 -fp $(pgrep -n php) 2>/dev/stdout)
Further more strings, can be replaced by more specific filter using grep, to get only what is inside double quotes:
grep -o '".\+[^"]"' | grep -o '[^"]\+[^"]'
however this may still print binary formats.
To avoid that, lets simplify the whole process, so lets define the following formatter alias:
alias format-strace='grep --line-buffered -o '\''".\+[^"]"'\'' | grep --line-buffered -o '\''[^"]*[^"]'\'' | while read -r line; do printf "%b" $line; done | tr "\r\n" "\275\276" | tr -d "[:cntrl:]" | tr "\275\276" "\r\n"'
where:
grep -o '".\+[^"]"' - select double-quoted string with quotes
grep -o '[^"]*[^"]' - select text within the double quotes
while read -r line - store each line into $line and do some action (help read)
printf "%b" $line - print line by expanding backslash escape sequences
tr "\r\n" "\275\276" - temporarily replace \r\n into \275\276
tr -d "[:cntrl:]" - remove all control characters
tr "\275\276" "\r\n" - restore new line endings
then the complete example to trace some command (e.g. php) can look like:
strace -e trace=read,write,recvfrom,sendto -s 1000 -fp $(pgrep -n php) 2>&1 | format-strace
Check for similar example: How to view the output of a running process in another bash session? at Unix.SE
I am going though some growing pains with Unix. My question:
I want to be able to print all my user defined variables in my shell. Let say I do the following in the shell:
$ x=9
$ y="Help"
$ z=-18
$ R="My 4th variable"
How would I go about printing:
x y z R
You should record your variables first at runtime with set, then compare it later to see which variables were added. Example:
#!/bin/bash
set | grep -E '^[^[:space:]]+=' | cut -f 1 -d = | sort > /tmp/previous.txt
a=1234
b=1234
set | grep -E '^[^[:space:]]+=' | cut -f 1 -d = | sort > /tmp/now.txt
comm -13 /tmp/previous.txt /tmp/now.txt
Output:
a
b
PIPESTATUS
Notice that there are still other variables produced by the shell but is not declared by the user. You can filter them with grep -v. It depends on the shell as well.
Add: Grep and cut could simply be just one sed a well: sed -n 's/^\([^[:space:]]\+\)=.*/\1/p'
Type set:
$ set
Apple_PubSub_Socket_Render=/tmp/launch-jiNTOC/Render
BASH=/bin/bash
BASH_ARGC=()
BASH_ARGV=()
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="3" [1]="2" [2]="51" [3]="1" [4]="release" [5]="x86_64-apple-darwin13")
BASH_VERSION='3.2.51(1)-release'
COCOS2DROOT=/Users/andy/Source/cocos2d
COLUMNS=80
DIRSTACK=()
...
(Oh, and BTW, you appear to have your variable syntax incorrect as you assign, say, A but print $A)
If variables are exported then you can use env command in Unix.
I have the following log file which contains lines like this
1345447800561|FINE|blah#13|txReq
1345447800561|FINE|blah#13|Req
1345447800561|FINE|blah#13|rxReq
1345447800561|FINE|blah#14|txReq
1345447800561|FINE|blah#15|Req
I am trying extract the first field from each line and depending on whether it belongs to blah#13 or blah#14, blah#15 i am creating the corresponding files using the following script, which seems quite in-efficient in terms of the number of temp files creates. Any suggestions on how I can optimize it ?
cat newLog | grep -i "org.arl.unet.maca.blah#13" >> maca13
cat newLog | grep -i "org.arl.unet.maca.blah#14" >> maca14
cat newLog | grep -i "org.arl.unet.maca.blah#15" >> maca15
cat maca10 | grep -i "txReq" >> maca10TxFrameNtf_temp
exec<blah10TxFrameNtf_temp
while read line
do
echo $line | cut -d '|' -f 1 >>maca10TxFrameNtf
done
cat maca10 | grep -i "Req" >> maca10RxFrameNtf_temp
while read line
do
echo $line | cut -d '|' -f 1 >>maca10TxFrameNtf
done
rm -rf *_temp
Something like this ?
for m in org.arl.unet.maca.blah#13 org.arl.unet.maca.blah#14 org.arl.unet.maca.blah#15
do
grep -i "$m" newLog | grep "txReq" | cut -d' ' -f1 > log.$m
done
I've found it useful at times to use ex instead of grep/sed to modify text files in place without using temps ... saves the trouble of worrying about uniqueness and writability to the temp file and its directory etc. Plus it just seemed cleaner.
In ksh I would use a code block with the edit commands and just pipe that into ex ...
{
# Any edit command that would work at the colon prompt of a vi editor will work
# This one was just a text substitution that would replace all contents of the line
# at line number ${NUMBER} with the word DATABASE ... which strangely enough was
# necessary at one time lol
# The wq is the "write/quit" command as you would enter it at the vi colon prompt
# which are essentially ex commands.
print "${NUMBER}s/.*/DATABASE/"
print "wq"
} | ex filename > /dev/null 2>&1