Counting chars from right - shell script - shell

I have a temperature sensor connected to a *nix system and the typical output is something like:
pi#raspberrypi $ cat /sys/bus/w1/devices/28-00000202070c/w1_slave
c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12187
The result comes without any comma, but is assumed that is always coming with 3 digits after the comma, so in this example would be 12.187º.
I have implemented a filter that places a comma after the second char, and it works most of the time:
grep t= | awk '{print $10;}'| sed 's/t\=//g' | sed 's/\([0-9][0-9]\)\([0-9]\)/\1,\2/g'
However, during winter, temperature drops below 10º and my filter returns values like 95,32º (when it should be 9,532º).
Is there any way of counting characters from the right, so I could always count with the 3 digital characters (and avoiding this problem in temperatures below 10º)?
Thanks,
Joaoabs

awk can handle floating point operations:
awk '/t=/ { sub(/t=/,"",$NF); print $NF/1000}' /sys/bus/w1/devices/28-00000202070c/w1_slave

If I understand you correctly, then what you want to do is :
sed 's/\([0-9][0-9][0-9]\)$/,\1/g'
$ in a regex means 'the end' so this searches for 3 digits right at the end of a string and replaces them with comma+the found digits.
(Note: This should be the last part of your pipe, with the beginning unchanged.)

Using awk
awk -F= '/t=/ {print $NF/1000}' /sys/bus/w1/devices/28-00000202070c/w1_slave
12.187
Or store it to a variable:
temp=$(awk -F= '/t=/ {print $NF/1000}' /sys/bus/w1/devices/28-00000202070c/w1_slave)
echo "$temp"
12.187

don't do grep|awk|sed|sed..., try this:
...|awk -F't=' -v OFS='t=' 'NF>1{$2=sprintf("%'\''d",$2)}7'
test with your data:
kent$ echo "c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12187"|awk -F't=' -v OFS='t=' 'NF>1{$2=sprintf("%'\''d",$2)}7'
c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12,187
awk has the format pattern in 'printf' for your requirement. just use it.
if you just want to have the t= value:
awk -F't=' -v OFS='t=' 'NF>1{printf "%'\''d\n",$2}'
with same input:
kent$ echo "c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12187"|awk -F't=' -v OFS='t=' 'NF>1{printf "%'\''d\n",$2}'
12,187

Related

how to grep a hex data area

I have a hex file, I need to extract a range of it to a text file
From range:
To Range:
I need Output: AC:E4:B5:9A:53:1C
i tried many but it not really correct requirements, Output: Binary file filehex matches
grep "["'\x9f\x87\x6f\x11'"-"'\x9f\x87\x70\x11'"]" filehex > test.txt
hope someone can help me
Use -a to force the text interpretation of the input.
Use -o to only output the matching part.
The expression you used doesn't make much sense. It matches any characters in the set \x9, \x87, \x6f, and then the range \x11-\x9f, etc.
You are rather interested in something that starts with \x9\x87\x6f\x11 and ends in \x9f\x87\x70\x11, and there can be anything in between.
You can use cut to remove the leading and trailing 4 bytes.
grep -oa $'\x9f\x87\x6f\x11.*\x9f\x87\x70\x11' hexfile | cut -b5-21
If you know the length of the string will always be 17 bytes, you can use .\{17\} instead of .*.
Ok I've build randomly one binary $file
with your string at a location making hd command to split them.
Note: regarding k314159' comment, I use hd to produce hexdump output similarto CentOS's hexdump tool.
One shoot using sed:
hd $file |sed -e 'N;/ 9f \+\(|.*\n[0-9a-f]\+ \+\|\)87 \+\(|.*\n[0-9a-f]\+ \+\|\)6f \+\(|.*\n[0-9a-f]\+ \+\|\)11 /p;D;'
000161c0 96 7a b2 21 28 f1 b3 32 63 43 93 ff 50 a6 9f 87 |.z.!(..2cC..P...|
000161d0 6f 11 0d 7a a5 a9 81 9e 32 9d fb 71 27 6d 60 f2 |o..z....2..q'm`.|
0002c3a0
Explanation:
N merge next line in current buffer
\(|.*\n[0-9a-f]\+ \+\|\) match a | followed by anything and a newline (\n), then immediately an hexadecimal number and a space OR nothing.
p print current buffer (two lines)
D Delete upto newline in current buffer, keep last line for next sed loop.
The last hexadecimal 00028d2a correspond to the size of my binary $file:
printf "%x\n" $(stat -c %s $file)
Using bash + grep:
printf -v var "\x9f\x87\x6f\x11"
IFS=: read -r offset _ < <(grep -abo "$var" $file)
hd $file | sed -ne "$((offset/16-1)),+4p"
000161a0 b7 8f 4a 4d ed 89 6c 0b 25 f9 e7 c9 8c 99 6e 23 |..JM..l.%.....n#|
000161b0 3c ba 80 ec 2e 32 dd f3 a4 a2 09 bd 74 bf 66 11 |<....2......t.f.|
000161c0 96 7a b2 21 28 f1 b3 32 63 43 93 ff 50 a6 9f 87 |.z.!(..2cC..P...|
000161d0 6f 11 0d 7a a5 a9 81 9e 32 9d fb 71 27 6d 60 f2 |o..z....2..q'm`.|
000161e0 15 86 c2 bd 11 d0 08 90 c4 84 b9 80 04 4e 17 f1 |.............N..|
Where you could read your string:
000161c0 9f 87 | ..|
000161d0 6f 11 |o. |
For testing, I've built my test file by:
dd if=/vmlinuz bs=90574 count=1 of=/tmp/testfile
printf '\x9f\x87\x6f\x11' >>/tmp/testfile
dd if=/vmlinuz bs=90574 count=1 >>/tmp/testfile
file=/tmp/testfile
Use grep to search for the original binary file, not the hex dump. Extending choroba's answer, I think you may have problems with grep trying to interpret your search pattern as UTF-8 or some other encoding. You should temporarily set the environment variable LC_ALL=C for grep to treat each byte individually. Also, you can use the -P option to enable use of lookbehind and lookahead in your pattern. So your command becomes:
LANG=C grep -oaP $'(?<=\x9f\x87\x6f\x11).*(?=\x9f\x87\x70\x11)' binary-file > test.txt
Proof that it works:
$ echo $'BEFORE\x9f\x87\x6f\x11AC:E4:B5:9A:53:1C\x9f\x87\x70\x11AFTER' | LANG=C grep -oaP $'(?<=\x9f\x87\x6f\x11).*(?=\x9f\x87\x70\x11)'
AC:E4:B5:9A:53:1C
$

Attempting to replace two character string with /dev/random hex

Example string
CA DA 00 17 11 38 88 C5 03
Desired output
AB 3C 6C 8F DA 88 24 78 6C
Commands attempted
$ tr -dc 0-9A-F < /dev/urandom filename ## prints too many chars
awk '{gsub(length($1)==2,{printf "%02")}}' filename ## syntax doesn't work, unsure how to add hex
$ sed 's/[a-z0-9]\{2\}//g' filename ## only replaces digits, unsure how to add hex as a replacement
I ended up using vim to do a partial conversion for some level of randomization.
:s/\d\d/AA/g
Can anyone provide a working solution?
It would be nice to see solutions (and explanations) leveraging tr/awk/sed for knowledge sharing purposes.
Thanks.
To replace each field with a random 2-digit hex number with awk is just:
$ awk -v seed="$RANDOM" 'BEGIN{srand(seed)} {for (i=1; i<=NF; i++) $i=sprintf("%02X",rand()*256)} 1' file
C7 A1 02 1A 4A 94 95 A0 1E
$ awk -v seed="$RANDOM" 'BEGIN{srand(seed)} {for (i=1; i<=NF; i++) $i=sprintf("%02X",rand()*256)} 1' file
1C 50 A9 D3 8B B0 24 9C 14
Hopefully it's very obvious what it's doing.
Here is an idea on how to get a random hex (mac address?)
awk -v seed=$RANDOM '
BEGIN{
srand(seed);
split("0 1 2 3 4 5 6 7 8 9 A B C D E F",hex," ");
for (i=1; i<=6; i++)
printf "%s%s ",hex[int(rand()*16)+1],hex[int(rand()*16)+1];
print ""
}'
D8 D9 BA 00 6A C6

Bash extracting multiple values

I'm working on extracting data from SNMP services. The output looks like this.
experimental.94.4.5.1.6.16.0.0.192.221.13.147.179.0.0.0.0.0.0.0.0.1 = Hex-STRING: 00 00 00 1A 85 95 13 F4
experimental.94.4.5.1.6.16.0.0.192.221.13.147.179.0.0.0.0.0.0.0.0.24 = Hex-STRING: 00 00 10 8E 0C F4 99 1C
experimental.94.4.5.1.6.16.0.0.192.221.31.68.71.0.0.0.0.0.0.0.0.10 = Hex-STRING: 00 00 17 10 45 A5 13 3C
I would like the output to be in three parts.
192.221.13.147.179 1 0000001A859513F4
192.221.13.147.179 24 0000108E0CF4991C
192.221.31.68.71 10 0000171045A5133C
I can accomplish all 3 individually using awk but not all at once.
awk '{print substr($0,28,35)}'
you can use:
awk 'BEGIN{FS="[.: ]"}{print $10"."$11"."$12"."$13"."$14" "$23" "$27$28$29$30$31$32$33$34}'

Remove 0 character

I have a text file that contains junk characters after some lines. As far as I know this is character 0 (not '0') although I am not sure. When I open the file in vi one such line looks like this:
ESH6^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#
I am only interested in the "ESH6" part of that line, I do not want the rest. If I do a hexdump -C that line looks like this (hexdump -C foo.txt | head -n 2, the "49705" is part of another line):
00000000 45 53 48 36 00 00 00 00 00 00 00 00 00 00 00 00 |ESH6............|
00000010 00 00 00 00 0a 0a 34 39 37 30 35 0a 0a 0a 0a 45 |......49705....E|
How can I remove all instances of this character from the file?
Use the tr command:
tr -d '\0' < foo.txt > output
This deletes (-d) all null bytes ('\0') from the standard input and writes everything else to standard output. The tr command is a pure filter; it only reads standard input and only writes to standard output.

Sed: get lines beginning with some prefix

I have file Blackberry jad file:
RIM-COD-URL-12: HelloWorld-12.cod
RIM-COD-Size: 68020
RIM-MIDlet-Icon-2-1: ____HOVER_ICON_res/icon/blackberry/icon-68.png,focused
RIM-COD-URL-11: HelloWorld-11.cod
RIM-MIDlet-Icon-Count-2: 1
RIM-COD-URL-10: HelloWorld-10.cod
RIM-MIDlet-Icon-Count-1: 1
MIDlet-Vendor: Vasia Pupkin
RIM-MIDlet-Icon-1-1: res\icon\blackberry\____HOVER_ICON_icon-68.png,focused
Manifest-Version: 1.0
RIM-MIDlet-Flags-1: 0
RIM-COD-SHA1-38: 9a c8 b3 35 72 de 34 5e 7a 0a 5b 9e c3 3a 65 4c 20 0f 8e 50
I just want to get lines begin with RIM-COD-.
Can you provide me solutions for awk or sed?
Use sed -n and only print lines that match RIM-COD.
sed -n -e '/^RIM-COD-/p' yourfile.txt
Try doing this :
awk '/^RIM-COD/' file.txt
Or
grep "^RIM-COD" file.txt
Or
sed -n '/^RIM-COD/p' file.txt

Resources