Example string
CA DA 00 17 11 38 88 C5 03
Desired output
AB 3C 6C 8F DA 88 24 78 6C
Commands attempted
$ tr -dc 0-9A-F < /dev/urandom filename ## prints too many chars
awk '{gsub(length($1)==2,{printf "%02")}}' filename ## syntax doesn't work, unsure how to add hex
$ sed 's/[a-z0-9]\{2\}//g' filename ## only replaces digits, unsure how to add hex as a replacement
I ended up using vim to do a partial conversion for some level of randomization.
:s/\d\d/AA/g
Can anyone provide a working solution?
It would be nice to see solutions (and explanations) leveraging tr/awk/sed for knowledge sharing purposes.
Thanks.
To replace each field with a random 2-digit hex number with awk is just:
$ awk -v seed="$RANDOM" 'BEGIN{srand(seed)} {for (i=1; i<=NF; i++) $i=sprintf("%02X",rand()*256)} 1' file
C7 A1 02 1A 4A 94 95 A0 1E
$ awk -v seed="$RANDOM" 'BEGIN{srand(seed)} {for (i=1; i<=NF; i++) $i=sprintf("%02X",rand()*256)} 1' file
1C 50 A9 D3 8B B0 24 9C 14
Hopefully it's very obvious what it's doing.
Here is an idea on how to get a random hex (mac address?)
awk -v seed=$RANDOM '
BEGIN{
srand(seed);
split("0 1 2 3 4 5 6 7 8 9 A B C D E F",hex," ");
for (i=1; i<=6; i++)
printf "%s%s ",hex[int(rand()*16)+1],hex[int(rand()*16)+1];
print ""
}'
D8 D9 BA 00 6A C6
Related
I have a hex file, I need to extract a range of it to a text file
From range:
To Range:
I need Output: AC:E4:B5:9A:53:1C
i tried many but it not really correct requirements, Output: Binary file filehex matches
grep "["'\x9f\x87\x6f\x11'"-"'\x9f\x87\x70\x11'"]" filehex > test.txt
hope someone can help me
Use -a to force the text interpretation of the input.
Use -o to only output the matching part.
The expression you used doesn't make much sense. It matches any characters in the set \x9, \x87, \x6f, and then the range \x11-\x9f, etc.
You are rather interested in something that starts with \x9\x87\x6f\x11 and ends in \x9f\x87\x70\x11, and there can be anything in between.
You can use cut to remove the leading and trailing 4 bytes.
grep -oa $'\x9f\x87\x6f\x11.*\x9f\x87\x70\x11' hexfile | cut -b5-21
If you know the length of the string will always be 17 bytes, you can use .\{17\} instead of .*.
Ok I've build randomly one binary $file
with your string at a location making hd command to split them.
Note: regarding k314159' comment, I use hd to produce hexdump output similarto CentOS's hexdump tool.
One shoot using sed:
hd $file |sed -e 'N;/ 9f \+\(|.*\n[0-9a-f]\+ \+\|\)87 \+\(|.*\n[0-9a-f]\+ \+\|\)6f \+\(|.*\n[0-9a-f]\+ \+\|\)11 /p;D;'
000161c0 96 7a b2 21 28 f1 b3 32 63 43 93 ff 50 a6 9f 87 |.z.!(..2cC..P...|
000161d0 6f 11 0d 7a a5 a9 81 9e 32 9d fb 71 27 6d 60 f2 |o..z....2..q'm`.|
0002c3a0
Explanation:
N merge next line in current buffer
\(|.*\n[0-9a-f]\+ \+\|\) match a | followed by anything and a newline (\n), then immediately an hexadecimal number and a space OR nothing.
p print current buffer (two lines)
D Delete upto newline in current buffer, keep last line for next sed loop.
The last hexadecimal 00028d2a correspond to the size of my binary $file:
printf "%x\n" $(stat -c %s $file)
Using bash + grep:
printf -v var "\x9f\x87\x6f\x11"
IFS=: read -r offset _ < <(grep -abo "$var" $file)
hd $file | sed -ne "$((offset/16-1)),+4p"
000161a0 b7 8f 4a 4d ed 89 6c 0b 25 f9 e7 c9 8c 99 6e 23 |..JM..l.%.....n#|
000161b0 3c ba 80 ec 2e 32 dd f3 a4 a2 09 bd 74 bf 66 11 |<....2......t.f.|
000161c0 96 7a b2 21 28 f1 b3 32 63 43 93 ff 50 a6 9f 87 |.z.!(..2cC..P...|
000161d0 6f 11 0d 7a a5 a9 81 9e 32 9d fb 71 27 6d 60 f2 |o..z....2..q'm`.|
000161e0 15 86 c2 bd 11 d0 08 90 c4 84 b9 80 04 4e 17 f1 |.............N..|
Where you could read your string:
000161c0 9f 87 | ..|
000161d0 6f 11 |o. |
For testing, I've built my test file by:
dd if=/vmlinuz bs=90574 count=1 of=/tmp/testfile
printf '\x9f\x87\x6f\x11' >>/tmp/testfile
dd if=/vmlinuz bs=90574 count=1 >>/tmp/testfile
file=/tmp/testfile
Use grep to search for the original binary file, not the hex dump. Extending choroba's answer, I think you may have problems with grep trying to interpret your search pattern as UTF-8 or some other encoding. You should temporarily set the environment variable LC_ALL=C for grep to treat each byte individually. Also, you can use the -P option to enable use of lookbehind and lookahead in your pattern. So your command becomes:
LANG=C grep -oaP $'(?<=\x9f\x87\x6f\x11).*(?=\x9f\x87\x70\x11)' binary-file > test.txt
Proof that it works:
$ echo $'BEFORE\x9f\x87\x6f\x11AC:E4:B5:9A:53:1C\x9f\x87\x70\x11AFTER' | LANG=C grep -oaP $'(?<=\x9f\x87\x6f\x11).*(?=\x9f\x87\x70\x11)'
AC:E4:B5:9A:53:1C
$
This question already has answers here:
How can I output null-terminated strings in Awk?
(4 answers)
Closed 7 years ago.
I have a function which outputs some file paths, I need these paths are separated by NUL charachter instead of new line \n character. I tried following code:
function myfunc
{
declare -a DUPS
# some commands to fill DUPS with appropriate file/folder paths
( for i in "${DUPS[#]}"; do echo "$i"; done )|sort|uniq|awk 'BEGIN{ORS="\x00";} {print substr($0, index($0, $2))}'
}
But if I pipe its output to hexdump or hd, no NUL character is diplayed. It seems that NUL character is not included in the awk output:
myfunc | hd
Will print:
00000000 2f 70 61 74 68 2f 6e 75 6d 62 65 72 2f 6f 6e 65 |/path/number/one|
00000010 2f 2f 70 61 74 68 2f 6e 75 6d 62 65 72 2f 74 77 |//path/number/tw|
00000020 6f 2f 2f 70 61 74 68 2f 6e 75 6d 62 65 72 2f 74 |o//path/number/t|
00000030 68 72 65 65 2f |hree/|
00000035
My awk version is:
~$ awk -W version
mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan
compiled limits:
max NF 32767
sprintf buffer 2040
Also any solution with other commands such as sed is acceptable for me.
NOTE: My question is not duplicate of enter link description here, because it asks for a solution that works on different machines with different awks. But I just need a solution that works on my own machine, so I could use any version of awk that could be installed on Ubuntu 14.04.
Gnu Awk v4.0.1 works just fine with your original program, but all the other awks I have kicking around (mawk, original-awk and busybox awk) produce the same NUL-free output as you seem to be experiencing. It appears that with those awks, using either print or printf to print a string with embedded NULs causes the NUL to be treated as a string terminator.
However, mawk and original-awk will output a real NUL if you use printf "%s",0;. So if you are using one of those, you could set ORS to the empty string and add {printf "%s", 0;} to the end of your awk program. (You'd need other more invasive modifications if your awk program uses next).
I don't know any way to convince busybox awk to print a NUL byte, so if that is what you are using you might want to consider choosing a real awk.
I have a temperature sensor connected to a *nix system and the typical output is something like:
pi#raspberrypi $ cat /sys/bus/w1/devices/28-00000202070c/w1_slave
c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12187
The result comes without any comma, but is assumed that is always coming with 3 digits after the comma, so in this example would be 12.187º.
I have implemented a filter that places a comma after the second char, and it works most of the time:
grep t= | awk '{print $10;}'| sed 's/t\=//g' | sed 's/\([0-9][0-9]\)\([0-9]\)/\1,\2/g'
However, during winter, temperature drops below 10º and my filter returns values like 95,32º (when it should be 9,532º).
Is there any way of counting characters from the right, so I could always count with the 3 digital characters (and avoiding this problem in temperatures below 10º)?
Thanks,
Joaoabs
awk can handle floating point operations:
awk '/t=/ { sub(/t=/,"",$NF); print $NF/1000}' /sys/bus/w1/devices/28-00000202070c/w1_slave
If I understand you correctly, then what you want to do is :
sed 's/\([0-9][0-9][0-9]\)$/,\1/g'
$ in a regex means 'the end' so this searches for 3 digits right at the end of a string and replaces them with comma+the found digits.
(Note: This should be the last part of your pipe, with the beginning unchanged.)
Using awk
awk -F= '/t=/ {print $NF/1000}' /sys/bus/w1/devices/28-00000202070c/w1_slave
12.187
Or store it to a variable:
temp=$(awk -F= '/t=/ {print $NF/1000}' /sys/bus/w1/devices/28-00000202070c/w1_slave)
echo "$temp"
12.187
don't do grep|awk|sed|sed..., try this:
...|awk -F't=' -v OFS='t=' 'NF>1{$2=sprintf("%'\''d",$2)}7'
test with your data:
kent$ echo "c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12187"|awk -F't=' -v OFS='t=' 'NF>1{$2=sprintf("%'\''d",$2)}7'
c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12,187
awk has the format pattern in 'printf' for your requirement. just use it.
if you just want to have the t= value:
awk -F't=' -v OFS='t=' 'NF>1{printf "%'\''d\n",$2}'
with same input:
kent$ echo "c3 00 4b 46 7f ff 0d 10 12 : crc=12 YES
c3 00 4b 46 7f ff 0d 10 12 t=12187"|awk -F't=' -v OFS='t=' 'NF>1{printf "%'\''d\n",$2}'
12,187
I have file Blackberry jad file:
RIM-COD-URL-12: HelloWorld-12.cod
RIM-COD-Size: 68020
RIM-MIDlet-Icon-2-1: ____HOVER_ICON_res/icon/blackberry/icon-68.png,focused
RIM-COD-URL-11: HelloWorld-11.cod
RIM-MIDlet-Icon-Count-2: 1
RIM-COD-URL-10: HelloWorld-10.cod
RIM-MIDlet-Icon-Count-1: 1
MIDlet-Vendor: Vasia Pupkin
RIM-MIDlet-Icon-1-1: res\icon\blackberry\____HOVER_ICON_icon-68.png,focused
Manifest-Version: 1.0
RIM-MIDlet-Flags-1: 0
RIM-COD-SHA1-38: 9a c8 b3 35 72 de 34 5e 7a 0a 5b 9e c3 3a 65 4c 20 0f 8e 50
I just want to get lines begin with RIM-COD-.
Can you provide me solutions for awk or sed?
Use sed -n and only print lines that match RIM-COD.
sed -n -e '/^RIM-COD-/p' yourfile.txt
Try doing this :
awk '/^RIM-COD/' file.txt
Or
grep "^RIM-COD" file.txt
Or
sed -n '/^RIM-COD/p' file.txt
So, currently i have created a code to do this as shown below. This code works and does what it is supposed to do after I echo the variables:
a=`awk 'NR==2 {print $1}' $coor`
b=`awk 'NR==3 {print $2}' $coor`
c=`awK 'NR==4 {print $3}' $coor`
....but i have to do this for many more lines and i want a more general expression. So I have attempted to create a loop shown below. Syntax wise i don't think anything is wrong with the code, but it is not outputting anything to the file "Cmain".
I was wondering if anyone could help me, I'm kinda new at scripting.
If it helps any, I can also post what i am trying to read.
for (( i=1; i <= 4 ; i++ )); do
for (( j=0; j <= 3 ; j++ )); do
B="`grep -n "cell" "$coor" | awk 'NR=="$i" {print $j}'`"
done
done
echo "$B" >> Cmain
You can replace your lines of awk with this one:
awk '{ for (i=1; i<=NF; i++) if (NR >= 2 && NR == i) print $(i - 1) }' file.txt
Tested input:
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70
71 72 73 74 75 76 77 78 79 80
Output:
11
22
33
44
55
66
77
awk 'BEGIN {f=1} {print $f; f=f+1}' infile > outfile
An alternative using sed and coreutils, assuming space separated input is in infile:
n=$(wc -l infile | cut -d' ' -f1)
for i in $(seq 1 $n); do
sed -n "${i} {p; q}" infile | cut -d' ' -f$i
done