I'm now doing it this way:
[root#~]# echo Aa|hexdump -v
0000000 6141 000a
0000003
[root#~]# echo -e "\x41\x41\x41\x41"
AAAA
But it's not exactly behaving as I wanted,
the hex form of Aa should be 4161,but the output is 6141 000a,which seems not making sense.
and when performing hex to ascii,is there another utility so that I don't need the prefix \x ?
The reason is because hexdump by default prints out 16-bit integers, not bytes. If your system has them, hd (or hexdump -C) or xxd will provide less surprising outputs - if not, od -t x1 is a POSIX-standard way to get byte-by-byte hex output. You can use od -t x1c to show both the byte hex values and the corresponding letters.
If you have xxd (which ships with vim), you can use xxd -r to convert back from hex (from the same format xxd produces). If you just have plain hex (just the '4161', which is produced by xxd -p) you can use xxd -r -p to convert back.
For the first part, try
echo Aa | od -t x1
It prints byte-by-byte
$ echo Aa | od -t x1
0000000 41 61 0a
0000003
The 0a is the implicit newline that echo produces.
Use echo -n or printf instead.
$ printf Aa | od -t x1
0000000 41 61
0000002
For single line solution:
echo "Hello World" | xxd -ps -c 200 | tr -d '\n'
It will print:
48656c6c6f20576f726c640a
or for files:
cat /path/to/file | xxd -ps -c 200 | tr -d '\n'
For reverse operation:
echo '48656c6c6f20576f726c640a' | xxd -ps -r
It will print:
Hello World
$> printf "%x%x\n" "'A" "'a"
4161
With bash :
a=abcdefghij
for ((i=0;i<${#a};i++));do printf %02X \'${a:$i:1};done
6162636465666768696A
I use:
> echo Aa | tr -d '\n' | xxd -p
4161
> echo 414161 | tr -d '\n' | xxd -r -p
AAa
The tr -d '\n' will trim any possible newlines in your input
I don't know how it crazy it looks but it does the job really well
ascii2hex(){ a="$#";s=0000000;printf "$a" | hexdump | grep "^$s"| sed s/' '//g| sed s/^$s//;}
Created this when I was trying to see my name in HEX ;)
use how can you use it :)
Text2Conv="Aa"
for letter in $(echo "$Text2Conv" | sed "s/\(.\)/'\1 /g");do printf '%x' "$letter";done
4161
The trick is using sed to parse the Text2Conv to format we can then seperate anf loop using for.
Finally got the correct thing
echo "Hello, world!" | tr -d '\n' | xxd -ps -c 200
here a little script I wrote to convert ascii to hex. hope it helps:
echo '0x'"`echo 'ASCII INPUT GOES HERE' | hexdump -vC | awk 'BEGIN {IFS="\t"} {$1=""; print }' | awk '{sub(/\|.*/,"")}1' | tr -d '\n' | tr -d ' '`" | rev | cut -c 3- | rev
SteinAir's answer above was helpful to me -- thank you! And below is a way it inspired, to convert hex strings to ascii:
for h in $(echo "4161" | sed "s/\(..\)/\1 /g"); do printf `echo "\x$h"`;done
Aa
echo -n Aa | hexdump -e '/1 "%02x"'; echo
according to http://mylinuxbook.com/hexdump/ you might use the hexdump format parameter
echo Aa | hexdump -C -e '/1 "%02X"'
will return 4161
to add an extra linefeed at the end, append another formatter.
BUT: the format given above will give multiplier outputs for repetitive characters
$ printf "Hello" | hexdump -e '/1 "%02X"'
48656C*
6F
instead of
48656c6c6f
jcomeau#aspire:~$ echo -n The quick brown fox jumps over the lazy dog | python -c "print raw_input().encode('hex'),"
54686520717569636b2062726f776e20666f78206a756d7073206f76657220746865206c617a7920646f67
jcomeau#aspire:~$ echo -n The quick brown fox jumps over the lazy dog | python -c "print raw_input().encode('hex')," | python -c "print raw_input().decode('hex'),"
The quick brown fox jumps over the lazy dog
it could be done with Python3 as well, but differently, and I'm a lazy dog.
echo append a carriage return at the end.
Use
echo -e
to remove the extra 0x0A
Also, hexdump does not work byte-per-byte as default. This is why it shows you bytes in a weird endianess and why it shows you an extra 0x00.
Related
It seems that here string is adding line break. Is there a convenient way of removing it?
$ string='test'
$ echo -n $string | md5sum
098f6bcd4621d373cade4e832627b4f6 -
$ echo $string | md5sum
d8e8fca2dc0f896fd7cb4cb0031ba249 -
$ md5sum <<<"$string"
d8e8fca2dc0f896fd7cb4cb0031ba249 -
Yes, you are right: <<< adds a trailing new line.
You can see it with:
$ cat - <<< "hello" | od -c
0000000 h e l l o \n
0000006
Let's compare this with the other approaches:
$ echo "hello" | od -c
0000000 h e l l o \n
0000006
$ echo -n "hello" | od -c
0000000 h e l l o
0000005
$ printf "hello" | od -c
0000000 h e l l o
0000005
So we have the table:
| adds new line |
-------------------------|
printf | No |
echo -n | No |
echo | Yes |
<<< | Yes |
From Why does a bash here-string add a trailing newline char?:
Most commands expect text input. In the unix world, a text file
consists of a sequence of lines, each ending in a
newline.
So in most cases a final newline is required. An especially common
case is to grab the output of a command with a command susbtitution,
process it in some way, then pass it to another command. The command
substitution strips final newlines; <<< puts one back.
fedorqui's helpful answer shows that and why here-strings (and also here-documents) invariably append a newline.
As for:
Is there a convenient way of removing it?
In Bash, use printf inside a process substitution as an "\n-less" alternative to a here-string:
... < <(printf %s ...)
Applied to your example:
$ md5sum < <(printf %s 'test')
098f6bcd4621d373cade4e832627b4f6
Alternatively, as user202729 suggests, simply use printf %s in the pipeline, which has the added advantage of not only using a more familiar feature but also making the command work in (more strictly) POSIX-compliant shells (in scripts targeting /bin/sh):
$ printf %s 'test' | md5sum
098f6bcd4621d373cade4e832627b4f6
As a "here doc" add a newline:
$ string="hello test"
$ cat <<_test_ | xxd
> $string
> _test_
0000000: 6865 6c6c 6f20 7465 7374 0a hello test.
Also a "here string" does:
$ cat <<<"$string" | xxd
0000000: 6865 6c6c 6f20 7465 7374 0a hello test.
Probably the easiest solution to get an string non-ending on newline would be printf:
$ printf '%s' "$string" | xxd
0000000: 6865 6c6c 6f20 7465 7374 hello test
It seems that here string is adding line break. Is there a convenient way of removing it?
$ string='test'
$ echo -n $string | md5sum
098f6bcd4621d373cade4e832627b4f6 -
$ echo $string | md5sum
d8e8fca2dc0f896fd7cb4cb0031ba249 -
$ md5sum <<<"$string"
d8e8fca2dc0f896fd7cb4cb0031ba249 -
Yes, you are right: <<< adds a trailing new line.
You can see it with:
$ cat - <<< "hello" | od -c
0000000 h e l l o \n
0000006
Let's compare this with the other approaches:
$ echo "hello" | od -c
0000000 h e l l o \n
0000006
$ echo -n "hello" | od -c
0000000 h e l l o
0000005
$ printf "hello" | od -c
0000000 h e l l o
0000005
So we have the table:
| adds new line |
-------------------------|
printf | No |
echo -n | No |
echo | Yes |
<<< | Yes |
From Why does a bash here-string add a trailing newline char?:
Most commands expect text input. In the unix world, a text file
consists of a sequence of lines, each ending in a
newline.
So in most cases a final newline is required. An especially common
case is to grab the output of a command with a command susbtitution,
process it in some way, then pass it to another command. The command
substitution strips final newlines; <<< puts one back.
fedorqui's helpful answer shows that and why here-strings (and also here-documents) invariably append a newline.
As for:
Is there a convenient way of removing it?
In Bash, use printf inside a process substitution as an "\n-less" alternative to a here-string:
... < <(printf %s ...)
Applied to your example:
$ md5sum < <(printf %s 'test')
098f6bcd4621d373cade4e832627b4f6
Alternatively, as user202729 suggests, simply use printf %s in the pipeline, which has the added advantage of not only using a more familiar feature but also making the command work in (more strictly) POSIX-compliant shells (in scripts targeting /bin/sh):
$ printf %s 'test' | md5sum
098f6bcd4621d373cade4e832627b4f6
As a "here doc" add a newline:
$ string="hello test"
$ cat <<_test_ | xxd
> $string
> _test_
0000000: 6865 6c6c 6f20 7465 7374 0a hello test.
Also a "here string" does:
$ cat <<<"$string" | xxd
0000000: 6865 6c6c 6f20 7465 7374 0a hello test.
Probably the easiest solution to get an string non-ending on newline would be printf:
$ printf '%s' "$string" | xxd
0000000: 6865 6c6c 6f20 7465 7374 hello test
Given that:
$ printf "love" | od -td4 -A n
1702260588
$ printf "lovehate" | od -td8 -A n
7310575196135911276
Is there a concise (ideally without loops, awk, sed, perl or python) way in Bash to convert the numbers 1702260588 and 7310575196135911276 to love and lovehate respectively?
Here's what I came up with:
alpha() {
(($1)) && printf "\x"$(printf "%02x" $(($1%256)))$(alpha $(($1/256)))"\n"
}
alpha 1702260588
alpha 7310575196135911276
Output:
love
lovehate
Edit: Here's an answer using the xxd utility:
# The echo is only necessary to get a newline at the end.
echo $(printf "%x" 1702260588 | xxd -r -p | rev)
echo $(printf "%x" 7310575196135911276 | xxd -r -p | rev)
Output:
love
lovehate
I'd like to put together a command that will print out a string of 32 hexadecimal digits. I've got a Python script that works:
python -c 'import random ; print "".join(map(lambda t: format(t, "02X"), [random.randrange(256) for x in range(16)]))'
This generates output like:
6EF6B30F9E557F948C402C89002C7C8A
Which is what I need.
On a Mac, I can even do this:
uuidgen | tr -d '-'
However, I don't have access to the more sophisticated scripting languages ruby and python, and I won't be on a Mac (so no uuidgen). I need to stick with more bash'ish tools like sed, awk, /dev/random because I'm on a limited platform. Is there a way to do this?
If you have hexdump then:
hexdump -vn16 -e'4/4 "%08X" 1 "\n"' /dev/urandom
should do the job.
Explanation:
-v to print all data (by default hexdump replaces repetition by *).
-n16 to consume 16 bytes of input (32 hex digits = 16 bytes).
4/4 "%08X" to iterate four times, consume 4 bytes per iteration and print the corresponding 32 bits value as 8 hex digits, with leading zeros, if needed.
1 "\n" to end with a single newline.
If you are looking for a single command and have openssl installed, see below. Generate random 16 bytes (32 hex symbols) and encode in hex (also -base64 is supported).
openssl rand -hex 16
There three ways that I know of:
#!/bin/bash
n=16
# Read n bytes from urandom (in hex):
xxd -l "$n" -p /dev/urandom | tr -d " \n" ; echo
od -vN "$n" -An -tx1 /dev/urandom | tr -d " \n" ; echo
hexdump -vn "$n" -e ' /1 "%02x"' /dev/urandom ; echo
Use one, comment out the other two.
Try:
xxd -u -l 16 -p /dev/urandom
Example output:
C298212CD8B55F2E193FFA16165E95E3
And to convert it back to binary:
echo -n C298212CD8B55F2E193FFA16165E95E3 | xxd -r -p
Here are a few more options, all of which have the nice property of providing an obvious and easy way to directly select the length of the output string. In all the cases below, changing the '32' to your desired string length is all you need to do.
#works in bash and busybox, but not in ksh
tr -dc 'A-F0-9' < /dev/urandom | head -c32
#works in bash and ksh, but not in busybox
tr -dc 'A-F0-9' < /dev/urandom | dd status=none bs=1 count=32
#works in bash, ksh, AND busybox! w00t!
tr -dc 'A-F0-9' < /dev/urandom | dd bs=1 count=32 2>/dev/null
EDIT: Tested in different shells.
If you want to generate output of arbitrary length, including even/odd number of characters:
cat /dev/urandom | hexdump --no-squeezing -e '/1 "%x"' | head -c 31
Or to maximize efficiency over readability/composeability:
hexdump --no-squeezing -e '/1 "%x"' -n 15 /dev/urandom
Here is a version not using dev/random:
awk -v len=32 'BEGIN {
srand('$RANDOM');
while(len--) {
n=int(rand()*16);
printf("%c", n+(n>9 ? 55 : 48));
};}'
you can also use od command like this
od -N32 -x < /dev/urandom | head -n1 | cut -b9- | sed 's/ //gi'
good luck
I'm trying to make a small function that removes all the chars that are not digits.
123a45a ---> will become ---> 12345
I've came up with :
temp=$word | grep -o [[:digit:]]
echo $temp
But instead of 12345 I get 1 2 3 4 5. How to I get rid of the spaces?
Pure bash:
word=123a45a
number=${word//[^0-9]}
Here's a pure bash solution
var='123a45a'
echo ${var//[^0-9]/}
12345
is this what you are looking for?
kent$ echo "123a45a"|sed 's/[^0-9]//g'
12345
grep & tr
echo "123a45a"|grep -o '[0-9]'|tr -d '\n'
12345
I would recommend using sed or perl instead:
temp="$(sed -e 's/[^0-9]//g' <<< "$word")"
temp="$(perl -pe 's/\D//g' <<< "$word")"
Edited to add: If you really need to use grep, then this is the only way I can think of:
temp="$( grep -o '[0-9]' <<< "$word" \
| while IFS= read -r ; do echo -n "$REPLY" ; done
)"
. . . but there's probably a better way. (It uses grep -o, like your solution, then runs over the lines that it outputs and re-outputs them without line-breaks.)
Edited again to add: Now that you've mentioned that you use can use tr instead, this is much easier:
temp="$(tr -cd 0-9 <<< "$word")"
What about using sed?
$ echo "123a45a" | sed -r 's/[^0-9]//g'
12345
As I read you are just allowed to use grep and tr, this can make the trick:
$ echo "123a45a" | grep -o [[:digit:]] | tr -d '\n'
12345
In your case,
temp=$(echo $word | grep -o [[:digit:]] | tr -d '\n')
tr will also work:
echo "123a45a" | tr -cd '[:digit:]'
# output: 12345
Grep returns the result on different lines:
$ echo -e "$temp"
1
2
3
4
5
So you cannot remove those spaces during the filtering, but you can afterwards, since $temp can transform itself like this:
temp=`echo $temp | tr -d ' '`
$ echo "$temp"
12345