The following steps are to be performed:
Do base64 decode on ASCII value.
To 'chirp' append the decoded value.
Generate sha256 on the "chirp<decoded_value>"
#!/bin/sh
a=$(echo MDkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDIgADAoR0WUZBTAkZv0Syvt+g5wGpb/HYHh22zAxCNP+ryTQ=|base64 -d)
b="chirp$a"
echo $b
echo -n $b | sha256sum
I am getting a value of:
f62e19108cfb5a91434f1bba9f5384f9039857743aa2c0707efaa092791e4420
But the expected value is:
6a29cb4....
Am I missing anything?
For binary data, as the base64-decoded data that your dealing with, I would not rely too much on echo, but just pipe the stuff, like this:
<<<'MDkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDIgADAoR0WUZBTAkZv0Syvt+g5wGpb/HYHh22zAxCNP+ryTQ=' base64 -d | cat <(echo -n chirp) - | sha256sum
That gives me the result that you expect, 6a29cb438954e8c78241d786af874b1c7218490d3024345f6e11932377a932b6 .
Here, cat gets two file descriptors as arguments, the first one streaming the word "chirp" and the second one forwarding the stdout of the previous command (base64 -d)
Related
I have a hash file that contains only the digest:
cat archive.hash
DBD47245A6E78626CBAD9A228C8956741BBA853E9D4A1DFCA6E335366C438C26341321E83B614834F78CD4D8FF71B05D5ADE8FAB3E0ECE5EF34A486D09882BB7
In order to accept the hash file as input and check, the sha512sum command requires that the file is formatted with a low chars digest followed by two spaces and the file name:
9a228c8956741bba853e9d4a1dfca6e335366c438c26341321e83b614834f78cd4d8ff71b05d5ade8fab3e0ece5ef34a486d09882bb7 archive.zip
I tried several syntax such as:
sha512sum -c <(echo "$(cat archive.hash |tr A-Z a-z) archive.zip")
But I receive this error:
sha512sum: /dev/fd/63: no properly formatted SHA512 checksum lines found
Analyzing the string:
echo "$(cat archive.hash |tr A-Z a-z) archive.zip"
The output is this:
archive.zip9a228c8956741bba853e9d4a1dfca6e335366c438c26341321e83b614834f78cd4d8ff71b05d5ade8fab3e0ece5ef34a486d09882bb7
It put the file name in front of the hash digest. How can this problem be avoided?
The below code is producing the expected results with username.
es_eh="$(aws cloudtrail --region us-east-1 lookup-events --lookup-attributes AttributeKey=EventSource,AttributeValue=route53.amazonaws.com --max-items 50 --start-time "${start_date}" --end-time "${end_date}" --output json)"
for row in $(echo "${es_eh}" | jq -r '.Events[] | #base64'); do
echo "${row}" | base64 --decode | jq -r '.Username'
done
I didn't understand the purpose of doing base64 encode and then doing decode of the same string inside loop to retrieve username.
This is not working when I remove base64 encode and decode.
for row in $(echo "${es_eh}" | jq -r '.Events[]'); do
echo "${row}"| jq -r '.Username'
done
Without the encoding, the output of the first jq is more than one row. The loop iterates over the lines and fails, as none of them contains a valid JSON. With the | #base64, each subobject is encoded to a single row, inflated back to a full JSON object by base64 --decode.
To see the rows, try outputting $row before processing it.
When you use $( ) without quotes around it, the result gets split into "words", but the shell's definition of a "word" is almost never what you want (and certainly has nothing to do with the json entries you want it split into). This sort of thing is why you should almost never use unquoted expansions.
Converting the output entries to base64 makes them wordlike enough that shell word splitting actually does the right thing. But note: some base64 encoders split their output into lines, which would make each line be treated as a separate "word". If jq's base64 encoding did this, this code would fail catastrophically on large events.
Transforming the for loop into a while loop should fix the problem :
while read -r row; do
echo "${row}" | jq -r '.Username'
done < <(echo "${es_eh}" | jq -c -r '.Events[]')
Note that in the outer jq, I used option -c to put output in a single ine.
I am trying to make a bash script work on MinGW and it seems the shell is not able to decode something like the following.
t=$(openssl rand -base64 64)
echo "$t" | base64 --decode
Resulting in,
Ԋ7▒%
▒7▒SUfX▒L:o<x▒▒䈏ţ94V▒▒▒▒uW;▒▒pxu▒base64: invalid input
Interestingly, if I output the base64 character and run the command as such, it works.
echo "+e5dcWsijZ83uR2cuKxIDJTTiwTvqB7J0EJ63paJdzGomQxw9BhfPvFTkdKP9I1o
g29pZKjUfDO8/SUNt+idWQ==" | base64 --decode
Anybody knows what I am doing wrong?
Thanks
I solved the above case by passing --ignore-garbage flag to the base64 decode. It ignores the non-alphabet characters.
echo "$t" | base64 --decode --ignore-garbage
However, I would still like to know how did I create "garbage" ;) ?
I think what has happened here is the base64 string contains some embedded spaces, and that causes the actual "invalid input" w (and what you observe to as garbage.)
The openssl rand -base64 64 command introduces some newlines (not spaces), for example,
openssl rand -base64 64 > b64.txt
... then viewing the b64.txt file in an editor I see two separate lines
tPKqKPbH5LkGu13KR6zDdJOBpUGD4pAqS6wKGS32EOyJaK0AmTG4da3fDuOI4T+k
abInqlQcH5k7k9ZVEzv8FA==
... and this implies there is a newline character between the 'k' and 'a'
So the base64 string has this embedded newline. The base64 -d can handle newlines (as demonstrated by your successful example), but, it cannot handle space char.
The newlines can get translated to spaces by some actions of the shell. This is very likely happening by the echo $t I.e. if t has newlines inside it, then echo will just replace then with single space. Actually, how it behaves can depend on shell options, and type of string quotes, if any, applied.
To fix the command, we can remove the newline before piping to the base64 -d command.
One way to do that is to use tr command, e.g. the following works on Linux:
t=$(openssl rand -base64 64 | tr -d '\n')
echo $t | base64 -d
... or alternatively, remove the spaces, again using tr
t=$(openssl rand -base64 64)
echo $t | tr -d ' ' | base64 -d
What would be the easiest way to convert the text produced by utilities such as sha512sum into a binary file?
I'd like to convert hex string like 77f4de214a5423e3c7df8704f65af57c39c55f08424c75c0931ab09a5bfdf49f5f939f2caeff1e0113d9b3d6363583e4830cf40787100b750cde76f00b8cd3ec (example produced by sha512sum) into a binary file (64 bytes long), in which each byte's value would be equivalent to a pair of letters/digits from the string. I'm looking for a solution that would require minimal amount of tools, so I'd be happy if this can be done easily with bash, sed or some utility from coreutils. I'd rather avoid xxd, as this doesn't seem to handle such string anyway (I'd have to add "addresses" and some whitespace).
I need the hash as a binary file, to convert it into an object file and link with the application that I'm writing. If there's another way to embed such string (in a binary form!) into application (via an array or whatever) - it's also a solution for me.
A bit of sed and echo might do:
for i in $(echo 77f4de214a5423e3c7df8704f65af57c39c55f08424c75c0931ab09a5bfdf49f5f939f2caeff1e0113d9b3d6363583e4830cf40787100b750cde76f00b8cd3ec | sed 's/../& /g'); do
echo -ne "\x$i"
done > output.bin
The sed command is splitting the hex string into bytes and the echo shows it as hexadecimal character.
Or in a shorter form with sha512sum output, as suggested in the comment:
echo -ne "$(sha512sum some-file.txt | sed 's/ .*$//; s/../\\x&/g')"
How about perl:
<<<77f4de214a5423e3c7df8704f65af57c39c55f08424c75c0931ab09a5bfdf49f5f939f2caeff1e0113d9b3d6363583e4830cf40787100b750cde76f00b8cd3ec \
perl -e 'print pack "H*", <STDIN>' > hash.bin
If you have openssl in your system and want a sha512 hash in binary form, you can use this:
openssl dgst -sha512 -binary somefile.txt
If you have node:
node -e "var fs = require('fs'); fs.writeFileSync('binary', new Buffer('77f4de214a5423e3c7df8704f65af57c39c55f08424c75c0931ab09a5bfdf49f5f939f2caeff1e0113d9b3d6363583e4830cf40787100b750cde76f00b8cd3ec', 'hex'))"
s="77f4de214a5423e3c7df8704f65af57c39c55f08424c75c0931ab09a5bfdf49f5f939f2caeff1e0113d9b3d6363583e4830cf40787100b750cde76f00b8cd3ec";
echo -n $s | xxd -r -p > file.bin
1 File(s) 64 bytes
Tested on Ubuntu 16.04.7
This problem is really leaving me astounded
Take a input file and create a HMAC_256 value from it using a private key
Base64 encode HMAC_256 hash
Code
#Create HMAC-SHA2 hash from shell parameter
filehash=`echo $1 | mac -a sha256_hmac -k test.key`
echo "HMAC_SHA256 hash : "$filehash
#Base64 encode filehash using openssl
filehash_64=`echo "$filehash" | /usr/sfw/bin//openssl enc -base64 | tr '\n' ' ' | cut -d " " -f2 `
echo "64 bit encoded hash : "$filehash_64
Using a test.key of
Bob123
Shell Input
Hello
Shell Output
SHA256 hash : 411796cfb1e6c30c1b39b589c79d6f8bf1fdde8d58fda4a6ec1e59538ecaa39a
64 bit encoded hash : ZWMxZTU5NTM4ZWNhYTM5YQo=
However if I go to these sites and do a HMAC_256 test they both generate a different hash
http://asecuritysite.com/encryption/hmac
http://jetcityorange.com/hmac/
They both Output a HMAC_256 hash of
a30410f584726f32ba3e6e823bfdecbdf28448d64e4ab8f11f6a2e66818b50fe
Why are they generating a different hash? I am assuming they are correct as they both have the same.
Does Solaris 10 have a bug with its MAC (Message Authentication Code)
tool?
Is it UTF8 or ASCII problem?
Is it a server problem, Windows / Unix?
I don't understand why I am generating a different hash to them, even though I am using the same hashing algorithm and key.
I'll guess that the problem is newline characters. The echo command puts a newline after "Hello", so if you don't want it, use "echo -n". Also make sure that there is no newline character in your key file.