AES (aes-cbc-256) encryption/decryption with openssl expected output truncated - bash

I wrote a script. Please pardon that I am not an expert in scripting.
Upon deciphering, the results gotten truncated.
[Message in text]: 0123456789abcdefghijklmnopqrstuvwxyz
message_input in hex: 303132333435363738396162636465666768696a6b6c6d6e6f707172737475767778797a0a
key: 788a1ca0bf1ab80f092841aabd77793f
hex string is too short, padding with zero bytes to length
c19f83afc1160ce81b0fc9906d513693386ccdd313b0f2884c698411441054e8
ciphered text: c19f83afc1160ce81b0fc9906d513693386ccdd313b0f2884c698411441054e8
IV: 7ecd3d63a8b74bb2f80d71a1c9d43359
deciphering ...
hex string is too short, padding with zero bytes to length
key: 788a1ca0bf1ab80f092841aabd77793f
iv: 7ecd3d63a8b74bb2f80d71a1c9d43359
answer: 30313233343536373839616263646566
Deciphered Message in hex: 30313233343536373839616263646566
deciphered text: 0123456789abcdef
The recovered deciphered text: 0123456789abcdef, ghijklmnopqrstuvwxyz gets truncated. This is supposed to be AES-CBC. Is there an option I did not turn on?
Here is the ciphering:
IV=$(openssl rand -hex 16)
get_key_for_ciphering; # key_for_ciphering gets populated
message_input=$(echo -n "${message_input//[[:space:]]/}") # remove spaces
echo "message_input in hex: "$message_input
echo "key": $key_for_ciphering;
ANS=$(echo "0: $message_input" | xxd -r | openssl enc -aes-256-cbc -iv $IV -K "$key_for_ciphering" | xxd -p)
ANS=$(echo -n "${ANS//[[:space:]]/}") # remove spaces
Here is the deciphering (message_input=$ANS):
get_key_for_ciphering; # key_for_ciphering gets populated
ANS=$(echo "0: $message_input" | xxd -r | openssl enc -aes-256-cbc -d -nopad -nosalt -K "$key_for_ciphering" -iv $IV | xxd -p) # -nopad -nosalt

Focusing in your question, the problem is in the xxd command. When converting a hex string to binary with xxd -r, you have to use the -p to tell xxd that is a plain hex string (no line breaks).
When converting back to hex with xxd -p, line breaks are added every 32 bytes. Unfortunately, xxd doesn't provide a flag to not include line breaks (you could use -c to set the number of columns, but it's limited to a max number). There are many option to remove line breaks, but one is appending a | tr -d '\n' to your command, as shown in the example below.
IV=$(openssl rand -hex 16)
key_for_ciphering=$(openssl rand -hex 16)
message_input="303132333435363738396162636465666768696a6b6c6d6e6f707172737475767778797a0a"
message_input=$(echo -n "${message_input//[[:space:]]/}") # remove spaces
echo "Message: $message_input"
echo "Key: $key_for_ciphering"
echo "IV: $IV"
ANS=$(echo "0: $message_input" | xxd -r -p | openssl enc -aes-256-cbc -iv $IV -K "$key_for_ciphering" | xxd -p | tr -d '\n')
ANS=$(echo -n "${ANS//[[:space:]]/}") # remove spaces
echo "Encrypted: $ANS"
ANS=$(echo "0: $ANS" | xxd -r -p | openssl enc -aes-256-cbc -d -nopad -nosalt -K "$key_for_ciphering" -iv $IV | xxd -p | tr -d '\n')
echo "Decrypted: $ANS"

---- Edit: ----
Doesn't work, as shell-parameters cannot contain binary zero. Possible fix with filters:
#!/bin/bash
tohex () {
perl -e 'binmode STDIN; while (<STDIN>) { print unpack "H*",$_; }'
}
fromhex () {
perl -e 'binmode STDIN; while (<STDIN>) { print pack "H*",$_; }'
}
binInput='0123456789abcdefghijklmnopqrstuvwxyz'
hexIV="$(openssl rand -hex 16)"
hexKey='788a1ca0bf1ab80f092841aabd77793f'
hexCipher="$(printf '%s' "$binInput" |\
openssl enc -aes-256-cbc -nosalt -iv "$hexIV" -K "$hexKey" | tohex)"
binResult="$(printf '%s' "$hexCipher" | fromhex |\
openssl enc -aes-256-cbc -d -iv "$hexIV" -K "$hexKey")"
if [ "$binInput" = "$binResult" ]; then echo OK;
fi
---- Original: ----
I think your problem lies in hexadecimal conversion. Try using perl pack/unpack:
tohex () {
perl -e 'print unpack "H*", "$ARGV[0]"' "$1"
}
fromhex () {
perl -e 'print pack "H*", "$ARGV[0]"' "$1"
}
message='0123456789abcdefghijklmnopqrstuvwxzy §"+!%/=()'
message_hex=$(tohex "$message")
message_cmp=$(fromhex "$message_hex")
if [ "$message" = "$message_cmp" ]; then echo OK; fi

Related

Syntax error while trying to generate a 2048bit long prime number

I'm trying to generate a 2048 bit long prime number, this is my code so far:
#!/bin/bash
generate_random() {
hex=$(head -c 256 /dev/urandom | xxd -p)
bc <<< "ibase=16; $hex"
}
p=$(generate_random)
echo "$p"
While running the script I get (standard_in) 1: syntax error followed by random zeroes.
Anyone knows what is causing this error and how can I fix it? I've tried with bash -x, but it doesn't add any useful information.
First, bc understands only upper-case letters as hex digits (at least by default). Second, you have separators in your xxd output, so you generate multiple numbers with bc later.
This should work:
#!/bin/bash
generate_random() {
hex=$(head -c 256 /dev/urandom | xxd -p -u | tr -d '\n')
bc <<< "ibase=16; $hex"
}
p=$(generate_random)
echo "$p"
-u flag to xxd instructs it to output upper-case letters as digits, and tr removes separators.
Example output:
84404284040092528807148386035025161100484110236893077703095592941720\
00537078513504880246726730474236368181068985417211434943913923235822\
01284401417146606673073772989889733010524123703686975444423088406509\
44767677616371794606797386146855833950295071249000795855185540560405\
62673903614333076371092344026999031152809898928396395497832309795471\
93897215963003601022703133486344387720277877558264139632520964120681\
97764906669023878701319760947789227343517474218584987497204300184084\
62846775760153647010072072799120566180042021620262646969602253704108\
06274157727080642084167983313757899766696995668747042179553171962777\
5716
To remove newline separators and backslashes, you can do
p_joined=$(echo "$p" | sed -z 's=\\\n==g')
echo "$p_joined"
instead.
An alternative way might be
printf -v hex '%s' $(od -v -An -N256 -x /dev/urandom)
read dec < <(bc <<< "ibase=16; ${hex^^}")
echo $dec

Bash OpenSSL is not equal to php openssl_encrypt value

I am trying to reach the same value of encryption in bash as it is from php , with no success .
Here is my php script
$message ='{"coupon_key":"011205358365345679","location_id":"288","device_key":"test_1234"}';
$key = "password";
$iv = base64_decode("dU+AyWweQYhAlGKLaxoS9w==");
$base64_iv = base64_encode($iv);
$base64_ev = base64_encode($encrypted_value);
$encrypted_value = openssl_encrypt(
$message,
'AES-256-CBC',
$key,
OPENSSL_RAW_DATA|OPENSSL_NO_PADDING,
$iv
);
$encode = base64_encode($encrypted_value);
//Zi7UcBwqM+gKZC9iZPgF3UHBXLUg1+Age/1+kRIfcstYBgGfYm7N1qCIFnm6jGn4AGQph/Q6hKjR1hYBT0wkJv8c8qFrHHZNUuraMfSRH3s=
$mac = hash_hmac('sha256', $base64_iv . $encode, $key);
echo '{"iv":"'.$base64_iv.'","value":"'.$encode.'","mac":"'.$mac.'"}'."<br>";
//{"iv":"dU+AyWweQYhAlGKLaxoS9w==","value":"Zi7UcBwqM+gKZC9iZPgF3UHBXLUg1+Age/1+kRIfcstYBgGfYm7N1qCIFnm6jGn4AGQph/Q6hKjR1hYBT0wkJv8c8qFrHHZNUuraMfSRH3s=","mac":"97fb6f4226a917322c7361af1d9b2949ad96691c1fc1a7f1c8379c71ae19f356"}
$retString2 = base64_encode('{"iv":"'.$base64_iv.'","value":"'.$encode.'","mac":"'.$mac.'"}');
echo $retString2;
//eyJpdiI6ImRVK0F5V3dlUVloQWxHS0xheG9TOXc9PSIsInZhbHVlIjoiWmk3VWNCd3FNK2dLWkM5aVpQZ0YzVUhCWExVZzErQWdlLzEra1JJZmNzdFlCZ0dmWW03TjFxQ0lGbm02akduNEFHUXBoL1E2aEtqUjFoWUJUMHdrSnY4YzhxRnJISFpOVXVyYU1mU1JIM3M9IiwibWFjIjoiOTdmYjZmNDIyNmE5MTczMjJjNzM2MWFmMWQ5YjI5NDlhZDk2NjkxYzFmYzFhN2YxYzgzNzljNzFhZTE5ZjM1NiJ9
and here is my bash version
password="password";
passwordhex=$(echo "$password" | xxd -c 256 -ps) ;
# iv2=$(hexdump -n 16 -e '4/4 "%08X" 1 "\n"' /dev/random) ;
iv=$(printf 'dU+AyWweQYhAlGKLaxoS9w==' | base64 -d )
biv=$(printf "$iv" | base64) ;
hexiv=$(printf "$iv" | xxd -c 256 -ps) ;
eString=$(printf '{"coupon_key":"011205358365345678","location_id":"288","device_key":"test_1234"}');
tttw=$(printf "$eString" | openssl aes-256-cbc -e -nosalt -a -A -K "$passwordhex" -iv "$hexiv") ;
echo "$tttw" ;
printf "${biv}${tttw}" |openssl dgst -sha256 -hmac abc -macopt hexkey:"$passwordhex" | sed 's/^.* //';
macopt2=$(printf "${biv}${tttw}" |openssl dgst -sha256 -hmac abc -macopt hexkey:"$passwordhex" | sed 's/^.* //');
echo "$macopt2"
finalString="{'iv':'$biv','value':'$tttw','mac':'$macopt2'}";
echo "$finalString";
sendHash=$(printf "$finalString" | base64 -w 0)
echo "$sendHash"
Starting from
tttw=$(echo -n $eString | openssl aes-256-cbc -e -nosalt -a -A -K $passwordhex -iv $hexiv) ;
it goes wrong , because $tttw is returning then
bKG5quB9/YQUsmlFvDHq2H+AfNGQuDfVztyi0dd5hCY7hLfaACnjD8SWlwqy0yy4hXUZSA2YcTXej/xtMg9vqEpoO6CDw9hk7+tUcYOOV5aOdVBnSLowmEllHt0JfjdE
instead of Zi7UcBwqM+gKZC9iZPgF3UHBXLUg1+Age/1+kRIfcstYBgGfYm7N1qCIFnm6jGn4AGQph/Q6hKjR1hYBT0wkJv8c8qFrHHZNUuraMfSRH3s=
Can anybody maybe see what option is going wrong here?
Note1: I have updated the bash script to use prinf instead of echo -n
Note 2 : Seems on my newer machine I get "hex string is too short, padding with zero bytes to length" from the line
tttw=$(printf $eString | openssl aes-256-cbc -e -nosalt -a -A -K $passwordhex -iv $hexiv) ;
Looks like the issue is near the very top:
passwordhex=$(echo "$password" | xxd -c 256 -ps) ;
So essentially the output of echo "$password" is password\n.
Using echo like this will append a line feed to the input for xxd which I assume you already know since the original script was packed with echo -n. I mentioned in the comments that printf is more portable. The accepted way to use printf with a variable is like this:
passwordhex=$(printf '%s' "$password" | xxd -c 256 -ps) ;
The reason for this is that if your variable has a format identifier (like %s), it would change the output unexpectedly. This would have also showed up on shellcheck.
With this one change, here is the output:
Zi7UcBwqM+gKZC9iZPgF3UHBXLUg1+Age/1+kRIfcssepjJ8+wUjTDAjPUMkGA+eF9EL284iD5UIzA+REyhMWLWbUJpPltHFk1+lhQyVlUXXVTw0FFV1G+iQfEWhbyg4
484123c33b54e446c61120112955cd15f3592f42e737c9fa24db266cdec954a2
484123c33b54e446c61120112955cd15f3592f42e737c9fa24db266cdec954a2
{'iv':'dU+AyWweQYhAlGKLaxoS9w==','value':'Zi7UcBwqM+gKZC9iZPgF3UHBXLUg1+Age/1+kRIfcssepjJ8+wUjTDAjPUMkGA+eF9EL284iD5UIzA+REyhMWLWbUJpPltHFk1+lhQyVlUXXVTw0FFV1G+iQfEWhbyg4','mac':'484123c33b54e446c61120112955cd15f3592f42e737c9fa24db266cdec954a2'}
eydpdic6J2RVK0F5V3dlUVloQWxHS0xheG9TOXc9PScsJ3ZhbHVlJzonWmk3VWNCd3FNK2dLWkM5aVpQZ0YzVUhCWExVZzErQWdlLzEra1JJZmNzc2Vwako4K3dValREQWpQVU1rR0ErZUY5RUwyODRpRDVVSXpBK1JFeWhNV0xXYlVKcFBsdEhGazErbGhReVZsVVhYVlR3MEZGVjFHK2lRZkVXaGJ5ZzQnLCdtYWMnOic0ODQxMjNjMzNiNTRlNDQ2YzYxMTIwMTEyOTU1Y2QxNWYzNTkyZjQyZTczN2M5ZmEyNGRiMjY2Y2RlYzk1NGEyJ30=

How to convert JSON Web Key Set (JWKS) public keys to PEM file using BASH?

Let's say you use AWS and you want to use ID based authentication using Cognito.
Then aws provides you with a public key you can verify the cognito payload with.
Let's also assume you don't want or cannot use any fancy libraries like jose since you are locked in a highly constrained environment.
The way to go is a BASH script that would make good old Brian Kernighan
proud.
You have to understand the encoding first.
Base64Url needs to be translated to Base64.
This is achieved using padding characters =
If the number of characters is divisible by 4 you don't need padding.
This relates to the binary digit representation.
Once that is taken care of you can translate Base64 to binary.
But how do I convert a JWKS/JWT to a PEM file this using BASH and BASH-programs only?
This is the solution I came up with following the officual documentation and sources listed below.
https://aws.amazon.com/premiumsupport/knowledge-center/decode-verify-cognito-json-token/
Please adapt input url or use json token directly and make sure you have jq installed.
Descriptive help is given in the function as comments.
Tested successfully on Ubuntu 18.04 and AmazonLinux2 (CentOS) as of 2020-04-28
#!/usr/bin/env bash
set -e
# FUNCTIONS
decodeBase64UrlUInt() { #input:base64UrlUnsignedInteger
local binaryDigits paddedStr
case $(( ${#1} % 4 )) in
2) paddedStr="$1==" ;;
3) paddedStr="$1=" ;;
*) paddedStr="$1" ;;
esac
binaryDigits=$( \
echo -n "$paddedStr" \
| tr '_-' '/+' \
| openssl enc -d -a -A \
| xxd -b -g 0 \
| cut -d ' ' -f 2 \
| paste -s -d '' \
)
echo "ibase=2; obase=A; $binaryDigits" | bc
# openssl enc:encoding; -d=decrypt; -a=-base64; -A=singleLineBuffer
# xxd "make-hexdump": -b=bits; -g=groupsize
# cut -d=delimiter; -f=field
# paste -s=serial|singleFile; -d=delimiter
}
base64UrlToHex() { #input:base64UrlString
local hexStr paddedStr
case $(( ${#1} % 4 )) in
2) paddedStr="$1==" ;;
3) paddedStr="$1=" ;;
*) paddedStr="$1" ;;
esac
hexStr=$( \
echo -n "$paddedStr" \
| tr '_-' '/+' \
| base64 -d \
| xxd -p -u \
| tr -d '\n' \
)
echo "$hexStr"
# base64 -d=decode
# xxd -p=-plain=continuousHexDump; -u=upperCase
# tr -d=delete
}
asn1Conf() { #input:hexStrPlainUpperCase
local e="$1"
local n="$2"
echo "
asn1 = SEQUENCE:pubkeyinfo
[pubkeyinfo]
algorithm = SEQUENCE:rsa_alg
pubkey = BITWRAP,SEQUENCE:rsapubkey
[rsa_alg]
algorithm = OID:rsaEncryption
parameter = NULL
[rsapubkey]
n = INTEGER:0x$n
e = INTEGER:0x$e
" | sed '/^$/d ; s/^ *//g' \
| openssl asn1parse \
-genconf /dev/stdin \
-out /dev/stdout \
| openssl rsa \
-pubin \
-inform DER \
-outform PEM \
-in /dev/stdin \
-out /dev/
# sed /^$/d=removeEmptyLines; /^ */=removeLeadingSpaces
}
main() {
local e n hexArr
local jwksUrl="$1"
local jwkJson=$(curl -sSSL $jwksUrl)
local kidList=$(jq -r '.keys[].kid' <<< "$jwkJson")
for keyId in $kidList; do
n=$(jq -r ".keys[] | select(.kid == \"$keyId\") | .n" <<< "$jwkJson")
e=$(jq -r ".keys[] | select(.kid == \"$keyId\") | .e" <<< "$jwkJson")
echo -e "\n$keyId"
# decodeBase64UrlUInt "$e"
# decodeBase64UrlUInt "$n"
asn1Conf $(base64UrlToHex "$e") $(base64UrlToHex "$n")
done
}
# MAIN
main 'https://cognito-idp.{region}.amazonaws.com/{userPoolId}/.well-known/jwks.json'
exit 0
Special thanks to:
Yury Oparin
https://www.yuryoparin.com/2014/05/base64url-in-bash.html
Cédric Deltheil
https://github.com/Moodstocks/moodstocks-api-clients/blob/master/bash/base64url.sh
Alvis Tang
https://gist.github.com/alvis/89007e96f7958f2686036d4276d28e47
Here are some options:
Either ignore base64 -d complaining of truncated input:
<<<'SGVsbG8geW91Cg' base64 -d 2>/dev/null ||:
Or fix the base64 padding with Bash before decoding:
base64URL='SGVsbG8geW91Cg'
printf -v pad_space '%*s' $((${#base64URL}%4)) ''
padded_base64="$base64URL${pad_space// /=}"
<<<"$padded_base64" base64 -d

OpenSSL decrypt AES 256bit (base64) encrypted password - wrong final block length

I am trying to decrypt aes-256-cdc encoded password using OpenSSL
#!/usr/bin/env bash
ak=BgL0cPoZQ4wZWOWl5mXBhlMsNbbZL2zvsWZXjuGy4Iw=
iv=cGEvcGWzE8t7CS3wbeoUFQ==
pass=RCQm23YHOCg3nxOl7CcQ7w==
#change format from base64 into hex
AES_KEY=$(echo "${ak}" | openssl base64 -d | xxd -p |tr -d '\n')
AES_IV=$(echo "${iv}" | openssl base64 -d | xxd -p)
ENCODED_PASSWORD=$(echo "${pass}" | openssl base64 -d | xxd -p)
echo "AES_KEY ${AES_KEY}"
echo "AES_IV ${AES_IV}"
echo "ENCODED_PASSWORD ${ENCODED_PASSWORD}"
#set password file
echo "${ENCODED_PASSWORD}" > in.txt
#decode password
openssl enc -nosalt -aes-256-cbc -d -iv ${AES_IV} -K ${AES_KEY} -in in.txt
this results in error message
AES_KEY 0602f470fa19438c1958e5a5e665c186532c35b6d92f6cefb166578ee1b2e08c
AES_IV 70612f7065b313cb7b092df06dea1415
ENCODED_PASSWORD 53b7adff6e85baedfa9dab80109ad67d
▒▒▒▒▒▒`$;▒▒▒▒%▒O▒Q▒▒▒S▒▒<7 7
bad decrypt
32624:error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length:evp_enc.c:518:
0602f470fa19438c1958e5a5e665c186532c35b6d92f6cefb166578ee1b2
e08c
I think that the problem is aes-key (ak) which has a newline char in it, which I am trying to remove with
|tr -d '\n'
a password should decode as
password
No, what's wrong is the encoding of in.txt. It should not be text at all, it should be binary.
In principle you would not get this error either if you'd use echo -n suppressing the final end-of-line within the ciphertext. The decryption would however still fail as it expects binary instead of an encoded binary value.
You may want to change the name of in.txt to in.bin if you want to keep a file. You should also be able to simply pipe the ciphertext into openssl using the standard input (stdin). In that case you may want to encode it first to store it in a shell variable and then decode before piping it to openssl to decrypt it.
For binary, use cat instead of echo.
If your $pass is long (more than 32 chars maybe), be sure to use openssl -A option, the reason is documented in the openssl manual.
With -A option, for encryption, the base64 encoded string will not be splitted in segments; For decryption, whole line is read to be decoded using base64.
Code example:
plaintxt='hello world"
pass=$(echo ${plaintxt} | openssl enc -aes-128-cbc -a -K ${AES_KEY} -iv ${AES_IV} -A )
echo "decoded password is: "
echo ${pass} | openssl enc -aes-128-cbc -d -a -K ${AES_KEY} -iv ${AES_IV} -A
this works as expected:
#!/usr/bin/env bash
#base64 encoded aes key, iv and password
ak=BgL0cPoZQ4wZWOWl5mXBhlMsNbbZL2zvsWZXjuGy4Iw=
iv=cGEvcGWzE8t7CS3wbeoUFQ==
pass=OfOXO+ruKFTCsBwGHynXwA==
#change format from base64 into hex, for openssl to consume, xxd -p -c32 is taking care of wrapping of the new lines
AES_KEY=$(echo ${ak} | openssl base64 -d | xxd -p -c32)
AES_IV=$(echo ${iv} | openssl base64 -d | xxd -p -c32)
echo "AES_KEY ${ak}"
echo "AES_IV ${iv}"
echo "encoded password ${pass}"
echo "decoded password is: "
echo ${pass} | openssl enc -aes-256-cbc -d -a -K ${AES_KEY} -iv ${AES_IV}
explanation:
openssl enc -aes-256-cbc -d
decode using aes-256-cbc algo
-a
in our case, it means that openssl will accept base64 encoded password
-K
aes key
-iv
aes iv

Generate random passwords in shell with one special character

I have the following code:
</dev/urandom tr -dc 'A-Za-z0-9##$%&_+=' | head -c 16
which is randomly generating passwords perfectly.
I want two changes:
It should only contain one special character listed above
It should choose a random length
I tried with length = $(($RANDOM%8+9))
then putting length as
</dev/urandom tr -dc 'A-Za-z0-9##$%&_+=' | head -c$length
but got no positive result.
#! /bin/bash
chars='##$%&_+='
{ </dev/urandom LC_ALL=C grep -ao '[A-Za-z0-9]' \
| head -n$((RANDOM % 8 + 9))
echo ${chars:$((RANDOM % ${#chars})):1} # Random special char.
} \
| shuf \
| tr -d '\n'
LC_ALL=C prevents characters like ř from appearing.
grep -o outputs just the matching substring, i.e. a single character.
shuf shuffles the lines. I originally used sort -R, but it kept the same characters together (ff1#22MvbcAA).
## Objective: Generating Random Password:
function random_password () {
[[ ${#1} -gt 0 ]] && { local length=${1}; } || { local length=16; }
export DEFAULT_PASSWORDLENGTH=${length};
export LC_CTYPE=C;
local random="$(
tr -cd "[:graph:]" < /dev/urandom \
| head -c ${length} \
| sed -e 's|\`|~|g' \
-e 's|\$(|\\$(|g';
)";
echo -e "${random}";
return 0;
}; alias random-password='random_password';
$ random-password 32 ;
)W#j*deZ2#eMuhU4TODO&eu&r)&.#~3F
# Warning: Do not consider these other options
# date +%s | sha256sum | base64 | head -c 32 | xargs -0;
# Output: MGFjNDlhMTE2ZWJjOTI4OGI4ZTFiZmEz
# dd if=/dev/urandom count=200 bs=1 2>/dev/null \
# | tr -cd "[:graph:]" \
# | cut -c-${length} \
# | xargs -0;
# Output: AuS*D=!wkHR.4DZ_la

Resources