Using bash i am executing following commands to filter information from a certificate
openssl s_client -connect google.com:443 < /dev/null > cert.pem
openssl x509 -in cert.pem -noout -subject > commonName
tr "," "\n" < commonName | sed -nr '/CN/p' | tr -d ' /t' > commonName
The last command leaves the file "commonName" empty, i wonder why this is.
If i instead append the file ">>" the desired filtered output is shown but unfiltered content remains.
Leaves file empty
tr "," "\n" < commonName | sed -nr '/CN/p' | tr -d ' /t' > commonName
Works but with undesired content
tr "," "\n" < commonName | sed -nr '/CN/p' | tr -d ' /t' >> commonName
Edit,
Might add that sending to a file with a new name works as intended. Changing "commonName" to "test" for example.
Thanks in advance!
/R
You can't edit the file with other commands and sed in only one pipe (the way you do it). You need a temporary file:
openssl s_client -connect google.com:443 < /dev/null > cert.pem
openssl x509 -in cert.pem -noout -subject > commonName
tr "," "\n" < commonName | sed -nr '/CN/p' | tr -d ' /t' > /tmp/temp
mv /tmp/temp commonName
And a better way to achieve the whole script, without temporary file:
openssl s_client -connect google.com:443 < /dev/null > cert.pem
openssl x509 -in cert.pem -noout -subject |
tr "," "\n" |
grep -o 'CN .*' > commonName
Related
The following contents in my shell file :
#!/usr/bin/env bash
api_params_with_timestamp="somethingXXXX"
SECRET_KEY="XXXXXXXXXXXXXX"
signature=$(echo -n "$api_params_with_timestamp" \
| openssl dgst -sha256 -hmac "$SECRET_KEY")
echo "signature"=$signature
execute this shell I got
signature=(stdin)= 3eee8d204c83381dddffgggg
I just need "3eee8d204c83381dddffgggg" part , what can I do
so that (stdin)= part would be skip ?!
Edit:
signature=$(echo -n "$api_params_with_timestamp" \
| openssl dgst -sha256 -hmac "$SECRET_KEY" | cut -c10-1000)
not perfect but works !
The openssl-dgst page doesn't provide a direct option to drop the "(stdin)= " portion of the output, but that can be done trivially with cut, grep or sed. For example:
with cut
signature=$(echo -n "$api_params_with_timestamp" \
| openssl dgst -sha256 -hmac "$SECRET_KEY" \
| cut -d ' ' -f2)
with grep
signature=$(echo -n "$api_params_with_timestamp" \
| openssl dgst -sha256 -hmac "$SECRET_KEY" \
| grep -E -o '[^ ]+$')
with sed
signature=$(echo -n "$api_params_with_timestamp" \
| openssl dgst -sha256 -hmac "$SECRET_KEY" \
| sed 's/^.*[[:space:]]//')
Note: you can also remove the line separator characters "\" and use the pipe for that purpose instead, e.g.
signature=$(echo -n "$api_params_with_timestamp" |
openssl dgst -sha256 -hmac "$SECRET_KEY" |
sed 's/^.*[[:space:]]//')
Either way is fine -- up to you.
Example Use/Output
$ api_params_with_timestamp="somethingXXXX"
> SECRET_KEY="XXXXXXXXXXXXXX"
> signature=$(echo -n "$api_params_with_timestamp" |
> openssl dgst -sha256 -hmac "$SECRET_KEY" |
> sed 's/^.*[[:space:]]//')
> echo $signature
46c86ea27df85b8ab9a8c565a344cadd1230f32ff1d773ba0fa5aa1076116d0b
I'm trying to write a bash script to get access tokens from Microsoft for my app registered in AD. I can't seem to get past this error though:
"error":"invalid_client","error_description":"AADSTS700027: The certificate with identifier used to sign the client assertion is not registered on application. [Reason - The key was not found., Thumbprint of key used by client: '356134 ...
The full gist can be found here: https://gist.github.com/smaring/3a3a6779a809beecc39624aada6e2b88
Here are some of the juicy bits ...
$ openssl pkcs12 -in <your-app>.pfx -out <your-app>.pem
$ openssl x509 -outform der -in <your-app>.pem -out ${PUBLIC_CERT_FILE}
$ openssl rsa -in <your-app>.pem -out ${PRIVATE_KEY_FILE}
x5t="$(sha1sum ${PUBLIC_CERT_FILE} | awk '{print $1;}' | openssl base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$// )"
read -r -d '' HEADER <<EOF
{
"alg": "RS256",
"typ": "JWT",
"x5t": "${x5t}"
}
EOF
HEADER_NO_WHITESPACE=$(echo "${HEADER}" | sed ':a; N; s/[[:space:]]//g; ta')
BASE64_ENCODED_HEADER=$(echo ${HEADER_NO_WHITESPACE} | openssl base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$// )
read -r -d '' PAYLOAD <<EOF
{
"aud": "https: //login.microsoftonline.com/${TENANT_ID}/oauth2/v2.0/token",
"exp": ${exp},
"iss": "${CLIENT_ID}",
"jti": "${jti}",
"nbf": ${nbf},
"sub": "${CLIENT_ID}"
}
EOF
PAYLOAD_NO_WHITESPACE=$(echo "${PAYLOAD}" | sed ':a; N; s/[[:space:]]//g; ta')
BASE64_ENCODED_PAYLOAD=$(echo ${PAYLOAD_NO_WHITESPACE} | openssl base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$// )
SIGNATURE=$( echo -n "${BASE64_ENCODED_HEADER}.${BASE64_ENCODED_PAYLOAD}" | \
openssl dgst -sha256 -binary -sign <(cat ${PRIVATE_KEY_FILE}) | \
openssl base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$// )
CLIENT_ASSERTION="${BASE64_ENCODED_HEADER}.${BASE64_ENCODED_PAYLOAD}.${SIGNATURE}"
curl -s -X POST \
--header \"Content-Type: application/x-www-form-urlencoded\" \
-d \"\
scope=https%3A%2F%2Fgraph.microsoft.com%2F.default&\
grant_type=client_credentials&\
client_assertion_type=urn%3Aietf%3Aparams%3Aoauth%3Aclient-assertion-type%3Ajwt-bearer&\
client_id=${CLIENT_ID}&\
client_assertion=${CLIENT_ASSERTION}\" \
https://login.microsoftonline.com/${TENANT_ID}/oauth2/v2.0/token"
After getting a Java version of this working I figured out that I was not generating the thumbprint properly. The fix was this:
thumbprint=$(openssl x509 -in ${publicCertFile} -fingerprint -noout | awk '{split($0,a,"="); print a[2]}' )
thumbprintHex=$(echo -e "\\x${thumbprint}" | sed 's/:/\\x/g' )
thumbprintHexString=$(echo -e "${thumbprint}" | sed 's/://g' )
x5t=$(echo -ne "${thumbprintHex}" | openssl base64 | sed s/\+/-/g | sed 's/\//_/g' | sed -E s/=+$// )
I have updated the gist to the fully working bash script.
Construct a JWT assertion in bash using client-credential with cert to get an access token from Microsoft Azure Directory --> https://gist.github.com/smaring/3a3a6779a809beecc39624aada6e2b88
I wrote a script. Please pardon that I am not an expert in scripting.
Upon deciphering, the results gotten truncated.
[Message in text]: 0123456789abcdefghijklmnopqrstuvwxyz
message_input in hex: 303132333435363738396162636465666768696a6b6c6d6e6f707172737475767778797a0a
key: 788a1ca0bf1ab80f092841aabd77793f
hex string is too short, padding with zero bytes to length
c19f83afc1160ce81b0fc9906d513693386ccdd313b0f2884c698411441054e8
ciphered text: c19f83afc1160ce81b0fc9906d513693386ccdd313b0f2884c698411441054e8
IV: 7ecd3d63a8b74bb2f80d71a1c9d43359
deciphering ...
hex string is too short, padding with zero bytes to length
key: 788a1ca0bf1ab80f092841aabd77793f
iv: 7ecd3d63a8b74bb2f80d71a1c9d43359
answer: 30313233343536373839616263646566
Deciphered Message in hex: 30313233343536373839616263646566
deciphered text: 0123456789abcdef
The recovered deciphered text: 0123456789abcdef, ghijklmnopqrstuvwxyz gets truncated. This is supposed to be AES-CBC. Is there an option I did not turn on?
Here is the ciphering:
IV=$(openssl rand -hex 16)
get_key_for_ciphering; # key_for_ciphering gets populated
message_input=$(echo -n "${message_input//[[:space:]]/}") # remove spaces
echo "message_input in hex: "$message_input
echo "key": $key_for_ciphering;
ANS=$(echo "0: $message_input" | xxd -r | openssl enc -aes-256-cbc -iv $IV -K "$key_for_ciphering" | xxd -p)
ANS=$(echo -n "${ANS//[[:space:]]/}") # remove spaces
Here is the deciphering (message_input=$ANS):
get_key_for_ciphering; # key_for_ciphering gets populated
ANS=$(echo "0: $message_input" | xxd -r | openssl enc -aes-256-cbc -d -nopad -nosalt -K "$key_for_ciphering" -iv $IV | xxd -p) # -nopad -nosalt
Focusing in your question, the problem is in the xxd command. When converting a hex string to binary with xxd -r, you have to use the -p to tell xxd that is a plain hex string (no line breaks).
When converting back to hex with xxd -p, line breaks are added every 32 bytes. Unfortunately, xxd doesn't provide a flag to not include line breaks (you could use -c to set the number of columns, but it's limited to a max number). There are many option to remove line breaks, but one is appending a | tr -d '\n' to your command, as shown in the example below.
IV=$(openssl rand -hex 16)
key_for_ciphering=$(openssl rand -hex 16)
message_input="303132333435363738396162636465666768696a6b6c6d6e6f707172737475767778797a0a"
message_input=$(echo -n "${message_input//[[:space:]]/}") # remove spaces
echo "Message: $message_input"
echo "Key: $key_for_ciphering"
echo "IV: $IV"
ANS=$(echo "0: $message_input" | xxd -r -p | openssl enc -aes-256-cbc -iv $IV -K "$key_for_ciphering" | xxd -p | tr -d '\n')
ANS=$(echo -n "${ANS//[[:space:]]/}") # remove spaces
echo "Encrypted: $ANS"
ANS=$(echo "0: $ANS" | xxd -r -p | openssl enc -aes-256-cbc -d -nopad -nosalt -K "$key_for_ciphering" -iv $IV | xxd -p | tr -d '\n')
echo "Decrypted: $ANS"
---- Edit: ----
Doesn't work, as shell-parameters cannot contain binary zero. Possible fix with filters:
#!/bin/bash
tohex () {
perl -e 'binmode STDIN; while (<STDIN>) { print unpack "H*",$_; }'
}
fromhex () {
perl -e 'binmode STDIN; while (<STDIN>) { print pack "H*",$_; }'
}
binInput='0123456789abcdefghijklmnopqrstuvwxyz'
hexIV="$(openssl rand -hex 16)"
hexKey='788a1ca0bf1ab80f092841aabd77793f'
hexCipher="$(printf '%s' "$binInput" |\
openssl enc -aes-256-cbc -nosalt -iv "$hexIV" -K "$hexKey" | tohex)"
binResult="$(printf '%s' "$hexCipher" | fromhex |\
openssl enc -aes-256-cbc -d -iv "$hexIV" -K "$hexKey")"
if [ "$binInput" = "$binResult" ]; then echo OK;
fi
---- Original: ----
I think your problem lies in hexadecimal conversion. Try using perl pack/unpack:
tohex () {
perl -e 'print unpack "H*", "$ARGV[0]"' "$1"
}
fromhex () {
perl -e 'print pack "H*", "$ARGV[0]"' "$1"
}
message='0123456789abcdefghijklmnopqrstuvwxzy ยง"+!%/=()'
message_hex=$(tohex "$message")
message_cmp=$(fromhex "$message_hex")
if [ "$message" = "$message_cmp" ]; then echo OK; fi
I am going to run acme-tiny on a central webserver in order to get the certificates for my two Nginx reverse proxies issued. The newly created certificates are published over https and available to the reverse proxies via download.
I want the Nginx servers to check the newly created certificates before replacing the old certificates with the new ones. For this purpose, I wrote the following bash script, which is run on each Nginx server. I would like to know whether I missed something or whether you have ideas for improvement. Or is there a better way to realise this?
#!/bin/bash
set -e
# Commands for deriving the public keys:
# openssl ec -in ecdsa.key -pubout > ecdsa_public_key.pem
# openssl rsa -in rsa.key -pubout > rsa_public_key.pem
curl -O https://example.org/ecdsa.pem
curl -O https://example.org/intermediate.pem
curl -O https://example.org/rsa.pem
# Are the certificates not expired?
# Have they (ecdsa.pem, rsa.pem) been recently issued (validity >= 80 days)?
openssl x509 -checkend 6912000 -noout -in intermediate.pem
openssl x509 -checkend 6912000 -noout -in ecdsa.pem
openssl x509 -checkend 6912000 -noout -in rsa.pem
# Do the private keys and certificates belong together?
openssl x509 -in ecdsa.pem -pubkey | \
sed -n '/-----BEGIN PUBLIC KEY-----/,/-----END PUBLIC KEY-----/p' | \
cmp - ecdsa_public_key.pem
openssl x509 -in rsa.pem -pubkey | \
sed -n '/-----BEGIN PUBLIC KEY-----/,/-----END PUBLIC KEY-----/p' | \
cmp - rsa_public_key.pem
# Is the certificate chain valid?
openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt intermediate.pem
openssl verify -CAfile intermediate.pem ecdsa.pem
openssl verify -CAfile intermediate.pem rsa.pem
# Are the certificates issued for the correct domain names?
(openssl x509 -noout -subject -in ecdsa.pem | awk -F'CN=' '{print $2}' | \
awk -F'/' '{print $1}'; \
openssl x509 -noout -text -in ecdsa.pem | grep "^[[:space:]]*DNS:" | \
xargs | tr ' ' '\n' | grep ^DNS | sed 's/^DNS://' | sed 's/,$//')| \
sort | uniq | cmp - domains.txt
(openssl x509 -noout -subject -in rsa.pem | awk -F'CN=' '{print $2}' | \
awk -F'/' '{print $1}'; \
openssl x509 -noout -text -in rsa.pem | grep "^[[:space:]]*DNS:" | \
xargs | tr ' ' '\n' | grep ^DNS | sed 's/^DNS://' | sed 's/,$//')| \
sort | uniq | cmp - domains.txt
UPDATE 1:
As I have stated I want to check every aspect of the certificate not just the validity of the certificate chain. It currently checks:
the chains validity,
whether private key and certificate match,
whether the cert has been recently issued making replacement of the older cert necessary
whether the cert has been issued for the desired domain names
Therefore, it isn't a duplicate to the linked questions. And, I would like to know whether I have forgotten some check or whether there is space for improvement. Below is an update to the bash script. Beware I have not tested it thoroughly, yet.
#!/bin/bash
set -e
# Execute the script:
# ( cd /path_to_workdir && \
# su - james -c "./check_cert.sh https://www.example.org intermediate.pem ecdsa.pem ecdsa_pubkey.pem /etc/ssl/certs/ca-certificates.crt domains.txt pass.txt" && \
# su - james -c "./check_cert.sh https://www.example.org intermediate.pem rsa.pem rsa_pubkey.pem /etc/ssl/certs/ca-certificates.crt domains.txt pass.txt" && \
# cat ecdsa.pem intermediate.pem > /etc/nginx/ssl/ecdsa_bundle.pem ) && \
# cat rsa.pem intermediate.pem > /etc/nginx/ssl/rsa_bundle.pem ) && \
# /etc/init.d/nginx reload
# Download URL, e.g. https://example.org
URL=$1
# Intermediate certificate of certificate chain
INTERMEDIATE=$2
# Issued certificate, e.g. ecdsa.pem
CERT=$3
# Public key derived from private key via:
# openssl ec -in ecdsa.key -pubout > ecdsa_pubkey.pem
# openssl rsa -in rsa.key -pubout > rsa_pubkey.pem
PUBKEY=$4
# ca-certificates file, e.g. /etc/ssl/certs/ca-certificates.crt
CACERTS=$5
# Domains that the cert should cover
DOMAINS=$6
# password-file.txt possible content:
# machine example.org login james password H3Llo
PASS=$7
# Download files if newer than local.
if [[ $(curl -s -O --netrc-file "$PASS" -w "%{http_code}" -z "$CERT" "$URL/$CERT") -eq 304 ]]; then
echo "No new certificate issued. Nothing to do!"
exit 1
fi
curl -s -O --netrc-file "$PASS" -z "$INTERMEDIATE" "$URL/$INTERMEDIATE"
# Is the certificate chain valid?
openssl verify -CAfile "$CACERTS" -untrusted "$INTERMEDIATE" "$CERT"
# Do the private keys and certificates belong together?
openssl x509 -in "$CERT" -pubkey | \
sed -n '/-----BEGIN PUBLIC KEY-----/,/-----END PUBLIC KEY-----/p' | \
cmp - "${PUBKEY}"
# Has the Let's Encrypt certificate been recently issued (validity >= 80 days)?
openssl x509 -checkend 6912000 -noout -in "$CERT"
# Are the certificates issued for the correct domain names?
(openssl x509 -noout -subject -in "$CERT" | awk -F'CN=' '{print $2}' | \
awk -F'/' '{print $1}'; \
openssl x509 -noout -text -in "$CERT" | grep "^[[:space:]]*DNS:" | \
xargs | tr ' ' '\n' | grep ^DNS | sed 's/^DNS://' | sed 's/,$//')| \
sort | uniq | cmp - "$DOMAINS"
The code might be run via cronjob:
0 1 * * * /usr/local/bin/update_cert.sh >/dev/null 2>&1
The content of update_cert.sh might be:
#!/bin/bash
( cd /path_to_workdir && \
su - james -c "./check_cert.sh https://www.example.org intermediate.pem ecdsa.pem ecdsa_pubkey.pem /etc/ssl/certs/ca-certificates.crt domains.txt pass.txt" && \
su - james -c "./check_cert.sh https://www.example.org intermediate.pem rsa.pem rsa_pubkey.pem /etc/ssl/certs/ca-certificates.crt domains.txt pass.txt" && \
cat ecdsa.pem intermediate.pem > /etc/nginx/ssl/ecdsa_bundle.pem ) && \
cat rsa.pem intermediate.pem > /etc/nginx/ssl/rsa_bundle.pem ) && \
/etc/init.d/nginx reload
I'm trying to figure out some Bash-foo to check that the output of three different commands is identical. I can do this with several lines of a Bash script, I'm just wondering if what I want to do is possible in one line with some fancy shell I/O redirection.
What I want to do is check that an SSL certificate matches up with a particular key and certificate signing request.
The commands look like this:
openssl x509 -noout -modulus -in certificate.crt | openssl md5
openssl rsa -noout -modulus -in privateKey.key | openssl md5
openssl req -noout -modulus -in CSR.csr | openssl md5
If the key, cert, and csr match up, all three of those commands should spit out identical output, like: "(stdin)= 95ce143e8418cf8a4f7dd718983ed4eb".
Here's a prototype:
[[ $(echo -e "blah\nblah\nblah" | uniq | wc -l) -eq 1 ]]
But I can't get from there to the final product. This doesn't work:
[[ $(openssl x509 -noout -modulus -in certificate.crt | openssl md5 && openssl rsa -noout -modulus -in privateKey.key | openssl md5 && openssl req -noout -modulus -in CSR.csr | openssl md5 | uniq | wc -l) -eq 1 ]]
One problem is maybe that my prototype generates all three lines of output from one command, but the real thing uses && a couple times.
cmp -s <( cmd1) <(cmd2) && cmp -s <( cmd1) <(cmd3)
Note that this construct executes cmd1 two times.
If you require single exec of each cmd, more complicated line would look something like:
cmd1|tee >( cmp -s <(cmd2) )|cmp -s <(cmd3)
Also for the second one, checking the result is complicated (you have to check PIPESTATUS array)
The problem is that you're piping only the last command "sub-pipeline" into uniq. Try this:
[[ $( { openssl x509 -noout -modulus -in certificate.crt | openssl md5 && openssl rsa -noout -modulus -in privateKey.key | openssl md5 && openssl req -noout -modulus -in CSR.csr | openssl md5; } | uniq | wc -l) -eq 1 ]]
The curly braces make the three "sub-pipelines" act as if together they are one command as far as uniq is concerned, analogously to your echo prototype.
You probably meant something like
((1 == $( ( echo 1 && sleep 1; echo 1 && sleep 1) | sort -u | wc -l ) ))
But it might be easier to do
x1=$( command1 )
x2=$( command2 )
x3=$( command3 )
if [[ $x1 == $x2 && $x2 == $x3 ]] ; then
echo The same.
fi
Edit:
This form should work as well, but reduce the number of stored variables.
x1=$(command1)
[[ $x1 == $(command2) && $x1 == $(command3) ]] && echo match