Storage service API call - bash

I try to create a file share on an existing Azure storage account via bash script. I only have the account name and key, but don't want to use login credentials.
This is what I have so far:
#!/bin/sh
DATE_ISO=$(date +"%Y-%m-%dT%H:%M:%S")
VERSION="2015-02-21"
curl --header "x-ms-version: ${VERSION}" --header "x-ms-date: ${DATE_ISO}" --header "Authorization: SharedKey mystorageaccount:?????" https://mystorageaccount.file.core.windows.net/myshare?restype=share
The documentation says, "Authorization" is required (syntax: Authorization="[SharedKey|SharedKeyLite] <AccountName>:<Signature>") and "Signature" is a Hash-based Message Authentication Code (HMAC) constructed from the request and computed by using the SHA256 algorithm, and then encoded by using Base64 encoding.
So how do I generate this Signature?

Try this to create Share with bash script.
#!/bin/sh
STORAGE_KEY="$1"
STORAGE_ACCOUNT="$2"
SHARE_NAME="$3"
DATE_ISO=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
VERSION="2015-12-11"
HEADER_RESOURCE="x-ms-date:$DATE_ISO\nx-ms-version:$VERSION"
URL_RESOURCE="/$STORAGE_ACCOUNT/$SHARE_NAME\nrestype:share"
STRING_TO_SIGN="PUT\n\n\n\n\n\n\n\n\n\n\n\n$HEADER_RESOURCE\n$URL_RESOURCE"
DECODED_KEY="$(echo -n $STORAGE_KEY | base64 -d -w0 | xxd -p -c256)"
SIGN=$(printf "$STRING_TO_SIGN" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$DECODED_KEY" -binary | base64 -w0)
curl -X PUT \
-H "x-ms-date:$DATE_ISO" \
-H "x-ms-version:$VERSION" \
-H "Authorization: SharedKey $STORAGE_ACCOUNT:$SIGN" \
-H "Content-Length:0" \
"https://$STORAGE_ACCOUNT.file.core.windows.net/$SHARE_NAME?restype=share"
Try this to create Directory under the specified share.
#!/bin/sh
STORAGE_KEY="$1"
STORAGE_ACCOUNT="$2"
SHARE_NAME="$3"
DIRECTORY_NAME="$4"
DATE_ISO=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
VERSION="2015-12-11"
HEADER_RESOURCE="x-ms-date:$DATE_ISO\nx-ms-version:$VERSION"
URL_RESOURCE="/$STORAGE_ACCOUNT/$SHARE_NAME/$DIRECTORY_NAME\nrestype:directory"
STRING_TO_SIGN="PUT\n\n\n\n\n\n\n\n\n\n\n\n$HEADER_RESOURCE\n$URL_RESOURCE"
DECODED_KEY="$(echo -n $STORAGE_KEY | base64 -d -w0 | xxd -p -c256)"
SIGN=$(printf "$STRING_TO_SIGN" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$DECODED_KEY" -binary | base64 -w0)
curl -X PUT \
-H "x-ms-date:$DATE_ISO" \
-H "x-ms-version:$VERSION" \
-H "Authorization: SharedKey $STORAGE_ACCOUNT:$SIGN" \
-H "Content-Length:0" \
"https://$STORAGE_ACCOUNT.file.core.windows.net/$SHARE_NAME/$DIRECTORY_NAME?restype=directory"

Related

binance "code":-1022,"msg":"Signature for this request is not valid. shell script

hi guys i try to many day to run this code but returnme always code":-1022,"msg":"Signature for this request is not valid , but i am sure is correct a code , anyone can controll my script for see if is all ok ?? thanks at all for advice
APIKEY="MY APIKEY"
APISECRET="MY SECRET KEY"
URLPART2="symbol=BNBUSDT&side=BUY&type=LIMIT&quoteOrderQty=10&price=270.3&stopPrice=270.3&newOrderRespType=FULL"
RECVWINDOW=50000
RECVWINDOW="recvWindow=$RECVWINDOW"
TIMESTAMP="timestamp=$(( $(date +%s) *1000))"
QUERYSTRING="&$URLPART2&$RECVWINDOW&$TIMESTAMP"
SIGNATURE=$(echo -n "$QUERYSTRING" | openssl dgst -sha256 -hmac $APISECRET | cut -c 10-)
SIGNATURE="signature=$SIGNATURE"
#curl -s -H "X-MBX-APIKEY: $APIKEY" "https://api.binance.com/api/v3/order/test?$URLPART2&$RECVWINDOW&$TIMESTAMP&$SIGNATURE"
curl -H "X-MBX-APIKEY: $APIKEY" -X POST "https://api.binance.com/api/v3/order/test?$URLPART2&$RECVWINDOW&$TIMESTAMP&$SIGNATURE"
echo```

Hashicorp Vault RSASSA-PSS Prehashed cannot be verified with OpenSSL

I am trying to use Hashicorp Vault to sign a file with RSASSA-PSS-4096. The file is too big for sending it to the server directly, so I want to prehash it locally and then send the digest via POST request to the Vault transit engine.
While the Vault signature verification works, the OpenSSL verification fails.
Please see my drafted script:
# Calculate SHA256 hash and convert to base64
sha256sum_base64=$(openssl dgst -sha256 -binary $1 | base64)
# Sign Hash Value with Vault
json_response=$(curl -s \
--header "X-Vault-Token: $(cat token)" \
--request POST \
--data-binary '{"input": "'"$sha256sum_base64"'", "prehashed": true, "signature_algorithm": "pss", "hash_algorithm": "sha2-256"}' \
http://127.0.0.1:8200/v1/transit/sign/rsa_4096)
# Extract base64 signature from the json response.
signature_base64=$(echo $json_response | python3 -c "import sys, json; print(json.load(sys.stdin)['data']['signature'])" | cut -d ":" -f 3)
# Convert signature from base64 to binary and write to file
sigfile=$1__signature.bin
echo $signature_base64 | openssl base64 -d -A -in - -out $sigfile
# Check whether signature is valid via OpenSSL
echo "OpenSSL --> " $(openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:32 -verify rsa_4096_pub.pem -signature $sigfile $1)
# Check whether signature is valid via Vault
signature_vaultformat="vault:v1:$signature_base64"
verify_response=$(curl -s \
--header "X-Vault-Token: $(cat token)" \
--request POST \
--data-binary '{"input": "'"$sha256sum_base64"'", "signature": "'"$signature_vaultformat"'", "prehashed": true, "signature_algorithm": "pss", "hash_algorithm": "sha2-256"}' \
http://127.0.0.1:8200/v1/transit/verify/rsa_4096)
echo "Vault Verify --> " $(echo $verify_response | python3 -c "import sys, json; print(json.load(sys.stdin)['data']['valid'])")
What could be the problem here? I played with rsa_pss_saltlen parameters (e.g. -1) without success. Is there another OpenSSL parameter I am missing? Do I need to consider something for EMSA-PSS?
Here is a proof-of-concept where you can sign a piece of text using the Transit secrets engine and then verify the signature using openssl rather than using the Transit secrets engine again.
# Define our plaintext
TEXT="abc123"
# Encode our plaintext with base64
B64_ENCODED_TEXT=$(echo $TEXT | base64)
# Reset the transit secrets engine
vault secrets disable transit
vault secrets enable transit
# Create a key called 'test' using 'rsa-2048'
vault write -f transit/keys/test \
type='rsa-2048'
# Export the public key from the transit secret engine key named 'test'
PUBLIC_KEY=$(vault read -format=json transit/keys/test | \
jq -r '.data.keys."1".public_key')
# Sign our base64 encoded text using our transit key named 'test' and
# capture the signature
SIGNATURE=$(vault write -format=json transit/sign/test/sha2-256 \
input="$B64_ENCODED_TEXT" \
signature_algorithm="pss" | \
jq -r '.data.signature')
# Demonstrate that we can use transit to verify our signature
printf "\nVerifying signature using Vault Transit...\n"
vault write transit/verify/test/sha2-256 \
signature_algorithm="pss" \
input=$B64_ENCODED_TEXT \
signature=$SIGNATURE
# Write out public key to a file
echo $PUBLIC_KEY > publickey.pem
# Remove the metadata from the Vault supplied signature and decode the
# signature using base64, writing the raw signature to a file
echo $SIGNATURE | cut -d':' -f3 | base64 -d > sig
# Write the non-encoded plaintext to a file
echo "$TEXT" > mytext
# Use openssl to verify the signature using the base64 decoded raw signature
# along with the public key and the non-encoded plaintext
printf "\nVerifying signature using openssl...\n"
openssl dgst \
-sha256 \
-verify publickey.pem \
-signature sig \
-sigopt rsa_padding_mode:pss \
mytext
Some important notes below:
Note that ALL data that is signed by Vault Transit secret engine must first be base64 encoded.
When using openssl to verify a signature, you must make sure that you are using the correct signature algorithm.
When Vault provides a signature, it's in the following format: vault:v1:8SDd3WHDOjf7mq69... where vault denotes that it was signed by Vault, v1 denotes the version of the key and the final part is the actual signature that is encoded using base64. The openssl utility requires that the signature is binary and not base64. In order to verify this signature with openssl, you must remove the first 2 parts of the Vault provided signature. You must then decode the base64 encoded signature and use the resultant binary signature when verifying with openssl.
When verifying with openssl you can not use use the base64 encoded version of the text, you must use the non-base64 encoded plaintext.

Loop through list for curl requests in bash

I have a bash script that sends a curl request and displays the response.
#!/bin/bash
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "V.S.23164117.AVG.10M"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$p"
My current output is just a value and the name of the tagId.
My query only allows for 1 tagId ( you can see above )
I want to be able to set a list of tagId's.
Then when I run this script it should loop through the list of tagId's and execute the curl request replacing the V.S.23164117.AVG.10M with each value
in the list.
Then output the entire list of results into a file.
list would be like so - (I would love to be able to enter this list into a seperate file and the bash script calls that file. Sometimes this list can be a few hundred lines.
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
V.S.23164115.AVG.10M
V.S.23164114.AVG.10M
output would like look so.
value tagId
value tagId
value tagId
100 V.S.23164117.AVG.10M
etc..
thank you for any help
You can loop over list of tags using a small script. I'm not 100% clean of the output format. You can change the 'echo' to match the required format.
Note minor change to quotes to allow variable expansion in the body.
The tags will be stored in a file, for examples, tags.txt
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
And the script will be use the file
#! /bin/bash
# Use user defined list of tags
tags=tags.txt
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
for tag in $(<$tags) ; do
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "'"$tag"'"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$tag $p"
done

"Invalid credentials" while doing a curl POST

I have a curl request in below format
curl -v -H "Content-Type:application/json" -H "x-user-id:xxx" -H "x-api-key:yyy" --data '{"logs":"'"${TEST_OUTPUT}"'","pass":"true | false"}' https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
This works fine while I do from my MAC terminal. But the same command throws
13:49:26 {
13:49:26 "status": "error",
13:49:26 "message": "Invalid credentials"
13:49:26 }
I saw this post but not sure how else would I send a json body without curly braces. I know that we can save it as a file.json and use the file as body.But for some reasons that cannot be implemented in my scenario
In general, you should avoid trying to build JSON using string interpolation. Use a tool like jq to handle any necessary quoting.
jq -n --argson o "$TEST_OUTPUT" '{logs: $o, pass: "true | false"}' |
curl -v -H "Content-Type:application/json" \
-H "x-user-id:xxx" \
-H "x-api-key:yyy" \
--data #- \
https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
However, if you can manage to correctly generate your JSON as you are now, you can just replace the jq command with echo:
echo '{"logs": ...' | curl ...
The #- argument to --data says to read from standard input.

Bash curl sign hmac

I'm trying to use the Bittrex API. The only example provided is the following. I'm not even sure what language this is. I'm trying to replicate this in bash. Full API detail is located here https://bittrex.com/Home/Api
$apikey='xxx';
$apisecret='xxx';
$nonce=time();
$uri='https://bittrex.com/api/v1.1/market/getopenorders?apikey='.$apikey.'&nonce='.$nonce;
$sign=hash_hmac('sha512',$uri,$apisecret);
$ch = curl_init($uri);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('apisign:'.$sign));
$execResult = curl_exec($ch);
$obj = json_decode($execResult);
I have tried several things but here is the latest I tried.
#Bash
apikey="mykey"
secret="mysecret"
nonce=`date +%s`
uri="https://bittrex.com/api/v1.1/market/getopenorders?apikey=$apikey&nonce=$nonce"
apisig=`echo -n "$uri" | openssl dgst -sha512 -hmac "$secret"`
curl -sG https://bittrex.com/api/v1.1/market/getopenorders?nonce="$nonce"&apikey="$apikey"&apisig="$apisig"
I get "{"success":false,"message":"APIKEY_NOT_PROVIDED","result":null}"
What you're missing are:
escaping & in query string
passing digest as header rather than parameter
So the code that worked for me is:
#!/bin/bash
apikey="mykey"
secret="mysecret"
nonce=`date +%s`
uri="https://bittrex.com/api/v1.1/market/getopenorders?apikey=$apikey&nonce=$nonce"
apisig=`printf %s "$uri" | openssl dgst -sha512 -hmac "$secret"| sed 's/^.*= //'`
curl -sG $uri --header "apisign: $apisig"

Resources