I have implemented a CURL script to execute API call which requires encrypted string as data, But it gets failed with 'failed to decrypt request'. I have tried with python also (requests and Crypto modules) but failed with same error message.
#!/usr/bin/env bash
output=$(openssl enc -aes-256-cbc -pass pass:xxxxxxx -in msg.txt -base64)
curl -X POST \
"https://api.swing.tradesmartonline.in/api/v1/login-token" \
-H "Content-Type: application/json" \
-d '{"data":"$output","app":"APPLICATION_ID"}'
Output is: {"status":"false","data":"","error_msg":"failed to decrypt
request."}
Where as I am able to decrypt the message in local using below command
echo "$output" | openssl enc -aes-256-cbc -base64 -pass pass:xxxxxxx -d
Thanks in advance
The shell will not expand your variables in a single quoted string. Either:
Use double quotes and escape embedded double quotes with \":
"{\"data\":\"$output\",\"app\":\"APPLICATION_ID\"}"
Use single quotes for static text and double quotes for what contain variables (this means less escaping):
'{"data":"'"$output"'","app":"APPLICATION_ID"}'
Related
I am trying to invoke the API using the curl command but unfortunately, I won't be able to keep the files. I am trying to pass the .key and .pem file's data in the command but I am not able to pass that correctly. Below is my command in my .sh file:
response=$(curl --key "$5" --cert "$6" -k -X "$2" -d "$payload" "$4")
I am calling the script below way:
key="${key}"
pem="${pem}"
bash ./Integration1.sh Provision POST "$payload" https://some-api.com/pr "$key" "$pem"
It gives the below error:
curl: (58) could not load PEM client certificate from -----BEGIN CERTIFICATE-----
This command works fine if I pass the file directly so, is there any way to pass the data via string variables in the curl command?
If you only have your key data in a variable and can't write it to a file yourself for some reason, an alternative solution is to use process substitution.
bash ./Integration1.sh Provision POST "$payload" https://some-api.com/pr \
<(printf '%s' "$key") \
<(printf '%s' "$pem")
This requires bash and still uses files under the hood, but it doesn't require you to manage the files yourself or know where they're located.
--key and --cert take the name of a file containing certificate data, not the certificate data itself.
... "$(cat my_client.key)" "$(cat my_client.pem)"
Should just be
... my_client.key my_client.pem
I am trying to use an expect script to pull configs from cisco devices. Dont ask why the client does not use a standard tool.
My issue is that they wish to use encrypted passwords for the session. I can encrypt and salt the password into a file and then use cat within the expect script to pull out the password into a variable... see code below, however, the exec command puts the string value into single quotes. Expect scripts I believe treat single quotes differently to double and certainly not like a string.
I can validate that the rest of the code works by putting the raw password into the script as a substitude and if I use 'puts' to display the variable it displays
'password'
My thought as I said is that the exec command is then putting the string back with the single quotes around them.
I have tried SED and TR piped into the end of the command string but that still gives the same result as I would expect.
Any thoughts
set salt "mrsalty"
set admincmd "cat /mrpwd.txt | openssl enc -aes-256-cbc -md sha512 -a -d -salt -pass pass:$salt"
set password [ exec sh -c $admincmd ]
(This is not really an answer, more of a long, formatted comment)
the exec command puts the string value into single quotes
No it does not.
A demo:
First, create an encrypted password file in the shell
$ salt=mrsalty
$ echo MyPass1234 | openssl enc -aes-256-cbc -md sha512 -a -e -salt -pass pass:$salt > pw.encrypted
$ cat pw.encrypted
U2FsdGVkX18bFSAmQbXqGjeSBVtE8AzJ2K8Lif4muB8=
Now in expect
expect1.1> set salt mrsalty
mrsalty
expect1.2> set file pw.encrypted
pw.encrypted
expect1.3> set password [exec openssl enc -aes-256-cbc -md sha512 -a -d -salt -pass pass:$salt -in $file]
MyPass1234
No single quotes there.
Even with storing the command in a variable and using sh
expect1.4> set admincmd "cat $file | openssl enc -aes-256-cbc -md sha512 -a -d -salt -pass pass:$salt"
cat pw.encrypted | openssl enc -aes-256-cbc -md sha512 -a -d -salt -pass pass:mrsalty
expect1.5> set password [exec sh -c $admincmd]
MyPass1234
How are you using $password later in the expect code?
I am optionally including a user and password in a curl request as follows:
declare creds=""
if [ -n "$user" ] && [ -n "$password" ]; then
creds="-u ${user}:${password}"
fi
output=$(curl ${creds} -X PUT -v --write-out "%{http_code}" "$url" \
-H 'Content-Type: application/json' -s -o /dev/null --data "${payload}")
This seems to work fine, but I'm getting this shellcheck warning:
Double quote to prevent globbing and word splitting
https://github.com/koalaman/shellcheck/wiki/SC2086
Puting quotes around it doesn't work, e.g. if I do this:
output=$(curl "${creds}" -X PUT -v --write-out "%{http_code}" "$url" \
-H 'Content-Type: application/json' -s -o /dev/null --data "${payload}")
then when the username and password are not supplied, this results in empty double quotes in the curl request curl "" -X PUT ..., which generates a <url> malformed error.
I could use an if-else for the curl command, but I'd rather avoid the duplication. Is the above approach acceptable despite the shellcheck warning?
You were doing right in putting quotes around the variable, but shellcheck doesn't catch the issue of storing commands in a variable which has its own downfalls. Since this being a an issue with function of the shell, shellcheck can't quite catch it out-of-the-box. When you did below
creds="-u ${user}:${password}"
and quoted "$creds", it is passed as one single argument word to curl instead of being broken down to -u and "${user}:${password}" separately. The right approach should have been to use an array to store the commands and expand it, so that the words are preserved and not split by the shell (foremost reason to quote the variable, as indicated by shellcheck)
creds=(-u "${user}:${password}")
and invoke
curl "${creds[#]}" <rest-of-the-cmd>
Also explore the following
I'm trying to put a command in a variable, but the complex cases always fail!
How to store a command in a variable in a shell script?
I have the following bash script that uses its arguments to hit a RESTful web service (via curl) and prints out both the curl request made as well as the response:
#! /bin/bash
# arguments:
# $1 - username
# $2 - password
#
# outputs:
# if the script exits with a non-zero status then something went wrong
# verify that we have all 6 required arguments and fail otherwise
if [ "$#" -ne 2 ]; then
echo "Required arguments not provided"
exit 1
fi
# set the script arguments to meaningful variable names
username=$1
password=$2
# login and fetch a valid auth token
req='curl -k -i -H "Content-Type: application/json" -X POST -d ''{"username":"$username","password":"$password"}'' https://somerepo.example.com/flimflam'
resp=$(curl -k -i -H "Content-Type: application/json" -X POST -d ''{"username":"$username","password":"$password"}'' https://somerepo.example.com/flimflam)
# echo the request for troubleshooting
echo "req = $req"
if [ -z "$resp" ]; then
echo "Login failed; unable to parse response"
exit 1
fi
echo "resp = $resp"
When I run this I get:
$ sh myscript.sh myUser 12345#45678
curl: (3) Port number ended with '"'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: 12345#45678"
100 1107 100 1093 100 14 2849 36 --:--:-- --:--:-- --:--:-- 2849
req = curl -k -i -H "Content-Type: application/json" -X POST -d {"username":"$username","password":"$password"} https://somerepo.example.com/flimflam
resp = HTTP/1.1 400 Bad Request...(rest omitted for brevity)
Obviously, I'm not escaping the various layers of single- and double-quotes inside the curl statement correctly, as is indicated by outputs like:
curl: (6) Could not resolve host: 12345#45678"
and:
req = curl -k -i -H "Content-Type: application/json" -X POST -d {"username":"$username","password":"$password"} https://somerepo.example.com/flimflam
where the username/password variables are not parsing.
In reality my script takes a lot more than 2 arguments, which is why I'm changing them to have meaningful variable names (such as $username instead of $1) so its more understandable and readable.
Can anyone spot where I'm going awry? Thanks in advance!
Update
I tried the suggestion which turns the req into:
curl -k -i -H 'Content-Type: application/json' -X POST -d "{'username':'myUser','password':'12345#45678'}" https://somerepo.example.com/flimflam
However this is still an illegal curl command and instead needs to be:
curl -k -i -H 'Content-Type: application/json' -X POST -d '{"username":"myUser","password":"12345#45678"}' https://somerepo.example.com/flimflam
First, as I said in a comment, storing commands in variables just doesn't work right. Variables are for data, not executable code. Second, you have two levels of quoting here: quotes that're part of the shell syntax (which are parsed, applied, and removed by the shell before the arguments are passed to `curl), and quotes that're part of the JSON syntax.
But the second problem is actually worse than that, because simply embedding an arbitrary string into some JSON may result in JSON syntax errors if the string contains characters that're part of JSON syntax. Which passwords are likely to do. To get the password (and username for that matter) embedded correctly in your JSON, use a tool that understands JSON syntax, like jq:
userinfo=$(jq -n -c --arg u "$username" --arg p "$password" '{"username":$u,"password":$p}')
Explanation: this uses --arg to set the jq variables u and p to the shell variables $username and $password respectively (and the double-quotes around the shell variables will keep the shell from doing anything silly to the values), and creates a JSON snippet with them embedded. jq will automatically add appropriate quoting/escaping/whatever is needed.
Then, to use it with curl, use something like this:
resp=$(curl -k -i -H "Content-Type: application/json" -X POST -d "$userinfo" https://somerepo.example.com/flimflam)
Again, the double-quotes around $userinfo keep the shell from doing anything silly. You should almost always put double-quotes around variables references in the shell.
Note that I never used the req variable to store the command. If you need to print the command (or its equivalent), use something like this:
printf '%q ' curl -k -i -H "Content-Type: application/json" -X POST -d "$userinfo" https://somerepo.example.com/flimflam
echo
The %q format specifier tells the shell to add appropriate quoting/escaping so that you could run the result as a shell command, and it'd work properly. (And the echo is there because printf doesn't automatically add a newline at the end of its output.)
try changing this:
req='curl -k -i -H "Content-Type: application/json" -X POST -d ''{"username":"$username","password":"$password"}'' https://somerepo.example.com/flimflam'
to this
req="curl -k -i -H 'Content-Type: application/json' -X POST -d \"{'username':'$username','password':'$password'}\" https://somerepo.example.com/flimflam"
and similarly for the resp
ah those pesky "curly" thingies...
how 'bout...
req="curl -k -i -H 'Content-Type: application/json' -X POST -d '{\"username\":\"$username\",\"password\":\"$password\"}' https://somerepo.example.com/flimflam"
This needs even more escaping:
With:
resp=$(curl -k -i -H "Content-Type: application/json" -X POST -d "{\"username\":\"$username\",\"password\":\"$password\"}" https://somerepo.example.com/flimflam)
In bash, the variables are still expanded when they're inside single quotes that are inside double quotes.
And you'll need the \" double quotes in the payload as per the JSON definition.
EDIT: I rerun the curl through a HTTP proxy and corrected the script line (see above, removed the single quotes). Results (in raw HTTP) are now:
POST /flimflam HTTP/1.1
Host: somerepo.example.com
User-Agent: curl/7.68.0
Accept: */*
Content-Type: application/json
Content-Length: 44
Connection: close
{"username":"user","password":"12345#abcde"}
(which should be fine)
I'm trying to pass several commands to an alpine container from powershell\cmd. The problem is: commands have quotes\single quotes\escaped quotes.
Sample command, the actual command is one line, I'm splitting it so it is easier to look at:
docker run neilpang/acme.sh bin/sh -c --%
"acme.sh --issue --dns dns_azure --dnssleep 10 --force -d "domain";
openssl pkcs12 -inkey /acme.sh/domain/domain.key -in /acme.sh/domain/domain.cer -export -out pfx -password pass:password;
curl -X POST -d "{\"certName\":\"certName1\",\"certData\":\"$(tail -n +2 /acme.sh/domain/domain.cer > b64; head -n -1 b64)\"}" url -H Content-Type:application/json;
curl -X POST -d "{\"certName\":\"certName2\",\"certData\":\"$(openssl base64 -in pfx -out b64; cat b64)\"}" url -H Content-Type:application/json"
What I've tried:
Use single quotes around the whole expression (after --%) returns something like quoted string not closed
Using single quotes around json works fine, but $() doesnt get interpreted, so I dont get proper output
escaping " before\after json
without " before\after json, results in some garbage
I never got it to work without --%, but it appears ps does some string manipulations even when using --%
did some experiments with cmd, no luck either.
I'm open to any way of making this work :)
for those requesting errors, here's a sample:
-n: line 1: syntax error: unexpected end of file (expecting ")") # quotes around expression
-p: line 1: syntax error: unterminated quoted string # single quotes around expression
no error, but certData empty # no quotes around json
no quotes in json >> not valid json >> doesnt get parsed # escaped quotes around json
executing the strings i'm trying to pass inside the containers results in the working "solution", whereas trying to pass them through docker run results in different defects described above. I'm not sure where these arise, from powershell (although, i firmly believe --% does the work here, and powershell passes everything properly), docker or bash itself. the closest i got to working is the ' version, but it doesnt evaluate expression inside $(), so that the problem
You'll find the quoting issues significantly easier if you turn your very long, very involved command line into a shell script, package it into a custom image, and then run that.
Shell script, let's call it run_acme.sh:
#!/bin/sh
acme.sh --issue --dns dns_azure --dnssleep 10 --force -d "domain"
openssl pkcs12 \
-inkey /acme.sh/domain/domain.key \
-in /acme.sh/domain/domain.cer \
-export \
-out pfx \
-password pass:password
CER_B64=$(tail -n +2 /acme.sh/domain/domain.cer | head -n -1)
curl -X POST \
--data-binary "{\"certName\":\"certName1\",\"certData\":\"$CER_B64\"}" \
-H 'Content-Type: application/json'
url
PFX_B64=$(openssl base64 -in pfx)
curl -X POST \
--data-binary "{\"certName\":\"certName1\",\"certData\":\"$PFX_B64\"}" \
-H 'Content-Type: application/json'
url
Dockerfile:
FROM neilpang/acme.sh
COPY run_acme.sh /bin
CMD ["/bin/run_acme.sh"]
Then you can docker build this like any other custom image, and docker run it, and the very long command line will be baked into the shell script in your image.
I'd suggest using splatting to make this significantly cleaner to maintain. Note: I'm uncertain whether your shell script needs to escape the quotes around the -d parameter.
$curl1 = '"{\"certName\":\"certName1\",' +
'\"certData\":\"$(tail -n +2 /acme.sh/domain/domain.cer > b64; head -n -1 b64)\"}"'
$curl2 = '"{\"certName\":\"certName2\",' +
'\"certData\":\"$(openssl base64 -in pfx -out b64; cat b64)\"}"'
$dockerArgs = #(
'run', 'neilpang/acme.sh', 'bin/sh', '-c'
'"acme.sh --issue --dns dns_azure --dnssleep 10 --force -d "domain";' +
'openssl pkcs12 -inkey /acme.sh/domain/domain.key -in /acme.sh/domain/domain.cer ' +
'-export -out pfx -password pass:password;' +
"curl -X POST -d $curl1 url -H Content-Type:application/json;" +
"curl -X POST -d $curl2 url -H Content-Type:application/json"""
)
docker.exe #dockerArgs
The powershell escape character is a grave-accent ` or you could double-up on the quotes "" to escape double-quotes "`"".