I need to execute a curl command like this:
#!/bin/bash
shopid=9932781
itemid=231873991
curl -sSb /tmp/cookies 'https://website.com' -H 'cookie: csrftoken=mytoken' -H 'x-csrftoken: mytoken' -H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary' -H 'referer: https://website.com' \
--data-binary $'------WebKitFormBoundary\r\nContent-Disposition: form-data; name="shopid"\r\n\r\n${shopid}\r\n------WebKitFormBoundary\r\nContent-Disposition: form-data; name="itemid"\r\n\r\n${itemid}\r\n------WebKitFormBoundary\r\nContent-Disposition: form-data; name="quantity"\r\n\r\n1\r\n------WebKitFormBoundary\r\nContent-Disposition: form-data; name="donot_add_quantity"\r\n\r\nfalse\r\n------WebKitFormBoundary\r\nContent-Disposition: form-data; name="update_checkout_only"\r\n\r\nfalse\r\n------WebKitFormBoundary\r\nContent-Disposition: form-data; name="source"\r\n\r\n\r\n------WebKitFormBoundary\r\nContent-Disposition: form-data; name="checkout"\r\n\r\ntrue\r\n------WebKitFormBoundary--\r\n'
The $'' quotes are necessary or else (ie. in the double-quoted case) \r\n won't work -- but with this form, $shopid and $item aren't replaced with their values.
How can I get both behaviors?
You need to make that code maintainable
binary_data=$( cat <<END_DATA | sed 's/$/\r/'
------WebKitFormBoundary
Content-Disposition: form-data; name="shopid"
${shopid}
------WebKitFormBoundary
Content-Disposition: form-data; name="itemid"
${itemid}
------WebKitFormBoundary
Content-Disposition: form-data; name="quantity"
1
------WebKitFormBoundary
Content-Disposition: form-data; name="donot_add_quantity"
false
------WebKitFormBoundary
Content-Disposition: form-data; name="update_checkout_only"
false
------WebKitFormBoundary
Content-Disposition: form-data; name="source"
------WebKitFormBoundary
Content-Disposition: form-data; name="checkout"
true
------WebKitFormBoundary--
END_DATA
)
curl_opts=(
-sSb /tmp/cookies
-H 'cookie: csrftoken=mytoken'
-H 'x-csrftoken: mytoken'
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary'
-H 'referer: https://website.com'
--data-binary "$binary_data"
)
curl "${curl_opts[#]}" 'https://website.com'
You can use multiple quoting styles within a single string. Thus:
$'Constant\r\n\n\r\n'"$bar"
...has the \r\ns parsed with $'...' rules, but the $bar expanded with double-quoting rules (such that the expansion also takes place).
The variables are not expanded in 'single quotes'.
It's important to learn how quoting works :
"Double quote" every literal that contains spaces/metacharacters and every expansion: "$var", "$(command "$var")", "${array[#]}", "a & b". Use 'single quotes' for code or literal $'s: 'Costs $5 US', ssh host 'echo "$HOSTNAME"'. See
http://mywiki.wooledge.org/Quotes
http://mywiki.wooledge.org/Arguments
http://wiki.bash-hackers.org/syntax/words
So, try this :
data="------WebKitFormBoundary
Content-Disposition: form-data; name='shopid'
${shopid}
------WebKitFormBoundary
Content-Disposition: form-data; name='itemid'
${itemid}
------WebKitFormBoundary
Content-Disposition: form-data; name='quantity'
1
------WebKitFormBoundary
Content-Disposition: form-data; name='donot_add_quantity'
false
------WebKitFormBoundary
Content-Disposition: form-data; name='update_checkout_only'
false
------WebKitFormBoundary
Content-Disposition: form-data; name='source'
------WebKitFormBoundary
Content-Disposition: form-data; name='checkout'
true
------WebKitFormBoundary--"
data="$(sed 's/$/\r/' <<< "$data")"
curl -sSb /tmp/cookies \
-H 'cookie: csrftoken=mytoken' \
-H 'x-csrftoken: mytoken' \
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundary' \
-H 'referer: https://website.com' \
--data-binary "$data" \
'https://website.com'
Related
I have tried to create a method in a bash script which should be able to perform a curl operation with a variable number of headers, however I seem to get stuck in that the curl command consider the header arguments to be multiple arguments as they contain spaces.
When I run the following line in bash I get a 201 response:
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
If I run the following:
#!/bin/bash
submit_request () {
full_path=/home/mat/globalmetadata.json
header_option=""
header_options=""
for header in "${#:1}"; do # looping over the elements of the $# array ($1, $2...)
header_option=$(printf " -H %s" "$header")
header_options=$(printf '%s%s' "$header_options" "$header_option")
done
echo Headers: $header_options
executable=curl
#response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" $header_option )
echo $response
}
submit_request "\"Content-Type: application/json\""
I get this output:
Headers: -H "Content-Type: application/json"
======= 3
* Trying 127.0.0.1:9200...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9200 (#0)
> POST /index-template/globalmetadata HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 3232
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
} [3232 bytes data]
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 406 Not Acceptable
< X-elastic-product: Elasticsearch
< content-type: application/json; charset=UTF-8
< content-length: 97
<
{ [97 bytes data]
* Connection #0 to host localhost left intact
* Could not resolve host: application
* Closing connection 1
406000
What I have noticed is that even though the headers are -H "Content-Type: application/json, curl says Could not resolve host: application. I suspect that it splits the arguments into two due to space between Content-Type: and application/json.
I tried to mix and match quotes and double quotes in all kinds of combination but nothing really worked.
#GordonDavisson is right, you'll have to put your header_options in an array. Just be aware that using the array "${header_options[#]}" when it is empty will result in an empty argument in your curl command, but You can get rid of this problem by storing the whole command in an other array.
submit_request () {
local -a header_options
for arg; do header_options+=(-H "$arg"); done
local -a curl_command=( \
curl \
-X POST \
localhost:9200/index-template/globalmetadata \
--write-out '%{http_code}' \
--silent \
--output /dev/null \
--verbose \
--data '#/home/mat/globalmetadata.json' \
"${header_options[#]}" \
)
local response=$( "${curl_command[#]}" )
printf '%s\n' "$response"
}
I am trying to authenticate to my openvpn-as server via curl command. My end goal is to be able to download the ovpn file as I would via the browser but via the cli. I was able to get the cURL command from the browser but I get a 403.
My curl command looks like this:
curl -iL 'https://192.168.35.229:943/?src=connect' \
-H 'Connection: keep-alive' \
-H 'Accept: text/plain, */*; q=0.01' \
-H 'X-OpenVPN: 1' \
-H 'X-Requested-With: XMLHttpRequest' \
-H 'X-CWS-Proto-Ver: 2' \
-H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' \
-H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36 OPR/70.0.3728.178' \
-H 'Origin: https://192.168.35.229:943' \
-H 'Sec-Fetch-Site: same-origin' \
-H 'Sec-Fetch-Mode: cors' \
-H 'Sec-Fetch-Dest: empty' \
-H 'Accept-Language: en-US,en;q=0.9' \
--data-raw 'username=dummy&password=dummy' \
--compressed \
--insecure
The response looks like this:
HTTP/1.1 302 Found
Transfer-Encoding: chunked
Set-Cookie: openvpn_sess_c6bec1b42f39a6d1034a54580d5422dc=36bce77626a6e403eb6ae9f02576d460; Expires=Wed, 16 Sep 2020 05:15:12 GMT; Path=/; Secure; HttpOnly
Server: OpenVPN-AS
Location: https://192.168.35.229:943/__session_start__/
Date: Wed, 16 Sep 2020 04:45:12 GMT
X-Frame-Options: SAMEORIGIN
Content-Type: text/html; charset=utf-8
HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Date: Wed, 16 Sep 2020 04:45:12 GMT
X-Frame-Options: SAMEORIGIN
Content-Type: text/html; charset=UTF-8
Server: OpenVPN-AS
<html><head><title>Forbidden</title></head><body><h1>Forbidden</h1>Request was forbidden.</body></html>#
I wasn't able to find any examples or instructions about this in the documentation. Is there a way to achieve this via CLI - via bash script ?
I was able to find a solution based on this article . The curl command would look like:
curl -u dummy:dummy -k https://192.168.35.229:943/rest/GetUserlogin
I'm building a lambda function that needs to receive several parameters, including eventually files.
curl -v -H "Authorization: blah" -F "index_name=test" -F "document=#./spec/fixtures/config/indexes/video.json" -X POST https://blah.execute-api.us-east-1.amazonaws.com/dev/index_document
I can see from inside my event['body'] that I'm receiving my data, but I can't find a way to parse the data and using Rack feels a bit weird.
<--------------------------298624d5b63e53bf
Content-Disposition: form-data; name="index_name"
test
--------------------------298624d5b63e53bf
Content-Disposition: form-data; name="document"; filename="video.json"
Content-Type: application/octet-stream
{
"settings": {
"index": {
"number_of_shards": 5
}, ...
Is there any other way to have my data back in an orderly fashion, like:
>>> event['body']['index_name']
"test"
I would recommend you have a look at the curb ruby gem
My goal is to send an email using curl with an html body with embedded image such as :
I'm sending the email like this :
curl "smtp://smtp.gmail.com:587" -v \
--mail-from "sender#gmail.com" \
--mail-rcpt "receiver#gmail.com" \
--ssl -u sender#gmail.com:secretpassword \
-T "message.txt" -k --anyauth
My message.txt looks like :
From: Some Name <sender#gmail.com>
To: Some Name <receiver#gmail.com>
Subject: example of mail
Reply-To: Some Name <sender#gmail.com>
Cc:
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="MULTIPART-MIXED-BOUNDARY"
--MULTIPART-MIXED-BOUNDARY
Content-Type: multipart/alternative; boundary="MULTIPART-ALTERNATIVE-BOUNDARY"
--MULTIPART-ALTERNATIVE-BOUNDARY
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64
Content-Disposition: inline
PGh0bWw+Cjxib2R5PgogICAgPGRpdj4KICAgICAgICA8cD5IZWxsbywgPC9wPgogICAgICAgIDxw
PlBsZWFzZSBzZWUgdGhlIGxvZyBmaWxlIGF0dGFjaGVkPC9wPgogICAgICAgIDxwPkFkbWluIFRl
YW08L3A+CiAgICAgICAgPGltZyBzcmM9ImFkbWluLnBuZyIgd2lkdGg9IjE1MCIgaGVpZ2h0PSI1
MCI+CiAgICA8L2Rpdj4KPC9ib2R5Pgo8L2h0bWw+Cg==
--MULTIPART-ALTERNATIVE-BOUNDARY--
--MULTIPART-MIXED-BOUNDARY
The html decoded is :
<html>
<body>
<div>
<p>Hello, </p>
<p>Please see the log file attached</p>
<p>Admin Team</p>
<img src="admin.png" width="150" height="50">
</div>
</body>
</html>
How can I embed admin.png in this html and attach another file log.txt
to this email using curl and bash ?
The solution I came up with is to base64 encode all the attachment (image and text file) and to include them into the uploaded file directly in the multipart/mixed body such as :
--MULTIPART-MIXED-BOUNDARY
Content-Type: image/png
Content-Transfer-Encoding: base64
Content-Disposition: inline
Content-Id: <admin.png>
iVBORw0KGgoAAAANSUhEUgAAAIAAAACgCAIAAABL8POqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAA
B3RJTUUH4AQNDwEVouBdqAAAG2xJREFUeNrtfX9oHFe25jdDBU5BG25BG7pABhXEkDJjSIsYIs1m
WbfJA8ubhcjjgdiTQNJOYCInj0RKYGIl8CbyPF4iZSCxEkgsB5LIgWQlL2Pcfow3bdgw0mMzox6e
....
--MULTIPART-MIXED-BOUNDARY
Content-Type: text/plain
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename=log.txt
c29tZSBsb2cgaW4gYSB0eHQgZmlsZSB0byBhdHRhY2ggdG8gdGhlIG1haWwK
--MULTIPART-MIXED-BOUNDARY--
The Content-Id header is used to identify the resource that can be referenced in the html with : cid: like :
<img src="cid:admin.png" width="150" height="50">
Here is a complete bash example to send an html email with and admin.png embedded image and a log.txt attached to it :
#!/bin/bash
rtmp_url="smtp://smtp.gmail.com:587"
rtmp_from="sender#gmail.com"
rtmp_to="receiver#gmail.com"
rtmp_credentials="sender#gmail.com:secretpassword"
file_upload="data.txt"
# html message to send
echo "<html>
<body>
<div>
<p>Hello, </p>
<p>Please see the log file attached</p>
<p>Admin Team</p>
<img src=\"cid:admin.png\" width=\"150\" height=\"50\">
</div>
</body>
</html>" > message.html
# log.txt file to attached to the mail
echo "some log in a txt file to attach to the mail" > log.txt
mail_from="Some Name <$rtmp_from>"
mail_to="Some Name <$rtmp_to>"
mail_subject="example of mail"
mail_reply_to="Some Name <$rtmp_from>"
mail_cc=""
# add an image to data.txt :
# $1 : type (ex : image/png)
# $2 : image content id filename (match the cid:filename.png in html document)
# $3 : image content base64 encoded
# $4 : filename for the attached file if content id filename empty
function add_file {
echo "--MULTIPART-MIXED-BOUNDARY
Content-Type: $1
Content-Transfer-Encoding: base64" >> "$file_upload"
if [ ! -z "$2" ]; then
echo "Content-Disposition: inline
Content-Id: <$2>" >> "$file_upload"
else
echo "Content-Disposition: attachment; filename=$4" >> "$file_upload"
fi
echo "$3
" >> "$file_upload"
}
message_base64=$(cat message.html | base64)
echo "From: $mail_from
To: $mail_to
Subject: $mail_subject
Reply-To: $mail_reply_to
Cc: $mail_cc
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=\"MULTIPART-MIXED-BOUNDARY\"
--MULTIPART-MIXED-BOUNDARY
Content-Type: multipart/alternative; boundary=\"MULTIPART-ALTERNATIVE-BOUNDARY\"
--MULTIPART-ALTERNATIVE-BOUNDARY
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64
Content-Disposition: inline
$message_base64
--MULTIPART-ALTERNATIVE-BOUNDARY--" > "$file_upload"
# add an image with corresponding content-id (here admin.png)
image_base64=$(curl -s "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_116x41dp.png" | base64)
add_file "image/png" "admin.png" "$image_base64"
# add the log file
log_file=$(cat log.txt | base64)
add_file "text/plain" "" "$log_file" "log.txt"
# add another image
#image_base64=$(curl -s "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_116x41dp.png" | base64)
#add_file "image/png" "something.png" "$image_base64"
# end of uploaded file
echo "--MULTIPART-MIXED-BOUNDARY--" >> "$file_upload"
# send email
echo "sending ...."
curl -s "$rtmp_url" \
--mail-from "$rtmp_from" \
--mail-rcpt "$rtmp_to" \
--ssl -u "$rtmp_credentials" \
-T "$file_upload" -k --anyauth
res=$?
if test "$res" != "0"; then
echo "sending failed with: $res"
else
echo "OK"
fi
Here's a shell script you could use.
I heavily derived from https://blog.ambor.com/2021/08/using-curl-to-send-e-mail-with.html
Passwords could be stored in a .netrc file as discussed on: https://everything.curl.dev/usingcurl/netrc but I haven't tried that.
Just make sure the image you need is embedded in the HTML using base64 encoding.
#!/bin/bash
declare -a VOPTS;
declare -a HOPTS;
sesAccess="sender.account.authentication#email.id" ;
sesSecret="sender.account.passwordXXXXXX";
sesFromName="Sender Full Name";
sesFromAddress='<sender#email.id>';
sesToName="Recipient Full Name";
sesToAddress="<recepient#email.id>"
sesSubject="Email Subject Line";
sesSMTP='mail.server.fqdn';
sesPort='465';
sesMessage=$'Test of line 1\nTest of line 2'
sesFile="$1"; # attachment is first argument
sesHTMLbody="/path/to/html/file.html"; # content of this file will be used to create HTML body
sesMIMEType=`file --mime-type "$sesFile" | sed 's/.*: //'`;
# sesMIMEType=`file -b --mime-type "$sesFile"`;
VOPTS=();
HOPTS=();
#Curl Options
VOPTS+=("-v");
VOPTS+=("--url"); VOPTS+=("smtps://$sesSMTP:$sesPort");
VOPTS+=("--ssl-reqd")
VOPTS+=("--user"); VOPTS+=("${sesAccess}:${sesSecret}");
VOPTS+=("--mail-from"); VOPTS+=("${sesFromAddress}");
VOPTS+=("--mail-rcpt"); VOPTS+=("${sesToAddress}");
#Header Options
HOPTS+=("-H"); HOPTS+=("Subject: ${sesSubject}");
HOPTS+=("-H"); HOPTS+=("From: ${sesFromName} ${sesFromAddress}");
HOPTS+=("-H"); HOPTS+=("To: ${sesToName} ${sesToAddress}");
curl "${VOPTS[#]}" -F '=(;type=multipart/mixed' -F "=<$sesHTMLbody;type=text/html;encoder=base64" -F "file=#$sesFile;type=$sesMIMEType;encoder=base64" -F '=)' "${HOPTS[#]}"
exit
Plus one, your code example is a bit misleading tough, in fact it doesn't work.
As you say, you just have to enclose each part in a boundary, in your case the delimiter is:
"--MULTIPART-MIXED-BOUNDARY".
Nonetheless it could be any other user defined string.
You are nesting multiple boundary types, which isn't necessary. HTML content doesn't need to be base64 encoded, it can go encoded if you will though.
You need a blank line after and before each delimiter.
I recommend taking a look at the W3 papers and examples: https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
The answer by #bertrand-martel was mostly right on my system running Ubuntu 22.04 and submitting to a postfix server in my network. The html email message came through just fine, and I had empty attachments named correctly. I had to edit the add_file function he defined and add in a blank line before the $3 at the very end. The email would not work without a blank line between the Content-Disposition: and the base64 encoding. I also had to insert a blank line after the --MULTIPART-ALTERNATIVE-BOUNDARY--
. Once I made those changes, the html message, the embedded image, and the attached file all came across correctly.
I would like to use curl to send an HTTP POST request with a multipart body, which consists of two parts part1 and part2 like this:
Content-Type: multipart/form-data;
boundary=v6jSBESHUN0nHdfaw-GnE4zGVcg0OLfL1
--v6jSBESHUN0nHdfaw-GnE4zGVcg0OLfL1
Content-Disposition: form-data; name="part1"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
abc
--v6jSBESHUN0nHdfaw-GnE4zGVcg0OLfL1
Content-Disposition: form-data; name="part2"
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
xyz
--v6jSBESHUN0nHdfaw-GnE4zGVcg0OLfL1--
Note that the content of part1 is abc and the content of part2 is xyz. The charset of part1 is ascii and the charset of part2 is utf-8.
I am trying to do it as follows:
curl -X POST -F "part1=abc" -F "part2=xyx" http://myhost/mypost
Unfortunately it did not do the trick. Besides, I did not find to how specify the charset of part2. How would you suggest send this request with curl ?
You can set the type with ;type=
curl -X POST -F "part1=abc" -F "part2=xyx;type=text/plain; charset=utf-8" http://myhost/mypost