I want to send an email with attached pdf file through the Sparkpost API with curl post.
To insert the pdf I use (my test.pdf is ~ 200KB)
"data":"'$(cat test.pdf} | base64 --wrap=0)'"
But somehow this doesn't work out showing the following error:
/usr/bin/curl: Die Argumentliste ist zu lang (original)
/usr/bin/curl: Argument list is too long
EDIT:
curl command
curl -X POST https://api.eu.sparkpost.com/api/v1/transmissions -H 'Authorization: <APIKEY>' -H 'Content-Type: application/json' -d '{
"options":{
"open_tracking":false,
"click_tracking":false,
"inline_css":false
},
"recipients":[
{
"address":{
"email":"user#domain.tld",
"name":"user"
}
}
],
"content":{
"from":{
"name":"sender",
"email":"sender#domain.tld"
},
"reply_to":"replyto#domain.tld",
"subject":"subject",
"text":"textbody",
"attachments":[
{
"name":"attachmentname.pdf",
"type":"application/pdf",
"data":"'$(cat test.pdf | base64 --wrap=0)'"
}
]
}
}'
This is coming up because you are trying to pass the entirety of the base64'd content on the command line. curl has the ability to load in data to POST from a file, which I'd recommend doing. More information can be found in the man page, but the basic format is this:
curl -X POST -d #filename.txt https://website.com/path
According to the curl manual, the -F option allows you to encode a file for base64, but limits the output to 76 characters.
Ex:
-F '=#localfile;encoder=base64'
Related
I am trying to send JSON data containing a mac address to an api using this command:
$value={ "pcModel": "KAT12", "displayType": "DELL U2311H", "graphicsType": "Microsoft Remote Display Adapter", "displayServiceTag": "HV8XP08Q079L", "ipAddress": "172.16.4.194", "recDate": "2022-10-06 16:57:55", "serviceTag": "18LQ9X1;Diskwear:(4.91TBW ; 15393 Hours)", "wolMode": "lanwithpxeboot;CC:101010-0118ZH;os:Ubuntu", "sleepState": "disable", "macAddress": "90:B1:1C:8E:D5:11", "hostName": "CI-KR95-05", "diskMode": "raid", "diskType": "Samsung SSD 850 PRO 512GB;TBW+Hrs:(4.91TB;15393 HrH) ;Clock:3.4GHz;Max Clock:3.67GHz(108%);RAM:32GB" }
curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85330rf562c4cd6c1fb1a64d1" -d "$value" --url "https://my_api.org/api/inventory/84:2b:2b:a0:0s2:18
but I get the following answer:
user#ubuntu:~$ curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85df90210cd1a827bc1518c4cd6c1fb1a64d1" -d "$value" --url "https:/my_api/api/inventory/84:2b:2b:a0:0s2:18"
curl: (3) URL using bad/illegal format or missing URL
I tried to escape the ":" colon characters with \ like this
curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85330rf562c4cd6c1fb1a64d1" -d "$value" --url "https://my_api.org/api/inventory/84\:2b\:2b\:a0\:0s\:18"
but I get no output and it sends nothing.
any Idea how to send this data without having the bad format error?
Thanks a lot
I removed the "?" from $value and added single quotes.
Try this:
#!/bin/bash
value='{ "pcModel": "KAT12", "displayType": "DELL U2311H", "graphicsType": "Microsoft Remote Display Adapter", "displayServiceTag": "HV8XP08Q079L", "ipAddress": "172.16.4.194", "recDate": "2022-10-06 16:57:55", "serviceTag": "18LQ9X1;Diskwear:(4.91TBW ; 15393 Hours)", "wolMode": "lanwithpxeboot;CC:101010-0118ZH;os:Ubuntu", "sleepState": "disable", "macAddress": "90:B1:1C:8E:D5:11", "hostName": "CI-KR95-05", "diskMode": "raid", "diskType": "Samsung SSD 850 PRO 512GB;TBW+Hrs:(4.91TB;15393 HrH) ;Clock:3.4GHz;Max Clock:3.67GHz(108%);RAM:32GB" }'
curl -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85330rf562c4cd6c1fb1a64d1" -d "$value" --url "https://my_api.org/api/inventory/84:2b:2b:a0:0s2:18"
Didn't your mother teach you how to format?
value='{
"pcModel": "KAT12",
"displayType": "DELL U2311H",
"graphicsType": "Microsoft Remote Display Adapter",
"displayServiceTag": "HV8XP08Q079L",
"ipAddress": "172.16.4.194",
"recDate": "2022-10-06 16:57:55",
"serviceTag": "18LQ9X1;Diskwear:(4.91TBW ; 15393 Hours)",
"wolMode": "lanwithpxeboot;CC:101010-0118ZH;os:Ubuntu",
"sleepState": "disable",
"macAddress": "90:B1:1C:8E:D5:11",
"hostName": "CI-KR95-05",
"diskMode": "raid",
"diskType": "Samsung SSD 850 PRO 512GB;TBW+Hrs:(4.91TB;15393 HrH) ;Clock:3.4GHz;Max Clock:3.67GHz(108%);RAM:32GB"
}'
$value is not a legal variable name in bash for declaration, only when you want to look it up. use value=
Also, you are missing a " after the URL.
$ curl "http:/cake.com/nom-nom
> "
curl: (3) URL using bad/illegal format or missing URL
The protocol schema is followed by colon-slash-slash ://. so you need one more:
https://
Maybe you want to read the URL rfc:
https://www.ietf.org/rfc/rfc2718.txt
2.1.2 Improper use of "//" following ":"
Contrary to some examples set in past years, the use of double
slashes as the first component of the of a URL
is not simply an artistic indicator that what follows is a URL:
Double slashes are used ONLY when the syntax of the URL's contains a hierarchical structure as described in RFC
2396. In URLs from such schemes, the use of double slashes indicates that what follows is the top hierarchical element for a
naming authority. (See section 3 of RFC 2396 for more details.)
URL schemes which do not contain a conformant hierarchical
structure in their should not use double
slashes following the ":" string.
Thanks for your answers.
I found the reasons why my code was not updating the API
1 - I needed to specify the "Content-Type: "application/json" parameter to tell the API that the data sent hat JSON format otherwise I saw in verbose mode that the data has a "x-www-form-urlencoded" format that the API can't understand.
2 - The data sent with curl must accept a value with the format like this
value='{
"pcModel": "KAT12",
"displayType": "DELL U2311H",
"diskType": "Samsung SSD .."
}'
3 - The curl request must be sent with double quotes around the url and the data ""
So the end request looks like this:
curl -k -X "PUT" -H "Content-Type: application/json"\
-H "Accept: application/json" -H "Authorization: Token 62d85df902101828g84kc4cd6c1fb1a64d1" \
--url "https://my_api.org/api/inventory/84:2b:2b:a0:0s2:18" \
-d "$value"
I need to send an HTML file as the body of an eamil to several customers. Our company will be using SendGrid for this, and I need to be able to send the email via API Curl Call.
The way that I'm doing it so far works for simple html or plain text:
curl -s --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SECRET_API_KEY" \
--header 'Content-Type: application/json' \
--data '{"personalizations":[{"to":[{"email":"my1#email.com"},{"email":"my2#email.com"}]}],"from":{"email":"info#somewhere.com"},"subject":"Testing sending emails via SendgridAPI","content":[{"type":"text\/html","value":"Test API Email From ME"}]}'
Now this works just fine. The problem is when I want to replace 'Test API Email From ME' with the contents of a rather large, complex HTML file. This has all the usual cli nightmares such as a mix of ' and " and new lines everywhere. I need to sanatize the HTML in order to accomplish three things:
The final result needs to be a valid command line string
The --data switch argument needs to remain a valid JSON enconded string
The HTML should not break.
What I do is I create the actual string command and the execute it using a scripting language. So I can perform any operation that I want on the html before inserting it in the value field of the content field. So my question is: what are the string operations that I should perform on the html so that I can send the email using this methodology?
Using jq and bash
I'll do it with static data, you may improve upon it
Define a JSON template for the API:
IFS='' read -r -d '' json_template <<'EOF'
{
"personalizations": [
{
"to": [
{ "email": "my1#email.com" },
{ "email": "my2#email.com" }
]
}
],
"from": { "email": "info#somewhere.com" },
"subject": "Testing sending emails via SendgridAPI",
"content": [
{
"type": "text/html",
"value": "Test API Email From ME"
}
]
}
EOF
Define the HTML content:
IFS='' read -r -d '' html_email <<'EOF'
<!doctype html>
<html>
<head>
title>Simple Email</title>
</head>
<body>
Test API Email From ME
</body
</html>
EOF
Replace the email content in the JSON with the HTML
json_data=$(
jq -c -n \
--arg html "$html_email" \
--argjson template "$json_template" \
'$template | .content[0].value = $html'
)
Send the query
curl -s --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SECRET_API_KEY" \
--header 'Content-Type: application/json' \
--data "$json_data"
Here is how you can compose a proper JSON data payload with jq so it can be sent to the API.
jq will ensure every values, recipients, from, subject and the html body will be respectively encoded to proper JSON data objects, arrays and strings before it is submitted as --data #- to curl:
I added comments everywhere, so it is very clear what is done at every step:
#!/usr/bin/env bash
recipients=(
'my1#email.com'
'my2#email.com'
)
from='info#somewhere.com'
subject='Testing sending emails via SendgridAPI'
# Streams null-delimited recipients array entries
printf '%s\0' "${recipients[#]}" |
# jq slurps the null-delimited recipients,
# read the raw html content into the jq $contentHTML variable
# and integrate it all as a proper JSON
jq --slurp --raw-input --rawfile contentHTML example.html \
--arg from "$from" \
--arg subject "$subject" \
'
# Fills the jq $recipient JSON array variable
# by splitting the null-delmited entries
# from the incoming stream
split( "\u0000") as $recipients |
{
"personalizations": [
{
# Uses the $recipients array that has been
# slurped from the input stream
"to": $recipients
}
],
"from": {
# Use the $from that has been passed as --arg
"email": $from
},
# Use the $subject that has been passed as --arg
"subject": $subject,
"content": [
{
"type": "text/html",
"value": $contentHTML
}
]
}
' |
# Get the resultant JSON piped into curl
# that will read the data from the standard input
# using --data #-
# rather than passing it as an argument, because
# the payload could exceed the maximum length of arguments
curl -s --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SECRET_API_KEY" \
--header 'Content-Type: application/json' \
--data #-
I am hitting a wall trying to build a script to save myself quite a good bit of time. I am working in a system in which I need to run a curl POST against a list of values. The list is about 400 lines long, so I am hoping to find a way of scripting this in Bash instead of running that call manually for each entry. Below are some details to help understand what I'm trying to accomplish:
If I were to be doing this task manually, each call would be formatted like the below:
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
This points to my JSON in the listed file which shows as the below:
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": "apple"
}
If I run the above, the call works, and the expected operation occurs. The issue is that I need to run this against roughly 400 values for that "memberInfo" field in the JSON payload. I'm trying to identify a way to run a single bash script, which will run this curl command over and over, and update the JSON payload to use each row in a file as the below:
memberList.txt
apple
banana
peach
pear
orange
.
.
And then maybe insert a pointer in my JSON for the "memberInfo" field over to this file.
Any and all help/suggestions are greatly appreciated!
.
This will do as you intend. Its a little convoluted but you might polish it a bit.
#!/bin/bash
function getString(){
echo $1 | python3 -c '
import json
import sys
payload="""
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": ""
}
"""
obj = json.loads(payload)
obj["memberInfo"] = sys.stdin.read().strip()
print(json.dumps(obj, indent = " "))
'
}
while read member
do
getString "$member" > json_payload.json
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
done <<< "$( cat fruits.txt )"
Hope it helps!
while read member; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '{"groupId": "12345678987654321","type": "serial","memberInfo": "$member"}'
done <members.txt
This will work if you only care about the memberInfo field, another method could be writing your json line by line to payloads.txt file.
payloads.txt
{"groupId": "12345678987455432","type": "stereo","memberInfo": "apple"}
{"groupId": "34532453453453465","type": "serial","memberInfo": "banana"}
...
then use this as the script
while read payload; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '$payload'
done <payloads.txt
here is a collection of bash scripting common uses I've had to use
https://github.com/felts94/advanced-bash/blob/master/bash_learn.sh
I need to create a shell script that calls my login API via curl.
The script should be able to store and process the response from curl api call.
myscript.sh
#!/bin/bash
echo "Extract bearer token from curl calling login api"
echo
# Check cURL command if available (required), abort if does not exists
type curl >/dev/null 2>&1 || { echo >&2 "Required curl but it's not installed. Aborting."; exit 1; }
echo
PAYLOAD='{"email": "dummy-user#acme.com", "password": "secret"}'
curl -s --request POST -H "Content-Type:application/json" http://acme.com/api/authentications/login --data "${PAYLOAD}"
My problem in the given script is:
it does not get the response of curl calling the API.
From the response json, get only the token value.
Sample Login API response:
{
"user": {
"id": 123,
"token": "<GENERATED-TOKEN-HERE>",
"email": "dummy-user#acme.com",
"refreshToken": "<GENERATED-REFRESH-TOKEN>",
"uuid": "1239c226-8dd7-4edf-b948-df2f75508888"
},
"clientId": "abc12345",
"clientSecretKey": "thisisasecret"
}
I only need to get the value of token and store it in a variable... I will use token value in other curl api call as bearer token.
What do I need to change in my script to extract the token value from the response of curl api call?
Thanks!
Your curl statement has an error in it. You are executing it with the target URL as an header field:
curl --request POST -H "Content-Type:application/json" -H http://acme.com/api/authentications/login --data "${PAYLOAD}"
^
|
Remove this header flag
Also the silent -s flag helps when curl is executed from scripts:
-s, --silent
Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute.
Afterwards you could store the data in a variable and execute a regular expression on it to extract the token you need for further processing.
The complete script could look like the following:
#!/bin/bash
echo "Extract bearer token from curl calling login api"
echo
# Check cURL command if available (required), abort if does not exists
type curl >/dev/null 2>&1 || { echo >&2 "Required curl but it's not installed. Aborting."; exit 1; }
echo
PAYLOAD='{"email": "dummy-user#acme.com", "password": "secret"}'
RESPONSE=`curl -s --request POST -H "Content-Type:application/json" http://acme.com/api/authentications/login --data "${PAYLOAD}"`
TOKEN=`echo $RESPONSE | grep -Po '"token":(\W+)?"\K[a-zA-Z0-9._]+(?=")'`
echo "$TOKEN" # Use for further processsing
An alternate solution to parsing JSON with regex is jq :
echo '{ "user": { "id": 123, "token": "<GENERATED-TOKEN-HERE>", "email": "dummy-user#acme.com", "refreshToken": "<GENERATED-REFRESH-TOKEN>", "uuid": "1239c226-8dd7-4edf-b948-df2f75508888" }, "clientId": "abc12345", "clientSecretKey": "thisisasecret" }' | jq -r '.user.token'
I have a curl request in below format
curl -v -H "Content-Type:application/json" -H "x-user-id:xxx" -H "x-api-key:yyy" --data '{"logs":"'"${TEST_OUTPUT}"'","pass":"true | false"}' https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
This works fine while I do from my MAC terminal. But the same command throws
13:49:26 {
13:49:26 "status": "error",
13:49:26 "message": "Invalid credentials"
13:49:26 }
I saw this post but not sure how else would I send a json body without curly braces. I know that we can save it as a file.json and use the file as body.But for some reasons that cannot be implemented in my scenario
In general, you should avoid trying to build JSON using string interpolation. Use a tool like jq to handle any necessary quoting.
jq -n --argson o "$TEST_OUTPUT" '{logs: $o, pass: "true | false"}' |
curl -v -H "Content-Type:application/json" \
-H "x-user-id:xxx" \
-H "x-api-key:yyy" \
--data #- \
https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
However, if you can manage to correctly generate your JSON as you are now, you can just replace the jq command with echo:
echo '{"logs": ...' | curl ...
The #- argument to --data says to read from standard input.