cURL command doesn't work from terminal prompt - bash

Following the documentation here under /campaigns I try the example with my api-key.
I am using the terminal window on a Mac (bash shell).
curl -X POST https://api.mailerlite.com/api/v2/campaigns \
-d '{
"subject": "Regular campaign subject",
"segments": 1291259,
"type": "regular"
}' \
-H "Content-Type: application/json" \
-H "X-MailerLite-ApiKey: <api-key>" \
I paste this whole thing into the prompt even though that doesn't seem the correct way to do this.
Part of the command is understood and then part way through, the shell asks this Display all 114 possibilities question:
~/: curl -X POST https://api.mailerlite.com/api/v2/campaigns \
> -d '{
>
Display all 114 possibilities? (y or n)
Followed by a directory listing, Followed by:
> campaign subject",
> "segments": 1291259,
> "type": "regular"
> }' \
> -H "Content-Type: application/json" \
> -H "X-MailerLite-ApiKey: <api-key>" \
>
{"error":{"code":400,"message":"Campaign type is missing"}}~/:
What is the proper way to issue a multi-line cURL command from a bash shell?

Replace tabs with spaces, and drop the last backslash, then it should work just fine.
curl -X POST https://api.mailerlite.com/api/v2/campaigns \
-d '{
"subject": "Regular campaign subject",
"segments": 1291259,
"type": "regular"
}' \
-H "Content-Type: application/json" \
-H "X-MailerLite-ApiKey: <api-key>"
It gives the following output to me:
$ curl -X POST https://api.mailerlite.com/api/v2/campaigns \
> -d '{
> "subject": "Regular campaign subject",
> "segments": 1291259,
> "type": "regular"
> }' \
> -H "Content-Type: application/json" \
> -H "X-MailerLite-ApiKey: <api-key>"
{"campaign_type":"regular","date":"2020-10-06 19:21:47","account_id":<id>,"id":<id>,"mail_id":<id>,"options":{"current_step":"step3","send_type":"regular","campaign_type":"regular","date":"2020-10-06 19:21:47"}}

Related

Mutiple varaibles in while loop of bash scripting

Suppose i have the below curl where i will be reading two of the varaibles from a file.how can we accomodate both the varaibles in a single while loop
while read p; do
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'HTTP_X_MERCHANT_CODE: STA' --header 'AK-Client-IP: 135.71.173.56' --header 'Authorization: Basic qwrewrereererweer' -d '{
"request_details": [
{
"id": "$p", #first dynamic varaible which will be fetched from the file file.txt
"id_id": "$q", #second dynamic varaible to be fetched from the file file.txt
"reason": "Pickup reattempts exhausted"
}
]
}' api.stack.com/ask
done<file.txt
file.tx will have two columns from which the dynamic variables whill be fetched to the above curl. Pls let me know how can we accomodate both the variable in the above while loop
i will need bit of a help regarding the same
Since you'll want to use a tool like jq to construct the payload anyway, you should let jq parse the file instead of using the shell.
filter='split(" ") |
{ request_details: [
{
id: .[0],
id_id: .[1],
reason: "Pickup reattempts exhausted"
}
]
}'
jq -cR "$filter" file.txt |
while IFS= read -r payload; do
curl -X POST --header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'HTTP_X_MERCHANT_CODE: STA' \
--header 'AK-Client-IP: 135.71.173.56' \
--header 'Authorization: Basic qwrewrereererweer' \
-d "$payload"
done
The -c option to jq ensures the entire output appears on one line: curl doesn't need a pretty-printed JSON value.
read accepts multiple target variable names. The last one receives all the content not yet read from the line. So read p reads the whole line, read p q would read the first token (separated by whitespace) into p and the rest into q, and read p q r would read the first two tokens into p and q and any remaining junk into r (for example if you want to support comments or extra tokens in file.txt).

argument list too long curl

Trying to solve "argument list too long"
I have been searching for a solution and found the closest one to my issue
curl: argument list too long
however the response is not clear as I am still having the issue "argument list too long"
curl -X POST -d #data.txt \
https://Path/to/attachments \
-H 'content-type: application/vnd.api+json' \
-H 'x-api-key: KEY' \
-d '{
"data": {
"type": "attachments",
"attributes": {
"attachment": {
"content": "'$(cat data.txt | base64 --wrap=0)'",
"file_name": "'"$FileName"'"
}
}
}
}'
thank you
Use jq to format your base64 encoded data string into a proper JSON string, and then pass the JSON data as standard input to the curl command.
#!/usr/bin/env sh
attached_file='img.png'
# Pipe the base64 encoded content of attached_file
base64 --wrap=0 "$attached_file" |
# into jq to make it a proper JSON string within the
# JSON data structure
jq --slurp --raw-input --arg FileName "$attached_file" \
'{
"type": "attachments",
"attributes": {
"attachment": {
"content": .,
"file_name": $FileName
}
}
}
' |
# Get the resultant JSON piped into curl
# that will read the data from the standard input
# using -d #-
curl -X POST -d #- \
'https://Path/to/attachments' \
-H 'content-type: application/vnd.api+json' \
-H 'x-api-key: KEY'
Per the linked answer
you are trying to pass the entirety of the base64'd content on the command line
This is a limitation of the shell, not curl. That is, the shell is responding with error argument list too long. The program curl is never even started.
The recommendation is
curl has the ability to load in data to POST from a file
Write the json data to some file /tmp/data.json using piping.(the commands will use piping | and file redirection > >> which can handle arbitrarily large amounts of data. Whereas, you cannot place arbitrarily large amounts of data into a single command, there is a limit).
echo -n '
{
"data": {
"type": "attachments",
"attributes": {
"attachment": {
"content": "' > /tmp/data.json
cat data.txt | base64 --wrap=0 >> /tmp/data.json
echo -n '",
"file_name": "'"$FileName"'"
}
}
}
}' >> /tmp/data.json
Pass that file path /tmp/data.json to the curl command using # so curl knows it's a file path.
curl -X POST -d #/tmp/data.json \
"https://Path/to/attachments" \
-H 'content-type: application/vnd.api+json' \
-H 'x-api-key: KEY'

for loop over sequence, then make string and post via json

Consider this simple bash script : (I am writing myurl as a placeholder for the correct URL; and myhost as the placeholder for the correct hostname for privacy here.)
for a in `seq 400 450`; do
curl --request POST \
--url myurl \
--header 'Content-Type: application/json' \
--header 'Host: myhost' \
--data '{
"devID": "21010010",
"count": 0,
"fileEnd": 1,
"line": "",
"file": "$a"
}'
done
This is rejected as bad request. I want, the output of seq, (e.g. the number 410) to be a quote delimated string. Notice the line "file": "$a". If I write "file": "410", this works perfectly, and also for any other file between 400 and 450 - as long as it is surrounded by "" quotes.
Question :
I want, that in each iteration of the loop, the curl request should include the number coming from the seq command surrounded by "". So the first call should be :
curl --request POST \
--url myurl \
--header 'Content-Type: application/json' \
--header 'Host: myhost' \
--data '{
"devID": "21010010",
"count": 0,
"fileEnd": 1,
"line": "",
"file": "400"
}'
The next one to be :
curl --request POST \
--url myurl \
--header 'Content-Type: application/json' \
--header 'Host: myhost' \
--data '{
"devID": "21010010",
"count": 0,
"fileEnd": 1,
"line": "",
"file": "401"
}'
and so on.
How can I do this? Thank you.
The --data portion is wrapped in single quotes ...
'{"devID": ... "file": "$a"}'
... which disables the expansion of variable references inside said single quotes.
Consider:
$ a=410
$ echo '"devID": "21010010", "file": "${a}"'
"devID": "21010010", "file": "${a}" # ${a} not expanded
You could wrap the --data section in double quotes, but then you need to also escape all of the nested double quotes.
Another idea is to break the --data section into 2x sections wrapped in single quotes and place the ${a} reference between the 2x sections, eg:
$ echo '"devID": "21010010", "file": "'${a}'"' # change "${a}" to "'${a}'"
"devID": "21010010", "file": "410" # ${a} successfully expanded
NOTE: There are several posts regarding the difference between single and double quotes, eg, this link

Scripted concatenation of strings in curl command payload

I use curl to test an user account creation API as follows:
curl -s -X POST "https://$APISERVER/users" \
-H 'Content-Type: application/json' \
-d '{ \
"username": "'$NEWUSERNAME'", \
"firstName": "'$NEWUSERFIRSTNAME'", \
"lastName": "'$NEWUSERLASTNAME'", \
"displayName": "'$NEWUSERDISPLAYNAME'", \
"password": "'$NEWUSERPASSWORD'" \
}'
and the variables are supplied via command line arguments:
APISERVER=http://localhost:8080
NEWUSERNAME=$1
NEWUSERPASSWORD=$2
NEWUSERFIRSTNAME=$3
NEWUSERLASTNAME=$4
# Calculated variable
NEWUSERDISPLAYNAME="${NEWUSERFIRSTNAME} ${NEWUSERLASTNAME}"
An example invocation of the script is as follows: ./test-new-user.sh jdoe Hello123 John Doe, resulting in the following variable values:
NEWUSERNAME=jdoe
NEWUSERPASSWORD=Hello123
NEWUSERFIRSTNAME=John
NEWUSERLASTNAME=Doe
(I intended NEWUSERDISPLAYNAME to be set to "John Doe")
But I get back an exception from the server, because the payload in the curl command appears to be cut-off, incomplete or malformed.
JSON parse error: Unexpected end-of-input in VALUE_STRING\n at [Source:
java.io.PushbackInputStream#2eda6052; line: 1, column: 293]; nested
exception is com.fasterxml.jackson.databind.JsonMappingException:
Unexpected end-of-input in VALUE_STRING\n at [Source:
java.io.PushbackInputStream#2eda6052; line: 1, column: 293]\n at
[Source: java.io.PushbackInputStream#2eda6052; line: 1, column: 142]
(through reference chain:
com.mycompany.api.pojos.NewUser[\"displayName\"])"
If I hard code value for displayName in the above curl command (as below), the user creation request goes through and works perfectly.
"displayName": "John Doe", \
I suspect it has to do with the space in displayName and how I insert the value for displayName using "'$NEWUSERDISPLAYNAME'". Is there a safe way to perform variable substitution in the curl command's POST request payload?
You need to quote shell variables:
curl -s -X POST "https://$APISERVER/users" \
-H 'Content-Type: application/json' \
-d '{ \
"username": "'"$NEWUSERNAME"'", \
"firstName": "'"$NEWUSERFIRSTNAME"'", \
"lastName": "'"$NEWUSERLASTNAME"'", \
"displayName": "'"$NEWUSERDISPLAYNAME"'", \
"password": "'"$NEWUSERPASSWORD"'" \
}'
In order to avoid excessive quoting, try this printf:
printf -v json -- '{ "username": "%s", "firstName": "%s", "lastName": "%s", "displayName": "%s", "password": "%s" }' \
"$NEWUSERNAME" "$NEWUSERFIRSTNAME" "$NEWUSERLASTNAME" "$NEWUSERDISPLAYNAME" "$NEWUSERPASSWORD"
curl -s -X POST "https://$APISERVER/users" \
-H 'Content-Type: application/json' \
-d "$json"
just use
$(echo $varname)
works for me

IBM Watson speech to test output only transcript or grep only transcript

I'm using curl with IBM Watson to produce a transcript but I can't seem to get an output where just the transcript is shown as shown below
Another method might be just to grep for the text in transcript":""
curl
curl -u user:password -X POST --header "Content-Type: audio/wav" --header "Transfer-Encoding: chunked" --data-binary #test.wav "https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true" > demo.txt
{
"results": [
{
"alternatives": [
{
"confidence": 0.302,
"transcript": "when to stop announced "
}
],
"final": true
},
{
"alternatives": [
{
"confidence": 0.724,
"transcript": "Russia is destroying western cheese and considering a ban Weston condoms and infection internet is reacting "
}
],
"final": true
To store the output into a file you can use the option -o
curl -u user:password -X POST --data-binary #test.wav
-o transcript.txt
--header "Content-Type: audio/wav" --header "Transfer-Encoding: chunked"
"https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true"
-o, --output <file>
Write output to <file> instead of stdout. Like in:
curl http://ibm.com.com -o "html_output.txt"
More info.
grep alone can't do quite what you're asking for because it doesn't get more granular than a single line. However, grep + sed can - sed can be used to perform regex replacement on the lines that grep spits out.
First pipe to grep: grep transcript, then pipe to sed: sed 's/.*"transcript": "\(.*\)".*/\1/g'
Here's the complete command:
curl -u user:password -X POST --header "Content-Type: audio/wav" --header "Transfer-Encoding: chunked" --data-binary #test.wav "https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true" | grep transcript | sed 's/.*"transcript": "\(.*\)".*/\1/g'
Note that by default, curl displays several lines of status when piping it's output. You can disable those with -s:
curl -s -u user:password -X POST --header "Content-Type: audio/wav" --header "Transfer-Encoding: chunked" --data-binary #test.wav "https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true" | grep transcript | sed 's/.*"transcript": "\(.*\)".*/\1/g'
Here's more info on sed if you're interested: http://www.grymoire.com/Unix/Sed.html
Update: I should also mention that there are SDKs available for a number of languages if you're interested in doing something more complex - https://github.com/watson-developer-cloud

Resources