Sending cURL data to multiple URLs - bash

My code consists of the following:
curl -k -X DELETE \
-H "X-Parse-Application-Id: staticid" \
-H "X-Parse-Master-Key: statickey" \
https://mystatic.url/dynamicvalue
I have a list of URLS:
https://mystatic.url/johndylan
https://mystatic.url/marypoppins
etc, included in a tobedeleted.txt file and I would like to modify my cURL code to be something like this (which I've tried but didnt work:
curl -k -X DELETE \
-H "X-Parse-Application-Id: staticid" \
-H "X-Parse-Master-Key: statickey" \
> tobedeleted.txt
or to something like this (which also I've tried but it didnt worked)
curl -k -X DELETE \
-H "X-Parse-Application-Id: staticid" \
-H "X-Parse-Master-Key: statickey" \
https://mystatic.url/$tobedeleted.txt
Note that i want to run the same cURL command, each time for each line of the file, so I guess that I would need something like foreach function, since this is a bash script.

You are going to want something like this:
while read value
do
curl -k -X DELETE \
-H "X-Parse-Application-Id: staticid" \
-H "X-Parse-Master-Key: statickey" \
https://mystatic.url/$value
done < tobedeleted.txt

Related

How to get the exact path of files in a folder using bash script?

I have a folder wherein there are two images
dog.jpeg
dog2.jpeg
and I need to send their path in the following queryImage.
#!/bin/bash
for i in /Users/user/Desktop/Short/*;
do
curl --location --request POST 'some_url' \
--header 'x-api-key: dummy-api-key' \
--form 'maxResults="25"' \
--form 'minConfidence="0.1"' \
**--form 'queryImage=#"$i"'**
done
When I try to give the path directly in the following way, we are able to retrieve the path and we get the required output.
#!/bin/bash
for i in /Users/user/Desktop/Short/*;
do
curl --location --request POST 'some_url' \
--header 'x-api-key: dummy-api-key' \
--form 'maxResults="25"' \
--form 'minConfidence="0.1"' \
**--form 'queryImage=#"/Users/user/Desktop/Short/dog.jpeg"'**
done
What am I giving wrong in the first one? How can we get the exact path in queryImage?
Please help. I am a newbie and can't share the details as this is for an internal project.
Thank you :)
Single quotes prevent the substitution at $i.
Use double quotes instead:
#!/bin/bash
for i in /Users/user/Desktop/Short/*
do
curl --location --request POST 'some_url' \
--header 'x-api-key: dummy-api-key' \
--form 'maxResults="25"' \
--form 'minConfidence="0.1"' \
**--form "queryImage=#'$i'"**
done

How to download file(*.txt or *.pdf) which received from GET api response through Curl command?

I want to download files(*.txt and *.pdf) from Ariba site through GET api request and want to automate the whole download process.
Initially I have used Postman for testing purpose which gives me result in the form of file content.
For eg. test.txt file is present on remote site, after GET request from Postman, the result I am getting, it is in form of content of file, so if 'abc' is written in file, I am getting 'abc' as response from test.txt file.
Now if I click on Send and Download button in Postman it gives me option to download file 'test.txt'
I have to automate this process to send GET response and get required file download at specific location. I am trying to use Curl script for this.
I have written corresponding Curl script and tried to execute it.
It gives me response in the form of file content.
curl -X GET \
'https://openapi.ariba.com/api/approval/v1/prod/invoices/INVASINV6-902/attachments/bnMyMDE5LzA0LzAzLzE1MjkyNDE4MQ==?realm=ProjectName&Content-Disposition=attachement' \
-H 'Accept: */*' \
-H 'Authorization: Bearer 7648d29a-db04-4046-b49c-5daed43a145c' \
-H 'Cache-Control: no-cache' \
-H 'Connection: keep-alive' \
-H 'Host: openapi.ariba.com' \
-H 'accept-encoding: gzip, deflate' \
-H 'apiKey: xxxxxxxxxxxxxxxxxx' \
-H 'cache-control: no-cache'
I want to write a curl script which will download file at specific location.
For eg. Above curl command give content(abc) from file Test.txt and not file Test.txt as output
Really appreciate your helpCurl Get Response
Postman Get Response
Finally I got answer to the question, I just need to remove -X GET from my curl script and at the end add -o to download file name.
Below is the final code:
curl 'https://openapi.ariba.com/api/approval/v1/prod/invoices/INVASINV6-902/attachments/bnMyMDE5LxxxxLzE1MjkyNDE4MQ==?realm=ProjectName' \
-H 'Accept: */*' \
-H 'Authorization: Bearer 77876887-xxxxx-42fb-b865-9cf8ff5c2b25' \
-H 'Cache-Control: no-cache' \
-H 'Connection: keep-alive' \
-H 'Host: openapi.ariba.com' \
-H 'accept-encoding: gzip, deflate' \
-H 'apiKey: XXXXXXXXXXXXXXXX' \
-H 'cache-control: no-cache' \
-H 'Content-Type: application/octet-stream' \
-H "Content-Transfer-Encoding: Binary" \
-o "Test.txt"
Above code give me file downloaded at specific location.
Thanks BlackPearl.

Unterminated quoted string in Capistrano curl command

I've tried just about every combination of single quotes, double quotes and backslashes that I can think of. If anyone could please point out what I'm doing wrong that'd be greatly appreciated.
run_locally do
execute "\\curl -X POST \"https://api.cloudflare.com/client/v4/zones/#{fetch(:cloudflare_zone)}/purge_cache\" \
-H \"X-Auth-Email: test#example.com\" \
-H \"X-Auth-Key: #{fetch(:cloudflare_api)}\" \
-H \"Content-Type: application/json\" \
--data {\"purge_everything\":true}\""
end
My capistrano script dies when it hits this every time. I feel like it's that last line but I'm not sure why.
Edit: I've gotten past that error, but now get "Malformed JSON in request body" back from Cloudflare.
run_locally do
execute "\\curl -X POST \"https://api.cloudflare.com/client/v4/zones/#{fetch(:cloudflare_zone)}/purge_cache\" \
-H \"X-Auth-Email: test#example.com\" \
-H \"X-Auth-Key: #{fetch(:cloudflare_api)}\" \
-H \"Content-Type: application/json\" \
--data \"{\"purge_everything\":true}\" "
end
So for some reason it doesn't like my --data section.
I'm not able to check this directly for you, but you can use other string constructs to do this in a clearer way which will hopefully highlight the issue for you.
run_locally do
execute %{curl -X POST "https://api.cloudflare.com/client/v4/zones/#{fetch(:cloudflare_zone)}/purge_cache" \
-H "X-Auth-Email: test#example.com" \
-H "X-Auth-Key: #{fetch(:cloudflare_api)}" \
-H "Content-Type: application/json" \
--data '{"purge_everything":true}'}
end
It's probably the nested double quotes in your JSON causing the issue.

Nested cURL call

I've one cron task to update my DDNS with my current ip address and do it through a cURL call.
The problem is one of the parameters to pass in the call is the CURRENT IP and in order to discover ir i need to do another cURL call.
I would like to know if is possible to nest two cURL calls in one single script in order to make my cron task avoiding extra scripts
example:
to get my current ip I use
curl ipinfo.io/ip
to update my ddns i need to do:
curl -X PUT "https://api.cloudflare.com/client/v4/zones/2wertyh/dns_records/23ertghj" \
-H "X-Auth-Email: tomatechines#gmail.com" \
-H "X-Auth-Key: 123ertgyh" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"qwsdfg.com.br","content":"MY-CURRENT-IP","ttl":1800,"proxied":false}'
how can i fit this two calls together in order to make my cron task
Use command substitution, like this:
curl -X PUT "https://api.cloudflare.com/client/v4/zones/2wertyh/dns_records/23ertghj" \
-H "X-Auth-Email: tomatechines#gmail.com" \
-H "X-Auth-Key: 123ertgyh" \
-H "Content-Type: application/json" \
--data '{"type":"A","name":"qwsdfg.com.br","content":"'"$(curl ipinfo.io/ip)"'","ttl":1800,"proxied":false}'
String argument for --data is composed from three concatenated parts, 'beginning' "$(curl ...)" 'ending' (more details see in this answer).

bash encapsulate command options in variables

I have a bunch of these to test my RESTful API
$CURL \
-v \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
-X POST \
-d '{"user":{"email":"user#example.com","password":"secret"}}' \
$URL/$PATH/sessions
I kinda want to shorten it to something like
CURLOPTS="-v -H 'Content-Type: application/json' -H 'Accept: application/json'"
$CURL \
$CURLOPTS \
-X POST \
-d '{"user":{"email":"user#example.com","password":"secret"}}' \
$URL/$PATH/sessions
but the options don't seem to be passed in. Any clues?
Short answer: see BashFAQ #50:'m trying to put a command in a variable, but the complex cases always fail!.
Long answer: Putting commands (or parts of commands) into variables and then getting them back out intact is complicated. The reason your script doesn't work is because of the order in which the shell parses the command line: it parses (and removes) quotes and escapes, then replaces variable values. By the time $CURLOPTS gets replaced, it's too late for the quotes to have their intended effect; instead, they're passed to curl as part of the arguments, which confuses curl greatly.
The solution: store the options in an array rather than a plain string:
CURLOPTS=(-v -H 'Content-Type: application/json' -H 'Accept: application/json')
$CURL \
"${CURLOPTS[#]}" \
-X POST \
-d '{"user":{"email":"user#example.com","password":"secret"}}' \
"$URL/$PATH/sessions"
You can use an array and trigger word splitting
$ set -x
$ CURLOPTS=(-v -H 'Content-Type: application/json' -H 'Accept: application/json')
$ : curl "${CURLOPTS[#]}"
+ : curl -v -H 'Content-Type: application/json' -H 'Accept: application/json'

Resources