I was reading the JIRA REST API tutorial and in their example curl request, they show
curl -D- -u username:password <rest-of-request>
What is the -D- syntax with the dash before and after?
To quote man curl:
-D, --dump-header <file>
Write the protocol headers to the specified file.
This option is handy to use when you want to store
the headers that a HTTP site sends to you. C
After a -D you normally give the name of the file where you want to dump the headers. As with many utilities, - is recognized as an alias to stdout. (if you're not familiar with that concept: when you launch the command from a terminal without redirection, stdout is the "terminal screen")
The -D- (without space) form is exactly the same as -D - (or on Linux at least, -D /dev/stdout)
Related
I have something like that (it replaces a special sequence in request and then it sends it with curl).
SPECIAL_SEQUENCE=My_value
sed -i -e "s|SPECIAL_SEQUENCE|$SPECIAL_SEQUENCE|g" file.txt
curl http://127.0.0.1:1478/ -X POST -d #file.txt
It does work ok. However, the problem for me is that it leaves changed the file file.txt. I could undo sed at the end of this script, but I wouldn't like do this (becase I often interrupt this script with Ctrl+C).
Can you give me some other ideas to deal with this ? In other words the final form of request can be known only during executing mentioned script.
Instead of modifying the file, modify it in the memory and pipe it to curl:
SPECIAL_SEQUENCE=My_value
sed -e "s|SPECIAL_SEQUENCE|$SPECIAL_SEQUENCE|g" file.txt | curl http://127.0.0.1:1478/ -X POST
I have five cURL statements that work fine by themselves and am trying to put them together in a bash script. Each cURL statement relies on a variable generated from a cuRL statement executed before it. I'm trying to figure out the smartest way to go about this. Here is the first cURL statement;
curl -i -k -b sessionid -X POST https://base/resource -H "Content-Type: application/json" -H "Authorization: Authorization: PS-Auth key=keyString; runas=userName; pwd=[password]" -d "{\"AssetName\":\"apiTest\",\"DnsName\":\"apiTest\",\"DomainName\":\"domainNameString\",\"IPAddress\":\"ipAddressHere\",\"AssetType\":\"apiTest\"}"
This works fine, it produces this output;
{"WorkgroupID":1,"AssetID":57,"AssetName":"apiTest","AssetType":"apiTest","DnsName":"apiTest","DomainName":"domainNameString","IPAddress":"ipAddressHere","MacAddress":null,"OperatingSystem":null,"LastUpdateDate":"2017-10-30T15:18:05.67-07:00"}
However, in the next cURL statement, I need to use the integer from AssetID in order to execute it. In short, how can I take the AssetID value and store it to a variable to be used in the next statement? In total, I'll be using 5 cURL statements and they rely on values generated in the preceeding statement to execute. Any insight on how is appreciated.
Download and install jq which is like sed for JSON data. You can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep does for unstructured data. Remember to replace '...' with your actual curl arguments
curl '...' | jq --raw-output '.AssetID'
and to store it in a variable use command-substitution syntax to run the command and return the result.
asset_ID=$( curl '...' | jq --raw-output '.AssetID' )
In the curl command, drop the -i flag to output only the JSON data without the header information.
I am trying to feed a GeoJSON to this data service using the following bash code.
curl -X POST -F "shape=$(cat myfile.geojson)" \
-F 'age=69' -o reconstructed_myfile.geojson \
https://dev.macrostrat.org/reconstruct
However, I am getting an "Argument list too long" error. I see a lot of questions open on stack related to this issue, but I do not understand how to convert the answers given in those threads to this specific case.
You should use <filename or #filename:
curl -X POST \
-F 'shape=<myfile.geojson' \
-F 'age=69' \
-o 'reconstructed_myfile.geojson' \
-- 'https://dev.macrostrat.org/reconstruct'
See man curl for details:
$ man curl | awk '$1 ~ /-F/' RS=
-F, --form <name=content>
(HTTP) This lets curl emulate a filled-in form in which a user has
pressed the submit button. This causes curl to POST data using the
Content-Type multi‐ part/form-data according to RFC 2388. This
enables uploading of binary files etc. To force the 'content' part to
be a file, prefix the file name with an # sign. To just get the
content part from a file, prefix the file name with the symbol <. The
difference between # and < is then that # makes a file get
attached in the post as a file upload, while the < makes a text field
and just get the contents for that text field from a file.
Looking at the man page for cURL:
-w, --write-out <format>
Make curl display information on stdout after a completed transfer.
Where it is possible to use this flag and append a string to the output of cURL. However I can only get this to append to the end of the output of cURL, because as the man page suggests, the -w flag appends after a completed transfer.
so doing:
curl -sS "http:/somewebsite" -w "hello_world"
will produce:
$
contentfromcurl
hello_world
....well how do you get the output to be
$
hello_worldcontentfromcurl
i.e. is it possible to get -w to prepend rather than append?
thanks to #Adrian, this is the final answer -
curl -sS "http:/somewebsite" | xargs echo "mystring"
cheers!
If you're really desperate you can make a code block and include an echo. The following will have the output you're looking for:
{ echo -n "hello_world"; curl -sS "http:/somewebsite"; }
As for getting the -w option to prepend, the answer is no:
-w, --write-out
Make curl display information on stdout after a completed transfer. The format is a string ...
Is this what you are after?
$ printf "bar\nquux\n"
bar
quux
$ printf "bar\nquux\n" | sed 's#^#foo#g'
foobar
fooquux
Obviously, you would replace printf with your curl invocation.
But this seems a bit like an XY-problem - what are you trying to accomplish?
Some research revealed a few useful stackexchange posts, namely expanding variable in CURL, but that given answer doesn't seem to properly handle bash variables that have spaces in them.
I am setting a variable to the output of awk, parsing a string for a substring (actually truncating to 150 characters). The string I am attempting to POST via curl has spaces in it.
When I use the following curl arguments, the POST variable Body is set to the part of the string before the first space.
curl -X POST 'https://api.twilio.com/2010-04-01/Accounts/GUID/SMS/Messages.xml' -d 'From=DIDfrom' -d 'To=DIDto' -d 'Body="'$smsbody'" -u SECGUID
smsbody is set as:
smsbody="$(echo $HOSTNAME$ $SERVICEDESC$ in $SERVICESTATE$\: $SERVICEOUTPUT$ | awk '{print substr($0,0,150)}')"
So the only portion of smsbody that is POSTed is $HOSTNAME$ (which happens to be a string without any space characters).
What is the curl syntax I should use to nest the bash variable properly to expand, but be taken as a single data field?
Seems pretty trivial, but I messed with quotes for a while without luck. I figure someone with better CLI-fu can handle it in a second.
Thanks!
It looks like you have an extra single quote before Body. You also need double quotes or the $smsbody won't be evaluated.
Try this:
curl -X POST 'https://api.twilio.com/2010-04-01/Accounts/GUID/SMS/Messages.xml' \
-d 'From=DIDfrom' -d 'To=DIDto' -d "Body=$smsbody" -u SECGUID
If the $s are still an issue (I don't think spaces are), try this to prepend a \ to them:
smsbody2=`echo $smsbody | sed 's/\\$/\\\\$/g'`
curl -X POST 'https://api.twilio.com/2010-04-01/Accounts/GUID/SMS/Messages.xml' \
-d 'From=DIDfrom' -d 'To=DIDto' -d "Body=$smsbody2" -u SECGUID
If I run nc -l 5000 and change the twilio address to localhost:5000, I see the smsbody variable coming in properly.
matt#goliath:~$ nc -l 5000POST / HTTP/1.1
Authorization: Basic U0VDR1VJRDphc2Q=
User-Agent: curl/7.21.6 (x86_64-apple-darwin10.7.0) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.5 libidn/1.20
Host: localhost:5000
Accept: */*
Content-Length: 45
Content-Type: application/x-www-form-urlencoded
From=DIDfrom&To=DIDto&Body=goliath$ $ in $: