Hiding data/ password while trying curl - bash [duplicate] - bash

I need to make a POST request via cURL from the command line. Data for this request is located in a file. I know that via PUT this could be done with the --upload-file option.
curl host:port/post-file -H "Content-Type: text/xml" --data "contents_of_file"

You're looking for the --data-binary argument:
curl -i -X POST host:port/post-file \
-H "Content-Type: text/xml" \
--data-binary "#path/to/file"
In the example above, -i prints out all the headers so that you can see what's going on, and -X POST makes it explicit that this is a post. Both of these can be safely omitted without changing the behaviour on the wire. The path to the file needs to be preceded by an # symbol, so curl knows to read from a file.

I need to make a POST request via Curl from the command line. Data for this request is located in a file...
All you need to do is have the --data argument start with a #:
curl -H "Content-Type: text/xml" --data "#path_of_file" host:port/post-file-path
For example, if you have the data in a file called stuff.xml then you would do something like:
curl -H "Content-Type: text/xml" --data "#stuff.xml" host:port/post-file-path
The stuff.xml filename can be replaced with a relative or full path to the file: #../xml/stuff.xml, #/var/tmp/stuff.xml, ...

If you are using form data to upload file,in which a parameter name must be specified , you can use:
curl -X POST -i -F "parametername=#filename" -F "additional_parm=param2" host:port/xxx

Most of answers are perfect here, but when I landed here for my particular problem, I have to upload binary file (XLSX spread sheet) using POST method, I see one thing missing, i.e. usually its not just file you load, you may have more form data elements, like comment to file or tags to file etc as was my case. Hence, I would like to add it here as it was my use case, so that it could help others.
curl -POST -F comment=mycomment -F file_type=XLSX -F file_data=#/your/path/to/file.XLSX http://yourhost.example.com/api/example_url

I was having a similar issue in passing the file as a param. Using -F allowed the file to be passed as form data, but the content type of the file was application/octet-stream. My endpoint was expecting text/csv.
You are able to set the MIME type of the file with the following syntax:
-F 'file=#path/to/file;type=<MIME_TYPE>
So the full cURL command would look like this for a CSV file:
curl -X POST -F 'file=#path/to/file.csv;type=text/csv' https://test.com
There is good documentation on this and other options here: https://catonmat.net/cookbooks/curl/make-post-request#post-form-data

I had to use a HTTP connection, because on HTTPS there is default file size limit.
https://techcommunity.microsoft.com/t5/IIS-Support-Blog/Solution-for-Request-Entity-Too-Large-error/ba-p/501134
curl -i -X 'POST' -F 'file=#/home/testeincremental.xlsx' 'http://example.com/upload.aspx?user=example&password=example123&type=XLSX'

Related

Redirect a cURL response to a cURL that POSTs, but not through a file

I 'd like to post directly a json object from a url(json) to another url
so the command goes as follows:
curl "<resource_link>.json" -o sample.json
curl -X POST "<my_link>" "Content-type: application/json" -d #sample.json
I 'd like to avoid this, so what is the solution? Is it something like that?
curl -X POST "<my_link>" "Content-type: application/json" -d "curl <resource_link>.json"
But it does not work? Also, this one post Stream cURL response to another cURL command posting the result
does not explain thouroughly and it is not working
Yes,
curl
manual explains the '#' but it does not explain about using another curl
Alternatievely, if I could save somewhere temporarily the 1st cURL response and use it in the other command(but not in a file)
You don't want -x POST in there so let's start with dropping that.
Send the results from the first transfer to stdout by not using -o, or telling -o to use stdout with -o-, and
Make sure your second transfer accepts the data to send on stdin, by using -d#-.
curl "<link>.json" | curl "<link2>" -H "Content-type: application/json" -d #-
With curl 7.82.0 and later
Starting with curl 7.82.0 you can do it even easier with the new --json option:
curl "<link>.json" | curl "<link2>" --json #-

Sending large file as data inside JSON via cURL

I'm trying to send a file as base64-encoded data via POST to the Bugzilla REST API as follows:
curl -X POST https://www.example.com/rest/bug/$id/attachment -H "Content-Type: application/json" \
-d "{\
\"login\" : \"$username\", \
\"password\" : \"$password\", \
\"ids\" : [ $id ], \
\"summary\" : \"...\", \
\"content_type\" : \"application/gzip\", \
\"data\" : \"$data\"\
}"
What I'm getting is an error from cURL that the argument list is too long. Presumably, this is because the file ($data) I'm trying to send is more than the shell maximum (the file is 11M). What I've seen online is that the best way to get around that is to have cURL read the data from a file using --data-binary. But since I need to send a username and password, I'd prefer not to have to save the entire file with them inside.
Is there some way to get around this maximum, or is there another way to send a large amount of data this way? I prefer native Linux tools, as I want this script to be portable.
You can try using a file like this, which is the recommended way.
curl -i \
-H 'Accept:application/json' \
-H 'Authorization:Basic $username:$password' \
-X POST -d #datafile.txt https://www.example.com/rest/bug/$id/attachment

Quoting of arguments passed to Curl command in Script file

I have been trying to execute curl command from a script file.
The command is as follows (the IP address is stored in variable ip):
curl -s -X POST ${ip1} -H \"content-type: application/json\" -d \''{"args":["org1","scatest'$j$i'","27-06-2018"]}'\'
It's throwing an error saying "cannot read property", if I execute from the script file.
Whereas if I execute from the command line then there is no problem.
Can anybody help me out why curl command is not getting executed from the script file?
Since the discussions under the comment has lead to a solution that works well. I'm gonna put it into answer for the record and in case anyone else runs into the same pitfall.
The problem was caused by excessive quoting which would result in extra quotes being made part of the request, correct form of the command was:
curl -s -X POST ${ip1} -H "content-type: application/json" \
-d '{"args":["org1","scatest'$j$i'","27-06-2018"]}'

POST multiple files with -d in curl

I'm using curl to create several classifications. I have written the json for the many classifications and they are in one folder. I would like to create all the classifications in one go. But using curl I can only create them one at a time. How could I make them in one request?
curl -u admin:admin -H "Content-Type: application/json" -X POST -d #pii.json http://127.0.0.1:21000/api/atlas/v2/types/typedefs
The curl manual for -d says 'Multiple files can also be specified'. How can I do this? All my attempts have failed.
Do I need a bash script instead? If so, could you help me - I'm not a coder and I'm struggling without an example!
Thanks in advance.
You probably don't want to use multiple -d with JSON data since curl concatenates multiple ones with a & in between. As described in the man page for -d/--data:
If any of these options is used more than once on the same command
line, the data pieces specified will be merged together with a
separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would
generate a post chunk that looks like 'name=daniel&skill=lousy'.
You can however easily and conveniently pass several files on stdin to let curl use them all in one go:
cat a.json b.json c.json | curl -d#- -u admin:admin -H "Content-Type: application/json" http://127.0.0.1:21000/api/atlas/v2/types/typedefs
(please note that -X POST has no place on a command line that uses -d)
I found the following to work in the end:
<fileToUpload.dat xargs -I % curl -X POST -T "{%}" -u admin:admin -H "Content-Type: application/json" http://127.0.0.1:21000/api/atlas/v2/types/typedefs
Where fileToUpload.dat contained a list of the .json files.
This seemed to work over Daniel's answer, probably due to the contents of the files. Hopefully this is useful to others if Daniel's solution doesn't work for them.
I needed to upload all the *.json files from a folder via curl and I made this little script.
nfiles=*.json
echo "Enter user:"
read user
echo "Enter password:"
read -s password
for file in $nfiles
do
echo -e "\n----$file----"
curl --user $user:$password -i -X POST "https://foo.bar/foo/bar" -H "Content-Type: application/json" -d "#$file"
done
Maybe fits your needs.

< was unexpected at this time. from curl command line when posting input data as an xml to rest service

when i post a raw string as an input to JSON REST Service call it is executing ex:
curl -d "{\"input1\": \"as\", \"input2\": \"ad\"}" -i -X POST -H "Content-Type:application/json" http://localhost/rtygies/Service1.svc/rest/receivedata1
But when i am posting as an xml as input it is giveng error as below:
curl -d "{\"input1\": \"<xml></xml>\", \"input2\": \"<xml></xml>\"}" -i -X POST -H "Content-Type:application/json" http://localhost/rtygies/Service1.svc/rest/receivedata1
Error: < was unexpected at this time
I am using curl in windows.
can any one say how to post xml as a string input to Rest service in JSON format from curl
Actually, I don't know what is the case,
but I got an error said < is unexpected this time
when I tried the following command (on windows):
curl -u admin:password -XPOST -H 'Content-type: text/xml' -d '<namespace><prefix>newWorkspace</prefix><uri>http://geoserver.org</uri></namespace>' http://localhost:8080/geoserver/rest/namespaces
Then I changed the single quote into double quotes and it worked.
Redirection symbols can be escaped by "
Try using
curl -d "{\"input1\": \""<xml></xml>"\", \"input2\": \""<xml></xml>"\"}" -i -X POST -H "Content-Type:application/json" http://localhost/rtygies/Service1.svc/rest/receivedata1
I had no luck with double-quotes, however escaping < and > with ^ worked for me.
So your curl becomes..
curl -d "{\"input1\": \"^<xml^>^</xml^>\", \"input2\": \"^<xml^>^</xml^>\"}" -i -X POST -H "Content-Type:application/json" http://localhost/rtygies/Service1.svc/rest/receivedata1

Resources