Figuring out a new API.
I’m trying to call an endpoint and populate a JSON file with the data. I see the following output:
admin#server:~$ curl -H "Authorization: Bearer <NOT DISPLAYED FOR THIS POST>" -o /home/admin/result.json https://www.endpoint.com/manage/query/run?id=55408&cmd=service&output=json
[1] 14493
[2] 14494
admin#server:~$ % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
It hangs here seemingly indefinitely until I hit enter and then the below displays
[1]- Done curl -H "Authorization: Bearer <NOT DISPLAYED FOR THIS POST>" -o /home/admin/result.json https://www.endpoint.com/manage/query/run?id=55408
[2]+ Done cmd=service
No error messages, and no data in result.json. Calling the same cURL command without the -o option also returns the same results, when normally I would expect to see the data pop up in my terminal. If I visit the endpoint URL in browser (auth token can be a URL parameter as well for this API), I see the exact data I want to download. Changing the Auth Token makes no difference in the output.
I know every API is different, and there are a hundred different troubleshooting questions I haven't addressed in this post, but has anyone experienced this type of output before with a cURL command? I've never seen this behavior before.
The API is for Slate, an SIS for universities, if that helps. Thank you!
Your URL has a & in it which is a bash syntax character to run a task in the background. Quote the URL to prevent it being interpreted as syntax.
admin#server:~$ curl -H "Authorization: Bearer <NOT DISPLAYED FOR THIS POST>" -o /home/admin/result.json "https://www.endpoint.com/manage/query/run?id=55408&cmd=service&output=json"
Related
HTTP GraphQl calls always return 200 - even if the response doesn't suite the request.
I'm trying to do a graphql call and understand if there's an error, like using $? and --fail but it doesn't help because of the always 200 response.
Even if graphql's output isn't according to input and contains error arrays, curl only cares about the http code, which is always 200.
Is there a way for curl to understand a graphql error? Like some kind of built in mechanism in to compare requested input to actual input and understand there's an error?
Perhaps I'm barking on the wrong tree here and should use some command line tool more dedicated to graphql? Thanks.
curl doesn't know anything in particular about GraphQL. You can pipe the output of curl to grep to check for the presence of errors and draw conclusions based on that as necessary.
ex:
curl --request POST \
--header 'content-type: application/json' \
--url http://localhost:4000/ \
--data 'your query data'| grep "errors"
I've been trying to learn how to use the Google Drive API to update a file in the Google Drive by using a resumable session.
I received a 'Forbidden' response to the upload content request.
Could you help me find missing or misused steps?
User is authorized with permissions:
drive.file (https://www.googleapis.com/auth/drive.file)
Execute a request to create resumable session:
PATCH 'https://www.googleapis.com/upload/drive/v3/files/1XIU63B-U8b9Fe1_UFFVvd7OOdS_ANqAj?uploadType=resumable
Retrieve session url:
https://www.googleapis.com/upload/drive/v3/files/1XIU63B-U8b9Fe1_UFFVvd7OOdS_ANqAj?uploadType=resumable&upload_id=AEnB2Uqew...
Send content by using resumable session:
PUT https://www.googleapis.com/upload/drive/v3/files/1XIU63B-U8b9Fe1_UFFVvd7OOdS_ANqAj?uploadType=resumable&upload_id=AEnB2Uqew...
I didn't find anything specific related to this step in the documentation, so I use regular upload documentation https://developers.google.com/drive/api/v3/manage-uploads#upload-resumable to update file in "Multiple chunks"
I get 403 error status code with 'Forbidden' reason and header with upload_id:
X-GUploader-UploadID: AEnB2Uqewr...
You want to update the existing file in Google Drive with the resumable upload method.
Unfortunately, from your question, I couldn't understand about the detail request body of your test. By this, I cannot replicate your situation. So in this answer, I would like to propose a sample flow for updating the existing file with the resumable upload.
Sample situation:
In this answer, as a sample situation, it supposes that a text file in Google Drive is updated by the resumable upload with the multiple chunks. And as the method for requesting, I use the curl command.
I prepared 2 files for 2 chunks. As the test situation, the 2 chuncs of 262,144 bytes and 37,856 bytes are uploaded. So total upload size is 300,000 bytes.
When you use the resumable upload, please be careful the following point.
Add the chunk's data to the request body. Create chunks in multiples of 256 KB (256 x 1024 bytes) in size, except for the final chunk that completes the upload. Keep the chunk size as large as possible so that the upload is efficient. Ref
Flow for updating a file with the resumable upload:
1. Initiate a resumable upload session
Create the session for uploading with the resumable upload. In this case, the existing file is updated, so the endpoint is PUT https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable. But as an important point, please use the method of PATCH instead of PUT. When PUT is used, location is not included in the response header. I thought that the official document might be not correct.
$ curl -X PATCH -i \
-H "Authorization: Bearer ###accessToken###" \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable"
If you want to update the file as the multipart upload, please use the following sample command. In this case, the filename is changed.
$ curl -X PATCH -i \
-H "Authorization: Bearer ###accessToken###" \
-H "Content-Type: application/json; charset=UTF-8" \
-d '{"name":"updatedFilename.txt"}' \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable"
When above sample command is run, 200 OK is returned, and the response header includes location like location: https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable&upload_id=###. For uploading the data, location is used as the endpoint.
2. Upload the 1st chunk
$ curl -X PUT -i \
-H "Content-Length: 262144" \
-H "Content-Range: bytes 0-262143/300000" \
-H "Content-Type: text/plain" \
-F "file=#data1.txt" \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable&upload_id=###"
When this curl command is run, 308 Resume Incomplete is returned. By this, it is found that the chunk could be correctly uploaded.
3. Upload the 2nd chunk (This is the last chunk of this sample flow.)
$ curl -X PUT -i \
-H "Content-Length: 37856" \
-H "Content-Range: bytes 262144-299999/300000" \
-H "Content-Type: text/plain" \
-F "file=#data2.txt" \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable&upload_id=###"
When this curl command is run, 200 OK is returned, and the file metadata is also returned. By this, it is found that the resumable upload could be correctly done.
Note:
In this case, the file is updated as the overwrite. So please be careful this.
In my environment, even when PUT is modified to PATCH for uploading the chunks, I could confirm that the above flow worked.
If in your environment, an error occurs, please try to test this modification.
About above sample situation, if you want to upload one chunk of 300,000 bytes, please use -H "Content-Length: 300000" -H "Content-Range: bytes 0-299999/300000".
References:
Perform a resumable upload
I am doing some basic piping of some simple raw code from github to terminal as shown here i.e.
curl https://raw.github.com/leachim6/hello-world/master/r/ruby.rb | ruby
When I try it, it doesn't produce "Hello World", but instead I just see
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
use
curl -sSL https://raw.github.com/leachim6/hello-world/master/r/ruby.rb | ruby
this should work
Update to explain
this URL is redirecting to
https://raw.githubusercontent.com/leachim6/hello-world/master/r/ruby.rb
so -L option was required to follow the redirection (-L, --location)
this option will make curl redo the request on the new place
sS to hide the progress bar and show errors if happened
to debug curl request you can use -v option which will make you see exactly what is happening
I'm using curl command inside logstash exec plugin to post a message to a stride group. The plugin documentation stated that they were using ruby system method to execute the command, so I'm trying to run it in my ruby IRB.
Escaping the double quotes with backslash character is giving error The request body cannot be parsed as valid JSON. Here is the full error
irb(main):050:0' --inf-ruby-2f3827a9-23243-13517-726000--
curl: (6) Couldn't resolve host 'first'
curl: (3) [globbing] unmatched close brace/bracket in column 9
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 165 0 87 100 78 60 54 0:00:01 0:00:01 --:--:-- 60
{
"statusCode": 400,
"message": "The request body cannot be parsed as valid JSON"
}=> true
I tried swapping double quotes with single quotes and using full double quotes everywhere. Nothing is working.
system("curl -X POST -H 'Content-Type: application/json' -H 'Authorization: Bearer blah-blah' -d '{\"body\":{\"version\":1,\"type\":\"doc\",\"content\":[{\"type\":\"paragraph\",\"content\":[{\"type\":\"text\",\"text\":\"My first message!\"}]}]}}' --url 'https://api.atlassian.com/site/blah-blah/conversation/blah-blah/message'")
Is there any way to make this work?
EDIT: I have tried running the cURL command in the terminal and it is working fine.
Besides the following answers on StackOverflow, NetHTTP has notoriously poor documentation but can be used to post what you're interested in.
I have implemented one shall script which send an https request to proxy with authorization header using GET request.
Here is my command :
curl -s -o /dev/null -w "%{http_code}" -X GET -H "Authorization: 123456:admin05" "https://www.mywebpage/api/request/india/?ID=123456&Number=9456123789&Code=01"
It takes around 12 second to wait and then sending request to proxy and revert back with some code like 200,400,500 etc..
Is it possible to reduce time and make it faster using CURL ?
Please advice me for such a case.
Thanks.
Use option -v or --verbose along with --trace-time
It gives details of actions begin taken along with timings.
Includes DNS resolution, SSL handshake, etc. Line starting with '>' means header/body being sent, '<' means being received.
Based on difference between operation sequence - you can decipher whether server is taking time to respond (time between request and response) or network latency or bandwidth(response taking) time.