upload zip file to google drive using curl - shell

I am trying to upload a zip file to Google drive account using curl.
The file is uploaded successfully but the filename is not getting updated. It gets uploaded with default filename i.e. "Untitled".
I am using below command.
curl -k -H "Authorization: Bearer cat /tmp/token.txt" -F "metadata={name : 'backup.zip'} --data-binary "#backup.zip" https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart

You can use Drive API v3 to upload the zip file. The modified curl code is as follows.
curl -X POST -L \
-H "Authorization: Bearer `cat /tmp/token.txt`" \
-F "metadata={name : 'backup.zip'};type=application/json;charset=UTF-8" \
-F "file=#backup.zip;type=application/zip" \
"https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart"
In order to use this, please include https://www.googleapis.com/auth/drive in the scope.

The answer above works fine and was the command I used in uploading my file to Google Drive using Curl. However, I didn't understand what scope was and all of the initial setup required to make this command work. Hence, for documentation purposes. I'll give a second answer.
Valid as at the time of writing...
Visit the Credentials page and create a new credential (this is assuming you have created a project). I created credentials for TVs and Limited devices, so the work flow was similar to:
Create credentials > OAuth client ID > Application Type > TVs and Limited Input devices > Named the client > Clicked Create.
After doing this, I was able to copy the Client ID and Client Secret when viewing the newly created credential.
NB: Only the variables with double asterisk from the Curl commands should be replaced.
Next step was to run the Curl command:
curl -d "client_id=**client_id**&scope=**scope**" https://oauth2.googleapis.com/device/code
Scope in this situation can be considered to be the kind of access you intend to have with the credential having the inputted client_id. More about scope from the docs For the use case in focus, which is to upload files, the scope chosen was https://www.googleapis.com/auth/drive.file.
On running the curl command above, you'll get a response similar to:
{ "device_code": "XXXXXXXXXXXXX", "user_code": "ABCD-EFGH",
"expires_in": 1800, "interval": 5, "verification_url":
"https://www.google.com/device" }
Next step is to visit the verification_url in the response in your browser, provide the user_code and accept requests for permissions. You will be presented with a code when all prompts have been followed, this code wasn't required for the remaining steps (but there may be some reasons to use it for other use cases).
Next step is to use the Curl command:
curl -d client_id=**client_id** -d client_secret=**client_secret** -d device_code=**device_code** -d grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Adevice_code https://accounts.google.com/o/oauth2/token
You will get a response similar to:
{ "access_token": "XXXXXXXXX", "expires_in": 3599,
"refresh_token": "XXXXXXXXX", "scope":
"https://www.googleapis.com/auth/drive.file", "token_type": "Bearer"
}
Now you can use the access token and follow the accepted answer with a Curl command similar to:
curl -X POST -L \
-H "Authorization: Bearer **access_token**" \
-F "metadata={name : 'backup.zip'};type=application/json;charset=UTF-8" \
-F "file=#backup.zip;type=application/zip" \
"https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart"

Related

how to download google docs document exported as pdf from terminal using wget or curl

I'm trying to download my master thesis from google docs as a pdf file..
The ui method (pdf > download > PDF document) fails because of internet connection and the size of the document which is quite big.
I want to download it using wget command from a Linux server which had a good internet connection, I tried a lot of solutions such this one, but they didn't work..
any help please!
If your server can access the user interface of Docs I strongly recommend you to download the PDF with it to save complexity. If the server doesn't offer that option you could use this curl command below to capture Files.export() from Drive API.
curl \
'https://www.googleapis.com/drive/v3/files/{DOC ID}/export?mimeType=application%2Fpdf' \
--header 'Authorization: Bearer {ACCESS TOKEN}' \
--header 'Accept: application/json' \
--compressed >> Doc.pdf
You'll need an access token for a single use, so could use OAuth 2.0 Playground to generate one by selecting the https://www.googleapis.com/auth/drive scope and clicking on Exchange authorization code for tokens.

Azure Office 365 Management APIs

I am trying to get a response from Microsofts Office 365 Management API using bash curl commands-
I get the token like this-
TOKEN=$(curl -X POST "https://login.microsoftonline.com/$TENANTID/oauth2/token" -d "grant_type=client_credentials&client_id=$CLIENTID&client_secret=$ACCESSCODE&resource=$RESOURCEURL" | jq -r '.access_token')
And then use the token to fetch the data like this-
RESULT=`curl -X GET -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" https://manage.office.com/api/v1.0/$TENANTID/ServiceComms/Services`
I do get a token back with the first command, so that works fine and my tenant/client/resource strings are correct
But the second one always gives
{
"error":{
"code":"","message":"Authorization has been denied for this request."
}
}
I pretty sure I have access-
What am I missing?
Turns out the resource URL i was using did not match the URL i was requesting data from

Authentication with search-tweets-ruby (premium API)

This question is about the search-tweets-ruby client provided by twitter for use with their premium and enterprise API's.
I am following the instructions and run into a 'Bad Authentication data' error when running the app (via terminal - Mac OS X) to retrieve tweets with a single rule.
The 'bearer token' and 'dev environment' are correct because a Curl request works.
The following are the contents of my ./config/config.yaml file
auth:
app_token: my_generated_bearer_token
labels:
environment: my_dev_environment_name
options:
search_type: premium
archive: fullarchive
max_results: 500
write_mode: standard-out
out_box: ./output
I'm not sure what I'm missing here, but would appreciate an assist. I haven't worked with the Twitter API before, although I've reviewed the documentation before asking on SO.
Thanks, everyone.
Update:
The first sample call, from the provided link is:
$ruby ./search-app.rb -r "snow profile_region:colorado has:media".
This yields a 'bad authentication error'.
I provided the contents of my yaml file, because presumably that is the only difference between the Curl request and the client app, if the 'bearer token' and 'environment name' work with Curl.
curl --request POST \
--url https://api.twitter.com/1.1/tweets/search/30day/prod.json \
--header 'authorization: Bearer AAAAAAAAAAAAAAAAAAAAAMLheAAAAAAA0%2BuSeid%2BULvsea4JtiGRiSDSJSI%3DEUifiRBkKG5E2XzMDjRfl76ZC9Ub0wnz4XsNiRVBChTYbJcE3F' \
--header 'content-type: application/json' \
--data '{
"query":"from:TwitterDev lang:en",
"maxResults": "100",
"fromDate":"201811010000",
"toDate":"201811062359"
}'
There is no code.
I think the issue is that your YAML file is configured for the full-archive search endpoint, yet your CURL-based call is going to the 30-day search endpoint. If you update the YAML file to point to the '30day' endpoint, I suspect it will succeed.

Unable to update file in the Google Drive by using resumable approach

I've been trying to learn how to use the Google Drive API to update a file in the Google Drive by using a resumable session.
I received a 'Forbidden' response to the upload content request.
Could you help me find missing or misused steps?
User is authorized with permissions:
drive.file (https://www.googleapis.com/auth/drive.file)
Execute a request to create resumable session:
PATCH 'https://www.googleapis.com/upload/drive/v3/files/1XIU63B-U8b9Fe1_UFFVvd7OOdS_ANqAj?uploadType=resumable
Retrieve session url:
https://www.googleapis.com/upload/drive/v3/files/1XIU63B-U8b9Fe1_UFFVvd7OOdS_ANqAj?uploadType=resumable&upload_id=AEnB2Uqew...
Send content by using resumable session:
PUT https://www.googleapis.com/upload/drive/v3/files/1XIU63B-U8b9Fe1_UFFVvd7OOdS_ANqAj?uploadType=resumable&upload_id=AEnB2Uqew...
I didn't find anything specific related to this step in the documentation, so I use regular upload documentation https://developers.google.com/drive/api/v3/manage-uploads#upload-resumable to update file in "Multiple chunks"
I get 403 error status code with 'Forbidden' reason and header with upload_id:
X-GUploader-UploadID: AEnB2Uqewr...
You want to update the existing file in Google Drive with the resumable upload method.
Unfortunately, from your question, I couldn't understand about the detail request body of your test. By this, I cannot replicate your situation. So in this answer, I would like to propose a sample flow for updating the existing file with the resumable upload.
Sample situation:
In this answer, as a sample situation, it supposes that a text file in Google Drive is updated by the resumable upload with the multiple chunks. And as the method for requesting, I use the curl command.
I prepared 2 files for 2 chunks. As the test situation, the 2 chuncs of 262,144 bytes and 37,856 bytes are uploaded. So total upload size is 300,000 bytes.
When you use the resumable upload, please be careful the following point.
Add the chunk's data to the request body. Create chunks in multiples of 256 KB (256 x 1024 bytes) in size, except for the final chunk that completes the upload. Keep the chunk size as large as possible so that the upload is efficient. Ref
Flow for updating a file with the resumable upload:
1. Initiate a resumable upload session
Create the session for uploading with the resumable upload. In this case, the existing file is updated, so the endpoint is PUT https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable. But as an important point, please use the method of PATCH instead of PUT. When PUT is used, location is not included in the response header. I thought that the official document might be not correct.
$ curl -X PATCH -i \
-H "Authorization: Bearer ###accessToken###" \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable"
If you want to update the file as the multipart upload, please use the following sample command. In this case, the filename is changed.
$ curl -X PATCH -i \
-H "Authorization: Bearer ###accessToken###" \
-H "Content-Type: application/json; charset=UTF-8" \
-d '{"name":"updatedFilename.txt"}' \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable"
When above sample command is run, 200 OK is returned, and the response header includes location like location: https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable&upload_id=###. For uploading the data, location is used as the endpoint.
2. Upload the 1st chunk
$ curl -X PUT -i \
-H "Content-Length: 262144" \
-H "Content-Range: bytes 0-262143/300000" \
-H "Content-Type: text/plain" \
-F "file=#data1.txt" \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable&upload_id=###"
When this curl command is run, 308 Resume Incomplete is returned. By this, it is found that the chunk could be correctly uploaded.
3. Upload the 2nd chunk (This is the last chunk of this sample flow.)
$ curl -X PUT -i \
-H "Content-Length: 37856" \
-H "Content-Range: bytes 262144-299999/300000" \
-H "Content-Type: text/plain" \
-F "file=#data2.txt" \
"https://www.googleapis.com/upload/drive/v3/files/[FILE_ID]?uploadType=resumable&upload_id=###"
When this curl command is run, 200 OK is returned, and the file metadata is also returned. By this, it is found that the resumable upload could be correctly done.
Note:
In this case, the file is updated as the overwrite. So please be careful this.
In my environment, even when PUT is modified to PATCH for uploading the chunks, I could confirm that the above flow worked.
If in your environment, an error occurs, please try to test this modification.
About above sample situation, if you want to upload one chunk of 300,000 bytes, please use -H "Content-Length: 300000" -H "Content-Range: bytes 0-299999/300000".
References:
Perform a resumable upload

How to download a big file from google drive via curl in Bash?

I wanna make a very simple bash script for downloading files from google drive via Drive API, so in this case there is a big file on google drive and I installed OAuth 2.0 Playground on my google drive account, then in the Select the Scope box, I choose Drive API v3, and https://www.googleapis.com/auth/drive.readonly to make a token and link.
After clicking Authorize APIs and then Exchange authorization code for tokens. I copied the Access tokenlike below.
#! /bin/bash
read -p 'Enter your id : ' id
read -p 'Enter your new token : ' token
read -p 'Enter your file name : ' file
curl -H "Authorization: Bearer $token" "https://www.googleapis.com/drive/v3/files/$id?alt=media" -o "$file"
but it won't work, any idea ?
for example the size of my file is 12G, when I run the code I will get this as output and after a second it back to prompt again ! I checked it in two computers with two different ip addresses.(I also add alt=media to URL)
-bash-3.2# bash mycode.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 166 100 166 0 0 80 0 0:00:02 0:00:02 --:--:-- 80
-bash-3.2#
the content of file that it created is like this
{
"error": {
"errors": [
{
"domain": "global",
"reason": "downloadQuotaExceeded",
"message": "The download quota for this file has been exceeded."
}
],
"code": 403,
"message": "The download quota for this file has been exceeded."
}
}
You want to download a file from Google Drive using the curl command with the access token.
If my understanding is correct, how about this modification?
Modified curl command:
Please add the query parameter of alt=media.
curl -H "Authorization: Bearer $token" "https://www.googleapis.com/drive/v3/files/$id?alt=media" -o "$file"
Note:
This modified curl command supposes that your access token can be used for downloading the file.
In this modification, the files except for Google Docs can be downloaded. If you want to download the Google Docs, please use the Files: export method of Drive API. Ref
Reference:
Download files
If I misunderstood your question and this was not the direction you want, I apologize.
UPDATE AS FOR MARCH 2021
Simply follow this guide here. It worked for me.
In summary:
For small files to download run
wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=FILEID' -O FILENAME
While if you are trying to download a quite large file you should try to run
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=FILEID' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=FILEID" -O FILENAME && rm -rf /tmp/cookies.txt
Simply substitute FILEID and FILENAME with your custom values.
FILEID can be found in your file share link (after the /d/ as illustrated in the article mantioned above).
FILENAME is simply the name you want to save the download as. Remember to include the right extension. For Example FILENAME = my_file.pdf if the file is a pdf.
This is a known bug
It has been reported in this Issue Tracker post. This is caused because as you can read in the documentation:
(about download url)
Short lived download URL for the file. This field is only populated
for files with content stored in Google Drive; it is not populated for
Google Docs or shortcut files.
So you should use another field.
You can follow the report by clicking on the star next to the issue
number to give more priority to the bug and to receive updates.
As you can read in the comments of the report, the current workaround is:
Use webContentlink instead
or
Change www.googleapis.com to content.googleapis.com

Resources