So I can successfully generate a temporary signed url on google cloud storage, with an expiry time etc.
However the signed URL still has the clearly visible bucket name and file name.
Now I understand that when the download occurs, the filename will be visible.. as we have downloaded that file. However it would be nice to obscure the bucket and filename in the URL?
Is this possible, the documentation does not give any clues and a google search session has not really given any results that help.
I don't think there's a way. Bucket naming best practices basically state that bucket and object names are "public", that you should avoid using sensitive information as part of those names, and advice you to use random names if you are concerned about name guessing/enumeration.
A possible workaround for this would be to proxy the "get" for the Cloud Storage objects using Cloud Functions or App Engine, so the app retrieves the objects from Cloud Storage and then send it to the client.
This is more costly, and would require to write more code.
I can think on another possible workaround which consists in protect your Signed URL by code (such as PHP), so that users cannot know what the URL is. Nevertheless, taking in account that you want to avoid any displayed-data on the network activity when downloading, you should test this workaround first to see if this works as intended.
Related
I'm writing an application that downloads publicly shared Google Drive files. My application already knows the IDs of the files it needs to download. Here are three possible ways I can do this, and none of them are satisfactory:
1.) Downloading the file using the url https://drive.google.com/uc?id=<fileID>&export=download works, but is subjected to a severe rate limit restriction because the request is sent with no authentication.
2.) Downloading the file with https://www.googleapis.com/auth/drive.file authentication doesn't work. This is because the documentation says it only provides access to "files created or opened by the app".
3.) Downloading the file with https://www.googleapis.com/auth/drive authentication works (without any other code modifications from the "drive.file" case). I can't use this option because of the required security assessment for applications that use this scope.
The only way forward seems to be using "drive.file" authentication, where these files are opened by the app. How should I do this? I think the Picker API is the intended way to do this from inside an application, but I don't see support for filtering by a specific file ID, and I would prefer a solution that doesn't require user input, especially since options (1) and (3) don't require user input.
(as a side note, the logic behind this security policy doesn't make sense to me. If (2) is not possible because of a security/authentication concern, then (1) should not be possible. If (2) is not possible because of a rate limit concern, then (3) should not be possible. I have been told that this is intended behavior.)
I understand that you are developing a web app that downloads a publicly shared Drive file. If my assumption is correct, then you only need to implement an OAuth 2.0 protocol. You can follow the basic steps as a reference. Here is a more in depth guide for web apps. Please, ask me any question if you have doubts about this approach.
UPDATE
Based on this IssueTracker answer from Google, this is expected behaviour. That is, you need to open the public file in your app first and then you would be able to download it.
I have an application that needs to upload file to Google Drive via the ordinary Google Drive API. It needs to upload, but it never needs to download anything, list directories, read metadata or anything like that. It basically uses Google Drive as a drop box to store results of some computations.
Because of the principle of least authority, I would like to give this application the authorization to create new files, but not read or modify anything. Is this possible? I cannot see anything like this in this list, so I suspect the answer is no, but would like more informed comments.
By principle of least authority "user/process should have necessities/privileges". A file created by an app is owned(create/modify/delete) by app and no access to rest of resources.
If this definition fits in your scope then use
https://www.googleapis.com/auth/drive.file
"Per-file access to files created or opened by the app"
As stated in Choose Auth Scopes
Auth scopes express the permissions you request users to authorize for your app. While many Drive apps can function with just the required set of scopes, you may need to consider using other available scopes.
As far as I know, you can choose from the list of scopes available for the Drive API, combine them or mix and match if necessary. And, you can also add other scopes if your app requires access to any other Google APIs as given in Google APIs scopes.
To learn more about scopes, you can watch the video of Google engineers discussing related tips and tricks within the given documentation. I hope that helps.
I am trying to upload files to a bucket on Google Cloud Storage, but I am having trouble figuring out how to set it up so that it is publicly writable and readable. In other words, I don't want to have to require authentication from the user in order to upload to the bucket. Does anybody know the steps to follow in order to set this up?
I would also like to know what I need to append to my request headers in order to upload files to this bucket. I am using the iOS API client, but if anybody knows what the raw headers are, I can probably figure out from there how to do it in iOS. At the moment, I am only including the following additional header: x-goog-project-id
For your first question, you can make your public your new uploaded objects with the following command that uses the gsutil acl syntax:
gsutil defacl ch -u allUsers:R gs://<bucket>
Now you need to give access to write to that bucket to everybody using the command:
gsutil acl ch -u allUsers:O gs://<bucket>
Regarding your other question, I'm not familiar with iOS but you can go to the bottom of this page and upload an object and you'll see the HTTP request that you can use in your code.
Also, there is Google API Client Library for Objetive-C and it seems that with that library you can manage Google Cloud Storage as per these files.
I hope it helps.
Please consider using signed URLs (https://developers.google.com/storage/docs/accesscontrol#Signed-URLs ) instead of making your bucket publicly writable. Having a publicly writable bucket can be an opening to various forms of abuse, and also could result in your getting a surprisingly high bill if your bucket is discovered by someone on the Internet who then uploads large amounts of data to it.
I'm letting people post files to my S3 account and I don't know the filename that they'll be posting.
How do I get a signed URL for them ahead of time so they can download whatever file they have posted? I want to do this ahead of time because I don't want to hit my server again.
I'm using the ruby aws sdk.
You can't.
You can generate a signed URL from a file that already exists on S3. You can't generate a URL before you know what the URL will be.
While you can't distribute pre-signed URLs without knowing the key, you can use the S3 POST technique to generate a signed policy, which can then be used to upload files. This requires building a simple web form, or other tool which can create an HTTP POST request.
Ideally, you would create a separate signed policy document for each user, that way you can revoke access on an individual basis. Also, the policy allows you to constrain the maximum file size, the key prefix, file types and other things.
I'm currently working on a rather interesting... project. I have a client who wants to allow form uploads (from a page presented on their server) specifically to their own google drive account. The platform being used is essentially LAMP.
Single (pre-authenticated) google drive account. Multiple otherwise anonymous upload sources (users).
They do not want users to be required to have their own google accounts (rules out simply using Picker on the user's own drive files).
They want some degree of backwards browser compatibility, such as IE8 (rules out XHR to form the post using HTML5's file API to read the filedata). They don't want to use flash/etc due to potential compatibility issues with certain mobile browsers.
What is working:
Authenticating (getting a refresh token, storing, using it to get access tokens as needed)
Uploading a file to the account without metadata
Result of file upload being sent to hidden iframe
Catching the iframe load event via jquery to at least know something has happened
Problems:
The REST API upload endpoint does not support CORS: there is no way to access the result iframe directly. (see: Authorization of Google Drive using JavaScript)
The return from a successful upload is only raw JSON, not JSONP.
There is seemingly no way to host anything with proper headers to open via browser on the googleapis.com domain, so easyXDM and similar multi-iframe with cross origin workaround communication javascript approaches are ruled out.
There is no way to embed a callback URL in the POST from the submit, the API does not allow for it.
The Picker displays errors on trying to upload if you pass it an Oauth2 token that is not for a user who is also authenticated in their browser (assumedly via cookie). Strangely enough you can show files from the Oauth2 token's matching account, but other than in a browser instance where the target Oauth2 token's account matches the already logged in user any file uploads fail with an ambiguous "Server rejected" message. This happens with all files and file types, including files working in an authenticated browser instance. I assume it's an authentication flow/scope issue of some sort. I haven't tried diving the Picker source.
All of the javascript Google Drive API upload examples seem to rely on using HTML 5 to get the file data, so anything of that nature seems to be ruled out.
While files are uploaded, there's no way other than guesstimating which file came from which user, since we can't get the file object ID from the result in our inaccessible iframe. At best we could make a very rough time based guess, but this is a terrible idea in case of concurrency issues.
We can't set the file name or any other identifier for the file (not even a unique folder) because the REST API relies on that metadata being sent via JSON in the post request body, not via form fields. So we end up with file objects in the drive with no names/etc.
We can't create the file with metadata populated server side (or via jquery/XHR, or the google javascript API client) and then update it with a form based upload because the update API endpoint exclusively works with PUT (tested).
We can't upload the files to our local server and then send them to google (proxy them) as the php ini is locked down to prevent larger file uploads (and back to restrictions imposed on using HTML5 or flash for why we can't chunk files/etc).
All of this has been both researched and to varying degrees tried.
At the moment this is going on hold (at least it was a useful way to learn the API and gain a sense of its limitations) and I'm just going to implement something similar on dropbox, but if anyone has any useful input it would be lovely!
e.g. is there any way to get this working with Drive? Have I overlooked something?
I also realize that this is probably essentially a less than intended use-case, so I'm not expecting miracles. I realize that the ideal flow would be to simply allow users to upload if necessary to their own google drives and then have them grant file access to our web app (via Picker or API+our own UI), but this becomes a problem when not all of our own users are necessarily already google account users. I know that google would OBVIOUSLY prefer we get even more people to sign up with them in order to have this happen, but making people sign up for a google account to use our app was ruled out (not out of any prejudice on our part, it was just too many added steps and potential user hurdles). Even simply having them sign in to google if they did have accounts was deemed unwanted for the basic LCD feature functionality, although it's likely to be added as an additional option on top of whatever becomes the base solution.
The biggest problem with the approach you described is you're introducing a big security issue. Allowing an anonymous user to directly upload to Drive from the client requires leaking a shared access token to anyone who comes by. Even with the limited drive.file scope, a malicious or even slightly curious user would be able to list, access (read/update/delete!) any file that was uploaded by that app.
Of course a public drop box feature is still useful, but you really need to proxy those requests to avoid revealing the access token. If your PHP environment is too restrictive, why not run the proxy elsewhere? You can host a simple proxy to handle the uploading just about anywhere -- app engine, heroku, etc. and support whatever features you need to ensure the metadata is set correctly for your app.