I try to migrate my project to aws-sdk 2. Need to use AWS SDK for Ruby - Version 2 for this.
I found all methods, but i cant change access to file (make public).
In later version i use this:
bucket.objects[file_path].acl = :public_read
But i cant find method for changing with new api version.
This is link to old api documentation
This is link to new api documentations
I presume here that you want to change the object ACL after it's been uploaded to S3. If you can, consider setting the ACL when the object is sent to S3 rather than after.
There's two ways to do it. They are both similar and perform the same action. Pick the one you like best or the one you are more comfortable with.
Using the Client API
client = Aws::S3::Client.new(region: myregion)
resp = client.put_object_acl({ acl: "public-read", bucket: mybucket, key: mykey })
Documentation:
http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Client.html#put_object_acl-instance_method
The Resource API
s3 = Aws::S3::Resource.new(region: myregion)
bucket = s3.bucket(mybucket)
object = bucket.object(mykey)
resp = object.acl.put({ acl: "public-read" })
Documentation:
http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/ObjectAcl.html#put-instance_method
Bonus
If absolutely all your objects inside your bucket needs to be public, you can set the default ACL on your whole bucket so that any object uploaded will be automatically public without having you to specify it. You do that by setting a bucket policy to your bucket.
Make a bucket public in Amazon S3
Related
I am developing a Flask app running on Heroku that allows users to upload images. The app has a page displaying the user's images in a table.
For developing purposes, I am saving the uploaded files to Heroku's ephemeral file system, and everything works fine: the images are correctly loaded and displayed (I am using the last method shown here implying the use of send_from_directory()). Now I have moved the storage to S3 and I am trying to adapt the code. I use boto3 to upload the files to the bucket: it works fine. My doubts are related to the download to populate the users' pages with their images.
As explained here, I could set the file as "public-read" and use the URL (I think this is what Flask-S3 does), but I'd rather prefer not to leave free access to the files. So, my solution attempt is to download the file to Heroku's filesystem and serve the image using again the send_from_directory() as follows:
app.py
#app.route('/download/<resource>')
def download_image(resource):
""" resource: name of the file to download"""
s3 = boto3.client('s3',
aws_access_key_id=current_app.config['S3_ACCESS_KEY'],
aws_secret_access_key=current_app.config['S3_SECRET_KEY'])
s3.download_file(current_app.config['S3_BUCKET_NAME'],
resource,
os.path.join('tmp',
resource))
return send_from_directory('tmp', # Heroku's filesystem
resource,
as_attachment=False)
Then, in the template I generate the URL for the image as follows:
...
<img src="{{ url_for('app.download_image',
resource=resource) }}" height="120" width="120">
...
It works, but I don't think this is the proper way for some reasons: among them, I should manage the Heroku's filesystem to avoid using up all the space between dynos restart (I should delete the images from the filesystem).
Which is the best/preferred way, also considering the performance?
Thanks a lot
The preferred way is to simply create a pre-signed URL for the image, and return a redirect to that URL. This keeps the files private in S3, but generates a temporary, time limited, URL that can be used to download the file directly from S3. That will greatly reduce the amount of work happening on your server, as well as the amount of data transfer being consumed by your server. Something like this:
#app.route('/download/<resource>')
def download_image(resource):
""" resource: name of the file to download"""
s3 = boto3.client('s3',
aws_access_key_id=current_app.config['S3_ACCESS_KEY'],
aws_secret_access_key=current_app.config['S3_SECRET_KEY'])
url = s3.generate_presigned_url('get_object', Params = {'Bucket': 'S3_BUCKET_NAME', 'Key': resource}, ExpiresIn = 100)
return redirect(url, code=302)
If you don't like that solution, you should at least look into streaming the file contents from S3 instead of writing it to the file system.
I have a host for my Laravel website and another (non-laravel) for stored files. Direct access to my files are blocked completely by default and I want to control access to them by creating temporary links in my Laravel site. I know how to code, just want to know the idea of how to do it (not details).
From the Laravel docs
Temporary URLs For files stored using the s3 or rackspace driver, you
may create a temporary URL to a given file using the temporaryUrl
method. This methods accepts a path and a DateTime instance specifying
when the URL should expire:
$url = Storage::temporaryUrl(
'file.jpg', now()->addMinutes(5)
);
You could also make your own solution by directing all image request through your own server and making sure the file visibility is set to private.
Here is an example of how a controller could return image from your storage
public function get($path)
{
$file = Storage::disk('s3')->get($path);
// Do your temp link solution here
return response($file, 200)->header('Content-Type', 'image/png');
}
What i am using right now is Flysystem provided in laravel.Laravel Flysystem integration use simple drivers for working with local filesystems, Amazon S3 and other some space provide also. So for this doesn't matter whether is a server is laravel server or not.
Even better, it's very simple in this to switch between server by just changing server configuration in API.
As far as I know we can create temporary Url for s3 and rackspace in this also by calling temporaryUrl method. Caching is already in this.
That's the thing.
If your files are uploaded on an AWS S3 server
then,
use Storage;
$file_path = "4/1563454594.mp4";
if( Storage::disk('s3')->exists($file_path) ) {
// link expiration time
$urlExpires = Carbon::now()->addMinutes(1);
try {
$tempUrl = Storage::disk('s3')->temporaryUrl($file_path, $urlExpires);
} catch ( \Exception $e ) {
// Unable to test temporaryUrl, its giving driver dont support it issue.
return response($e->getMessage());
}
}
Your temporary URL will be generated, After given expiration time (1 minute). It will expire.
I'm loosing my mind.
I'm using Shrine (https://github.com/janko-m/shrine) with Google Cloud Storage (https://github.com/renchap/shrine-google_cloud_storage), but when I start the PUT call I get this:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>
<StringToSign>
PUT
image/jpeg
1518399402
/mybucket.appspot.com/7d5e4aad1e3a737fb8d2c59571fdb980.jpg
</StringToSign>
</Error>
I followed this info (http://shrinerb.com/rdoc/classes/Shrine/Plugins/PresignEndpoint.html) for presign_endpoint, but still nothing:
class FileUploader < Shrine
plugin :presign_endpoint, presign_options: -> (request) do
filename = request.params["filename"]
extension = File.extname(filename)
content_type = Rack::Mime.mime_type(extension)
{
content_type: content_type
}
end
end
I tried with and without this (restarting the Rails server everytime).
Where am I wrong?
I also tried with Postman with a PUT to that URL and withtout any content-type. But still nothing.
I read here: https://github.com/GoogleCloudPlatform/google-cloud-node/issues/1976 and here: https://github.com/GoogleCloudPlatform/google-cloud-node/issues/1695
How can I try without Rails?
Is there a REPL (or similar) to try with my credentials and with a file?
You have several options for just uploading a file to Google Cloud Storage as you can see in the official docs here.
For example if you want to use Ruby Client Library, you can use this code:
# project_id = "Your Google Cloud project ID"
# your-bucket-name = "Your Google Cloud Storage bucket name"
# local_file_path = "Path to local file to upload"
# storage_file_path = "Path to store the file in Google Cloud Storage"
require "google/cloud/storage"
storage = Google::Cloud::Storage.new(project: "your-project_id")
bucket = storage.bucket "your-bucket-name"
file = bucket.create_file "local_file_path", "storage_file_path"
puts "Uploaded #{file.name}"
You will need to:
Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key.
as you can see in the Cloud Storage Client Libraries docs here.
However, it looks like the github repo you are using makes use of signed URLs. If you need to create a signed URL you have the instructions here. It exists already the signed_urls method for Ruby in the official repo.
Anyway, the error you were getting is because there was something wrong being made with the signed urls. And precisely there was a commit to the repo you were referencing (https://github.com/renchap/shrine-google_cloud_storage) 7 days ago with the name Fix presigned uploads. It looks like this commit should fix the "PUT" upload (before a "GET" was being signed, therefore it wouldn't work). Still I didn't try it so I don't know if it is actually working.
I am trying to use the amazon S3 versioning.
So I manage to create a object and display versions and select the version of a well Preci.
But I block or it is for public display url of a document versioning
s3 = Aws::S3::Client.new
prefix = "path/file"
obj = s3.list_object_versions(bucket: 'bucket', prefix: prefix)
I get the version and I want are public url
obj.versions.first
I can not find the method that allows me to have the public URL
Thank you
Per the docs, the version object (ie, what you get upon obj.versions.first) has a version_id key.
You can use that ID to get that version using REST - you just add a versionId=<THE_ID> to your query string.
But I'm not really sure that counts as a public URL. It probably needs you to be authenticated - I don't think it's a good idea to have every version of an object publicly accessible, so probably you just can't.
Thank you for your answer,
Actually my url is not secure, so I used a test url public but that does not work because it says access denied, and yet I added him:
#bucket = Aws::S3::Bucket.new(BUCKET)
url = #bucket.object(name)
url.public_url(acl: 'public-read')
So adding a new permission I manage to not have the access denied but now he told me that the file is not yet well I use the public_url,
I have forgotten something?
#bucket = Aws::S3::Bucket.new(BUCKET)
url = #bucket.object(#name)
url.public_url
Say I have a domain, media.coolsite.com. Also say I have a bucket in Google Cloud Storage with the same name. I also have the cname setup properly, so that when I visit media.coolsite.com in the browser, it renders index.html in the bucket.
The bucket also contains a file 01.mp3 that I am trying to get a signed URL for. I can do this with the following code:
gcloud = Gcloud.new
storage = gcloud.storage
bucket = storage.bucket 'media.coolsite.com'
file = bucket.file '01.mp3'
signed_url = file.signed_url
This returns a URL like so:
https://storage.googleapis.com/media.coolstuff.com/01.mp3?GoogleAccessId=...&Expires=...etc...
But what I would like is a URL like this:
http://media.coolstuff.com/01.mp3?GoogleAccessId=...&Expires=...etc...
Note the different host: media.coolstuff.com instead of storage.googleapis.com.
How can I do this using the gcloud gem?