I am trying to do it like this:
AWS.config(
:access_key_id => '...',
:secret_access_key => '...'
)
s3 = AWS::S3.new
bucket_name = 'bucket_name'
key = "#{File.basename(avatar_big)}"
s3.buckets[bucket_name].objects[key].write(:file => avatar_big_path)
This working well for a file, the file is uploaded to the root of the set up bucket.
However, how to upload it into the foloder photos that is located in root?
I've tried
key = "photos/#{File.basename(avatar_big)}"
but this doesn't work.
EDIT: error message
Thank you
I had the same issue as the OP. This is what worked for me:
key = "photos/example.jpg"
bucket = s3.buckets[bucket_name]
filepath = Pathname.new("path/to/example.jpg")
o = bucket.objects[key]
o.write(filepath)
Something I would check out would be the object key you are trying to use. There's not much documentation on what are the restrictions are (see this and this) but the one shown in error message looks suspicious to me.
Try including the the path in the file key:
s3.buckets[bucket_name].objects[key].write(:file => "photos/#{avatar_big_path}")
Related
I want to create a ruby script which will take barcodes from a text file, search a webservice for that barcode and download the result.
First I tried to test the webservice download. In a file when I hardcode the query things work fine:
result_download = open('http://webservice.org/api/?query=barcode:78686112327', 'User-Agent' => 'UserAgent email#gmail.com').read
It all works fine.
When I try to take the barcode from a textfile and run the query I run into problems.
IO.foreach(filename) {|barcode| barcode
website = "'http://webservice.org/api/?query=barcode:"+barcode.to_str.chomp + "', 'User-Agent' => 'UserAgent email#gmail.com'"
website = website.to_s
mb_metadata = open(website).read
}
The result of this is:
/home/user/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:37:in `initialize': No such file or directory # rb_sysopen - http://webservice.org/api/?query=barcode:78686112327', 'User-Agent' => 'UserAgent email#gmail.com' (Errno::ENOENT)
I can't figure out if this problem occurs because the string I generate somehow isn't a valid url and ruby is trying to open a non-existent file, or is the issue that I am doing all this in a for loop and the file/url doesn't exist there. I have tried using open(website).write instead of open(website).read but that produces the same error.
Any help would be much appreciated.
The error message you get explicitly states, that there is no such file:
http://webservice.org/api/?query=barcode:78686112327', 'User-Agent' => 'UserAgent email#gmail.com'.
You try to pass all the parameters to open method using 1 big string (website), which is wrong. You should do it like that.
IO.foreach(filename) do |barcode|
website = "http://webservice.org/api/?query=barcode:#{barcode.to_str.chomp}"
mb_metadata = open(website, 'User-Agent' => 'UserAgent email#gmail.com').read
end
Combining both Intervention Image and Amazon S3, I'd like to be able to pull a file from S3 and then use Image to do some cropping. This is what I have so far, why does Storage::get() return false?
$path = 'uploads/pics/123.jpeg';
$exists = Storage::disk('s3')->exists($path); // returns true
$image = Storage::disk('s3')->get($path); // returns false
From the S3 side of things, the bucket permissions are set to 'Everyone', the Storage::getVisibility() returns public... I'm not sure why I can't load the image as if it were a local image.
After digger deeper into the code I found this message
"Error executing "GetObject" on "file"; AWS HTTP error: file_exists(): open_basedir restriction in effect. File(/etc/pki/tls/certs/ca-bundle.crt) is not within the allowed path(s): (paths)"
First it seems that my server don't have this file, but it have! The file is located in another folder.
/etc/ssl/certs/ca-certificates.crt
So, to solve my problem in Ubuntu I have to create this folder /etc/pki/tls/certs and after that, symlink to the correct file:
cd /etc/pki/tls/certs;
sudo ln -s /etc/ssl/certs/ca-certificates.crt ca-bundle.crt;
Edit your php.ini and add /etc/pki/tls/certs/ca-bundle.crt to the open_basedir configuration.
Restart your php server!
For me it solves the problem, hope it helps!
Since Dec 2020, Amazon S3 now provides strong read-after-write consistency in all regions, rendering this answer obsolete. For more details, refer to the Amazon S3 Strong Consistency page.
This shouldn't be an issue anymore.
The old answer below has been kept for reference purposes & for providing a reason for the bounty previously awarded.
From the Amazon S3 documentation:
Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3 bucket in all regions with one caveat. The caveat is that if you make a HEAD or GET request to the key name (to find if the object exists) before creating the object, Amazon S3 provides eventual consistency for read-after-write.
Given the example code where the path is static and the exists call is made prior to the get, I'm conjecturing that you're being hit with eventual consistency. Your get should eventually return. Try:
$backoff = 0;
while (false === ($image = Storage::disk('s3')->get($path))) {
if (5 < $backoff) {
throw new \RuntimeException;
}
sleep(pow(2, $backoff++));
}
If you are using laravel 5 , than apply for this method.
$photo = $attributes['banner_image'];
$s3 = AWS::createClient('s3');
try {
$response = $s3->putObject([
'Bucket' => 'gfpressreleasephotos',
'Key' => str_random(8) . '.' . str_replace(' ', '-', strtolower($photo->getClientOriginalName())),
'Body' => fopen($photo->getRealPath(), 'r'),
'ACL' => 'public-read',
]);
if ($response->get('ObjectURL') != null) {
$photourl = $response->get('ObjectURL');
} else {
$photourl = $response->get('Location');
}
$attributes['banner_image'] = $photourl;
} catch (S3Exception $e) {
return "There was an error uploading the file.\n";
}
I am using aws-sdk-ruby for deleting a file saved in a bucket in my amazon s3 account, but i can't figure out why i am able to delete the desired file from S3 bucket using the following code.
this is my code
require 'aws-sdk-v1'
require 'aws-sdk'
ENV['AWS_ACCESS_KEY_ID'] = "XXXXXXX"
ENV["AWS_SECRET_ACCESS_KEY"] = '/ZZZZZZZZ'
ENV['AWS_REGION'] = 'us-east-1'
s3 = Aws::S3::Resource.new
bucket = s3.bucket('some-bucket')
obj = bucket.object('https://s3.amazonaws.com/some-bucket/38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg')
obj.delete
The documentation tells that it should look like this:
s3 = Aws::S3.new
bucket = s3.buckets['some-bucket']
object = bucket.objects['38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg']
object.delete
Please note:
the square brackets,
that the object's key doesn't include the domain and
instead of creating an instance of Aws::S3::Resource create an instance of AWS::S3
If you use API version 3 (aws-sdk-s3 (1.81.1)) you should do something like below:
s3 = Aws::S3::Client.new
s3.delete_object(bucket: 'bucket_name', key: 'bucket_folder/file.txt')
it should be:
objs = bucket.objects('https://s3.amazonaws.com/some-bucket/38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg')
objs.each {|obj| obj.delete}
With the aws-sdk v2: I had to do this: (see Doc)
$s3.bucket("my-bucket").objects(prefix: 'my_folder/').batch_delete!
(delete is deprecated in favor of batch_delete)
Useful post: https://ruby.awsblog.com/post/Tx1H87IVGVUMIB5/Using-Resources
I am trying to create a form that will upload files to AWS S3. I have searched all around for an answer but I am getting the error "TypeError at /upload can't convert Symbol into Integer"
Here is the block of code
post '/upload' do
s3 = AWS::S3.new(
:access_key_id => 'X',
:secret_access_key => 'X')
bucket = s3.buckets['X']
title = params['title']
desc = params['desc']
file = params['file'][:tempfile]
s3.buckets['indio'].objects[title].write(:file => file)
end
I get the error on the line
file = params['file'][:tempfile]
Can someone point out what I am doing wrong?
Typically the error can't convert Symbol into Integer hints to the fact that you are trying to access an array with a non-integer.
From this I suspect the params['file'] is an array or a string, and not whatever you think it is.
Find out exactly what you got in params['file'] and continue from there.
Am trying to copy an image from bucket to another bucket in s3.
AWS.config(
:access_key_id => 'Bucket one key',
:secret_access_key => 'bucket one secret key'
)
s3 = AWS::S3.new
bucket1 = s3.buckets["Bucket_one"]
bucket2 = s3.buckets["Bucket_two"]
obj1 = bucket1.objects["source_key"]
obj2 = bucket2.objects["destination_key"]
obj1.copy_to(obj2)
Can you guide me as to how to retrieve the source key of a file already uploaded in S3? I have the destination_key,bucket_one and bucket_two.
Image on S3 is not a single file but a number of files names of which starts from the name of your image.
Thus you need to know the name of your image.
Once you get it, the could should look like:
bucket1 = s3.buckets["Bucket_one"]
bucket2 = s3.buckets["Bucket_two"]
bucket1.objects.with_prefix(image_name).each do |source_object|
source_object.copy_to(bucket2.objects[source_object.key])
end