I am using aws-sdk-ruby for deleting a file saved in a bucket in my amazon s3 account, but i can't figure out why i am able to delete the desired file from S3 bucket using the following code.
this is my code
require 'aws-sdk-v1'
require 'aws-sdk'
ENV['AWS_ACCESS_KEY_ID'] = "XXXXXXX"
ENV["AWS_SECRET_ACCESS_KEY"] = '/ZZZZZZZZ'
ENV['AWS_REGION'] = 'us-east-1'
s3 = Aws::S3::Resource.new
bucket = s3.bucket('some-bucket')
obj = bucket.object('https://s3.amazonaws.com/some-bucket/38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg')
obj.delete
The documentation tells that it should look like this:
s3 = Aws::S3.new
bucket = s3.buckets['some-bucket']
object = bucket.objects['38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg']
object.delete
Please note:
the square brackets,
that the object's key doesn't include the domain and
instead of creating an instance of Aws::S3::Resource create an instance of AWS::S3
If you use API version 3 (aws-sdk-s3 (1.81.1)) you should do something like below:
s3 = Aws::S3::Client.new
s3.delete_object(bucket: 'bucket_name', key: 'bucket_folder/file.txt')
it should be:
objs = bucket.objects('https://s3.amazonaws.com/some-bucket/38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg')
objs.each {|obj| obj.delete}
With the aws-sdk v2: I had to do this: (see Doc)
$s3.bucket("my-bucket").objects(prefix: 'my_folder/').batch_delete!
(delete is deprecated in favor of batch_delete)
Useful post: https://ruby.awsblog.com/post/Tx1H87IVGVUMIB5/Using-Resources
Related
I use this code to upload files to S3:
s3 = Aws::S3::Resource.new(...)
obj = s3.bucket(s3_bucket).object("final/path")
success = obj.upload_file('my/local/file')
How can I change the storage class to Standard-Infrequent Access?
Try to pass storage_class option to #upload_file:
success = obj.upload_file('my/local/file', storage_class: "STANDARD_IA")
I'm running into an issue uploading to S3 with version 2 of the sdk.
When running:
Aws.config.update({
region: 'us-east-1',
credentials: Aws::Credentials.new(credentials['key'],credentials['secret'],
s3_server_side_encryption: :aes256)
})
s3 = Aws::S3::Resource.new
bucket = 'VandalayIndustriesAccountingData'
s3_file_path = "folder/filename.tar.gz"
s3_object = s3.bucket(bucket).object(s3_file_path)
s3_object.upload_file(artifact_location)
I get the following error:
Aws::S3::Errors::InvalidToken
-----------------------------
The provided token is malformed or otherwise invalid.
When I remove the s3_server_side_encryption setting it changes to an access denied error.
I've been trying to find documentation around doing this with v2 of the API, but everything online seems to rely on the bucket object having a write method which doesn't seem to exist in v2 of the API.
http://docs.aws.amazon.com/AmazonS3/latest/dev/SSEUsingRubySDK.html
I'm likely just not finding the correct document in the v2 api. I'd like to avoid using v1 and v2 of the api but may fall back to that.
upload_file takes arguments similar to write
Aws.config.update({
region: 'us-east-1',
credentials: Aws::Credentials.new(credentials['key'],credentials['secret'],
)
})
s3 = Aws::S3::Resource.new
bucket = 'VandalayIndustriesAccountingData'
s3_file_path = "folder/filename.tar.gz"
s3_object = s3.bucket(bucket).object(s3_file_path)
s3_object.upload_file(artifact_location, server_side_encryption: :AES256)
I'm trying to upload in image to s3 using the aws-sdk. I'm able to retrieve my bucket
s3 = Aws::S3::Client.new
resp = s3.list_buckets
bucket = resp.buckets.select {|x| x.name == "mybucket"}[0]
>> bucket
>> #<struct Aws::S3::Types::Bucket name="mybucket", creation_date=2015-09-05 19:23:49 UTC>
I now have my bucket. Looking at the aws documentation and heroku's documentation I should be able to call bucket.presigned_post, however I get NoMethodError: undefined method 'presigned_post' for #<Aws::S3::Types::Bucket:0x007ff583bece10>
What am I missing here? Do I not have the correct s3 bucket object?
Aws::S3::Types::Bucket is not the same as Aws::S3::Bucket. Only the latter has #presigned_post. It appears that Aws::S3::Client#list_buckets returns information about buckets, not the bucket objects (which you have to create yourself).
Have you tried:
bucket = Aws::S3::Bucket.new('mybucket', client: s3)
Can any one provide me a good documentation for uploading files to S3 using asw-sdk Version 2. I checked out the main doc and in v1 we used to do like
s3 = AWS::S3.new
obj = s3.buckets['my-bucket']
Now in v2 when I try as
s3 = Aws::S3::Client.new
am ending up with
Aws::Errors::MissingRegionError: missing region; use :region option or export region name to ENV['AWS_REGION']
Can anyone help me with this?
As per official documentation:
To use the Ruby SDK, you must configure a region and credentials.
Therefore,
s3 = Aws::S3::Client.new(region:'us-west-2')
Alternatively, a default region can be loaded from one of the following locations:
Aws.config[:region]
ENV['AWS_REGION']
Here's a complete S3 demo on aws v2 gem that worked for me:
Aws.config.update(
region: 'us-east-1',
credentials: Aws::Credentials.new(
Figaro.env.s3_access_key_id,
Figaro.env.s3_secret_access_key
)
)
s3 = Aws::S3::Client.new
resp = s3.list_buckets
puts resp.buckets.map(&:name)
Gist
Official list of AWS region IDs here.
If you're unsure of the region, the best guess would be US Standard, which has the ID us-east-1 for config purposes, as shown above.
If you were using a aws.yml file for your credentials in Rails, you might want to create a file config/initializers/aws.rb with the following content:
filename = File.expand_path(File.join(Rails.root, "config", "aws.yml"))
config = YAML.load_file(filename)
aws_config = config[Rails.env.to_s].symbolize_keys
Aws.config.update({
region: aws_config[:region],
credentials: Aws::Credentials.new(aws_config[:access_key_id], aws_config[:secret_access_key])
})
The config/aws.yml file would need to be adapter to include the region.
development: &development
region: 'your region'
access_key_id: 'your access key'
secret_access_key: 'your secret access key'
production:
<<: *development
I am trying to do it like this:
AWS.config(
:access_key_id => '...',
:secret_access_key => '...'
)
s3 = AWS::S3.new
bucket_name = 'bucket_name'
key = "#{File.basename(avatar_big)}"
s3.buckets[bucket_name].objects[key].write(:file => avatar_big_path)
This working well for a file, the file is uploaded to the root of the set up bucket.
However, how to upload it into the foloder photos that is located in root?
I've tried
key = "photos/#{File.basename(avatar_big)}"
but this doesn't work.
EDIT: error message
Thank you
I had the same issue as the OP. This is what worked for me:
key = "photos/example.jpg"
bucket = s3.buckets[bucket_name]
filepath = Pathname.new("path/to/example.jpg")
o = bucket.objects[key]
o.write(filepath)
Something I would check out would be the object key you are trying to use. There's not much documentation on what are the restrictions are (see this and this) but the one shown in error message looks suspicious to me.
Try including the the path in the file key:
s3.buckets[bucket_name].objects[key].write(:file => "photos/#{avatar_big_path}")