Using the aws-sdk-s3 gem, I am currently able to upload items to buckets and create signed URLs, and am trying to determine if an object exists in a bucket or not. All the documentation I see says client.bucket('bucketname') should work, but in my case it is not. I've tried:
client.bucket('bucketname')
client.bucket['bucketname']
client.buckets('bucketname')
client.buckets['bucketname']
but none work. This suggestion using head_object is a possibility (https://github.com/cloudyr/aws.s3/issues/160), but I'm still curious why bucket isn't working.
DOCS:
https://gist.github.com/hartfordfive/19097441d3803d9aa75ffe5ecf0696da
https://docs.aws.amazon.com/sdk-for-ruby/v3/api/index.html#Resource_Interfaces
You should call bucket or buckets on Aws::S3::Resource instance and not on Aws::S3::Client instance as an error states.
And the links you provided as well as docs show that:
s3 = Aws::S3::Resource.new(
region: 'us-east-1',
credentials: Aws::InstanceProfileCredentials.new()
)
bucket = s3.bucket('my-daily-backups')
Related
how can I get ALL records from route53?
referring code snippet here, which seemed to work for someone, however not clear to me: https://github.com/aws/aws-sdk-ruby/issues/620
Trying to get all (I have about ~7000 records) via resource record sets but can't seem to get the pagination to work with list_resource_record_sets. Here's what I have:
route53 = Aws::Route53::Client.new
response = route53.list_resource_record_sets({
start_record_name: fqdn(name),
start_record_type: type,
max_items: 100, # fyi - aws api maximum is 100 so we'll need to page
})
response.last_page?
response = response.next_page until response.last_page?
I verified I'm hooked into right region, I see the record I'm trying to get (so I can delete later) in aws console, but can't seem to get it through the api. I used this: https://github.com/aws/aws-sdk-ruby/issues/620 as a starting point.
Any ideas on what I'm doing wrong? Or is there an easier way, perhaps another method in the api I'm not finding, for me to get just the record I need given the hosted_zone_id, type and name?
The issue you linked is for the Ruby AWS SDK v2, but the latest is v3. It also looks like things may have changed around a bit since 2014, as I'm not seeing the #next_page or #last_page? methods in the v2 API or the v3 API.
Consider using the #next_record_name and #next_record_type from the response when #is_truncated is true. That's more consistent with how other paginations work in the Ruby AWS SDK, such as with DynamoDB scans for example.
Something like the following should work (though I don't have an AWS account with records to test it out):
route53 = Aws::Route53::Client.new
hosted_zone = ? # Required field according to the API docs
next_name = fqdn(name)
next_type = type
loop do
response = route53.list_resource_record_sets(
hosted_zone_id: hosted_zone,
start_record_name: next_name,
start_record_type: next_type,
max_items: 100, # fyi - aws api maximum is 100 so we'll need to page
)
records = response.resource_record_sets
# Break here if you find the record you want
# Also break if we've run out of pages
break unless response.is_truncated
next_name = response.next_record_name
next_type = response.next_record_type
end
I'm trying to copy data from one bucket to another using the ruby "aws-sdk" gem version 3.
My code is shown below:
temporary_object = #temporary_bucket.object(temporary_path)
permanent_object = #permanent_bucket.object(permanent_path)
temporary_object.copy_to(permanent_object)
However I keep getting the error Aws::S3::Errors::NoSuchKey: The specified key does not exist. Which makes sense as the permanent bucket doesn't exist at this moment however I thought that using copy_to creates the bucket if it does not exist.
Any advice would be very helpful.
Thanks
trying to figure AWS S3 API and failing miserably...
I currently got a bucket which consist lots of videos.
I need to request all the videos as an object, that will have the video meta-data which I set once uploading, and the link to share the video.
Problem is I'm getting the object without any of the above...
What Iv'e got so far -
AWS.config.update({accessKeyId: 'id', secretAccessKey: 'key', region: 'eu-
west-1'});
var s3 = new AWS.S3();
var params = {
Bucket: 'Bucket-name',
Delimiter: '/',
Prefix: 'resource/folder-with-videos/'
}
s3.listObjects(params, function (err, data) {
if(err)throw err;
console.log(data);
});
Thanks for reading :)
UPDATE - found that when using getObject and adding ExposeHeader to the CORS setting I can indeed get the metadata I set.
problem is getObject only works on a specific Object (video in my case).
any Idea how I can get all the object like listObject and have values of each object like I do on getObject?
Only solution I can think of is doing listObject to get a list of all the objects, and then by this result to do for each object an getObject ajax?... rip UX
thanks :)
A couple of things to sort out first.
As per the documentation, http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#listObjects-property listObjects API returns exactly what is mentioned in the callback parameters. Using a delimiter causes the S3 to list objects only one level below the prefix given. It won't traverse all objects recursively.
In order to get the URL, you can use http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property. I am not really sure what you meant by meta-data.
Im trying to delete an object on S3 using the ruby aws-sdk (version 2). It works fine, but it returns this
<struct Aws::S3::Types::DeleteObjectOutput delete_marker=nil,version_id=nil, request_charged=nil>
Which doesnt make sense because in the documentation it says the response should be of the type:
resp.delete_marker #=> true/false
resp.version_id #=> String
resp.request_charged #=> String, one of "requester"
Why am I becoming nil? I want to know if the object was deleted or not. I am getting that response both when i succeed in deleting the object and when I dont.
This is the code Im using to delete the object:
creds = Aws::Credentials.new(user_access_key,
user_secret_key,
session_token)
s3 = Aws::S3::Client.new( region: 'eu-west-1',
credentials: creds)
key = "myKey.csv"
r = s3.delete_object(bucket: "myBucket",
key: key)
Your delete object was successful. The Amazon S3 API only returns those values under certain circumstances. In this case, your object was not in a versioned bucket (no version id or delete marker boolean), and is not configured for request-payer.
As a general rule, if the SDK does not raise an error from the response, it is successful. In this case, the API reference documentation may be confusing as it does not clearly indicate that these values may be nil.
aws-sdk-ruby Aws::CloudFormation::Client#delete_stack Documentation
The amazon documentation for glacier doesn't seem to include any Ruby examples and the documentation itself is rather sparse.
I gather I need to instantiate a Glacier Client object and then use the upload_multipart_part method to access the Glacier API, but not sure how to massage the arguments to pass to upload_multipart_part. How do I calculate the checksums that AWS is looking for using ruby? And what is upload_id?
UPDATE
Figured out most of this by reading Amazon's documentation. They don't appear to have any code samples for ruby, the github repo of code examples does not cover Glacier. But by looking at the raw API documentation and some Java and PHP examples, it looks like I'd do this:
client = AWS::Glacier::Client.new(access_key_id: ACCESS_KEY_ID, secret_access_key: SECRET_ACCESS_KEY)
resp = client.initiate_multipart_upload(account_id: ACCOUNT_ID, vault_name: 'My Vault', archive_description: "Backup of some stuff", part_size: PART_SIZE_IN_BYTES)
And if all goes well, the Amazon API response should include a unique upload_id which I would then use in subsequent calls using client.upload_multipart_part().
I'm guessing the checksums can be calculated like this:
Digest::SHA256.file(file_to_upload).hexdigest
UPDATE 2
Seems like this has already been solved:
https://github.com/fog/fog
For anyone else interested, this link is helpful, pretty much covers most of what you need:
http://www.spacevatican.org/2012/9/4/using-glacier-with-fog/
It doesn't really provide all details on how to instantiate a glacier object:
glacier = Fog::AWS::Glacier.new({
:aws_access_key_id => YOUR_ACCESS_KEY_ID,
:aws_secret_access_key => YOUR_SECRET_ACCESS_KEY
})