So I am trying to write a simple script to connect to AWS s3 and create a bucket but I keep getting Access Denied (Aws::S3::Errors::AccessDenied)
This is my code
require 'aws-sdk'
require 'csv'
def test()
creds = CSV.read('accessKeys.csv')
s3_client = Aws::S3::Client.new(
region: 'us-west-2',
credentials: Aws::Credentials.new(creds[1][0], creds[1][1]),
)
s3 = Aws::S3::Resource.new(client: s3_client)
s3.create_bucket({
bucket: "dns-complaint-bucket",
})
end
test()
I have also attached AmazonS3FullAccess policy to the IAM user that I am using.
Related
As Softlayer or IBM Cloud has moved from Swift based Object Storage to S3 based Cloud Object Storage. I am using fog/aws instead of fog/softlayer.
The below is the code:
require 'fog/aws'
fog_properties = {
provider: 'AWS',
aws_access_key_id: username,
aws_secret_access_key: api_key
}
#client = Fog::Storage.new(fog_properties)
#client.directories
But it failed even with valid key and id.
<Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.\</Message><AWSAccessKeyId>####</AWSAccessKeyId><RequestId>####</RequestId><HostId>##</HostId></Error>
End Point IBM COS uses is "https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints"
When I tried to use fog alone(require 'fog'). It throws the below error:
Unable to activate google-api-client-0.23.9, because mime-types-2.99.3 conflicts with mime-types (~> 3.0) (Gem::ConflictError)
Please suggest how to resolve these issues.
https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints"
This is not an endpoint but a list of endpoints in JSON.
Choose the endpoint for your bucket location.
For example if your bucket is in us-south the public endpoint is
https://s3.us-south.cloud-object-storage.appdomain.cloud
The following code worked for IBM Cloud Objects Storage
properties = {
region: region,
endpoint: URI('https://s3.us-south.cloud-object-storage.appdomain.cloud'),
credentials: Aws::Credentials.new(access_key_id, secret_access_key)
}
Aws.config.update(properties)
#client = Aws::S3::Client.new
Properties for the config can also be set as ENV variables.
Below are few basic operations performed on COS.
List all the bucker names
#client.list_buckets.buckets.map(&:name)
Create Bucket
#client.create_bucket(bucket: )
Upload a file
#client.put_object(bucket: , key: , body: )
Download a file
#client.get_object(bucket: , key: )
Delete a file
#client.delete_object(bucket: , key: )
Delete a Bucket
#client.delete_bucket(bucket: )
I'm trying to generate a presigned url for an s3 bucket using Ruby.
client = Aws::S3::Client.new(
region: 'eu-west-1', #or any other region
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
)
#signer = Aws::S3::Presigner.new(client: client)
#signer.presigned_url(
:put_object,
bucket: ENV['S3_PROFILES_BUCKET'],
key: "test-#{SecureRandom.uuid}"
)
I try and take the url that is returned from this, something like:
"https://some-bucket.s3.eu-west-1.amazonaws.com/test-4ad40444-e907-4748-a025-a12515580450?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATTSSBDQFDFFX36UU4%2F20191204%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20191204T002242Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=31b0a90127f43e79462713b101b5fc80146c50f800cfce31c493d206ea142333"
When I try and make a POST (or PUT) request to this URL with an image binary (I'm using Postman) I get an error about the signature not being correct.
My use case is:
I want to execute a lambda function for the PUT event of an existing S3 bucket.
The problem is that you in CloudFormation or CDK you cannot add notifications for an existing bucket, only for buckets that are created.
To get around this I am trying to use a custom resource that adds the Lambda function to the PutNotification. I have this working fine in CloudFormation but I am trying to use CDK now to do something similar.
To simulate what I have in CloudFormation I need to add a bucket policy to the existing bucket granting permission to the action s3:PutBucketNotification to the lambda execution role principal.
In CloudFormation I do this like this:
NotificationBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref BucketName
PolicyDocument:
Statement:
- Effect: "Allow"
Action:
- 's3:PutBucketNotification'
Resource: !Sub "arn:aws:s3:::${BucketName}"
Principal:
AWS: !GetAtt LambdaExecutionRole.Arn
I am attempting to create the bucket policy and add the statement to it in CDK but I need the Arn of the Lambda Function's Arn
const bucket = Bucket.fromBucketName(this, "Bucket", "my-bucket-name");
const bucketConfigurationFunction = new lambda.SingletonFunction(this, "bucketConfigurationFunction ", {
runtime: lambda.Runtime.NODEJS_8_10,
code: lambda.Code.asset('lambda/bucket-configuration'),
handler: 'lambda_function.handler',
timeout: cdk.Duration.seconds(300),
uuid: '72561a5f-e772-4365-b3d1-f59e8ddc60b1'
})
const bucketPolicy = new BucketPolicy(this, "TargetBucketPolicy", {
bucket: bucket
})
const bucketPolicyStatement = new PolicyStatement()
bucketPolicyStatement.addActions("s3:PutBucketNotification");
//Need to put the execution role arn here but role is undefined
bucketPolicyStatement.addArnPrincipal(bucketConfigurationFunction.role.roleArn)
I have read the CDK creates a lambda function execution role automatically however when I try to access the role Arn to add it as the principal in the policy statement, it is undefined.
Am I doing this totally the wrong way?
Because TypeScript is quite strict on checking optional variables and the role is generated at runtime, you need to box it using and if but that's still fine. For example this works:
const bucketPolicyStatement = new iam.PolicyStatement()
bucketPolicyStatement.addActions("s3:PutBucketNotification");
if (bucketConfigurationFunction.role) {
bucketPolicyStatement.addArnPrincipal(bucketConfigurationFunction.role.roleArn)
}
const bucketPolicy = new s3.BucketPolicy(this, "TargetBucketPolicy", {
bucket: bucket,
})
bucketPolicy.document.addStatements(bucketPolicyStatement);
I need to get temporary credentials with specific permission that my app can use to working with S3 over API.
The problem is that policy variable ${aws:userid} don't work in my case.
Steps for reproduce:
Create policy for allow all s3 actions to user-specific folder.
policy = AWS::STS::Policy.new(version: '2012-10-17')
policy.allow({
actions: ["s3:*"],
resources: ["arn:aws:s3:::my-bucket/folder/${aws:userid}/*"]
})
Get temporary credential with previous policy.
sts = AWS::STS.new(:access_key_id => 'ADMIN_ACCESS_KEY',
:secret_access_key => 'ADMIN_SECRET_KEY')
session = sts.new_federated_session("custom_user", policy: policy)
Now session return all required credential params and user_id like 811395474237:custom_user.
And i expect that this user_id will be in ${aws:userid}, but it don't so.
s3u = AWS::S3.new(session.credentials)
bucket = s3u.buckets['my-bucket']
file = bucket.objects.create("folder/#{session.user_id}/user.txt", 'test file', :acl => :public_read)
This return me Access Deny error.
Which should be in ${aws:userid} when use credentials from sts.new_federated_session("custom_user", policy: policy)?
My aws Profile looks like:
[my_aws_profile]
aws_access_key_id = XXXXXXXXXXXXXXXXXX
aws_secret_access_key = YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
region = us-east-1
My call:
s3 = Aws::S3::Client.new(region: 'us-east-1', credentials: Aws::SharedCredentials.new(profile_name: 'my_aws_profile'))
s3.put_object_acl(
bucket: $global_config['bucket'], key: someFile_key_json, grant_read: grant_read)
Rerturns
Aws::S3::Errors::MissingSecurityHeader: Your request was missing a required header
What am I doing wrong? ...
How do I fix this?