I need to get temporary credentials with specific permission that my app can use to working with S3 over API.
The problem is that policy variable ${aws:userid} don't work in my case.
Steps for reproduce:
Create policy for allow all s3 actions to user-specific folder.
policy = AWS::STS::Policy.new(version: '2012-10-17')
policy.allow({
actions: ["s3:*"],
resources: ["arn:aws:s3:::my-bucket/folder/${aws:userid}/*"]
})
Get temporary credential with previous policy.
sts = AWS::STS.new(:access_key_id => 'ADMIN_ACCESS_KEY',
:secret_access_key => 'ADMIN_SECRET_KEY')
session = sts.new_federated_session("custom_user", policy: policy)
Now session return all required credential params and user_id like 811395474237:custom_user.
And i expect that this user_id will be in ${aws:userid}, but it don't so.
s3u = AWS::S3.new(session.credentials)
bucket = s3u.buckets['my-bucket']
file = bucket.objects.create("folder/#{session.user_id}/user.txt", 'test file', :acl => :public_read)
This return me Access Deny error.
Which should be in ${aws:userid} when use credentials from sts.new_federated_session("custom_user", policy: policy)?
Related
Broad Overview: I am trying to create a .Net Core 3.1 WebApi backend that is authenticated against Amazon Cognito. I want to use the Amazon-hosted sign-in page(s) provided by Cognito. I want to leverage Cognito Identity Pool to provide temporary scoped credentials for users after they have logged in. I cannot figure out how to exchange the Cognito token to create the Credentials to call AWS services.
Technology Overview
.NET Core 3.1 WebApi
Amazon Cognito User Pool for initial authentication
Amazon Identity Pool for defining permissions (Roles) for logged in users
Deployed on AWS via API Gateway + Lambda using the AWS Serverless framework (basically CloudFormation)
Currently both of the following work:
Add [Authorize] attribute to a controller endpoint and access the URL in a browser. This re-directs me to the Cognito-hosted login page and, upon successful login, returns me back to the controller/endpoint and I am authorized.
Create a separate Client application and login to AWS Cognito. Pass the JWT token in the Authorization HTTP header when calling APIs from the client and the Authorization succeeds and API access is granted.
In both cases, the access to the API is permitted however the AmazonServiceClient instances that are created in the WebApi are granted the permissions associated with the Lambda function (which is the proper behavior).
Problem
I need to create AmazonServiceClients whose credentials match the Role defined by the Cognito Identity Pool.
To do this, I need to exchange token provided by logging into Cognito User Pool for temporary credentials in the Identity Pool.
Virtually ALL examples and documentation I can find on this process define how to manually login to Cognito using the API (not the hosted web UI), and then using the API response to create a CognitoUser and then get credentials from the Identity Pool using that user.
The closest (though super brief) documentation I can find to do what I need is from AWS here: https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/cognito-creds-provider.html
// Authenticate user through Facebook
string facebookToken = GetFacebookAuthToken();
// Add Facebook login to credentials. This clears the current AWS credentials
// and retrieves new AWS credentials using the authenticated role.
credentials.AddLogin("graph.facebook.com", facebookAccessToken);
While that example uses Facebook, conceptually it should be the same for any provider (Facebook, Google, Twitter, OpenId, etc.).
My Current Attempt
I have registered CognitoAWSCredentials as a Scoped service as it is user-specific and therefore should only exist as long as the API request session exists.
RegionEndpoint region = Configuration.GetAWSOptions().Region;
services.AddScoped(_ => new CognitoAWSCredentials(Settings.CognitoIdentityPoolId, region));
I have created an event handler that gets triggered when the OpenIdConnect event 'OnTokenValidated' is fired. This happens after I login to the Cognito hosted web UI and am redirected back to my API.
In this handler I can call:
CognitoAWSCredentials creds = services.BuildServiceProvider().GetRequiredService<CognitoAWSCredentials>();
creds.AddLogin( ... ??? ...);
(note: since I'm setting all this up in the Startup.ConfigureServices(IServiceCollection services) method, I am building an IServiceProvider instance each time authentication succeeds... which may be inefficient but I haven't figured out another way to access a scoped service inside the ConfigureServices method)
All this preamble to say that I cannot find a set of values for the AddLogin call which allow this test call to succeed:
ImmutableCredentials immCreds = creds.GetCredentials();
Relevant Data Structures
In the event handler where I can call AddLogin, I have access to: Microsoft.AspNetCore.Authentication.OpenIdConnect.TokenValidatedContext which in particular contains:
Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectMessage with:
access_token
id_token
refresh_token
System.IdentityModel.Tokens.Jwt.JwtSecurityToken with:
{
{
"alg": "RS256",
"kid": "**************************"
}. {
"at_hash": "**************************",
"sub": "**************************",
"email_verified": true,
"iss": "https://cognito-idp.ca-central-1.amazonaws.com/**************************",
"cognito:username": "**************************",
"nonce": "**************************",
"aud": "**************************",
"event_id": "**************************",
"token_use": "id",
"auth_time": 1595260191,
"exp": 1595263791,
"iat": 1595260191,
"email": "**************************"
}
}
I have tried using the iss value as the providerName in AddLogin, and either the access_token or id_token but neither work.
Does anyone know what I need to use for AddLogin in order for Cognito to create Identity Pool credentials for me based upon a JWT token from a Cognito User Pool login?
unless I missed it, I haven't seen documentation that states this, but even though all the Issuer fields on the various data structures include the 'https://', you need to strip it before using the Issuer as the providerName on the AddLogin call. ugh.
CognitoAWSCredentials creds = services.BuildServiceProvider().GetRequiredService<CognitoAWSCredentials>();
string shortIssuer = tokenValidatedContext.SecurityToken.Issuer;
if (shortIssuer.StartsWith("https://", StringComparison.InvariantCultureIgnoreCase)) shortIssuer = shortIssuer.Substring("https://".Length);
if (shortIssuer.StartsWith("http://", StringComparison.InvariantCultureIgnoreCase)) shortIssuer = shortIssuer.Substring("http://".Length);
creds.AddLogin(shortIssuer, tokenValidatedContext.TokenEndpointResponse.IdToken);
now, the above code has a problem as the services.BuildServiceProvider(). part means the credentials object I modify isn't global (only local to the service provider I built here I think), but that's a different issue - just noting that in case anyone is copying this code.
services...<other authentication setup>...
.AddOpenIdConnect(OpenIdConnectDefaults.AuthenticationScheme, options =>
{
options.ClientId = Settings.CognitoClientId;
options.MetadataAddress = CognitoMetadataAddress;
options.ResponseType = OpenIdConnectResponseType.Code;
options.SaveTokens = true;
options.UsePkce = true;
options.TokenValidationParameters = new TokenValidationParameters()
{
ValidateIssuer = true,
ValidIssuers = new string[] { Settings.CognitoAuthority },
ValidateAudience = true,
ValidAudiences = new string[] { Settings.CognitoClientId }
};
options.Events = new OpenIdConnectEvents() {
OnTokenValidated = tokenValidatedContext => {
CognitoAWSCredentials creds = services.BuildServiceProvider().GetRequiredService<CognitoAWSCredentials>();
string shortIssuer = tokenValidatedContext.SecurityToken.Issuer;
if (shortIssuer.StartsWith("https://", StringComparison.InvariantCultureIgnoreCase)) shortIssuer = shortIssuer.Substring("https://".Length);
if (shortIssuer.StartsWith("http://", StringComparison.InvariantCultureIgnoreCase)) shortIssuer = shortIssuer.Substring("http://".Length);
creds.AddLogin(shortIssuer, tokenValidatedContext.TokenEndpointResponse.IdToken);
return Task.CompletedTask;
}
};
})
(some code removed to focus on specifically the OpenId Connect event and the CognitoAWSCredentials init)
I'm building a terraform template to create Azure resources including Keyvault Secrets. The customer Subscription policy doesn't allow anyone to update/delete/view keyvault secrets.
If I run terraform apply for the first time, it will work perfectly. However, running the same template again will give you the following error: Error:
Error updating Key Vault "####" (Resource Group "####"): keyvault.VaultsClient#Update: Failure responding to request: StatusCode=403 --
Original Error: autorest/azure: Service returned an error. Status=403 Code="RequestDisallowedByPolicy" Message="Resource '###' was disallowed by policy. Policy identifiers: '[{\"policyAssignment\":{\"name\":\"###nis-deny-keyvault-acl\", ...
on ..\..\modules\azure\keyvault\main.tf line 15, in resource "azurerm_key_vault" "keyvault":
15: resource "azurerm_key_vault" "keyvault" {
How can I get my CI/CD working while that means terraform apply will be continuously running?
Is there a way to pass this policy in terraform?
Is there a way to prevent terraform from updating KV once it created (other than locking the resource)?
Here is the Keyvault module:
variable "keyvault_id" {
type = string
}
variable "secrets" {
type = map(string)
}
locals {
secret_names = keys(var.secrets)
}
resource "azurerm_key_vault_secret" "secret" {
count = length(var.secrets)
name = local.secret_names[count.index]
value = var.secrets[local.secret_names[count.index]]
key_vault_id = var.keyvault_id
}
data "azurerm_key_vault_secret" "secrets" {
count = length(var.secrets)
depends_on = [azurerm_key_vault_secret.secret]
name = local.secret_names[count.index]
key_vault_id = var.keyvault_id
}
output "keyvault_secret_attributes" {
value = [for i in range(length(azurerm_key_vault_secret.secret.*.id)) : data.azurerm_key_vault_secret.secrets[i]]
}
And here is the module from my template:
locals {
secrets_map = {
appinsights-key = module.app_insights.app_insights_instrumentation_key
storage-account-key = module.storage_account.primary_access_key
}
output_secret_map = {
for secret in module.keyvault_secrets.keyvault_secret_attributes :
secret.name => secret.id
}
}
module "keyvault" {
source = "../../modules/azure/keyvault"
keyvault_name = local.kv_name
resource_group_name = azurerm_resource_group.app_rg.name
}
module "keyvault_secrets" {
source = "../../modules/azure/keyvault-secret"
keyvault_id = module.keyvault.keyvault_id
secrets = local.secrets_map
}
module "app_service_keyvault_access_policy" {
source = "../../modules/azure/keyvault-policy"
vault_id = module.keyvault.keyvault_id
tenant_id = module.app_service.app_service_identity_tenant_id
object_ids = module.app_service.app_service_identity_object_ids
key_permissions = ["get", "list"]
secret_permissions = ["get", "list"]
certificate_permissions = ["get", "list"]
}
Using Terraform for provisioning and managing a keyvault with that kind of limitations sounds like a bad idea. Terraforms main idea is to monitor the state of your resources - if it is not allowed to read the resource it becomes pretty useless. Your problem is not even that Terraform is trying to update something, it fails because it wants to check the current state of your resource and fails.
If your goal is just to create secrets in a keyvault, I would just us the az keyvault commands like this:
az login
az keyvault secret set --name mySecret --vault-name myKeyvault --value mySecretValue
An optimal solution would of course be that your service principal that you use for executing Terrafom commands has the sufficient rights to perform the actions it was created for.
I know this is a late answer, but for future visitors:
The pipeline running the Terraform Plan and Apply will need to have proper access to the key vault.
So, if you are running your CI/CD from Azure Pipelines, you would typically have a service connection that your pipeline uses for authentication.
The service connection you use for Terraform is most likely based on a service principal that has contributor rights (at least at resource group level) for it to provisioning anything at all.
If that is the case, then you must add a policy giving that same service principal (Use the Service Principals Enterprise Object Id) to have at least list, get and set permissions for secrets.
I'm sure that this is a simple error, but I'm interested in writing a program that collects information on all of my github repositories. While this seems simple enough to do with Octokit, I've run into issues associated with authenticating my session.
client = Octokit::Client.new \
:login => 'MY_USER_NAME',
:password => 'MY_PASSWORD'
puts client
user = client.user("MY_USER_NAME", :headers => { "PERSONAL_ACCESS_TOKEN_NAME" => "TOKEN" })
puts user
Unfortunately this results in the following:
GET https://api.github.com/users/mccoleman75225: 401 - Must specify two-factor authentication OTP code. // See: https://developer.github.com/v3/auth#working-with-two-factor-authentication (Octokit::OneTimePasswordRequired)
How does someone go about authenticating their session?
As of January 2022, you can create a PAT (Personal Access Token) in your GitHub Developer Settings and use that to connect through the Octokit client like so:
client = Octokit::Client.new(:access_token => "<Your Personal Access Token>")
user = client.user
user.login
# => "monacat"
Here's a step-by-step guide on how to create a PAT. Try to select the correct permissions when creating your token or you'll get back a 403 error with a message explaining the missing scope. You can always go back and edit your scopes later though.
Sources:
Octokit.rb — Authentication
GitHub API Authentication - Personal Access Tokens
Looks like you have 2 Factor Authentication enabled on your account so you'll need to add your 2FA token:
client = Octokit::Client.new \
:login => 'defunkt',
:password => 'c0d3b4ssssss!'
client.create_authorization(:scopes => ["user"], :note => "Name of token",
:headers => { "X-GitHub-OTP" => "<your 2FA token>" })
# => <your new oauth token>
See documentation
I'm unable to a folder by providing an id to that folder using Boxr gem. Previously I didn't has the enterprise settings as shown in this post which I have now fixed. I'm creating a token using JWT authentication get_user_token method the following way.
token = Boxr::get_user_token("38521XXXX", private_key: ENV.fetch('JWT_PRIVATE_KEY'), private_key_password: ENV.fetch('JWT_PRIVATE_KEY_PASSWORD'), public_key_id: ENV.fetch('JWT_PUBLIC_KEY_ID'), client_id: ENV.fetch('BOX_CLIENT_ID'), client_secret: ENV.fetch('BOX_CLIENT_SECRET'))
I then pass this this token when creating a client.
client = Boxr::Client.new(token)
when I check the current user on client this is what I get:
client.current_user
=> {"type"=>"user",
"id"=>"60853XXXX",
"name"=>"OnlineAppsPoC",
"login"=>"AutomationUser_629741_06JgxiPtPj#boxdevedition.com",
"created_at"=>"2018-10-04T08:41:32-07:00",
"modified_at"=>"2018-10-04T08:41:50-07:00",
"language"=>"en",
"timezone"=>"America/Los_Angeles",
"space_amount"=>10737418240,
"space_used"=>0,
"max_upload_size"=>2147483648,
"status"=>"active",
"job_title"=>"",
"phone"=>"",
"address"=>"",
"avatar_url"=>"https://app.box.com/api/avatar/large/6085300897"}
When I run client.methods I see there is folder_from_id however when I call that method I get the following error:
pry(#<FormsController>)> client.folder_from_id("123456", fields: [])
Boxr::BoxrError: 404: Not Found
from /usr/local/bundle/gems/boxr-1.4.0/lib/boxr/client.rb:239:in `check_response_status'
I have the following settings:
I also authorize the application. Not sure what else to do.
token = Boxr::get_user_token(user_id,
private_key: ENV.fetch('JWT_PRIVATE_KEY'),
private_key_password: ENV.fetch('JWT_PRIVATE_KEY_PASSWORD'),
public_key_id: ENV.fetch('JWT_PUBLIC_KEY_ID'),
client_id: ENV.fetch('BOX_CLIENT_ID'),
client_secret: ENV.fetch('BOX_CLIENT_SECRET'))
client = Boxr::Client.new(token.access_token)
folder = client.folder_from_id(folder_id)
client.upload_file(file_path, folder)
For anybody using C# and BOXJWT.
You just need to have a boxManager set up and will get you with anything you need, say BoxFile, Folder etc.
If you have the folderID, well & good, but if you need to retrieve, this can be done as shown below:
string inputFolderId = _boxManager.GetFolder(RootFolderID).Folders.Where(i => i.Name == boxFolder).FirstOrDefault().Id; //Retrieves FolderId
Folder inputFolder = _boxManager.GetFolder(inputFolderId);
I followed the quick start and am attempting to create a user using the google-api-ruby-client.
I've set up access in the google api console. And I can get this to work using the API explorer.
But when I try using the ruby client, I'm getting a resource not found: domain error.
Here's the code:
def self.create_user
# Initialize the client.
client = Google::APIClient.new(
:application_name => 'MYAPP',
:application_version => '0.0.1'
)
# Authorization
# Load our credentials for the service account
key = Google::APIClient::KeyUtils.load_from_pkcs12(KEY_FILE, KEY_SECRET)
client.authorization = Signet::OAuth2::Client.new(
token_credential_uri: 'https://accounts.google.com/o/oauth2/token',
audience: 'https://accounts.google.com/o/oauth2/token',
scope: 'https://www.googleapis.com/auth/admin.directory.user',
issuer: ACCOUNT_ID,
signing_key: key)
# Request a token for our service account
client.authorization.fetch_access_token!
# Load API Methods
admin = client.discovered_api('admin', 'directory_v1')
# Make an API call.
result = client.execute(
admin.users.update,
name: { familyName: 'testy', givenName: 'testerson' },
password: '!password12345!',
primaryEmail: 'ttesterson#my-actual-domain.com'
)
result.data
end
Here's the response:
"error"=>{"errors"=>[{"domain"=>"global", "reason"=>"notFound", "message"=>"Resource Not Found: domain"}], "code"=>404, "message"=>"Resource Not Found: domain"}
Why?
After a bit of documentation reading, there were two things that I needed to fix.
I hadn't set up the proper authorization for my test service account.
You have to go to the Apps Console > Security > Advanced > Manage API client access and add the client url for your service account as well as any specific permissions that you want to add
As seen in this question, it seems that you need to create a user object rather than just passing in parameters.
Here's my updated code:
# Authorization happens here ....
api = client.discovered_api('admin', 'directory_v1')
new_user = api.users.insert.request_schema.new(
name: { familyName: 'Testy', givenName: 'Testerson' },
primaryEmail: 'ttttesterson#<domain-redacted>.com',
password: 'password123'
)
result = client.execute(
api_method: api.users.insert,
body_object: new_user
)