Short version:
How to correctly set up application permissions and/or role assignments and/or something else that I'm missing, so that application id/secret (OAuth client credentials) can be used to create shared mailboxes?
So far I've tried couple combinations of permissions/roles, e.g. Exchange.ManageAsApp with User Administrator (fe930be7-5e62-47db-91af-98c3a49a38b1), Exchange administrator (29232cdf-9323-42fd-ade2-1d097af3e4de) and bunch of other.
Details:
I have a bunch of powershell scripts used to automate various tasks on Exchange Online. So far I've been using basic auth, which I was able to successfully convert into OAuth password flow.
But to get rid of dependency on service account completely, I'd prefer to use credentials flow. In background I'm trying to do something like this:
var authenticationContext = new AuthenticationContext($"https://login.microsoftonline.com/{TenantId}", false, _tokenCache);
var clientCredential = new ClientCredential(ClientId, ClientSecret);
var authenticationResult = await authenticationContext.AcquireTokenAsync(Resource, clientCredential);
var username = "OAuthUser#" + TenantId;
var password = authenticationResult.CreateAuthorizationHeader();
var executor = new ExolExecutor(username, password);
await executor.Execute(Script, cancellationToken);
where executor does the regular thing:
Create PSSession to https://outlook.office365.com/powershell-liveid?BasicAuthToOAuthConversion=true
Executes powershell script using
using PowerShell powershell = PowerShell.Create();
powershell.Runspace = runspace;
powershell.AddScript(script);
...
await Task.Factory.FromAsync(powershell.BeginInvoke(input, output), powershell.EndInvoke);
Remove PSSession
So far so good. Works perfectly fine with Get-Mailbox -ResultSize 1.
But when trying to create new shared mailbox New-Mailbox -Name "pko222" -DisplayName "pko222" -Alias "pko222" -Shared, I'm getting
CategoryInfo.Activity: New-Mailbox
CategoryInfo.Category: 1001
CategoryInfo.Reason: ADOperationException
CategoryInfo.TargetName:
CategoryInfo.TargetType:
ErrorDetails.Message:
ErrorDetails.RecommendedAction:
Exception.Message: Active Directory operation failed on DB7PR01A03DC005.EURPR01A003.prod.outlook.com. This error is not retriable. Additional information: Access is denied.
Active directory response: 00000005: SecErr: DSID-03152612, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0
FullyQualifiedErrorId: [Server=BEXP281MB0087,RequestId=88419a8e-78a4-4967-9bca-71d40feb5150,TimeStamp=10/6/2020 11:57:38 AM] [FailureCategory=Cmdlet-ADOperationException] 2C0312E5,Microsoft.Exchange.Management.RecipientTasks.NewMailbox
JWT token looks something like this:
{
"aud": "https://outlook.office365.com",
"iss": "https://sts.windows.net/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/",
"iat": 1601985127,
"nbf": 1601985127,
"exp": 1601989027,
"aio": "E2RgYFCOsw1iZj34elV49CH5zyd5AQ==",
"app_displayname": "XXXXXXXXXXX",
"appid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"appidacr": "1",
"idp": "https://sts.windows.net/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/",
"oid": "zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz",
"rh": "0.AAAAv9y4fwZQ_0G6_d1kLKJ_sarAXb_REQFHhc2EM1FNn9tIAAA.",
"roles": ["User.Read.All", "full_access_as_app", "Mail.ReadWrite", "MailboxSettings.ReadWrite", "User.ReadBasic.All", "Mailbox.Migration", "Mail.Read", "Mail.Send", "MailboxSettings.Read", "Exchange.ManageAsApp"],
"sid": "qqqqqqqq-qqqq-qqqq-qqqq-qqqqqqqqqqqq",
"sub": "zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz",
"tid": "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy",
"uti": "CRytfXbD80y3ATmQvd-VAQ",
"ver": "1.0",
"wids": ["29232cdf-9323-42fd-ade2-1d097af3e4de", "88d8e3e3-8f55-4a1e-953a-9b9898b8876b", "fe930be7-5e62-47db-91af-98c3a49a38b1", "9360feb5-f418-4baa-8175-e2a00bac4301", "62e90394-69f5-4237-9190-012177145e10", "0997a1d0-0d1d-4acb-b408-d5ca73121e90"]
}
Fyi i managed to make it work on my side.
you just need to add the following param in the connection uri
&email=SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}#yourtenantname.onmicrosoft.com
so the connection uri looks like :
https://outlook.office365.com/PowerShell-LiveId?BasicAuthToOAuthConversion=true&email=SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}#yourtenantname.onmicrosoft.com
just change the suffix from 'yourtenantname' with ... your tenant name! dont put the tenant guid !
https://learn.microsoft.com/en-us/answers/questions/451006/pssession-and-modern-auth.html
I'm unable to a folder by providing an id to that folder using Boxr gem. Previously I didn't has the enterprise settings as shown in this post which I have now fixed. I'm creating a token using JWT authentication get_user_token method the following way.
token = Boxr::get_user_token("38521XXXX", private_key: ENV.fetch('JWT_PRIVATE_KEY'), private_key_password: ENV.fetch('JWT_PRIVATE_KEY_PASSWORD'), public_key_id: ENV.fetch('JWT_PUBLIC_KEY_ID'), client_id: ENV.fetch('BOX_CLIENT_ID'), client_secret: ENV.fetch('BOX_CLIENT_SECRET'))
I then pass this this token when creating a client.
client = Boxr::Client.new(token)
when I check the current user on client this is what I get:
client.current_user
=> {"type"=>"user",
"id"=>"60853XXXX",
"name"=>"OnlineAppsPoC",
"login"=>"AutomationUser_629741_06JgxiPtPj#boxdevedition.com",
"created_at"=>"2018-10-04T08:41:32-07:00",
"modified_at"=>"2018-10-04T08:41:50-07:00",
"language"=>"en",
"timezone"=>"America/Los_Angeles",
"space_amount"=>10737418240,
"space_used"=>0,
"max_upload_size"=>2147483648,
"status"=>"active",
"job_title"=>"",
"phone"=>"",
"address"=>"",
"avatar_url"=>"https://app.box.com/api/avatar/large/6085300897"}
When I run client.methods I see there is folder_from_id however when I call that method I get the following error:
pry(#<FormsController>)> client.folder_from_id("123456", fields: [])
Boxr::BoxrError: 404: Not Found
from /usr/local/bundle/gems/boxr-1.4.0/lib/boxr/client.rb:239:in `check_response_status'
I have the following settings:
I also authorize the application. Not sure what else to do.
token = Boxr::get_user_token(user_id,
private_key: ENV.fetch('JWT_PRIVATE_KEY'),
private_key_password: ENV.fetch('JWT_PRIVATE_KEY_PASSWORD'),
public_key_id: ENV.fetch('JWT_PUBLIC_KEY_ID'),
client_id: ENV.fetch('BOX_CLIENT_ID'),
client_secret: ENV.fetch('BOX_CLIENT_SECRET'))
client = Boxr::Client.new(token.access_token)
folder = client.folder_from_id(folder_id)
client.upload_file(file_path, folder)
For anybody using C# and BOXJWT.
You just need to have a boxManager set up and will get you with anything you need, say BoxFile, Folder etc.
If you have the folderID, well & good, but if you need to retrieve, this can be done as shown below:
string inputFolderId = _boxManager.GetFolder(RootFolderID).Folders.Where(i => i.Name == boxFolder).FirstOrDefault().Id; //Retrieves FolderId
Folder inputFolder = _boxManager.GetFolder(inputFolderId);
I get an error AWS::S3::Errors::InvalidRequest The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. when I try upload file to S3 bucket in new Frankfurt region. All works properly with US Standard region.
Script:
backup_file = '/media/db-backup_for_dev/2014-10-23_02-00-07/slave_dump.sql.gz'
s3 = AWS::S3.new(
access_key_id: AMAZONS3['access_key_id'],
secret_access_key: AMAZONS3['secret_access_key']
)
s3_bucket = s3.buckets['test-frankfurt']
# Folder and file name
s3_name = "database-backups-last20days/#{File.basename(File.dirname(backup_file))}_#{File.basename(backup_file)}"
file_obj = s3_bucket.objects[s3_name]
file_obj.write(file: backup_file)
aws-sdk (1.56.0)
How to fix it?
Thank you.
AWS4-HMAC-SHA256, also known as Signature Version 4, ("V4") is one of two authentication schemes supported by S3.
All regions support V4, but US-Standard¹, and many -- but not all -- other regions, also support the other, older scheme, Signature Version 2 ("V2").
According to http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html ... new S3 regions deployed after January, 2014 will only support V4.
Since Frankfurt was introduced late in 2014, it does not support V2, which is what this error suggests you are using.
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html explains how to enable V4 in the various SDKs, assuming you are using an SDK that has that capability.
I would speculate that some older versions of the SDKs might not support this option, so if the above doesn't help, you may need a newer release of the SDK you are using.
¹US Standard is the former name for the S3 regional deployment that is based in the us-east-1 region. Since the time this answer was originally written,
"Amazon S3 renamed the US Standard Region to the US East (N. Virginia) Region to be consistent with AWS regional naming conventions." For all practical purposes, it's only a change in naming.
With node, try
var s3 = new AWS.S3( {
endpoint: 's3-eu-central-1.amazonaws.com',
signatureVersion: 'v4',
region: 'eu-central-1'
} );
You should set signatureVersion: 'v4' in config to use new sign version:
AWS.config.update({
signatureVersion: 'v4'
});
Works for JS sdk.
For people using boto3 (Python SDK) use the below code
from botocore.client import Config
s3 = boto3.resource(
's3',
aws_access_key_id='xxxxxx',
aws_secret_access_key='xxxxxx',
config=Config(signature_version='s3v4')
)
I have been using Django, and I had to add these extra config variables to make this work. (in addition to settings mentioned in https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html).
AWS_S3_REGION_NAME = "ap-south-1"
Or previous to boto3 version 1.4.4:
AWS_S3_REGION_NAME = "ap-south-1"
AWS_S3_SIGNATURE_VERSION = "s3v4"
Similar issue with the PHP SDK, this works:
$s3Client = S3Client::factory(array('key'=>YOUR_AWS_KEY, 'secret'=>YOUR_AWS_SECRET, 'signature' => 'v4', 'region'=>'eu-central-1'));
The important bit is the signature and the region
AWS_S3_REGION_NAME = "ap-south-1"
AWS_S3_SIGNATURE_VERSION = "s3v4"
this also saved my time after surfing for 24Hours..
Code for Flask (boto3)
Don't forget to import Config. Also If you have your own config class, then change its name.
from botocore.client import Config
s3 = boto3.client('s3',config=Config(signature_version='s3v4'),region_name=app.config["AWS_REGION"],aws_access_key_id=app.config['AWS_ACCESS_KEY'], aws_secret_access_key=app.config['AWS_SECRET_KEY'])
s3.upload_fileobj(file,app.config["AWS_BUCKET_NAME"],file.filename)
url = s3.generate_presigned_url('get_object', Params = {'Bucket':app.config["AWS_BUCKET_NAME"] , 'Key': file.filename}, ExpiresIn = 10000)
In Java I had to set a property
System.setProperty(SDKGlobalConfiguration.ENFORCE_S3_SIGV4_SYSTEM_PROPERTY, "true")
and add the region to the s3Client instance.
s3Client.setRegion(Region.getRegion(Regions.EU_CENTRAL_1))
With boto3, this is the code :
s3_client = boto3.resource('s3', region_name='eu-central-1')
or
s3_client = boto3.client('s3', region_name='eu-central-1')
For thumbor-aws, that used boto config, i needed to put this to the $AWS_CONFIG_FILE
[default]
aws_access_key_id = (your ID)
aws_secret_access_key = (your secret key)
s3 =
signature_version = s3
So anything that used boto directly without changes, this may be useful
Supernova answer for django/boto3/django-storages worked with me:
AWS_S3_REGION_NAME = "ap-south-1"
Or previous to boto3 version 1.4.4:
AWS_S3_REGION_NAME = "ap-south-1"
AWS_S3_SIGNATURE_VERSION = "s3v4"
just add them to your settings.py and change region code accordingly
you can check aws regions from:
enter link description here
For Android SDK, setEndpoint solves the problem, although it's been deprecated.
CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(
context, "identityPoolId", Regions.US_EAST_1);
AmazonS3 s3 = new AmazonS3Client(credentialsProvider);
s3.setEndpoint("s3.us-east-2.amazonaws.com");
Basically the error was because I was using old version of aws-sdk and I updated the version so this error occured.
in my case with node js i was using signatureVersion in parmas object like this :
const AWS_S3 = new AWS.S3({
params: {
Bucket: process.env.AWS_S3_BUCKET,
signatureVersion: 'v4',
region: process.env.AWS_S3_REGION
}
});
Then I put signature out of params object and worked like charm :
const AWS_S3 = new AWS.S3({
params: {
Bucket: process.env.AWS_S3_BUCKET,
region: process.env.AWS_S3_REGION
},
signatureVersion: 'v4'
});
Check your AWS S3 Bucket Region and Pass proper Region in Connection Request.
In My Senario I have set 'APSouth1' for Asia Pacific (Mumbai)
using (var client = new AmazonS3Client(awsAccessKeyId, awsSecretAccessKey, RegionEndpoint.APSouth1))
{
GetPreSignedUrlRequest request1 = new GetPreSignedUrlRequest
{
BucketName = bucketName,
Key = keyName,
Expires = DateTime.Now.AddMinutes(50),
};
urlString = client.GetPreSignedURL(request1);
}
In my case, the request type was wrong. I was using GET(dumb) It must be PUT.
Here is the function I used with Python
def uploadFileToS3(filePath, s3FileName):
s3 = boto3.client('s3',
endpoint_url=settings.BUCKET_ENDPOINT_URL,
aws_access_key_id=settings.BUCKET_ACCESS_KEY_ID,
aws_secret_access_key=settings.BUCKET_SECRET_KEY,
region_name=settings.BUCKET_REGION_NAME
)
try:
s3.upload_file(
filePath,
settings.BUCKET_NAME,
s3FileName
)
# remove file from local to free up space
os.remove(filePath)
return True
except Exception as e:
logger.error('uploadFileToS3#Error')
logger.error(e)
return False
Sometime the default version will not update. Add this command
AWS_S3_SIGNATURE_VERSION = "s3v4"
in settings.py
For Boto3 , use this code.
import boto3
from botocore.client import Config
s3 = boto3.resource('s3',
aws_access_key_id='xxxxxx',
aws_secret_access_key='xxxxxx',
region_name='us-south-1',
config=Config(signature_version='s3v4')
)
Try this combination.
const s3 = new AWS.S3({
endpoint: 's3-ap-south-1.amazonaws.com', // Bucket region
accessKeyId: 'A-----------------U',
secretAccessKey: 'k------ja----------------soGp',
Bucket: 'bucket_name',
useAccelerateEndpoint: true,
signatureVersion: 'v4',
region: 'ap-south-1' // Bucket region
});
I was stuck for 3 days and finally, after reading a ton of blogs and answers I was able to configure Amazon AWS S3 Bucket.
On the AWS Side
I am assuming you have already
Created an s3-bucket
Created a user in IAM
Steps
Configure CORS settings
you bucket > permissions > CORS configuration
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>```
Generate A bucket policy
your bucket > permissions > bucket policy
It should be similar to this one
{
"Version": "2012-10-17",
"Id": "Policy1602480700663",
"Statement": [
{
"Sid": "Stmt1602480694902",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::harshit-portfolio-bucket/*"
}
]
}
PS: Bucket policy should say `public` after this
Configure Access Control List
your bucket > permissions > acces control list
give public access
PS: Access Control List should say public after this
Unblock public Access
your bucket > permissions > Block Public Access
Edit and turn all options Off
**On a side note if you are working on django
add the following lines to you settings.py file of your project
**
#S3 BUCKETS CONFIG
AWS_ACCESS_KEY_ID = '****not to be shared*****'
AWS_SECRET_ACCESS_KEY = '*****not to be shared******'
AWS_STORAGE_BUCKET_NAME = 'your-bucket-name'
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
# look for files first in aws
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
# In India these settings work
AWS_S3_REGION_NAME = "ap-south-1"
AWS_S3_SIGNATURE_VERSION = "s3v4"
Also coming from: https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html
For me this was the solution:
AWS_S3_REGION_NAME = "eu-central-1"
AWS_S3_ADDRESSING_STYLE = 'virtual'
This needs to be added to settings.py in your Django project
Using PHP SDK Follow Below.
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
$client = S3Client::factory(
array(
'signature' => 'v4',
'region' => 'me-south-1',
'key' => YOUR_AWS_KEY,
'secret' => YOUR_AWS_SECRET
)
);
Nodejs
var aws = require("aws-sdk");
aws.config.update({
region: process.env.AWS_REGION,
secretAccessKey: process.env.AWS_S3_SECRET_ACCESS_KEY,
accessKeyId: process.env.AWS_S3_ACCESS_KEY_ID,
});
var s3 = new aws.S3({
signatureVersion: "v4",
});
let data = await s3.getSignedUrl("putObject", {
ContentType: mimeType, //image mime type from request
Bucket: "MybucketName",
Key: folder_name + "/" + uuidv4() + "." + mime.extension(mimeType),
Expires: 300,
});
console.log(data);
AWS S3 Bucket Permission Configuration
Deselect Block All Public Access
Add Below Policy
{
"Version":"2012-10-17",
"Statement":[{
"Sid":"PublicReadGetObject",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::MybucketName/*"
]
}
]
}
Then Paste the returned URL and make PUT request on the URL with binary file of image
Full working nodejs version:
const AWS = require('aws-sdk');
var s3 = new AWS.S3( {
endpoint: 's3.eu-west-2.amazonaws.com',
signatureVersion: 'v4',
region: 'eu-west-2'
} );
const getPreSignedUrl = async () => {
const params = {
Bucket: 'some-bucket-name/some-folder',
Key: 'some-filename.json',
Expires: 60 * 60 * 24 * 7
};
try {
const presignedUrl = await new Promise((resolve, reject) => {
s3.getSignedUrl('getObject', params, (err, url) => {
err ? reject(err) : resolve(url);
});
});
console.log(presignedUrl);
} catch (err) {
if (err) {
console.log(err);
}
}
};
getPreSignedUrl();
I'm using node-windows to set up my application to run as a Windows Service. I am using node-config to manage configuration settings. Of course, everything is working fine when I run my application manually using node app.js command. When I install it as a service and it starts, the configuration settings are empty. I have production.json file in ./config folder, and I can set NODE_ENV to production in the install script. I can confirm that the variable is set correctly and still nothing. log.info('CONFIG_DIR: ' + config.util.getEnv('CONFIG_DIR')); produces undefined even if I explicitly set it in env value for the service. Looking for any insight.
install script:
var Service = require('node-windows').Service;
var path = require('path');
// Create a new service object
var svc = new Service({
name:'Excel Data Import',
description: 'Excel Data Import Service.',
script: path.join(__dirname, "app.js"), // path application file
env:[
{name:"NODE_ENV", value:"production"},
{name:"CONFIG_DIR", value: "./config"},
{name:"$NODE_CONFIG_DIR", value: "./config"}
]
});
// Listen for the "install" event, which indicates the
// process is available as a service.
svc.on('install',function(){
svc.start();
});
svc.install();
app script:
var config = require('config');
var path = require('path');
var EventLogger = require('node-windows').EventLogger;
var log = new EventLogger('Excel Data Import');
init();
function init() {
log.info("init");
if(config.has("File.fileFolder")){
var pathConfig = config.get("File.fileFolder");
log.info(pathConfig);
var DirectoryWatcher = require('directory-watcher');
DirectoryWatcher.create(pathConfig, function (err, watcher) {
//...
});
}else{
log.info("config doesn't have File.fileFolder");
}
}
I know this response is very late, but also i had the same problem, and here is how i solved it :
var svc = new Service({
name:'ProcessName',
description: 'Process Description',
script: require('path').join(__dirname,'bin\\www'),
env:[
{name: "NODE_ENV", value: "development"},
{name: "PORT", value: PORT},
{name: "NODE_CONFIG_DIR", value: "c:\\route-to-your-proyect\\config"}
]
});
When you are using windows, prefixing your enviroment variables with $ , is not required.
Also, when your run script isn´t on the same dir as your config dir, you have to provide a full path to your config dir.
When you have errors with node-windows , is also helpful dig into the error log. It is located on rundirectory/daemon/processname.err.log
I hope this will help somebody.