Docker fails to provide creds for awslogs logging driver - macos

Question

My docker-compose file:
version: "2"
services:
app:
build:
# Build an image from the Dockerfile in the current directory
context: .
ports:
- 5000:5000
environment:
PORT: 5000
NODE_ENV: production
And docker-compose.override
version: "2"
networks:
# This special network is configured so that the local metadata
# service can bind to the specific IP address that ECS uses
# in production
credentials_network:
driver: bridge
ipam:
config:
- subnet: "169.254.170.0/24"
gateway: 169.254.170.1
services:
# This container vends credentials to your containers
ecs-local-endpoints:
# The Amazon ECS Local Container Endpoints Docker Image
image: amazon/amazon-ecs-local-container-endpoints
volumes:
# Mount /var/run so we can access docker.sock and talk to Docker
- /var/run:/var/run
# Mount the shared configuration directory, used by the AWS CLI and AWS SDKs
# On Windows, this directory can be found at "%UserProfile%\.aws"
- $HOME/.aws/:/home/.aws/
environment:
# define the home folder; credentials will be read from $HOME/.aws
HOME: "/home"
# You can change which AWS CLI Profile is used
AWS_PROFILE: "default"
networks:
credentials_network:
# This special IP address is recognized by the AWS SDKs and AWS CLI
ipv4_address: "169.254.170.2"
# Here we reference the application container that we are testing
# You can test multiple containers at a time, simply duplicate this section
# and customize it for each container, and give it a unique IP in 'credentials_network'.
app:
logging:
driver: awslogs
options:
awslogs-region: eu-west-3
awslogs-group: sharingmonsterlog
depends_on:
- ecs-local-endpoints
networks:
credentials_network:
ipv4_address: "169.254.170.3"
environment:
AWS_DEFAULT_REGION: "eu-west-3"
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: "/creds"
I have added my AWS credentials to my mac using the aws configure command and the credentials are stored correctly in ~/.aws/credentials.
I am using docker 2.2.0.4, docker-compose 1.25.4 and docker-machine 0.16.2.
When I run docker-compose up I get the following error:
ERROR: for scraper Cannot start service scraper: Failed to initialize logging driver: NoCredentialProviders: no valid providers in chain.
Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors
ERROR: Encountered errors while bringing up the project.
I believe this is because I need to set the AWS credentials in the Docker Daemon but I cannot work out how this is done on macOs High Sierra.

Answer 1

We need to pass the credential to Docker daemon. On Systemd based Linux, as the docker daemon is managed by systemd, we need to setup systemd configuration for docker daemon.
Pass Credentials to the awslogs Docker Logging Driver on Ubuntu
For mac, we need to find a way to do the similar in Mac, and need to understand how Mac docker daemon is configured and started.
Apparently there is an issue with Mac docker daemon not being able to pass environment variables, so it would limit the options. The person posted the issue ended up with docker-cloudwatchlogs
Add ability to set environment variables for docker daemon
There are a few ways on Mac mentioned in the stackoverflow.
How do I provide credentials to the docker awslogs driver using Docker for Mac?
However, if this is about docker compose, there could be another way to pass AWS credential via environment variables using Docker Composer features.
Environment variables in Compose
Or simply setup environment variable when running docker compose command line.
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY= ... AWS_SESSION_TOKEN= ... docker-compose up ...
Please refer to Docker for Mac can't use loaded environment variable from file as well.
Regarding the AWS IAM permission, please make sure the AWS account of the AWS credential has the IAM permission as specified in the Docker document.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow",
"Resource": "*"
}
]
}