My folder structure looks like this
- root-dir
-- docker
-- src //contains laravel application
---.env
-- docker-compose.yml
As you might know in both laravel .env and docker-compose.yml files you need to specify the connection settings to db
// .env
DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=homestead
DB_USERNAME=homestead
DB_PASSWORD=secret
// docker-compose.yml
environment:
- MYSQL_ROOT_PASSWORD=secret
- MYSQL_DATABASE=homestead
- MYSQL_USER=homestead
- MYSQL_PASSWORD=secret
Is there a way where I can make docker-compose to "read" the settings from the .env file, since the last one is not tracked by git? so basically if I have to change settings I have to do it only in one file and also to not track the credentials on git for docker-compose.yml
You can do it like(From docker documentation https://docs.docker.com/compose/environment-variables/#the-env-file):
The “.env” file
You can set default values for any environment variables referenced in the Compose file, or used to configure Compose, in an environment file named .env:
$ cat .env
TAG=v1.5
$ cat docker-compose.yml
version: '3'
services:
web:
image: "webapp:${TAG}"
You can also use:
The “env_file” configuration option
You can pass multiple environment variables from an external file through to a service’s containers with the ‘env_file’ option, just like with docker run --env-file=FILE ...:
web:
env_file:
- web-variables.env
Related
Example
Consider this example docker-compose file with custom .env file:
version: '3'
services:
service_example:
build:
dockerfile: Dockerfile
context: .
args:
AAA: ${AAA}
command: python3 src/service/run.py
env_file:
- custom_env.env
custom_env.env:
AAA=qqq
When I run docker-compose config I get the following output:
WARNING: The AAA variable is not set. Defaulting to a blank string.
services:
service_example:
build:
args:
AAA: '' <----------------------------- ??????
context: /Users/examples
dockerfile: Dockerfile
command: python3 src/service/run.py
environment:
AAA: qqq
version: '3'
Question
Why AAA is unset in build section?
What should I do to set it properly (to the value provided from custom file: AAA=qqq)?
I've also noticed that if I change the env file name to the default setting mv custom_env.env .env and remove env_file section from docker-compose.yml - everything will be just fine:
services:
service_example:
build:
args:
AAA: qqq
context: /Users/examples
dockerfile: Dockerfile
command: python3 src/service/run.py
version: '3'
Quick Answer
docker-compose --env-file custom_env.env config
Answers Explanation
Question-1: Why AAA is unset in build section?
Because the env file specified in env_file property custom_env.env is specific for the Container only, i.e. those variables are to be passed to container while running not during image build.
Question-2: What should I do to set it properly (to the value provided from custom file: AAA=qqq)?
To provide environment variables for build step in docker-compose file using custom env file, we need to specify the custom file path. Like
Syntax: docker-compose --env-file FILE_PATH config
Example: docker-compose --env-file custom_env.env config
Question-3: How .env works?
Because that is the default file for which docker-compose looks for.
Summary
So, In docker-compose for current scenario we can consider 2 stages for specifying env
Build Stage(Image)
Running Stage(Container)
For Build stage - we can use .env default file or we can use --env-file option to specify custom env file
For Running Stage - we can specify environment variables using environment: property or we can use env_file: property to specify a env file
References
https://docs.docker.com/compose/env-file/
https://docs.docker.com/compose/environment-variables/
https://docs.docker.com/compose/compose-file/#env_file
https://docs.docker.com/compose/compose-file/#environment
enter image description here
Please help me. When gitlab CI instance run then error migrate of laravel , can't connect host mysql
enter image description here
First:
Use Docker with environment variables for deploy everyone :D
Second:
Make cut .env in CI script and show.
Your sed is not working right. It substitutes the variable name with the value, and not your data in the variable value.
Should be something like:
sed -i "s|DB_HOST=|DB_HOST=${DB_HOST}|g" .env
Third: Don't use .env.example for building .env. Build .env file from empty.
You didn't set up your MySQL properly. For services you have MySQL, but no port of it is exposed, NO MySQL root username, password, or database is set.
It should be something like...
services:
mysql:
image: mysql:5.7
env:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: laravel
ports:
- 33306:3306
Then use that in your project Environment, try this for a complete yml.
If you want SQLite instead of MySQL you can try this
I'm new to sre. i building an aws codepipeline using cdk. i need to pass the rds instance information from my rds stack to my codepipeline(ec2) stack. I need a .env file in my ec2 instances. based on my research i saw there is something called environment variables that can do it for me instead of generating a .env file from codebuild. i set up a few environment variables(plain text) in codebuild and try to pass those environment variables into the ec2 instances that was deployed from the codedeploy. i was able to get the correct environment variable values in buildspec.yml. but when i tried to run echo $DB_HOST in ec2 terminal. i got nothing. here is my set up:
codebuild environment variables:
buildspec.yml
version: 0.2
env:
exported-variables:
- DB_HOST
- DB_PORT
- DB_DATABASE
- DB_PASSWORD
- DB_USERNAME
phases:
install:
commands:
- echo $DB_HOST
- export DB_HOST=$DB_HOST
pre_build:
commands:
- export DB_HOST=$DB_HOST
artifacts:
files:
- '**/*'
name: myname-$(date +%Y-%m-%d)
my appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /var/www/html/
hooks:
BeforeInstall:
- location: script/BeforeInstall.sh
runas: root
AfterInstall:
- location: script/AfterInstall.sh
runas: root
AfterInstall.sh
#!/bin/bash
# Set permissions to storage and bootstrap cache
sudo chmod -R 0777 /var/www/html/storage
sudo chmod -R 0777 /var/www/html/bootstrap/cache
#
cd /var/www/html
#
# Run composer
composer install --ignore-platform-reqs
please help me to pass those environment variables from codebuild to codedeploy ec2. or is there any other way to generate .env file for codebuild?
You can't do this the way you expect it. The proper way is to pass them through SSM Secrets Manager or SSM Paramter Store.
So in your setup, CodeBuild will populate the SSM Secrets Manager or SSM Paramter Store (or you populate them before hand youself), and CodeDeploy will read these secret stores for the parameters.
I found a way to work around with it. here is my solution:
since I'm able to get all the environment variables in build stage. i manage to build a .env file in build stage. I have a few environment variables coming in to build stage from secret manager or as plain text.
first, i created a .env.exmaple file in my project root directory:
...
APP_ENV=local
APP_KEY=
...
DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=
DB_USERNAME=root
DB_PASSWORD=
MAIL_MAILER=smtp
MAIL_HOST=smtp.sendgrid.net
MAIL_PORT=587
MAIL_USERNAME=apikey
MAIL_PASSWORD=
...
second, i updated my buildspec.yml file and replace each value with environment variable values using sed commands
version: 0.2
env:
exported-variables:
- DB_HOST
- DB_DATABASE
- DB_PASSWORD
- DB_USERNAME
secrets-manager:
MAIL_PASSWORD: "email-token:MAIL_PASSWORD"
AWS_ACCESS_KEY_ID: "aws-token:AWS_ACCESS_KEY_ID"
AWS_SECRET_ACCESS_KEY: "aws-token:AWS_SECRET_ACCESS_KEY"
AWS_DEFAULT_REGION: "aws-token:AWS_DEFAULT_REGION"
AWS_BUCKET : "aws-token:AWS_BUCKET"
AWS_URL : "aws-token:AWS_URL"
phases:
build:
commands:
- cp .env.example .env
- sed -i "s/DB_HOST=127.0.0.1/DB_HOST=$DB_HOST/g" .env
- sed -i "s/DB_DATABASE=/DB_DATABASE=$DB_DATABASE/g" .env
- sed -i "s/DB_USERNAME=root/DB_USERNAME=$DB_USERNAME/g" .env
- sed -i "s/DB_PASSWORD=/DB_PASSWORD=$DB_PASSWORD/g" .env
- sed -i "s/APP_ENV=local/APP_ENV=$APP_ENV/g" .env
- sed -i "s/MAIL_PASSWORD=/MAIL_PASSWORD=$MAIL_PASSWORD/g" .env
...
- sed -i "s#AWS_URL=#AWS_URL=$AWS_URL#g" .env
artifacts:
files:
- '**/*'
name: myname-$(date +%Y-%m-%d)
in this way, i am able to create a .env file for the deploy stage.
one thing to notice here is that if your value contain / (for example url), you need to use # to instead of / for sed commands
I created a new Laravel project and installed Sail with composer require laravel/sail --dev followed by php artisan sail:install and sail up to get the project up and running in Docker.
By doing these actions my .env file changed from
DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=laravel_docker
DB_USERNAME=root
DB_PASSWORD=
to
DB_CONNECTION=pgsql
DB_HOST=pgsql
DB_PORT=5432
DB_DATABASE=laravel_docker
DB_USERNAME=sail
DB_PASSWORD=password
And now I got two running Docker containers:
laravel-docker_laravel.test_1
laravel-docker_pgsql_1
I'm able to run the basic user migration with sail artisan migrate.
Next up I want to connect Tableplus (or Postico) with my Postgresql database running in Docker. Therefore I filled in the following information:
When trying to connect I got ERROR FATAL: role "sail" does not exist.
Can someone help me out?
Edit 1: Adding a screenshot from some terminal commands. I can connect to the database in the docker container, see a list of the tables and get a table with all the rows from the users table (inserted with a seeder using Laravel Sail)
Edit 2: docker-compose ps
Thanks to this post I found a working solution.
Changing to a different port in the docker-compose.yml file fixed the issue.
Before:
pgsql:
image: 'postgres:13'
ports:
- '${FORWARD_DB_PORT:-5432}:5432'
After:
pgsql:
image: 'postgres:13'
ports:
- '${FORWARD_DB_PORT:-5632}:5432'
After changing the port number to 5632 in TablePlus I'm able to connect to the database.
I have the same problem "role "sail" does not exist". In this case we need to use command ./vendor/bin/sail down -v to delete the existing Docker volume data. After that you can do ./vendor/bin/sail up command.
I define a variable in an .env file:
LOCAL_IP=127.0.0.1
When using this in the following docker-compose.yml I can see that the variable is expanded to 127.0.0.1 by running docker-compose config.
docker-compose.yml
services:
my-service:
environment:
# This gets expanded to http://127.0.0.1/services
SERVICE_ENDPOINT: http://${LOCAL_IP}/services
Now I'd like to reference the value of LOCAL_IP in another variable in the same file like:
LOCAL_IP=127.0.0.1
SOME_OTHER_VAR=$LOCAL_IP
Updated docker-compose.yml
services:
my-service:
environment:
SERVICE_ENDPOINT: http://${SOME_OTHER_VAR}/services
Running docker-compose config then gives me:
services:
my-service:
environment:
# I'd have expected http://127.0.0.1/services
SERVICE_ENDPOINT: http://$$LOCAL_IP:8090/services