AWS | Laravel | ECS(G/B) | CodeDeploy unfinish - laravel

I'm using Laravel.
I want to deploy it to ECS (B/G) to see how it works.
In the development environment, Laravel is running.
I was able to launch my Laravel project on EC2 using docker.
I want to use Fargate for the first time and deploy to ECS!
Also, CodeBuild has completed successfully.
appspec.yml
version: 0.0 Resources:
- TargetService:
Type: AWS::ECS::Service
Properties:
TaskDefinition: "<TASK_DEFINITION>"
LoadBalancerInfo:
ContainerName: "nginx"
ContainerPort: "80"
taskdef.json
{
"taskRoleArn": "arn:aws:iam::**********:role/ecsTaskExecutionRole",
"executionRoleArn": "arn:aws:iam::**********:role/ecsTaskExecutionRole",
"containerDefinitions": [
{
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/****-system",
"awslogs-region": "******",
"awslogs-stream-prefix": "ecs"
}
},
"entryPoint": [
"sh",
"-c"
],
"command": [
"php artisan config:cache && php artisan migrate && chmod -R 777 storage/ && chmod -R 777 bootstrap/cache/"
],
"cpu": 0,
"environment": [
{
"name": "APP_ENV",
"value": "staging"
}
],
"workingDirectory": "/var/www/html",
"image": "<IMAGE1_NAME>",
"name": "php"
},
{
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/****-system",
"awslogs-region": "****",
"awslogs-stream-prefix": "ecs"
}
},
"portMappings": [
{
"hostPort": 80,
"protocol": "tcp",
"containerPort": 80
}
],
"environment": [
{
"name": "APP_ENV",
"value": "staging"
}
],
"workingDirectory": "/var/www/html",
"image": "**********.dkr.ecr.**********.amazonaws.com/**********-nginx:latest",
"name": "nginx"
}
],
"placementConstraints": [],
"memory": "2048",
"family": "*****-system",
"requiresCompatibilities": [
"FARGATE"
],
"networkMode": "awsvpc",
"cpu": "1024",
"volumes": []
}
CodeDeploy stopped at INSTALL, and there are no errors.
As you can see in the capture, we can confirm that "<TASK_DEFINITION>" has been replaced.
I'd like to know if there's any information I'm missing.
I'm not sure how to set environment variables such as ".env", so I'm thinking this might be the cause.
CodeDeploy Failed
Revision
Task Definitions
ECR
ECR nginx
ECR src(laravel)

If you want to change .env file to set env variable, you may use ssh connection to your webserver and run nano .env command at root folder, to write the file.
You can also modify the file using ftp connection.

Related

Bash redirection in docker container failing when ran in ECS task on Amazon Linux 2 instances

I am trying to run an ECS task that contains 3 containers - postgres, redis, and an image from a private ECR repository. The custom image container definition has a command to wait until the postgres container can receive traffic via a bash command
"command": [
"/bin/bash",
"-c",
"while !</dev/tcp/postgres/5432; do echo \"Waiting for postgres database to start...\"; /bin/sleep 1; done; /bin/sh /app/start-server.sh;"
],
When I run this via docker-compose on my local machine through docker it works, but on the Amazon Linux 2 EC2 machine this is printed when the while loop runs:
/bin/bash: line 1: postgres: Name or service not known
/bin/bash: line 1: /dev/tcp/postgres/5432: Invalid argument
The postgres container runs without error and the last log from that container is
database system is ready to accept connections
I am not sure if this is a docker network issue or an issue with amazon linux 2's bash not being compiled with --enable-net-redirections which I found explained here
Task Definition:
{
"networkMode": "bridge",
"containerDefinitions": [
{
"environment": [
{
"name": "POSTGRES_DB",
"value": "metadeploy"
},
{
"name": "POSTGRES_USER",
"value": "<redacted>"
},
{
"name": "POSTGRES_PASSWORD",
"value": "<redacted>"
}
],
"essential": true,
"image": "postgres:12.9",
"mountPoints": [],
"name": "postgres",
"memory": 1024,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "metadeploy-postgres",
"awslogs-region": "us-east-1",
"awslogs-create-group": "true",
"awslogs-stream-prefix": "mdp"
}
}
},
{
"essential": true,
"image": "redis:6.2",
"name": "redis",
"memory": 1024
},
{
"command": [
"/bin/bash",
"-c",
"while !</dev/tcp/postgres/5432; do echo \"Waiting for postgres database to start...\"; /bin/sleep 1; done; /bin/sh /app/start-server.sh;"
],
"environment": [
{
"name": "DJANGO_SETTINGS_MODULE",
"value": "config.settings.local"
},
{
"name": "DATABASE_URL",
"value": "<redacted-postgres-url>"
},
{
"name": "REDIS_URL",
"value": "redis://redis:6379"
},
{
"name": "REDIS_HOST",
"value": "redis"
}
],
"essential": true,
"image": "the private ecr image uri built from here https://github.com/SFDO-Tooling/MetaDeploy",
"links": [
"redis"
],
"mountPoints": [
{
"containerPath": "/app/node_modules",
"sourceVolume": "AppNode_Modules"
}
],
"name": "web",
"portMappings": [
{
"containerPort": 8080,
"hostPort": 8080
},
{
"containerPort": 8000,
"hostPort": 8000
},
{
"containerPort": 6006,
"hostPort": 6006
}
],
"memory": 1024,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "metadeploy-web",
"awslogs-region": "us-east-1",
"awslogs-create-group": "true",
"awslogs-stream-prefix": "mdw"
}
}
}
],
"family": "MetaDeploy",
"volumes": [
{
"host": {
"sourcePath": "/app/node_modules"
},
"name": "AppNode_Modules"
}
]
}
The corresponding docker-compose.yml contains:
version: '3'
services:
postgres:
environment:
POSTGRES_DB: metadeploy
POSTGRES_USER: postgres
POSTGRES_PASSWORD: sample_db_password
volumes:
- ./postgres:/var/lib/postgresql/data:delegated
image: postgres:12.9
restart: always
redis:
image: redis:6.2
web:
build:
context: .
dockerfile: Dockerfile
command: |
/bin/bash -c 'while !</dev/tcp/postgres/5432; do echo "Waiting for postgres database to start..."; /bin/sleep 1; done; \
/bin/sh /app/start-server.sh;'
ports:
- '8080:8080'
- '8000:8000'
# Storybook server
- '6006:6006'
stdin_open: true
tty: true
depends_on:
- postgres
- redis
links:
- redis
environment:
DJANGO_SETTINGS_MODULE: config.settings.local
DATABASE_URL: postgres://postgres:sample_db_password#postgres:5432/metadeploy
REDIS_URL: redis://redis:6379
REDIS_HOST: redis
volumes:
- .:/app:cached
- /app/node_modules
Do I need to recompile bash to use --enable-net-redirections, and if so how can I do that?
Without bash's net redirection feature, your best bet is to use something like nc or netcat (if available) to determine if the port is open. If those aren't available, it may be worth modifying your app logic to better handle database failure cases.
Alternately, a potential better approach would be:
Adding a healthcheck to the postgres image.
Modifying the web service's depends_on clause "long syntax" to add a dependency on postgres being service_healthy instead of the default service_started.
This approach has two key benefits:
The postgres image likely has the tools to detect if the database is up and running.
The web service no longer needs to manually check if the database is ready or not.

Packer provisioners don't save installed packages

I have encountered an issue during provisioning with HashiCorp Packer for virtualbox-iso on Alpine Linux v3.16.
Provisioning script runs OK, and it logs that build has finished, however when I open the outputted ovf file in VirtualBox moved files and docker are not present.
I would be grateful for any advice.
I run packer build packer-virtualbox-alpine-governator.json
packer-virtualbox-alpine-governator.json file:
{
"variables": {
"password": "packer"
},
"builders": [
{
"type": "virtualbox-iso",
"memory": 8192,
"guest_os_type": "Other_64",
"iso_url": "https://dl-cdn.alpinelinux.org/alpine/v3.16/releases/x86_64/alpine-standard-3.16.0-x86_64.iso",
"iso_checksum": "file:https://dl-cdn.alpinelinux.org/alpine/v3.16/releases/x86_64/alpine-standard-3.16.0-x86_64.iso.sha256",
"ssh_username": "root",
"ssh_password": "{{user `password`}}",
"shutdown_command": "poweroff",
"hard_drive_interface": "sata",
"boot_command": [
"root<enter><wait>",
"setup-alpine<enter><wait>us<enter><wait>us<enter><wait><enter><wait><enter><wait><enter><wait><enter><wait5>{{user `password`}}<enter><wait>{{user `password`}}<enter><wait><enter><wait><enter><wait><enter><wait15><enter><wait>openssh<enter><wait>openssh-full<enter><wait5>test123<enter><wait5>test123<enter><wait><enter><wait><enter><wait>sda<enter><wait>sys<enter><wait>y<enter><wait30>",
"echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config<enter><wait>",
"/etc/init.d/sshd restart<enter><wait5>"
]
}
],
"provisioners": [
{
"type": "shell",
"inline": ["mkdir -p /opt/site/governator"]
},
{
"type": "file",
"source": "files/docker-compose.yaml",
"destination": "/opt/site/"
},
{
"type": "file",
"source": "files/governator.conf",
"destination": "/opt/site/governator/"
},
{
"type": "shell",
"scripts": [
"scripts/alpine/install-docker-on-alpine.sh"
]
}
]
}
./scritps/alpine/install-docker-on-alpine.sh
#! /bin/ash
cat > /etc/apk/repositories << EOF; $(echo)
https://dl-cdn.alpinelinux.org/alpine/v$(cut -d'.' -f1,2 /etc/alpine-release)/main/
https://dl-cdn.alpinelinux.org/alpine/v$(cut -d'.' -f1,2 /etc/alpine-release)/community/
https://dl-cdn.alpinelinux.org/alpine/edge/testing/
EOF
apk update
apk add docker
addgroup $USER docker
rc-update add docker boot
service docker start
apk add docker-compose
sync

what is the password for accessing apps that binded with config service in pivotal cloud foundry

I am very new to spring boot in pivotal cloud foundry. I am able to deploy the apps and bind them to config-server. When i access the app, it is asking for login details. I tried with pivotal credentials it didn't worked. I am struggling from days to find the credentials for this. Can someone help where can i find the username and password.
I have checked VCAP_SERVICES env variable but no luck. Below is my VCAP_SERVICES
{
"VCAP_SERVICES": {
"p-config-server": [
{
"binding_name": null,
"credentials": {
"access_token_uri": "https://p-spring-cloud-services.uaa.run.pivotal.io/oau
th/token",
"client_id": "p-config-server-066a4553-a934-4d52-b016-30e2ab7e6d90",
"client_secret": "rHnPwJj613pP",
"uri": "https://config-da1be911-4c82-41a3-a3bd-0b06ea295480.cfapps.io"
},
"instance_name": "config-server",
"label": "p-config-server",
"name": "config-server",
"plan": "trial",
"provider": null,
"syslog_drain_url": null,
"tags": [
"configuration",
"spring-cloud"
],
"volume_mounts": []
}
],
"p-service-registry": [
{
"binding_name": null,
"credentials": {
"access_token_uri": "https://p-spring-cloud-services.uaa.run.pivotal.io/oau
th/token",
"client_id": "p-service-registry-ae9f69cf-258d-45ea-821e-1fa4db37f1a3",
"client_secret": "XJOLKHZqwssd",
"uri": "https://eureka-89a144f4-7dc7-4b6d-8152-c4414b748ec0.cfapps.io"
},
"instance_name": "service-registry",
"label": "p-service-registry",
"name": "service-registry",
"plan": "trial",
"provider": null,
"syslog_drain_url": null,
"tags": [
"eureka",
"discovery",
"registry",
"spring-cloud"
],
"volume_mounts": []
}
]
}
}
{
"VCAP_APPLICATION": {
"application_id": "1b398fd8-61f2-47e3-875d-d3fe7574513a",
"application_name": "customer-service",
"application_uris": [
"customer-service-demo.cfapps.io"
],
"application_version": "d0243f11-b7af-4bb3-b3a3-b7a4c95e6298",
"cf_api": "https://api.run.pivotal.io",
"limits": {
"disk": 1024,
"fds": 16384,
"mem": 1024
},
"name": "customer-service",
"organization_id": "8e425cae-75a0-433a-a8a7-79c4f81d7c85",
"organization_name": "MSA_springboot",
"process_id": "1b398fd8-61f2-47e3-875d-d3fe7574513a",
"process_type": "web",
"space_id": "8902b90c-9964-4f9d-b426-ca1635589ebe",
"space_name": "development",
"uris": [
"customer-service-demo.cfapps.io"
],
"users": null,
"version": "d0243f11-b7af-4bb3-b3a3-b7a4c95e6298"
}
}
User-Provided:
SPRING_PROFILES_ACTIVE: dev
No running env variables have been set
No staging env variables have been set```[enter image description here][1]
[1]: https://i.stack.imgur.com/9kNWC.png

Mesos/Marathon - reserved resources for role are not offered for Marathon app

I have assigned slave resources to the particular role ("app-role") by set --default_role="app-role" parameter to ExecStart for slave service ( /etc/systemd/system/dcos-mesos-slave.service). Next I have restarted slave agent:
sudo systemctl daemon-reload
sudo systemctl stop dcos-mesos-slave.service
sudo rm -f /var/lib/mesos/slave/meta/slaves/latest
sudo systemctl start dcos-mesos-slave.service
and verified by: curl master.mesos/mesos/slaves.
After that I expect marathon app with acceptedResourceRoles attribute will receive only these particular resource offers, but it does not happen (the app is still in waiting state).
Why does marathon didn't receive it? How should this be done to make it work?
{
"id": "/basic-4",
"cmd": "python3 -m http.server 8080",
"cpus": 0.5,
"mem": 32,
"disk": 0,
"instances": 1,
"acceptedResourceRoles": [
"app-role"
],
"container": {
"type": "DOCKER",
"volumes": [],
"docker": {
"image": "python:3",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 8080,
"hostPort": 0,
"servicePort": 10000,
"protocol": "tcp",
"name": "my-vip",
"labels": {
"VIP_0": "/my-service:5555"
}
}
],
"privileged": false,
"parameters": [],
"forcePullImage": false
}
},
"portDefinitions": [
{
"port": 10000,
"protocol": "tcp",
"name": "default",
"labels": {}
}
]
}
This works only if marathon is started with --mesos_role set.
In the context of the question this should be: --mesos_role 'app-role'.
If you set --mesos_role other, Marathon will register with Mesos for this role – it will receive offers for resources that are reserved
for this role, in addition to unreserved resources.
If you set default_accepted_resource_roles *, Marathon will apply this default to all AppDefinitions that do not explicitly
define acceptedResourceRoles. Since your AppDefinition defines that
option, the default will not be applied (both are equal anyways).
If you set "acceptedResourceRoles": [""] in an AppDefinition (or the AppDefinition inherits a default of ""), Marathon will only
consider unreserved resources for launching of this app.
More: https://mesosphere.github.io/marathon/docs/recipes.html

How to mount HDFS in a Docker container

I made an application Dockerized in a Docker container. I intended to make the application able to access files from our HDFS. The Docker image is to be deployed on the same cluster where we have HDFS installed via Marathon-Mesos.
Below is the json to be POST to Marathon. It seems that my app is able to read and write files in the HDFS. Can someone comment on the safety of this? Would files changed by my app correctly changed in the HDFS as well? I Googled around and didn't find any similar approaches...
{
"id": "/ipython-test",
"cmd": null,
"cpus": 1,
"mem": 1024,
"disk": 0,
"instances": 1,
"container": {
"type": "DOCKER",
"volumes": [
{
"containerPath": "/home",
"hostPath": "/hadoop/hdfs-mount",
"mode": "RW"
}
],
"docker": {
"image": "my/image",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 8888,
"hostPort": 0,
"servicePort": 10061,
"protocol": "tcp",
}
],
"privileged": false,
"parameters": [],
"forcePullImage": true
}
},
"portDefinitions": [
{
"port": 10061,
"protocol": "tcp",
"labels": {}
}
]
}
You might have a look at the Docker volume docs.
Basically, the volumes definition in the app.json would trigger the start of the Docker image with the flag -v /hadoop/hdfs-mount:/home:RW, meaning that the host path gets mapped to the Docker container as /home in read-write mode.
You should be able to verify this if you SSH into the node which is running the app and do a docker inspect <containerId>.
See also
https://mesosphere.github.io/marathon/docs/native-docker.html

Resources