This is my buildspec.yml:
phases:
install:
runtime-versions:
java: corretto11
pre_build:
commands:
- COMMIT_ID_SHORT=`echo "${CODEBUILD_RESOLVED_SOURCE_VERSION}" | cut -c1-8`
- TAG=`echo "${MAJOR}.${MINOR}.${COMMIT_ID_SHORT}"`
- aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin ${ECR_URL}
- export CODEARTIFACT_AUTH_TOKEN=`aws codeartifact get-authorization-token --domain cloud-infra-packages --domain-owner 715371302281 --query authorizationToken --output text`
build:
commands:
- gradle clean build
- docker build -t ${APP}:${TAG} -f Dockerfile
- docker tag ${APP}:${TAG} ${ECR_URL}/${ECR_URL}:${TAG}
But I got a error message:
the gradle version is not supported, and I found the gradle version of codebuild standard image is 5.6.0, so how to modify the gradle version when build the project in AWS codebuild.
already fixed the problem, we should use ./graldew to build the project, not gralde.
Related
I use Laravel Vapor for deploying our microservices based on Laravel. This works very good so far, if the app with their dependencies is not too large. But if it is then it gets a little bit tricky.
Vapor provides a Docker runtime for this case where you are able to deploy apps up to 10GB size.
For local development we usually use Laradock.io because its easy and flexible.
That means if we deploy from our local environment it easy to enter the workspace container and and run the vapor deploy commands. After enabling Docker Client for the workspace container it works with the vapor Docker runtime properly.
But now we integrated the deployment process into Gitlab CI Pipeline. That works very well for our small services with Vapor PHP runtime.
But for the Docker runtime I desperate on the CI deployment.
The docker runtime needs an installed docker instance where vapor will be invoked. That means in the Gitlab-ci.yml I have to add an image with installed Docker and PHP to invoke the Vapor scripts.
So I created an docker image base on the laradock workspace container but the Gitlab-runner exits always with the error message no docker deamon is available.
This is the related part of my GitLab-CI yml (the image is only local available):
testing:
image:
name: lexitaldev/vapor-docker-deploy:latest
pull_policy: never
securityContext:
privileged: true
environment: testing
stage: deploy
only:
- test
script:
- composer install
- php vendor/bin/vapor deploy test
This is the specific output:
Error Output:
================
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the
docker daemon running?
I've tried to use the standard 'laravelphp/vapor:php80' image and install docker over the script section as well.
before_script:
- apk add docker
- addgroup root docker
But nothing helped. It seems to be there is a problem with the docker.sock.
Did anybody managed to add Vapor Docker Runtime deployment to CI scripts?
Best,
Michael
I would like to tell you, that you only need to add the Service: dind, but after you do that, it will throw an error, related to the image that Gitlab create for your pipelines. So you need to create a runner with volumes, privileged flag, and tags.
I did it, using gitlab-runner on my machine.
sudo gitlab-runner register -n \
--url {{ your_url }} \
--registration-token {{your_token}} \
--executor docker \
--description "{{ Describe your runner }}" \
--docker-image "docker:20.10.12-alpine3.15" \
--docker-privileged \
--docker-volumes="/certs/client" \
--docker-volumes="cache" \
--docker-volumes="/var/run/docker.sock:/var/run/docker.sock"
--tag-list {{ a_tag_for_your_pipeline }}
Once you did that, you would need to use a docker stable version in your gitlab-ci.yml file. For some reason, it doesn't work when I was trying to use version 20 or latest
image: docker:stable
services:
- name: docker:stable:dind
before_script:
- echo $CI_JOB_TOKEN | docker login $CI_REGISTRY -u $CI_REGISTRY_USER --password-stdin
build:
tags:
- {{the tag you defined in your runner}}
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
script:
- echo $IMAGE_TAG
- docker build -t $CI_REGISTRY_IMAGE -f {{your Dockerfile}} .
- docker push $CI_REGISTRY_IMAGE
All the variables are previously defined in Gitlab, so don't worry, you can "copy & paste". Also, I added some advices that Gitlab mention on its documentation when you need to register your Docker container in Gitlab container.
Laravel vapor is completely developed on aws platform but not used aws code pipeline to deploy codes. Has anyone tried aws code pipeline to deploy vapor codes?
I can deploy ubuntu server and install required PHP extensions and run vapor deploy staging command in aws codedeploy. Wondering is there any better way to deploy laravel vapor.
Finally, I made the below changes in the buildspec.yml file and used the ubuntu instance.
install:
commands:
- nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2 &
- timeout 15 sh -c "until docker info; do echo .; sleep 1; done"
#- command
#- echo Logging in to Amazon ECR
pre_build:
commands:
- docker run --name myvapor -d -e VAPOR_API_TOKEN=MY_VAPOR_API_TOKEN --volume $(pwd):/~ --workdir /~ teamnovu/laravel-vapor
- docker exec myvapor vapor deploy staging
I am using Circle CI to deploy a microservice to a Digital Ocean droplet and had a few questions about whether my approach is the right one.
My microservice is built using docker-compose, and therefore requires a docker-compose.yml file to pull, start the images that constitute it.
In a nutshell, my deployment approach would be:
Merge branch to master will kick off a CircleCI build
CircleCI will run unit tests
Upon all tests passing, docker-compose build and docker-compose push to Docker Hub
Stop all running images of that service on remote server.
Remove dangling images, and local networks.
Download the relevant docker-compose.yml, Dockerfile and docker-compose.env files.
Pull using docker-compose pull
Start images using docker-compose up
I am using this configuration in CircleCI:
version: 2.1
jobs:
build:
docker:
- image: "circleci/node:10.16.0"
steps:
- checkout
- run:
name: Update to latest npm version
command: "sudo npm install -g npm#latest"
- restore_cache:
key: dependency-cache-{{ checksum "package-lock.json" }}
- run:
name: Install dependencies
command: npm install
- run:
name: Install `docker-compose`
command: |
curl -L https://github.com/docker/compose/releases/download/1.19.0/docker-compose-`uname -s`-`uname -m` > ~/docker-compose
chmod +x ~/docker-compose
sudo mv ~/docker-compose /usr/local/bin/docker-compose
- setup_remote_docker:
docker_layer_caching: false
- run:
name: Build using `docker-compose`
command: |
docker-compose build
- run:
name: Login for Docker Hub
command: |
echo "$DOCKER_PASSWORD" | docker login --username $DOCKER_USERNAME --password-stdin
- run:
name: Push to Docker Hub
command: |
docker-compose push
- run: ssh-keyscan $DIGITALOCEAN_HOST >> ~/.ssh/known_hosts
- add_ssh_keys:
fingerprints:
- fo:of:fe:ef:af
- run:
name: Remove currently running containers
command: |
ssh root#$DIGITALOCEAN_HOST ./deploy_image.sh
I am planning on creating a bash script to handle steps 4 to 8 from my list above.
Is it a good idea to have a script take care of the Docker steps?
Or is there a better way to have a more "native" CircleCI configuration?
I'm trying to deploy a java maven project on aws with Gitlab CI/CD.
This is my .gitlab-ci.yml
image: maven:3-jdk-8
services:
- docker:dind
stages:
- test
- build
- deploy
maven-test:
stage: test
script:
- echo "Test stage"
- mvn clean validate compile test -B
maven-build:
stage: build
script:
- echo "Build stage"
- mvn install -B -DskipTests
artifacts:
paths:
- ./target/*.jar
maven-deploy:
stage: deploy
script:
- echo "Deploy stage"
- scp -v -o StrictHostKeyChecking=no -I "mykey.pem" ./target/*.jar ubuntu#xxxxxxx.com:*.jar
when: manual
If I execute the scp command on a terminal in my pc then the jar is uploaded in aws ec2 instance while in gitlab I have errors and the jar is not uploaded.
This is my first approach with Gitlab CI and aws, so can someone explain step by step what I need to do to deploy the project in aws ec2 instance with Gitlab CI?
Thanks!
Since you have not posted much about your problem nor did you post the error I will just suggest a few things to look at:
From a GitLab perspective:
Are you sure that the "mykey.pem" is available within the repository when running that command(maven-deploy) on the the gitlab-runner.?
Also are you sure that you are using a docker gitlab-runner, if you are not then you can't use the image: directive and therefore it might not not have mvn/scp locally.
You might want to look into the dependencies directive and ensure you make that artifact available in next task. This should be done by default!
From an AWS perspective:
Make sure that the ubuntu target machine/server has port 22 exposed to the EC2 machine running the gitlab-runner.
Edit:
If the error you are receiving is with the pem files permissions then take a look at this resolution for AWS EC2 pem file issue. Another similar resolution is here.
Seems like if you put chmod 400 mykey.pem before the scp it might fix your problem.
My use case is the following: I want to deploy a PHP app to docker after a branch is updated and then leave it running for manual testing.
So I would like to access it under a certain URL.
What I did so far is to create a gitlab docker runner
gitlab-ci-multi-runner register --url "https://git.example.com/ci" --registration-token xxxx \
--description "dockertest" \
--executor docker \
--docker-image "php:7.0-apache" \
--docker-services mariadb:latest
And I have a .gitlabci.yml
stages:
- deploy
job-deploy-docker:
only:
- docker-ci-test
stage: deploy
environment: docker
tags:
- docker
image: php:7.0-apache
services:
- mariadb
script:
- apt-get update && apt-get --assume-yes install mysql-client
- mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mariadb -e "create database ci"
The deploy runs Job succeeded and then the container is destroyed.
How can I avoid destroying the container. Of course it should be cleaned up or reused if a new push is done to a branch.
How can I make the web app accessible under a certain external URL (branchname.ci.example.com)
Is that even the right approach or should I do it differently?