GitLab Shared runner, deploy spring boot microservice app to custom server - spring-boot

Before i start, let me tell you that i'm newbie to Gitlab CI file :)
i'm looking to automate deployments of spring boot microservice app to custom server (a namecheap VPS).
(I'm using the Gitlab shared runner)
i have used jHipster ci-cd to generate the .gitlab-ci.yml file. Doing that, i have: build, package, release stages working.
i even could see the image repository built in Gitlab Container registery.
What last, is the deployment. i know that i have to use a docker container to deploy it, but i don't know how.
I'm stuck in deploying the image repository to my VPS.
(as it's my first microservice, my VPS is still new, having only java installed).
Here is my .gitlab-ci.yml file:
#image: jhipster/jhipster:v6.9.0
image: openjdk:11-jdk
cache:
key: "$CI_COMMIT_REF_NAME"
paths:
- .maven/
stages:
- check
- build
# - test
# - analyze
- package
- release
- deploy
before_script:
- chmod +x mvnw
- git update-index --chmod=+x mvnw
- export NG_CLI_ANALYTICS="false"
- export MAVEN_USER_HOME=`pwd`/.maven
nohttp:
stage: check
script:
- ./mvnw -ntp checkstyle:check -Dmaven.repo.local=$MAVEN_USER_HOME
maven-compile:
stage: build
script:
- ./mvnw -ntp compile -P-webpack -Dmaven.repo.local=$MAVEN_USER_HOME
artifacts:
paths:
- target/classes/
- target/generated-sources/
expire_in: 1 day
#maven-test:
# stage: test
# script:
# - ./mvnw -ntp verify -P-webpack -Dmaven.repo.local=$MAVEN_USER_HOME
# artifacts:
# reports:
# junit: target/test-results/**/TEST-*.xml
# paths:
# - target/test-results
# - target/jacoco
# expire_in: 1 day
maven-package:
stage: package
script:
- ./mvnw -ntp verify -Pprod -DskipTests -Dmaven.repo.local=$MAVEN_USER_HOME
artifacts:
paths:
- target/*.jar
- target/classes
expire_in: 1 day
# Uncomment the following line to use gitlabs container registry. You need to adapt the REGISTRY_URL in case you are not using gitlab.com
docker-push:
stage: release
variables:
REGISTRY_URL: registry.gitlab.com
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG-$CI_COMMIT_SHA
dependencies:
- maven-package
script:
- ./mvnw -ntp compile jib:build -Pprod -Djib.to.image=$IMAGE_TAG -Djib.to.auth.username=gitlab-ci-token -Djib.to.auth.password=$CI_BUILD_TOKEN -Dmaven.repo.local=$MAVEN_USER_HOME
docker-deploy:
image: docker:stable-git
stage: deploy
script:
when: manual
only:
- master
Thanks for your help :)

Related

Gitlab pipeline error With CD/CI for AWS ec2 debian instance: This job is stuck because you don't have any active runners online

I want to create a CI/CD pipeline between gitlab and aws ec2 deployment.
My repository is nodejs/express web server project.
And I created a gitlab-ci.yaml
image: node:latest
cache:
paths:
- node_modules/
stages:
- build
- test
- staging
- openMr
- production
before_script:
- apt-get update -qq && apt-get install
Build:
stage: build
tags:
- node
before_script:
- yarn config set cache-folder .yarn
- yarn install
script:
- npm run build
Test:
stage: test
tags:
- node
before_script:
- yarn config set cache-folder .yarn
- yarn install --frozen-lockfile
script:
- npm run test
Deploy to Production:
stage: production
tags:
- node
before_script:
- mkdir -p ~/.ssh
- echo -e "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- bash ./gitlab-deploy/.gitlab-deploy.prod.sh
environment:
name: production
url: http://ec2-url.compute.amazonaws.com:81
When I push a new commit pipeline failed on build step. And I get a warning as :
This job is stuck because you don't have any active runners online or
available with any of these tags assigned to them: node
I checked my runner on gitlab settings/CI/CD
After that I checkked server
admin#ip-111.222.222.111:~$ gitlab-runner
statusRuntime platform arch=amd64 os=linux pid=18787 revision=98daeee0 version=14.7.0
FATAL: The --user is not supported for non-root users
You need to remove the tag node from your jobs. Runner tags are used to define which runner should pick up your jobs (https://docs.gitlab.com/ee/ci/runners/configure_runners.html#use-tags-to-control-which-jobs-a-runner-can-run). As there is no runner available which supports the tag node, your job gets stuck.
It doesn't look like your pipeline has any special requirements so you can just remove the tag so it can be picked up by every runner.
The runner that can be seen in your screenshot supports the tag shop_service_runner. So another option would be to change the tag node to shop_service_runner which would lead to this runner (and every runner with the same tags) being able to pick up this job.

Travis doesn't deploy on heroku. Invalid credentials

I'm trying to set up automatic deploy on heroku with travis. I get this error when travis try to deploy:
API request failed.
Message: Invalid credentials provided.
Reference:
failed to deploy
This is my travis file:
jobs:
include:
- language: python
python:
- "3.6"
install:
- pip install -r Deployment/requirements.txt
script:
- python -c "print ('Testing some script')"
branches:
only:
- master
- develop
before_deploy:
- cd Deployment
deploy:
- provider: heroku
skip_cleanup: true
api_key:
secure: b3AVdCtJ2e/+Gu1...
app:
master: motorent-deploy
develop: motorent-apitest
- language: android
dist: trusty
env:
global:
- ANDROID_API_LEVEL=29
- ANDROID_BUILD_TOOLS_VERSION=29.0.3
- extra-google-google_play_services
- extra-google-m2repository
- extra-android-m2repository
- addon-google_apis-google-$ANDROID_API_LEVEL
android:
licenses:
- 'android-sdk-preview-license-.+'
- 'android-sdk-license-.+'
- 'google-gdk-license-.+'
components:
- tools
- platform-tools
- android-$ANDROID_API_LEVEL
- build-tools-$ANDROID_BUILD_TOOLS_VERSION
- extra-google-google_play_services
- extra-google-m2repository
- extra-android-m2repository
- addon-google_apis-google-$ANDROID_API_LEVEL
addons:
apt:
packages:
ant
before_install:
- touch $HOME/.android/repositories.cfg
- yes | sdkmanager "platforms;android-29"
- yes | sdkmanager "build-tools;29.0.3"
before_script:
- cd AndroidApp
- chmod +x gradlew
script:
- ./gradlew build check
As you can see I have two differents projects in the same repository, but it's not important, because the android test works well. What doesn't work is the deploy of Flask project. The solutions that I have found talk about the need to encrypt the api_key. I have tested it with Travis encrypt $(heroku auth:token) but it doesn't work either.
I've been trying to find the error for a long time but I don't know what it can be.
I had the same error.
Here are steps that I performed to fix it.
Firstly I tried the command: heroku auth:token
but the output was:
› Warning: token will expire 06/06/2021
› Use heroku authorizations:create to generate a long-term token
Then I tried the command: heroku authorizations:create
One line from the output contained Token: <created_heroku_auth_token>
I took the value of it (<created_heroku_auth_token>)
and I went to
https://travis-ci.org/github/<my_github_user>/<my_repo>/settings
where I created new environment variable:
HEROKU_AUTH_TOKEN with value of my <created_heroku_auth_token>
Then in my .travis.yml I changed value of api_key to:
api_key: $HEROKU_AUTH_TOKEN
After pushing this change, the deployment to heroku went fine.

Caching on gitlab-ci isn't working when using Kubernetes cluster

When using gitlab-ci on a java+maven project; the maven artifacts are not cached. They are getting downloaded every-time.
I have deployed the gitlab-runner as kubernetes.
And the build time logs show
Creating cache edu-erp...
.m2/repository/: found 761 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
.gitlab-ci.yml
image: maven:latest
variables:
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version"
MAVEN_OPTS: "-Djava.awt.headless=true -Dmaven.repo.local=./.m2/repository"
cache:
paths:
- ./.m2/repository
# keep cache across branch
key: "$CI_COMMIT_REF_NAME"
stages:
- build
- test
build:
stage: build
cache:
key: edu-erp
paths:
- .m2/repository/
script:
- "mvn clean compile $MAVEN_CLI_OPTS"
artifacts:
paths:
- target/
test:
stage: test
cache:
key: edu-erp
script:
- "mvn test $MAVEN_CLI_OPTS"
The cache should be available across builds.

Unable to deploy spring-boot app in Heroku using bitbucket pipelines

I'm trying to use bitbucket pipelines to do deploys in heroku and I created a simple web application using spring-boot.
I'm wondering if any of you have ideas about what is need it in "bitbucket-pipelines.yml" file? This is my first time using heroku with bitbucket pipeline and I'm very lost.
My file looks like this one:
image: maven:3.3.9
clone:
depth: full
pipelines:
default:
- step:
script: # Modify the commands below to build your repository.
- mvn -B clean install -P heroku # -B batch mode makes Maven less verbose
- git push https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP_NAME.git HEAD
I would recommend the following steps:
Follow Step 1 mentioned in Deploy to Heroku in Bitbucket Pipelines' documentation.
Please edit your "bitbucket-pipelines.yml" file with the following content:
image: maven:3.3.9
clone:
depth: full
pipelines:
default:
- step:
script:
- git push https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP_NAME.git HEAD
- mvn clean package
- kill -9 $(lsof -t -i:<your_app_port_number> -sTCP:LISTEN)
- java -jar target/<your-app-name>.jar &
NOTE:
Replace < your_app_port_number > and < your-app-name > with appropriate values.
Remember to check your "bitbucket-pipelines.yml" file with the online validator.
Here is my bitbucket-pipelines.yml where spring profile is specified.
image: maven:3.3.9
clone:
depth: full
pipelines:
default:
- step:
name: Deploy to Heroku
deployment: test # set to test, staging or production
script:
- mvn -B clean package -Dspring.profiles.active=prod # -B batch mode makes Maven less verbose
- git push https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP_NAME.git HEAD
I have a Profile with:
web: java -Dserver.port=$PORT $JAVA_OPTS -Dspring.profiles.active=prod -jar target/my-api-1.0.0-SNAPSHOT.jar
Make sure you have configured $HEROKU_API_KEY and $HEROKU_APP_NAME on BitBucket like:

GitLab Runner cache for Gradle is not working

I'm using GitLab Runner as CI to build an Android project however the cache is not working.
Here's my .gitlab-ci.yml. It was modified from https://gist.github.com/daicham/5ac8461b8b49385244aa0977638c3420.
image: runmymind/docker-android-sdk:latest
variables:
GRADLE_USER_HOME: $CI_PROJECT_DIR/.gradle
stages:
- build
debug:
stage: build
script:
- set +e
- du -sh $CI_PROJECT_DIR/.gradle/wrapper
- du -sh $CI_PROJECT_DIR/.gradle/caches
- set -e
- ./gradlew assembleDebug
- mkdir artifacts
- cp mobile/build/outputs/apk/*.apk artifacts/
- cp wear/build/outputs/apk/*.apk artifacts/
cache:
paths:
- .gradle/wrapper/
- .gradle/caches/
- build/
- mobile/build/
- wear/build/
artifacts:
name: "project_${CI_JOB_NAME}_${CI_COMMIT_REF_NAME}_${CI_COMMIT_SHA}"
expire_in: 2 weeks
paths:
- artifacts/
And the log:
Running with gitlab-ci-multi-runner 9.0.0 (08a9e6f)
Using Docker executor with image runmymind/docker-android-sdk:latest ...
Using docker image sha256:d696fa13188c8d2d121c86cf526201b363c1e34ee7b163d6ce1ab1718f91a5e6 ID=sha256:d696fa13188c8d2d121c86cf526201b363c1e34ee7b163d6ce1ab1718f91a5e6 for predefined container...
Pulling docker image runmymind/docker-android-sdk:latest ...
Using docker image runmymind/docker-android-sdk:latest ID=sha256:474ac98077a496f2f71aa22ce4eebcea966c2960a061d4a59babe81ff007009b for build container...
Running on runner-8ce5d03c-project-72-concurrent-0 via outrage...
Cloning repository...
Cloning into '/builds/User/android-project'...
Checking out 015d01d0 as master...
Skipping Git submodules setup
Checking cache for default...
Successfully extracted cache
$ set +e
$ du -sh $CI_PROJECT_DIR/.gradle/wrapper
du: cannot access '/builds/User/android-project/.gradle/wrapper': No such file or directory
$ du -sh $CI_PROJECT_DIR/.gradle/caches
du: cannot access '/builds/User/android-project/.gradle/caches': No such file or directory
$ set -e
$ ./gradlew assembleDebug
Downloading https://services.gradle.org/distributions/gradle-3.4.1-all.zip
Unzipping /builds/User/android-project/.gradle/wrapper/dists/gradle-3.4.1-all/c3ib5obfnqr0no9szq6qc17do/gradle-3.4.1-all.zip to /builds/User/android-project/.gradle/wrapper/dists/gradle-3.4.1-all/c3ib5obfnqr0no9szq6qc17do
Set executable permissions for: /builds/User/android-project/.gradle/wrapper/dists/gradle-3.4.1-all/c3ib5obfnqr0no9szq6qc17do/gradle-3.4.1/bin/gradle
Starting a Gradle Daemon (subsequent builds will be faster)
Download https://jcenter.bintray.com/com/android/tools/build/gradle/2.3.0/gradle-2.3.0.pom
(more downloads)
I have also tried using gradle argument to set gradle user home, aggressively specifying .gradle/ for cache, etc, but none of them worked.
Any ideas?
If you use Gitlab < 9.0,you add to specify the cache has to be shared among different pipelines.
Try to add key: $CI_PROJECT_NAME under cache:
image: runmymind/docker-android-sdk:latest
variables:
GRADLE_USER_HOME: $CI_PROJECT_DIR/.gradle
stages:
- build
debug:
stage: build
script:
- set +e
- du -sh $CI_PROJECT_DIR/.gradle/wrapper
- du -sh $CI_PROJECT_DIR/.gradle/caches
- set -e
- ./gradlew assembleDebug
- mkdir artifacts
- cp mobile/build/outputs/apk/*.apk artifacts/
- cp wear/build/outputs/apk/*.apk artifacts/
cache:
key: $CI_PROJECT_NAME
paths:
- .gradle/wrapper/
- .gradle/caches/
- build/
- mobile/build/
- wear/build/
artifacts:
name: "project_${CI_JOB_NAME}_${CI_COMMIT_REF_NAME}_${CI_COMMIT_SHA}"
expire_in: 2 weeks
paths:
- artifacts/

Resources