I am trying to create a pipeline for an existing application. It is a React/Java Spring Boot application. It usually gets bundled into a single war file and uploaded to ElasticBeanstalk. I created my codebuild project and when I run it manually it will generate a war file that I can then upload to ElasticBeanstalk and everything works correctly. The buildspec for that is below:
version: 0.2
phases:
install:
commands:
- echo Nothing to do in the install phase...
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- mvn -Pprod package -X
post_build:
commands:
- echo Build completed on `date`
- mv target/cogcincinnati-0.0.1-SNAPSHOT.war cogcincinnati-0.0.1-SNAPSHOT.war
artifacts:
files:
- cogcincinnati-0.0.1-SNAPSHOT.war
When I run this build step in my pipeline it generates a zip file that gets dropped onto S3. My deploy step takes that build artifact and sends it to ElasticBeanstalk. Elasticbeanstalk does not give me any errors, but when I navigate to my url, I get a 404.
I have tried uploading the zip directly to Elasticbeanstalk and I get the same result. I have unzipped the file and it does appear to have all of my project files.
When I look at the server logs, I do not see any errors. I don't understanding why codebuild appears to be generating a war file when I run it manually, but a zip when executed in code pipeline.
Change artifacts war file name to ROOT.war this will resolve your problem actually your application is deployed successfully but on a different path, this is tomcat inbuild functionality by changing the ROOT it will run the application on '/'
So updated buildspec.yml will be
version: 0.2
phases:
install:
commands:
- echo Nothing to do in the install phase...
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- mvn -Pprod package -X
post_build:
commands:
- echo Build completed on `date`
- mv target/cogcincinnati-0.0.1-SNAPSHOT.war ROOT.war
artifacts:
files:
- ROOT.war
Seems your application is failing, you should review the logs from the Beanstalk environment, specially:
"tomcat8/catalina.out"
"tomcat8/catalina.[date].log"
[1] Viewing logs from Amazon EC2 instances in your Elastic Beanstalk environment - https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.logging.html
For more details about using Tomcat platform on EB environment, you can refer to this document:
- Using the Elastic Beanstalk Tomcat platform - https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-tomcat-platform.html
About the folder structuring in your project, please refer to this document:
- Structuring your project folder - https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-tomcat-platform-directorystructure.html
Try adding discard-paths: yes at the end of the buildspec.yml file. That will help you resolving the path error.
Related
I am trying to deploy my Go 1.14 microservices application on to Google's Flexible environment. I read that there are issues with finding GOROOT, and how it is unable to obtain the correct dependencies.
I wanted to use the flexible environment because I needed to do port forwarding. Since my domain name was used to run the actual application, I wanted port 8081 to run my microservices.
I followed the instruction from this link:
https://blog.cubieserver.de/2019/go-modules-with-app-engine-flexible/
I tried option 3.
This is my gitlab-ci.yaml configurations file.
# .gitlab-ci.yaml
stages:
- build
- deploy
build_server:
stage: build
image: golang:1.14
script:
- go mod vendor
- go install farmcycle.us/user/farmcycle
artifacts:
paths:
- vendor/
deploy_app_engine:
stage: deploy
image: google/cloud-sdk:270.0.0
script:
- echo $SERVICE_ACCOUNT > /tmp/$CI_PIPELINE_ID.json
- gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
- gcloud --quiet --project $PROJECT_ID app deploy app.yaml
after_script:
- rm /tmp/$CI_PIPELINE_ID.json
This my app.yaml configuration file
runtime: go
env: flex
network:
forwarded_ports:
- 8081/tcp
When I deployed this using the Git CI pipeline. Build stage passes, but the Deploy stage failed.
Running with gitlab-runner 13.4.1 (e95f89a0)
on docker-auto-scale 72989761
Preparing the "docker+machine" executor
Preparing environment
00:03
Getting source from Git repository
00:04
Downloading artifacts
00:02
Executing "step_script" stage of the job script
00:03
$ echo $SERVICE_ACCOUNT > /tmp/$CI_PIPELINE_ID.json
$ gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
Activated service account credentials for: [farmcycle-hk1996#appspot.gserviceaccount.com]
$ gcloud --quiet --project $PROJECT_ID app deploy app.yaml
ERROR: (gcloud.app.deploy) Staging command [/usr/lib/google-cloud-sdk/platform/google_appengine/go-app-stager /builds/JLiu1272/farmcycle-backend/app.yaml /builds/JLiu1272/farmcycle-backend /tmp/tmprH6xQd/tmpSIeACq] failed with return code [1].
------------------------------------ STDOUT ------------------------------------
------------------------------------ STDERR ------------------------------------
2020/10/10 20:48:27 staging for go1.11
2020/10/10 20:48:27 Staging Flex app: failed analyzing /builds/JLiu1272/farmcycle-backend: cannot find package "farmcycle.us/user/farmcycle/api" in any of:
($GOROOT not set)
/root/go/src/farmcycle.us/user/farmcycle/api (from $GOPATH)
GOPATH: /root/go
--------------------------------------------------------------------------------
Running after_script
00:01
Running after script...
$ rm /tmp/$CI_PIPELINE_ID.json
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1
This was the error. Honestly I am not really sure what this error is and how to fix it.
Surprisingly, even using the latest Go runtime, runtime: go1.15, go modules appear to not be used. See golang-docker.
However, Flex builds your app into a container regardless of runtime and so, in this case, it may be better to use a custom runtime and build your own Dockerfile?
runtime: custom
env: flex
Then you get to use e.g. Go 1.15 and go modules (without vendoring) and whatever else you'd like. For a simple main.go that uses modules e.g.:
FROM golang:1.15 as build
ARG PROJECT="flex"
WORKDIR /${PROJECT}
COPY go.mod .
RUN go mod download
COPY main.go .
RUN GOOS=linux \
go build -a -installsuffix cgo \
-o /bin/server \
.
FROM gcr.io/distroless/base-debian10
COPY --from=build /bin/server /
USER 999
ENV PORT=8080
EXPOSE ${PORT}
ENTRYPOINT ["/server"]
This ought to be possible with Google's recently announced support for buildpacks but I've not tried it.
I have a CI/CD working well at Amazon.
Lets say I have a simple html with a url homolog.test.com.
Is there a way when I merge the repository to master, the post build of CodeBuild & CodeDeploy run thru all files to find homolog. to replace with www.
The point is that I dont wanna change by hand all files to test in homolog and then replace again by hand to merge to master...I would like to have homolog working fine and when I merge to master it would replace at the deploy time the urls...
I know that the correct would be have the correct urls in the corret repository, but to facilitate the tests would be nice to work in this other way not too trivial...
A "sed" shell command would work? like this example? Unix Shell Loop through files and replace texts
would be something like this on the post build section? how I specify the correct folder at S3?
sed "s/homolog./www./g" *.html -i
My BuildSpec is very simple:
version: 0.2
phases:
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
build:
commands:
- cp dev-ops/config.production.js src/config.js
- export PUBLIC_URL=/app/
- npm run build
post_build:
commands:
- aws s3 sync site-beta s3://www.test.com/ --cache-control max-age=3600
- aws s3 sync build s3://www.test.com/app/ --cache-control max-age=3600
artifacts:
files:
- build/**/*
I configured a Google could demo project and created a cluster for it in the GitLab Serverless settings for a Hello world Spring boot application. The only information I find on deploying applications is https://docs.gitlab.com/ee/user/project/clusters/serverless/#deploying-serverless-applications which might explain how to deploy a Ruby application only. I'm not sure about that because none of the variables used in the script are explained and the hint
Note: You can reference the sample Knative Ruby App to get started.
is somehow confusing because if I'm not familiar with building Ruby applications which I'm not, that won't get me started.
Following the instruction to put
stages:
- build
- deploy
build:
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
only:
- master
script:
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE
deploy:
stage: deploy
image: gcr.io/triggermesh/tm#sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5
only:
- master
environment: production
script:
- echo "$CI_REGISTRY_IMAGE"
- tm -n "$KUBE_NAMESPACE" --config "$KUBECONFIG" deploy service "$CI_PROJECT_NAME" --from-image "$CI_REGISTRY_IMAGE" --wait
in .gitlab-ci.yml causes the deploy stage to fail due to
$ tm -n "$KUBE_NAMESPACE" --config "$KUBECONFIG" deploy service "$CI_PROJECT_NAME" --from-image "$CI_REGISTRY_IMAGE" --wait
2019/02/09 11:08:09 stat /root/.kube/config: no such file or directory, falling back to in-cluster configuration
2019/02/09 11:08:09 Can't read config file
ERROR: Job failed: exit code 1
My Dockerfile which allows to build locally looks as follows:
FROM maven:3-jdk-11
COPY . .
RUN mvn --batch-mode --update-snapshots install
EXPOSE 8080
CMD java -jar target/hello-world-0.1-SNAPSHOT.jar
(the version in the filename doesn't make sense for further deployment, but that's a follow-up problem).
Reason is a mismatch in the environment value specified in .gitlab-ci.yml and the GitLab Kubernetes configuration, see https://docs.gitlab.com/ee/user/project/clusters/#troubleshooting-missing-kubeconfig-or-kube_token for details.
I have a multi module maven project i want do build and deploy with AWS Codestar on EC2. This almost works like a charm now.
Local build works, app can be accessed on port 5000.
Codestar build is ok, upload is ok, deployment seems to be ok.
But i cannot reach the app on port 80 (404 not found). SERVER_PORT is set to 5000, should be translated to 80 on AWS.
Now the funny thing about the story: if i deploy the WAR manually (local or downloaded from CodeBuild), both can be accessed on AWS on port 80. But the Codestar uploaded and deployed one cannot.
I'm pretty much out of ideas. The logs don't show anything usable. I'm willing to provide them, though.
Here is the buildspec.yml. I'm still convinced that there is something wrong in there...
version: 0.2
phases:
install:
commands:
- echo Entering install phase...
- wget http://mirror.olnevhost.net/pub/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
- tar xzvf apache-maven-3.3.9-bin.tar.gz -C /opt/
- export PATH=/opt/apache-maven-3.3.9/bin:$PATH
build:
commands:
- echo Entering BUILD phase...
- echo Build started on `date`
- mvn install
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- 'jweb-web/target/*.war'
We have a separate build for the frontend and backend of the application, where we need to pull the dist build of frontend to backend project during the build. During the build the 'curl' cannot write to the desired location.
In detail, we are using SpringBoot as backend for serving Angular 2 frontend. So we need to pull the frontend files to src/main/resources/static folder.
image: maven:3.3.9
pipelines:
default:
- step:
script:
- curl -s -L -v --user xxx:XXXX https://api.bitbucket.org/2.0/repositories/apprentit/rent-it/downloads/release_latest.tar.gz -o src/main/resources/static/release_latest.tar.gz
- tar -xf -C src/main/resources/static --directory src/main/resources/static release_latest.tar.gz
- mvn package -X
As a result of this the build fails with output of CURL.
* Failed writing body (0 != 16360)
Note: I've tried the same with maven-exec-plugin, the result was the same. The solution works on local machine naturally.
I would try running these commands from a local docker run of the image you're specifying (maven:3.3.9). I found that the most helpful way to debug things that were behaving differently in Pipelines vs. in my local environment.
To your specific question yes, you can download external content from the Pipeline run. I have a Pipeline that clones other repos from BitBucket via HTTP into the running container.