How do I properly create buildspec.yml file for Laravel Application using AWS CodePipeline - laravel

I am using AWS CodePipeline for the first time and trying to figure out how to properly create my buildspec.yml file for my Laravel application. There are few resources on the internet.
I have the following in my buildspec.yml file currently:
version: 0.2
phases:
install:
commands:
- curl -s https://getcomposer.org/installer | php
- mv composer.phar /usr/local/bin/composer
- php --version
build:
commands:
- echo Build started on `date`
- echo Installing composer deps
- composer install
- cp extra/.env ./
- php artisan cache:clear
post_build:
commands:
- echo Build completed on `date`
artifacts:
type: zip
files:
- '**/*'
name: clyde-$(date +%Y-%m-%d)
The CodeBuild is successful and this does deploy to Elastic Beanstalk. I did change the configuration in Elastic Beanstalk so the root is /public (for Laravel). However, when I go to the URL, the first line of code run presents an error like below:
View [inc\navbar] not found. (View: /var/app/current/resources/views/layouts/app.blade.php)
This leads me to believe something is not built properly.

To make it work, it will need to use a complete Pipeline: CodeCommit-->CodeBuild-->CodeDeploy
Inside your Artifact bucket there will be two objects generated in the process:
s3://codepipeline-us-east-1-<001122334455>/SourceArtif/
s3://codepipeline-us-east-1-<001122334455>/BuildArtif/
The first one is obtained in the initial phase of the pipeline from CodeCommit.
The second one is created by CodeBuild. The resultant zip file will be exactly the same as that one from CodeCommit. So it seems, the CodeBuild is only testing but not saving the Artifact with results from the instructions specified in buildspec.yml.
The third phase, CodeDeploy will obtain the code from the Artifact and it will need to Build again via scripts referred by appspec.yml.
version: 0.0
os: linux
files:
- source: /
destination: /web/project/html
hooks:
BeforeInstall:
- location: scripts/install_dependencies.sh
timeout: 300
runas: root
AfterInstall:
- location: scripts/build_again.sh
timeout: 600
runas: user
ApplicationStart:
- location: scripts/start_application.sh
timeout: 300
runas: root
The build_again.sh file will need to include same commands you are using in buildspec.yml (build section), then your Laravel project should be working.

Related

AWS Amplify Build Error - Mismatched Brackets Found In The Schema

End Desire:
To build my AWS Amplify project in the dev environment, using full-stack CI/CD. So for example, pushing my changes to Code Commit will build the backend resources.
Build Logs:
2023-01-06T06:19:26.457Z [INFO]: [33mNote: It is recommended to run this command from the root of your app directory[39m
2023-01-06T06:19:27.797Z [WARNING]: - Initializing your environment: dev
2023-01-06T06:19:29.164Z [WARNING]: - Building resource api/project
2023-01-06T06:19:33.756Z [WARNING]: - Building resource auth/project
2023-01-06T06:19:33.902Z [WARNING]: - Building resource storage/project
2023-01-06T06:19:33.939Z [WARNING]: ✔ Initialized provider successfully.
2023-01-06T06:19:34.452Z [WARNING]: ✖ There was an error initializing your environment.
2023-01-06T06:19:34.453Z [INFO]: 🛑 Syntax Error: mismatched brackets found in the schema. Unexpected ) at line 239 in the schema.
2023-01-06T06:19:34.453Z [INFO]: Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/
buildspec.yml:
version: 0.1
frontend:
phases:
preBuild:
commands:
- npm i
build:
commands:
- npm run build
artifacts:
baseDirectory: .next
files:
- '**/*'
cache:
paths:
- node_modules/**/*
Checks:
I have checked the graphql.schema for errors.
I have removed the testing section in the buildspec.
I have added amplify push --simple as recommended here
I've ran npm run build on my local CLI successfully
In the AWS Amplify console, under build settings, the AWS CLI version was "latest". Changing the AWS CLI version to 10.4.0 fixed the issue.
View AWS CLI Versioning here: https://www.npmjs.com/package/#aws-amplify/cli?activeTab=versions
Rather than going to 10.4.0, go back a couple versions from the latest AWS CLI version at the time.

Gitlab runner cache miss file after stage complete

Summary
My gitlab-ci.yml has 3 stage for deploy an application to okd pod
Application running spring boot on tomcat:8
Sometimes, the cache.zip is not update after stage complete so that the next step can't run correctly
Steps to reproduce
My gitlab-ci run the following stage
Stage 1: run test compile ---> OK
Stage 2: package war file as output for deploy ---> Gitlab-ci log show success but the cache.zip has not war file (just sometimes cache.zip not have war file, sometimes it run correctly)
Stage 3: Deploy war file to pod ---> Because of war file not exists in cache.zip, script error -> failed
.gitlab-ci.yml
image: openshift/origin-cli
stages:
- build
- test
- staging
cache:
paths:
- .m2/repository
- target
- artifact
validate:jdk8:
stage: build
script:
- 'mvn test-compile'
only:
- master
image: maven:3.3.9-jdk-8
verify:jdk8:
stage: test
script:
- 'mvn verify'
- 'mvn package' # =====> this command generate war file
only:
- master
image: maven:3.3.9-jdk-8
staging:
script:
- "mkdir -p artifact"
- "cp ./target/*.war ./artifact/" # ======> Sometimes error at this line because of previous step not add war file into cache
- "oc start-build $APP"
- "rm -rf ./target/* && rm -rf ./artifact/*" # Remove war & class file, only cache m2 lib
stage: staging
variables:
APP: $CI_PROJECT_NAME
environment:
name: staging
url: http://$CI_PROJECT_NAME-staging.$OPENSHIFT_DOMAIN
only:
- master
Actual behavior
Sometimes cache not have war file after test stage complete (is this depends on war file size?)
Expected behavior
War file update into cache after test stage for staging stage deploy
Relevant logs and/or screenshots
ScreenShot
job log
Running with gitlab-runner 13.7.0 (943fc252)
on gitlab-runner-node1 y6awygsj
Preparing the "docker" executor
00:01
Using Docker executor with image openshift/origin-cli ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:7ebb6be01117a50344d63f77c385a13302afecd33480b97c36a518d4f5ebc25a for openshift/origin-cli with digest docker.io/openshift/origin-cli#sha256:509e052d0f2d531b666b7da9fa49c5558c76ce5d286456f0859c0a49b16d6bf2 ...
Preparing environment
00:00
Running on runner-y6awygsj-project-489-concurrent-0 via gitlab.runner.node1...
Getting source from Git repository
00:01
Fetching changes...
Reinitialized existing Git repository in /builds/my-project/.git/
Checking out b4c97428 as master...
Removing .m2/
Removing artifact/
Removing target/
Skipping Git submodules setup
Restoring cache
00:05
Checking cache for default-23...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
Executing "step_script" stage of the job script
00:01
$ mkdir -p artifact
$ cp ./target/*.war ./artifact/
cp: cannot stat './target/*.war': No such file or directory
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1
Environment description
config.toml
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "gitlab-runner-node1"
url = "https://gitlab.mycompany.vn/"
token = "y6awygsj9zks18nU6PDt"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
dns = ["192.168.100.1"]
tls_verify = false
image = "alpine:latest"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/mnt/nfs/nfsshare-gitlab/cache:/cache"]
shm_size = 0
pull_policy = "if-not-present"
Used GitLab Runner version
Version: 13.7.0
Git revision: 943fc252
Git branch: 13-7-stable
GO version: go1.13.8
Built: 2020-12-21T13:47:06+0000
OS/Arch: linux/amd64
Possible fixes
Re-run test stage until cache has war file
Let's go step by step.
First, regarding how to manage the files between stages.
It's true that you could directly access to the files between jobs and stages if both run on the same environment, but that's not always the case (even if both runners are using the same nfs share directory) and you should use artifacts for that.
When you define an artifact within a job, you're specifying a list of files that are attached to the job when it succeeds, fails or always, depending on the configuration you have.
By default, all artifacts from previous stages are passed to each job, but in any case you can use dependencies to also define from which jobs you want to fetch artifacts from.
So basically you should use the following .gitlab-ci.yml
image: openshift/origin-cli
stages:
- build
- test
- staging
cache:
paths:
- .m2/repository
validate:jdk8:
stage: build
script:
- 'mvn test-compile'
only:
- master
image: maven:3.3.9-jdk-8
verify:jdk8:
stage: test
script:
- 'mvn verify' # =====> verify already includes: validate, compile, test and package
artifacts:
paths:
- target/[YOUR_APP_NAME].war
only:
- master
image: maven:3.3.9-jdk-8
staging:
dependencies:
- verify:jdk8
script:
- "mkdir -p artifact"
- "cp ./target/[YOUR_APP_NAME].war ./artifact/"
- "oc start-build $APP"
stage: staging
variables:
APP: $CI_PROJECT_NAME
environment:
name: staging
url: http://$CI_PROJECT_NAME-staging.$OPENSHIFT_DOMAIN
only:
- master
Also, notice that I deleted the mvn package instruction. I would recommend you to take a look into the Build Lifecycle Basics of Maven.

How to properly set up CircleCI plugin

I am trying to integrate CircleCI plugin with my spring-pet-clinic project. I was following the instruction on CircleCI web page. I have created .circleci folder inside my project root folder.
Inside .circleci I have added config.yml file and copy-pasted config from CircleCI page
My config was like this:
# Use the latest 2.1 version of CircleCI pipeline processing engine,
see https://circleci.com/docs/2.0/configuration-reference/
version: 2.1
# Use a package of configuration called an orb, see
https://circleci.com/docs/2.0/orb-intro/
orbs:
# Declare a dependency on the welcome-orb
welcome: circleci/welcome-orb#0.3.1
# Orchestrate or schedule a set of jobs, see
https://circleci.com/docs/2.0/workflows/
workflows:
# Name the workflow "Welcome"
Welcome:
# Run the welcome/run job in its own container
jobs:
- welcome/run
After I ran the project CircleCI has thrown an error. Especially this one: "Config Processing Error: Don't rerun"
$#!/bin/sh -eo pipefail
# No configuration was found in your project. Please refer to
https://circleci.com/docs/2.0/ to get started with your
configuration.
#
# -------
# Warning: This configuration was auto-generated to show you
the
message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
Spin Up Environment looks like this
Build-agent version 1.0.10572-3ce00c85 (2019-04-
15T22:09:28+0000)
Docker Engine Version: 17.05.0-ce
Kernel Version: Linux b0a81c56acff 4.4.0-144-generic
#170~14.04.1-
Ubuntu SMP Mon Mar 18 15:02:05 UTC 2019 x86_64 Linux
Starting container bash:4.4.19
using image
bash#sha256:9f0a4aa3c9931bd5fdda51b1b2b74a0398a8eabeaf9519d807e010b9d9d41993
Using build environment variables
BASH_ENV=/tmp/.bash_env-5cbebf83d4b030000849b60f-0-build
CI=true
CIRCLECI=true
CIRCLE_BRANCH=master
CIRCLE_BUILD_NUM=5
CIRCLE_BUILD_URL=https://circleci.com/gh/sajmon2325/Spring-
Pet-
Clinic/5
CIRCLE_COMPARE_URL=
CIRCLE_JOB=Build Error
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
CIRCLE_PREVIOUS_BUILD_NUM=4
CIRCLE_PROJECT_REPONAME=Spring-Pet-Clinic
CIRCLE_PROJECT_USERNAME=sajmon2325
CIRCLE_REPOSITORY_URL=git#github.com:sajmon2325/Spring-Pet-
Clinic.git
CIRCLE_SHA1=48f6db114b41c338e606de32d8648c64ba5119fd
CIRCLE_SHELL_ENV=/tmp/.bash_env-5cbebf83d4b030000849b60f-0-
build
CIRCLE_STAGE=Build Error
CIRCLE_USERNAME=sajmon2325
CIRCLE_WORKFLOW_ID=2789d93e-f1e4-4c81-93f1-846f7d38c107
CIRCLE_WORKFLOW_JOB_ID=670105ca-617e-445e-9b5e-6ac57f6af8da
CIRCLE_WORKFLOW_UPSTREAM_JOB_IDS=
CIRCLE_WORKFLOW_WORKSPACE_ID=2789d93e-f1e4-4c81-93f1-
846f7d38c107
CIRCLE_WORKING_DIRECTORY=~/project
Using environment variables from project settings and/or
contexts
CIRCLE_JOB=**REDACTED**
So at first I thought that I have only a skeleton of CircleCI configuration, that's why I have edited my config.yml file to look like this (the actual version)
# Java Maven CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-java/ for more details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
- image: circleci/openjdk:11-browsers-legacy
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/repo
environment:
# Customize the JVM maximum heap limit
MAVEN_OPTS: -Xmx3200m
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "pom.xml" }}
# fallback to using the latest cache if no exact match
is
found
- v1-dependencies-
- run: mvn install -DskipTests
- run: mvn dependency:go-offline
- save_cache:
paths:
- ~/.m2
key: v1-dependencies-{{ checksum "pom.xml" }}
# run tests!
- run: mvn integration-test
But even this is not working. I still have the same error:
$#!/bin/sh -eo pipefail
# No configuration was found in your project. Please refer to https://circleci.com/docs/2.0/ to get started with your configuration.
#
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
I just need to successfully integrate CircleCi plugin with my project. If you need to see my repo, here is the link: https://github.com/sajmon2325/Spring-Pet-Clinic.git
The problem is that .circleci is not in the root of the repository. It is currently in sfg-pet-clinic/, and the CircleCI build process won't find it there.

Cloud formation lambda not picking jar from code build

I tried to use Code Pipeline to automate the code deployment. It uses Git Hub -> Code Build -> Cloud Formation as mentioned in wiki
AWS Automation of Lambda
I managed to get the pipeline run after few changes suggested by this thread
However whenever I am using the code pipeline, the Lambda test fails saying the class is not found.
In order to verify, I uploaded the jar directly in AWS lambda console and it worked fine.
I also verified the jar which is built by aws code build in the S3 "MyAppBuild" folder and it contains jar file in target/app-1.0-SNAPSHOT.jar in a zip file along with my SamTemplate.yml.
This is the SamTemplate.yml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Outputs the time
Parameters:
SourceBucket:
Type: String
Description: S3 bucket name for the CodeBuild artifact
SourceArtifact:
Type: String
Description: S3 object key for the CodeBuild artifact
Resources:
TimeFunction:
Type: AWS::Serverless::Function
Properties:
Handler: com.xxx.Hello::handleRequest
Runtime: java8
CodeUri:
Bucket: !Ref SourceBucket
Key: !Ref SourceArtifact
Events:
MyTimeApi:
Type: Api
Properties:
Path: /TimeResource
Method: GET
Here is the buildSpec.yaml
version: 0.2
phases:
build:
commands:
- echo Build started on `date`
- mvn test
post_build:
commands:
- echo Build completed on `date`
- mvn package
install:
commands:
- aws cloudformation package --template-file SamTemplate.yaml --s3-bucket codepipeline-us-east-1-xxxx
--output-template-file NewSamTemplate.yaml
artifacts:
type: zip
files:
- SamTemplate.yaml
- target/app-1.0-SNAPSHOT.jar
Any suggestions to try on?
I use maven.
Finally, after a few tries I found a probable solution for the packaging with aws code build, cloud formation, and lambda.
The whole point is that code build creates a wrapper zip of all files mentioned in artifacts:
This is the same zip file which must be given to aws lambda.
In order for aws lambda to accept a zip as valid, classes should be root folder, dependent libs should be in libs folder.
So I managed to do this as my build spec.
version: 0.2
phases:
install:
commands:
- aws cloudformation package --template-file SamTemplate.yaml --s3-bucket codepipeline-us-east-1-XXXXXXXX
--output-template-file NewSamTemplate.yaml
build:
commands:
- echo Build started on `date`
- gradle build clean
- gradle test
post_build:
commands:
- echo Build started on `date`
- gradle build
- mkdir -p deploy
- cp -r build/classes/main/* deploy/
- cp NewSamTemplate.yaml deploy/
- cp -r build/libs deploy/
- ls -ltr deploy
- ls -ltr build
- echo Build completed on `date`
- echo Build is complete
artifacts:
type : zip
files:
- '**/*'
base-directory : 'deploy'

Codedeploy-agent error when run in amazon linux instance

When I create a new deploy in AWSCodeDeploy with GitHub I receive this fail message:
Error CodeScriptFailed
Script Namescripts/stop_server.sh
MessageScript at specified location: scripts/stop_server.sh run as user ubuntu failed with exit code 1
Log TailLifecycleEvent - ApplicationStop
Script - scripts/stop_server.sh
[stderr]su: user ubuntu does not exist
But, my instance is an Amazon Linux Instance and don't have a ubuntu user, anybody know anything about this?
The script that a try to run is:
# scripts/stop_server.sh
#!/bin/bash
forever stop .
My appspec.yml file:
version: 0.0
os: linux
files:
- source: /
destination: /home/ec2-user
hooks:
AfterInstall:
- location: scripts/install_dependencies.sh
timeout: 5
runas: root
ApplicationStart:
- location: scripts/start_server.sh
timeout: 5
runas: root
ApplicationStop:
- location: scripts/stop_server.sh
timeout: 5
runas: root
Codedeploy-agent version agent_version: OFFICIAL_1.0-1.1095_rpm
Application stop usually refers to appspec.yml in previous successful deployment archive. Either empty /opt/codedeploy-agent/deployment-archive/deployment-instructions/ or you may use BeforeInstall hook to execute stop script.
The code deploy create temporary application inside /opt/code-deploy/.... path. If the deployment fails normally it starts from the temporary directory next time. If you want to get rid of the error pointing by deployment, you should check the particular script file in the temporary directory and edit it.

Resources