I have a sonarqube server running on top of Azure and a CICD pipeline configured using Google cloud build on top of GCP. Do you have an idea about how to include the sonarqube connection information in my cloudbuild file as a custom build step? I'm using gradle to build my build and test my images.
There's a sonarqube community cloud builder: https://github.com/GoogleCloudPlatform/cloud-builders-community/tree/master/sonarqube
There is an example of using it as a step here: https://github.com/GoogleCloudPlatform/cloud-builders-community/blob/master/sonarqube/examples/cloudbuild.yaml
Below sample code worked for me
#static code analysis by sonarqube
- name: 'maven:3.6.1-jdk-8'
entrypoint: 'bash'
args:
- -c
- |
unset MAVEN_CONFIG \
&& echo "104.199.71.165 sonarqube.ct.blue.cdtapps.com" > /etc/hosts \
&& mvn sonar:sonar -q -Dsonar.login=5531b1a2d571c0482a3d45f605830e08ccf5f245 \
'-Dsonar.projectKey=odp.df.pubsub-sftp' \
'-Dsonar.projectName=ODP-DF-PUBSUB-SFTP' \
'-Dsonar.host.url=https://sonarqube.ct.blue.cdtapps.com' \
'-Dsonar.qualitygate.wait=true' \
'allow_failure: true'
dir: 'dataflows/generic/pubsub-sftp/src'
id: 'sonarqube-analysis'
Related
I am using Github Actions to automate the process to push a docker image generated with the help of the maven plugin from Spring boot (mvn spring-boot:build-image), but I receive a maven error:
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
default-cli of goal
org.springframework.boot:spring-boot-maven-plugin:3.0.0:build-image
failed: Error response received when pushing image: denied: requested
access to the resource is denied
Using the following configuration:
- name: Build image & push
run: |
cd myFolder
mvn -X spring-boot:build-image \
--batch-mode --no-transfer-progress \
-Dspring-boot.build-image.publish=true \
-Dspring-boot.build-image.imageName="MY_USER/demo-ms:0.1.0" \
-DCI_REGISTRY=https://index.docker.io/v1 \
-DCI_REGISTRY_USER=${{ secrets.DOCKERHUB_USERNAME }} \
-DCI_REGISTRY_PASSWORD=${{ secrets.DOCKERHUB_TOKEN }}
https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/#build-image.examples.docker.auth
What I am missing?
Many thanks in advance
Juan Antonio
In github actions, it is possible to run a Script, so I found an alternative:
At github action level:
- name: Build image & push
run: |
cd myFolder
./build-spring-boot.sh ${{ secrets.DOCKERHUB_USERNAME }} ${{ secrets.DOCKERHUB_PASSWORD }}
At script level:
docker login "https://index.docker.io/v1/" -u="$1" -p="$2"
mvn spring-boot:build-image \
--batch-mode --no-transfer-progress
IMAGE_NAME=$(mvn help:evaluate -Dexpression=docker.image.name -q -DforceStdout)
echo $IMAGE_NAME
docker push $IMAGE_NAME
In this way, you can publish your image.
I am using Sonarqube and CircleCI for code quality scan.
However, I don't know how if it is possible to start up a Sonarqube Server on CircleCI and use it to run the scanner.
This is my current config.yaml
version: 2.1
executors:
scanner:
docker:
- image: openjdk:11
commands:
check-code-quality:
description: Check Code Quality
parameters:
sonar_server_url:
type: string
description: "URL of your SonarQube server. e.g.: http://my.sonarqube,server:9000"
default: "$SONAR_SERVER"
sonar_login:
description: "Authentication key (sonar.login paramter) to access SonarQube and perform analysis"
type: string
default: "$SONAR_TOKEN"
sonar_sources:
description: "Where the files are located?"
type: string
default: "$SONAR_SOURCES"
steps:
- run:
name: Install Sonarqube scanner
command: |
wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.2.0.1873.zip
unzip sonar-scanner-cli-4.2.0.1873.zip
- run:
name: Run Sonarscanner
command: |
export SONAR_SCANNER_OPTS="-Xmx2048m"
eval ./sonar-scanner-4.2.0.1873/bin/sonar-scanner \
-Dsonar.projectKey=projectKey
-Dsonar.host.url=<< parameters.sonar_server_url >> \
-Dsonar.sources=<< parameters.sonar_sources >> \
-Dsonar.login=<< parameters.sonar_login >>
jobs:
check-code-job:
executor: scanner
steps:
- check-code-quality
workflows:
check-code-quality-flow:
jobs:
- check-code-job:
context: lineclass
There is an error log when the job being executed:
...
Caused by: java.lang.IllegalStateException: Fail to get bootstrap index from server
at org.sonarsource.scanner.api.internal.BootstrapIndexDownloader.getIndex(BootstrapIndexDownloader.java:42)
at org.sonarsource.scanner.api.internal.JarDownloader.getScannerEngineFiles(JarDownloader.java:58)
at org.sonarsource.scanner.api.internal.JarDownloader.download(JarDownloader.java:53)
at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:76)
... 7 more
Caused by: java.net.ConnectException: Failed to connect to localhost/127.0.0.1:9000
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.R...
This indicates that the Sonarqube Server is missing.
If you have experience running sonar-scanner on CircleCI please help.
Thank you.
After I change the image to sonarqube:8.9-community and fix the missing \ in the sonar-scanner command (at the end of -Dsonar.projectKey), it works.
I have 2 Gitlab repos:
Project A
Integration tests for Project A
I want to stop the pipeline / build of Project A if the integration tests fail but currently the Project A pipeline passes even if the integration tests fail.
My .gitlab-ci.yml for Project A defines these 7 stages:
stages:
- build
- test
- publish
- dev-deployment
- staging-deployment
- trigger-integration-tests
- prod-deployment
The second last stage (trigger-integration-tests) kicks off the integration tests project by using the Gitlab API call with curl:
trigger-integration-tests:
stage: trigger-integration-tests
image: ubuntu:16.04
script:
- apt-get update && apt-get install -y curl
- "curl -X POST -F token=$INTEGRATION_TESTS_TOKEN -F variables[PROJECT_ID]=$CI_PROJECT_ID -F variables[BRANCH_NAME]=$CI_COMMIT_REF_NAME -F ref=master https://gitlab.mycompany.com/api/v4/projects/123/trigger/pipeline"
allow_failure: false
only:
- master
I tried adding the allow_failure: false flag but that didn't help so I'm looking for more ideas.
I found the trigger-and-wait technique but wasn't sure if there's a more simple solution.
As answered on a previous question, you could do the following:
From the main project, using a Python/Bash script:
Trigger the integration tests pipeline (and capture the pipeline ID)
Poll the status of the pipeline, using the captured ID (which can be running, pending, failed, canceled or skipped)
Raise an exception / error if it has failed...
See here for an example python script to achieve this.
I tried to use Code Pipeline to automate the code deployment. It uses Git Hub -> Code Build -> Cloud Formation as mentioned in wiki
AWS Automation of Lambda
I managed to get the pipeline run after few changes suggested by this thread
However whenever I am using the code pipeline, the Lambda test fails saying the class is not found.
In order to verify, I uploaded the jar directly in AWS lambda console and it worked fine.
I also verified the jar which is built by aws code build in the S3 "MyAppBuild" folder and it contains jar file in target/app-1.0-SNAPSHOT.jar in a zip file along with my SamTemplate.yml.
This is the SamTemplate.yml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Outputs the time
Parameters:
SourceBucket:
Type: String
Description: S3 bucket name for the CodeBuild artifact
SourceArtifact:
Type: String
Description: S3 object key for the CodeBuild artifact
Resources:
TimeFunction:
Type: AWS::Serverless::Function
Properties:
Handler: com.xxx.Hello::handleRequest
Runtime: java8
CodeUri:
Bucket: !Ref SourceBucket
Key: !Ref SourceArtifact
Events:
MyTimeApi:
Type: Api
Properties:
Path: /TimeResource
Method: GET
Here is the buildSpec.yaml
version: 0.2
phases:
build:
commands:
- echo Build started on `date`
- mvn test
post_build:
commands:
- echo Build completed on `date`
- mvn package
install:
commands:
- aws cloudformation package --template-file SamTemplate.yaml --s3-bucket codepipeline-us-east-1-xxxx
--output-template-file NewSamTemplate.yaml
artifacts:
type: zip
files:
- SamTemplate.yaml
- target/app-1.0-SNAPSHOT.jar
Any suggestions to try on?
I use maven.
Finally, after a few tries I found a probable solution for the packaging with aws code build, cloud formation, and lambda.
The whole point is that code build creates a wrapper zip of all files mentioned in artifacts:
This is the same zip file which must be given to aws lambda.
In order for aws lambda to accept a zip as valid, classes should be root folder, dependent libs should be in libs folder.
So I managed to do this as my build spec.
version: 0.2
phases:
install:
commands:
- aws cloudformation package --template-file SamTemplate.yaml --s3-bucket codepipeline-us-east-1-XXXXXXXX
--output-template-file NewSamTemplate.yaml
build:
commands:
- echo Build started on `date`
- gradle build clean
- gradle test
post_build:
commands:
- echo Build started on `date`
- gradle build
- mkdir -p deploy
- cp -r build/classes/main/* deploy/
- cp NewSamTemplate.yaml deploy/
- cp -r build/libs deploy/
- ls -ltr deploy
- ls -ltr build
- echo Build completed on `date`
- echo Build is complete
artifacts:
type : zip
files:
- '**/*'
base-directory : 'deploy'
I'm developing an android library.
My project is using Gradle and don't have any application modules. Just library.
I've connected this one to the Travis CI but every my push console was showing that my building process has been failing.
This is my travis.yml
language: android
android:
components:
- build-tools-22.0.1
- android-22
- extra-android-m2repository
licenses:
- android-sdk-license-.+
before_install:
- chmod +x gradlew
before_script:
- echo no | android create avd --force -n test -t android-21 --abi armeabi-v7a
- emulator -avd test -no-skin -no-audio -no-window &
- curl http://is.gd/android_wait_for_emulator > android-wait-for-emulator
- chmod u+x android-wait-for-emulator
- ./android-wait-for-emulator
- adb shell input keyevent 82 &
And error I got:
* What went wrong:
Execution failed for task ':library:connectedAndroidTest'.
com.android.builder.testing.ConnectedDevice > runTests[test(AVD) - 5.0] FAILED
com.android.builder.testing.api.DeviceException: com.android.ddmlib.InstallException: Failed to establish session
at com.android.builder.testing.ConnectedDevice.installPackages(ConnectedDevice.java:108)
null
So do I must do to resolve this problem? Thanks!
Add atleast one image file in your travis.yml like this
- sys-img-armeabi-v7a-android-19
- sys-img-x86-android-17