I want my integration tests to run in parallel on Circleci.
I read this document https://circleci.com/blog/how-to-boost-build-time-with-test-parallelism/ and I setup my job like this
platform_component_test:
working_directory: *workspace_root
executor: ubuntu-machine
parallelism: 16
steps:
- prepare_workspace
- run:
name: 'Run Platform Component tests'
command:
./gradlew platform:componentTest -PtestFilter="`circleci tests glob "platform/src/componentTest/java/**/*.java"|circleci tests split`"
By looking at the UI, I see that each of the 16 containers that are spawn execute all the tests.
Am I missing something?
I ended up slightly modifying this and incorporating what I learned from here and here to build this:
- run:
name: Run tests in parallel
# Use "./gradlew test" instead if tests are not run in parallel
command: |
cd module-with-tests-to-run/src/test/kotlin
# Get list of classnames of tests that should run on this node
CLASSNAMES=$(circleci tests glob "**/**Test.kt" \
| cut -c 1- | sed 's#/#.#g' \
| sed 's/.\{3\}$//' \
| circleci tests split --split-by=timings --timings-type=classname)
cd ../../../..
# Format the arguments to "./gradlew test"
GRADLE_ARGS=$(echo $CLASSNAMES | awk '{for (i=1; i<=NF; i++) print "--tests",$i}')
echo "Prepared arguments for Gradle: $GRADLE_ARGS"
./gradlew clean module-with-tests-to-run:test $GRADLE_ARGS
note: I tried to get the formatting right but I might have goofed.
Related
I am trying to update contents of a file from a variable with sed during Gitlab CI job. The variable comes from artifacts of the previous stage version. If simplified, my job looks something like this:
build-android-dev:
stage: build
dependencies:
- version
only:
- mybranch
before_script:
- PUBSPEC_VERSION="`cat app-pubspec-version`"
- echo "Pubspec version - $PUBSPEC_VERSION"
script:
- >
sed -i -E "s/^(version: )(.+)$/\1${PUBSPEC_VERSION}/g" pubspec.yaml
- cat pubspec.yaml | grep version
interruptible: true
tags:
- macmini-flutter
Unfortunatelly, the job fails with the following error message:
$ sed -i -E "s/^(version: )(.+)$/\1${PUBSPEC_VERSION}/g" pubspec.yaml
sed: 1: "s/^(version: )(.+)$/\14 ...": \1 not defined in the RE
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit status 1
PUBSPEC_VERSION coming from artifacts is the following:
$ echo "Pubspec version - $PUBSPEC_VERSION"
Pubspec version - 4.0.0+2
I am able to execute the command on my local Ubuntu (Linux) machine without any issues:
$ export PUBSPEC_VERSION=4.0.0+2
$ sed -i -E "s/^(version: )(.+)$/\1${PUBSPEC_VERSION}/g" pubspec.yaml
$ cat pubspec.yaml | grep version
version: 4.0.0+2
The remote machine where Gitlab Runner is started is MacOS. Not sure whether it matters.
As you can see, I also use folding style in my CI configuration like proposed here in order to avoid inproper colon interpretation.
I googled for solutions to solve the issue but it seems that I don't need to escape (though I also tried) group parentheses in my regular expression because I use extended regular expression.
So I'm stuck on it...
P.S. I don't have access to the shell of remote MacOS.
is MacOS.
-i takes a suffix argument, so -E is the backup suffix to create. Yuo would want:
- sed -i '' -E 's/...'
Based on this question on StackOverflow, I added test coverage into my Gitlab CI/CD YAML file, and I expect the result gets uploaded into codeclimate.
- go get github.com/axw/gocov/gocov
- export CC_TEST_REPORTER_ID=My_REPO_ID
- curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
- chmod +x ./cc-test-reporter
- ./cc-test-reporter before-build
- gocov test -v ./... -coverprofile=out
- ./cc-test-reporter format-coverage --input-type gocov out
- ./cc-test-reporter upload-coverage
The script runs test successfully in all my packages, and the output of CI/CD shows that all tests in different packages ran, but uploaded report into codeclimate only shows there is only one file with test-coverage and that is the last test package, instead of showing all.
I mixed the solution with another answer in the same stackoverflow link and finally create a bash file and ran it from my yaml file like this so the report output contain all package reports, this is my script:
go get github.com/axw/gocov/gocov
export CC_TEST_REPORTER_ID=My_ID
curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
chmod +x ./cc-test-reporter
./cc-test-reporter before-build
for pkg in $(go list ./... | grep -v vendor); do
go test -coverprofile=$(echo $pkg | tr / -).cover $pkg
done
echo "mode: set" > c.out
grep -h -v "^mode:" ./*.cover >> c.out
rm -f *.cover
./cc-test-reporter after-build
Inside .gitlab-ci.yml we define a variable (which is just the artifactId name for the project) ARTIFACT_ID: myMicroservice-1
This variable ARTIFACT_ID is sent to a general microservice which has all the scripts to publish/deploy docker, etc.
How can I read this variable direct from POM file?
pom:
<artifactId>myMicroservice-1</artifactId>
.gitlab-ci.yml:
variables:
SKIP_UNIT_TESTS_FLAG: "true"
ARTIFACT_ID: myMicroserverName
IS_OSL: "true"
KUBERNETES_NAMESPACE: test
Here is how we do it.
Value is extracted from the pom.xml based on its XPath.
We use xmllint tool from libxml2-utils, but there are various other tools for that.
Then value is saved as an environment variable in a file, which is passed to further GitLab jobs as artifact.
stages:
- prepare
- build
variables:
VARIABLES_FILE: ./variables.txt # "." is required for sh based images
POM_FILE: pom.xml
get-version:
stage: prepare
image: ubuntu
script:
- apt-get update
- apt-get install -y libxml2-utils
- APP_VERSION=`xmllint --xpath '/*[local-name()="project"]/*[local-name()="version"]/text()' $POM_FILE`
- echo "export APP_VERSION=$APP_VERSION" > $VARIABLES_FILE
artifacts:
paths:
- $VARIABLES_FILE
build:
stage: build
image: docker:latest
script:
- source $VARIABLES_FILE
- echo "Here use $APP_VERSION as you like"
This is the script to grab information from pom, in this case, artifactId:
- export myARTIFACT_ID=$(mvn exec:exec -q -Dexec.executable=echo -Dexec.args='${project.artifactId}')
- if [[ "$myARTIFACT_ID" == *finishWithWhatEverName]]; then export myVariable="false"; else export myVariable="true";
Then you can use myVariable to whatever you want.
I tried Angel's solution, but got an error:
/bin/bash: line 87: mvn: command not found
Finally I succeeded when I used the following to extract the tag value:
- export ARTIFACT_ID=$(cat pom.xml | grep "<artifactId>" | head -1 | cut -d">" -f2 | cut -d"<" -f1 | awk '{$1=$1;print}')
jobname:
stage: stage
before_script:
- export "MAVEN_ID=$(mvn help:evaluate -Dexpression=project.id -q -DforceStdout)"
- >
IFS=: read -r MAVEN_GROUPID MAVEN_ARTIFACTID MAVEN_PACKAGING MAVEN_VERSION <<< ${MAVEN_ID}
script:
- >
echo -e "groupId: ${MAVEN_GROUPID}\nartifactId: ${MAVEN_ARTIFACTID}\nversion: ${MAVEN_VERSION}\npackaging: ${MAVEN_PACKAGING}"
mvn help:evaluate -Dexpression=project.id -q -DforceStdout prints artifact identification information in the following format: com.group.id:artifactid:packaging:version
MAVEN_ID variable is parsed using IFS based on the colon (:) as a separator to get common maven variables like artifactId, groupId, version and packaging (explanation)
later these variables can be used in the code, for example for echoing values
IFS is a bash feature, thus corresponding GitLab runner should have bash installed
What follows is my buildspec.yml
build:
commands:
- 'IMAGE_TAG=$(cat package.json | grep version | head -1 | awk -F: ''{ print $2 }'' | sed ''s/[",]//g'')'
- echo $IMAGE_TAG
- docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
- docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
Here's the output from the relevant build:
[Container] 2018/12/12 22:06:42 Running command IMAGE_TAG=$(cat package.json | grep version | head -1 | awk -F: '{ print $2 }' | sed 's/[",]//g')
[Container] 2018/12/12 22:06:42 Running command echo $IMAGE_TAG <<< GOOD
1.0.0 <<<< PERFECT
[Container] 2018/12/12 22:06:42 Running command docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
invalid argument "gotbot-air:" for t: invalid reference format
See 'docker build --help'. <<<<<< OH NO
[Container] 2018/12/12 22:06:42 Command did not exit successfully docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG . exit status 125
As you can clearly see on the marked lines, somehow the variable $IMAGE_TAG is set correctly to 1.0.0 when echo'd, yet on the very next line of execution in my build script, it seems to have disappeared.
Please note I am using version 0.2 of the specification.
EDIT: It may be important that my other environment variables are either declared at the top in env or native code build variables, it might be that im getting some different execution environment when running the docker command?
Check the version of your BuildSpec file. Change it to version: 0.2.
In version 0.1, AWS CodeBuild runs each build command in a separate instance of the default shell in the build environment. In version 0.2, AWS CodeBuild runs all build commands in the same instance of the default shell in the build environment.
I am trying to generate an nyc/istanbul coverage report on my project (using mocha). I can get this to run from the command line with the following command:
npm test --reporter mocha-bamboo-reporter test/
which is essentially running
nyc mocha --recursive "mocha-bamboo-reporter" "test/"
The tricky part is that I need to run this in bamboo which doesn't seem to support chaining of/multiple commands in one task.
My bamboo config has a mocha test runner:
node_modules/mocha/bin/mocha --reporter mocha-bamboo-reporter --recursive
I have an istanbul/nyc instrument task:
./node_modules/nyc/bin/nyc.js instrument test/ .nyc_output
And then a coverage report task:
./node_modules/nyc/bin/nyc.js report
When this runs I get no data in the report file:
25-Apr-2018 14:27:28 ----------|----------|----------|----------|----------|-------------------|
25-Apr-2018 14:27:28 File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
25-Apr-2018 14:27:28 ----------|----------|----------|----------|----------|-------------------|
25-Apr-2018 14:27:28 All files | 0 | 0 | 0 | 0 | |
25-Apr-2018 14:27:28 ----------|----------|----------|----------|----------|-------------------|
Any help would be greatly appreciated.
Figured this out. The best way to do it was to create a custom script within my package.json file and then call that from Bamboo.
package.json
"scripts": {
"test-ci": "nyc -a --reporter=clover mocha --recursive --reporter mocha-bamboo-reporter"
Bamboo - Node.js task
run-script test-ci