I cannot make the gradle incremental feature work on a bitbucket pipeline with multiple steps.
What I like to have is:
A step that compile the code
A step that test the code being compiled (without a need to recompile)
Reading the doc I added the build folder and even the .gradle folder as artifacts of the compile step:
default:
- step:
name: Compile the application
caches:
- gradlew
script:
- ./gradlew core:compileJava
artifacts:
- .gradle/**
- build/**
- step:
name: Run the unit tests
caches:
- gradlew
script:
- ./gradlew core:test
Using the find command I can see the compiled classes and the .gradle files
into the unit test step.
However the unit test step always rerun the compilation process and using the -i option it outputs:
Task ':core:compileJava' is not up-to-date because:
Output property 'options.generatedSourceOutputDirectory' file /opt/atlassian/pipelines/agent/build/core/build/generated/sources/annotationProcessor/java/main has been removed.
Output property 'options.headerOutputDirectory' file /opt/atlassian/pipelines/agent/build/core/build/generated/sources/headers/java/main has been removed.
This seems like a trivial setup am I doing something wrong?
Related
I have a repository in which I want to separate the source code and documentation a little bit. So my directory structure looks like this:
/
- .gitlab-ci.yml
- readme.md
+ doc/
- doc_files.txt
+ src/
+ server/
- gradlew
- source_code.java
So from the root of the repo, I need to execute ./src/server/gradlew build to build the project.
Now I need to trigger gradle build via Gitlab-CI, i.e., I would execute said command exactly. This is what I imagine for my gitlab-ci.yml:
stages:
- build
server_build:
stage: build
image: gradle:7.5.1-jdk18-alpine
script:
- './gradlew build'
Anyhow, gradlew seems to not like being executed from any other directory and fails with the following:
> ./src/server/gradlew build
FAILURE: Build failed with an exception.
* What went wrong:
Directory '/home/user/git/project' does not contain a Gradle build.
A Gradle build should contain a 'settings.gradle' or 'settings.gradle.kts' file in its root directory. It may also contain a 'build.gradle' or 'build.gradle.kts' file.
//...
BUILD FAILED in 668ms
So I tried to set the directory with -p and/ or pointing with -b directly to the settings.gradle in src/server/ - anyhow, the I get the following errors:
> ./src/server/gradlew -b settings.gradle -p src/server/ build
FAILURE: Build failed with an exception.
* What went wrong:
A problem occurred configuring root project 'server'.
> Could not open proj generic class cache for build file '/home/user/git/project/src/server/settings.gradle' (/home/user/.gradle/caches/7.5.1/scripts/qpzgz2vuqu1er7i9j90pmbyq).
> BUG! exception in phase 'semantic analysis' in source unit '_BuildScript_' Unsupported class file major version 63
// ...
BUILD FAILED in 667ms
And, quite obviously, Gitlab-CI fails. So I tried to tell Gitlab-CI to cd into the directory /src/server/ before executing gradlew in the .gitlab-ci.yml:
cd src/server && ./gradlew build
Anyhow, this does not work either and fails like this (output from the runner's job in Gitlab-CI):
// ...
$ cd src/server && ./gradlew build
Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain
Caused by: java.lang.ClassNotFoundException: org.gradle.wrapper.GradleWrapperMain
Cleaning up project directory and file based variables
ERROR: Job failed: exit code 1
How can I tell Gitlab-CI to use gradlew build from the root of the repository given the existing directory structure?
So, I actually solved it by stumbling upon the solution by chance. To cut to the chase, I had to re-add the file gradle/wrapper/gradle-wrapper.jar, which was not uploaded due to .gitignore.
All in all, the gitlab-ci.yml looks like this:
stages:
- build
server_build:
stage: build
image: gradle:7.5.1-jdk18-alpine
script:
- cd src/server
- ./gradlew build
The gitlab-ci.yml is located in the root of the repository, the entire Java/Spring Boot-project is located in /src/server/. This includes the gradle.build, gradlew, and so on (specifically: everything you'd get out of the Spring Initializr https://start.spring.io/).
So after all, no path configurations or so were necessary.
Can you export GRADLE_USER_HOME in before_script of .gitlab-ci.yml file.
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
You can get reference configuration at https://gist.github.com/daicham/5ac8461b8b49385244aa0977638c3420#file-gitlab-ci-yml
You should try setting the setting.gradle file in you root directory and build.gradle in the src/server directory. The using the -p should work fine.
I have a test project that is built with Maven (Java). I can either execute the test from IntelliJ manually or from the command line by writing mvn test.
I put the project on CircleCI and it generated a yml file. And it was able to execute the tests on pipelines as well without any issue at first
Then I made something stupid. Initially tests were in this root: src/main/java/api/test. But I decided to move the test class to this root: src/test/java/api.
I did this change because src/test/java was the actual test folder created automatically when you create a Maven project. (The other test folder was created by me manually so I thought this was not the best practice and therefore decided to move the test class to src/test/java. Basically what I did is, I created a package named api under src/test/java and moved the test class to this package. After that I deleted src/main/java/api/test as it is empty now.
After this change, I didn't observe any issue. I could still run the tests from IntelliJ or from the command line by mvn test command. But after I commit my changes, I just checked the pipelines and saw this:
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.793
My project is still built and tested on CircleCI but it obviously does not execute the test class. I dont know why it is happening. I checked the yml file and there is nothing related with paths and it was working before. Here is my yml file:
# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1
jobs:
# Below is the definition of your job to build and test your app, you can rename and customize it as you want.
build-and-test:
# These next lines define a Docker executor: https://circleci.com/docs/2.0/executor-types/
# You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# Be sure to update the Docker image tag below to openjdk version of your application.
# A list of available CircleCI Docker Convenience Images are available here: https://circleci.com/developer/images/image/cimg/openjdk
docker:
- image: cimg/openjdk:11.0
steps:
# Checkout the code as the first step.
- checkout
# Use mvn clean and package as the standard maven build phase
- run:
name: Build
command: mvn -B -DskipTests clean package
# Then run your tests!
- run:
name: Test
command: mvn test
workflows:
# Below is the definition of your workflow.
# Inside the workflow, you provide the jobs you want to run, e.g this workflow runs the build-and-test job above.
# CircleCI will run this workflow on every commit.
# For more details on extending your workflow, see the configuration docs: https://circleci.com/docs/2.0/configuration-reference/#workflows
sample:
jobs:
- build-and-test
As I said the same yml file was working fine before I moved the test class. Now it probably can not locate my test file. What could be the problem here and how can I solve this? Any help is appreciated.
I have a gitlab pipeline running on a windows machine with Windows 7 and powershell 4.0.
The .yaml has the typical 3 stages: build, test and deploy.
For the second stage I want to perform some simple tests that generate a log file which should be available after the test stage finishes.
Here the script section from the test:
script:
- '$exitCode = (start-process C:\app_versions\app_20181211\bin\app.exe -PassThru -Wait).ExitCode'
- 'cat .\TestLogs\BasicFunctionsTestPlan.log'
- 'exit $exitCode'
artifacts:
paths:
- .\TestLogs
expire_in: 1 year
Here I had one problem, after the test run has finished the stage finishes always successfully even if the test themselves failed. Then I had to force the script exit with an error code in case the application tells me that the tests failed.
This caused the second problem: the artifacts link do not get created even they are available (my test produce it anyway).
Probably if I knew how to tell gitlab that the test failed in a more clean way, the artifacts would be available anyway.
I agree that the log file is not an artifact but I would like to keep that file in order to check how the tests have performed, maybe there is a better way to save this file.
Thanks in advance for your help!
EDIT:
Looks like there were more people having the same issue here, maybe it helps understanding better the problem.
I had the same question, but it's easily solved:
You can use artifacts:when to upload artifacts on job failure or despite the
failure.
artifacts:when
source: Gitlab CI yaml reference: artifacts:when
Introduced in GitLab 8.9 and GitLab Runner v1.3.0.
artifacts:when is used to upload artifacts on job failure or despite the
failure.
artifacts:when can be set to one of the following values:
on_success - upload artifacts only when the job succeeds. This is
the default.
on_failure - upload artifacts only when the job
fails.
always - upload artifacts regardless of the job status.
Example:
To upload artifacts only when job fails:
job:
artifacts:
when: on_failure
allow_failure
BTW: you can tell Gitlab CI to continue to the next job after a job failure with allow_failure: true
source: Gitlab CI yaml Reference: allow_failure
job1:
stage: test
script:
- execute_script_that_will_fail
allow_failure: true
So combined it could look something like:
job1:
stage: test
script:
- execute_script_that_will_fail
allow_failure: true
artifacts:
when: always # or 'on_failure'
paths:
- resulting_artifacts
I have the following build configuration for a multi-project gradle file:
stages:
- test
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
cache:
paths:
- .gradle/wrapper
- .gradle/caches
- build
test :
dependencies: []
image: openjdk:x
stage: test
script:
- ./gradlew test --debug
On the GitLab, between builds with no changes to source files, I get:
Up-to-date check for task ':x:compileJava' took 1.117 secs. It is not up-to-date because:
No history is available.
I'm not sure why it says this, as I would expected the task history to be restored from cache. I see this in the logs between runs:
Creating cache default...
.gradle/wrapper: found 207 matching files
.gradle/caches: found 5058 matching files
build: found 2743 matching files
When I re-run on my local machine, I can see the tests are not being re-run:
> Skipping task ':x:compileJava' as it is up-to-date (took 0.008 secs).
More confusing is dependencies are cached perfectly, it just keeps rerunning tests when I have made no code changes.
As far as I know the history that gradle is missing is also stored in the .gradle folder, but not in the caches or wrapper subfolder. If you tell Gitlab to cache the complete .gradle folder the problem should go away.
See also this example:
https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Gradle.gitlab-ci.yml
No matter what I write to .travis.yml Travis CI always executes ./gradlew assemble. I would like to run ./gradlew test instead.
Contents of my .travis.yml:
language: java
script:
- ./gradlew test
Feel free to fix my failed build:
https://github.com/RadoBuransky/scala-struct
The script ./gradle assemble is run as part of the install section of your build, it's the default.
If you want to skip this step, you can override it like so:
install: true
Docs: https://docs.travis-ci.com/user/customizing-the-build/#Skipping-the-Installation-Step