I'd like to know whether there's a way to kind of 'query' the state of an Maven execution from within a Shell script (used for build process).
The point is that I would like to let the whole build script fail as soon as a single error appears within one of the Shell scripts Maven executions.
e.g.
(0) mvn -f .../someDir clean
(1) mvn -f .../1/pom.xml install
(2) mvn -f .../2/pom.xml -PgenerateWadl
So if e.g. there occurs an error within (0), then (1) and (2) must no more be executed, but instead the build script should quit with an error message directly after (0).
I'm not THAT much into Shell scripting, but I know of the $? variable to get the return value of an earlier execution. But as Maven simply seems to write errors out to the console, this might not work, does it?
I would have liked to research more information concerning the "$?", but it's quite hard to google for it.
A simple way is to use the -e option.
set -e
mvn -f .../somedir clean
mvn -f .../otherdir install
mvn -f .../thirddir -PgenerateWadl package
This will automatically have the shell exit with error if any command in sequence exits with a non-zero status (unless it is part of an explicit test such as an if or while or a chain like a || b).
You can watch this happen by inserting a call to false between any of the Maven calls.
See the set POSIX spec for details on set -e.
mvn ... && mvn ... && mvn ...
The execution will only proceed if the previous one was successful.
Related
I am running cucumber java tests in jenkins job using maven build as below
After its done in post step under windows batch command execution script I am rerunning failures
mvn clean verify -Dcucumber.options="#rerun.txt --tags #Retry1" -Dthreadcount="2"
After this step every time execution fails. I have read somewhere that if I add any of the below line it should pass but still it fails every time.
echo "exit script"
true
command || true
exit 0
Also added #!/bin/sh at start of script
Please advise.
Since this is supposed to be executed on Windows you may try
mvn clean verify -Dcucumber.options="#rerun.txt --tags #Retry1" -Dthreadcount="2" & echo note: ignoring exit code
or
cmd /c mvn clean verify -Dcucumber.options="#rerun.txt --tags #Retry1" -Dthreadcount="2" & echo note: ignoring exit code
(sorry, don't have Jenkins with Windows agent to test this at hand).
I am using Azure Devops to build and deploy my git repo to a third party vps. I do this by logging into the server from Azure Devops through SSH, executing a shell script to pull git repo, and build it with ie. vue-cli and Laravel.
When the bash script is executed I receive a lot of errors on nearly all commands although everything is succeeding - can anyone tell me how to get rid of these unless something is really failing (would be nice to fail if npm build exit with code 1 for instance).
See screenshot below.
Screenshots are only really helpful for visual issues. You can use PasteBin or etc to share long logs if necessary.
According to this issue Azure just follows the lead of whatever shell it's running code in. So, in Bash it continues unless explicitly told to stop.
To easily change this behavior you can add set -e (or set -o errexit) at the start of your script. The errexit option causes Bash to exit as soon as a command/etc returns a non-zero exit code.
Another worthy addition is the set -o pipefail option. If you've got any pipes like command1 | command2 this will return the first non-zero exit code from a chain of pipes of any length as the result. So, if command1 fails above but command2 succeeds it would return the failure code from command1 instead of overwriting it.
Finally, set -u (or -o nounset) causes an error when unset variables are encountered during parameter expansion. If running in a non-interactive shell, it will also exit.
Many scripts combine these by running set -euo pipefail at the beginning to stop them from running after the first problem is encountered.
If you want to explicitly force a bash script to exit you can use || and && accordingly. The expression command || exit will exit if the command fails and command && exit will exit if the command succeeds.
This seems to be one bug starting from npm V.3.10.8. You can check this discussion.
As a workaround you can add this script to package.json and run the command with --no-optional switch:
"optionalDependencies": {
"fsevents": "*"
},
Also, there's possibility that your NPM version is too old. You can use Node.js tool installer task with version spec = 12.x to install higher node.js and npm versions.
In a TeamCity job I'm trying to read Maven project version from the root pom.xml e.g. 1.0.0-SNAPSHOT and change the build number to 1.0.0-N where N is a value of %build.counter%. The goal is to replace SNAPSHOT with a build identifier.
So far I tried to combine build-helper:parse-version together with exec:exec to call echo to write ##teamcity[buildNumber '1.0.0-1'] in the build log and change the build number.
On my workstation I can execute following Maven command to print the ##teamcity instruction in the command line:
mvn build-helper:parse-version exec:exec \
-Dexec.executable="echo" \
-Dexec.args="##teamcity[buildNumber '${parsedVersion.majorVersion}.${parsedVersion.minorVersion}.${parsedVersion.incrementalVersion}-%build.counter%']"
However when I define a Maven Step in my TeamCity build then TeamCity parses the ##teamcity instruction directly from the Maven command line invocation printed in the build logs resulting in following Maven call:
mvn build-helper:parse-version exec:exec \
-Dexec.executable="echo" \
-Dexec.args=""
and following being done by TeamCity:
##teamcity[buildNumber '${parsedVersion.majorVersion}.${parsedVersion.minorVersion}.${parsedVersion.incrementalVersion}-11']
Is there a way to escape the parameter containing ##teamcity in the Maven command line invocation so TeamCity ignores it during Maven invocation but passes it to exec:exec Maven goal so it can output:
##teamcity[buildNumber '1.0.0-11']
Please let me know if there is an easier way to achieve my goal using TeamCity Build Steps and without modifying project's pom.xml. Do I have to use PowerShell or Batch script for this?
Interesting use case. I tried experimenting with this and got the positive result.
The echo command was omitting the single quote in the final output so I never managed to get TeamCity to consume the service message.
Instead of calling the Maven build step, I defined Command Line build step that launched a script. The scripts task is to generate the desired output:
##teamcity[buildNumber '1.0.0-61']
The script itself is really ugly, but kinda works. Perhaps you can come up with more elegant solution:
#!/bin/bash
export ARGUMENTS="teamcity[buildNumber 'Q\${parsedVersion.majorVersion}.\${parsedVersion.minorVersion}.\${parsedVersion.
mvn build-helper:parse-version exec:exec -Dexec.executable="echo" -Dexec.args="$ARGUMENTS" | grep teamcity | sed "s/Q/\'/g" | awk '{print "##"$0}'
For some reason, if I put ## into the initial string and didn't use awk at the end of the command sequence, then it didn't work and I ended up with a result like '1.0.0-???', hence the hack with awk at the end.
Hope this helps.
I've been trying to find ways to cut my Jenkins build time as much as possible, and thanks to this helpful SO post, I found pbzip2: Utilizing multi core for tar+gzip/bzip compression/decompression
Works great! A 6 min compression time was brought down to 2 mins on my machine with the following:
tar -v -c --use-compress-program=pbzip2 -f parallel.tar.bzip2 myapplication.app
But Jenkins just barfs with a Execute Shell task where I put in the above command:
+ tar -v -c --use-compress-program=pbzip2 -f parallel.tar.bz2 myapplication.app
a myapplication.appBuild step 'Execute shell' marked build as failure
The fact that the "Build step" line is getting mashed together with the output from the tar tells me it might be a background process issue that tar/pbzip2 is introducing.
I've tried introducing a #!/bin/bash -xe and get the same results. I've tried wrapping the tar command in an if statement. I've also tried putting tar in a background thread itself with & and waiting for it. Same result.
Is there any techniques I could implement to help the Jenkins process out?
Found out that even though I can run this command as the jenkins user through command line, pbzip2 wasn't defined in the PATH for the Jenkins job. Pretty misleading since there wasn't useful output.
I am writing a bash script to automate the build process. There are two major build blocks, one is an ant task and one is a plain old mvn clean install. I want to do something when there is build error coming from either of this two build processes.
The problem is, these builds will contain test failures or errors from time to time, but the end result is successful. And I believe that the status code ($?) return by these processes should be 0 no matter the build fail or succeed, I could be wrong.
So what is the best way for my script to detect the end result (build fail/succeed) without catching the false info during the mid build (test errors, etc) from them?
mvn clean test
if [[ "$?" -ne 0 ]] ; then
echo 'could not perform tests'; exit $rc
fi
$? is a special shell variable that contains the exit code (whether it terminated successfully, or not) of the most immediate recently executed command.
-ne stands for "not equal". So here we are testing if the exit code from mvn clean is not equal to zero.
There are a few issues against Maven 2 returning incorrect return codes (i.e. always returning 0). Notably MNG-3651 that was fixed in Maven 2.0.9.
In older versions, mvn.bat ended with this line:
exit /B %ERROR_CODE%
From Maven 2.0.9 onwards, the last line was changed to this:
cmd /C exit /B %ERROR_CODE%
So a non-0 return code is returned if the build fails. In the case of a build ERROR the return code is 1. If you are unable to upgrade to 2.0.9+, you could consider modifying mvn.bat as above to return the correct code.
Correct solution for unix/linux:
mvn clean install
rc=$?
if [ $rc -ne 0 ] ; then
echo Could not perform mvn clean install, exit code [$rc]; exit $rc
fi
The "if" statement itself is a command and if it is successful, it will reset the $? variable to 0. Same goes for echo. So, you have to use an intermediary local var, for example $rc to store the return code from "mvn clean install", then it can be passed to the "exit" command as well.
According to the Ant manual:
the ant start up scripts (in their Windows and Unix version) return the return code of the java program. So a successful build returns 0, failed builds return other values.
Maven also returns a non-zero exit code on error. Here's a link showing how to interrogate this status using the Maven Invocation API.
So it seems to me that you should be able to explicitly handle the return codes in your script
. Presumably you can ignore error codes relating to tests etc. if those are not a concern to you.
exec error codes in Ant are operating system-specific. These may help you:
a list of error codes for Linux
a list of error codes for Windows
Here is exactly what I do to get the result you want.
<exec executable="${env.M2_HOME}/bin/mvn" dir="${basedir}"
failonerror="true" osfamily="unix">
<arg value="-DskipTests=${argSkipTests}"/>
<arg value="-Doffline=${argOffline}"/>
<arg line="${projectsLine}"/>
<arg line="${resumeFromLine}"/>
<arg line="${alsoMakeLine}"/>
<arg line="${alsoMakeDependentsLine}"/>
<arg line="${commandsLine}"/>
</exec>
There's a command built-in to bash which performs exactly this.
# exit when any command fails
set -e
I put this at the top of my bash scripts, which I copied from this excellent resource.
# keep track of the last executed command
trap 'last_command=$current_command; current_command=$BASH_COMMAND' DEBUG
# echo an error message before exiting
trap 'echo "\"${last_command}\" command filed with exit code $?."' EXIT