I am trying to dynamically build a bash command to be run in a github action. My step looks like this:
- name: Build PREPROD
run: |
# Generate dart-define values from the preprod.env file
DART_DEFINES_PREPROD="$(scripts/some_script.sh)"
echo $DART_DEFINES_PREPROD
# Build the apk
flutter build apk ${DART_DEFINES_PREPROD}
What my script in scripts/some_script.sh does, is convert the content of an environment file like this one:
APP_NAME=nameOfMyApp preprod
APP_SUFFIX=.preprod
into bash arguments that look like this (this is what appears when the echo $DART_DEFINES_PREPROD is run in my job
--dart-define APP_NAME="nameOfMyApp preprod" --dart-define APP_SUFFIX=".preprod"
So to me, it should be running a command like
flutter build apk --dart-define APP_NAME="nameOfMyApp preprod" --dart-define APP_SUFFIX=".preprod"
which is what I want, and is valid when I run this locally on my machine.
Yet, when the step pasted above is ran on the github Action agent, the result is the following error:
Target file "preprod"" not found.
Error: Process completed with exit code 1.
And I just don't understand what happens here. I've tried many other syntaxes that all produces the same result.
I know the issue is coming from the space in my APP_NAME variable, because if I replace it by something else without any space, my step is working fine. But since that value is included in double quotes, I juste don't understand why it is interpreted as two commands, instead of juste one information.
Any help / clue is appreciated !
Thanks !
Update 1:
Ok so following #"Benjamin W." comment, I read your links (thanks) and I think I'm doing the right thing, yet it still fails. Here is my updated code:
# For clarity I removed the .sh script file to show what I'm doing inside of it
dart_defines_args=()
input="scripts/envs/preprod.env"
while IFS='=' read -r name value
do
dart_defines_args+="--dart-define "
dart_defines_args+="$name=\"$value\" "
done < "$input"
echo "${dart_defines_args[#]}"
# Build the apk
flutter build apk ${dart_defines_args[#]}
I also did the same test with
flutter build apk "${dart_defines_args[#]}"
But I still end up with a
Target file "--dart-define APP_NAME="nameOfMyApp preprod" --dart-define APP_SUFFIX=".preprod"
or
Target file "preprod"" not found.
depending on whether or not I have double quotes around ${dart_defines_args[#]}
I think the problem is about variable expansion.
Try change:
flutter build apk ${DART_DEFINES_PREPROD}
for
flutter build apk "${DART_DEFINES_PREPROD}"
I am trying to execute multiple windows batch commands in jenkins one after the other. The problem is that if any the of the project/build fails, it never gonna execute next windows batch commands.
My question is How to execute next windows batch command if previous execution fails?
Help me with this regard.
When you say "multiple windows batch commands", do you mean:
Multiple Execute Windows Batch Command build steps
Or multiple lines of commands within a single build step?
If configuring multiple build steps, you just need to make sure that the last command of the build step does not return anything other than 0. You can do this by adding either of the following as the very last statement in your build step:
either exit /b 0
or echo "All done"
As for multiple lines within the same build step, default implementation of Execute Windows Batch Command does not break if one line/statement fails (which is different from default Execute Shell implementation). As long as the very last statement returns 0, the build step will not fail, and any lines failing in between does not matter.
Once again, you can reference to the above list to make sure that the last line always returns 0
Re "if any the of the project/build fails"
Do you mean "if any of the batch commands fails"?
See Conditional BuildStep Plugin.
In your job config scroll down to:
Build
[ For each command that can fail Add build step at the bottom of this Build section ]
Conditional step (single)
Run? | Not
! | Execute Windows batch commands
Commands | ... your commands ...
[ Click Advanced... ]
On evaluation failure | Fail the build
Builder | Set the build result
Result | Success
Or add just one Conditional step (single) section and write all of your commands into:
Commands | ... your commands ...
Or maybe Conditional step (multiple) is the way to go for you. I haven't used this so far, so I'm not much of a help there.
I want to pass some information to another build step. E.g. for build tagging. How do I do that?
I've tried ##teamcity[setParameter name='xxx' value='111'] in my script, but it doesn't seem to do anything.
Well, first you need to define custom parameter in Build configuration -> Parameters. Then you should set it like in question, but with one nuance: you should echo command! And there isn't a word about that in docs :(
In the end, you need to do this: echo ##teamcity[setParameter name='xxx' value='111'] in your script, and then, in next build step you could use it as usual Teamcity variable: %xxx%.
P.S. xxx would be initialized only in next build step, so don't use it for anything else but setting value in this build step.
The horror stories I found while searching for an answer for this one...
OK, I have a .sh script which pretty much does everything Jenkins supposed to do:
checks out sources from SVN
build the project
deploys the project
cleans after itself
So in Jenkins I only have to 'build' the project by running the script in an Execute Shell command.
The script is ran (the sources are downloaded, the project is build/deploy) but then it marks the build as a failure:
Build step 'Execute shell' marked build as failure
Even if the script was successfully ran! I tried closing the script with:
exit 0 (still marks it as failure)
exit 1 (marks it as failure, as expected)
no exit command at all (marks it as failure)
When, how and why does Execute Shell mark my build as a failure?
First things first, hover the mouse over the grey area below. Not part of the answer, but absolutely has to be said:
If you have a shell script that does "checkout, build, deploy" all by itself, then why are you using Jenkins? You are foregoing all the features of Jenkins that make it what it is. You might as well have a cron or an SVN post-commit hook call the script directly. Jenkins performing the SVN checkout itself is crucial. It allows the builds to be triggered only when there are changes (or on timer, or manual, if you prefer). It keeps track of changes between builds. It shows those changes, so you can see which build was for which set of changes. It emails committers when their changes caused successful or failed build (again, as configured as you prefer). It will email committers when their fixes fixed the failing build. And more and more. Jenkins archiving the artifacts also makes them available, per build, straight off Jenkins. While not as crucial as the SVN checkout, this is once again an integral part of what makes it Jenkins. Same with deploying. Unless you have a single environment, deployment usually happens to multiple environments. Jenkins can keep track of which environment a specific build (with specific set of SVN changes) is deployed it, through the use of Promotions. You are foregoing all of this. It sounds like you are told "you have to use Jenkins" but you don't really want to, and you are doing it just to get your bosses off your back, just to put a checkmark "yes, I've used Jenkins"
The short answer is: the exit code of last command of the Jenkin's Execute Shell build step is what determines the success/failure of the Build Step. 0 - success, anything else - failure.
Note, this is determining the success/failure of the build step, not the whole job run. The success/failure of the whole job run can further be affected by multiple build steps, and post-build actions and plugins.
You've mentioned Build step 'Execute shell' marked build as failure, so we will focus just on a single build step. If your Execute shell build step only has a single line that calls your shell script, then the exit code of your shell script will determine the success/failure of the build step. If you have more lines, after your shell script execution, then carefully review them, as they are the ones that could be causing failure.
Finally, have a read here Jenkins Build Script exits after Google Test execution. It is not directly related to your question, but note that part about Jenkins launching the Execute Shell build step, as a shell script with /bin/sh -xe
The -e means that the shell script will exit with failure, even if just 1 command fails, even if you do error checking for that command (because the script exits before it gets to your error checking). This is contrary to normal execution of shell scripts, which usually print the error message for the failed command (or redirect it to null and handle it by other means), and continue.
To circumvent this, add set +e to the top of your shell script.
Since you say your script does all it is supposed to do, chances are the failing command is somewhere at the end of the script. Maybe a final echo? Or copy of artifacts somewhere? Without seeing the full console output, we are just guessing.
Please post the job run's console output, and preferably the shell script itself too, and then we could tell you exactly which line is failing.
Simple and short answer to your question is
Please add following line into your "Execute shell" Build step.
#!/bin/sh
Now let me explain you the reason why we require this line for "Execute Shell" build job.
By default Jenkins take /bin/sh -xe and this means -x will print each and every command.And the other option -e, which causes shell to stop running a script immediately when any command exits with non-zero (when any command fails) exit code.
So by adding the #!/bin/sh will allow you to execute with no option.
In my opinion, turning off the -e option to your shell is a really bad idea. Eventually one of the commands in your script will fail due to transient conditions like out of disk space or network errors. Without -e Jenkins won't notice and will continue along happily. If you've got Jenkins set up to do deployment, that may result in bad code getting pushed and bringing down your site.
If you have a line in your script where failure is expected, like a grep or a find, then just add || true to the end of that line. That ensures that line will always return success.
If you need to use that exit code, you can either hoist the command into your if statement:
grep foo bar; if [ $? == 0 ]; then ... --> if grep foo bar; then ...
Or you can capture the return code in your || clause:
grep foo bar || ret=$?
I 've tried all mentioned options (even changing sh to bash without -xe params), the only one option worked for me is:
<command-which-returns-not-zero> || exit 0
Plain and simple:
If Jenkins sees the build step (which is a script too) exits with non-zero code, the build is marked with a red ball (= failed).
Why exactly that happens depends on your build script.
I wrote something similar from another point-of-view but maybe it will help to read it anyway:
Why does Jenkins think my build succeeded?
So by adding the #!/bin/sh will allow you to execute with no option.
It also helped me in fixing an issue where I was executing bash script from Jenkins master on my Linux slave. By just adding #!/bin/bash above my actual script in "Execute Shell" block it fixed my issue as otherwise it was executing windows git provided version of bash shell that was giving an error.
Try and always find where exactly its failing by adding the following line into your "Execute shell" Build step.
#!/bin/sh -xe
By adding the -x you will print each and every command that ran (including the lines from embedded scripts) and that will help in spotting the root cause.
Removing the -e option i.e. running #!/bin/sh will allow you to execute with no option, which is really a bad idea as Bryan explained well in one of the answers.
The problem is with no option Jenkins will ignore errors and continue execution of subsequent steps (if there are any) which will leave your process in an consistent state. If this is for a production build or deployment, that may have a bad impact.
Once you find the problem area, run the same failing command from the directory as jenkins-user manually, to get to the exact error/rootcause.
In Jenkins ver. 1.635, it is impossible to show a native environment variable like this:
$BUILD_NUMBER or ${BUILD_NUMBER}
In this case, you have to set it in an other variable.
set BUILDNO = $BUILD_NUMBER
$BUILDNO
I must be doing something differently than what was asked and answered here because the solution does not appear to be working for me: TeamCity, passing an id generated in one build step to a later build step
I want use a string generated by one build step in a final build step. So far I have set up an environmental variable called "TEST" that is empty. Both build steps use the Command Line Runner.
Build Step #1:
\##teamcity[setParameter name='env.TEST' value='test']
Build Step #2:
echo $TEST
echo %env.TEST%;
Placeholder for now, but if I could access the test string ('test') set in Build Step 1 I would be so happy.
you need to echo that string, e.g.
echo "##teamcity[setParameter name='env.TEST' value='test']"
What I found is that with long values, as soon as TeamCity is breaking down the log output into two separate lines echo will not work anymore - you have to use Write-Host instead.
Write-Host "##teamcity[setParameter name='env.TEST' value='test']"
This should always work, just a side note - this value will be available only on subsequent build steps.
I think you have an extra "\" in there. Try removing that and add double quotes around it and it should work.
"##teamcity[setParameter name='env.TEST' value='test']"
If it doesn't work try using Powershell runner type as I'm using that for setting it and it works.
To expand on the above answers, with powershell it would look like so in build step 1:
Write-Host "##teamcity[setParameter name='env.TEST' value='$test']"
...and you can use the value like this in step 2:
echo %env.Test%
Also as a note, you'll have to set env.Test in the TC build parameters to be equal to something. I just used a space since I know the value will be set via ps script. Hope this helps.
It has to be printed to STDOUT, I use cat with heredoc to avoid having to escape single quotes in the event of using variables to dynamically set config parameters. What is heredoc?
MYVARNAME=MYVALUE
cat <<EOF
##teamcity[setParameter name='myConfParameter' value='$MYVARNAME']
EOF
Result:
##teamcity[setParameter name='myConfParameter' value='MYVALUE']
Documentation
Here is official ticket about addition double quotes and echo (Write-Host - OS dependency).