Retry failed xcodebuild test - bash

I am building custom Jenkins build script (sh) for iOS app build/test checks.
But sometimes UI test fails just because of timing issue, so I want it to re run few more times to make sure the issue is real.
for (( ATTEMPT=1; ATTEMPT<=2; ATTEMPT++ ))
do
xcodebuild [flags] test #add_result_saving_mechanism
#if failed, do smth to go to next attempt. Else - break
if SOME_KIND_OF_FAIL_CHECK; then
continue
else
break
fi
fi
I used xcpretty before, so was able to read $PIPESTATUS and react accordingly, but xcpretty is crashing for xcodebuild test for some reason, so looking ways to do without it
xcodebuild [flags] test | xcpretty
STATUS="${PIPESTATUS}"
if [ "$STATUS" != "0" ]; then
FAILURE_MSG="${TARGET} (${BUILD_NAME}) failed UI/Unit testing"
#try next attempt if available
continue
else
break
fi
How can I handle these retries without pipes/xcpretty?

From the jenkins perspective - it always used to bail early if a script encountered an error, so you could try this type of syntax to prevent that (in the jenkins config this is, not in the script itself)
jenkins_build_script.sh || true
# continue with things...
Also, if you're having trouble capturing the failure itself - try piping the xcodebuild output to a log file and then grep-ing for the errors you're anticipating.

Since you said it's a script run by Jenkins you can handle the retry from Jenkins pipeline instead of inside the shell script.
As in the example from the docs
pipeline {
agent any
stages {
stage('Deploy') {
steps {
retry(3) {
sh './flakey-deploy.sh'
}
}
}
}
}
You can read about it here
Hope this will help good luck.

Related

Can cypress CLI distinguish between an internal error and exactly one failing test?

From the cypress run docs:
"When Cypress finishes running tests, it exits. If there are no failed tests, the exit code will be 0. If there are any test failures, then the exit code will match the number of tests that failed. If Cypress could not run for some reason (for example if no spec files were found) then the exit code will be 1."
Because we want to retrieve the number of failed tests for use later in our Azure DevOps pipeline and also cause the pipeline to fail if cypress errors out for some internal reason, I'm using the following bash script:
run_cypress () {
node_modules/.bin/cypress run $(SpecFile) --browser chrome --headless
case $? in
1)
return 1
;;
0)
# Set some pipeline variables
return 0
;;
*)
NUM_TESTS=$?
# Set some pipeline variables
return 0
;;
esac
}
run_cypress
We are now running into the entirely predictable corner case where we have a test suite that has exactly one failing test. This is causing the whole pipeline to fail when it shouldn't.
Is there any way to reliably distinguish between "cypress has returned 1 because of a fatal error" and "cypress returned 1 because one test failed"?
You could check if the test results file exists, which should indicate that the test suite ran all the way through.
So, if the exit code is 1 and there are no results file present, consider it a fatal error.
npx cypress run --browser chrome
if [ $? -eq 1 ] && [ ! -f reports/result.json ]; then
echo "Fatal error"
exit 255;
fi

How to force successful build if post test fails in circleci 1.0

In my circle.yml file I have a post test that runs after the normal tests, and only when you are building master. I am trying to find a way to alert if this post test succeeds or fails, but have the build pass regardless of success or failure. Note that the build should still fail if any of the tests in the normal test suite fail. It is only this post test that I wish to see test failure for, but still have the build succeed.
test:
post:
- |
if [ master == $CIRCLE_BRANCH ]; then
npm run extra-tests
fi
disclaimer: CircleCI Developer Evangelist
You can do the following:
test:
post:
- |
if [ master == $CIRCLE_BRANCH ]; then
npm run extra-tests || true
fi
The double pipe is an "or" in Bash. If the command to the left succeeds (exit code 0), then we move onto the next line, ending the if block. If it fails, then the command to the right of || runs, which always succeeds.
Just be careful as you'll only know if any of these "extra tests" fail by logging into CircleCI's website, viewing the build, and expanding the build output for that section.

Executing Maven task from shell script and getting error codes

I'm executing a Maven deploy task from a bash script however even if the Maven task fails the script will continue and complete without errors.
I have tried the -e flag but that causes the deploy to fail. I also tried the following (pseudo code)
result_code= mvn deploy
if [$result_code -gt 0];then
exit 1
Any suggestions how i can identify if the deploy was successful?
result_code=mvn deploy is not the way to get return status
you can try e.g. :
#!/bin/bash
mvn deploy
STATUS=$?
if [ $STATUS -eq 0 ]; then
echo "Deployment Successful"
else
echo "Deployment Failed"
fi
In addition if anyone using Windows 10, here is the example which I use:
mvn deploy
if not %ERROR_CODE%==0 goto error
echo SUCCESS
goto end
:error
echo FAILED
:end
Just one other possible reason a person's mvn task may return 0 despite failing: be careful about piping the output of maven to other programs. For example, I'm using grcat (which grc is build on top of), which will always return exit code 0;
\mvn compile | grcat ~/conf.mvn
I'm not sure how to retain my nice color coding of the output. There is color_maven.sh out there but that has other issues.

How to stop Xcode build from shell script

I am currently working on an app in Xcode where I've added a Build Phase that runs a shell script.
The script looks for resources from the Desktop and copies them to the app . If the files/folders don't exist, the script should cancel the build of the app.
I have tried various things to stop the build such as xcodebuild clean but I can't quite figure it out. Here is what I have:
if [ -d ~/Desktop/MyFolder ]; then
cp -r ~/Desktop/MyFolder ${CONFIGURATION_BUILD_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}/MyFolder
else
#Stop the build
fi
Is there a way to have the script tell Xcode to stop the build? If so, how can I do it?
You need to return a non-zero exit code from the script:
exit 1
Another useful technique is for the script to produce an output with an "error: " prefix.
This will cause Xcode to show the error within the build logs, and if you Xcode settings stop building on failure - it will stop right there.
In addition to this, you could also trigger a warning by printing out "warning: " prefix.
Example:
if [ -d ~/Desktop/MyFolder ]; then
cp -r ~/Desktop/MyFolder ${CONFIGURATION_BUILD_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}/MyFolder
else
echo "error: Copy failed!"
fi

How to abort build if run script phase gives an error

I would like to use a run scripts target to tag my revision in mercurial and upload it to a server. I created a new run scripts target that depends on the other target building my app, then I added two run script phases directly one after another.
Now my question: how can I prevent executing run script phase #2 if run script phase #1 gives an error (return code of script is unequal 0)?
The second script would upload the files to a server, so I only want to execute this phase if everything went right until then.
Your first script can abort the build with "exit 1", like this:
if [ error ] then
echo "There are some errors, etc..."
exit 1
fi
A solution would be to generally stop building when errors occur:
XCode -> Preferences -> Building -> Build options "Continue building after errors".
If you uncheck that, the build will stop if your script returns something unequal 0 and the second script will not be executed. If you use xcodebuild, the name for the setting is "PBXBuildsContinueAfterErrors".
exit 1 aborts the build.
echo "error: will flag it up as an error within Xcode.
echo "error: Everything has failed!"
exit 1

Resources