I'm using shell script to run protractor tests.
I want to make sure that if the test fails (exit code != 0) then it will run again - three times most.
I'm already using Teamcity, but Teamcity sends the 'FAIL' email and only then tries again. I want the test will run three times before sending a message.
this is part of my script:
if [ "$#" -eq 0 ];
then
/usr/local/bin/protractor proactor-config.js --suite=sanity
now I want to somehow check whether the Exit Code was 0 and of not - run again.
Thanks.
I wrote a small module to do this called protractor flake. It can be used via the cli
# defaults to 3 attempts
protractor-flake -- protractor.conf.js
Or programatically.
One nice thing here is that it will only re-run failed spec files, instead of your test suite.
There is a long standing feature request for this in the protractor issue queue. It probably won't be baked into the core of the framework.
function to check status
function test {
"$#"
local status=$?
if [ $status -ne 0 ]; then
echo "error with $1" >&2
fi
return $status
}
test command1
test command2
If you use protractor with cucumber-js, you can choose to give each scenario (or all scenarios tagged as unstable) a number of retries:
./node_modules/cucumber/bin/cucumber-js --help
...
--retry <NUMBER_OF_RETRIES> specify the number of times to retry failing test cases (default: 0) (default: 0)
--retryTagFilter <EXPRESSION> only retries the features or scenarios with tags matching the expression (repeatable).
This option requires '--retry' to be specified. (default: "")
Unfortunately if every failed scenario has been successfully retried, Protractor will still return with exit code 1:
https://github.com/protractor-cucumber-framework/protractor-cucumber-framework/issues/176
As a workaround, when starting the protractor I append the following to its command line:
const directory = 'build';
ensureDirSync(directory);
const cucumberSummary = join(directory, 'cucumberSummary.log');
protractorCommandLine += ` --cucumberOpts.format=summary:${cucumberSummary} \
|| grep -P "^(\\d*) scenarios? \\(.*?\\1 passed" ${cucumberSummary} \
&& rm ${cucumberSummary}`;
Related
A branch job in which the branch property of the trigger property is using a variable will always fail with reason: downstream pipeline can not be created.
Steps to reproduce
Set up a downstream pipeline with a trigger property as you would normally.
Add a branch property to the trigger property. Write the name of an existing branch on the downstream repository, like master/main or the name of a feature branch.
Run the pipeline and observe that the downstream pipeline is successfully created.
Now change the branch property to use a variable instead, like branch: $CI_TARGET_BRANCH.
Manually run the CI pipeline with that, setting variable through the GitLab GUI.
The job will instantly fail with reason: downstream pipeline can not be created.
Code example
The goal is to create a GitLab CI config that runs the pipeline of a specified downstream branch. The bug occurs when attempting to do it with a variable.
This works, creating a downstream pipeline like normal. But the branch name is hardcoded:
stages:
- deploy
deploy:
variables:
environment: dev
stage: deploy
trigger:
project: group/project
branch: foo
strategy: depend
This does not work; although TARGET_BRANCH is set successfully, the job fails because the downstream pipeline can not be created:
stages:
- removeme
- deploy
before_script:
- if [ -z "$TARGET_BRANCH" ]; then TARGET_BRANCH="main"; fi
- echo $TARGET_BRANCH
test_variable:
stage: removeme
script:
- echo $TARGET_BRANCH
deploy:
variables:
environment: dev
stage: deploy
trigger:
project: group/project
branch: $TARGET_BRANCH
strategy: depend
If you know what I'm doing wrong, or you have something that does work with variable expansion of the branch property, please share it (along with your GitLab version). Alternate solutions are also welcome, but this one seems like it should work.
GitLab Version on which bug occurs
Self-hosted GitLab Community Edition 12.10.7
What is the current bug behavior?
The job always fails for reason: downstream pipeline can not be created.
What is the expected correct behavior?
The branch property should be set to the value of the variable and the downstream pipeline should be created as normal, just as if you simply hardcoded/typed the name of the branch.
More details
The ability to use variable expansion in the trigger branch property was added in v12.4, and it's explicitly mentioned in the docs.
I searched for other .gitlab-ci.yml / GitLab config files. Every single one that attempted to use variable expansion in the branch property had it commented out, saying it was bugged for an unknown reason (example.
I haven't been able to find a repository in which someone claimed to have a working variable expansion for the branch property of the trigger property.
Unfortunately, the alternate solutions are either (a) hardcoding every downstream branch name into the GitLab CI config of the upstream project, or (b) not being able to test changes to the downstream GitLab CI config without first committing them to master/main, or having to use only/except.
TL;DR: How to use the value of a variable for the branch property of a bridge job? My current solution makes it so the job fails and the downstream pipeline isn't created.
this is a 'works as designed', and gitlab will improve in upcoming releases.
trigger job will pretty weak b/c it is not a full job that runs on a runner. Therefore most of the trigger configuration needs to be hardcoded.
I use direct API calls to trigger downstream jobs passing the CI_JOB_TOKEN which links the upstream job to downstream as the trigger does
API calls give you full control
curl -X POST \
-s \
-F token=${CI_JOB_TOKEN} \
-F "ref=${REF_NAME}" \
-F "variables[STAGE]=${STAGE}" \
"${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/trigger/pipeline"
now this will not wait and monitor for when the job is done so you will need to code for that if you need to wait for the downstream job to finish,
Moreover, CI_JOB_TOKEN cannot be used to get the status of the downstream job, so you will another token for that.
- |
DOWNSTREAM_RESULTS=$( curl --silent -X POST \
-F token=${CI_JOB_TOKEN} \
-F "ref=${DOWNSTREAM_PROJECT_REF}" \
-F "variables[STAGE]=${STAGE}" \
-F "variables[SLS_PACKAGE_PATH]=.serverless-${STAGE}" \
-F "variables[INVOKE_SLS_TESTS]=false" \
-F "variables[UPSTREAM_PROJECT_REF]=${CI_COMMIT_REF_NAME}" \
-F "variables[INSTALL_SLS_PLUGINS]=${INSTALL_SLS_PLUGINS}" \
-F "variables[PROJECT_ID]=${CI_PROJECT_ID}" \
-F "variables[PROJECT_JOB_NAME]=${PROJECT_JOB_NAME}" \
-F "variables[PROJECT_JOB_ID]=${PROJECT_JOB_ID}" \
"${CI_SERVER_URL}/api/v4/projects/${DOWNSTREAM_PROJECT_ID}/trigger/pipeline" )
echo ${DOWNSTREAM_RESULTS} | jq .
DOWNSTREAM_PIPELINE_ID=$( echo ${DOWNSTREAM_RESULTS} | jq -r .id )
echo "Monitoring Downstream pipeline ${DOWNSTREAM_PIPELINE_ID} status..."
DOWNSTREAM_STATUS='running'
COUNT=0
PIPELINE_API_URL="${CI_SERVER_URL}/api/v4/projects/${DOWNSTREAM_PROJECT_ID}/pipelines/${DOWNSTREAM_PIPELINE_ID}"
echo "Pipeline api endpoint => ${PIPELINE_API_URL}"
while [ ${DOWNSTREAM_STATUS} == "running" ]
do
if [ $COUNT -eq 0 ]
then
echo "Starting loop"
fi
if [ ${COUNT} -ge 350 ]
then
echo 'TIMEOUT!'
DOWNSTREAM_STATUS="TIMEOUT"
break
elif [ $(( ${COUNT} % 60 )) -eq 0 ]
then
echo "Downstream pipeline status => ${DOWNSTREAM_STATUS}"
echo "Count => ${COUNT}"
sleep 10
else
sleep 10
fi
DOWNSTREAM_CALL=$( curl --silent --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" ${PIPELINE_API_URL} )
if [ $COUNT -eq 0 ]
then
echo ${DOWNSTREAM_CALL} | jq .
fi
DOWNSTREAM_STATUS=$( echo ${DOWNSTREAM_CALL} | jq -r .status )
COUNT=$(( ${COUNT} + 1 ))
done
#pipeline status is running, failed, success, manual
echo "PIPELINE STATUS => ${DOWNSTREAM_STATUS}"
if [ ${DOWNSTREAM_STATUS} != "success" ]
then
exit 2
fi
I have the following shell script. There is a deploy function, that is used later in the script to deploy containers.
function deploy {
if [[ $4 -eq QA ]]; then
echo Building a QA Test container...
docker create \
--name=$1_temp \
-e DATABASE=$3 \
-e SPECIAL_ENV_VARIABLE \
-p $2:9696 \
... # skipping other project specific settings
else
docker create \
--name=$1_temp \
-e DATABASE=$3 \
-p $2:9696 \
... # skipping some project specific stuff
fi
During the deployment, I have to do some tests on the application (which is in containers). I use different containers for that, however I need to pass one additional parameter to the deploy function for my QA_test container, because I need a different setting in the docker create. Hence, why I put the if statement in the begining, which checks if the 4th argument equals 'QA', and if it does it creates a specific container with special env variables, otherwise if it has just the 3 arguments, it creates a 'normal' one. I was able to run the code with two seperate deploy functions, but I want to make my code better for readabiity. Anyway, this is how it should go:
Step 1: Normal tests:
deploy container_test 9696 test_database # 3 parameters
run tests... (this is not relevant to the question)
Step 2: QA testing:
deploy container_qa_test 9696 test_database QA # 4 parameters, so I can create a
# a special container
run tests... (again, not relevant to the question)
Step 3: If they are successful, deploy a production-ready container:
deploy production_container 9696 production_database # 3 parameters again
However what happens according to the log:
Step 1: test_container is created. However its created with the upper if, but there is not a 4th parameter that equals QA, however it executes it.
Step 2: this runs normal.
Step 3: production container is built as a QA container
It never reaches the else part, even if the condition is not satisfied. Can anyone give me some tips?
just change [[ $4 -eq QA ]] to :
if [[ "$4" == "QA" ]]; then
-eq used to comapre numbers ....
I'm currently debugging a shell script, which acts as a master-script in a data pipeline. In order to run the pipeline, you feed a bunch of arguments into the shell script. From there, the shell script sequentially calls 6 different scripts [4 in R, 2 in Python], writes out stuff to log files, and so on. Basically, my idea is to use this script to automate a data pipeline that takes a long time to run.
Right now, if any of the individual R or Python scripts break within the shell script, it just jumps to the next script that it's supposed to call. However, running script 03.py requires the data input to scripts 01.R and 02.R to be fully run and processed, otherwise 03 will produce erroneous output data which will then be written out and further processed in later scripts.
What I want to do is,
1. Break the overall shell script if there's an error in any of the R scripts
2. Output a message telling me where this error happened [line of individual R / python script]
Here's a sample of the master.sh shell script which calls the individual scripts.
#############
# STEP 2 : RUNNING SCRIPTS
#############
# A - 01.R
#################################################################
# log_file - this needs to be reassigned for every individual script
log_file=01.log
current_time=$(date)
echo "Current time: $current_time"
echo "Now running script 01. Log file output being written to $log_file_dir$log_file."
Rscript 01.R -f $input_file -s $sql_db > $log_file_dir$log_file
# current time/date
current_time=$(date)
echo "Current time: $current_time"
# B - 02.R
#################################################################
log_file=02.log
current_time=$(date)
echo "Current time: $current_time"
echo "Now running script 02. Log file output being written to $log_file_dir$log_file"
Rscript 02.R -f $input_file -s $sql_db > $log_file_dir$log_file
# PRINT OUT TIMINGS
current_time=$(date)
echo "Current time: $current_time"
This sequence is repeated throughout the master.sh script until script 06.R, after which it collates some data retrieved from output files and log files, and prints them to stout.
Here's some sample output that gets printed by my current master.sh, which shows how the script just keeps moving even though 01.R has produced an error.
file: test-data/minisample.txt
There are a total of 101 elements in file.
Using the main database.
Writing log-files to this directory: log_files/minisample/.
Writing output-csv with classifications to output/minisample.csv.
Current time: Wed Nov 14 18:19:53 UTC 2018
Now running script 01. Log file output being written to log_files/minisample/01.log.
Loading required package: stringi
Loading required package: dplyr
Attaching package: ‘dplyr’
The following objects are masked from ‘package:stats’:
filter, lag
The following objects are masked from ‘package:base’:
intersect, setdiff, setequal, union
Loading required package: RMySQL
Loading required package: DBI
Loading required package: methods
Loading required package: hms
Error: The following 2 arguments need to be provided:
-f <input file>.csv
-s <MySQL db name>
Execution halted
Current time: Wed Nov 14 18:19:54 UTC 2018
./master.sh: line 95: -1: substring expression < 0
./master.sh: line 100: -1: substring expression < 0
./master.sh: line 104: -1: substring expression < 0
Total time taken to run script 01.R:
Average time taken per user to run script 01.R:
Total time taken to run pipeline so far [01/06]:
Average time taken per user to run pipeline so far [01/06]:
Current time: Wed Nov 14 18:19:54 UTC 2018
Now running script 02. Log file output being written to log_files/minisample/02.log
Seeing as the R script 01.R produces an error, I want the script master.sh to stop. But how?
Any help would be greatly appreciated, thanks in advance!
As another user mentioned, simply running set -e will make your script terminate on first error. However, if you want more control, you can also check the exit status with ${?} or simply $? assuming your program gives an exit code of 0 on success, and non-zero otherwise.
#!/bin/bash
url=https://nosuchaddress1234.com/nosuchpage.html
error_file=errorFile.txt
wget ${url} 2> ${error_file}
exit_status=${?}
if [ ${exit_status} -ne 0 ]; then
echo -n "wget ${url} "
if [ ${exit_status} -eq 4 ]; then
echo "- Network failure."
elif [ ${exit_status} -eq 8 ]; then
echo "- Server issued an error response."
else
echo "- Other error"
fi
echo "See ${error_file} for more details"
exit ${exit_status};
fi
I like to put some boilerplate at the top of most scripts like this -
trap 'echo >&2 "ERROR in $0 at line $LINENO, Aborting"; exit $LINENO;' ERR
set -u
While coding at debugging, I usually add
set -x
And a lot of trace "comments" with colons -
: this will parse its args but only show under set -x
Then the trick is to make sure any errors you know are ok are handled.
Conditionals consume the errors, so those are safe.
if grep foo nonexistantfile
then : do the success stuff
else : if you *want* a failout here, just call false
false here will abort # args don't matter :)
fi
By the same token, if you just want to catch and ignore a known possible error -
ls $mightNotExist ||: # || says "do on fail"; : is an alias for "true"
Just always check your likely errors. Then the only thing that will crash your script is a fail.
I'm working on a custom Nagios script that will monitor cPanel to make sure it is running and give back a status depending on what it gets from an output of service cpanel status. This is what I have:
##############################################################################
# Constants
cpanelstate="running..."
ALERT_OK="OK - cPanel is running"
ALERT_CRITICAL="CRITICAL - cPanel is NOT running"
###############################################################################
cpanel=$(service cpanel status | head -1)
if [ "$cpanel" = "$cpanelstate" ]; then
echo $ALERT_OK
exit 0
else
echo $ALERT_CRITICAL
exit 2
fi
exit $exitstatus
When I run the script, this is the output I get:
root#shared01 [/home/mvelez]# /usr/local/nagios/libexec/check_cpanel
CRITICAL - cPanel is NOT running
When I run the script, cPanel IS RUNNING but this is the output I get. As a matter of fact, no matter what the status reports for cPanel this is the output that comes out. When I comment out the ELSE, ECHO and EXIT 2 statement:
#else
# echo $ALERT_CRITICAL
# exit 2
It gives back a blank output:
root#shared01 [/home/mvelez]# /usr/local/nagios/libexec/check_cpanel
root#shared01 [/home/mvelez]#
I'm not sure what I'm not doing correctly as I am very new to bash scripting and trying to learn as I go along. Thank you in advanced for any and all help very very much!
The code below should work, but you might need to run it with sudo, because 'service' might not be available for ordinary users.
#!/bin/bash
##############################################################################
# Constants
cpanelstate="running"
ALERT_OK="OK - cPanel is running"
ALERT_CRITICAL="CRITICAL - cPanel is NOT running"
###############################################################################
cpanel=$(service apache2 status | head -1)
echo CPANEL $cpanel
if [[ $cpanel == *$cpanelstate* ]]; then
echo $ALERT_OK
exit 0
else
echo $ALERT_CRITICAL
exit 2
fi
#Oleg Gryb's answer solves your problem, but as for why your original script didn't work:
[ "$cpanel" = "$cpanelstate" ] compared the full command output - e.g., cpsrvd (pid 10066) is running..., against a substring of the expected output, running... for equality, which will obviously fail.
The solution is to use bash's pattern matching, provided via the right-hand side of its [[ ... ]] conditional (bash's superior alternative to the [ ... ] conditional):
[[ "$cpanel" == *"$cpanelstate" ]]
* represents any sequence of characters, so that this conditional returns true, if $cpanel ends with $cpanelstate (note how * must be unquoted to be recognized as a special pattern char.)
Can I run multiple Test Cases from multiple scripts but have a single output that either says "100% Pass" or "X Failed" and lists out the failed tests?
For example I want to see something like:
>runtests.rb all #runs all the scripts in the directory
Finished in 4.523 Seconds
100% Pass
>runtests.rb category #runs all the scripts in a specified sub-directory
Finished in 2.1 Seconds
2 Failed:
test_my_test
test_my_test_2
1 Error:
test_my_test_3
I use the built-in MiniTest::Unit along with the autotest command that is part of ZenTest and get output like:
autotest
/Users/tinman/.rvm/rubies/ruby-1.9.2-p290/bin/ruby -I.:lib:test -rubygems -e "%w[test/unit tests/test_domains.rb tests/test_regex.rb tests/test_vlan.rb tests/test_nexus.rb tests/test_switch.rb tests/test_template.rb].each { |f| require f }"
Loaded suite -e
Started
........................................
Finished in 0.143375 seconds.
40 tests, 276 assertions, 0 failures, 0 errors, 0 skips
Test run options: --seed 62474
Is that similar to what you are talking about?