We run our protractor regression tests in gitlab CI and we have jasmine HTML reports. Right now it is only the QA team that monitors and checks failure if any.
But we would like to make it more visible. The devs have also asked us if we can make it visible in a single place instead of having to go to gitlab job and browse for artifacts.Also would it be possible to have an overview of pass/fail tests over time.
I'm not sure how and where to start. Any pointers would be appreciated.
You're looking for the expose_as keyword for artifacts. The full docs are here: https://docs.gitlab.com/ee/ci/yaml/#artifactsexpose_as.
If you use expose_as with your artifacts, Gitlab CI will link them to any application Merge Request with the name you give in this field.
For example (from the docs):
test:
script: ["echo 'test' > file.txt"]
artifacts:
expose_as: 'artifact 1'
paths: ['file.txt']
In this example, a Merge Request for this pipeline will have a link called "artifact 1" that opens the file "file.txt".
This also works for directories, but if there's more than one file it will open in the job's artifacts browser (like you currently do).
There are some caveats, like:
If you use a variable in the artifacts path field, expose_as won't work
Max of 10 artifacts can be exposed
Glob patters won't work
If Gitlab Pages is enabled, some file extensions will be automatically rendered using Pages (html, xml, txt, etc.).
Related
Postman collection and it's folder arrangement
I want to run this ITEM-API postman collection in jenkins. I have exported it with github repo. I have also done configuration. But here when will I run test, it will give result for all ITEM-API collection. But inside ITEM-API collection, there are different folders of API, for eg. CHARTS, IAMGES, SYNDICATION, INTEREST, etc. SO how can use newman command to run test for particular api. Let's say, I want to run test for CHARTS api, then how can I use newman command for that in jenkins or groovy file in github repo.
please please help.
I have a shell script that logs on to the given server, finds all the wars running on the same and prints their name and versions to the console.
I need to take this output and display it on a page in confluence.
I went through this plugin for Jenkins and can connect and print static stuff on the page I provide.
The output that my script creates is dynamic, how do I, say store it in a variable and use it in the: Post-build Actions -> Publish to confluence -> Wiki Markup Replacements.
I have googled around and played with Jenkins to get these results, to no avail. I have seen '/env-vars.html' page, but none of these are of use to me, for my data is dynamic.
I referred this to accomplish this task.
We are archiving a lot of html reports in a Jenkins pipeline (scripted Pipeline). These are accessible through a link "Last Successful Artifacts" on the job page as usual. But we would like to create an additional custom link that that points to one of these reports (that is being generated whether the build is successful or not).
I found the DocLink plugin, but it's not listed on the pipeline compatibility list and I'm not able to figure out how this eventually could be used in a pipeline.
The HTML Publisher Plugin is another one I was looking at. But it’s not suited for our use case, since it requires us to gather all reports and publish them again. It also puts all the content in an iframe, but all we need is link to one of the already archived html reports.
Here is a example to add summary links to a build
manager.createSummary("document.png").appendText("<a href='"+ pom.url + "'>View Maven Site</a>", false)
As that method accepts HTML and can be used for XSS you need to approve them.
https://jenkins.io/doc/book/managing/script-approval/
for more examples look here: https://wiki.jenkins.io/display/JENKINS/Groovy+Postbuild+Plugin
For pipeline the Badge plugin was extracted from the Groovy Postbuild plugin and if can create the summary using something like:
createSummary icon:'package.png', text: "<a href='$pom.url'>View Maven Site</a>"
My current Cucumber file looks like this:
Feature: Test Online application Page
Scenario: Visit application home page and test links
Scenario: Visit application Login and Validate login
So now I would like to add few more scenarios may be for API testing in the same file. So i was thinking to create a new Feature for that instead of using the Feature: Test Online application Page. This way i dont need to create a separate feature file for API testing.
Feature: Test Online application Page
Scenario: Visit application home page and test links
Scenario: Visit application Login and Validate login
Feature: Test application API's
Scenario: validate Login API
Is it possible to have multiple features within a single feature file and is that a good practice? I just need to test one API and I will run API tests along with online tests. I will still separate them using #online and #api tags.
It is not possible to have multiple feature inside single feature file. If you create multiple feature inside single feature file, you will get Gherkin Parser exception while running cucumber scenarios. So the answer is NO.
C:/Users/ABC/RubymineProjects/XYZ.feature: Lexing error on line 47: 'Feature test google'. See https://github.com/cucumber-attic/gherkin2/wiki/LexingError for more information. (Cucumber::Core::Gherkin::ParseError)
Well, it is obviously not a good practice. It is best to put a single feature in a feature file. You should create new feature files for doing this. But you can add any number of scenarios in a single feature file.
The corresponding steps may or may not be in a single step file.
In BDD, cucumber is designed for the non-technical audience as well.
Writing scenario and steps definition in Gherkin Language or simple English is must support other audience.
All scenario should be executed Independently. No dependency on other scenario or feature file
In my past experience, Adding more complexity will add more flaky tests and High Maintenance cost
Agree with #philip John
You can create a text file with the features
and when executing the file it adds all the files in the order defined in the file.
#order-execution.txt
In this example, the file was created in the project root. file content
./features/records/country.feature
./features/records/company.feature
"scripts": {
"test:company": "cucumber-js #order-execution.txt --tags \"#company\" -f json:result/records/company.json",
},
this is the same as riding this way
"scripts": {
"company": "cucumber-js ./features/records/country.feature ./features/records/company.feature --tags \"#company\" -f json:result/records/company.json",
},
I am trying to figure out how to run unit tests, using Google Test, and send the results to TeamCity.
I have run my tests, and output the results to an xml, using a command-line argument --gtest_output="xml:test_results.xml".
I am trying to get this xml to be read in TeamCity. I don't see how I can get XML Reports passed to TeamCity during build/run...
Except through XML report Processing:
I added XML Report Processing, added Google Test, then...
it asks me to specify monitoring rules, and I added the path to the xml file... I don't understand what monitoring rules are, or how to create them...
[Still, I can see nowhere in the generated xml, the fact that it intends to talk to TeamCity...]
In the log, I have:
Google Test report watcher
[13:06:03][Google Test report watcher] No reports found for paths:
[13:06:03][Google Test report watcher] C:\path\test_results.xml
[13:06:03]Publishing internal artifacts
And, of course, no report results.
Can anyone please direct me to a proper way to import the xml test results file into TeamCity ? Thank you so much !
Edit: is it possible that XML Report Processing only processes reports that were created during build ? (which Google Test doesn't do?) And is ignoring the previously generated reports, as "out of date", while simply saying that it can't find them - or are in the wrong format, or... however I should read the message above ?
I found a bug report that shows that xml reports that are not generated during the build are ignored, making a newbie like me believe that they may not be generated correctly.
Two simple solutions:
1) Create a post build script
2) Add a build step that calls the command line executable with the command-line argument. Example:
Add build step
Add build feature - XML report processing
I had similar problems getting it to work. This is how I got it working.
When you call your google test executable from the command line, prepend %teamcity.build.checkoutDir% to the name of your xml file to set the path to it like this:
--gtest_output=xml:%teamcity.build.checkoutDir%test_results.xml
Then when configuring your additional build features on the build steps page, add this line to your monitoring rules:
%teamcity.build.checkoutDir%test_results.xml
Now the paths match and are in the build directory.