We have 3 types of tests, unit, functional and acceptance.
The first 2 can be run with phpunit or other tools on top of it like codeception. So in CI the deploy script will run all these tests and if one fail build will fail and merge request will be cancelled.
But in CI deploy script how to run acceptance tests? These tests need to be run in browser in an already deployed build. Is there a workaround for that? Maybe run acceptance tests after build succeeds?
But then revert will be a pain.
You can parallelize the test jobs as advised in every CI/CD pipeline.
But in CI deploy script how to run acceptance tests?
For this you will need dedicated Test infrastructure, like available browsers on the server. After build step is successful - run all Test steps.
The parallel jobs can be setup like this:
Related
I can give the URL in cy.visit to point to deployed application, but I am not clear on the setup, as I don't have CI/CD yet. The goal is to be able to run this test on a button click, not having to checkout workspace and build the application.
I can give the URL in cy.visit to point to deployed application, but I am not clear on the setup, as I don't have CI/CD yet. The goal is to be able to run this test on a button click, not having to checkout workspace and build the application.
Yes, you could test your application directly on a staging/preproduction environment on AWS without A CI/CD.
However, I would not recommend using Cypress against a production environment because for E2E testing to be efficient, you will probably create/delete/edit many things, which may not be suited for a prod env.
Finally, it depends on what you try to achieve with your tests. In general, you will use Cypress to ensure your product works as expected after adding a new feature, for example. You always want to test against the latest version of your code.
On the setup, the E2E tests can be packaged in the project and run manually until you have a CI to execute them automatically.
https://docs.cypress.io/guides/guides/command-line#cypress-run
I am using the new Jenkins pipeline DSL which I really like. My Jenkinsfile is probably fairly typical and compiles / unit tests code in the master branch of GIT using maven, does a docker build, deploys to staging etc. Towards the end of the pipeline there is a manual step where a user has to confirm if a build goes to production e.g.
stage name: 'Production Deploy', concurrency:1
input 'Do you want to deploy to production?'
node {
sh "./bin/production-deploy.sh"
}
However, the build blocks until someone accepts / declines. Is there a way to automatically decline the input if someone else kicks the build off (by merging code to the master branch)?
I sugest you separate the Continuos Integration pipeline of Continuos Delivery pipeline. In the CI pipeline you build, test, deploy from develop stage to staging stage, then someone validate the staging deploy and when all staging tests are OK in a next step you execute the DC pipeline; is when you deploy to production stage. in that way you have an independent livecycle develoment process and an independent delivery process.
We are using Fitnesse slim in our project. As of now we were executing the fitnesse tests (slim) from wiki page and we were checking the results manually from wiki itself.
Now, we are planning to implement CI for our project. So, is there a way to integrate fitnesse slim with AnthilPro (CI tool) and can run fitnesse tests / suits automatically and send the execution results to specified set of email IDs ?
Kindly help me out. I know that, many have already implemented this. But, i am not getting the exact info i am looking for anywhere.
Thank you,
Chethan
I'm not familiar with anthillpro, but I run fitnesse test using their junit runner on teamcity and Jenkins.
This produces test results those ci tools pick up automatically, and html reports for people to see the test details.
Mailing people based on build outcome is a standard feature of the tools. I assume for anthill also.
Would that work for you?
My fitnesse baseline project contains a maven setup to run a fitnesse suite with a single maven call: mvn clean test-compile failsafe:integration-test.
I assume that making a job the checks out a project, executes a maven call and then e-mails based on test outcome is just as simple to setup in anthill as in other ci tools.
I want to run my protractor test (remote server) using teamcity project.
I'm new to teamcity. I want to call the scripts from teamcity.
can I just refer the script at its location without having 'checkout directory'? since I don't do checkout, I just want to run the scripts and control it remotely.
what is the best way to combine protractor tests with teamcity? the target is to make it run on hourly basis and use the results - send it on mail, run again if failed etc.
thanks.
You can run test from teamcity targets without checking out any code. You can leave the vcs settings as blank if you want to .
Teamcity targets can automatically send email for test failures to individuals or groups. Normally emails are sent to developers who have commits in a particular run, however since you are not tying in any VCS to your build step, you can sign up a group of people who will receive a mail whenever tests fail using the "Notifications" tab
I have a job configured to automatically deploy an application to a running server. I would like to know if Jenkins provides a way to verify if this application was deployed successfully. Ideally, the build should fail when the deployment fails.
Jenkins has a rundeck plugin which might perhaps be useful.
Rundeck deploys the application, and can be setup to trigger a post-deployment build on Jenkins to perform tasks like running integration tests.
Jenkins could be used to report the status of the deployment, however it might make more sense to use rundeck's dashboard instead.