Assumption
API tests are written in the same API code repository using GoLang
API tests are run as part of the Pull Request CI build.
An API test is basically starting gRPC server, making request from a golang based gRPC client and verifying the response
Flow
Dev starts building a new endpoint for an API
Dev does not write any API tests (please note that I mention specifically API tests and not unit tests)
Dev creates a Pull Request for the changes
The Pull Request should identify that there are no tests written for the new changes and block the PR
My question
How could we build a CI flow like this to achieve point 4?
I'm aware that code coverage is one way to go but I'm not fully convinced if that would be the correct approach for API tests?
Is it possible to find code coverage of gRPC service based on such API tests? If yes, then how?
Related
I would like to set up some end-to-end tests to make sure my Google Sign In button is working. I'd like to use tools such as:
Cypress web tests
Datadog live monitoring
Mobile E2E tests
to ensure my set-up works.
Is it possible to whitelist certain requests from Google's bot detection? Either by IP or header etc?
I believe with Cypress there is a way of mocking the responses to fix this issue, but with more 'black-box' testing like the other approaches, this isn't possible.
Problem Statement: I'm building an .NET Core based REST API. It has unit tests and Postman Tests. When I check-in the code to my GIT HUB, I want my GIT HUB Action to RUN my unit tests and also run my postman tests which calls the actual Rest API. The REST API in turn calls the database to do it works.
As part of my github action script definition, I think I will need to launch a container for my REST API and another container to host my database. The Postman test then just hits the API container to test the various endpoints. Once testing completes, the containers go away.
Question: Is this approach viable/practical? How can I go about building this in github actions?
Tech stack: Postgress, .net core 6, Postman.
Im currently looking into K6 to stress test our api. For this I want to set up a github action to run a stress test before deploying to production.
The issue is that in all guides I found for this, the user is using some url pointing to some webpage and not using the api that should be testet.
Is it even possible to do what I want?
What is the correct way here?
We are using XRAY in our Jira instance for running so far manual test and record metrics for our QA department.
We would like now to move a step further in order to implement Test Automation integrated with X-ray.
X-ray can use cucumber syntax for test case
What is the way to integrate Cypress with X-ray ?
How can we create automation test in Jira and send them to Cypress ?
How to get cypress test result back to X-ray for reporting and tracking ?
We are new to cypress and automation test using those environment, thanks for sharing your findings
regards
If using Cypress, you can use the report produced by the test runner to integrate with Xray. If you're using Cucumber together with Cypress, you'll use "cypress-cucumber-preprocessor" package; you'll need to decide which workflow to adopt - in other words, you need to decide which will be the master for editing/managing your Gherkin specification.
The flows are slightly different depending on that.
This tutorial details the two flows; it is for Xray on Jira Cloud but it can easily be adapted if you are using Jira on-premise (server/Datacenter).
We have been writing our mobile app tests with Calabash/Xamarin.UITest for a while using backdoor methods to redirect our app's base api url to a mock HTTP server to make tests repeatable without incurring unnecessary server costs.
As Xamarin has announced they are phasing out Calabash (and Xamarin.UITest with it, as it relies on the Calabash server component) we have been working on migrating our test suite to Appium to comply with Xamarin's recommendations. Our simpler tests were migrated easily but a lot of our testing relies on said mock HTTP server with backdoor configuration which Appium does not seem to support, rendering our tests impossible to port without a way to recompile our app with the mock HTTP server address built in, which quite frankly, sucks.
Anyone knows an alternative to said backdoor method? Or an alternative to the whole mock HTTP server overall?
We want our tests to run in parallel in the Xamarin Test Cloud service so issuing a QA/Test environment for this is completely undesirable.