We have been writing our mobile app tests with Calabash/Xamarin.UITest for a while using backdoor methods to redirect our app's base api url to a mock HTTP server to make tests repeatable without incurring unnecessary server costs.
As Xamarin has announced they are phasing out Calabash (and Xamarin.UITest with it, as it relies on the Calabash server component) we have been working on migrating our test suite to Appium to comply with Xamarin's recommendations. Our simpler tests were migrated easily but a lot of our testing relies on said mock HTTP server with backdoor configuration which Appium does not seem to support, rendering our tests impossible to port without a way to recompile our app with the mock HTTP server address built in, which quite frankly, sucks.
Anyone knows an alternative to said backdoor method? Or an alternative to the whole mock HTTP server overall?
We want our tests to run in parallel in the Xamarin Test Cloud service so issuing a QA/Test environment for this is completely undesirable.
Related
Our cloud-deployed app is composed of (simplified):
A web front-end
Some back-end services
A database
When developing the front-end, I can easily debug by running the front-end locally, and redirecting its back-end calls to the actual services, since their endpoint routes are public.
However, I also want to debug back-end service code, together with the front-end. I can run back-end services locally, but they can't access the database, since the database doesn't have any publicly-accessible endpoint.
Question: How can I conveniently develop the service code? I can think of these options:
Expose the database publicly, maybe just the dev env's database. This doesn't sound like a good practice security-wise, and I haven't found a way to do it in my cloud platform (CloudFoundry).
Test everything using local unit- and component- tests. We do this, but can't cover everything, and certainly not the integration with the front-end.
Deploy my code changes to a dev environment, and test that way. This is what we do now, but:
It's a much slower development turn-around that running locally
I can't connect a debugger to the deployed app, so I must debug using logs, which again is slow
We have a limited number of dev environments, and this creates contention for them.
Somehow deploy a replica of the database locally as well, using some kind of test data.
Tech details: For cloud we use CloudFoundry over AWS. My back-end services are written in C# + .NET core 5. Locally, we develop them using Visual Studio 2019 on Windows.
For now, I managed to expose my database locally using an SSH tunnel. Specifically, by running cf ssh [AppName] -L [local_port]:[db_hostname]:[port], with [db_hostname]:[port] taken from the app's configuration, taken by running cf env [AppName].
I am a complete beginner working as a tester, I have found multiple resources in google about web APIs and its HTTP methods (GET,PUT,POST,DELETE). However I am confused what is the equivalent of these 'HTTP methods' for a windows desktop application? How do one actually perform a Windows application API test? I have no experience testing windows application APIs and would like to understand it first before making any test plans.
In Google's latest docs, they say to test Go 1.12+ apps locally, one should just go build.
However, this doesn't take into account all the routing etc that would happen in the app engine utilizing the app.yaml config file.
I see that the dev_appserver.py is still included in the sdk. But it doesn't seem to work in Windows 10.
How does one test their Go App Engine App locally with the app.yaml. ie: as an actual emulated app engine app.
Thank you!
On one hand, if your application consists of just the default service I would recommend to follow #cerise-limón comment suggestion. In general, it is recommended for the routing logic of the application to be handled within the code. Although I'm not a Go programmer, for single service applications that use static_files and static_dir there shouldn't be any problems when testing the application locally. You might also deploy the new version without promoting traffic to it in order to test it as explained here.
On the other hand, if your application is distributed across multiple services and the routing is managed through the dispatch.yaml configuration file you might follow two approaches:
Test each service locally one by one. This could be the way to go if each service has a single responsibility/functionality that could be tested in isolation from the other services. In fact, with this kind of architecture the testing procedure would be more or less the same as for single service applications.
Run all services locally at once and build your own routing layer. This option would allow to test applications where services needs to reach one another in order to fulfill the requests made to them.
Another approach that is widely used is to have a separate project for development purposes where you could just deploy the application and observe it's behavior in the App Engine environment. As for applications with highly coupled services it would be the easiest option. But it largely depends on your budget.
Helow
Is it possible to use Jmeter to make a performance test on a mobile app which doesn´t connect to internet, such as calendar or calculator; an app that doesn´t use http protocol?
Thank you.
No.
JMeter acts on protocol level, it doesn't actually clicks links or buttons, it sends requests over variety of supported protocols (not limited to HTTP) and records response times.
If your application doesn't have a backend, i.e. it is completely standalone, you normally don't need to load test it as it won't have more than one user. The only thing you can do is to check resources impact like CPU or RAM usage, battery consumption, most long running code parts, etc. Check out the following materials:
Android Performance Profiling Tools
iOS App Performance: Instruments & beyond
Performance Profiling on Windows Phone 8.1 with Visual Studio
For more detailed explanation of the concept of simulating multiple non-browser applications users check out How to Run Performance Tests of Desktop Applications Using JMeter guide.
You don't need internet connection in order to test like a local website, service or app. And yes, it is possible.
If your app is locally then the app will be available on a localhost port ( db, api, website). If you plan to test from the UI perspective a mobile or desktop app then you are using the wrong tool and you should use other tools like Ranorex or Test Complete.
I am using Xcode 7, Swift, and the new UI Automation announced at WWDC 15. I want to test some things in an oracle database after some of my UI automation tests have run.
Can this be done? If so how do you do it? I can't seem to find any sort of documentation on the subject.
The mobile phone doesn't guarantee a permanent and stable network connection and is a big problem for direct connection to DB.
For mobile platforms you need to create a web service on a server that will do all the work between database and mobile application. It is not recommended to connect directly to databases, except when it is local databases(SQLite).
I just upvoted #lurie-manea answer, in addition to that you shouldn't rely on Xcode UI test for testing your DB transaction once you have your web service implemented since you'd be able to test the DB state with XCTest(app tests) or if you using AppCode for writing code, it has nice plugin to test restful web services,
https://www.jetbrains.com/objc/help/testing-restful-web-services.html
Hope this helps.