As a beginner, I'm researching towards whether is it possible to run automated test scripts of Coded UI in Headless mode for Azure DevOps CI/CD pipeline.
My intention is to automate UI of Desktop Applications. Along with this, I have concerned about MS Winapp Driver as well but it was concluded that, Coded UI is the ideal tool for my objective.
Researches Done:
1.) Created a Coded UI project and run a test scenario through Visual Studio - Successful
2.) Run the same scenario (C# file) using command prompt (using MSTest switches) - Successful
It will be highly appreciable, if one can come up with some best solution(s) or sample project.
According to this link(UI testing considerations), you can not use Headless for Desktop app.
if you are running UI tests for desktop apps, Windows agents must be configured to run as an interactive process with auto-logon enabled
Related
Is there any way we can automate desktop applications using cypress .. Any Plug in or tool available ?
Unfortunately, desktop app automation is not possible with cypress or with any cypress plugin.
Cypress code for interacting with elements runs inside the browser, which makes it impossible to interact with anything outside the browser.
You can try this tools https://github.com/clicknium/clicknium-docs. It can automate both desktop application and web application.
I have a set of Selenium WebDriver tests that work okay on three different machines; unfortunately, they don't seem to work consistently on a mentor's machine. I'm not sure if I coded the WebDriverWait conditions poorly, or if there is just something odd with my mentor's machine.
I need a consistent environment to run the unit tests in that is friendly for all developers on my team. We have a few test servers at our disposal, but some of our developers prefer to work on local boxes. Most of our developers should have access to Azure Devops, but they tend to prefer to use an in-house solution for CM.
Is there a way for me force my unit tests to run in Azure without having to setup a CI/CD pipeline? If I do that, will the Visual Studio subscribers on my team incur billing charges from Microsoft each time they run my unit tests?
NOTEs:
I can probably setup a CI/CD pipeline for our own in-house CM solution to run the unit tests, but our in-house CM solution is not necessarily Microsoft friendly and my organization's slot for a CI/CD expert is currently empty
I didn't readily notice how to run unit tests in Azure without CI/CD during Bing and Google searches
I'm afraid the answer is NO. The Azure Pipelines supports automatically build and test of code projects to make them available to others. It works with just about any language or project type. Azure Pipelines combines continuous integration (CI) and continuous delivery (CD) to constantly and consistently test and build your code and ship it to any target. I don't see other tools in Azure to run unit tests in Azure without pipeline.
Is there an alternative to Soasta with the enterprise license for automated functional testing? I'm looking for a solution where we can record clicks, gestures and input etc and play it back.
Thanks in advance
Xamarin UI Test. Titanium renders native components, so in your application you can write end-to-end test with it. There is a test recorder (only for Visual Studio) and a REPL which you can use to write your actions one by one and to have a look at the components tree. It's everything based on the compiled file so there aren't integration problems. After you can also automate those tests on Xamarin Test in Cloud.
https://developer.xamarin.com/guides/testcloud/uitest/
https://www.xamarin.com/test-cloud/recorder
https://www.xamarin.com/test-cloud
I am working on developing a Xamarin hybrid (html+js+css) application for Android using Visual Studio for iOS.
My client wants to upload builds (of the app) to the Xamarin Test Cloud. I am new to Xamarin, so I have no idea about Xamarin Test Cloud. I have done some research and found that we can write test cases in our application and send them to Xamarin Test Cloud.
So far I understand it, Xamarin Test Cloud is only for app testing purposes, not for sharing buildAs with multiple users or clients. Am I right or not?
Please let me know more about Xamarin Test Cloud, i.e what are the uses of it.
You are partly correct.
Xamarin Test Cloud is an automated testing solution. You are able to run UI-tests on every platform. This will allow you to see how the application performs on the many android devices and versions, as well as the different versions of iOS. Xamarin Test Cloud however is limited to UI testing (and crash reporting based on your testing scenario). Unit Tests and Integration tests (mostly technical tests) cannot be performed by Xamarin Test Cloud.
In order to distribute the app to test-users you should use HockeyApp. HockeyApp has recently be acquired by microsoft (source) and allows you to distribute your app to a closed group of people for feedback (usually used for User Acceptance Tests (UAT)). Note: HockeyApp will soon be integrated with Microsofts' Application Insights service where it offers more features (insights in performance of your app, time used, time looked at Activity X etc) - see how to set this up
So in short:
Xamarin Test Cloud - Automated UI Testing on hundreds of devices
HockeyApp - User Acceptance Tests with distribution
Xamarin Insights (or Microsoft Application Insights) - insight in the usage of your app on different levels.
Correct. TestCloud is used to run your app on thousands of devices at once to figure out how your app behaves and how it looks on many different device quickly and without having to actually buy thousands of devices. It is not for sharing your app with specific users. If you want to share your app with specific users, you can either use HockeyApp for iOS and Android or you can use TestFlight for iOS and the Google Play Store's Beta program for Android.
In short Xamarin.Test.Cloud is a cloud service for automated UI tests.Those UI tests can be written in Calabash and/or C# NUnit project in form of unit tests. Most of the developers use C# version form od the framework. This Xamarin product is called Xamarin.UITest which is created locally with Xamarin tools and uploaded to Xamarin.Test.Cloud.
More in depth...
Xamarin.UITest which most of developers use is basically NUnit C# project in Viusal Studio or Xamarin.Studio. It loads user's apk (Andriod) or ipa/app (iOS) file and prepares everything for testing.
Testing might include:
exploring UI (detecting content, widgets/views/controls names and content)
interacting with UI (tapping, scrolling)
recording interactions from 2. so they can be used in automated tests
https://developer.xamarin.com/guides/testcloud/uitest/
API is not complex, but needs some [short] time for learning. Maybe more for tricks like working with pickers (especially in Xamarin.Forms), backdoors, gestures etc. My daughter is 15 and she's doing talks on Xamarin.UITest, because it is that simple and nice for kids to learn coding and feel "productive".
Sample interaction:
app.Tap( c => c.Marked("Login") );
Another useful command is: tree - to see Visual Tree (DOM) of the Ui presented.
NOTE:
Xamarin.UITest is not limited to apps written in Xamarin!
It can be used with Android app written in Java, iOS app written in objective-c/swift!
Xamarin.UITest differs for hybrid apps, because the DOM is actually HTML DOM. Hence - try exploring DOM with tree command.
With Xamarin.UITest and nunit runner you can test locally on your devices and/or emulators/simulators. 15 minutes limit is removed if I can recall correctly.
Xamarin.Test.Recorder is useful tool for exploring and recording Xamarin.UITests. On Mac this should be standalone app and on Windows you'll need Visual Studio Enterprise (restriction).
https://developer.xamarin.com/guides/testcloud/testrecorder/
Xamarin.Inspector from package called Xamarin.Interactive is another standalone tool for analysing and inspecting UI of the app.
https://developer.xamarin.com/guides/cross-platform/inspector/
When UI test is written - the project output (dlls) are uploaded to the Xamarin.Test.Cloud. To get project output you'll need Xamarin.Studio or Visual Studio with Xamarin tools installed.
Before upload there are several steps user needs to do like: selecting devices (number depends on pricing), getting APIKey for particular tests. Uploading is done through commandline...
https://developer.xamarin.com/guides/testcloud/uitest/working-with/submitting-tests-to-xamarin-test-cloud/
Then user goes to XTC (Xamarin.Test.Cloud) portal and checks results (reports, screenshots etc).
This is how it looks like behind the scenes (this is what I like to show during my talks):
https://youtu.be/PQMBCoVIABI?t=3432
Handsome Danish guys (which cannot play handball) on stage and 2000+ devices on the shelves (it was 1900+ devices in 2015-09) in the datacenter...
For sharing - use HockeyApp
For monitoring - use Xamarin.Insights
NOTE: HockeyApp and Xamarin.Insight are being merged to single product for beta-testing and monitoring (TestFlight like).
I am looking for a solution to simulate vxml application in a desktop/web browser.
Challenge
i have few hundreds of unit test cases in every new vxml application development.
it is not possible to always make a test call to test every single unit test cases i have. if i can test vxml application in a simulator it will be more easy to test it..
i am aware of that avaya Dialog designer/AAOD have a feature to test application in their inbuild Application simulator which comes with AAOD/DD Eclipse package but this tool can run AAOD/DD projects in workspace alone.
Please refer : Chapter 16 in Avaya dev guide
AAOD's Tool looks like this
is it possible to customize to use this tool to simulate static VXML / Nuance NDF based VXML application ?
or any other tools available ?
Unit testing of IVR applications is challenging and it is near impossible to hit ever possible path in the call flow for an application of any size. I am not aware if you can customize Avaya's tool for your needs. But there is a product out there that may fit some of your requirements. It is Voiyager by Syntellect. It looks like they have a free trial now if you want to try it out. I saw a demonstration of it at SpeechTEK a few years ago and it was pretty impressive. The only reason I did not pursue it at the time is that it was pretty pricey.
Avaya od/dd is a platform dependent tool(IDE) where you can create (vxml+java), test and debug application in the tool. But code which is created outside(Avaya OD/DD tool) the environment is not possible. Because code generated in avaya OD is java format, while executing in the voice browser it is converted into vxml for that u need to give webLM a license to avaya tool.You can use some ivr automation tools like hammer etc.But you need to pay for the software.
VXML apps are just WARs. The only session information transferred between invocations of pages is the session id. We have used Apache JMeter to create apps.
What you can do it try and see if what you want to do can be done via the web browser, by going to http://<server>/<Application/Start there fill the boxes you require (most of the time just filling the ones marked ANI and DNIS will be enough). If your call can be tested this way you can use the tools for web testing, without using Orchestration Designer.
If your app has a CTI/AES connector you may not be able to do this, because the web server tries to connect to AES upon submitting the first page.
As stated above , there are things that cannot be unit tested .
Nuance NDF and SVF applications come with test pages where you can simulate the call flow ; similar tools exit for grammars .