We are developing a Xamarin iOS app and we need to automate the test cases. There are two choices right now.
1. Xcode UI Test
2. Katalon
Before choosing a tool, I want to know your personal experience or recommendations about the above tools.
Currently, we have only the iOS version. But in future, we might have to do this app for Android. Also, we need to compare the values which display in the App with the values in the database or desktop version of the App.
Another thing is, we need to run these tests in the cloud devices (Probably in Visual Studio App centre)
Well, lets go through some specs of your two choices
XCTest:
positive points:
- native tool (but you obv don't care about native tools, since your app is in Xamarin)
- a lot of support from forums/Apple etc
- free
- easy to use, if you have native app
- easy to add to CI pipeline
negative points:
- no support for Android
- not supporting desktop-mobile comparison (it may be done with some 3rd party tools tho )
Katalon
positive points:
- supports iOS and Android (beware, that this does not mean, you can use the same code - you will still need to write two sets of methods etc, but Katalon can be used for Android, XCTest not)
- free - to some point (paid support)
- in theory supports the thing with mobile + web data comparison (not gonna be easy, but surely possible)
negative points:
- not as much support - forums etc
- writing complicated methods is not very easy (but if you have app with a few views, than it will work well)
- implementation to CI is harder
You have to select, which solution suits you better, and though I am not a fan of 3rd party testing tolls (cuz its pain to set it up correctly etc), Katalon might be better for you in your scenario (iOS + Web + maybe Android).
Hope this helps you :-)
Related
I have found a lot of articles (and forms) about Ionic vs Xamarin but they always talk about Ionic 1 or Xamarin when it was still paid for (so before march 2016 when it was acquired by Microsoft).
I have to research Ionic 2 vs (current) Xamarin and I know that both technologies have made huge advancements. Can anyone help me on my way with some sources or is it still too early to ask this question?
Xamarin: With one year of experience, I have realise it is really flexible IDE to use. The amazing part about Xamarin is you have 2 option, i) go native ii) go cross platform, this make it open on what kind usage you prefer and how you want to go about.
OS Support/Deployment to: Windows, iOS, Android, Mac
Languages Used: XML, C#
Ionic: especially Ionic-2, It amazing for all HTML, CSS, and JS users to build application for web, mobile applications. I haven't seen its deployment for windows phone(if it is, then i am not sure) With Ionic, forget all your MVVM structured coding. But the new implementation of version-2 parallel with angular-2(typescript) it might make it possible to implement.
OS Support/Deployment to: iOS, Android, Web(HTML/CSS)
languages Used: HTML, CSS, Angular-2
Star Rating
Xamarin - Mobile and PC Dev - 4/5 stars on usage of IDE and deployment. There are immediate help available upon stuck through Xamarin Forums. Documentation is little vague.
Ionic-2 - Mobile and Web Dev - 3/5 stars on usable and compatibility. Its hard to find some answers to specific question, rest ionic docs are great at documentation and implementation.
Hope this helps. PS: This is completely my own view as i have used both of this tools personally. Comments are welcomed.
I am working on developing a Xamarin hybrid (html+js+css) application for Android using Visual Studio for iOS.
My client wants to upload builds (of the app) to the Xamarin Test Cloud. I am new to Xamarin, so I have no idea about Xamarin Test Cloud. I have done some research and found that we can write test cases in our application and send them to Xamarin Test Cloud.
So far I understand it, Xamarin Test Cloud is only for app testing purposes, not for sharing buildAs with multiple users or clients. Am I right or not?
Please let me know more about Xamarin Test Cloud, i.e what are the uses of it.
You are partly correct.
Xamarin Test Cloud is an automated testing solution. You are able to run UI-tests on every platform. This will allow you to see how the application performs on the many android devices and versions, as well as the different versions of iOS. Xamarin Test Cloud however is limited to UI testing (and crash reporting based on your testing scenario). Unit Tests and Integration tests (mostly technical tests) cannot be performed by Xamarin Test Cloud.
In order to distribute the app to test-users you should use HockeyApp. HockeyApp has recently be acquired by microsoft (source) and allows you to distribute your app to a closed group of people for feedback (usually used for User Acceptance Tests (UAT)). Note: HockeyApp will soon be integrated with Microsofts' Application Insights service where it offers more features (insights in performance of your app, time used, time looked at Activity X etc) - see how to set this up
So in short:
Xamarin Test Cloud - Automated UI Testing on hundreds of devices
HockeyApp - User Acceptance Tests with distribution
Xamarin Insights (or Microsoft Application Insights) - insight in the usage of your app on different levels.
Correct. TestCloud is used to run your app on thousands of devices at once to figure out how your app behaves and how it looks on many different device quickly and without having to actually buy thousands of devices. It is not for sharing your app with specific users. If you want to share your app with specific users, you can either use HockeyApp for iOS and Android or you can use TestFlight for iOS and the Google Play Store's Beta program for Android.
In short Xamarin.Test.Cloud is a cloud service for automated UI tests.Those UI tests can be written in Calabash and/or C# NUnit project in form of unit tests. Most of the developers use C# version form od the framework. This Xamarin product is called Xamarin.UITest which is created locally with Xamarin tools and uploaded to Xamarin.Test.Cloud.
More in depth...
Xamarin.UITest which most of developers use is basically NUnit C# project in Viusal Studio or Xamarin.Studio. It loads user's apk (Andriod) or ipa/app (iOS) file and prepares everything for testing.
Testing might include:
exploring UI (detecting content, widgets/views/controls names and content)
interacting with UI (tapping, scrolling)
recording interactions from 2. so they can be used in automated tests
https://developer.xamarin.com/guides/testcloud/uitest/
API is not complex, but needs some [short] time for learning. Maybe more for tricks like working with pickers (especially in Xamarin.Forms), backdoors, gestures etc. My daughter is 15 and she's doing talks on Xamarin.UITest, because it is that simple and nice for kids to learn coding and feel "productive".
Sample interaction:
app.Tap( c => c.Marked("Login") );
Another useful command is: tree - to see Visual Tree (DOM) of the Ui presented.
NOTE:
Xamarin.UITest is not limited to apps written in Xamarin!
It can be used with Android app written in Java, iOS app written in objective-c/swift!
Xamarin.UITest differs for hybrid apps, because the DOM is actually HTML DOM. Hence - try exploring DOM with tree command.
With Xamarin.UITest and nunit runner you can test locally on your devices and/or emulators/simulators. 15 minutes limit is removed if I can recall correctly.
Xamarin.Test.Recorder is useful tool for exploring and recording Xamarin.UITests. On Mac this should be standalone app and on Windows you'll need Visual Studio Enterprise (restriction).
https://developer.xamarin.com/guides/testcloud/testrecorder/
Xamarin.Inspector from package called Xamarin.Interactive is another standalone tool for analysing and inspecting UI of the app.
https://developer.xamarin.com/guides/cross-platform/inspector/
When UI test is written - the project output (dlls) are uploaded to the Xamarin.Test.Cloud. To get project output you'll need Xamarin.Studio or Visual Studio with Xamarin tools installed.
Before upload there are several steps user needs to do like: selecting devices (number depends on pricing), getting APIKey for particular tests. Uploading is done through commandline...
https://developer.xamarin.com/guides/testcloud/uitest/working-with/submitting-tests-to-xamarin-test-cloud/
Then user goes to XTC (Xamarin.Test.Cloud) portal and checks results (reports, screenshots etc).
This is how it looks like behind the scenes (this is what I like to show during my talks):
https://youtu.be/PQMBCoVIABI?t=3432
Handsome Danish guys (which cannot play handball) on stage and 2000+ devices on the shelves (it was 1900+ devices in 2015-09) in the datacenter...
For sharing - use HockeyApp
For monitoring - use Xamarin.Insights
NOTE: HockeyApp and Xamarin.Insight are being merged to single product for beta-testing and monitoring (TestFlight like).
I am trying to evaluate whether Xamarin would be a good option for my project. The project is a large, complex app for Android and iOS with a lot of client-server communication. The user interface is a major focus and has to be really fast and smooth. Also, we plan to make large use of UX graphic effects (comparable to the Spotify app).
For now we are planning to go for two separate native apps using Java/Objective-C. However, the possibility of cross-platform code sharing would be very convenient for us of course.
Most opinions I've heard so far say that Xamarin - although far better than HTML5 apps - cannot match the UX of a native app. Also, I tested the following applications made with Xamarin (on Android):
Rdio
MarketWatch
Busch Gardens Discovery Guide
Sqor
Storyo
From my impression, none of them could quite match the speed and smoothness of a good native app.
If our focus is on a top notch user experience, would Xamarin really be a viable option? Can it really match a native UX? I am particularly looking for opinions from developers who have experience with large and complex cross-platform Xamarin applications. A few critical voices would be very helpful.
Thank you a lot!
I'm on the Rdio mobile development team, so I can make some personal reflections from that standpoint.
Xamarin allows you to write native applications in C#. Any slowness, jankiness, ugliness or bad-appiness usually has nothing to do with the Xamarin layer itself.
You save some time being able to share core business logic between your different clients, but you're still writing the UI from scratch, specific to the platform. You're just writing it in C#.
But while you save that time, you're spending it in other ways. All of those SDKs you want to use probably aren't compatible with Xamarin out of the box. You won't be pod install'ing that iOS framework, and you might be reinventing the wheel for handfuls of things. Xamarin takes advantage of the NuGet repo so you have a library of components that handle many of the things most people need (Analytics, Testing, Facebook SDK, JSON parsing, Database, etc etc) but it doesn't cover everything. And it certainly doesn't cover stuff that's out the day of an Apple or Google product announcement.
Any 3rd party code that you do want to import into your project will be done through writing custom bindings. While not usually difficult, it is time consuming. Xamarin has a team of people that specialize in assisting you in this. This fact speaks to the process being messy at times.
So while the slowness, jankiness, ugliness or bad-appiness probably isn't the fault of Xamarin, it might be the fault of you spending time in places you normally wouldn't, or not being able to take advantage of features you normally would. If that 3rd party partner SDK is giving you problems, your troubleshooting may take twice as long because there's a layer that you don't control.
UI is a wash. You're writing it from scratch anyway.
Business logic is shared. Depending on the app that might be a win if you architect your application to take advantage of it.
Compatibility / bleeding edge ability will be lacking. That might not matter to you at all, or you might be the person wanting to take advantage of that hot new API in the next OS release the day it's announced.
My personal thought, without knowing specifics, is if you want to build an application that you plan on being around years from now, and that will take advantage of the latest and greatest, I'd tell you to write natively for each platform. Unless you can really see huge gains in sharing that business logic the upfront gains are minimal. Or if you really like C#.
Xamarin uses native controls. So you design a fully native UI per platform. The users can't see that your App is made with Xamarin or Java/Objective-C.
There are sometimes performance issues in conjunction with the platform independent UI wrapper Xamarin.Forms. But you're not forced to use it. When you have still performance issues in your Xamarin.Android or Xamarin.iOS app then you produce them in your code.
There are benchmark results for Android apps comparing Xamarin.Android and Java apps: Does anyone have benchmarks (code & results) comparing performance of Android apps written in Xamarin C# and Java?
As you can see Xamarin's internal performance became better and better over the time.
Conclusion: Yes, you can write smooth native Apps using Xamarin.
I have some knowledge of basic PHP developing however right now I am thinking about developing a phone app. I do however have a bit of a conundrum in that I love WP7 and I have so many ideas for features which can't be done on iOS and Android, however at the same time I am completely aware that my target users (filmmakers) generally have iOS or android phones.
Is there a way I can develop the app, which I will get a developer to do rather than myself, with WP7 as the lead platform and then make a relatively simple conversion to the other platforms or do apps generally have to be completely reworked for each platform?
Also, just to though it out there, any idea of where to go to find a freelance app developer?
Thanks
James
I think you have a few options, but the first question I would have is: where comes the data from. Are you the owner of the data, and do you already have a datasource from where you can get the data from? If not I would suggest take some time to write a decent 'backend' from which the different platforms easily can get the data from.
Also all different platforms have their own pro/cons which you cannot share that easily. If you build a WP7 app in silverlight/xna, you cannot directly port it to a ios/android variant, also functionality will probably look/react different. You will need different solutions foreach platform with this approach.
Another option would be to use an platform like phonegap which can create a solution that will work on multiple platforms. (It uses HTML5 to get it work on the different platforms)
Build your app on iOS and android first, and when you have an app to base WP7 development then contact a developer that has made similar apps to yours.
I want to implement web application using xcode?is it possible?if yes then then please advicse me?
example:http://pinoypay.com/iphone/
While I have no personal experience with it I think you are looking for Dashcode rather than Xcode for developing web applications such as the one you linked.
It's located in /Developer/Applications/Dashcode after installing the developer tools.
You can read more about using Dashcode to develop web applications in the Dashcode User Guide.
Is it possible? Yes.
Is it convenient or pleasant? No.
I'm sorry but from the question, it appears you are trying to run before you can walk. Or possibly even crawl.
XCode is not required to build an iPhone styled website, only to write an iPhone application. You would be better off starting with some HTML-CSS tutorials.
XCode is an IDE. You can use it to build a web application but you must use some kind of web application framework in the first place.
The best option is to use one of the frameworks becoming available and maturing.
In the first instance as has been said look at Dashcode, particularly the latest release as this has templates for web based apps and will actually generate the code for dual, Browser and iPhone app so the resultant code will function in both.
The there are a number of third party solutions of varying quality, ease of use and documentation.
In no particular order there is:
IUI library
webapp
iWebkit
jqtouch (this is jquery based)
phone-universal
quickconnect (which has large parts of an OReilly book devoted to it)
All of these give you the javascript and the css to make a web app look and feel and act like a native iPhone app (to one degree or another).
Dashcode is excellent for some solutions as it gives you access to all the Apple tools and has reasonable IDE. The result then runs on a normal web server.
Quickconnect actually has Dashcode templates which you load into Dashcode and build the first part of the app in, then this is exported and imported into Xcode and you completed the project in Xcode and deliver it as a Web App for the iPhone.
Thee other frameworks can be used and developed with any tools, Textmate, Eclipse etc. and the resulting code it then run on a normal web servers
In my opinion, yes.
And while xCode does not target to the web development as other software (like Brackets, DW, etc.) it has a nice feature: a drop-down menu of javascript functions, which would be extremely useful to have in other web development IDEs...