Okay, Emma is killing me. I spent two days on this already.
There two issues with Emma now
It crashes partly through unit testing
It complaints about class stamp mismatch. : com.vladium.emma.EMMARuntimeException: [CLASS_STAMP_MISMATCH] runtime version of class xxx in the coverage data is not consistent with the version of this class in the metadata, possibly because stale metadata is being used for report generation.
While I can live with the crashes, it only often but I don't intent to fix it,
INSTRUMENTATION_RESULT: shortMsg=Process crashed.
[exec] INSTRUMENTATION_CODE: 0
However, I do like to see my coverage result.
I have tried:
clean the device
wiped out entire workspace (in Jenkins)
using the command prompt (bypass jenkins, ant clean, ant emma debug install test)
My understanding of this error is the apk installed and under unit test contains different binaries and hence its metadata info is out of date with the coverage.em generated.
I can confirm this is IMPOSSIBLE in my case, as I clead, ant clean and even wiped out entire work space and the phone memory. It just doesn't make any sense to me now.
Please HELP~
I am using Android sdk r16, NDK 5c, and the default settings from Ant and Emma.
I'm seeing the exact same problem with my Android project (which uses a native code third-party library via JNI). It turns out that the EMMA error message is slightly misleading in this case. It's triggered because the statically generated coverage meta-data on the host (coverage.em) does not match the coverage data collected during run-time of the application on the device (coverage.ec). In fact, it's not only a "mismatch", but the coverage.ec is completely missing. This is due to the crash in native code: On Android, a crash in native code will shut down the associated Java process, too, and in fact even restart the VM. This means EMMA is killed, too, and it does not have chance to flush the coverage.ec file to disk any more, which is why it's missing.
I've wondered whether it would make sense to make EMMA flush the coverage.ec file more often before the crash, but then again it's questionable how valuable an incomplete coverage info would be as
the file format probably is just not build in a way that makes sense to be parsed incrementally,
a partial coverage.ec still would mismatch a complete coverage.em,
coverage generated from a partial coverage.ec would report the wrong coverage with respect to all your tests in the suite.
So, long story short, the best way to address this is to fix your native code to not crash.
I met with this problem too when using EMMA for android. It's because the coverage.ec you pull from sdcard is not consistent with coverage.em. Try rebuild the instrumented version of the project.
Try the following steps:
ant clean release
ant instrument
ant installi
adb pull /mnt/sdcard/coverage.ec
use the coverage.em ( newly generated under the bin folder of the project through step 2 ) and coverage.ec to generate the coverage report
You can refer to EMMA for Android for detailed information.
Related
My Xcode project is very large, in terms of number of source files, and lines of code.
When I make a change to only the XCTest code, and hit run on a single test, a single test file, or even the whole test suite, Xcode re-compiles my host application first, even if no files have changed in the host application. It does so every time I make any change to any test file.
This is posing an efficiency challenge with writing tests, because small adjustments in the test code take a long time to re-run and re-validate. (upwards of 3 minutes per run - yes it's that big of a project, and we are already taking many steps to cut down the build time, by inspecting the build logs, etc...)
Using Xcode 10.3 with Swift 4.2
the tests are using:
#testable import MyModuleName
Is there a way I can prevent Xcode from re-building the whole project, if I've only changed tests?
I found the problem.
We have some custom scripts under the Project's Build Phases section that actually change some of the compilable assets in the codebase. We also have a Carthage dependency copy stage.
Commenting out both of these stages from Build phases makes it so that the whole project does not recompile when only changing tests. These scripts were actually making changes to the compilable code, making XCode force re-compilation.
Moral of the story: Check your build phases for any steps that could be modifying your source code, including possibly resource files that are part of the target. We will try to do this in a more automated way in the future, without having to comment out these build steps.
I have TeamCity set up to build some Visual Studio solutions and package them into an artifact. This works just fine most of the time.
However, once in a while (maybe 1 in 100 builds), the files built from TeamCity does not run. We use a lot of dependency injection in our code and some times our application refuses to start because it can't find the correct dependencies in the files. There are no missing files in the artifact, though.
We then force TeamCity to build everything once more (by adding a new line to a file and commiting it), and suddenly our application starts again.
It is then obvious to me that there is something TeamCity has not built correctly, though I'm unable to discover what it is, as all the build logs report no errors or warnings of any kind.
What can cause this strange behavior, and what can I do to prevent it from happening in the future (or what can I turn on to get more extensive logging to try to find out what happens when it fails)?
Check the errors when Team City does not run. Does it say why it cannot run? Sometimes have Team City fail a build simply because the version control system had a hiccup. Based on the error in Team City's log it can be obvious that it was only because of that. If you do not do clean builds try doing that and seeing if it makes the intermittent problems go away.
I've tried building it:
parallel (helps a bit)
daemon=true
preDexLibraries true/false ( no big difference )
incremental true
offline
It takes >40s EVERY time. It doesn't matter if I change anything or not. Building it after "gradle clean" takes >50s.
I hope that I'm doing something wrong. Eclipse/ant could build consecutive builds WITH CHANGES in <10s.
Any help how to bring this <10s appreciated.
Yes, we have a lot of modules, some jar libraries, some maven dependancies. Still don't get why would it take >40s for two consecutive builds with no changes.
Gradle used: 2.2.1
Android studio: not really relevant, usually building it only with Gradle.
EDIT: adding some profiling logs.
:app:dexProjDebug 22.541s
:app:shrinkProjDebugMultiDexComponents 4.279s
:app:compileProjDebugJava 3.478s
:app:packageProjDebug 2.591s
:app:processProjDebugResources 2.590s
:app:packageAllProjDebugClassesForMultiDex 2.536s
:app:createProjDebugMainDexClassList 2.126s
You need to properly define inputs and outputs even for library projects with source code. (Gradle Documentation). Do you get UP-TO-DATE message when you rebuild second time onwards?
If you have no changes to the other modules, you can build the app itself using gradlew :app:build. This will eliminate the time Gradle spends on building the other modules. You can build any module separately like this as well, using gradlew :moduledirectoryname:build. If you need to build one module and the app, you can execute both tasks at once to avoid going through Gradle's configuration stage twice: gradlew :app:build :moduledirectoryname:build
If possible, update to Gradle 2.4. It's significantly faster (claiming 20-40%).
One of the most significant build time reducers is the preDex task. Check whether preDex is running in your build. It increases build time the first time you run it, but dramatically reduces build times in subsequent builds, as most of your SDKs and libraries will already be dexed. Note however that it won't run under some conditions, e.g. if you use multidex.
Btw Google are well aware of build times with Gradle and Android Studio and are going to introduce significant improvements in upcoming releases, including Jack and Jill which will allow compiling from source code directly to dex. You can already try to experiment with them - see here: http://tools.android.com/tech-docs/jackandjill.
Also worth watching this video from recent Google I/O: https://youtu.be/f7ihSQ44WO0?t=327
Download Genymotion emulator. It is very fast and good to run android apps. It will save you a lot of time.
Here is the link - GenyMotionDownload
I have an Xcode 4.5 project for iOS 6.0 and I am trying to set up test coverage for Jenkins with Cobertura.
I have set up a test target and have configured both the main target and the test target correctly (Instrument Program Flow and Generate Test Coverage Files both on YES).
When the coverage report is generated (.gcda file), I translate it into XML using the gcovr tool (version 2.4).
When I run the test on Jenkins, everything works fine and I can see the report through the Cobertura plugin. The only problem is that in the Cobertura report all the values for "Conditionals" are "N/A", while I see that there are conditionals (if-constructions) in the covered files.
My guess is that something happens in the translation to XML resulting in a faulty conversion of branches/conditionals, but I cannot find what it is.
The strange thing is that for some if constructions it does recognise the conditional. For example if I use the "STAssertEquals()" macro from the "SenTestingKit" library, the if constructions there are recognised.
I hope someone can help me with a fix for this problem or at least knows the cause. Please do not hesitate to ask me if you need more information.
Nicolas
my team is currently using buildbot to automate overnight and continuous-integration builds and regression tests. For builds and unit tests, the builder just invokes a script which syncs the sources from p4 and then runs xcodebuild. The regression tests are also launched by a shell script, and are themselves combinations of shell scripts and AppleScripts. The builds are all performed on one system but then buildbot triggers tests to happen on multiple Macs, with different architecture and OS combinations. The things I like about buildbot are the automatic triggers (so the tests run only if and when the build succeeds), and the reporting including the waterfall view to see the overall status.
However, I see a number of problems too. The "master" process which coordinates the builds is either leaky or just has a huge working set which means that it consumes a couple of gigabytes of memory. Occasionally network problems mean that a slave will be lost; rather than retrying later it will just fail the build. In fact if the slave is supposed to be triggered for a dependent build, it will fail the first build after it has otherwise successfully completed.
So what are the rest of you using to automate your Xcode builds and unit tests? How do those solutions work for you? Anything you would recommend?
I use TeamCity in a windows environment but I believe it will work on Mac and has xcode build agents.
Another similar question here.
Our team use Hudson with a script which converts the Unit Test output from XCode to NUnit format.
I know this is an older question, but for those using TFS to hold their Xcode projects, I created a custom build activity to make automating Xcode builds via TFS easier. The code is hosted on Codeplex here: http://tfsxcodebuild.codeplex.com/.
Hope someone finds it useful!
We worked out what was using all the memory - log files - older buildbot keeps all logs forever and keeps them in memory (possibly until a restart).
Newer buildbot can be configured to keep a fixed amount of history.
Buildbot build log files should be limited in size to avoid the problem.
I use the XcodeBuilder that's part of CruiseControl. Of course it helps that I wrote it. :)
But I did use it on a real multiple person project for an iPhone app (Surf) that's for sale in the app store.
Now that it's three years later I have my own answer to provide to this question. I'm using Jenkins, mainly for the plug-ins that it provides. It has a plug-in for building targets in Xcode projects (or schemes in workspaces). There's a plug-in to run the Clang static analyzer. It interfaces with my bug-tracker system, it'll automatically push builds to Testflight too.