Teamcity builds are taking forever to complete because on every build, it's downloading all the dependencies (and sub-sub-sub dependencies) which is quiet pointless. I've been facing build time outs after waiting for 20-30 minutes which is extremely frustrating and annoying.
I'm wondering if there is a configuration which can let us choose when to re-download all the dependencies and when not to.
Here are the configurations:
I'm suspecting it's because of the -Pgenerate-dependencies flag. Not able to find much about it though.
Related
My company's builds are being uploaded to a Nexus repository. I don't have the source of the publishing scripts (obviously). I need evidence that there is something funny going on.
GIT log
That "publishMavenJavaPublicationToMavenRepository" task is taking up to 1 hour to run. Any sane developer would consider that excesive, right?
Well, the people in charge of the build process thinks everything is A-OK as long as the builds are getting uploaded. Even if it takes 1 hour or more. I complained that the build died when it reached 1 hour. They increased the timeout to 3 hours and called it a day.
Is there a Gradle plugin or property that prints in the log the upload/download speed of binaries to/from repositories?
I am suffering the slow download of maven.
I use the default configuration for Spring Boot starter project.
Group: com.example
Artifact: Demo
STS takes too much time to sync the content. is there a way to speed this up? for example, download the whole thing separately that the project needs, and then import them locally.
If you look carefully at the screenshot you'll notice that it says that Maven download operation is blocked by user operation.
The comments saying the build is slower under IDE (particularly Eclipse) are wrong in the sense that they are sort of a mental shortcuts. They are based on observation (it may indeed take longer time to achieve the end result) but that does not mean the build/download itself is slower. The thing is that Eclipse performs way more operations than just the build alone and sometimes those end up waiting on one another (as your screenshot clearly indicates).
With that in mind, if you run your build on the command line it may complete a lot faster as most likely it will not compete for resources with other tasks. But keep in mind that will keep Eclipse out of sync with what is actually on the file system. Eventually Eclipse will figure that out and try to sync. Sometimes it may not and you would have to do it manually. In both cases, depending on the size and amount of the projects and the number and complexity of changes made, it may take significant time to sync.
To summarize, it's not "slow download of maven" what you experience but multiple tasks competing for resources and waiting on each other. There is no point to pre-download all dependencies as this is not a recurring operation. Maven ONLY downloads missing dependencies. Once they are in local repo it will not try to download them again (unless you force it to).
I've tried building it:
parallel (helps a bit)
daemon=true
preDexLibraries true/false ( no big difference )
incremental true
offline
It takes >40s EVERY time. It doesn't matter if I change anything or not. Building it after "gradle clean" takes >50s.
I hope that I'm doing something wrong. Eclipse/ant could build consecutive builds WITH CHANGES in <10s.
Any help how to bring this <10s appreciated.
Yes, we have a lot of modules, some jar libraries, some maven dependancies. Still don't get why would it take >40s for two consecutive builds with no changes.
Gradle used: 2.2.1
Android studio: not really relevant, usually building it only with Gradle.
EDIT: adding some profiling logs.
:app:dexProjDebug 22.541s
:app:shrinkProjDebugMultiDexComponents 4.279s
:app:compileProjDebugJava 3.478s
:app:packageProjDebug 2.591s
:app:processProjDebugResources 2.590s
:app:packageAllProjDebugClassesForMultiDex 2.536s
:app:createProjDebugMainDexClassList 2.126s
You need to properly define inputs and outputs even for library projects with source code. (Gradle Documentation). Do you get UP-TO-DATE message when you rebuild second time onwards?
If you have no changes to the other modules, you can build the app itself using gradlew :app:build. This will eliminate the time Gradle spends on building the other modules. You can build any module separately like this as well, using gradlew :moduledirectoryname:build. If you need to build one module and the app, you can execute both tasks at once to avoid going through Gradle's configuration stage twice: gradlew :app:build :moduledirectoryname:build
If possible, update to Gradle 2.4. It's significantly faster (claiming 20-40%).
One of the most significant build time reducers is the preDex task. Check whether preDex is running in your build. It increases build time the first time you run it, but dramatically reduces build times in subsequent builds, as most of your SDKs and libraries will already be dexed. Note however that it won't run under some conditions, e.g. if you use multidex.
Btw Google are well aware of build times with Gradle and Android Studio and are going to introduce significant improvements in upcoming releases, including Jack and Jill which will allow compiling from source code directly to dex. You can already try to experiment with them - see here: http://tools.android.com/tech-docs/jackandjill.
Also worth watching this video from recent Google I/O: https://youtu.be/f7ihSQ44WO0?t=327
Download Genymotion emulator. It is very fast and good to run android apps. It will save you a lot of time.
Here is the link - GenyMotionDownload
Jenkins is very slow on the first visit in the early morning. This also happens if we restart the Jenkins server.
So the first developer accessing the jenkins has sometimes wait up to 3 minutes.
We thought it was an issue with the build load. So we moved all build-jobs to a jenkins slave. But the issue has not been solved with it.
Since the performance seems to be always bad after restart. I assume jenkins reads a lot of stuff into memory.
How can we enforce jenkins to precache stuff? We can give jenkins a hughe amount of RAM if needed. Hopefully the cache will solve the speed issues.
It would be difficult to definitively say what is causing your Jenkins instance to take so long to startup, though it is highly likely that one of your installed plugins is the offender.
For example:
JENKINS-17837 - Downstream Build View Plugin
JENKINS-24945 - Global Build Stats Plugin
Surely there are other plugins besides these two which have bugs/issues resulting in slow startup times as well. Some may have had issues and subsequent versions have addressed the performance issues.
Therefore, performing the following steps should at least reveal the culprit if not entirely fix the issue:
Upgrade Jenkins to the latest, stable version
Upgrade all plugins to their latest, stable versions
Restart Jenkins
If startup remains slow, remove your plugins one at a time (ensuring to save their respective configurations if necessary)
If upgrading to the latest versions does not improve your startup time, step 4 should reveal the culprit.
I'm running TeamCity 5. I have one particular configuration that is preserving 3500+ past builds, with artifacts, from the last year or so. It's taking up almost 7 gigs of space so I need to clean up old builds.
The configuration is set to clean up everything older than 30 days or 5 builds. None of the builds are pinned. No configurations on the server depend on the artifacts, and even if they did I have un-set "Prevent dependency artifacts clean-up". Other build configurations clean up, but this particular one doesn't.
Anybody have ideas on what to check next?
I figured it out. I wasn't understanding how TeamCity is preserving old builds. It's preserving all builds since the fifth oldest successful build, and the fifth successful build was a year ago. I reduced the cleanup rule to "Clean artifacts more than 30 days older than the last build and older than the 1st successful build", and got rid of some of the history. Now, to get the devs to fix their unit tests so this is successful more often... ;)
Also, you can leave the "-th successful build" field blank and limit just on days.