In our TeamCity deployment, we have used S3 Artifacts Storage to store build artifacts in S3 as they are built. This is very useful as we get to leverage the stability of S3 to store and restore these files should TeamCity need to be re-deployed.
I was wondering if there is a similar way to store the hidden artifacts of a build, i.e. build logs, build settings, etc in S3 in the same manner? These hidden artifacts accumulate to thousands or hundreds of thousands of files, and also log files can be particularly large. Having to copy these files to a fresh TeamCity deployment can make the restore time very long, more than 12 hours.
It would be great if these files, or at the least, the build logs, could be stored in S3 and cached on the TeamCity server as necessary based on user request. Any ideas of whether this can or has been done?
Related
My team has using TeamCity to automate some tiresome maintenance tasks, and over time, we've found we want to re-use common pieces, so we've got some common functions in a repository on Bitbucket.
For better or for worse, our Bitbucket has a daily backup/maintenance period that, when active, blocks all of our builds from running with the following error:
Failed to collect changes ... Bitbucket is currently unavailable
I've looked at the various checkout modes, though we're generally limited to checking out files on the server (Rather than agent). I had figured that if the files are checked out to the server, then if Bitbucket were unavailable, there would be some way to fall back on "Whatever is already there". Especially as we don't have Clean build checked.
Is there some way that we can fall back on whatever is already checked out on the TeamCity server? Or do we need to set up some kind of redundancy?
Yes you can.
you need to simply build a project in a directory location of a teamcity machine and then call the run from that location instead.
You can have a build with No VCS at all. This build run will utilize the folder you mentioned instead of cloning a VCS.
I do have a project in TeamCity, that has a build configuration for the master release branch. This is compiled, every time a new version of our product is released.
In order to be able to pinpoint the introduction of errors, I do need a big retention time for some artifacts on this build configuration. As some other artifacts are rather big (full cd installation packages), my server's hard drive gets pretty full when simply upping the cleanup interval of this configuration.
Is it possible to configure two different cleanup intervals somehow? I would love to have a big retention time for the really important artifacts, while throwing the big ones away early.
I currently use TeamCity 9.0.3
Let's say for example, that my project has two artifacs:
smallupdatepack.zip (32 mb)
reallybigupdatecd.iso (700 mb)
I would like to configure TeamCity in a way that has the .iso kept for e.g. the last 10 builds, but the .zip is kept for the last 150 builds.
What I do not want, is a solution where all the .zip files are kept forever, while only the .iso files are deleted by an interval, which is all that seemed possible to me by using the build configuration's setting's artifact patterns alone.
You can specify custom cleanup rules for porjects/targets in Build History Clean-up page.
In your case, you can have a aggressive cleanup for all builds and a lenient cleanup for the Project/target for the master build
I have uploaded an example via an image below , if it helps
If you edit any of the settings, you can set individual period for artefacts. You can setup artefacts cleanup per target. However, for the same target you cannot setup different cleanup rules for multiple artefacts.
The answer by #Biswajit_86 looks like it's the only thing available for setting special clean up rules. I looked at it and it seems like the configuration specific settings should override the project settings and give you what you need, but maybe it doesn't work that way. Try it out and see if it works. If not, file a bug/suggestion with JetBrains.
The only other thing I could think of was to create a separate build configuration that only publishes the artifacts that you want to keep longer than your default rule. Give it a snapshot dependency on the configuration that creates the files and check the box to run on the same build agent. That way it doesn't need to rebuild them and can just publish what was already created. Set up a build trigger so that this new configuration runs whenever the other one finishes. Then set the clean up rules for this configuration to the longer retention setting.
A noob question, but googling and stack overflow search didn't seem to yield an answer.
Can someone explain what exactly are teamcity artifacts?
From the documentation
"Typically these include distribution packages, WAR files, reports,
log files, etc. When creating a build configuration, you specify artifacts
of your build at the General Settings page.
It doesn't really explain to me what an artifact is. A .Net oriented answer will be very helpful. I have a couple of builds already working on teamcity, but I'm not sure what exactly I would need an artifact for ?
thank you
Artifacts are the files you want the TeamCity server to store so that they can be downloaded after the build has finished. They will be downloadable from the TeamCity dashboard from each build.
For a .NET project you might choose the store the output of the compiler (i.e. .exe and .dll files), and the log files from running unit tests. You might just have a Windows Installer package (i.e. .msi).
It is completely up to you what gets stored for your specific needs. Just note that build artifacts do take up disk space on the TeamCity server, so if yours are large you'll want to configure the Build History Clean-up rules.
In the infrastructure I've inherited, the guys use jenkins and maven to construct jars and config files.
So, jenkins checks out some code, then maven builds something. Jenkins post build task then copies files from mavens target directory and copies them to a "build share" (eeks), for example
Source
**/target/*.jar
Destination
releases/env_name/project/lib
Source
**/target/classes/some-service.xml
Destination
releases/env_name/project/conf
I dare say this is not ideal.
In the new provisioning project, the jars from the build, are deployed into Nexus. In turn, I then use the maven rpm plugin to build RPMs using GAVs to identify which jars are packaged. I haven't captured the config files.
Could I publish the config files to Nexus? Yes they're small, but still artifacts. They would also have a maven snapshot or release coordinate, which could fit in with my releases. I have not tried, but I imagine I can add a section to my existing RPM build pom so that the desired plain text files are uploaded to Nexus when I run mvn deploy.
Some of the files require modification post deployment to suit an environment - this adds further head ache to this.
The poms that define the RPMs will also grow significantly in length. Not a problem per se, if I can ascertain this is the right way.
The question: How have others managed jars and config files deployment in a multi-environment set up?
Since you are asking how others have done it, I will say how I do, which may or may not be appropriate in your situation.
The binaries (.jar or whatnot) are build by Jenkins from the source code in SVN.
The SVN source code has configuration files, but most of the time, those are configured for DEV or Local environments. I let them have it there so that developers' local builds can be simple, but really as far as I am concerned they don't exist.
The SVN also has a separate location (not accessible by DEV, only by Release Managers and Systems teams), and this location contains configuration files for all environments (including DEV environment, when deployed from the CI server)
Since we have a lot of environment specific settings, it makes sense to keep separate copies of each environment.
If a new configuration parameter is added, it is merged from DEV config files all the way to PROD, just like other source code changes are merged, with the exception that only RM or Systems team can do it.
The deployment process (in our case a shell script, but could be anything) takes care of pulling the correct configuration files from SVN and pushing them, along with binaries, to the remote server. Once server is deployed, the "in-jar" configuration is wholly replaced by the configuration files that came from SVN.
I organize the configuration file by ENV, then by TYPE, for example: QA/WEB, QA/API and then PROD/WEB, PROD/API. So I don't have to maintain configuration files for each remote machine.
However there are times when configuration is specific to the remote machine, such as an IP address of the machine. In these cases, the SVN configuration file contains a token, like [local_ip]. When the deployment script pushes this file to remote server, it then knows to replace the token with real IP of the remote machine
I've read about the disadvantages (especially this one) of using SVN to store build artificats (large binary files). Hudson was suggested as an alternative.
How does Hudson handle these files?
Edit: My project is not Java-based.
Hudson can create/keep an archive of build artifacts, and provides a nice browser view for inspecting them.
You need to enable Archive the Artifacts in the job definition.
Hudson is basically using flat file storage. You can find those files within Hudson in the jobs/builds/ folders. I'm not sure I'd say, "Use Hudson as an alternative to checking in file to source control" but using something as an alternative is a decent idea if it provides:
authoritative place to store
versioned binaries access control
checksums for tamper resistance
release meta-data (environment information; approval level)
retention periods
I'm not sure how well Hudson scores on those marks, but I think it does at least some of that. SVN is non-terrible as a solution there as well, but really struggles with retention periods (old builds tend to eat disk space like crazy) and isn't terribly well optimized for large binaries - most SCM systems are optimized for smallish text files.
I stole the list above from this presentation: http://www.anthillpro.com/html/resources/webinars/Role_of_Binary_repositories_in_Software_Configuration_Management.html (registration required)
We use Jenkins for our builds, but we also store the artifacts from the builds. Like Eric said above, Hudson/Jenkins store artficats using flat file storage. It is organized based on the build.
Some things I have noticed from use (in reponse to Eric's questions about an alternative to souce control for binaries):
Each build stores it's own artifact, so you do have a versioning of sorts.
You can use the fingerprinting option when archiving. This will allow you to differentiate between versions and also check for corruption.
Retention periods are completely up to you. We keep artifacts forever.
FYI, our projects are not Java either (they are C/C++) and our artifacts are tar.gz/zip files and documents.
It may or may not be the best way to store binaries, but it is definitely decent as long as you have regular backups (weekly in our case) and your disk is fault tolerant.