So I am new to the continuous integration world and of course, like everyone else, have been thrown into the task of setting it up and integrating it.
My company is .NET based and handles MVC applications, regular websites, form applications, as well as windows services. We have a ton of solutions in our repository and most of these solutions have trouble building in our CI that I am testing out (TeamCity). My local PC being the testing server.
Anyways, I have been reviewing a ton of documentation regarding TeamCity on their confluence pages but a lot of it is very technical and I'm getting tired of reading things more than once. So with that being said, there were some terms and concepts that I was a little confused on that I was hoping someone could explain pretty plainly maybe with an example.
Build Parameters -
I understand what parameters are and know what they are used for. Why would I use one in a build?
2.Snapshot dependencies -
What are these and When should I use them? I am having trouble visualizing this.
Artifact Dependencies -
I understand what artifacts are, I don't know why a project would need them to build however.
Any input would be greatly appreciated. Thanks guys.
I could give you some examples.
Build Parameters
It gives you a way to configure your build. For example, you may want to control your build version in Teamcity. You can use build parameters to specify major, minor version (with default value) for your build.
Personally, I try to avoid to use it as we should control configuration in source code or somewhere.
Snapshot dependencies
It is mostly used to build a chain of builds. For example, you have two builds. One is building source and other one is deploying to development environment. If you want to trigger deployment build after each source code build is successful, you need to add snapshot dependency to deployment build.
Artifact Dependencies
The same example as above, you want to deploy the artifact build in other build. In this situation, you need to use artifact dependency to copy the artifact into deployment build to upload somewhere etc.
Or sometimes you may have different projects, some projects are common libraries. If you don't have a artifact repository, you can use artifact dependency when those common libraries are required by other projects.
Related
I've spent years trying to deploy libraries that use native code to Maven Central. I've run into the following problems:
There weren't any good plugins for building native code using Maven. native-maven-plugin was a very rigid build system that, among other things, made it difficult to debug the resulting binaries. You'd have to manually synchronize the native-maven-plugin build system with the native IDE you use for debugging.
Maven did not replace variables in deployed pom.xml files: MNG-2971, MNG-4223. This meant that libraries had to declare platform-specific dependencies once per Maven profile (as opposed to declaring the dependency once and setting a different classifier per profile); otherwise, anyone who depended on your library had to re-define those same properties in their project file in order to resolve transitive dependencies. See Maven: Using inherited property in dependency classifier causes build failure.
Jenkins had abysmal support for running similar logic across different platforms (e.g. "shell" vs "batch" tasks, and coordinating a build across multiple machines)
Running Windows, Linux and Mac in virtual machines was way too slow and fragile. Even if you got it working, attempting to configure the VMs as Jenkins slaves was a lesson in frustration (you'd get frequent intermittent build errors).
Maven Central requires a main jar for artifacts that are platform-specific: OSSRH-975
Sonatype OSS Repository Hosting and maven-release-plugin assumed that it would be possible to release a project in an atomic manner from a single machine but I need to build the OS-specific bits on separate machines.
I'm going to use this Stackoverflow question to document how I've managed to overcome these limitations.
Here is how I overcame the aforementioned problems:
I used CMake for building native code. The beauty of this system is that it generates project files for your favorite (native) IDE. You use the same project files to compile and debug the code. You no longer need to synchronize the two systems manually.
Maven didn't support CMake, so I built my own plugin: https://github.com/cmake-maven-project/cmake-maven-project
I manually hard-coded platform-specific dependencies into each Maven profile, instead of defining the dependency once with a different classifier per profile. This was more work, but it doesn't look like they will be fixing this bug in Maven anytime soon.
I plan to investigate http://www.mojohaus.org/flatten-maven-plugin/ and https://github.com/mjiderhamn/promote-maven-plugin as alternatives in the near future.
Jenkins pipeline does a good job of orchestrating a build across multiple machines.
Running Jenkins slaves on virtual machines is still very error-prone but I've managed to workaround most of the problems. I've uploaded my VMWare configuration steps and Jenkins job configuration to help others get started.
I now create an empty JAR file for platform-specific artifacts in order to suppress the Sonatype error. This was actually recommended by Sonatype's support staff.
It turns out that maven-release-plugin delegates to other plugins under the hood. Instead of invoking it, I do the following:
Use mvn versions:set to change the version number from SNAPSHOT to a release and back.
Tag and commit the release myself.
Use nexus-staging:rc-open, nexus-staging:deploy -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}, and nexus-staging:rc-close to upload artifacts from different platforms into the same repository. This is called a Staging Workflow (referenced below).
Upon review, release the repository to Maven Central.
Important: do not enable <autoReleaseAfterClose> in the nexus-staging plugin because it closes the staging repository after each deploy instead of waiting for all deploys to complete.
Per https://issues.sonatype.org/browse/NEXUS-18753 it isn't possible to release SNAPSHOT artifacts atomically (there is no workaround). When releasing SNAPSHOTs, you need to skip rc-open, rc-close and invoke nexus-staging:deploy without -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}. Each artifact will be uploaded into a separate repository.
See my Requirements API for a real-life example that works.
Other quirks to watch out for:
skipNexusStagingDeployMojo must be false in last reactor module (otherwise no artifacts will be deployed): https://issues.sonatype.org/browse/NEXUS-12365. The best workaround is to use Maven profiles to omit whatever modules you want when deploying (don't use skipNexusStagingDeployMojo at all)
skipLocalStaging prevents deploying multiple artifacts into the same repository: https://issues.sonatype.org/browse/NEXUS-12351
I have only little experience using maven with eclipse. One of the job descriptions which I received has "Workflow management using Maven" as a required skill. What does this mean ? What do they possibly expect?
I think they want you to correct them? :D
I'm not sure what they refer to. I would guess it relates to the developer workflow of creating and delivering software with eclipse (?) and maven.
So setting up a project from scratch is often done from an maven archetype (a project template if you like). A lot of open source frameworks offer archetypes to start with.
For existing projects you would check out the code from version control and import it into eclipse. the m2eclipse plugin is required to do that (but I think its quite common to have it)
Then there is building the software. Which is done through executing maven phases (which will then execute plugins). See maven-phases for more details. Maven phases have default plugins that execute (for example compile will run the compiler plugin).
So your workflow would look like this: you modify the files. compile them, test them, package them, deploy the artifacts into the maven repository. the maven install phase will store the artifacts in you local repository, the maven deploy phase will upload them into the company's repository.
From there the the files are installed. Yet you can use maven plugins to install the software into a application server. That depends on the traditions of the company.
I would not think of workflow as some strict step by step think like BPMN. Development is usually done with huge amounts of personal practices (are tests written in advance or while implementing, and so on).
Hope that will help :)
We check all of our source code's dependent third-party JARs into source control along with our source code. When needed, we manually download updates to third party JARs and replace those JARs that are in source control with the newer versions. We haven't felt the need to use Maven yet as this process seems simple enough for us. But are we missing something of great value by not using Maven? Or does our scenario not warrant using Maven?
"JARs dont change much", I hear this all the time.....
Storing jars in the SCM is simple in the beginning of the project. Over time the number of jars gets larger and larger.... Wait 2 or 3 years and nobody remembers where the jars came from, what their licensing terms were and most commonly what versions are being used (important to know when analysing security vulnerabilities).....
The best article I've read recently making the case for a repository manager is:
http://www.sonatype.com/people/2012/07/wait-you-dont-have-a-repository-manager/
A little irreverant, but does make a valid point about the kind of technical inertia one encounters all the time.
Switching a project team from ANT to Maven can be scary.... Maven works quite differently, so I find it is best deployed with greenfield or adventurous project teams. For the old-school ANT users, I recommend using the Apache ivy plugin. Ivy allows such teams to outsource the management of their dependencies but keep the build technology they're comfortable with.
Ultimately the biggest benefit of using Maven are not dependency management. It's the standized build process. I've seen several failed attempts to create a "standard" ANT build process. Problem every build engineer has his opinion on what the standard should be.... Maven's approach of forcing users to write build plugins may appear restrictive in the beginning, but just like the iPhone eventually developers discover "there's a Maven plugin for that" :-)
When it comes to dependency management Maven really can be quite valuable. As Mark O'Connor suggests, running a local repository manager would likely be better than checking the artifacts into source control.
There are many tools (like m2e in eclipse) that can help with dependency management and provide valuable feedback on which modules or dependencies require which other dependencies. Maven will also make sure to get the appropriate version of a dependency even if different modules depend on different versions of a given library. That will help prevent duplicate versions of the same jar showing up in your deployed project as long as they have the same group and artifact id.
Even for a very simple project I don't think I would resort to checking dependencies into the source control system.
It's not only about 3rd Party Libraries. Mostly if you have multiple repositories. In our case, we had four repositories with lots of inter- and intra-dependencies.
Actually I started this answer and then I had to go for 15 minutes to talk to some colleague about a problem happened after someone forgot to update the .jar of one project in the other's lib directory.
And it looks more professional :)
Hi there i need some information or general tips on a problem with maven.
Context:
We just migrated one big eclipse project into 4 maven project. (Thats one step in the good direction!)
We were building that/those project with an ant script (build.xml) We were selecting the task to do "on-demand"
To keep it simple here are the 4 project : Core, Client, Server, Admin.
Each of those maven project build into a jar. This have been establish and it is working perfectly. Core is a dependency to Client and Server.
We use Jenkins-CI and Artifactory on a remote server.
Problem:
I need to create some kind of "parent project" that will build all those other maven project and add some task "on demand" that we were doing with an ant script.
Exemple: We want to build locally (So we don't use jenkins and artifactory on this side) for our developper so they can test manually their update (yes we have no test for now, we are working on a legacy system). On this build, we do not want to obfuscate the code or sign our jar..etc
We also want a "customer build" (The real release that we push on the server, so it does use jenkins and artifactory) That will add some task on some of the 4 project like obfuscating the code, signing the jar ..etc
For this "customer build", we need to be able to select our dependency of a library "on-the-fly" or more like "On demand". Our program is an extension to another software and all our customer don't use the same version. To make it simple the library "y" can be y-2.0.1.jar or y-2.0.2.jar.. etc
All of those "task" i need can be done in different maven-plugin with no problem.
Question: What would be a good practice to solve my problem. We would really like to get rid of our ant script. Also we are cleaning a big big dirty project so i would like a clean solution without a lots of duplicated stuff or lots of manually task to do each time we want to build either locally or on the remote server for our customer.
Idea: I though i could use different maven profile in all those 4 project as i saw there:
Ant to Maven - multiple build targets But i will have a seriously huge pom.xml for each project with lots of duplicated stuff so I really don't like this idea. I though we could have a parent maven project but this would contain no code so i think I'm wrong with this idea also.
Thank for answering and for your time!
Going with Maven Profiles is the right thing to do for this kind of customization. Then you'll probably have developerProfile and releaseProfile or such.
And yes, your poms will be big and complicated.
Looks like your demands are a little bit to much for what Maven can provide out-of-the-box, and it's not the best tool for doing highly-customized builds. Since (as I understand) you are on pretty early days with your new build infrastucture, I'd advice to look at Gradle. You could reuse your ant tasks and both Jenkins-CI and Artifactory work great with Gradle.
we have several projects in our company with many dependencies among them. we have gathered our projects (EJB,Web, Enterprise Application and ...), on a SVN server, now we have difficulties with our projects, because after checking out a project from SVN, we have to waste a long time to gather the required libraries and projects for the checked-out project to build.
i want to some help about that, how the maven technology could help us to easily gather the dependencies and build our projects automatically. we use myeclipse IDE.
moreover i need some help and resources, to learn how to achieve this goal.
any guide and help will be appreciated.
thanks in advance.
Depending on the amount of projects you have, and how complex their build is, switching to Maven can be a hell of a task. It might still be worthwhile, mainly because Maven forces you to do things "the Maven way"1, and that such a "Maven way" is becoming more standard with time: you might end up with a simplified/more standardized build procedure.
If your projects are currently built using ant, chances are that ivy can quite more easily solve your problem of offloading JARs from your VCS system, with a minimal impact to your current build configuration.
Otherwise, if you really think Maven is the right tool for you, I strongly suggest you to have a look at the official tutorial, the POM reference and start playing with simple, self generated projects to start having an idea about how to deal with Maven. Using the archetype:generate mechanism you have access to dozens (hundreds?) of sample projects (archetypes, in Maven jargon) that illustrate you how to deal with different kinds of projects, from simple JAR to more complex ones, maybe leveraging exotic frameworks or products. Have a look at the generated POM for such projects and try to understand how it works.
Integration with Eclipse is achieved via two different plugins. I'm not using Eclipse anymore since quite a while, but I think m2eclipse is still probably the best one to start with.
1- WARN: his is one of the things people coming to Maven heavily dislike more often than not