Now that Bazel (http://bazel.io/) has been opensourced, is there an incremental process by which I can gradually migrate (a large repository) from Maven to Bazel?
I work on Bazel. No, as far as we know there is no such process. I wish.
We have been running some migrations from other build systems to Bazel; the evidence isn't conclusive, but it's difficult to even envision how an incremental process would look like. There are some scenarios where we can envision one build system generating configuration files for another (like gyp), but then you still need to switch wholesale.
In the two years since Ulf responded, there's been a few efforts to assist with maven to bazel migration. In particular, the Bazel team is creating a tool to assist with this: https://github.com/bazelbuild/migration-tooling
The tool generates expansive WORKSPACE files from a set of pom files or maven coordinates. In the ideal case, you can pass the path to your maven project, and then it will generate a bzl file you can load into the WORKSPACE file.
More commentary on how to manage external dependencies can be found here: https://bazel.build/versions/master/docs/external.html
And another update (2018)...
There is a dedicated guide on migrating from Maven build tool to Bazel.
And on a general note, it’s best to have both build tools running in parallel until you have fully migrated your development team, CI system, and any other relevant integrations. You can run Maven and Bazel in the same repository.
https://docs.bazel.build/versions/master/migrate-maven.html
September 2019 update
Wix published a tool called Exodus to automatically migrate your Maven project to Bazel.
Additionally, rules_jvm_external has support for managing transitive Maven artifact dependencies.
May 2018 update
Here's another update using Jadep, a BUILD file generator for your Java projects.
There's a tutorial by the author who migrated google-java-format to Bazel: https://github.com/cgrushko/text/blob/master/migrating-gjf-to-bazel.md
Related
I am new to gradle. I am looking forward to migrating from maven to gradle.
I had few requirements:
Existing project is maven based, and is generating a fat jar/uber jar. I am planning to split this into multiple projects, and creating smaller/thinner jars/libraries
I am currently evaluating the Multi-project Build support.
I have to also edit the Java source code, automatically, like making the java source modifications based on certain conditions
Publish the project as maven based, as other projects which need these split-up jars are still maven based.
I suppose Maven plugin can be used for publishing?
Would Gradle be a good, scalable solution for these two requirements which I am looking into currently?
Also please provide some pointers around these two topics.
Gradle has very good multi-project support, far better than Maven's. You can start with this documentation section
You can setup compilation of generated/auto-edited sources as well. Take a look at this forum post, discussing compilation of sources created from database using hbm2dao
You can setup publishing of projects using the Maven plugin. pom.xml files will be generated automatically
I've spent years trying to deploy libraries that use native code to Maven Central. I've run into the following problems:
There weren't any good plugins for building native code using Maven. native-maven-plugin was a very rigid build system that, among other things, made it difficult to debug the resulting binaries. You'd have to manually synchronize the native-maven-plugin build system with the native IDE you use for debugging.
Maven did not replace variables in deployed pom.xml files: MNG-2971, MNG-4223. This meant that libraries had to declare platform-specific dependencies once per Maven profile (as opposed to declaring the dependency once and setting a different classifier per profile); otherwise, anyone who depended on your library had to re-define those same properties in their project file in order to resolve transitive dependencies. See Maven: Using inherited property in dependency classifier causes build failure.
Jenkins had abysmal support for running similar logic across different platforms (e.g. "shell" vs "batch" tasks, and coordinating a build across multiple machines)
Running Windows, Linux and Mac in virtual machines was way too slow and fragile. Even if you got it working, attempting to configure the VMs as Jenkins slaves was a lesson in frustration (you'd get frequent intermittent build errors).
Maven Central requires a main jar for artifacts that are platform-specific: OSSRH-975
Sonatype OSS Repository Hosting and maven-release-plugin assumed that it would be possible to release a project in an atomic manner from a single machine but I need to build the OS-specific bits on separate machines.
I'm going to use this Stackoverflow question to document how I've managed to overcome these limitations.
Here is how I overcame the aforementioned problems:
I used CMake for building native code. The beauty of this system is that it generates project files for your favorite (native) IDE. You use the same project files to compile and debug the code. You no longer need to synchronize the two systems manually.
Maven didn't support CMake, so I built my own plugin: https://github.com/cmake-maven-project/cmake-maven-project
I manually hard-coded platform-specific dependencies into each Maven profile, instead of defining the dependency once with a different classifier per profile. This was more work, but it doesn't look like they will be fixing this bug in Maven anytime soon.
I plan to investigate http://www.mojohaus.org/flatten-maven-plugin/ and https://github.com/mjiderhamn/promote-maven-plugin as alternatives in the near future.
Jenkins pipeline does a good job of orchestrating a build across multiple machines.
Running Jenkins slaves on virtual machines is still very error-prone but I've managed to workaround most of the problems. I've uploaded my VMWare configuration steps and Jenkins job configuration to help others get started.
I now create an empty JAR file for platform-specific artifacts in order to suppress the Sonatype error. This was actually recommended by Sonatype's support staff.
It turns out that maven-release-plugin delegates to other plugins under the hood. Instead of invoking it, I do the following:
Use mvn versions:set to change the version number from SNAPSHOT to a release and back.
Tag and commit the release myself.
Use nexus-staging:rc-open, nexus-staging:deploy -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}, and nexus-staging:rc-close to upload artifacts from different platforms into the same repository. This is called a Staging Workflow (referenced below).
Upon review, release the repository to Maven Central.
Important: do not enable <autoReleaseAfterClose> in the nexus-staging plugin because it closes the staging repository after each deploy instead of waiting for all deploys to complete.
Per https://issues.sonatype.org/browse/NEXUS-18753 it isn't possible to release SNAPSHOT artifacts atomically (there is no workaround). When releasing SNAPSHOTs, you need to skip rc-open, rc-close and invoke nexus-staging:deploy without -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}. Each artifact will be uploaded into a separate repository.
See my Requirements API for a real-life example that works.
Other quirks to watch out for:
skipNexusStagingDeployMojo must be false in last reactor module (otherwise no artifacts will be deployed): https://issues.sonatype.org/browse/NEXUS-12365. The best workaround is to use Maven profiles to omit whatever modules you want when deploying (don't use skipNexusStagingDeployMojo at all)
skipLocalStaging prevents deploying multiple artifacts into the same repository: https://issues.sonatype.org/browse/NEXUS-12351
I have only little experience using maven with eclipse. One of the job descriptions which I received has "Workflow management using Maven" as a required skill. What does this mean ? What do they possibly expect?
I think they want you to correct them? :D
I'm not sure what they refer to. I would guess it relates to the developer workflow of creating and delivering software with eclipse (?) and maven.
So setting up a project from scratch is often done from an maven archetype (a project template if you like). A lot of open source frameworks offer archetypes to start with.
For existing projects you would check out the code from version control and import it into eclipse. the m2eclipse plugin is required to do that (but I think its quite common to have it)
Then there is building the software. Which is done through executing maven phases (which will then execute plugins). See maven-phases for more details. Maven phases have default plugins that execute (for example compile will run the compiler plugin).
So your workflow would look like this: you modify the files. compile them, test them, package them, deploy the artifacts into the maven repository. the maven install phase will store the artifacts in you local repository, the maven deploy phase will upload them into the company's repository.
From there the the files are installed. Yet you can use maven plugins to install the software into a application server. That depends on the traditions of the company.
I would not think of workflow as some strict step by step think like BPMN. Development is usually done with huge amounts of personal practices (are tests written in advance or while implementing, and so on).
Hope that will help :)
So I am new to the continuous integration world and of course, like everyone else, have been thrown into the task of setting it up and integrating it.
My company is .NET based and handles MVC applications, regular websites, form applications, as well as windows services. We have a ton of solutions in our repository and most of these solutions have trouble building in our CI that I am testing out (TeamCity). My local PC being the testing server.
Anyways, I have been reviewing a ton of documentation regarding TeamCity on their confluence pages but a lot of it is very technical and I'm getting tired of reading things more than once. So with that being said, there were some terms and concepts that I was a little confused on that I was hoping someone could explain pretty plainly maybe with an example.
Build Parameters -
I understand what parameters are and know what they are used for. Why would I use one in a build?
2.Snapshot dependencies -
What are these and When should I use them? I am having trouble visualizing this.
Artifact Dependencies -
I understand what artifacts are, I don't know why a project would need them to build however.
Any input would be greatly appreciated. Thanks guys.
I could give you some examples.
Build Parameters
It gives you a way to configure your build. For example, you may want to control your build version in Teamcity. You can use build parameters to specify major, minor version (with default value) for your build.
Personally, I try to avoid to use it as we should control configuration in source code or somewhere.
Snapshot dependencies
It is mostly used to build a chain of builds. For example, you have two builds. One is building source and other one is deploying to development environment. If you want to trigger deployment build after each source code build is successful, you need to add snapshot dependency to deployment build.
Artifact Dependencies
The same example as above, you want to deploy the artifact build in other build. In this situation, you need to use artifact dependency to copy the artifact into deployment build to upload somewhere etc.
Or sometimes you may have different projects, some projects are common libraries. If you don't have a artifact repository, you can use artifact dependency when those common libraries are required by other projects.
We check all of our source code's dependent third-party JARs into source control along with our source code. When needed, we manually download updates to third party JARs and replace those JARs that are in source control with the newer versions. We haven't felt the need to use Maven yet as this process seems simple enough for us. But are we missing something of great value by not using Maven? Or does our scenario not warrant using Maven?
"JARs dont change much", I hear this all the time.....
Storing jars in the SCM is simple in the beginning of the project. Over time the number of jars gets larger and larger.... Wait 2 or 3 years and nobody remembers where the jars came from, what their licensing terms were and most commonly what versions are being used (important to know when analysing security vulnerabilities).....
The best article I've read recently making the case for a repository manager is:
http://www.sonatype.com/people/2012/07/wait-you-dont-have-a-repository-manager/
A little irreverant, but does make a valid point about the kind of technical inertia one encounters all the time.
Switching a project team from ANT to Maven can be scary.... Maven works quite differently, so I find it is best deployed with greenfield or adventurous project teams. For the old-school ANT users, I recommend using the Apache ivy plugin. Ivy allows such teams to outsource the management of their dependencies but keep the build technology they're comfortable with.
Ultimately the biggest benefit of using Maven are not dependency management. It's the standized build process. I've seen several failed attempts to create a "standard" ANT build process. Problem every build engineer has his opinion on what the standard should be.... Maven's approach of forcing users to write build plugins may appear restrictive in the beginning, but just like the iPhone eventually developers discover "there's a Maven plugin for that" :-)
When it comes to dependency management Maven really can be quite valuable. As Mark O'Connor suggests, running a local repository manager would likely be better than checking the artifacts into source control.
There are many tools (like m2e in eclipse) that can help with dependency management and provide valuable feedback on which modules or dependencies require which other dependencies. Maven will also make sure to get the appropriate version of a dependency even if different modules depend on different versions of a given library. That will help prevent duplicate versions of the same jar showing up in your deployed project as long as they have the same group and artifact id.
Even for a very simple project I don't think I would resort to checking dependencies into the source control system.
It's not only about 3rd Party Libraries. Mostly if you have multiple repositories. In our case, we had four repositories with lots of inter- and intra-dependencies.
Actually I started this answer and then I had to go for 15 minutes to talk to some colleague about a problem happened after someone forgot to update the .jar of one project in the other's lib directory.
And it looks more professional :)