my company provides a lot of java projects for customers (typical consulting solutions). These projects are versioned in git and built with gradle and therefore contain a gradlew as recommended by Gradle (https://docs.gradle.org/current/userguide/gradle_wrapper.html).
I am trying to find a solution to the task of updating the Gradle version for each of these individual projects. As you can imagine this is quite tedious if done manually.
The best solution I have come up yet is running a Jenkins job for each project that checks a web service for our current company-default Gradle version, runs
./gradlew wrapper --gradle-version <VERSION>
and pushes the result. Is there any better or even kind-of-standard way?
Related
I've spent years trying to deploy libraries that use native code to Maven Central. I've run into the following problems:
There weren't any good plugins for building native code using Maven. native-maven-plugin was a very rigid build system that, among other things, made it difficult to debug the resulting binaries. You'd have to manually synchronize the native-maven-plugin build system with the native IDE you use for debugging.
Maven did not replace variables in deployed pom.xml files: MNG-2971, MNG-4223. This meant that libraries had to declare platform-specific dependencies once per Maven profile (as opposed to declaring the dependency once and setting a different classifier per profile); otherwise, anyone who depended on your library had to re-define those same properties in their project file in order to resolve transitive dependencies. See Maven: Using inherited property in dependency classifier causes build failure.
Jenkins had abysmal support for running similar logic across different platforms (e.g. "shell" vs "batch" tasks, and coordinating a build across multiple machines)
Running Windows, Linux and Mac in virtual machines was way too slow and fragile. Even if you got it working, attempting to configure the VMs as Jenkins slaves was a lesson in frustration (you'd get frequent intermittent build errors).
Maven Central requires a main jar for artifacts that are platform-specific: OSSRH-975
Sonatype OSS Repository Hosting and maven-release-plugin assumed that it would be possible to release a project in an atomic manner from a single machine but I need to build the OS-specific bits on separate machines.
I'm going to use this Stackoverflow question to document how I've managed to overcome these limitations.
Here is how I overcame the aforementioned problems:
I used CMake for building native code. The beauty of this system is that it generates project files for your favorite (native) IDE. You use the same project files to compile and debug the code. You no longer need to synchronize the two systems manually.
Maven didn't support CMake, so I built my own plugin: https://github.com/cmake-maven-project/cmake-maven-project
I manually hard-coded platform-specific dependencies into each Maven profile, instead of defining the dependency once with a different classifier per profile. This was more work, but it doesn't look like they will be fixing this bug in Maven anytime soon.
I plan to investigate http://www.mojohaus.org/flatten-maven-plugin/ and https://github.com/mjiderhamn/promote-maven-plugin as alternatives in the near future.
Jenkins pipeline does a good job of orchestrating a build across multiple machines.
Running Jenkins slaves on virtual machines is still very error-prone but I've managed to workaround most of the problems. I've uploaded my VMWare configuration steps and Jenkins job configuration to help others get started.
I now create an empty JAR file for platform-specific artifacts in order to suppress the Sonatype error. This was actually recommended by Sonatype's support staff.
It turns out that maven-release-plugin delegates to other plugins under the hood. Instead of invoking it, I do the following:
Use mvn versions:set to change the version number from SNAPSHOT to a release and back.
Tag and commit the release myself.
Use nexus-staging:rc-open, nexus-staging:deploy -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}, and nexus-staging:rc-close to upload artifacts from different platforms into the same repository. This is called a Staging Workflow (referenced below).
Upon review, release the repository to Maven Central.
Important: do not enable <autoReleaseAfterClose> in the nexus-staging plugin because it closes the staging repository after each deploy instead of waiting for all deploys to complete.
Per https://issues.sonatype.org/browse/NEXUS-18753 it isn't possible to release SNAPSHOT artifacts atomically (there is no workaround). When releasing SNAPSHOTs, you need to skip rc-open, rc-close and invoke nexus-staging:deploy without -DstagingProfileId=${stagingProfileId} -DstagingRepositoryId=${stagingRepositoryId}. Each artifact will be uploaded into a separate repository.
See my Requirements API for a real-life example that works.
Other quirks to watch out for:
skipNexusStagingDeployMojo must be false in last reactor module (otherwise no artifacts will be deployed): https://issues.sonatype.org/browse/NEXUS-12365. The best workaround is to use Maven profiles to omit whatever modules you want when deploying (don't use skipNexusStagingDeployMojo at all)
skipLocalStaging prevents deploying multiple artifacts into the same repository: https://issues.sonatype.org/browse/NEXUS-12351
Now that Bazel (http://bazel.io/) has been opensourced, is there an incremental process by which I can gradually migrate (a large repository) from Maven to Bazel?
I work on Bazel. No, as far as we know there is no such process. I wish.
We have been running some migrations from other build systems to Bazel; the evidence isn't conclusive, but it's difficult to even envision how an incremental process would look like. There are some scenarios where we can envision one build system generating configuration files for another (like gyp), but then you still need to switch wholesale.
In the two years since Ulf responded, there's been a few efforts to assist with maven to bazel migration. In particular, the Bazel team is creating a tool to assist with this: https://github.com/bazelbuild/migration-tooling
The tool generates expansive WORKSPACE files from a set of pom files or maven coordinates. In the ideal case, you can pass the path to your maven project, and then it will generate a bzl file you can load into the WORKSPACE file.
More commentary on how to manage external dependencies can be found here: https://bazel.build/versions/master/docs/external.html
And another update (2018)...
There is a dedicated guide on migrating from Maven build tool to Bazel.
And on a general note, it’s best to have both build tools running in parallel until you have fully migrated your development team, CI system, and any other relevant integrations. You can run Maven and Bazel in the same repository.
https://docs.bazel.build/versions/master/migrate-maven.html
September 2019 update
Wix published a tool called Exodus to automatically migrate your Maven project to Bazel.
Additionally, rules_jvm_external has support for managing transitive Maven artifact dependencies.
May 2018 update
Here's another update using Jadep, a BUILD file generator for your Java projects.
There's a tutorial by the author who migrated google-java-format to Bazel: https://github.com/cgrushko/text/blob/master/migrating-gjf-to-bazel.md
I'm in the latter stages of setting up a CI environment for my project. I'm using Maven, Jenkins and Artifactory Pro and can successfully build my project and deploy it's artifacts to Artifactory. I have also written a bash script to retrieve the resulting artifacts of a specific build from Artifactory and copy them somewhere.
The main part I'm missing right now is automated versioning. I've looked at enabling Artifactory release management, which is really cool, but involves the rebuilding of the project. I'm really trying to follow the mantra of 'Build Once, Deploy Anywhere', so any rebuilding is a no-no.
My question boils down to: Is there an automated way (either with one of the aforementioned tools, or a plugin) to handle versioning, without rebuilding an artifact?
Artifactory Pro allows you to easily extend Artifactory's behavior with your own plugins written in groovy. (https://www.jfrog.com/confluence/display/RTF/User+Plugins)
You can find here, an example of Promote extension, that will change your artifacts versions without the needs of new build.
You can find more usefully examples in the GitHub "artifactory-user-plugins" repository.
I have a project, which is written using the Play Framework, say myproject-web. It is mostly a thin HTTP layer over another project, which forms the core of the entire business logic, called myproject-engine. In my build setup, myproject-web is a sbt project, whereas myproject-engine is a Gradle one.
What I want to achieve is that Play recognize myproject-engine as a dependency, and invoke gradle to build it whenever I try to build the play application (either on run, or automatically, as it happens in the dev mode) or when I do play dist. Is it possible? What is most important for me is that it automatically loads any dependencies that myproject-engine has.
Eventually, the state I want it to reach is that I host my Maven repo for these projects and then SBT can simply pull this package from over there and will get all its dependencies. Is this rather easy to setup? Even if it is, is it relatively easy to maintain?
As #Peter-Niederwieser pointed out in his comment, I think the only viable solution is to have a maven/ivy/gradle repository where the myproject-engine Gradle project is published to. With the correct resolvers the project becomes yet another project dependency, regardless of the build tool it uses.
See Resolvers in the official documentation of sbt.
Hi there i need some information or general tips on a problem with maven.
Context:
We just migrated one big eclipse project into 4 maven project. (Thats one step in the good direction!)
We were building that/those project with an ant script (build.xml) We were selecting the task to do "on-demand"
To keep it simple here are the 4 project : Core, Client, Server, Admin.
Each of those maven project build into a jar. This have been establish and it is working perfectly. Core is a dependency to Client and Server.
We use Jenkins-CI and Artifactory on a remote server.
Problem:
I need to create some kind of "parent project" that will build all those other maven project and add some task "on demand" that we were doing with an ant script.
Exemple: We want to build locally (So we don't use jenkins and artifactory on this side) for our developper so they can test manually their update (yes we have no test for now, we are working on a legacy system). On this build, we do not want to obfuscate the code or sign our jar..etc
We also want a "customer build" (The real release that we push on the server, so it does use jenkins and artifactory) That will add some task on some of the 4 project like obfuscating the code, signing the jar ..etc
For this "customer build", we need to be able to select our dependency of a library "on-the-fly" or more like "On demand". Our program is an extension to another software and all our customer don't use the same version. To make it simple the library "y" can be y-2.0.1.jar or y-2.0.2.jar.. etc
All of those "task" i need can be done in different maven-plugin with no problem.
Question: What would be a good practice to solve my problem. We would really like to get rid of our ant script. Also we are cleaning a big big dirty project so i would like a clean solution without a lots of duplicated stuff or lots of manually task to do each time we want to build either locally or on the remote server for our customer.
Idea: I though i could use different maven profile in all those 4 project as i saw there:
Ant to Maven - multiple build targets But i will have a seriously huge pom.xml for each project with lots of duplicated stuff so I really don't like this idea. I though we could have a parent maven project but this would contain no code so i think I'm wrong with this idea also.
Thank for answering and for your time!
Going with Maven Profiles is the right thing to do for this kind of customization. Then you'll probably have developerProfile and releaseProfile or such.
And yes, your poms will be big and complicated.
Looks like your demands are a little bit to much for what Maven can provide out-of-the-box, and it's not the best tool for doing highly-customized builds. Since (as I understand) you are on pretty early days with your new build infrastucture, I'd advice to look at Gradle. You could reuse your ant tasks and both Jenkins-CI and Artifactory work great with Gradle.