Use Gradle composite build only locally and use dependencies in CI/CD - gradle

I store library artifacts after compilation with Gradle in an internal Artifactory repository that is not accessible from the Internet. When I'm working offline or outside the company, I cannot access that repository.
One application ("web-application") that uses such libraries has configured the following settings in build.gradle:
...
repositories {
mavenCentral()
maven {
url "https://artifactory.company.com"
}
}
...
dependencies {
compile(group: 'com.company', name: 'common-lib', version: '1.0.+', changing: true)
}
...
dependencyManagement {
}
resolutionStrategy {
cacheChangingModulesFor 0, 'seconds'
cacheDynamicVersionsFor 0, 'seconds'
}
}
This works fine for CI/CD and when I am in the office. It ensures the most recent bugfix-version of a library is used so not every project has to be touched whenever there is a bump of the bugfix counter.
To work offline, I came across the Composite Build feature of Gradle: https://docs.gradle.org/current/userguide/composite_builds.html
This way, I can check-out the code locally and add those settings:
web-application/settings.gradle
rootProject.name='web-application'
includeBuild '../common-lib'
common-lib/settings.gradle
rootProject.name='common-lib'
In addition, I have to remove the dependency in web-application/build.gradle:
dependencies {
/* compile(group: 'com.company', name: 'common-lib', version: '1.0.+', changing: true) */
}
Lastly, IntelliJ IDEA asks me whether to add a dependency on the module common-lib, which I accept. This works fine but then again, I can't commit the changes in build.gradle as this would break CI/CD pipelines that rely on checked-in libraries.
What is the correct way to make both scenarios work?

Related

Gradle repositories conflict forces me to use repository in two places

I have a Gradle multi-project build. Which has a project myApplication that uses a library utils.
The library requires a JasperReports dependency with a custom repository. There is a particular Jasper dependency that produces an error when retrieved from mavenCentral().
If I build utils by itself, it works. Because it fetches the problem jar from the custom repo.
However, if I build myApplication it produces the maven repository error, because it tries to solve the dependency using its mavenCentral() repo.
myApplication/build.gradle.kts:
plugins {
id("application")
}
repositories {
mavenCentral()
}
application {
mainClass.set("com.myapp.SomeMainClass")
}
dependencies {
implementation(project(":utils"))
}
utils/build.gradle.kts:
plugins {
id ("java-library")
}
repositories {
maven {
url = uri("http://jaspersoft.jfrog.io/jaspersoft/third-party-ce-artifacts/")
isAllowInsecureProtocol = true
}
mavenCentral()
}
dependencies {
implementation("net.sf.jasperreports:jasperreports:6.18.1")
}
If I include the custom repository in myApplication/build.gradle.kts the project builds ok, but I think adding the custom repo seems like a bad practice, and I think Gradle documentation discourages it.
I think this is actually using a direct dependency in myApplication instead of fixing a transitive dependency. But of that I'm not so sure, and completely lost on how to solve it. I'm really struggling reading the documentation.
Is there a way for me to have Gradle just compile :utils first, and then have :myAplication use the dependency "as is"?

Having trouble with Gradle dependencies not updating

I have a project with 3 common jars shared across multiple repositories. I'm using Intellij 2019.4. In my gradle build, I included the following method:
dependencyManagement {
resolutionStrategy {
// don't cache SNAPSHOT modules; always retrieve them from the maven cache
cacheChangingModulesFor 0, 'seconds'
}
}
That's supposed to tell Gradle not to cache snapshots, but mine are still being cached. I build and install one of the common jars, and the maven repo has it, but the Gradle cache still has a jar from over 24 hours ago. Is there a different method I should be using? I want Gradle to ALWAYS use what's in the .m2 repo.
Gradle will only search for modules in the declared repositories.
This means that if you need a library, SNAPSHOT or not, from you local Maven repository, you need to declare it as a repository in Gradle.
repositories {
mavenLocal() {
content {
includeModule("my.org", "someLib")
}
}
// other repositories
}
However, there are caveats in adding mavenLocal() to your repositories, so make sure to use repository content filtering to only search your local Maven repo for those SNAPSHOT dependencies, as shown above.
Try to add changing = true into individual SNAPSHOT dependencies' configClosure.
implementation("group:module:1.0-SNAPSHOT") {changing = true}
Then cacheChangingModulesFor should apply to them:
configurations.all() {
resolutionStrategy {
cacheChangingModulesFor 0, "seconds"
}
}
With version latest.integration, this would require the build-number added into the version - but, this would keep the build reproducible, since one can switch back to the previous build of the library.
There also is a CLI option --refresh-dependencies, which refreshes them all.
The Gradle manual explains it, too: Declaring a changing version.
Forcibly deleting them before build would be another option.

how to set archiveBaseName for local .m2 repository

I'm trying to upgrade a dependency to a project that will ultimately become a dependency to my project. I've made the upgrade and I want to test it locally before I put it out on the repo to be used. I'm learning Gradle and a few Google searches showed me how to add the project to the settings.gradle file. But the dependency project uses aliases for their dependencies (see build.gradle below).
settings.gradle
include ':TransportImpl'
Changed to:
include ':TransportImpl', ':jeromq'
project(':jeromq').projectDir = new File("../zeromq/jeromq")
build.gradle
//project.ext.set("JEROMQ", 'THIRD-PARTY:jeromq:0.4.2')
project.ext.set("JEROMQ", ':jeromq')
If I uncomment the original line (shown commented above), because that apk is in the repo it gets recognized. I'm guessing that this only works for external libraries.
Other things I have tried:
//project.ext.set("JEROMQ", 'C:/Users/username/.m2/repository/THIRD_PARTY/jeromq/0.5.1-SNAPSHOT/jeromq-0.5.1-SNAPSHOT-jeromq.jar')
//project.ext.set("JEROMQ", 'C:\\Users\\username\\.m2\\repository\\THIRD_PARTY\\jeromq\\0.5.1\\jeromq-0.5.1-jeromq.jar')
//implementation filetree(dir: 'C:\\Users\\username\\.m2\\repository\\THIRD_PARTY\\jeromq\\0.5.1', include:['jeromq-0.5.1-jeromq.jar'])
Can anyone give me a tip on how I can assign a variable that points to the local repository and use that variable to set an archiveBaseName?
New Information:
gradle.build for our jeromq project
apply plugin : 'maven'
apply plugin : 'maven-publish'
// Top-level build file where you can add configuration options common to all sub-projects/modules.
ext {
// Nexus paths
nexusUrl='https://nexus.path'
Releases='/Private_Releases'
nexusUsername = project.findProperty("nexusUsername") ?: (System.getenv("NEXUS_USERNAME") ?: "user_name"
nexusPassword = project.findProperty("nexusPassword") ?: (System.getenv("NEXUS_PASSWORD") ?: "password")
// Project versions
jeromqVersion = "0.5.1-SNAPSHOT"
}
allprojects {
// Read only repositories for dependencies; this should never be used to publish
repositories {
mavenCentral()
jcenter()
}
}
The project that uses it as a dependency finds it using the following from its build.gradle file:
// Create aliases for dependencies
project.ext.set("EASY_MOCK", 'Test:easymock:3.5.1')
project.ext.set("OBJENESIS", 'Test:objenesis:2.6')
// **************** HERE ***************************
// THIRD-PARTY is configured to look on the nexus server
project.ext.set("JEROMQ", 'THIRD-PARTY:jeromq:0.4.2') ... or 0.5.1 or 0.5.1-SNAPSHOT ...
allprojects {
// Read only repositories for dependencies; this should never be used to publish
repositories {
mavenCentral()
mavenLocal()
// maven {
// // trying to add my local repo,
// // BUT this still does not change where THIRD-PARTY is pointing to
// url 'file://C:/Users/me/.m2/repository/THIRD_PARTY/jeromq/0.5.1-SNAPSHOT/jeromq-0.5.1-SNAPSHOT-jeromq.jar'
// }
maven {
name 'ReleasesName'
url "$nexusUrl$ReleasesName
}
}
maven {
name 'ReleasesNameSnapshots'
url "$nexusUrl$ReleasesNameSnapshots"
credentials {
username "${rootProject.ext.nexusReadOnlyUsername}"
password "${rootProject.ext.nexusReadOnlyPassword}"
}
}
jcenter {
url "https://jcenter.bintray.com/"
}
}
The only reason I need the alias for that dependency is because it is used in other places.
I'm not entirely sure what you are asking, but I think what you are trying is completely off.
The build you are trying to include is a Maven build, not a Gradle build, so it is unlikely you can simply treat it as it were a Gradle build.
And even if it were a Gradle build, including it like you did would not be the right way. How you tried it is for including multiple projects of a multi-project build, not including external libraries.
If it were a Gradle build, you would use a composite build, which effectively replaces a declared binary dependency by the build output of a "sub-build". But afair this only works cleanly with a Gradle build.
Why don't you simply mvn install your modified jeromq version, add mavenLocal() to your dependencies and depend on that just installed version? That would be the usual way for locally testing new Maven built dependencies.

Generate Project: HUGE downloads

I created my first LibGDX project using the Libgdx Project generator. I then opened the project in IntelliJ, upon which it asked me to index the repositories in the build.gradle file. The remote repositories in question were:
Maven2
all of the oss.sonatype snapshots
all of the oss.sonatype releases
All together this totalled about 5 to 6 GB of stuff, that intelliJ happily started downloading. Because it was taking ages to complete, I started looking through some of those repositories. Typically the point of using repo managers like gradle is that for the libraries you're using, you don't downloading the whole thing, just the parts you need. However, sonatype snapshots and releases are full of huge amounts of files and lots and lots of different versions/releases.
My question is: Is supposed to be downloading all of those files or did I miss something? Is there a way I can only download the bits and pieces that I need?
Here is my complete build.gradle:
buildscript {
repositories {
mavenLocal()
mavenCentral()
maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
jcenter()
}
dependencies {
}
}
allprojects {
apply plugin: "eclipse"
apply plugin: "idea"
version = '1.0'
ext {
appName = "HelloWorld"
gdxVersion = '1.9.6'
roboVMVersion = '2.3.0'
box2DLightsVersion = '1.4'
ashleyVersion = '1.7.0'
aiVersion = '1.8.0'
}
repositories {
mavenLocal()
mavenCentral()
maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
maven { url "https://oss.sonatype.org/content/repositories/releases/" }
}
}
project(":desktop") {
apply plugin: "java"
dependencies {
compile project(":core")
compile "com.badlogicgames.gdx:gdx-backend-lwjgl:$gdxVersion"
compile "com.badlogicgames.gdx:gdx-platform:$gdxVersion:natives-desktop"
compile "com.badlogicgames.gdx:gdx-box2d-platform:$gdxVersion:natives-desktop"
compile "com.badlogicgames.gdx:gdx-tools:$gdxVersion"
}
}
project(":core") {
apply plugin: "java"
dependencies {
compile "com.badlogicgames.gdx:gdx:$gdxVersion"
compile "com.badlogicgames.gdx:gdx-box2d:$gdxVersion"
}
}
tasks.eclipse.doLast {
delete ".project"
}
Not required to download all injected library with all versions.
If you build with gradle then it only download required artifact with only version that you injected.
Indexing a maven repository
A maven repository is made up of artifact and different versions of those artifact. My guess is that when an IDE like IntelliJ for example reads the remote repository, it will create a separate local cache/mini database of library names+versions available in the repository on machine. That is the indexing part. This makes searching for library names/latest versions etc much faster, as it is now being done consulting the index on your machine instead of accessing the repository each time.
Its nothing major and won't affect your builds, but what your IDE is simply telling is that whenever its looking for a version/dependency name (during auto-complete etc) its having to go online and check for those details (instead of just checking the local cache).
Advantage of indexing - search speed
Disadvantage - you have to ensure the index is kept up to date otherwise your won't always get up to date search results (new versions etc) also somehow 5 to 6 GB is download is painful.
You can disable this notification, If you want by clicking Disable:
Unindexed remote maven repositories found. Disable...
or on Mac OSX Go to
IntellijIDEA > Preferences > Appearance & Behavior > Notifications > Unindexed maven repositories gradle detection > Uncheck
Leave Unindexed remote maven repo, each time if artifact with particular version not found in local repo then it search to remote.

Gradle with a local pom dependency picks up wrong version

My build.gradle references a local maven pom. I have enabled the mavenLocal() repository and have added the jar as a compile time dependency (eg. my-local-lib, as shown below).
repositories {
mavenCentral()
mavenLocal()
}
dependencies {
compile (group: 'com.company', name: 'my-local-lib', version: '1.0-SNAPSHOT')
}
Gradle indeed picks it up and adds it as a dependency. My-local-lib, however, is itself dependent on another library as specified in its pom.xml, but gradle fails to pick up the correct version specified in the pom.xml, and instead chooses a much earlier version. This particular jar dependency is not a dependency on any other library.
Is this a known issue? Could it be due to my-local-lib being a SNAPSHOT version? Is there a way that I can enforce gradle to respect the versions specified in the libraries?
Try to add the following piece of code:
configurations.all {
resolutionStrategy {
cacheChangingModulesFor 0, 'seconds'
}
}
to build.gradle script.

Resources