Generate Project: HUGE downloads - gradle

I created my first LibGDX project using the Libgdx Project generator. I then opened the project in IntelliJ, upon which it asked me to index the repositories in the build.gradle file. The remote repositories in question were:
Maven2
all of the oss.sonatype snapshots
all of the oss.sonatype releases
All together this totalled about 5 to 6 GB of stuff, that intelliJ happily started downloading. Because it was taking ages to complete, I started looking through some of those repositories. Typically the point of using repo managers like gradle is that for the libraries you're using, you don't downloading the whole thing, just the parts you need. However, sonatype snapshots and releases are full of huge amounts of files and lots and lots of different versions/releases.
My question is: Is supposed to be downloading all of those files or did I miss something? Is there a way I can only download the bits and pieces that I need?
Here is my complete build.gradle:
buildscript {
repositories {
mavenLocal()
mavenCentral()
maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
jcenter()
}
dependencies {
}
}
allprojects {
apply plugin: "eclipse"
apply plugin: "idea"
version = '1.0'
ext {
appName = "HelloWorld"
gdxVersion = '1.9.6'
roboVMVersion = '2.3.0'
box2DLightsVersion = '1.4'
ashleyVersion = '1.7.0'
aiVersion = '1.8.0'
}
repositories {
mavenLocal()
mavenCentral()
maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
maven { url "https://oss.sonatype.org/content/repositories/releases/" }
}
}
project(":desktop") {
apply plugin: "java"
dependencies {
compile project(":core")
compile "com.badlogicgames.gdx:gdx-backend-lwjgl:$gdxVersion"
compile "com.badlogicgames.gdx:gdx-platform:$gdxVersion:natives-desktop"
compile "com.badlogicgames.gdx:gdx-box2d-platform:$gdxVersion:natives-desktop"
compile "com.badlogicgames.gdx:gdx-tools:$gdxVersion"
}
}
project(":core") {
apply plugin: "java"
dependencies {
compile "com.badlogicgames.gdx:gdx:$gdxVersion"
compile "com.badlogicgames.gdx:gdx-box2d:$gdxVersion"
}
}
tasks.eclipse.doLast {
delete ".project"
}

Not required to download all injected library with all versions.
If you build with gradle then it only download required artifact with only version that you injected.
Indexing a maven repository
A maven repository is made up of artifact and different versions of those artifact. My guess is that when an IDE like IntelliJ for example reads the remote repository, it will create a separate local cache/mini database of library names+versions available in the repository on machine. That is the indexing part. This makes searching for library names/latest versions etc much faster, as it is now being done consulting the index on your machine instead of accessing the repository each time.
Its nothing major and won't affect your builds, but what your IDE is simply telling is that whenever its looking for a version/dependency name (during auto-complete etc) its having to go online and check for those details (instead of just checking the local cache).
Advantage of indexing - search speed
Disadvantage - you have to ensure the index is kept up to date otherwise your won't always get up to date search results (new versions etc) also somehow 5 to 6 GB is download is painful.
You can disable this notification, If you want by clicking Disable:
Unindexed remote maven repositories found. Disable...
or on Mac OSX Go to
IntellijIDEA > Preferences > Appearance & Behavior > Notifications > Unindexed maven repositories gradle detection > Uncheck
Leave Unindexed remote maven repo, each time if artifact with particular version not found in local repo then it search to remote.

Related

how to set archiveBaseName for local .m2 repository

I'm trying to upgrade a dependency to a project that will ultimately become a dependency to my project. I've made the upgrade and I want to test it locally before I put it out on the repo to be used. I'm learning Gradle and a few Google searches showed me how to add the project to the settings.gradle file. But the dependency project uses aliases for their dependencies (see build.gradle below).
settings.gradle
include ':TransportImpl'
Changed to:
include ':TransportImpl', ':jeromq'
project(':jeromq').projectDir = new File("../zeromq/jeromq")
build.gradle
//project.ext.set("JEROMQ", 'THIRD-PARTY:jeromq:0.4.2')
project.ext.set("JEROMQ", ':jeromq')
If I uncomment the original line (shown commented above), because that apk is in the repo it gets recognized. I'm guessing that this only works for external libraries.
Other things I have tried:
//project.ext.set("JEROMQ", 'C:/Users/username/.m2/repository/THIRD_PARTY/jeromq/0.5.1-SNAPSHOT/jeromq-0.5.1-SNAPSHOT-jeromq.jar')
//project.ext.set("JEROMQ", 'C:\\Users\\username\\.m2\\repository\\THIRD_PARTY\\jeromq\\0.5.1\\jeromq-0.5.1-jeromq.jar')
//implementation filetree(dir: 'C:\\Users\\username\\.m2\\repository\\THIRD_PARTY\\jeromq\\0.5.1', include:['jeromq-0.5.1-jeromq.jar'])
Can anyone give me a tip on how I can assign a variable that points to the local repository and use that variable to set an archiveBaseName?
New Information:
gradle.build for our jeromq project
apply plugin : 'maven'
apply plugin : 'maven-publish'
// Top-level build file where you can add configuration options common to all sub-projects/modules.
ext {
// Nexus paths
nexusUrl='https://nexus.path'
Releases='/Private_Releases'
nexusUsername = project.findProperty("nexusUsername") ?: (System.getenv("NEXUS_USERNAME") ?: "user_name"
nexusPassword = project.findProperty("nexusPassword") ?: (System.getenv("NEXUS_PASSWORD") ?: "password")
// Project versions
jeromqVersion = "0.5.1-SNAPSHOT"
}
allprojects {
// Read only repositories for dependencies; this should never be used to publish
repositories {
mavenCentral()
jcenter()
}
}
The project that uses it as a dependency finds it using the following from its build.gradle file:
// Create aliases for dependencies
project.ext.set("EASY_MOCK", 'Test:easymock:3.5.1')
project.ext.set("OBJENESIS", 'Test:objenesis:2.6')
// **************** HERE ***************************
// THIRD-PARTY is configured to look on the nexus server
project.ext.set("JEROMQ", 'THIRD-PARTY:jeromq:0.4.2') ... or 0.5.1 or 0.5.1-SNAPSHOT ...
allprojects {
// Read only repositories for dependencies; this should never be used to publish
repositories {
mavenCentral()
mavenLocal()
// maven {
// // trying to add my local repo,
// // BUT this still does not change where THIRD-PARTY is pointing to
// url 'file://C:/Users/me/.m2/repository/THIRD_PARTY/jeromq/0.5.1-SNAPSHOT/jeromq-0.5.1-SNAPSHOT-jeromq.jar'
// }
maven {
name 'ReleasesName'
url "$nexusUrl$ReleasesName
}
}
maven {
name 'ReleasesNameSnapshots'
url "$nexusUrl$ReleasesNameSnapshots"
credentials {
username "${rootProject.ext.nexusReadOnlyUsername}"
password "${rootProject.ext.nexusReadOnlyPassword}"
}
}
jcenter {
url "https://jcenter.bintray.com/"
}
}
The only reason I need the alias for that dependency is because it is used in other places.
I'm not entirely sure what you are asking, but I think what you are trying is completely off.
The build you are trying to include is a Maven build, not a Gradle build, so it is unlikely you can simply treat it as it were a Gradle build.
And even if it were a Gradle build, including it like you did would not be the right way. How you tried it is for including multiple projects of a multi-project build, not including external libraries.
If it were a Gradle build, you would use a composite build, which effectively replaces a declared binary dependency by the build output of a "sub-build". But afair this only works cleanly with a Gradle build.
Why don't you simply mvn install your modified jeromq version, add mavenLocal() to your dependencies and depend on that just installed version? That would be the usual way for locally testing new Maven built dependencies.

Cant resolve rar dependencies in Gradle

I'm having a problem fetching active-mq in my gradle project.
It says Could not find activemq-rar
dependencies {
compile 'org.apache.activemq:activemq-rar:5.15.6'
}
Even after adding the type
dependencies {
compile 'org.apache.activemq:activemq-rar:5.15.6#rar'
}
I remember I have hacked it by adding that dependency manually as an artefact in my Nexus 1 but now when migrated to Nexus 3 and its more strict I can't get this fetched. Any Ideas?
And Nexus 3 is not happy storing rar files at all.
https://issues.sonatype.org/browse/NEXUS-11712
Is this compoennt already in your NXRM repository? If so, since you're running v3.15+, you can simply navigate to the component in NXRM UI and in the right side panel there are dependency snippets that will help you how to include a component in your project. Also, make sure that your build.gradle points to the right repositories.
Here's the config I tried. NXRM proxy to Maven Central:
A minmal build.gradle:
plugins {
id 'java'
}
sourceCompatibility = '1.8'
targetCompatibility = '1.8'
version = '1.0.0-SNAPSHOT'
repositories {
maven {
url 'http://localhost:2001/repository/maven-central'
}
}
dependencies {
implementation 'org.apache.activemq:activemq-rar:5.15.8#rar'
}
Then build your app $ gradle build shich should yield success and you should see the activemq-rar-5.15.8.rar in your repository.

Gradle : Something like Maven Parent POM

In our company, many of the different projects use similar technology stack and will have many common features.
So, we want to maintain the common features, dependencies etc. in one common file and refer it in the other projects.
In maven, it is something like creating a separate maven project with the common dependency information and refer that in the other projects as .
I want to do something similar to the maven parent project in gradle, which can be used by all different projects.
I googled for that, but could not find a concise information on how to do that.
We are not allowed to use external thirdparty plugins.
It would be great if someone could explain it how to do that.
in gradle you can do that, but for it you need to have external plugin, otherwise it is not possible at least for now. I have achieved it in this way:
buildscript {
repositories { jcenter() }
dependencies {
classpath 'com.netflix.nebula:nebula-dependency-recommender:4.3.0'
}
}
allprojects {
apply plugin: 'nebula.dependency-recommender'
apply plugin: 'groovy'
apply plugin: 'java'
sourceCompatibility = 1.8
targetCompatibility = 1.8
repositories {
mavenCentral()
jcenter()
maven { url "http://repo1.maven.org/maven2/" }
maven { url "REPOSITORY_OF_YOUR_PARENT_POM.XML" }
}
dependencyRecommendations {
mavenBom module: 'YOUR_PARENT_POM_GROUP:YOUR_PARENT_POM_ID:YOUR_PARENT_POM_VERSION'
}
}
where:
REPOSITORY_OF_YOUR_PARENT_POM.XML - any system like nexus or something else accessible for maven
YOUR_PARENT_POM_GROUP - your parent pom project group (e.g. com.foo.bar.parent)
YOUR_PARENT_POM_ID - your parent pom id (e.g. projects-parent)
YOUR_PARENT_POM_VERSION - your parent pom project version (e.g. 1.0.1)
so, if the external dependency to netflix.nebula is fine , than you can go in this way
Gradle has many extension mechanisms for leveraging build logic located outside of the main script.
A simple thing that can be done is to use an external build script, which can be sourced from the local file system or through an URL, see the documentation on this topic.
If that solution gets too problematic, then you can move to packaging a real plugin that others can apply and potentially configure.
This will allow you to configure much more than dependencies for example.

How do I generate maven-metata.xml with maven-publish and the artifactory-gradle-plugin?

I've written a Gradle plugin in groovy and used Gradle to build it. I've got a local network Artifactory server that I publish the results to using the Gradle Artifactory plugin and the maven-publish plugin in Gradle. I have another Gradle build script that relies on this plugin as a dependency. I've been able to get this all working if I list my dependency with a specific version. I've tried to use a maven version range (ex. '[1.0,2.0)'), but this fails saying it can't find maven-metadata.xml. I checked Artifactory, and sure enough, it isn't there. What do I need to do to produce it, preferably during the build of the plugin?
Here is the build.gradle file for my custom gradle plugin:
buildscript {
repositories {
maven {
url "${artifactory_contextUrl}/plugins-release"
credentials {
username = "${artifactory_user}"
password = "${artifactory_password}"
}
}
}
dependencies {
classpath group: 'org.apache.directory.studio', name: 'org.apache.commons.io', version: '2.4'
classpath group: 'org.jfrog.buildinfo', name: 'build-info-extractor-gradle', version: '2.0.9'
}
}
plugins {
id 'com.jfrog.artifactory' version '3.0.1'
}
apply plugin: 'groovy'
apply plugin: 'maven-publish'
artifactory {
contextUrl = "${artifactory_contextUrl}"
publish {
repository {
repoKey = 'plugins-snapshot-local'
username = "${artifactory_user}"
password = "${artifactory_password}"
maven = true
}
defaults {
publications ('mavenJava')
}
}
resolve {
repository {
repoKey = 'libs-release'
username = "${artifactory_user}"
password = "${artifactory_password}"
maven = true
}
}
}
dependencies {
compile gradleApi()
compile localGroovy()
}
publishing {
publications {
mavenJava(MavenPublication) {
from components.java
}
}
}
I've searched the Gradle, Artifactory, and Maven documentation to understand maven-metadata.xml and how to generate and deploy. It makes sense what it is, and I could probably build one manually, but I can't find anything that explains specifically how to automatically generate it in Gradle with either the maven-publish plugin or the artifactory-gradle-plugin. I don't want to have to manually update the file since that would defeat the automation effort, and I don't want to switch to mvn since I've already invested so much in Gradle.
A groupId had to be added to the publications section. Once implemented, a maven-metadata.xml file was published to the artifact repository.
publishing {
publications {
mavenJava(MavenPublication) {
groupId = 'com.group'
}
}
}
I had the same problem and it turned out that the Artifactory repository was not a Maven repo, it was a Generic repo. It took me forever to notice because I didn't create the repo and I assumed it was a Maven repo, and deploying/resolving otherwise was working as expected.
After switching to a Maven repo, the maven-metadata.xml's were generated upon publishing.
maven-metadata.xml should be handled by Artifactory.
What is your local repository layout in Artifactory?
The accepted answer is correct. And I upvoted it. However, there is also this caveat.
I have a multi module project, so I will use "allprojects". If you have a monolith/single-jar ( :( ).. you can use use a different scope than "allprojects".
They key here is that you set the "group". (and version as well)
allprojects {
apply plugin: 'java-library'
apply plugin: 'maven-publish'
apply plugin: 'com.jfrog.artifactory'
repositories {
jcenter()
}
group = 'com.group'
version = '1.0-SNAPSHOT'
}
Ok, now build.gradle (which in my multi-module project is not the root-build.gradle) (but values in a root build.gradle would be similar)
below is the entire contents of my non-root build.gradle file
// the "name" variable inside the publications/myPublicationName block is getting overwritten. so create a variable here to capture the name (as the artifactid)
def artifactIdForPublicationBlockHolder = "${name}"
dependencies {
testImplementation group: 'junit', name: 'junit', version: junitVersion
}
println("hey.here.read.me")
println("group=${group}")
println("version=${version}")
println("artifactId=${name}")
publishing {
publications {
myCustomPublicationName(MavenPublication) {
// groupId, artifactId and version have defaults, so do not arbitrarily override : https://docs.gradle.org/current/userguide/publishing_maven.html#publishing_maven:publications
//your value below could be slightly different, look for *.jar after you do ./gradlew. build (note, this path value (of "./") is relative to the non-root-build.gradle path, not the overall root-build.gradle
"./build/libs/${artifactIdForPublicationBlockHolder}-${version}.jar"
}
}
}
As the link says, you'll get defaults for
// groupId, artifactId and version have defaults, so do not arbitrarily override : https://docs.gradle.org/current/userguide/publishing_maven.html#publishing_maven:publications
You just have to set these values, like I show above with the
group = 'com.group'
version = '1.0-SNAPSHOT'
code
After going thru the grinder a few times with the
myCustomPublicationName(MavenPublication)
I find the least amount of stuff I custom-set, the better. and prefer to piggy back on the defaults...which means setting the values which drive the defaults... in the build.gradle .. and not to set the myCustomPublicationName(MavenPublication)
altering the values inside
myCustomPublicationName(MavenPublication)
should be reserved (IMHO) to when the defaults don't work for you. which usually is a very small minority of time.
note:
"${name}" at the top of my non-root-gradle.build is getting populated by the directory structure of my multi-module project.
i don't know how it works in a non multi-module since i never write monoliths.
settings.gradle example in my code:
rootProject.name = 'com.me.myproject-rootProjectName'
include ':source:java:mydatalayer'
include ':source:java:mybizlogic'
include ':source:java:mydomain'
Bonus Quote below:
Moreover, modular decomposition represents a key component of software
quality. If you have a tightly coupled system, when you tweak one
component, the whole system breaks. If you thought in terms of APIs,
the intermodular boundaries are clear, so you can maintain and improve
one module without affecting the others.
Massive refactorings prove difficult. If you built something as a
monolithic system and then find you had repeated code all over the
place, and you want to refactor it properly, you'll have a massive
job. In contrast, if you wrote it as components, but you got some of
the component boundaries a little wrong, you can tweak them easily.
-- Joshua Bloch, former Chief Java Architect at Google. Modular Decomposition Link

Dependenciies in Gradle not working correctly

We have a project that is using Java 1.5 and we are trying to convert from Maven to Gradle.
We have a repository that is local to us containing all the versions of all the jars we need as the dev environment has no access to the internet.
The problem we are seeing is that it cannot find the commons-io jar and keeps trying to goto the external maven repo. we have not even set that up so where is it finding it from?
we have repositories and dependencies set up in the All projects section as follows
allprojects {
apply plugin: 'java'
sourceCompatibility = 1.5
targetCompatibility = 1.5
project.tasks.withType(AbstractCompile, { AbstractCompile ac -> ac.options.bootClasspath = "C:/Program Files/java/1.5.0_14/jre/lib/rt.jar" })
repositories {
mavenLocal()
maven { url "http://internalrepo/maven-local" }
}
dependencies {
compile "org.apache.commons:commons-io:1.3.2"
}
But its reporting
Could not resolve org.apache.commons:commons-io:1.3.2.
inconsistent module metadata found
even though it works fine in Maven using mvn install
Gradle will never query a repo that isn't set up. mavenlocal() is misspelled (should be mavenLocal()), which will make the build fail. "Inconsistent metadata" could mean that the group ID, artifact ID, or version in the POM doesn't match the one in the build script. mavenLocal() should only be used if the Gradle build needs to exchange artifacts with local Maven builds.
Found the issue,
Unbeknownst to me there was a hidden repo in the maven settings.xml in the maven install folder.
Adding that resolved the issue.

Resources