Gradle still downloads dependencies already available in local repo - gradle

I have set gradle up to use a local maven repository (gmaven_stable file) using the "offline.gradle" file method in the ".gradle\init.d" directory.
I constantly update the local maven repo with already downloaded files in the gradle dependency cache (.gradle\caches\modules-2\files-2.1).
However, I have realized that gradle still downloads some dependencies during "gradle sync" and "compile time" most of which I am already having in my local maven repo. Dependencies like "junit" , "google play services" and many others are always downloaded whenever I hit "Build" or "Run" button in Android Studio.
I would much appreciate if I would be guided to stop this behaviour in gradle. I am spending a lot of time gathering dependencies to build my local maven repo because of low and costly internet access in my area. It is quite annoying whenever that scarce internet I have access to is used by "Gradle" to re-download those dependencies I have already cached in my local maven repo.

Gradle has a built-in option that will avoid network access:
The --offline command line switch tells Gradle to always use dependency modules from the cache, regardless if they are due to be checked again. When running with offline, Gradle will never attempt to access the network to perform dependency resolution. If required modules are not present in the dependency cache, build execution will fail.
https://docs.gradle.org/current/userguide/dynamic_versions.html#sec:offline-mode
I constantly update the local maven repo with already downloaded files in the gradle dependency cache (.gradle\caches\modules-2\files-2.1).
You should not be manually messing with Gradle's internal cache folder. Instead, you place your JARs in a folder else where you can access them such as your home directory. Then, you can configure a flat directory repository:
repositories {
flatDir {
dirs("mylibs")
}
}
https://docs.gradle.org/current/userguide/declaring_repositories.html#sub:flat_dir_resolver
You will also need to download and define various Gradle plugins that are used in your project as well and configure them the same way as the application repositories:
// settings.gradle.kts
pluginManagement {
repositories {
flatDir {
dirs("mylibs")
}
}
}

Related

Make a Gradle project work without an internet connection

Given a Gradle project, that has:
Runtime/Compile-time dependencies.
Source/Javadoc Jars of dependencies.
Gradle plugins.
All of which are retrieved from the internet.
How can I download all of these online dependencies to a local folder, and then use them locally in a Gradle project?
The goal is to end up with a project that doesn't rely on an internet connection at all.
You can do a one-time download of your required dependencies to a local folder, and add a local "maven repo", besides the dependencies, it should also work for sources.
repositories {
maven {
url 'file://C:/LocalRepo'
}
}

Idea (via Gradle) "Could not resolve:" dependencies from Sonatype Nexus Proxy of mavenCentral()

I have a hosted Sonatype Nexus repository on a local network. It has a Maven group containing a proxy repository for the public Maven repository. In the past this configuration was superb and I encountered few issues.
I recently configured https and ssl on the Nexus repository as Docker would not easily log into insecure Nexus Docker registries during CI/CD processes. I did end up re-configuring the Maven repositories at this point.
Then I updated build.gradle:
repositories {
maven {
credentials {
username "${nexusUsername}"
password "${nexusPassword}"
}
name = 'RepositoryName'
url = "https://${nexusURL}:${nexusPort}/repository/maven-public"
}
}
with the nexus* variables defined in ~/.gradle/gradle.properties:
nexus<Variable>=<value>
I have also added the appropriate certificate to the java jre keystore with the keytool and added the certificate in Idea's settings (File > Settings... : Tools > Server Certificates).
When trying to download dependencies (through the Maven proxy) using Gradle (by clicking "import changes" on the pop-up notification in Intellij Idea) the Build output shows "Could not resolve: <dependency>" for each dependency. This behavior is consistent across all of my projects (even ones that previously were able to resolve dependencies).
I have, under most circumstances, been able to get the dependencies to resolve through Nexus when running a Gradle task (:dependencies, :idea, :build) from the project's build.gradle file from a command line. The resulting downloads are not available to the project in Idea. However, after the dependencies have been resolved once, the artifacts are cached in Nexus's Maven proxy repository allowing Gradle/Idea to correctly resolve all dependencies.
What could be causing Gradle/Idea's failure to resolve artifacts through Nexus's Maven proxy? Is there a way to get Gradle/Idea to correctly resolve dependencies through the Nexus Maven group/proxy?
For now I've just added mavenCentral() to the repository list in build.gradle but I would prefer to only include the Nexus Maven group in the future. Caching resources for 1GB/s download is really nice. I would also like to better understand Gradle/Idea and what is causing this issue.
More information:
Intellij Idea has been reinstalled to version 2018.2.6 Build #IC-182.5107.16 during the process of trying to fix this issue. The old version is lost to history.
Gradle has been updated to version 4.10.2. Previous version was 4.5.1.
I've printed each of the nexus* variables via println to ensure the values were correct.
No configuration of Idea's settings for Gradle (local Gradle distribution, default Gradle wrapper, Gradle 'wrapper' task configuration) managed to resolve dependencies.
Every configuration of deleting at least one of ./.idea, ./.gradle, and ~/.gradle/caches was tried.
Idea is not in offline-mode. Sequences of toggling offline-mode and "Refresh all Gradle projects" did not change the outcome.
I have run an Idea configuration of Gradle's dependencies task with --warning-mode all --debug and compared the log to the output of gradle dependencies --warning-mode all --debug on command line. The logs seem to be producing the same statements (in wildly different orders) until the "Could not resolve:" message appears in the Idea output. I did not find any nearby error messages that would explain the failure. If it would help diagnose the issue I can upload those files.
I have tried setting the repository to point directly at the Nexus Maven proxy instead of the Maven group. This did not allow Gradle/Idea to resolve dependencies.
Should I be using a http/https proxy for Gradle? I don't understand the goal of using a proxy in this context.
I have not done anything with Grail. I don't know what Grail is and suspect I do not currently need it.
OS is Windows 10.
Dependencies are not resolved when using compile or implementation in build.gradle.
Transitive dependencies don't seem to be relevant.
I must not have added the certificate to the correct jre installation. I added it more recently and the issue was resolved.
Also, superstitious notes for anyone else having a similar issue:
I added the certificate (a wildcard certificate) under the alias (using the -alias command line parameter for keytool): *.example.com
I also added it under an alias for the full address: nexus.example.com
I don't know whether either of those had any impact on Idea/Gradles' success in resolving artifacts. I believe it was working before I added the second alias.

When does gradle store in .m2 and when in cache?

In which scenario will gradle store artifacts in the directory .m2 and in which scenario will it store them in gradle\caches?
I am trying to resolve my issue wherein I have a dependency within my local build
Gradle will read from your local maven repository only when you declare it as a valid repository:
repositories {
mavenLocal()
}
Gradle will write to your local maven repository only when you publish artifacts and tell it to publish to the local maven repo.
If you are using the maven plugin, when executing the task install
If you are using the maven-publish plugin, when executing the task publishToMavenLocal
Gradle will use its own internal cache for all resolved dependencies, including ones coming from the local maven repository.
For example, if you use a dependency org:foo:1.0 from your maven local repository, the metadata and artifact will be copied to the Gradle cache on first resolution. From then on, the dependency will be resolved from the Gradle cache.
However, if the dependency is changing, such as when using a -SNAPSHOT version, the Gradle cache will by default keep the last one resolved for 24h. After which it will perform a new resolution, hitting again the local maven repository in this example.
See the documentation for controlling that cache duration for dynamic and/or changing dependencies.

Why gradle not creating local repository and downloading same dependencies for every project

I'm using Gradle with Java Project in IntelliJ Idea.
I see that Gradle downloading dependencies for first time on opening project.
But there is another project with same dependencies then also it's re-downloading those libs. why?
Why doesn't it maintain Maven like local repository even after configured?
repositories {
mavenLocal()
mavenCentral()
jcenter()
}
How can Gradle maintain local repository and next it should first check local repo and go for download if no matching dependencies found?
With that piece of code you instruct gradle to look at the local maven repository, then at the central maven repository and last in JCenter when looking for dependencies. The first one it finds your dependency it takes it from.
It does not instruct Gradle to put resolved dependencies to the local maven repository though. This is e. g. helpful if you have two projects in two separate builds, one depends on the other and you install the first dependency to the local maven repository with the respective Gradle task and then build the second one, depending on the version you just built and installed.
Gradle has a resolution cache though in ~/.gradle/caches/modules-2/files-2.1/ where it caches all downloaded dependencies and also reuses this from different builds.

Fetch all dependencies, put them in a new local Maven repository using Gradle

I have a Gradle project with several subprojects and many, many dependencies. I would like to have a simple way to tell Gradle to download all dependencies (including those under buildscript!) and put them in a local Maven/Ivy repository for later use. The Gradle script should then be able to pull all dependencies from the local repository.
Background: I need to build the application on a server which has absolutely no access to any public Maven repositories, so all dependencies must already be present on the host. I've tried a flat directory, but I have not found it easy to resolve the transitive dependencies, and managing them by hand is not an option. Copying the Gradle cache did not work, either.
Can anyone suggest something? Thanks.
I found a solution, namely to install Apache Archive (http://archiva.apache.org/) locally and set it up as a proxy. Then I copied the entire installation to the target-server and disabled the remote repositories. The dependencies could then be fetched locally.

Resources