How to use maven or gradle on an unreliable network? - maven

Naive use of the built-in settings sometimes results in hanging connections. It is likely some network issue outside of my control.
I would like to know how I can set a timeout and a retry for both downloading artifacts and publishing artifacts.
I have found a connection timeout and read timeout setting for HTTP wagon, but I have not found a retry option.
For publishing, I found a retry option, but no timeout setting.
http://maven.apache.org/guides/mini/guide-http-settings.html#Connection_Timeouts
https://maven.apache.org/plugins/maven-deploy-plugin/deploy-mojo.html
Edit what I want is the equivalent of curl's --max-time, --retry and --retry-delay options for maven and gradle downloads and publishes.

I tried to dig into this topic as well, but did not find much. Except one thing. Since Gradle 4.3 (thanks to PR) you can put to gradle.properties something like:
systemProp.org.gradle.internal.http.connectionTimeout=120000
systemProp.org.gradle.internal.http.socketTimeout=120000
Maybe once https://github.com/gradle/gradle/issues/4629 would be resolved, life will be much easier.

Gradle has built-in Maven compatability and it will create the .m2 or .gradle archive directories for you. To retry, all you need to do is just re-run your build over and over until all artifacts are complete.

Related

Why does mvn command take forever?

$ mvn dependency:tree -Dincludes=com.fasterxml.jackson.core
[INFO] Scanning for projects...
Downloading from central: https://repo.maven.apache.org/maven2/org/jooq/jooq-codegen-maven/3.9.1/jooq-codegen-maven-3.9.1.pom
I am executing this command, but it takes forever and got stuck here. My network is all good and I tried the command at different time but ended with the same. What might be the problem?
I've seen all kinds of problems with this. No idea what your specific issue is but here's a few possibilities
Your traffic is going through some kind of stateful package inspection/firewall/antivirus/IDS/bluecoat/etc and it's interfering and/or silently dropping the connection
There's some issues validating the certificate
The server is offline, which happens from time to time but is unlikely
Your settings.xml file is redirectly elsewhere and that "elsewhere" location is misbehaving.
A few suggestions to work around the issue
try disabling https (i know) by using settings.xml to remap the apache repo from https to http.
try disabling any antivirus/firewall (temporarily) to try and isolate the problem.
check to make sure your hard drive is not full
To troubleshoot run mvn with --debug flat to see any issues to find if mvn is stuck on any command. Based on this, if you still need help, please update this question
mvn --debug ...

Groovy install HTTPBuilder now that Codehaus shutdown?

I'm a groovy n00b and trying to use http-builder, but ALL of the examples on the web just reference the same #Grab statement which doesn't work. I assume it is because codehaus.org isn't hosting groovy stuff anymore. I've tried downloading the source from github and building it with Maven, but the build fails.
How and where am I supposed to get httpbuilder for groovy?
Things I've already tried:
Deleting the grapes directory from this post didn't work.
I got this code snippet from this other post, but it doesn't work for me either.
#Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.7' )
def http = new groovyx.net.http.HTTPBuilder('http://www.codehaus.org')
println http
Here is the error in the IntelliJ console:
/Users/kenny/Sites/inadaydevelopment.com/reports/fetch_windows_appstore_report.groovy
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during conversion: Error grabbing Grapes --
[download failed: xerces#xercesImpl;2.9.1!xercesImpl.jar,
download failed: xml-apis#xml-apis;1.3.04!xml-apis.jar]
EDIT 1:
Tried running it from the command line, still same error message.
Tried grab artifacts in IntelliJ, but that failed too:
I wondered if it was Mac related, and bingo. I uploaded this script to my CentOS server and it ran just fine. There is something related to MacOSX+groovy that is causing the problem.
Got it! It looks like it was the maven cache that was the problem.
I found the solution on this page:
rm -rf ~/.m2/repository ~/.groovy/grapes
I had previously tried removing the ~/.groovy/grapes cache, but that didn't fix the problem. Removing the ~/.m2/repository is what actually did it for me.
This works fine from the groovy console and from the command line.
The http-builder project is hosted on Maven Central.
Grab will actually use JCenter, but JCenter mirrors Maven Central so this resolution works.
Not sure why you would have trouble resolving transitive dependencies... it may be that you are using a proxy, for example... could also be your settings for Maven or Ivy... Check the Groovy Grapes documentation to see if you might inadvertently have something configured that causes this problem.
Also, try from groovyConsole or the command-line to rule out some conflicts within IntelliJ.
From IntelliJ, point to the Grab annotation and hit Alt+Enter. Then select grab artifacts and Enter.
It should work (works for me) and you should be able to run the script without problems.
If it complains about Ivy not being in the classpath, just add Ivy to the module dependencies and it will work.

Tell gradle to bypass dependency checks

I am at a clients site, behind a firewall. Im trying to compile but gradle keeps trying to check my dependencies. The corporate firewall explicitly blocks maven downloads so my build is failing. Now I have compiled before, so the dependencies do already exist in my [user]/.gradle folder, but its been more than 24 hours so gradle is trying to do its daily "lets check the repo and make sure nothing changed stuff."
Is there a command switch or anything that im just simply not seeing here to tell gradle to bypass this version check and simply compile the code? I would even be happy with a command switch that says I don't care if dependency resolution failed, compile anyways.
Try the --offline command line switch.
You still have to get it to compile once to grab the dependencies online, so --offline only works once you have compiled successfully one time.
The only way I was able to get this to work was to tether my phone to my machine and connect to my cellular service for internet instead. That way, I bypassed my company's proxy and firewall and was able to download the dependencies once.
After that, I was able to then compile when connected to the corporate network instead. Just don't clean your project or else you'll be back where you are right now.
I found this question because I wanted to short-circuit the Maven dependency checks in a git pre-commit hook that runs gradle check for better performance. I ended up doing:
if ! ( gradle --offline check || gradle check ); then
exit 1
fi
which tries once with --offline, and then tries again without --offline if the first try fails.

Forcing authentication on Maven release

I have a problem getting Maven to release to a Nexus server. Seemingly, it refuses to use my provided username and password (but there might be other problems as well).
When I first type 'mvn release:perform', I get a'not authorized'-error. However, some files are created on the Nexus, namely a pom with checksums etc. When I try a second time (without changing anything), I get a different error: '400 bad request'
When I delete the files and try again, I get the first error once again.
I have run this with the -X flag to see if I can make any sense of what is happening, and I have discovered that the first time I run the command, maven omits my username and password provided in settings.xml:
[INFO] [DEBUG] Using connector WagonRepositoryConnector with priority 0 for http://nexus.example.com/content/repositories/releases
When I run it the second time, it includes my credentials:
[INFO] [DEBUG] Using connector WagonRepositoryConnector with priority 0 for http://nexus.example.com/content/repositories/releases/ as developers
Notice it says 'as developers'
Of course I don't know that the fact that it prints it differently actually means anything, but it seems that way.
When I allow redeploy for the releases repository in Nexus, I always get the first variant (not authorized).
If anyone can tell me how I might force Maven to use my credentials (if that is indeed what it is not doing) or on what else might be wrong, I would be very happy.
I have got it working now, by specifying in the maven release plugin that it only deploy, and not deploy and deploy site as is default.
mvn site:deploy fails with the error: Wagon protocol 'http' does not support directory copying.
Of course, my original error message did not refer very much to site at all.
Way to produce useful error messages, Maven!
I found a way to force preemptive authentication here: http://maven.apache.org/guides/mini/guide-http-settings.html (it didn't solve my problem, but it is an answer to the title.)

how to prevent gradle from downloading dependencies

We would like to have a script that does "svn update" and if the depedency.gradle file is in that list of updates, we would like to run a task that ONLY updates dependencies so the developers machine is up to date. What would that task be? I don't see it when running "gradle tasks". Looking for an updatejars or something.
When we build our project, we don't want it to check for jar updates at all!!!! most because that only needs to be done in 2 situations which are #1 above and when someone is updating the dependency.gradle file themselves. For the second thing, they can just run "gradle updatejars" once I know the answer to question #1 that is.
Any ideas? I am just getting into gradle and we really want to keep a consistent environment where when we run our update script, it gets the source code AND the jars in one atomic sweep and we are no longer bothered by checking the repositories every build.
It would be nice to know how to do it by changing the build.gradle file if possible. If not, is there a command line option? (The build.gradle obviously would give me a command line option which is why I prefer that method as I could say compile does not depend on downloading jars).
Regarding the second question. As far as I understand, Gradle will not attempt to do remote lookups or try to download the jar if it is already in the local cache. This should be true for jars declared with a static version, e.g. testCompile 'junit:junit:4.10'.
If you have dynamic versions, e.g. 1.+ or 1.0-SNAPSHOT, etc. then Gradle has to do a check every now and then. You can fine tune the cache expiry for such dependencies.
To make sure Gradle does not do remote lookups you can also use --offline option. See this doc for details.
With regard to svn update, you have at least 3 options:
Try to use an SvnKit plugin for Gradle
Use the ant svn task (here's how to do svn checkout)
Run external command from Gradle. Use the ExecPlugin or just implement it yourself using Groovy API.
Looks like the 1st question I can do with the answer in this post
how to tell gradle to download all the source jars
so I can just gradle eclipse and it will download new jars and update my classpath...nice.

Resources