I'm running Spark Notebook and struggling with dependancies. The server runs in an Amazon VPC, so has no direct Internet access.
In order to load dependancies, created a repo in an S3 bucket and this works fine, getting all the POM's and JAR's until it gets to
Installing the s3:// URLStreamHandler via
java.net.URL.setURLStreamHandlerFactory
Creating a new Ivy URLHandlerDispatcher to handle s3:// URLs
[info] :: resolving dependencies :: org.scala-lang#scala;2.10.4
[info] confs: [compile, test, runtime]
[info] Resolving default#ffx-analytics_2.10;1.1 ...
[info] found default#ffx-analytics_2.10;1.1 in s3-repo
[info] Resolving org.scala-lang#scala-library;2.10.4 ...
[info] found org.scala-lang#scala-library;2.10.4 in s3-repo
[info] [2.10.4] org.scala-lang#scala-library;2.10.4
[info] Resolving com.github.nscala-time#nscala-time_2.10;1.4.0 ...
[info] found com.github.nscala-time#nscala-time_2.10;1.4.0 in s3-repo
[info] Resolving joda-time#joda-time;2.4 ...
[info] found joda-time#joda-time;2.4 in s3-repo
[info] Resolving org.joda#joda-convert;1.2 ...
[info] found org.joda#joda-convert;1.2 in s3-repo
[info] Resolving ua_parser#ua-parser;1.3.1-SNAPSHOT ...
[info] found ua_parser#ua-parser;1.3.1-SNAPSHOT in s3-repo
[info] Resolving org.yaml#snakeyaml;1.10 ...
[info] found org.yaml#snakeyaml;1.10 in s3-repo
[info] Resolving commons-collections#commons-collections;3.2.1 ...
[info] Resolving org.apache.commons#commons-parent;9 ...
[info] Resolving org.apache#apache;4 ...
[error] Server access Error: Connection timed out url=https://repo1.maven.org/maven2/org/apache/apache/4/apache-4.jar
So I tried to download the JAR from another server (with internet access) like so:
ubuntu#ip-10-126-49-161:~$ wget https://repo1.maven.org/maven2/org/apache/apache/4/apache-4.jar
--2015-09-07 00:37:42-- https://repo1.maven.org/maven2/org/apache/apache/4/apache-4.jar
Connecting to 172.16.100.11:8080... connected.
Proxy request sent, awaiting response... 404 Not Found
2015-09-07 00:37:45 ERROR 404: Not Found.
I cant find an apache-4.jar anywhere. Looking at the POM there is no JAR for it.
Any tips would be appreciated.
UPDATE #1:
I tried importing another JAR that lead down a similar path:
[info] Resolving net.sf.opencsv#opencsv;2.0 ...
[info] found net.sf.opencsv#opencsv;2.0 in s3-repo
[info] Resolving commons-codec#commons-codec;1.10 ...
[info] Resolving org.apache.commons#commons-parent;35 ...
[info] Resolving org.apache#apache;15 ...
When I look at the POM for Apache 35, it lists Apache 15 as a parent.
These parents appear to be a real nightmare. Hopefully it is just something simple I need to switch on to make this all work.
I found the answer here:
https://theholyjava.wordpress.com/2011/01/26/using-ivy-with-pom-xml/
"Ivy ignores the packaging=pom on the parent module and will always try to find a .jar for it, thus wasting precious time. The workaround is to create a fake, empty .jar, for example via echo “” > emptyJarToSatisfyIvy.jar."
Related
I am trying to build Metasfresh ERP backend (server-side) suite https://github.com/metasfresh/metasfresh/tree/master/backend and my maven build stops at the project https://github.com/metasfresh/metasfresh/tree/master/backend/de.metas.adempiere.adempiere/migration because of missing resrouces required by the maven-assembly-plugin.
Specifically, the maven log says:
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) # de.metas.adempiere.adempiere.migration-sql ---
[INFO] Building jar: C:\Workspace-Brugere\metasfresh\backend\de.metas.adempiere.adempiere\migration\target\de.metas.adempiere.adempiere.migration-sql-10.0.0.jar
[INFO]
[INFO] --- maven-assembly-plugin:3.0.0:single (make-postgresql-assembly) # de.metas.adempiere.adempiere.migration-sql ---
Downloading: https://repository.apache.org/snapshots/de/metas/metasfresh-assemblies/maven-metadata.xml
Downloading: https://repo.metasfresh.com/repository/mvn-master/de/metas/metasfresh-assemblies/maven-metadata.xml
Downloaded: https://repo.metasfresh.com/repository/mvn-master/de/metas/metasfresh-assemblies/maven-metadata.xml (8.7 kB at 5.5 kB/s)
Downloading: https://repo.metasfresh.com/repository/mvn-master/de/metas/de.metas.parent.general/5.159.1-21669+master/de.metas.parent.general-5.159.1-21669+master.pom
Downloading: https://repo.metasfresh.com/repository/mvn-master/de/metas/de.metas.parent.general/5.159.1-21672+master/de.metas.parent.general-5.159.1-21672+master.pom
Downloading: https://repo.metasfresh.com/repository/mvn-master/de/metas/de.metas.parent.general/5.159.1-21673+master/de.metas.parent.general-5.159.1-21673+master.pom
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] de.metas.parent .................................... SUCCESS [ 2.944 s]
...
[INFO] de.metas.monitoring ................................ SUCCESS [ 10.034 s]
[INFO] de.metas.adempiere.adempiere.patched-ecs ........... SUCCESS [ 13.617 s]
[INFO] de.metas.adempiere.adempiere.migration-sql ......... FAILURE [ 36.118 s]
[INFO] metasfresh-report .................................. SKIPPED
...
[INFO] metasfresh-dist-dist ............................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:36 min
[INFO] Finished at: 2020-12-25T19:01:04+02:00
[INFO] Final Memory: 146M/1207M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:3.0.0:single (make-postgresql-assembly)
on project de.metas.adempiere.adempiere.migration-sql:
Execution make-postgresql-assembly of goal org.apache.maven.plugins:maven-assembly-plugin:3.0.0:single failed:
Plugin org.apache.maven.plugins:maven-assembly-plugin:3.0.0 or one of its dependencies could not be resolved:
Failed to collect dependencies at org.apache.maven.plugins:maven-assembly-plugin:jar:3.0.0 ->
de.metas:metasfresh-assemblies:jar:5.159.1-21669+master:
Failed to read artifact descriptor for de.metas:metasfresh-assemblies:jar:5.159.1-21669+master:
Could not find artifact de.metas:de.metas.parent.general:pom:5.159.1-21669+master in metasfresh-repo
(https://repo.metasfresh.com/repository/mvn-master/) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
I checked and the maven-assembly-plugin reads resources from the
https://repo.metasfresh.com/repository/mvn-master/de/metas/metasfresh-assemblies/maven-metadata.xml
and it have entry for 5.1159.1-21669+master:
<version>5.159.1-21667+master</version>
<version>5.159.1-21668+master</version>
<version>5.159.1-21669+master</version>
<version>5.159.1-21670+master</version>
<version>5.159.1-21671+master</version>
<version>5.159.1-21672+master</version>
<version>5.159.1-21673+master</version>
<version>5.159.1-21674+master</version>
but there is no entry at link https://repo.metasfresh.com/repository/mvn-master/de/metas/de.metas.parent.general/5.159.1-21669+master/de.metas.parent.general-5.159.1-21669+master.pom
So - this is very strange thing with this project. Maybe there are some administrative/commercial/organizational issues in this open source project and they are keeping forbidden some necessary files intentionally.
My question is - how can I ask maven plugins to skip missing resources? And just see what is happening?
Maybe I can download maven-metadata.xml, remove the entries for which there are no versions and then hope that maven-assembly-plugin will use local (handcrafted) mave-metadata.xml?
de.metas:metasfresh-assemblies:jar:5.159.1-21669+master's POM contains:
...
<parent>
<groupId>de.metas</groupId>
<artifactId>de.metas.parent.general</artifactId>
<version>5.159.1-21669+master</version>
...
</parent>
...
but, as you found, there is no such artifact.
This is also not a resource (as in src/[main|test]/resources), but a parent POM. You can't "skip" parent POMs, i.e. you can't build child POMs if the parent is not available. This is like trying to compile a sub-class if the super-class isn't available in the classpath.
Willing to use the gradle-enterprise-maven-extension
to migrate a project from maven to gradle,
I cloned the maven-build-scan-quickstart provided by the gradle team.
But runing mvn install, as specified, after acceptation of gradle terms of service,
I receive the following message:
UnknownHostException: scans-in.gradle.com
I tried to configure the proxy in several manners:
Using commannd line option:
mvn install -Dhttps.proxyHost=localhost:8888 -Dhttps.proxyPort=localhost:8888
Using a gradle.properties file in a directory at %GRADLE_USER_HOME% directory location:
(proxy is also configured in maven's settings.xml and need no password it works fine when using maven in standard ways)
gradle.properties :
systemProp.http.proxyHost=localhost
systemProp.http.proxyPort=8888
systemProp.http.nonProxyHosts=*.nonproxyrepos.com|localhost|127.0.0.1
systemProp.https.proxyHost=localhost
systemProp.https.proxyPort=8888
systemProp.https.nonProxyHosts=*.nonproxyrepos.com|localhost|127.0.0.1
But I still receive the same message:
~/github/maven-build-scan-quickstart
$ mvn install
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.830 s
[INFO] Finished at: 2019-12-31T13:09:19+01:00
[INFO] ------------------------------------------------------------------------
[INFO] 7 goals, 7 executed
Publishing a build scan to scans.gradle.com requires accepting the Gradle Terms of Service defined at https://gradle.com/terms-of-service. Do you accept these terms? (yes/no): yes
[INFO] Gradle Terms of Service accepted.
[INFO]
[INFO] Publishing build scan...
[INFO]
[INFO] A network error occurred.
[INFO]
[INFO] The hostname 'scans-in.gradle.com' could not be resolved.
[INFO] You may be disconnected from the Internet.
[INFO]
[INFO] If you require assistance with this problem, please report it via https://gradle.com/help/plugin and include the following information via copy/paste.
[INFO]
[INFO] ----------
[INFO] Maven version: 3.6.2
[INFO] Extension version: 1.3.3
[INFO] Request URL: https://scans-in.gradle.com/in/maven/3.6.2/1.3.3
[INFO] Request ID: 1234567e-abcd-23de-c2d3-3fbbccd14a32
[INFO] Exception: java.net.UnknownHostException: scans-in.gradle.com
[INFO] ----------
[INFO]
I am trying to use a maven virtual repository that is linked up to an SBT local repository to resolve dependencies, but I keep getting 404s when I try to download the artifacts. They are listed fine on the UI, but right clicking on the artifacts and hitting "Download" give me a 404 error. Has anyone run into this problem. We're using JFrogs cloud hosted version of Artifactory on AWS. Our virtual Gradle repository is setup the same way and works as expected ...
mvn install output:
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building my-app 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for com.example:myjar:jar:1.0-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.220 s
[INFO] Finished at: 2018-04-13T10:41:55-03:00
[INFO] Final Memory: 9M/309M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project my-app: Could not resolve dependencies for project com.mycompany.app:my-app:jar:1.0-SNAPSHOT: Failure to find com.example:myjar:jar:1.0-SNAPSHOT in https://example.jfrog.io/example/maven was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
It seems that there are differences between snapshot policies between your local repositories referenced from the virtual one.
https://www.jfrog.com/jira/browse/RTFACT-17521
I am new to camel and I've been trying to run one of the examples found in camel called camel-example-twitter-websocket, which may be found here.
When I run mvn compile it works successfully
$ mvn compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Camel :: Example :: Twitter WebSocket 2.17-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.4:enforce (enforce-maven) # camel-example-twitter-websocket ---
[INFO]
[INFO] --- maven-bundle-plugin:2.3.7:cleanVersions (versions) # camel-example-twitter-websocket ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) # camel-example-twitter-websocket ---
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) # camel-example-twitter-websocket ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default) # camel-example-twitter-websocket ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) # camel-example-twitter-websocket ---
[INFO] Nothing to compile - all classes are up to date
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.504 s
[INFO] Finished at: 2016-03-18T15:27:30-04:00
[INFO] Final Memory: 22M/437M
[INFO] ------------------------------------------------------------------------
But when I run the second step mvn camel:run it does not work and I get the following output
$ mvn camel:run
[INFO] Scanning for projects...
[WARNING] The POM for org.apache.camel:camel-maven-plugin:jar:2.17-20151107.033312-28 is invalid, transitive dependencies (if any) will not be av ailable, enable debug logging for more details
[WARNING] Failed to retrieve plugin descriptor for org.apache.camel:camel-maven-plugin:2.17-SNAPSHOT: Plugin org.apache.camel:camel-maven-plugin: 2.17-SNAPSHOT or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.camel:camel-maven-plugin:jar:2. 17-SNAPSHOT
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no dependency information available
[WARNING] Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in https://repo.maven.apache.org/maven2 w as cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced
[WARNING] The POM for org.apache.maven.plugins:maven-surefire-plugin:jar:{maven-surefire-plugin-version} is missing, no dependency information av ailable
[WARNING] Failed to retrieve plugin descriptor for org.apache.maven.plugins:maven-surefire-plugin:{maven-surefire-plugin-version}: Plugin org.apa che.maven.plugins:maven-surefire-plugin:{maven-surefire-plugin-version} or one of its dependencies could not be resolved: Failure to find org.apa che.maven.plugins:maven-surefire-plugin:jar:{maven-surefire-plugin-version} in https://repo.maven.apache.org/maven2 was cached in the local repos itory, resolution will not be reattempted until the update interval of central has elapsed or updates are forced
Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml
Downloading: https://repository.apache.org/content/repositories/snapshots/org/apache/maven/plugins/maven-metadata.xml
Downloading: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml
Downloading: https://repository.apache.org/content/repositories/snapshots/org/codehaus/mojo/maven-metadata.xml
Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (13 KB at 36.9 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (20 KB at 57.4 KB/sec)
Downloaded: https://repository.apache.org/content/repositories/snapshots/org/apache/maven/plugins/maven-metadata.xml (9 KB at 9.7 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.140 s
[INFO] Finished at: 2016-03-18T15:28:29-04:00
[INFO] Final Memory: 27M/327M
[INFO] ------------------------------------------------------------------------
[ERROR] No plugin found for prefix 'camel' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] availabl e from the repositories [local (C:\Users\myNameHere\.m2\repository), apache.snapshots (https://repository.apache.org/content/repositories/snapshots/), central (https://repo.maven.apache.org/maven2)] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoPluginFoundForPrefixException
It looks like this is more of a Maven issue than it is a Camel issue. I put in the default .m2/settings.xml file (before I had a mirror configured for my companies Nexus repository. I thought that might be messing things up so I took it all out). Out of desperation I even tried adding in the plugin repository from here http://camel.apache.org/maven-2-snapshot-repository-in-pom.html but that did not work.
Read the readme file how to run the examples.
Some examples run using mvn camel:run and others using mvn exec:java and what else.
And you run from the master branch in the source code. Instead you should download a version of Camel such as the latest 2.16.2 and use that. It ships the examples in the examples directory: http://camel.apache.org/download
If you run from master branch in the source code. You need to rebuild Camel first, see building: http://camel.apache.org/building.html
I've tested myself, and putting the plugin 2.16 don't solve it neither.
But in readme, I found this :
We have described this in more details at the Camel twitter documentation:
http://camel.apache.org/twitter
You will need to compile this example first:
mvn compile
To run the example type
mvn exec:java
Then open a browser to see live twitter updates in the web page
http://localhost:9090
Worked for me, I hope it will work for you too. I'll investigate the problem with the plugin that doesn't seems to export the prefix latter.
I have a Jenkins/Maven build as well as Artifactory. All instances run in AWS EC2. Here what is happening. I have had a dozen successful builds but when the IP has since changed. I made the changes in Jenkins to our new FQDN but for some reason the old FQDN is getting picked up somewhere. He is the console output:
[INFO] Scanning for projects...
[INFO] [buildinfo] Properties file found at '/tmp/buildInfo626490319749387713.properties'
[INFO] Initializing Artifactory Build-Info Recording
[HUDSON] Collecting dependencies info
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building MS2.0 Security 0.0.14-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] Downloading: http://artifactory_old:8081/artifactory/plugins-release/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
[INFO] Downloading: http://artifactory_old:8081/artifactory/plugins-snapshot/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
The problem is the address above is not what I have configured globally and I have no idea where the old address is coming from.
Any insight would be greatly appriciated.