I have installed Artifactory 3.2.0 (free version) on my computer and it is relatively easy to use but I am confused when it comes to viewing the dependencies between artifacts.
As a trial run, I uploaded an artifact with default pom, but added in
<dependencies>
<dependency>
<groupId>text</groupId>
<artifactId>trial2</artifactId>
<version>1.0</version>
<type>txt</type>
</dependency>
</dependencies>
which is the dependency declaration of another file. However I do not see anything indicating the two dependencies between the two artifacts. Is this what artifactory is like? Or is there something wrong with my installation of artifactory?
Typically, binary repository does not include dependency manager of its own, so it does not know about the dependencies between artifacts. This is due to the fact that there is a whole plead of dependency managers, each of them resolves dependencies a bit differently (specially, when it comes to transitive dependencies). Even one tool can switch the resolution strategy between versions.
That means that doing static dependency analysis correctly is almost impossible, so when you just upload the files, Artifactory doesn't know about the dependencies.
This changes, when Artifactory has a chance to do a runtime dependency analysis. If you run a build (effectively resolving the artifacts using the dependency manager of choice) and Artifactory is aware of it (by using one of the Build Integration plugins, Maven plugin or Gradle plugin), then boom! you can see all the dependencies in the build browser:
Related
I'm trying to build a p2 repository from Tycho feature artifacts which are deployed in a remote Maven repository, without having to install the artifacts into the local Maven repository first (as in Tycho fails to resolve reference from product to eclipse-feature from a different reactor build), and without having to build all features and the repository together in a single reactor build.
Background
I have a multi-module Tycho project that builds several Eclipse plugins and features.
So that I can build each module separately - and so that I can reference OSGI artifacts in our Nexus Maven repository - I have enabled <pomDependencies>consider</pomDependencies> in my target platform, and added Maven dependencies between the modules or to the repository artifacts as usual with <dependency/> elements.
This works well - I can build the features or run the plugin tests without their dependant plugins being either in my local Maven repository or in the same reactor build. For example, when I run mvn test on a plugin test project, the relevant dependencies will be downloaded from Nexus and Tycho will happily resolve the Import-Packages in my manifest against these, build everything and run the tests. So far so good.
I would like to generate a p2 repository from these features so that I can install them in Eclipse from an update site, and the advertised way to do this is with the eclipse-repository packaging type. But here the plan falls down - Tycho doesn't seem to be able to resolve feature dependencies when building repositories in the same way as it can resolve plugin dependencies when building features. All attempts yield:
[ERROR] Cannot resolve project dependencies:
[ERROR] Software being installed: my.eclipse.repository raw:0.0.1.'SNAPSHOT'/format(n[.n=0;[.n=0;[-S]]]):0.0.1-SNAPSHOT
[ERROR] Missing requirement: my.eclipse.repository raw:0.0.1.'SNAPSHOT'/format(n[.n=0;[.n=0;[-S]]]):0.0.1-SNAPSHOT requires 'my.prj.eclipse.project.feature.feature.group 0.0.0' but it could not be found
There are two ways I have successfully built the p2 repository:
As part of the same reactor build. If I make the eclipse-repository a module within the Tycho multi-module project, and build the whole project at once with e.g. mvn verify, the features are resolved fine. But I don't want to do this. I would prefer to build modules individually. This means our CI can have an indicator for each module, and we can immediately see what module tests have failed in; it gives us opportunities for parallelising builds; and we avoid having to be constantly running builds on modules that haven't changed. It would be a shame to have to use a monolithic Maven build.
If I install the Tycho project into my local Maven repository, by running mvn install on the dependency. But I don't want to do this either, because this would mean the build is inherently irreproducable, as it would be sensitive to the state of the local repository. Our CI is currently set up to maintain a Maven repository per job and to completely wipe it at the start of execution, to shield us from this potential messiness.
So my question is: is there a third way? Is there any way I can get the Tycho plugin responsible for building eclipse-repository packaging types to download features from a remote Maven repository? Or any other way I can build the p2 repository from plugins that have been individually built and deployed to the Maven repository?
Things I've tried include:
specifiying the Maven feature depedencies as both jar and eclipse-feature
explicitly adding the features to the target platform, like
...
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<dependency-resolution>
<extraRequirements>
<requirement>
<type>eclipse-feature</type>
<id>my.prj.eclipse.project.feature</id>
<versionRange>0.0.0</versionRange>
</requirement>
...
The closest thing I've found to a decent solution is have a multi-module Tycho project that just contains the repository and features.
feature-project
|- feature1 (eclipse-feature)
|- feature2 (eclipse-feature)
|- repository (eclipse-repository)
Building this works - all plugins added to the top-level POM are downloaded from Nexus, available for inclusion in each feature and included in the generated repository.
However this is far from ideal because I can no longer store my features logically alongside my plugins; they need to be in separate project hierarchies. Attempting to build the features and repository separately, like with mvn clean verify -pl :feature1,feature2,repository, fails presumably due to Bug 380152.
Is there a better way? Any help would be gratefully received.
Many thanks
(As an aside: building the repository with mvn clean verify -Dtycho.localArtifacts=ignore will succeed if the features are present in the local Maven repository, and won't show you the warning that artifacts are being resolved from the local repo... is this a bug?)
I am pretty impressed by your thorough analysis. You've almost got everything covered which is possible with the current Tycho version (0.22.0) - except for the solution which is so unintuitive that I wouldn't have expected anyone to be able to guess it (see below). Note however that there is a small fix required to also make the solution work for SNAPSHOT artifacts.
But first, I'd like to provide some technical (and historical) background for what you have observed:
pomDependencies=consider only works for plug-ins: The use case for this functionality was to allow referencing plug-ins (or more precisely OSGi bundles) from Maven repositories. So when the flag is set and the project has dependencies to JARs, Tycho will check if they are OSGi bundles, generate the p2 metadata for them on-the-fly, and add them to the target platform. There is no similar support for feature JARs because these usually don't exist in Maven repositories.
But what about Tycho-built projects? These may deploy into Maven repositories! Yes, this is true, and this is why I tried to extend the pomDependencies concept to allow for what you are trying to do. The idea was that every time Tycho considers a POM dependency for the target platform, it also checks if the p2 index files ...-p2metadata.xml and ...-p2artifacts.xml exist. However this turned out to infer a massive performance penalty because it generally takes very long for a Maven repository server to figure out that an artifact does not exist. So the remote download was disabled, and replaced with a look-up in the local Maven repository. In this way, two Tycho builds could set -Dtycho.localArtifacts=ignore and would still be able to exchange the artifacts specified in the POM via the local Maven repository.
Knowing these implementation details, we get to the following solution: Instead of only adding a POM dependency from the repository to the feature artifact, you also need to add dependencies to the p2metadata and p2artifacts files. Example:
<dependencies>
<dependency>
<groupId>myproject</groupId>
<artifactId>myproject.feature</artifactId>
<version>0.1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>myproject</groupId>
<artifactId>myproject.feature</artifactId>
<version>0.1.0-SNAPSHOT</version>
<classifier>p2metadata</classifier>
<type>xml</type>
</dependency>
<dependency>
<groupId>myproject</groupId>
<artifactId>myproject.feature</artifactId>
<version>0.1.0-SNAPSHOT</version>
<classifier>p2artifacts</classifier>
<type>xml</type>
</dependency>
</dependencies>
This makes Maven also download these p2 index files, so Tycho recognizes the main artifact as Tycho artifact. In this way, you can also get an eclipse-feature into the target platform via POM dependencies - at least almost: With 0.22.0, the repository build passes, but the feature.jar artifact is missing. I already debugged this issue, and it is easy to fix.
Obviously the syntax with three <dependency> elements for every actual dependency is not nice. It should be possible to boil this down to a single p2artifacts element - but this is more work. In case you are interested in this feature, you could open an enhancement request in Tycho's issue tracker.
I am using MAVEN3 and I have a project XXX with version 1.0.0-SNAPSHOT and it is being uploaded to artifactory.I can find the jar uploaded to artifactory as XXX-1.0.0-SNAPSHOT.jar.
Another project YYY uses XXX-1.0.0-SNAPSHOT.jar as its dependency and it is declared as
<dependency>
<groupId>...<...>
<arti...>XXX</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
When I try to build YYY maven is trying to download XXX jar file from artifactory and it fails to find.
I get the following error
Could not resolve dependencies for project com......YYY:jar:1.0.0-SNAPSHOT: Could not find artifact com....XXX:jar:1.0.0-20130710.163046-1 in repo1 (https://artifactory.xxx.com/artifactory/REPO1)
I can see that it tries to download a jar with specific timestamp.How can I tell maven not to use timestamp to download my dependency.
Any help on this is appreciated.
Support for non-unique snapshots was removed in Maven 3 (see related JIRA issue).
I found an Artifactory document mentioning one way to handle this, near the bottom of the page.
My team has noticed problems if developers publishing snapshots to the same group/artifact in the snapshot repository are not all using the same Maven version. Either everyone needs to stick with Maven 2, or everyone uses Maven 3. Mixing doesn't work well.
On mvnrepositry, when you search for a certain module, there's a link to download the binary. For some versions it has a pom.xml file available for download instead of the jar. What are you supposed to do with that pom.xml? It seems like if I specify a version that does not have a downloadable jar, but instead downloadable pom.xml, my maven build will fail. Is what I'm seeing correct?
Modules that only have pom files are maven modules with pom packaging. They are used to aggregate multiple modules into one unit. You can use such a module as a dependency for your maven project. Maven will download the pom file, analyze the dependencies included in that pom file and download those & add it to your automatically.
Even modules that have jars (jar packaging) have a pom file associated with them. This pom file defines the other dependencies that are required for using it. Maven will automatically process and fetch those dependencies (transitive dependencies).
This makes specifying and managing dependency for any project. You will specify the top level modules that your projects directly depends on and other things required will automatically figured out and downloaded. It also makes it easier when you have upgrade to a new version - all the transitive dependencies will get upgraded automatically.
One of the reason that cause this is because of licensing issue.
License for such JARs prohibit public redistribution in such approach. So someone provide only the POM so that you can get the JAR yourself and install it to your local maven repo/ internal repo, together with the POM provided.
Consider the following scenario: My application has some dependencies on my own POM-first artifacts (built with pure Maven) and some dependencies on my own manifest-first artifacts (built with Tycho). For the POM-first artifacts, Tycho resolves the exact the version I specified in the POM. For the manifest-first artifacts, Tycho resolves the locally built units which may have a higher version.
In my concrete case, I specified a dependency in the pom.xml to the manifest-first artifact in version 1.2.0, but I get warning "The following locally built units have been used to resolve project dependencies" with version 1.3.0.2012xxx.
I have already found following bugs and discussions, but I don't understand why there is a difference in Tycho resolving POM-first and manifest-first dependencies.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=355367
http://dev.eclipse.org/mhonarc/lists/tycho-user/msg01673.html
Dependency-resolution is a two-step process in Tycho:
First, Tycho computes the so-called target platform, which is
the set of artifacts that are considered for dependency resolution.
In this step, Tycho evaluates the POM dependencies according to the
Maven rules and adds the result to the target platform. Also, all
Tycho artifacts you have built locally with mvn install are added
to the target platform.
Then, Tycho resolves your project's dependencies (from the
MANIFEST.MF, feature.xml, etc.) according to the OSGi rules. Unlike
in Maven dependencies, OSGi dependencies are typically specified as
version ranges. So if you write
Require-Bundle: my.bundle;bundle-version="1.2.0"
you are saying that you want version 1.2.0 or later. You typically don't want to specify an exact version here, because this would have implications on the runtime. But you do want to control what happens at build time, which is why there are various ways to control the content of the target platform.
In your particular case, you have your POM-first artifacts in the target platform in the version specified in the POM. Then, you also seem to be specifying a POM dependency to your Tycho artifacts (which is uncommon, but okay), so you will have the Tycho artifacts in the specified version in the target platform. But since you have also built a newer version of the Tycho artifacts locally with mvn install, these will also be in the target platform. The dependency resolution (step 2) hence has a choice between two versions, an will typically pick the later version.
To prevent the locally built artifacts from being added to the target platform, you can delete the file ~/.m2/repository/.meta/p2-local-metadata.properties. (I'm assuming you already know this, but just to be sure. Bug 355367 will also bring a alternative, more convenient option in 0.16.0.)
And now I finally get to your question why the behaviour is different for POM-first artifacts compared to Tycho artifacts:
Assume multiple Tycho artifacts are built together in the same reactor. Then each artifact can use the other artifacts as dependencies without needing any specific target platform configuration, e.g. you don't need POM dependencies to the artifacts in the same reactor. (Or in other words: upstream artifacts from the same reactor are automatically part of a module's target platform.) So in order to support re-builds of parts of a Tycho reactor (after a mvn install of the full reactor), locally installed Tycho artifacts need to be added to every module's target platform. Tycho can't know if they were originally part of the same reactor, so it just adds them all.
For POM-first artifacts, the reference to the artifact is always there (through the Maven configuration inheritance), even if only a part of a Tycho reactor is built. Therefore there doesn't need to be any mechanism for picking up any version of the locally built POM-first artifacts, but Tycho can add the exactly specified version to the target platform.
For those interested to force tycho to ignore local artifacts when resolving the target platform, add the CLI tycho.localArtifacts=ignore as in e.g.
mvn clean install -Dtycho.localArtifacts=ignore
More details can be found on the Tycho-Target Eclipse Wiki
I have a few dependencies like this. For example : jdic
In my pom.xml, I defined a dependency
<dependency>
<artifactId>jdic</artifactId>
<groupId>jdic</groupId>
<version>0.8.6</version>
</dependency>
And I have a remote repository (internal server): eg http://repo/thirdparty/
And jdic can be found in http://repo/thirdparty/jdic/jar/jdic.jar
As you notice, the naming is not conventional groupId:artifactId:version, instead its just jdic.jar
so when I run maven compile
mvn clean compile
maven tells me that it cannot resolve dependencies.
I'm aware that we can just download those jars to the local repository .m2/repository and run
mvn compile (ie without clean)
and it the jar will not be a problem. But is there any other way that I can make it retrieve from the remote despite its unconventional name and lack of metadata/pom info for those dependencies?
I already have a mirror to this internal repository that overrides the central
You could install the jdic in your internal/mirror repository with the version (jdic-0.8.6.jar).
If you are using a repository manager and you uploaded the jar to it, it would automagically create a pom for it as well as ensure the dependency was created with version, as per maven convention.
A remote Maven repository is NOT any web server putting files in any way you want. There are lots of convention to follow. One of them is the way to represent the version (which is one essential element of an artifact). With your "remote repository" it is clear that there is no way Maven can find the artifact.
Setup a real Maven remote repository to host such files. You can have a look in Nexus and Artifactory.