Felix maven-bundle-plugin transitive dependency issue - osgi

I'm new to OSGI and trying to deploy my first application. I've a spring dependency in my pom. While deploying I realized that Felix runtime requires all transitive dependencies to install the bundle properly. Since then, I'm sort of struggling to resolve this issue. I've tried embedded-dependency and embedded-transitive options, but of no luck. Here's my pom.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.test</groupId>
<artifactId>taxonomydaobundle</artifactId>
<version>1.0.0</version>
<packaging>bundle</packaging>
<name>Taxonomy Dao Bundle</name>
<url>http://maven.apache.org</url>
<repositories>
<repository>
<id>fusesource</id>
<url>http://repo.fusesource.com/maven2</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</repository>
<repository>
<id>apache-public</id>
<url>https://repository.apache.org/content/groups/public/</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</repository>
</repositories>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>com.test</groupId>
<artifactId>taxonomymodelbundle</artifactId>
<version>1.0.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>2.5.5</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.0.1</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Export-Package>com.test.taxonomy.api.*;version=1.0.0
</Export-Package>
<Import-Package>com.test.taxonomy.message.*;version=1.0.0,
*
</Import-Package>
<Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>
<Embed-Transitive>true</Embed-Transitive>
</instructions>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.1</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
Initially, I was trying mvn install but it was only including the direct dependencies and not the transitive ones. After reading the felix maven plugin documentation, I tried mvn org.apache.felix:maven-bundle-plugin:bundleall. But, the execution is failing as its not able to get the required jar files from the repository its using. Looking at the logs, I can see that it's referring to http://repo1.maven.org/maven2 repository which doesn't have the required versions. For e.g. this one is from hessian 3.1.3,among other ones.
[INFO] Unable to find resource 'hessian:hessian:pom:3.1.3' in repository central
(http://repo1.maven.org/maven2)
I'll appreciate if someone can share their experience in this regard.
-Thanks

In the OSGi environment the common way to deploy required libraries is to deploy them as own bundles. Embedding a library is an option that is most used (as I know) if the library is not OSGi enabled and does not matter for other bundles in the OSGi container.
So, if your bundle depends on other libraries you should first have a look, if these libraries are OSGi enbabled and install them as own bundles in the OSGi container.
If you use a library that is not OSGi enabled you can have a look to some places that offer OSGi enabled "wrapped" bundles of these libraries, e.g.
http://www.springsource.com/repository
http://repo.fusesource.com/
Spring is OSGi enabled - it should be possible for you to deploy these bundles on their own. I also recommend to have a look at Spring Dynamic Modules (documentation). If you build Spring enabled bundles you can also start with an OSGi Container that's already prepared with these bundles. I made good expirience with FUSE ESB (which is free (based on apache servicemix) OSGi container (and more) with commercial support).
So although I did not answer your concrete questions I shared some of my experience and hope that will help you. Good luck!

Run Maven with the -X option, which will get the bundle plugin to print what it thinks the dependency tree is
[DEBUG] Will bundle the following dependency tree
org.springframework:spring-remoting:jar:2.0.8:runtime
aopalliance:aopalliance:jar:1.0:compile
commons-httpclient:commons-httpclient:jar:3.0.1:compile
junit:junit:jar:3.8.1:compile
(commons-logging:commons-logging:jar:1.0.3:compile - omitted for conflict with 1.1)
commons-codec:commons-codec:jar:1.2:compile
commons-logging:commons-logging:jar:1.1:compile
log4j:log4j:jar:1.2.12:compile
logkit:logkit:jar:1.0.1:compile
avalon-framework:avalon-framework:jar:4.1.3:compile
(javax.servlet:servlet-api:jar:2.3:compile - omitted for conflict with 2.4)
hessian:hessian:jar:3.0.20:compile
Unfortunately you can not tweak your project's pom depdnencyManagement section to remove "optional" dependencies because maven-bundle-plugin is ignoring this.
According to FELIX-954 its pretty much going to visit all optional dependencies.
The only advice I can offer is to find those optional artifacts and make sure Maven can resolve them.

There is a bug in the transitive dependencies management for the maven-bundle-plugin that causes it to look up optional dependencies (some of which may not exist). I suspect this is what is happening when it tries to pull in hessian.
I'll second K. Claszen's idea of looking for versions of your bundles on springsource or fusesource, but if there are bundles you cannot find, the maven-bundle-plugin provides a bundleall goal (it's described in the goals section). This goal will cause the bundle plugin to create bundles for all of the transitive dependencies of your application. You can then use the ones you couldn't find in springsource/fusesource. Once you've done this, you won't need to embed the jars into your bundle, which will make them available to other bundles as well.
One caveat that you may run into is the need to deploy the bundles such that no bundle is deployed before one of its dependencies. In the past, I have used the Eclipse Virgo server to get around this. It allows you to place all of your bundles in a repository and have the server determine the deployment order.

Related

How can I use Maven locations in Eclipse Target files that come from custom repositories

I am trying to use a maven location in a Tycho build, that comes from a different repository than maven central. Below is the part of the definition that provides this on PDE. Tycho does not seem to honour the repository declaration and fails on resolution. Assuming I am observing this correctly, I have the following
Question: How can I hint Tycho to understand that these maven coordinates should be obtained from the associated repositories?
If this is not a current feature, but there is an issue, please comment as well, so this can be found and tracked.
<location includeDependencyScope="compile" includeSource="true" missingManifest="generate" type="Maven">
<dependencies>
<dependency>
<groupId>com.github.amlorg</groupId>
<artifactId>amf-api-contract_2.12</artifactId>
<version>5.0.2</version>
<type>jar</type>
</dependency>
<dependency>
<groupId>org.raml</groupId>
<artifactId>webapi-parser</artifactId>
<version>0.5.0</version>
<type>jar</type>
</dependency>
</dependencies>
<repositories>
<repository>
<id>mulesoft.releases</id>
<url>https://repository-master.mulesoft.org/nexus/content/repositories/releases</url>
</repository>
<repository>
<id>mulesoft.public</id>
<url>https://repository.mulesoft.org/nexus/content/repositories/public/</url>
</repository>
</repositories>
</location>

Inherit child-dependencies from maven dependency

I have two separate maven projects.
Project A contains utility classes and similar stuff. It also uses jetbrains annotations in some interfaces to mark parameters as Nullable and NotNullable.
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<version>20.1.0</version>
</dependency>
Project B uses some of the Utilities of Project A. It includes it as a dependency from my repository.
<dependency>
<groupId>org.test.group</groupId>
<artifactId>Utilities</artifactId>
<version>1.0</version>
</dependency>
I can access the classes from my utilities dependency just fine. But i do not see any of the annotations on the parameters. I also can't access the jetbrains annotations in project B in any of my classes there. I'd have to add the jetbrains dependency in project B as well to do so.
Is there any way to inherit the dependencies of another dependency?
I looked at other questions and found this similar one. Tho his solution was to set the optional-parameter to false which i am not even using. Perhaps also something that needs to be configured in the maven build? Currently im running my build with goals clean package deploy without any special additional configuration.
I know gradle builds allow for implementation and api dependencies, where one of them forwards the dependency to other projects that include it and the other doesn't.
Edit: Here the full configuration of my projects
Local nexus running under localhost:8081 containing my artifacts. Local TeamCity running under localhost:8080 used for the builds and deployment to the repository.
Project A pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.test.group</groupId>
<artifactId>Utilities</artifactId>
<version>1.0</version>
<name>Utilities</name>
<dependencies>
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<version>20.1.0</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<distributionManagement>
<repository>
<id>local_nexus</id>
<name>Deployment</name>
<url>http://localhost:8081/repository/org.test.group/</url>
</repository>
</distributionManagement>
</project>
Project B pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.test.group</groupId>
<artifactId>TestProject</artifactId>
<version>1.0</version>
<name>TestProject</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.test.group</groupId>
<artifactId>Utilities</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>local_nexus</id>
<name>Deployment</name>
<url>http://localhost:8081/repository/org.test.group/</url>
</repository>
</repositories>
<distributionManagement>
<repository>
<id>local_nexus</id>
<name>Deployment</name>
<url>http://localhost:8081/repository/org.test.group/</url>
</repository>
</distributionManagement>
</project>
Edit 2:
I've made some progress. Since i didn't change Project A's version after adding the annotations my local repository didn't fetch the new version. After a purge using mvn dependency:purge-local-repository the updated state of version 1.0 was available.
mvn dependency:tree prints now
[INFO] org.test.TestProject:jar:1.0
[INFO] \- org.test:Utilities:jar:1.0:compile
[INFO] \- org.jetbrains:annotations:jar:20.1.0:compile
Tho my IDE (Intellij) still doesn't recognize the annotation classes inside TestProject. Any idea why this is failing now?
You inherit annotations automatically, no configuration needed.
You don't inherit it, if it is of scope provided. Look at mvn dependency:tree to find out about the place and scope of the annotations library.
BTW: mvn clean package deploy is a waste of time, just use mvn clean deploy.
The issue was caused by my local maven repository. Since i didn't change the version of Project A's artifact (always 1.0) my local repository kept providing me with the old state of version 1.0.
Using mvn dependency:purge-local-repository i was able to cleanse the old 1.0 and load the new 1.0. Tho still my IDE refused to imports annotations dependency. At this point it was listed by mvn dependency:tree tho.
After tempering with the IDE for a while i decided to update my version to 1.1. Once Project A's 1.1 was built and Project B's dependency was updated to 1.1 it worked.
So basically it resulted from poor versioning of my projects, which interfered with my local maven repository.

mvn- look for wrong url in maven repositories

When I run mvn package to compile a maven project it downloads the jar file from
wrong URLs. It adds org/dnosproject/ to the URL which is wrong.
Downloading: https://mvnrepository.com/artifact/io.github.dnos-project/dnos-lib-all/org/dnosproject/onos-port-protobuf/1.1.5/onos-port-protobuf-1.1.5.jar
<repositories>
<repository>
<id>dnos-lib-all</id>
<name>dnos-lib</name>
<url>https://mvnrepository.com/artifact/io.github.dnos-project/dnos-lib-all</url>
<layout>default</layout>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>io.github.dnos-project</groupId>
<artifactId>dnos-lib-all</artifactId>
<version>1.1.5</version>
</dependency>
The website mvnrepository.com is a kind of search engine across multiple real Maven repositories. If you look at the link you mentioned in your <repository/> configuration, you'll notice they list that dependency as available in "Central", in fact here. "Central" is configured by default, so you don't need a <repository/> configuration for this dependency. Instead, you just need the correct <dependency/> entry:
<dependency>
<groupId>io.github.dnos-project</groupId>
<artifactId>dnos-lib-all</artifactId>
<version>1.1.5</version>
</dependency>
You already had this in the snippet you posted, so just removing the <repository/> configuration should do the trick.

Could not find metadata in local

Locally on the machine i have an Artifactory installed with maven repositories, and i have a very simple .pom file for my project which points to it:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>MYGROUP</groupId>
<artifactId>NAME</artifactId>
<packaging>jar</packaging>
<version>VERSION</version>
<build>
<sourceDirectory>SRCFOLDER</sourceDirectory>
<testSourceDirectory>TESTFOLDER</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>local-artifactory</id>
<name>Artifactory</name>
<url>http://localhost:8081/artifactory/repo</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
<checksumPolicy>warn</checksumPolicy>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>local-artifactory-plugins</id>
<name>Artifactory Plugins</name>
<url>http://localhost:8081/artifactory/repo</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>MYGROUP</groupId>
<artifactId>DEPENDENCY1</artifactId>
<version>bla-SNAPSHOT</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>MYGROUP</groupId>
<artifactId>DEPENDENCY2</artifactId>
<version>bla-SNAPSHOT</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
So i have local-artifactory repository declared with the intention that it will reload everything every time. Also i have couple of dependencies which are SNAPSHOTS.
Now i'm using maven 3.0.5, and when i'm compiling it it works fine, but it's not refreshing snapshots at all. Never. Ever. Log file looks like this:
[DEBUG] Could not find metadata MYGROUP:DEPENDENCY1:bla-SNAPSHOT/maven-metadata.xml in local (/opt/myuser/.m2/repository)
[DEBUG] Using connector WagonRepositoryConnector with priority 0 for http://localhost:8081/artifactory/repo
Downloading: http://localhost:8081/artifactory/repo/MYGROUP/DEPENDENCY1/bla-SNAPSHOT/maven-metadata.xml
Downloaded: http://localhost:8081/artifactory/repo/MYGROUP/DEPENDENCY1/bla-SNAPSHOT/maven-metadata.xml (314 B at 13.9 KB/sec)
[DEBUG] Writing resolution tracking file /opt/myuser/.m2/repository/MYGROUP/DEPENDENCY1/bla-SNAPSHOT/resolver-status.properties
[DEBUG] Could not find metadata MYGROUP:DEPENDENCY1:bla-SNAPSHOT/maven-metadata.xml in local (/opt/myuser/.m2/repository)
[DEBUG] Skipped remote update check for MYGROUP:DEPENDENCY1:bla-SNAPSHOT/maven-metadata.xml, already updated during this session.
So it constantly complains that it can't find local maven-medatada.xml and of course he's completely right - it's not there. And fails to do ANYTHING with it.
So i've tried to use dependency:purge-local-repository to purge local inventory and guess what? Apparently it can't purge it because there is no maven-metadata.xml in each artifact, what an amazing functionality!
Each .m2 artifact directory apart of .jar and .pom contains maven-metadata-local-artifactory.xml(with the correct copy of maven-metadata.xml from the artifactory, current), _mave.repositories and resolver-status.properties.
Maven is just out of the box, there is no configuration in .m2 and the configuration in the /conf folder is default without any single change.
UPDATE: I've just manually downloaded maven-metadata.xml from the artifactory and placed it as maven-metadata.xml inside .m2 for a dependency - no effect, it is still "missing" it. So i don't have any idea what it wants from me anymore. It just can't update dependencies because "already updated during this session". Well i don't know what it updated exactly, but apparently something else.
UPDATE2: Contents of maven-metadata.xml on the server:
<metadata>
<groupId>MYGROUP</groupId>
<artifactId>DEPENDENCY1</artifactId>
<version>bla-SNAPSHOT</version>
<versioning>
<snapshot>
<buildNumber>1</buildNumber>
</snapshot>
<lastUpdated>20130322155759</lastUpdated>
</versioning>
</metadata>
My maven complained that it can not find metadata in local, blah blah... and basically, what happened is that i managed to unplug my power cable in mid-maven-build and its metadata in the relevant local .m2 subdirectory got corrupted. I went in, deleted the artifacts in question from from their .m2 location and re-tried the build. All worked fine. Just wanted to leave this message for those that have a similar problem to mine, but stumbled upon your question.
From the maven metadata xml file it looks like you are using non-unique snapshot in your repository. Maven 3 does not supports non-unique, only snapshot pom file with TIMESTAMP-BUILDNUMBER instead of SNAPSHOT. Do you have:
http://localhost:8081/artifactory/repo/MYGROUP/DEPENDENCY1/bla-SNAPSHOT/DEPENDENCY1-bla-XXXXX-Y.pom
file?

Maven internal library pom version management

Are there any tools and/or best practices to help with managing a large set of internal libraries versions across many dependent projects. Allow me to elabore,
Say you have products A, B, C.
These products depend on libraries X, Y, Z, and there are dependencies within those as well.
In this world, we like to keep all versions on the latest released version of all dependencies (no snapshot dependencies except in working copies). This allows us to release the product at any time and force all projects to be run in CI (Hudson).
The issue is in keeping the pom.xml's all up-to-date. Currently, we use a custom maven plugin that, as part of each release, searches our SCM for pom.xml's that depend on the project being released and updates it. This is similar to the versions-maven-plugin except that you don't need a working copy of each project to do it.
There has got to be a better way. What do other teams do about many shared libraries across many projects? What is the best way to organize this? Multi-module works in some cases, but most of our libraries are fairly independent and used by too many other projects to (a) decide which multi-module it would belog to and (b) the hierarchy that would work for this.
managing dependencies in Maven works fine with dependencyManagement - I think you are familiar with it. You can outsource this dependencyManagement to a dedicated POM: a so called BOM POM (see http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html).
In your case this BOM POM might be shared by your products A, B and C. Due to that fact you should define a Maven project just containing this BOM POM with all the dependency management for thirdparty software X, Y and Z. This project can be released and you are able to add a dependency on your BOM POM with the scope import.
Your BOM POM:
<project>
<groupId>com.acme</groupId>
<artifactId>my-thirdparty-bom-pom</artifactId>
<version>1.0-SNAPSHOT</version>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>...</groupId>
<artifactId>X</artifactId>
<version>...</version>
</dependency>
<dependency>
<groupId>...</groupId>
<artifactId>Y</artifactId>
<version>...</version>
</dependency>
<dependency>
<groupId>...</groupId>
<artifactId>Z</artifactId>
<version>...</version>
</dependency>
</dependencies>
<dependencyManagement>
...
</project>
Sample import for project A
<project>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.acme</groupId>
<artifactId>my-thirdparty-bom-pom</artifactId>
<version>1.0-SNAPSHOT</version>
<scope>import</scope>
<dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>...</groupId>
<artifactId>Y</artifactId>
<!-- the version is managed by the thirdparty bom imported above -->
</dependency>
</dependencies>
...
</project>
The closest thing is to use Dependency Version Ranges and then in a profile override your pom's repositories changing snapshots to false, making sure to include your profile during release with "-P".
<dependency>
<groupId>a</groupId>
<artifactId>a</artifactId>
<version>[3.8,4.0)</version>
</dependency>
...
<profiles>
<profile>
<id>your-release</id>
<repositories>
<repository>
...
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
</profile>
<profiles>
Another pattern is the "bom" whereby you create a pom (packaging: pom) that declares a list of dependencies to make management easier (less to change).

Resources