Using Camel-hdfs in a karaf bundle - maven

I am trying to implement an camel route to send files in a HDFS server with an OSGI bundle, using java language with no blue print, but i can't make it work due tu hdfs scheme not being found while creating the route.
The code of the class has been tested as a Jar and works. The issue is in karaf, which seems to not be able to use camel-hdfs for the bundle, even if the camel-hdfs bundle is shown using the list command.
Here's the pom file of the project :
<modelVersion>4.0.0</modelVersion>
<groupId>the.group</groupId>
<artifactId>receiveFile</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>receiveFile</name>
<packaging>bundle</packaging>
<dependencies>
<dependency>
<groupId>org.osgi</groupId>
<artifactId>osgi_R4_core</artifactId>
<version>1.0</version>
<scope>provided</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-hdfs</artifactId>
<version>2.10.4</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>2.10.4</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.3.7</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Import-Package>*</Import-Package>
<Export-Package>activation</Export-Package>
<Private-Package>activation</Private-Package>
<Bundle-Activator>activation.Activator</Bundle-Activator>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
I tried also Embed-Dependency with the transitive option, but it still doesn't work and i'm kind of stuck right now.
Karaf prints the following error :
org.osgi.framework.BundleException: Exception in activation.Activator.start() of bundle group.receiveFile.
The log :
org.osgi.framework.BundleException: Exception in activation.Activator.start() of bundle group.receiveFile.
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:806)[osgi-3.6.2.R36x_v20110210.jar:]
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:755)[osgi-3.6.2.R36x_v20110210.jar:]
at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:370)[osgi-3.6.2.R36x_v20110210.jar:]
at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:284)[osgi-3.6.2.R36x_v20110210.jar:]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundle(DirectoryWatcher.java:1244)[6:org.apache.felix.fileinstall:3.2.4]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundles(DirectoryWatcher.java:1216)[6:org.apache.felix.fileinstall:3.2.4]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.startAllBundles(DirectoryWatcher.java:1205)[6:org.apache.felix.fileinstall:3.2.4]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.process(DirectoryWatcher.java:500)[6:org.apache.felix.fileinstall:3.2.4]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:291)[6:org.apache.felix.fileinstall:3.2.4]
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route route1027 at: >>> To[hdfs://hadoopServer/received] <<< in route: Route[[From[file://toSend/]] -> [To[hdfs://hadoopServer/rece... because of Failed to resolve endpoint: hdfs://hadoopServer/received due to: No component found with scheme: hdfs
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:879)
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:172)
at org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:722)
at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:1789)
at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:1575)
at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:1444)
at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:60)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:1412)
at activation.Activator.start(Activator.java:24)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:783)[osgi-3.6.2.R36x_v20110210.jar:]
at java.security.AccessController.doPrivileged(Native Method)[:1.7.0_11]
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:774)[osgi-3.6.2.R36x_v20110210.jar:]
... 8 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: hdfs://hadoopServer/received due to: No component found with scheme: hdfs
at org.apache.camel.impl.DefaultCamelContext.getEndpoint(DefaultCamelContext.java:485)
at org.apache.camel.util.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:50)
at org.apache.camel.model.RouteDefinition.resolveEndpoint(RouteDefinition.java:187)
at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:108)
at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:114)
at org.apache.camel.model.SendDefinition.resolveEndpoint(SendDefinition.java:61)
at org.apache.camel.model.SendDefinition.createProcessor(SendDefinition.java:55)
at org.apache.camel.model.ProcessorDefinition.makeProcessor(ProcessorDefinition.java:461)
at org.apache.camel.model.ProcessorDefinition.addRoutes(ProcessorDefinition.java:179)
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:876)
... 19 more
Karaf version : 2.2.8
Maven : m2e plugin, 3.0.4
Thanks in advance.

I would suggest to use a blueprint XML file to bootstrap your application. Just to setup a < camelContext >.
Otherwise you would have to do a lot of manual osgi setup and whatnot in your activator to properly setup Camel for OSGi. All that is taken care of when using camel-blueprint.
If you still want to do it from an Activator, then check out camel-core-osgi and see what we do there.

Related

Using a 3rd party dependency within a Maven OSGi bundle

After feeling like I had a grasp on how OSGi is utilized, I tried to go about adding a 3rd party dependency, specifically log4j2, to my application that is utilizing apache felix and bundling with the maven-bundle-plugin. Unfortunately, it seems as if I am stuck in dependency hell.
I have tried using numerous maven-bundle tactics like Import-Package, Embed-Dependency, wrapImportPackage, Embed-Transitive, and setting specific version numbers just to name a few. Below is what my pom looks like for this plugin:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>ParentId</artifactId>
<groupId>ParentGroupId</groupId>
<version>x.x.x</version>
</parent>
<groupId>ParentGroupId.ParentId</groupId>
<artifactId>thisModule</artifactId>
<packaging>bundle</packaging>
<name>thisModule</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.osgi</groupId>
<artifactId>org.osgi.core</artifactId>
<version>6.0.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>AM</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.12.1</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.12.1</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>3.5.1</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-SymbolicName>${project.name}</Bundle-SymbolicName>
<Bundle-Activator>moduleActivator</Bundle-Activator>
<Embed-Dependency>
AM,
gson,
log4j-api,
log4j-core
</Embed-Dependency>
</instructions>
</configuration>
</plugin>
</plugins>
<finalName>${project.artifactId}</finalName>
</build>
The most progress I feel like I have had is with the above pom, where I am embedding the log4j api and core directly into the bundle, but it seems as if OSGi is incapable of downloading and bundling the compile dependencies that the log4j api and core are dependent on. It successfully builds using maven but then when I deploy the EAR and JAR I get this error at runtime (when the plugin is trying to boot up):
Caused By: org.osgi.framework.BundleException: Unresolved constraint in bundle thisModule [2]: Unable to resolve 2.0: missing requirement [2.0] package; (package=com.conversantmedia.util.concurrent)
error that will name a specific dependency that log4j needs. What I DON'T want to do is include every single dependency and their mother inside the Embed-Dependency tag.
What am I doing wrong here?
Also note: Due to constraints, my only option here is to use apache felix and OSGi.
Below are other examples of modifications I made to the above POM and their resulting outputs:
Removing both log4j-api and log4j-core from Embed-Dependency and adding in a <wrapImportPackage>;</wrapImportPackage> tag. Doing so resulted in this output, which is extremely common and happens whenever I try to import:
Caused By: org.osgi.framework.BundleException: Unresolved constraint in bundle thisModule [2]: Unable to resolve 2.0: missing requirement [2.0] package; (&(package=org.apache.logging.log4j)(version>=2.12.0)(!(version>=3.0.0)))
Adding * to Embed-Dependency as well as adding <Embed-Transitive>true</Embed-Transitive>:
Caused By: org.osgi.framework.BundleException: Unresolved constraint in bundle thisModule [2]: Unable to resolve 2.0: missing requirement [2.0] package; (package=android.dalvik)
Embedding a logging libary is a bad idea. After all you want to configure the logging in a central place which is very hard when each bundle embeds a logging framework.
In most cases the safe bet is to simple keep the maven-bundle-plugin config empty and let it do its thing.
I personally always use slf4j for logging in OSGi. You simply depend on the slf4j-api. The maven-bundle-plugin creates the import package statements automatically.
Then at runtime you simply deploy a logging framework that supports the logging api you want.
For apache karaf this is already the case by default. If you use bndtools or your own application assembly based on plain felix then check out my osgi best practices example.
It shows how to use slf4j-api in your own bundles and also how to configure the log system in karaf and bndtools based applications.

NoClassDefFoundError using Xerces in OSGi environment

I've built a simple bundle which instantiate a org.apache.xerces.util.XMLCatalogResolver class.
In my pom, I've had these dependencies and plugin :
<build>
<plugins>
<!-- Plugin to create bundle -->
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.5.4</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Import-Package>*</Import-Package>
<Export-Package>myXMLResolver</Export-Package>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>2.16.1</version>
<type>bundle</type>
<scope>provided</scope>
</dependency>
<!-- xerces is provided in /lib/endorsed/ folder of ServiceMix -->
<dependency>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<version>2.11.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.servicemix.bundles</groupId>
<artifactId>org.apache.servicemix.bundles.xmlresolver</artifactId>
<version>1.2_5</version>
<type>bundle</type>
<scope>compile</scope>
</dependency>
</dependencies>
</build>
Testing the bundle "locally" (within the IDE) is working fine.
Deploying it into the /deploy folder is also fine (its status is "Active").
When running it (through Camel route), I get this stacktrace :
Caused by: java.lang.NoClassDefFoundError: org/apache/xml/resolver/CatalogManager
at org.apache.xerces.util.XMLCatalogResolver.init(Unknown Source)[:]
at org.apache.xerces.util.XMLCatalogResolver.<init>(Unknown Source)[:]
at org.apache.xerces.util.XMLCatalogResolver.<init>(Unknown Source)[:]
at myXMLResolver.MainBean.run(MainBean.java:17)[241:test.myXMLResolver:0.0.1.SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.7.0_95]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)[:1.7.0_95]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_95]
at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_95]
at org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:408)[199:org.apache.camel.camel-core:2.16.1]
at org.apache.camel.component.bean.MethodInfo$1.doProceed(MethodInfo.java:279)[199:org.apache.camel.camel-core:2.16.1]
at org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:252)[199:org.apache.camel.camel-core:2.16.1]
... 19 more
If I'm not wrong, the class CatalogManager from package org.apache.xml.resolver existed during compilation but not in run-time. So I deploy org.apache.servicemix.bundles.xmlresolver dependency into /deploy folder.
When executing in Karaf console the command bundle:headers <xmlresolver_bundle_id>, I get :
Export-Package =
...
org.apache.xml.resolver;uses:="org.apache.xml.resolver.helpers,javax.xml.parsers,org.apache.xml.resolver.readers";version=1.2,
...
Nevertheless, same stacktrace appears.
Even if I add the package org.apache.xml.resolver in the <Import-Package> I get the same error.
Any idea of what's going on ?
I'm using ServiceMix 6.1.0 (w/ Karaf 3.0.5), running on Java 7.
Edit :
Here is the bundle's MANIFEST.MF :
Manifest-Version: 1.0
Bnd-LastModified: 1461311084908
Build-Jdk: 1.8.0_72-internal
Bundle-ManifestVersion: 2
Bundle-Name: myXMLResolver
Bundle-SymbolicName: test.myXMLResolver
Bundle-Version: 0.0.1.SNAPSHOT
Created-By: Apache Maven Bundle Plugin
Export-Package: myXMLResolver;version="0.0.1";uses:="org.apache.camel,org.apache.xerces.util"
Import-Package: org.apache.camel;version="[2.16,3)",org.apache.xerces.util;version="[2.11,3)"
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.7))"
Tool: Bnd-2.4.1.201501161923
If I add the package org.apache.xml.resolver in the <Import-Package>, Import-Package from MANIFEST become : org.apache.camel;version="[2.16,3)",org.apache.xerces.util;version="[2.11,3)",org.apache.xml.resolver;version="[1.2,2)"
Edit #2 : workaround
A workaround can be done :
Put the bundle org.apache.servicemix.bundles.xmlresolver in
<SMX_HOME>/lib/endorsed folder
In the file <SMX_HOME>/etc/config.properties, add
org.apache.xml.resolver to the property
org.osgi.framework.system.package.extra.
But it doesn't look like a clean method...
Edit #3 : issue opened on Jira : https://issues.apache.org/jira/browse/SM-3020

Intellij + springboot + maven + spring-loaded

I have an issue where debugging stops working in Intellij, when applying Spring-loaded as a dependency to the maven plugin.
Situation 1 (working):
Using a autogenerated spring-boot maven (pom.xml) file with a declared "Spring-boot-maven-plugin" I can debug my my app using the debug-maven command in intellij. The pom file looks like this:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
Situation 2 (not working):
Following the guide lines here: spring-boot-hot-swapping one should add a dependency for spring-loaded, to make use of spring-loaded. The pom file now looks like:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>springloaded</artifactId>
<version>1.2.3.RELEASE</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
Debugging the same run-configuration as before does not work. The break-points simply never turns into a "checkmark" and the code never suspends. Code is now hotswapping as one would expect... If I remove the plugin-dependency again, debugging starts working again...
Please help me make situation two work!
IntelliJ 14.1.3 Ultimate, Java 7, Spring boot 1.2.4, Spring loaded 1.2.3
Since it seems there is no one with an answer to this (nor in IntelliJ forums) i've created an issue:
https://youtrack.jetbrains.com/issue/IDEA-142078
Did you try with spring-devtools (since 1.2.3) ? Use this dependency :
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
</dependency>
There is already a ticket here, it seams the same issue.

Is Apache Aries running in Felix?

I'm trying to build a Blueprint bundle to run in Apache Felix. I tried to make it running but I didn't succeed.
The blueprint bundle works fine in Karaf but not in Felix. Is it any documentation or a running example on the web to explain how to run a Blueprint bundle only with plain Felix. I suppose I have to manually add Aries to Felix platform but it didn't seem to work.
To be more precise, I want a simple service to see that it's loaded from a blueprint.xml XML config file as a Blueprint bundle. The service may have only one dummy method or even just a constructor with a println in it. That service class I want to refer it in OSGI-INF/blueprint/blueprint.xml so it will be loaded when the Blueprint bundle is loaded by Felix.
After spending some time trying to solve this problem I found the solution. So, you need the following bundles to be installed into your Felix (tested with v.4.4.1) in order to make Aries Blueprint running:
org.apache.aries.blueprint : org.apache.aries.blueprint : 1.1.0
org.apache.aries : org.apache.aries.util : 1.1.0
org.apache.aries.proxy : org.apache.aries.proxy : 1.0.1
org.apache.felix : org.apache.felix.configadmin : 1.8.0
one implementation of SLF4J (in this case will be PAX Logging):
org.ops4j.pax.logging : pax-logging-api : 1.4
org.ops4j.pax.logging : pax-logging-service : 1.4 (you may exclude log4j : log4j because is not needed)
These jars will enable Aries Blueprint in Felix (but only the XML configuration version). If you want to use annotations, you have to add also annotation related Jars.
Here is a pom to ease your work. Just run it and all the jar needed to be installed in felix will be in your target folder.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.aries</groupId>
<artifactId>blueprint-felix-assembly</artifactId>
<version>1.0-SNAPSHOT</version>
<name>Blueprint Felix Jar Assembly</name>
<packaging>pom</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<pax.logging.version>1.4</pax.logging.version>
<aries.version>1.1.0</aries.version>
<aries.proxy.version>1.0.1</aries.proxy.version>
<felix.config.admin.version>1.8.0</felix.config.admin.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.felix</groupId>
<artifactId>org.apache.felix.configadmin</artifactId>
<version>${felix.config.admin.version}</version>
</dependency>
<dependency>
<groupId>org.ops4j.pax.logging</groupId>
<artifactId>pax-logging-api</artifactId>
<version>${pax.logging.version}</version>
</dependency>
<dependency>
<groupId>org.ops4j.pax.logging</groupId>
<artifactId>pax-logging-service</artifactId>
<version>${pax.logging.version}</version>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.aries.blueprint</groupId>
<artifactId>org.apache.aries.blueprint</artifactId>
<version>${aries.version}</version>
</dependency>
<dependency>
<groupId>org.apache.aries</groupId>
<artifactId>org.apache.aries.util</artifactId>
<version>${aries.version}</version>
</dependency>
<dependency>
<groupId>org.apache.aries.proxy</groupId>
<artifactId>org.apache.aries.proxy</artifactId>
<version>${aries.proxy.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<excludeTransitive>true</excludeTransitive>
<outputDirectory>${project.build.directory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Aries should run very well on Apache Felix it does not require Apache Karaf to run. In fact we are using plain equinox for our integration tests.
You can take a look at the integration test base class to see which bundles you need.

Problems with maven apache mina pom.xml

I have following entries in my pom.xml.
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-core</artifactId>
<version>2.0.4</version>
</dependency>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-filter-compression</artifactId>
<version>2.0.7</version>
</dependency>
I am getting "Missing artifact org.apache.mina:mina-core:bundle:
2.0.7" error in my pom.xml .
Could someone please help in resolving this error.
Add to your pom file:
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
</plugin>
</plugins>
Addition to the accepted answer, an explanation why this is necessary:
The various MINA dependencies rely on OSGi bundle artifacts rather than standard JAR files.
As such, it's necessary to add support for these bundle to Maven using the Apache Felix maven-bundle-plugin.
See https://stackoverflow.com/a/5409602 for a good explanation of OSGi bundles, with links to more info.

Resources