How to import Apache Kafka in OSGi / Karaf - maven

I am trying to import and use Apache Kafka producer within an application of a much bigger project (onosproject.org) developed with Karaf. My application is referencing four services of which only one is satisfied, resulting in unsatisfied component:
onos> scr:list | grep com.foo.bar
115 | UNSATISFIED | com.foo.bar.AppComponent
After removing the code that required the three unsatisfied service I got the same behaviour which looks even stranger.
onos> scr:details com.foo.bar.AppComponent
Component Details
Name : com.foo.bar.AppComponent
State : UNSATISFIED
Properties :
service.pid=com.foo.bar.AppComponent
component.name=com.foo.bar.AppComponent
component.id=115
References
Reference : hostService
State : satisfied
Multiple : single
Optional : mandatory
Policy : static
Service Reference : Bound Service ID 1023 (org.onosproject.net.host.impl.HostManager)
Judging from logs, activation works fine which makes me believe that something in pom.xml 's dependencies is incorrect. I am building using servicemix:
<dependency>
<groupId>org.apache.servicemix.bundles</groupId>
<artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
<version>0.8.2.2_1</version>
</dependency>
with maven-bundle-plugin:
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.5.3</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Import-Package>net.jpountz*;version="[1.2.0,1.3.0)";resolution:=optional,
javax.management*,
org.slf4j*;resolution:=optional,
org.xerial.snappy;resolution:=optional,
sun.misc;resolution:=optional,
sun.nio.ch;resolution:=optional
</Import-Package>
<Export-Package>
org.apache.kafka.*
</Export-Package>
</instructions>
</configuration>
</plugin>
I don't know if I have to, but I also exposing servicemix bundle via features.xml:
<bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.kafka-clients/0.8.2.2_1</bundle>
I have also tried building with org.apache.kafka/kafka-clients/0.8.2.2 instead the servicemix but I had the same behavior.
Please find a much more detailed description of my tries/logs in this github repo.
Any suggestions will be appreciated! :)

Related

Spring boot unpack application before executing

I got this error when running cucumber test via spring-boot jar
io.cucumber.core.exception.CompositeCucumberException: There were 2 exceptions:
io.cucumber.core.exception.CucumberException(The resource jar:file:/Users/XTZ/IdeaProjects/eq-data/target/eq-data-0.0.1.jar!/BOOT-INF/lib/xyz-service-starter-2.3.10-1.jar!/com/xyz is located in a nested jar.
This typically happens when trying to run Cucumber inside a Spring Boot Executable Jar.
Cucumber currently doesn't support classpath scanning in nested jars.
Feel free to send a pull request to make this possible!
You can avoid this error by unpacking your application before executing.)
My question is how to unpack before executing(unpacking your application before executing)?
The Spring Boot executable format describes how Spring packages its executable jar files. For example:
example.jar
|
+-META-INF
| +-MANIFEST.MF
+-org
| +-springframework
| +-boot
| +-loader
| +-<spring boot loader classes>
+-BOOT-INF
+-classes
| +-com
| +-example
| +-project
| +-StepDefinitions.class
+-lib
+-com.example:utilities:8.0.1.jar
+-com.example:models:4.2.0.jar
By default Cucumber scans the entire classpath for step definitions. This includes BOOT-INF/classes, com.example:utilities:8.0.1.jar and com.example:models:4.2.0.jar. Cucumber can not scan inside the latter two.
If your step definitions and features are located in BOOT-INF/classes you do not have to unpack anything. Rather you have to instruct Cucumber to look for step definitions in a specific package.
If for example you are using the io.cucumber.core.cli.Main you can do this by setting the cucumber.glue=com.example.project property in cucumber.properties or passing --glue com.example.project. Otherwise see the documentation for your test runner on how to configure properties.
If your step definitions are located in either or both of the libraries you have to instruct the spring-boot-maven-plugin to unpack these dependencies. Additionally you have to instruct cucumber to only scan the packages in that specific library i.e: com.example.utilities and/or com.example.models.
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
<configuration>
<requiresUnpack>
<dependency>
<groupId>com.example</groupId>
<artifactId>utilities</artifactId>
</dependency>
<dependency>
<groupId>com.example</groupId>
<artifactId>models</artifactId>
</dependency>
</requiresUnpack>
</configuration>
</plugin>
</plugins>
There is also a Gradle equivalent.

What Build Module in IntelliJ actually does for maven module

When I run
mvn clean install
for my maven module then it compiles fine. No issues.
But when I open my pom.xml file in IntelliJ and I choose to Build -> Build module then I get following issues:
Information:javac 1.8.0_144 was used to compile java sources
Information:Module "mymodule" was fully rebuilt due to project configuration/dependencies changes
Information:09.10.2017 21:16 - Compilation completed with 3 errors and 3 warnings in 23s 991ms
C:\somepath\mymodule\pom.xml
Error:Error:osgi: [mymodule] Exception: java.lang.ClassNotFoundException: org.apache.sling.bnd.models.ModelsScannerPlugin not found, parent: java.net.URLClassLoader#29453f44 urls:[] exception:java.lang.ClassNotFoundException: org.apache.sling.bnd.models.ModelsScannerPlugin
Error:Error:osgi: [mymodule] Failed to load plugin org.apache.sling.bnd.models.ModelsScannerPlugin;generatePackagesHeader=true, error: java.lang.ClassNotFoundException: org.apache.sling.bnd.models.ModelsScannerPlugin not found, parent: java.net.URLClassLoader#29453f44 urls:[] exception:java.lang.ClassNotFoundException: org.apache.sling.bnd.models.ModelsScannerPlugin
Error:Error:osgi: [mymodule] Cannot load the plugin org.apache.sling.bnd.models.ModelsScannerPlugin
This is a module with AEM code and it uses maven-sling-plugin. It works fine for other developers in the project. Because it's working when executed directly from maven I'm trying to understand what IntelliJ does in the background. But actually, my problem is those compilation issues.
From what I've found IntelliJ does not call maven when Build is done. Any ideas how can I find differences between running from IntelliJ and directly from Maven?
What happens here is that the ModelScanner plugin can't be found using the current ClassLoader. The reason for this can be that you are using IntelliJ IDEA Ultimate which comes with a OSGI plugin already pre-installed called 'Osmorc'. If this OSGI plugin is active it will determine the classloader to be used for building OSGI related projects.
So simply de-activating this Osmorc plugin in IntelliJ should allow your build to revert to the classloader from the ModelScannerPlugin mentioned in the configuration of your the maven-bundle-plugin in your projects POM.xml file which should solve the problem.
If this still results in a similar Maven build error, then make sure to add a Maven dependency 'org.apache.sling.bnd.model' to your maven-bundle-plugin in your POM.xml file.
<!-- Apache Felix Bundle Plugin -->
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>3.3.0</version>
<inherited>true</inherited>
<extensions>true</extensions>
<executions>
<!-- Configure extra execution of 'manifest' in process-classes phase to make sure SCR metadata is generated before unit test runs -->
<execution>
<id>scr-metadata</id>
<goals>
<goal>manifest</goal>
</goals>
<configuration>
<supportIncrementalBuild>true</supportIncrementalBuild>
</configuration>
</execution>
</executions>
<configuration>
<exportScr>true</exportScr>
<instructions>
<!-- Enable processing of OSGI DS component annotations -->
<_dsannotations>*</_dsannotations>
<!-- Enable processing of OSGI metatype annotations -->
<_metatypeannotations>*</_metatypeannotations>
<_plugin>org.apache.sling.bnd.models.ModelsScannerPlugin;generatePackagesHeader=true</_plugin>
</instructions>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.sling</groupId>
<artifactId>org.apache.sling.bnd.models</artifactId>
<version>1.0.0</version>
</dependency>
</dependencies>
</plugin>
could you please check your core pom file. it should contain a plugin section like this:
<plugin> <!-- Enable registration of Sling Models classes via bnd plugin --> org.apache.sling.bnd.models.ModelsScannerPlugin, <!-- Allow the processing of SCR annotations via a bnd plugin --> org.apache.felix.scrplugin.bnd.SCRDescriptorBndPlugin;destdir=${project.build.outputDirectory} </plugin>
but if you created a project using aem archetype the tag looks like' <_plugin>

ClassCastException with OSGI bundle

I am working on OSGi bundle, which uses javax.ws.rs-api (2.0.1). Karaf is already having jsr311-api (1.1.1) loaded as bundle. When I try to load my OSGi bundle, I see the following exception. Is there a way we can ignore the previously loaded bundle?
The activate method has thrown an exception
java.lang.LinkageError: ClassCastException: attempting to castbundle://137.0:1/javax/ws/rs/ext/RuntimeDelegate.class to bundle://177.0:1/javax/ws/rs/ext/RuntimeDelegate.class
at javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:146)[137:javax.ws.rs.jsr311-api:1.1.1]
at javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:120)[137:javax.ws.rs.jsr311-api:1.1.1]
at javax.ws.rs.core.UriBuilder.newInstance(UriBuilder.java:95)[137:javax.ws.rs.jsr311-api:1.1.1]
at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119)[137:javax.ws.rs.jsr311-api:1.1.1]
Your bundle must import only the packages you need versions.
You have to create META-INF\MANIFEST.MF with Import-Package header, which will contain the list of packages required only versions.
Import-Package: javax.ws.rs.ext,version="2.0.1"
List all the packages that cause the conflict.
I think here they are:
javax.ws.rs,version="2.0.1"
javax.ws.rs.client,version="2.0.1"
javax.ws.rs.container,version="2.0.1"
javax.ws.rs.core,version="2.0.1"
javax.ws.rs.ext,version="2.0.1"
You can specify a range of versions : [2.0.1, 3) and so on.
Real example:
Import-Package: org.osgi.service.blueprint; version="[1.0.0, 2.0.0)"
You can use maven-bundle-plugin to create requered MANIFEST.MF:
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.3.7</version>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
<Bundle-Description>${project.description}</Bundle-Description>
<Import-Package>
javax.ws.rs;version=2.0.1,
javax.ws.rs.client;version=2.0.1,
javax.ws.rs.container;version=2.0.1,
javax.ws.rs.core;version=2.0.1,
javax.ws.rs.ext;version=2.0.1,
*,
org.apache.camel.osgi
</Import-Package>
<Export-Package>
your.package
</Export-Package>
</instructions>
</configuration>
</plugin>
Don't forget to install bundle version 2.0.1

Got NoClassDefFoundError when invoking Axis2 webservice deployed with maven in weblogic

I'm trying to deploy axis2 webservices in weblogic server using maven. The project has maven modules and one of that is a war that i have defined the axis servlet in. The wsdl was there, so i used wsdl2code plugin to generate the xmlbean and schema and put that in a jar module. Structure is as below.
--lv-ear (ear with dependency on war)
|
--lv-ws
|
--lv-ws-ccid (jar module with skeleton and xmlbeans)
|
--lv-ws-ecs (jar module with skeleton and xmlbeans)
|
--lv-ws-web (war module with dep on jar modules)
|
--WEB-INF
|
--conf/axis2.xml
--services/ccid/services.xml
I built and deployed the ear to weblogic domain. The war was deployed successfully as part of ear and services were deployed. I am able to access wsdl files. When I tried to call the service, i got the below ClassNotFoundException for a schema file.
Caused by: java.lang.ClassNotFoundException: schemaorg_apache_xmlbeans.system.s2104B1E6E09A2A85656B3E630BA151C1.TypeSystemHolder
I saw that the random string in that path differed fo me. So I tried to call again and got below NoClassDefFoundError, which persists even after I tried with different approaches.
java.lang.NoClassDefFoundError: Could not initialize class com.lv.ws.ccid.xmlbean.InputDocument
at com.lv.ws.ccid.xmlbean.InputDocument$Factory.parse(InputDocument.java:463)
at com.lv.ws.ccid.CcidMessageReceiverInOut.fromOM(CcidMessageReceiverInOut.java:332)
at com.lv.ws.ccid.CcidMessageReceiverInOut.invokeBusinessLogic(CcidMessageReceiverInOut.java:46)
at org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
at org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:114)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:173)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:173)
at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:144)
I searched for this and found something that told to configure app server for axis2 based on http://axis.apache.org/axis2/java/core/docs/app_server.html . When I tried it I got below error.
weblogic.xml.stax.XmlStreamInputFactory can not be cast to javax.xml.stream.XmlInputFactory
After discarding that configuration, I did some other possible deployments by having the webservice skeleton and xmlbean files in an aar and put the aar inside WEB-INF/services. I also tried putting Class-Path entry in MANIFEST.MF in ear / war for the jar files, to no avail. Still I got the same NoClassDefFoundError. Can you please give me suggestions on fixing that?
Fixed it now. This was due to my lack of experience with Axis. Issue was that, I had the generated schema and xmlbean files moved to src folder and then just tried to use normal jar function and dependency to deploy.
Now, I removed them from src folder and use wsdl2code and axis2-aar plugins to generate the xmlbean and schema files dynamically and then package them in aar. Then I deployed the aar to webapp and it worked fine. I have listed the plugin configuration inside <build> below.
<plugins>
<plugin>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-wsdl2code-maven-plugin</artifactId>
<version>1.5.4</version>
<executions>
<execution>
<id>ccid-ws</id>
<goals>
<goal>wsdl2code</goal>
</goals>
</execution>
</executions>
<configuration>
<packageName>com.lv.ws.ccid</packageName>
<wsdlFile>${basedir}/src/main/resources/META-INF/ccid.wsdl</wsdlFile>
<databindingName>xmlbeans</databindingName>
<syncMode>sync</syncMode>
<unpackClasses>true</unpackClasses>
<namespaceToPackages>https://mdm.com/portal/ws/services/ccid=com.lv.ws.ccid.xmlbean</namespaceToPackages>
<outputDirectory>${basedir}/target/generated-sources</outputDirectory> <generateServerSide>false</generateServerSide>
<generateServicesXml>true</generateServicesXml>
<skipWSDL>true</skipWSDL>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2-aar-maven-plugin</artifactId>
<version>1.6.2</version>
<extensions>true</extensions>
<executions>
<execution>
<phase>prepare-package</phase>
<goals>
<goal>aar</goal>
</goals>
</execution>
</executions>
<configuration>
<aarName>ccid</aarName>
<includeDependencies>false</includeDependencies>
<outputDirectory>${project.build.directory}/aar</outputDirectory>
</configuration>
</plugin>
</plugins>

service provider and OSGI issue

I have written an osgi bundle, where i have dependency for a jar(bsf-all.jar), which contains service provider as follows (under META_inf/services/);
bsh.engine.BshScriptEngineFactory
com.sun.script.freemarker.FreeMarkerScriptEngineFactory
com.sun.script.groovy.GroovyScriptEngineFactory
com.sun.script.jacl.JaclScriptEngineFactory
com.sun.script.jaskell.JaskellScriptEngineFactory
com.sun.script.java.JavaScriptEngineFactory
com.sun.phobos.script.javascript.RhinoScriptEngineFactory
com.sun.phobos.script.javascript.EmbeddedRhinoScriptEngineFactory
com.sun.script.jawk.JawkScriptEngineFactory
com.sun.script.jelly.JellyScriptEngineFactory
com.sun.script.jep.JepScriptEngineFactory
com.sun.script.jexl.JexlScriptEngineFactory
com.sun.script.jruby.JRubyScriptEngineFactory
com.sun.script.judo.JudoScriptEngineFactory
com.sun.script.juel.JuelScriptEngineFactory
com.sun.script.jython.JythonScriptEngineFactory
com.sun.script.ognl.OgnlScriptEngineFactory
org.pnuts.scriptapi.PnutsScriptEngineFactory
com.sun.script.scheme.SchemeScriptEngineFactory
com.sun.script.velocity.VelocityScriptEngineFactory
com.sun.script.xpath.XPathScriptEngineFactory
com.sun.script.xslt.XSLTScriptEngineFactory
When i check my bundle state via OSGI console , it is Active and there is no any dependency issue..
But when i try to use it(means after the server up and running)
server throws "Class not found" issue ;
java.lang.ClassNotFoundException: com.sun.phobos.script.javascript.RhinoScriptEngineFactory
This particular class is in my dependency jar(bsf-all.jar) and that class is exposed via the service provider..
I suspect there is a class loading issue with OSGi and java service provider..
My pom.xml is as follows;
<dependencies>
<dependency>
<groupId>org.apache.bsf</groupId>
<artifactId>bsf-all</artifactId>
<version>${bsf.version}</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>rhino</groupId>
<artifactId>js</artifactId>
<version>1.6R7</version>
<optional>true</optional>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
<Bundle-Name>${project.artifactId}</Bundle-Name>
<Export-Package>
org.apache.bsf.*,
org.mozilla.javascript.*,
org.pnuts.scriptapi.*,
com.sun.script.*,
com.sun.phobos.script.*,
bsh.engine.*,
javax.script.*,
</Export-Package>
<Import-Package>
com.sun.*
</Import-Package>
<DynamicImport-Package>*</DynamicImport-Package>
<Embed-Dependency>js;scope=compile|runtime;inline=false;</Embed-Dependency>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
I'm using Equinox OSGi implementation..How can i edit my pom to overcome above issue?
I feel that there are too many fundamental errors in this POM to begin speculating about the cause of the CNFE. Your <Export-Package> statement is of most concern. Why are you repackaging and exporting the whole of BSF, Rhino, Phobos and even parts of the JDK inside your bundle??
As I implied in my answer to your earlier question: you are getting bogged down in low-level details without, I think, having a good understanding of what you are trying to achieve at the high level. Therefore even if somebody posts an answer that gets you past this particular issue, you will still not have a working architecture.
Please step back and describe at a high level what you are trying to achieve and why. Then we can offer a solution based on good OSGi practices.

Resources