Not able to connect Google Cloud Datastore with AEM 6.5 : Bundle has resolved dependencies - maven

I have a requirement to connect google cloud datastore from AEM. I have added the dependencies in main pom and core pom.
MAIN POM
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>libraries-bom</artifactId>
<version>16.4.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>de here
Core POM
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-datastore</artifactId>
</dependency>
When I deploy my bundle is in insatlled state and has the following errors
com.google.auth -- Cannot be resolved
com.google.auth.oauth2 -- Cannot be resolved
com.google.cloud -- Cannot be resolved
com.google.cloud.datastore -- Cannot be resolved

try adding those libs to the embed-dependencies in the POM. Also check that your libs are inside the .jar file generated.
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<!-- <Embed-Dependency> artifactId1, artifactId2;inline=true </Embed-Dependency> -->
<Export-Package>we.retail.core.model*</Export-Package>
<Private-Package>we.retail.core*</Private-Package>
<Sling-Model-Packages>
we.retail.core.model
</Sling-Model-Packages>
</instructions>
</configuration>
</plugin>
In the embed you can try something like: google-cloud-datastore
Also take care with the import type that you're using.
Link to Maven
Hope you can find the answer

If you do not need those, exclude them from the <Import-Packages>:
<Import-Package>
!com.google.auth.*,
*
</Import-Package>

Related

Transitive Dependency: Using Elasticsearch Rest High Client problem in AEM

I am trying to use Java High Level Rest Client in Adobe Experience Manager to finish project of comparison between Lucene, Solr and Elasticsearch search engines.
I am having some problems with elasticsearh implementation.
Here is the code:
Dependency in the parent pom.xml (the same is defined in core pom.xml)
<!-- Elasticseach dependencies -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.4.0</version>
</dependency>
The only line of code that I am using that is from dependencies above
try (RestHighLevelClient client = new
RestHighLevelClient(RestClient.builder(new HttpHost(server, port,
protocol),
new HttpHost(server, secondPort, protocol)));)
{
}
catch (ElasticsearchException e)
{
LOG.error("Exception: " + e);
}
protocol = "http", server = "localhost", port = 9200, secondPort =
9201
Error
Dependencies from IntelliJ
I know that there is usually problem with dependencies versions, but all are 7.4.0 in this case. Also elasticsearch 7.4.0v is running locally on 3 nodes.
This project is done on We.Retail project so it is easy to replicate. Also all the code with this error is available here:
https://github.com/tadijam64/search-engines-comparison-on-we-retail/tree/elasticsearch-integration
AEM 6.4v.
Any info or idea is appreciated.
UPDATE
I tried with adding the following to embed these dependencies externally since they are not OSGi dependencies:
<build>
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-scr-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Embed-Dependency>org.apache.servicemix.bundles.solr-solrj, log4j, noggit, zookeeper,
elasticsearch-rest-high-level-client
</Embed-Dependency>
<Embed-Transitive>true</Embed-Transitive>
<Embed-Directory>OSGI-INF/lib</Embed-Directory>
<Export-Package>we.retail.core.model*</Export-Package>
<Import-Package>
*;resolution:=optional
</Import-Package>
<Private-Package>we.retail.core*</Private-Package>
<Sling-Model-Packages>
we.retail.core.model
</Sling-Model-Packages>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
The error remains. I also tried adding it to the "export-package", but nothing helps.
And by Elasticsearch documentation, all I need to use Elasticsearch is
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.4.0</version>
</dependency>
but then NoClassDefFoundErrors occurs. It seems like a problem with transitive dependencies maybe. Not sure, but any idea is appreciated.
Some other suggestions can be found here: https://forums.adobe.com/thread/2653586
I have also tried adding it's transitive dependencies like org.elasticsearch and org.elasticsearch.client, but it does not work. The same error, just other class.
AEM version 6.4, Java version: jdk1.8.0_191.jdk
So my guess was right, transitive dependencies were not included altho <Embed-Transitive>true</Embed-Transitive> exists.
The following is necessary when running elasticsearch as a search engine on AEM the problem:
I have added all transitive dependencies in pom.xml (versions are defined in parent/pom.xml):
<!-- Elasticsearch -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-x-content</artifactId>
</dependency>
<dependency>
<groupId>org.elasticsearch.plugin</groupId>
<artifactId>rank-eval-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-imaging</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.elasticsearch.plugin</groupId>
<artifactId>lang-mustache-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpasyncclient</artifactId>
</dependency>
It is important to add all third-party dependencies as <Embed-Dependency> inside maven-bundle-plugin like this:
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Embed-Dependency>org.apache.servicemix.bundles.solr-solrj, noggit,
elasticsearch-rest-high-level-client,
elasticsearch,
elasticsearch-rest-client,
elasticsearch-x-content,
elasticsearch-core,
rank-eval-client,
lang-mustache-client,
httpasyncclient;
</Embed-Dependency>
<Embed-Transitive>true</Embed-Transitive>
<Embed-Directory>OSGI-INF/lib</Embed-Directory>
<Export-Package>we.retail.core.model*</Export-Package>
<Import-Package>
*;resolution:=optional
</Import-Package>
<Private-Package>
we.retail.core*
</Private-Package>
<Sling-Model-Packages>
we.retail.core.model
</Sling-Model-Packages>
<_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>
</instructions>
</configuration>
</plugin>
Important to notice:
All third-party dependencies (the ones outside of OSGi) must be included in the "Embed-Dependency"
"Embed-Transitive" must be set to true to include transitive dependencies
"Import-Package" must include "*;resolution:=optional" to exclude all dependencies that could not be resolved so that the program can run
normally
For some reason, there was an error in compile time when "elasticsearch" dependency was added which is not important for this
task, so I've decided to ignore it this way:
<_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>
Though challenging, I finally resolved it. There are many similar or the same problems on Google, so I hope this will help someone. Thanks to everyone that tried to help.

Add external library .jar to Spring boot .jar internal /lib

I have an external .jar that cannot be imported from public repositories using pom.xml, it's sqljdbc41.jar.
I can run the project locally from my IDE, and everything will work. I referenced the library after downloading it like so:
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>sqljdbc41</artifactId>
<version>4.1</version>
<scope>system</scope>
<systemPath>${basedir}/lib/sqljdbc41.jar</systemPath>
</dependency>
When I run mvn clean package to create my .jar file and try to run the created .jar, a mistake will pop up, which mentions the SQL Server references are not valid. I then extracted my .jar file and true enough, everything that is referenced in the pom.xml file properly gets downloaded and added, however, my SQL Server does not.
I can, in a very hacky way* just manually add the sqljdbc41.jar to my /lib folder after it's been compiled as a .jar, and it'll work, however that seems highly unoptimal. What would be a better approach?
*Opening the .jar file with Winrar, going to the /lib folder, manually selecting my sqljdbc41.jar file, then make sure to select the No Compression option bottom left where Winrar gives you compression options, in case you find this by Google and no one answered.
you can set 'includeSystemScope' to true.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
You could install the sqljdbc41.jar in your local repository :
mvn install:install-file -Dfile=path/to/sqljdbc41.jar -DgroupId=com.microsoft.sqlserver -DartifactId=sqljdbc41 -Dversion=4.1 -Dpackaging=jar
And then declare the dependency as a standard dependency :
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>sqljdbc41</artifactId>
<version>4.1</version>
</dependency>
If you use a remote artifact repository (nexus, archiva...) you also need to deploy the artifact on this repository. You can find more here : https://maven.apache.org/guides/mini/guide-3rd-party-jars-remote.html
Another way, you can put it into the resources folder, such as resources/lib/xxx.jar, then config the pom.xml like this:
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>sqljdbc41</artifactId>
<version>4.1</version>
<scope>system</scope>
<systemPath>${basedir}/src/main/resources/lib/sqljdbc41.jar</systemPath>
</dependency>
In Spring Boot: I also faced similar issue and below code helped me.
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.5.7.RELEASE</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
</plugins>
</build>
It works for me:
project {root folder}/libs/ojdbc-11.2.0.3.jar
pom.xml:
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc</artifactId>
<version>11.2.0.3</version>
<scope>system</scope>
<systemPath>${basedir}/libs/ojdbc-11.2.0.3.jar</systemPath>
</dependency>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
In my case, the fault was providing a version number without "dot" in tag:
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<scope>system</scope>
<version>1</version>
<systemPath>${basedir}/src/main/resources/lib/tools.jar</systemPath>
</dependency>
This one works:
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<scope>system</scope>
<version>1.8</version>
<systemPath>${basedir}/src/main/resources/lib/tools.jar</systemPath>
</dependency>
When Spring-Boot projects are used with maven or gradle plugins they packaged the applicaiton by default as executable jars.
These executable jars cannot be used as dependency in any another Spring-Boot project because the executable jar add classes in BOOT-INF/classes folder. This means that they cannot be found when the executable jar is used as a dependency because the dependency jar will also have the same class path structure as shown below.
If we want to use project-A as a maven dependency in project-B then we must have two artifacts. To produce the two artifacts, one that can be used as a dependency and one that is executable, a classifier must be specified. This classifier is applied to the name of the executable archive, leaving the default archive for use as a dependency.
To configure a classifier of exec in Maven, you can use the following configuration:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<classifier>exec</classifier>
</configuration>
</plugin>
</plugins>
</build>
So the MAJIC WORD here is <classifier>exec</classifier> this will create a jar structure as below and then it could easily be conusmed by spring-boot project as maven dependency jar on class path.
The above plugin need to be add in project-A pom that is going to be used as dependency in project-B. Same is explained in spring documentation section 16.5. as well.
In order to work through the local repository, the target .jar file that we will work with must be in the s2 folder. Several methods can be used for this:
The file can be taken manually and put in the relevant place (not
preferred). The same process can be done by installing it via the
console.
Relevant Remote URL is written in the .pom file dependencies and
automatically places it in the s2 folder when Intellij is refreshed
(validate) in the IDE used.
The same process can be done by addressing the .pom file dependencies via the centeral repository.
Attention: ComponentScan should not be forgotten for the related jar work on SpringBot.

Problems with maven apache mina pom.xml

I have following entries in my pom.xml.
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-core</artifactId>
<version>2.0.4</version>
</dependency>
<dependency>
<groupId>org.apache.mina</groupId>
<artifactId>mina-filter-compression</artifactId>
<version>2.0.7</version>
</dependency>
I am getting "Missing artifact org.apache.mina:mina-core:bundle:
2.0.7" error in my pom.xml .
Could someone please help in resolving this error.
Add to your pom file:
<plugins>
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
</plugin>
</plugins>
Addition to the accepted answer, an explanation why this is necessary:
The various MINA dependencies rely on OSGi bundle artifacts rather than standard JAR files.
As such, it's necessary to add support for these bundle to Maven using the Apache Felix maven-bundle-plugin.
See https://stackoverflow.com/a/5409602 for a good explanation of OSGi bundles, with links to more info.

maven ignoring findbugs suppressFBWarnings annotation

I have 2 projects that I am using the FindBugs plugin in maven to identify bugs. I am also using the #SuppressFBWarnings annotation to ignore specific bugs.
With the first project, I added the dependancies to the pom.xml and both the findbugs report and the annotation worked fine. With the second project, the report gets generated, but it still identifies bugs that I have suppressed using the annotation.
I run mvn clean install site to generate the reports on my machine in the build folder.
Each of the 2 projects I mentioned, have sub-projects with their own pom.xml files in their sub-directories, so in the parent directory, I also have a pom.xml. This directory layout is mirrored identically in both of the main projects.
Here is the XML I added to the parent poms under the <reporting> tag:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<xmlOutput>true</xmlOutput>
<findbugsXmlOutput>true</findbugsXmlOutput>
<fork>true</fork>
<threshold>Low</threshold>
<effort>Min</effort>
</configuration>
</plugin>
Also, in this same parent pom, I added this to the <dependencyManagement><dependencies> section:
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>annotations</artifactId>
<version>2.0.1</version>
</dependency>
This is identical in both of the main projects poms.
Now, in the sub-projects where I actually use the #SuppressFBWarnings annotation, and only in that particular sub-project, I have this under <dependencies>:
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>annotations</artifactId>
<version>2.0.1</version>
</dependency>
Also, this is mirrored in the other working project. I copied and pasted directly.
One project works perfect and I can successfully suppress false positives. The other project completely ignores the #SuppressFBWarnings anotation, and I can't seem to fix it.
Is there something I'm missing here?
I think that if an annotation is not found, instead of giving an error, it will just ignore it? How can I tell if its not found?
Hopefully this is a simple fix.
Thanks.
#SuppressFBWarnings was introduced with the annotation in version 3. That's why it should look like this:
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>annotations</artifactId>
<version>3.0.1</version>
</dependency>
Try to add annotations artifcat to the plugin dependencies :
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<xmlOutput>true</xmlOutput>
<findbugsXmlOutput>true</findbugsXmlOutput>
<fork>true</fork>
<threshold>Low</threshold>
<effort>Min</effort>
</configuration>
<dependencies>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>annotations</artifactId>
<version>2.0.1</version>
</dependency>
</dependencies>
</plugin>
Ensure that the dependency added is in between the dependencies tags.
Like this:
<dependencies>
<dependency>
<groupId>something</groupId>
<artifactId>something</artifactId>
<version>something</version>
</dependency>
<dependencies>

How to package dependencies with maven nbm plugin

I have modified netbeans module jar that I want to package with my module. How do I do this ?
Dependency in context
<dependency>
<groupId>org.netbeans.modules</groupId>
<artifactId>org-netbeans-modules-db-dataview</artifactId>
<version>${netbeans.platform.version}</version>
<scope>system</scope>
<systemPath>${dbdataview}</systemPath>
</dependency>
mvn nbm configuration
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>nbm-maven-plugin</artifactId>
<version>3.5</version>
<extensions>true</extensions>
<configuration>
<additionalArguments>${netbeans.run.params.ide}</additionalArguments>
<netbeansInstallation>/home/venkat/netbeans-7.1.1/</netbeansInstallation>
<keystore>keystore</keystore>
<keystorealias>ezondaice</keystorealias>
<keystorepassword>ezondaice</keystorepassword>
</configuration>
</plugin>
Edit
I have the following dependency as well. This one seems to get packaged with the nbm inside
/netbeans/modules/ext/org.yaml/. Not sure why the other dependency is not being packaged into nbm.
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.11-SNAPSHOT</version>
</dependency>

Resources