In my project I have dependency on 'org.nd4j:nd4j-native-platform:0.6.0' which brings me transitive dependencies:
Gradle: org.nd4j:nd4j-native:linux-ppc64le:0.6.0
Gradle: org.nd4j:nd4j-native:macosx-x86_64:0.6.0
Gradle: org.nd4j:nd4j-native:windows-x86_64:0.6.0
Gradle: org.nd4j:nd4j-native:linux-x86_64:0.6.0
I want to exclude nd4j-native:linux-ppc64le and nd4j-native:macosx-x86_64 since my application does not support these platforms. I write in my Gradle file:
configurations {
all.collect { configuration ->
configuration.exclude(group: 'org.nd4j', module: 'nd4j-native', classifier: 'linux-ppc64le')
}
}
Gradle says:
Error:(44, 0) Could not set unknown property 'classifier' for object of type org.gradle.api.internal.artifacts.DefaultExcludeRule.
It seems that gradle does not support exclusion by classifier.
How to exclude such a transitive dependencies?
Update: Gradle allows us to exclude dependencies, but what if we have several dependencies with the same id and group but different classifiers?
I have faced the same issued. I have used a deeplearning4j library with Gradle dependency.
compile group: 'org.nd4j', name: 'nd4j-native-platform', version: '1.0.0-beta'
compile group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta'
When I use this it is also downloading other platform classifiers and its size is almost 500MB. but my use case is specific to the Windows platform so i don't need other classifiers for Linux and Android and other platforms.If I exclude the group it is also excluding the classifier for the windows also . And in Gradle as of my knowledge, we can not exclude specific classifiers.
So the question was how to remove the specific classifier. What I found strange is when I made jar file of the project and extracted jar it shows me the org.nd4j:nd4j-native:linux-ppc64le:0.6.0 and other jars but when I generate dependency tree it is not showing me the specific jar in the tree.
So in order to find out in which specific module and project the jar is coming from I made a separate maven project and with this
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-native-platform</artifactId>
<version>1.0.0-beta</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>1.0.0-beta</version>
</dependency>
and then I have generated a dependency tree. It showed me the jars in the dependency tree.
What I did is I have removed the whole module and I have added the required classifier in a particular module with a specific version and it worked for me.
compile (group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta')
{
exclude group: 'org.bytedeco.javacpp-presets', module: 'opencv-platform'
exclude group: 'org.bytedeco.javacpp-presets', module: 'leptonica-platform'
exclude group: 'org.bytedeco.javacpp-presets', module: 'hdf5-platform'
}
compile (group: 'org.nd4j', name: 'nd4j-native-platform', version: '1.0.0-beta')
{
exclude group: 'org.bytedeco.javacpp-presets', module: 'openblas-platform'
}
compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta', classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1', classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1', classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1',classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1',classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1',classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1',classifier: "windows-x86_64"
Doing this reduced my jar size to almost 250MB
How to exclude such a transitive dependencies?
I think, the only way is to exclude all transitive dependencies by it's module or group and manually provide dependencies on libraries for platforms your application supports. Because classifiers are supported in dependency declaration.
And the same way you can handle the case, when you have a number of dependencies with the same module and grooup, but with different classifiers. Just add such dependencies manually with it's classifier property.
Related
I want to build a fat jar for tesseract. With the following build settings I get a jar of about 68 MB with dependencies for all supported platforms:
dependencies {
implementation group: 'org.bytedeco', name: 'tesseract-platform', version: '4.1.1-1.5.3'
}
jar {
manifest { attributes 'Main-Class': 'BasicExample' }
from { configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) } }
}
In order to reduce this size, I tried to only include the dependencies for my platform following this guideline - but without success, see also this SO question and #Samuel Audet's answer to it (I want to stick with gradle). Therefore I decided to manually include only the required dependencies from the POM file:
dependencies {
implementation group: 'org.bytedeco', name: 'tesseract', version: '4.1.1-1.5.3'
implementation group: 'org.bytedeco', name: 'tesseract', version: '4.1.1-1.5.3', classifier: 'windows-x86_64'
implementation group: 'org.bytedeco', name: 'leptonica', version: '1.79.0-1.5.3', classifier: 'windows-x86_64'
}
This reduces the size of the jar to about 7 MB and works fine (at least for the basic example). Nevertheless I get warnings:
Warning: Could not load ...: java.lang.UnsatisfiedLinkError: no jnijavacpp in java.library.path: ...
Comparing the two jars I found that the small jar lacks the lib path with all the so libs along with header, cmake and pkgconfig files and I assume that this is the reason for the warnings.
So my questions are:
If these files are apparently not necessary to run the jar, why are they included normally?
How can I prevent these warnings while keeping the jar size small?
Any other way to build a jar with just the required dependencies for one platform is of course also welcomed.
I'll have to update that guide a little bit, thanks for reporting! We now also need javacpp-platform, which would give us something like this for windows-x86_64 in this case:
dependencies {
implementation group: 'org.bytedeco', name: 'tesseract', version: '4.1.1-1.5.3'
implementation group: 'org.bytedeco', name: 'tesseract', version: '4.1.1-1.5.3', classifier: 'windows-x86_64'
implementation group: 'org.bytedeco', name: 'leptonica', version: '1.79.0-1.5.3', classifier: 'windows-x86_64'
implementation group: 'org.bytedeco', name: 'javacpp', version: '1.5.3', classifier: 'windows-x86_64'
}
This is needed to fix some issues at load time as explained in this issue:
https://github.com/bytedeco/javacv/issues/1305
UPDATE: It's now possible to use Gradle JavaCPP to do this more easily this way:
plugins {
id 'java-library'
id 'org.bytedeco.gradle-javacpp-platform' version "$javacppVersion"
}
// We can set this on the command line too this way: -PjavacppPlatform=linux-x86_64,macosx-x86_64,windows-x86_64,etc
ext {
javacppPlatform = 'linux-x86_64,macosx-x86_64,windows-x86_64,etc' // defaults to Loader.getPlatform()
}
dependencies {
api "org.bytedeco:tesseract-platform:$tesseractVersion"
}
I have written code for Google Analytics API in Liferay DXP. I have imported the following jars by writing them in build.gradle
compile group: 'commons-logging', name: 'commons-logging', version: '1.1.1'
compile group: 'com.google.api-client', name: 'google-api-client', version: '1.22.0'
compile group: 'com.google.api-client', name: 'google-api-client-appengine', version: '1.22.0'
compile group: 'com.google.api-client', name: 'google-api-client-gson', version: '1.22.0'
compile group: 'com.google.http-client', name: 'google-http-client-jackson2', version: '1.22.0'
compile group: 'com.google.api-client', name: 'google-api-client-java6', version: '1.22.0'
compile group: 'com.google.api-client', name: 'google-api-client-servlet', version: '1.22.0'
compile group: 'com.google.apis', name: 'google-api-services-analytics', version: 'v3-rev132-1.22.0'
compile group: 'com.google.http-client', name: 'google-http-client', version: '1.22.0'
compile group: 'com.google.http-client', name: 'google-http-client-appengine', version: '1.22.0'
compile group: 'com.google.http-client', name: 'google-http-client-gson', version: '1.22.0'
compile group: 'com.google.http-client', name: 'google-http-client-jackson2', version: '1.22.0'
compile group: 'com.google.http-client', name: 'google-http-client-jdo', version: '1.22.0'
compile group: 'com.google.oauth-client', name: 'google-oauth-client', version: '1.22.0'
compile group: 'com.google.oauth-client', name: 'google-oauth-client-appengine', version: '1.22.0'
compile group: 'com.google.oauth-client', name: 'google-oauth-client-java6', version: '1.22.0'
compile group: 'com.google.oauth-client', name: 'google-oauth-client-jetty', version: '1.22.0'
compile group: 'com.google.oauth-client', name: 'google-oauth-client-servlet', version: '1.22.0'
compile group: 'com.google.code.gson', name: 'gson', version: '2.1'
compile group: 'org.apache.httpcomponents', name: 'httpcore', version: '4.0.1'
compile group: 'com.fasterxml.jackson.core', name: 'jackson-core', version: '2.1.3'
compile group: 'javax.jdo', name: 'jdo2-api', version: '2.3-eb'
compile group: 'org.mortbay.jetty', name: 'jetty', version: '6.1.26'
compile group: 'org.mortbay.jetty', name: 'jetty-util', version: '6.1.26'
compile group: 'com.google.code.findbugs', name: 'jsr305', version: '1.3.9'
compile group: 'javax.transaction', name: 'transaction-api', version: '1.1'
Also I have included the same in bnd.bnd file using the following code:
Include-Resource: #google-api-client-1.22.0.jar,#google-http-client-1.22.0.jar,#google-api-client-java6-1.22.0.jar,#google-http-client-jackson2-1.22.0.jar,#google-http-client-gson-1.22.0.jar,#google-oauth-client-1.22.0.jar,#google-oauth-client-java6-1.22.0.jar,#gson-2.1.jar,#google-api-services-analytics-v3-rev132-1.22.0.jar,#jackson-core-2.1.3.jar,#commons-logging-1.1.1.jar,#google-api-client-appengine-1.22.0.jar,#google-api-client-gson-1.22.0.jar,#google-api-client-servlet-1.22.0.jar,#google-api-services-analytics-v3-rev132-1.22.0.jar,#google-http-client-appengine-1.22.0.jar,#google-http-client-jackson2-1.22.0.jar,#google-http-client-jdo-1.22.0.jar,#google-oauth-client-appengine-1.22.0.jar,#google-oauth-client-jetty-1.22.0.jar,#google-oauth-client-servlet-1.22.0.jar,#gson-2.1.jar,#httpcore-4.0.1.jar,#jackson-core-2.1.3.jar,#jdo2-api-2.3-eb.jar,#jetty-6.1.26.jar,#jetty-util-6.1.26.jar,#jsr305-1.3.9.jar,#transaction-api-1.1.jar
The module compiles fine and I am able to get jar. But when I deploy this jar on server I keep getting Unresolved requirement:Import-Package: error.
I know that the issue is with the Transitive dependencies. I read somewhere on Liferay community that the things that we mention in build.gradle is available at compile time and it does not looks for Transitive dependencies at compile time but at runtime we need to have transitive dependencies as well.
Isnt there any way that Gradle can download transitive dependencies on its own instead of me mentioning them individually in build.gradle file.
With Include-Resource (or -includeresource) specifying #some.jar, you're copying all of the jar's content into your own jar. This is a build instruction.
For compile time, gradle can identify and download transitive dependencies, but note that some of them might be optional - in that case it's questionable if you want to have them all. And also: Note that a compile time dependency does not necessarily mean that this dependency must be packaged for runtime. In fact, the packaging that you do, with IncludeResource, is done by bnd, not by gradle at all.
What you specify here, including 30 jars in your own jar, is not best practice. If they're OSGi bundles, you should just deploy them to the OSGi runtime separately. If they're not OSGi bundles, you should look for places where they're transformed to OSGi bundles. And only then should you include the left-over non-bundles in your own jar, unless you find better options (e.g. turn them into bundles yourself and send pullrequests upstream)
For demonstration: Let me unroll your list of jars from the IncludeResource directive, that you have entered in one line:
#google-api-client-1.22.0.jar
#google-http-client-1.22.0.jar
#google-api-client-java6-1.22.0.jar
#google-http-client-jackson2-1.22.0.jar
#google-http-client-gson-1.22.0.jar
#google-oauth-client-1.22.0.jar
#google-oauth-client-java6-1.22.0.jar
#gson-2.1.jar
#google-api-services-analytics-v3-rev132-1.22.0.jar
#jackson-core-2.1.3.jar
#commons-logging-1.1.1.jar
#google-api-client-appengine-1.22.0.jar
#google-api-client-gson-1.22.0.jar
#google-api-client-servlet-1.22.0.jar
#google-api-services-analytics-v3-rev132-1.22.0.jar
#google-http-client-appengine-1.22.0.jar
#google-http-client-jackson2-1.22.0.jar
#google-http-client-jdo-1.22.0.jar
#google-oauth-client-appengine-1.22.0.jar
#google-oauth-client-jetty-1.22.0.jar
#google-oauth-client-servlet-1.22.0.jar
#gson-2.1.jar (oops, repeated)
#httpcore-4.0.1.jar
#jackson-core-2.1.3.jar
#jdo2-api-2.3-eb.jar
#jetty-6.1.26.jar
#jetty-util-6.1.26.jar
#jsr305-1.3.9.jar
#transaction-api-1.1.jar
While I'd discourage to keep up including 30 jars in your own jar, there's another option for those dependencies that aren't OSGi Bundles: If there are any required (non-optional) transitive dependencies that would need to be included, I understand that the compileInclude directive for gradle will automatically include them.
I've recently recorded a video about the different gradle options, that soon (after proper edits) will be added to the (free) OSGi Basics lesson on Liferay University - the preliminary edit of the video might help understand what's happening when you include third party code in your own jars- you can find it here.
Unable to use Spring's reactive WebClient in a Spring boot based Web Application which runs on Tomcat. Application startup itself is failing due to unresolved dependencies.
I looked at https://github.com/spring-projects/spring-boot/issues/9690 but unfortunately thats not an option for me as I cannot set the WebApplication type to None
My gradle file:
dependencies {
compile(group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa') {
exclude(module: 'tomcat-juli')
}
compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-redis'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web'
compile group: 'org.springframework.session', name: 'spring-session'
compile group: 'org.hibernate', name: 'hibernate-java8'
compile group: 'org.hibernate', name: 'hibernate-validator', version: '5.2.4.Final'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-guava'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-hibernate5'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-jsr310'
compile group: 'com.google.guava', name: 'guava'
compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.3.2'
compile group: 'org.postgresql', name: 'postgresql'
compile group: 'io.jsonwebtoken', name: 'jjwt', version:'0.9.0'
compile group: 'com.nimbusds', name: 'nimbus-jose-jwt', version:'5.12'
// Spring managed dependency is not working, so resorted to using specific version
// compile('org.springframework.boot:spring-boot-starter-webflux')
compile group: 'org.springframework.boot', name: 'spring-boot-starter-webflux', version: '2.1.4.RELEASE'
/*
compile group: 'org.springframework', name: 'spring-web-reactive', version: '5.0.0.M1'
compile group: 'org.springframework', name: 'spring-web', version: '5.1.6.RELEASE'
*/
implementation 'com.auth0:jwks-rsa:0.7.0
}
bootRepackage {
enabled = false
}
Error:
Caused by: java.lang.NoClassDefFoundError: org/springframework/http/client/reactive/ClientHttpConnector
at org.springframework.web.reactive.function.client.WebClient.create(WebClient.java:144)
I then tried adding,
compile group: 'org.springframework', name: 'spring-web-reactive', version: '5.0.0.M1'
Which then complained about something else, which made me add
compile group: 'org.springframework', name: 'spring-web', version: '5.1.6.RELEASE'
But then another error shows up while starting my application, which makes me think there is some other problem.
To be precise,I simply want to call a MicroService using Reactive WebClient from within another MicroService (which is Spring Boot based Web Application).
Using RestTemplate works fine but I just wanted to try the reactive WebClient and stuck with error.
Due to the way we host our dependencies, managed dependencies are not working. Our dependency server is unable to resolve them. That's the reason for having specific version of spring boot dependencies
org.springframework.http.client.reactive.ClientHttpConnector is part of the spring-web jar so it should be available if use use spring-boot-starter-webflux or spring-boot-starter-web. If you're writing a traditional servlet based application then you should be using spring-boot-starter-web.
Check the output of gradle dependencies to ensure that you have a spring-web jar. If you do, the chances are it's somehow been corrupted and you should clear your gradle cache.
I'd also recommend that you use Spring Boot's managed dependencies rather than specifify version numbers on each dependency. This will ensure that you get versions that are known to work well together.
Currently I am learning Nexus and gradle integration and stuck with one issue.
One file OJDBC14.jar is has already been uploaded on nexus earlier. So when we mentioned that jar in build.gradle dependency, it downloads from nexus.
OJDBC14
dependencies {
compile 'mysql:mysql-connector-java:8.0.9-rc'
compile 'ojdbc:ojdbc:14'
testCompile group: 'junit', name: 'junit', version: '4.12'
compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.5'
compile 'org.apache.poi:poi:4.0.0'
compile 'org.apache.poi:poi-ooxml:4.0.0'
compile 'com.aventstack:extentreports:3.1.2'
compile 'org.jsoup:jsoup:1.11.3'
compile 'com.google.code.gson:gson:2.8.5'
compile 'com.sparkjava:spark-core:2.8.0'
compile group: 'org.apache.commons', name: 'commons-math3', version: '3.0'}
Recently I have uploaded new jar file OJDBC8.jar on nexus using UI but build.gradle cannot download that file from nexus.
OJDBC8
dependencies {
compile 'mysql:mysql-connector-java:8.0.9-rc'
compile 'ojdbc:ojdbc:14'
compile 'ojdbc:ojdbc:8'
testCompile group: 'junit', name: 'junit', version: '4.12'
compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.5'
compile 'org.apache.poi:poi:4.0.0'
compile 'org.apache.poi:poi-ooxml:4.0.0'
compile 'com.aventstack:extentreports:3.1.2'
compile 'org.jsoup:jsoup:1.11.3'
compile 'com.google.code.gson:gson:2.8.5'
compile 'com.sparkjava:spark-core:2.8.0'
compile group: 'org.apache.commons', name: 'commons-math3', version: '3.0'}
I checked the .gradle log but it does not showing any error message.
Is there any other settings to do when we upload the jar using UI?
You are trying to add two different versions of the same module ojdbc:ojdbc for the same configuration compile, but this is not supported by Gradle, see more details about dependencies resolution here. In your case, unless you change the default resolution strategy, the latest version of this module will be used, so version 14.
So I think your issue is not connected to this particular library version 8 in your Nexus: you can try to remove/comment the dependency compile 'ojdbc:ojdbc:14' and keep only compile 'ojdbc:ojdbc:14': Gradle should be able to download/use this version 8.
I am creating a multi module project with Spring Boot for deploying my application and using gradle as my build tool.
Now, I am creating independent deployments for some modules in the project. Some of the project requires Embedded tomcat and some do not. All the common dependencies have been put on a common project and all other modules are dependent on this common project.
Most of the other deployment modules require an embedded tomcat application server and other web components (provided by org.springframework.boot', name: 'spring-boot-starter-web) so this has been included in the build.gradle for common project
Here is the build.gradle for common project:
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web', version: springBootVersion
compile group: 'org.springframework.boot', name: 'spring-boot-starter-actuator', version: springBootVersion
compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa', version: springBootVersion
compile group: 'org.springframework.data', name: 'spring-data-jpa', version: springDataJpaVersion
Now, one of the other modules which is going to be deployed independently does not require this embedded tomcat and other mvc jars which comes with including spring-boot-starter-web but requires the other transitive dependencies from common project. As such, I want to exclude the transitive dependency for
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web', version: springBootVersion
I am doing it like this in the build.gradle other project
dependencies {
compile project(':common') {
exclude group 'org.springframework.boot', module:'spring-boot-starter-web'
}
}
But while building it is throwing this error:
startup failed: build file
'/Users/user1/Documents/repo/storm/build.gradle': 30: expecting '}',
found ',' # line 30, column 44.
oup 'org.springframework.boot', module:'
changing it to:
dependencies {
compile project(':common'){
exclude group 'org.springframework.boot:spring-boot-starter-web'
}
}
throws:
Could not find method exclude() for arguments [parent-project name] on
project ':common'.
How can I exclude transitive dependency here?
You need to invoke exclude on compile not on project:
dependencies {
compile(project(':common')) {
exclude group: 'org.springframework.boot', module:'spring-boot-starter-web'
}
}