i am using Flink Connector Kafka 1.8.0. (depends on kafka-clients 2.0.1)
https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka_2.12/1.8.0
using gradle:
// https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka
compile group: 'org.apache.flink', name: 'flink-connector-kafka_2.12', version: '1.8.0'
i wonder if i can force it to use kafka-clients 2.4.0
i wodering if flink supports it, and even so, how it can be configured through
gradle?
can you assist?
Yes, you can configure that by simply explicitly including the dependency with the version that you want
compile group: 'org.apache.flink', name: 'flink-connector-kafka_2.12', version: '1.8.0'
compile group: 'org.apache.kafka', name: 'kafka_2.13', version: '2.4.0'
You can test which version has been chosen with
gradle dependencies
Related
I have already seen in some project, that in some of them is using libs and libs group: instead of implementation or deprecated compile. After local switch to implementation everything looks fine and works correctly.
Example:
libs group: 'com.google.code.gson', name: 'gson', version: '2.8.6'
instead of
implementation group: 'com.google.code.gson', name: 'gson', version: '2.8.6'
Is there any difference between them?
libs, implementation, compile are known as dependency configurations (configurations for short) in Gradle: https://docs.gradle.org/current/userguide/declaring_dependencies.html#sec:what-are-dependency-configurations
They are essentially a "bucket" to place dependency in. The Java plugin defines quite a few configurations: https://docs.gradle.org/current/userguide/java_plugin.html#sec:java_plugin_and_dependency_management
The libs configuration you mentioned is not part of any standard/provided Gradle plugin. So, it is provided/created by some other plugin you have applied to your project. Or you have defined the configuration yourself in your project somewhere.
The implementation configuration, provided by the Java plugin, are for implementation details of your application or library. In other words, these are "private" to your application/library and will not be available to consumers' classpath.
I want to use Elasticsearch's Client Java class within a Liferay 7 SP4 FP30 module, so I wrote this build.gradle:
dependencies {
compileOnly group: "com.liferay", name: "com.liferay.portal.search.elasticsearch", version: "2.1.14"
compileOnly group: "com.liferay", name: "org.elasticsearch", version: "2.2.0.LIFERAY-PATCHED-1"
compileOnly group: "biz.aQute.bnd", name: "biz.aQute.bndlib", version: "3.1.0"
compileOnly group: "com.liferay", name: "com.liferay.osgi.util", version: "3.0.0"
compileOnly group: "com.liferay", name: "com.liferay.portal.spring.extender", version: "2.0.0"
compileOnly group: "com.liferay.portal", name: "com.liferay.portal.kernel", version: "2.0.0"
compileOnly group: "com.liferay", name: "com.liferay.portal.security.audit.api", version: "2.0.0"
compileOnly group: "com.liferay", name: "com.liferay.portal.configuration.metatype", version: "2.0.0"
compileOnly group: "org.osgi", name: "org.osgi.compendium", version: "5.0.0"
}
... and a Java class containing code such as import com.liferay.portal.search.elasticsearch.connection.ElasticsearchConnectionManager; and Client client = elasticsearchConnectionManager.getClient();
It builds fine.
But when I try to start the module, this error happens:
org.osgi.framework.BundleException: Could not resolve module: mymodule [548]
Unresolved requirement: Import-Package: com.liferay.portal.search.elasticsearch.connection
Why is this happening? My build.gradle does not mention this module ending in .connection, and Maven does not seem to have any such module.
#gjoranv is correct, just because you in is on your gradle.build it does not mean it will be in your environment.
First things first, the error is due to the lack of a used package, in Java's conventional sense. So you will need a module, as represented by a jar file, that makes this package public.
As liferay is pretty version dependent when it comes to Elastic Search, and relies on accident versions, you might get away with using not exposed packages, and forcing the exposure, normally through a Uber module.
If you are feeling lucky, you can also use compileInclude, instead of compileOnly. Including the library this way will possibly make a mess, as it will embed the jar inside your jar and expose all packages.
Another possibility, which normally is way less aggressive is to embed the jar, and set the classpath inside your bundle. To do this you just need to declare your dependency as compile, and add the classpath in your bnd.bnd file. (it sounds harder than it is, it should be a trivial process)
Another issue to have in mind is the alignment with your ElasticSearch and you liferay deployment:2.2-2.4.x but this is just because you might fall into class conversion exceptions and API mismatch if your objects are used by other bundles or when interfacing with an old ES.
Embedding example:
gradle.build
compile "org.apache.httpcomponents:httpclient"
compile "org.apache.httpcomponents:httpcore"
bnd.bnd
-includeresource: lib/httpclient.jar=httpclient-4.5.3.jar,\
lib/httpcore.jar=httpcore-4.4.6.jar
Bundle-ClassPath: ., lib/httpclient.jar, lib/httpcore.jar
I'm not familiar with Liferay and gradle, but I've been working with OSGi (apache felix) and maven for a long time. The error message indicates that your bundle uses the package com.liferay.portal.search.elasticsearch.connection, but the runtime environment does not have a bundle that exports that package. The package in question is contained in the first dependency mentioned in your build.gradle, but it's not exported. If you like, you can open the bundle jar and peek into its manifest.mf by downloading it from the maven central repo.
Since the package is not exported (only com.liferay.portal.search.elasticsearch.settings is), I assume it's a signal that it's not intended for external use. So maybe you should check if there's another way of doing what you want.
From looking at the Liferay docs for using 3rd party libraries, it seems you are trying to expand the library into your module. Maybe you could try the embedding strategy instead, if you still need to use the .connection package.
My Spring-boot application started failing once Jackson 2.9 was released on 2nd of March. I am using Gradle for building and Spring boot version 1.5.2 which depends on Jackson-core 2.8.7.
In addition I need Opentok SDK which I have added as dependency:
compile group: 'com.tokbox', name: 'opentok-server-sdk', version: '2.3.2'
I suppose the reason is the Opentok SDK dependency definition which allows downloading newer JAR for Jackson which then creates a mismatch of libraries as several versions of Jackson JARs are downloaded:
https://github.com/opentok/Opentok-Java-SDK/blob/master/build.gradle
dependencies {
...
compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '[2.3.1,2.99999)'
How to sort this out?
I am not an expert of Gradle but could I somehow force Opentok to use 2.8.7 version?
I cannot deliver at the moment at all so please help.
I think this should be useful: https://docs.gradle.org/current/userguide/dependency_management.html#sub:version_conflicts
As well as the guide here: https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.dsl.DependencyHandler.html#N1627D
Approaches might differ, but you can set Gradle to force = true for Spring's jackson-databind dependency.
This is how I sorted it out
compile ('com.tokbox:opentok-server-sdk:2.3.2')
{
// Jackson 2.9 is not compatible with Spring boot 1.4.4 - 1.5.2
exclude group: 'com.fasterxml.jackson.core', module: 'jackson-databind'
}
In my project I have dependency on 'org.nd4j:nd4j-native-platform:0.6.0' which brings me transitive dependencies:
Gradle: org.nd4j:nd4j-native:linux-ppc64le:0.6.0
Gradle: org.nd4j:nd4j-native:macosx-x86_64:0.6.0
Gradle: org.nd4j:nd4j-native:windows-x86_64:0.6.0
Gradle: org.nd4j:nd4j-native:linux-x86_64:0.6.0
I want to exclude nd4j-native:linux-ppc64le and nd4j-native:macosx-x86_64 since my application does not support these platforms. I write in my Gradle file:
configurations {
all.collect { configuration ->
configuration.exclude(group: 'org.nd4j', module: 'nd4j-native', classifier: 'linux-ppc64le')
}
}
Gradle says:
Error:(44, 0) Could not set unknown property 'classifier' for object of type org.gradle.api.internal.artifacts.DefaultExcludeRule.
It seems that gradle does not support exclusion by classifier.
How to exclude such a transitive dependencies?
Update: Gradle allows us to exclude dependencies, but what if we have several dependencies with the same id and group but different classifiers?
I have faced the same issued. I have used a deeplearning4j library with Gradle dependency.
compile group: 'org.nd4j', name: 'nd4j-native-platform', version: '1.0.0-beta'
compile group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta'
When I use this it is also downloading other platform classifiers and its size is almost 500MB. but my use case is specific to the Windows platform so i don't need other classifiers for Linux and Android and other platforms.If I exclude the group it is also excluding the classifier for the windows also . And in Gradle as of my knowledge, we can not exclude specific classifiers.
So the question was how to remove the specific classifier. What I found strange is when I made jar file of the project and extracted jar it shows me the org.nd4j:nd4j-native:linux-ppc64le:0.6.0 and other jars but when I generate dependency tree it is not showing me the specific jar in the tree.
So in order to find out in which specific module and project the jar is coming from I made a separate maven project and with this
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-native-platform</artifactId>
<version>1.0.0-beta</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>1.0.0-beta</version>
</dependency>
and then I have generated a dependency tree. It showed me the jars in the dependency tree.
What I did is I have removed the whole module and I have added the required classifier in a particular module with a specific version and it worked for me.
compile (group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta')
{
exclude group: 'org.bytedeco.javacpp-presets', module: 'opencv-platform'
exclude group: 'org.bytedeco.javacpp-presets', module: 'leptonica-platform'
exclude group: 'org.bytedeco.javacpp-presets', module: 'hdf5-platform'
}
compile (group: 'org.nd4j', name: 'nd4j-native-platform', version: '1.0.0-beta')
{
exclude group: 'org.bytedeco.javacpp-presets', module: 'openblas-platform'
}
compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta', classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1', classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1', classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1',classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1',classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1',classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1',classifier: "windows-x86_64"
Doing this reduced my jar size to almost 250MB
How to exclude such a transitive dependencies?
I think, the only way is to exclude all transitive dependencies by it's module or group and manually provide dependencies on libraries for platforms your application supports. Because classifiers are supported in dependency declaration.
And the same way you can handle the case, when you have a number of dependencies with the same module and grooup, but with different classifiers. Just add such dependencies manually with it's classifier property.
How to specify latest redis client dependency for logback-redis-appender in gradle
https://mvnrepository.com/artifact/com.cwbase/logback-redis-appender/1.1.5
I want to use 2.9.0 dependency of https://mvnrepository.com/artifact/redis.clients/jedis instead of 2.5.2 being used right now.
Add to build.gradle:
compile group: 'redis.clients', name: 'jedis', version: '2.9.0'
or
dependencies {
compile "redis.clients:jedis:2.9.0"
}