By reading the documentation I understood that with Apache Flink 1.3 I should be able to use Elasticsearch 5.x.
However, in my pom.xml :
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-elasticsearch5_2.10</artifactId>
<version>1.3.0</version>
</dependency>
I got this :
Dependency
"org.apache.flink:flink-connector-elasticsearch5_2.10:1.3.0" not found
Any idea why this dependency is unfound ?
This was a bug in the 1.3.0 release and is being fixed for 1.3.1 (which is due very soon). See the mailing list for more details.
Use the following in the pom.xml
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-elasticsearch5_2.10</artifactId>
<version>1.3.1</version>
</dependency>
Related
Please share me the dependencies information about using ES 5.6.7 with springboot ?
As i have ES 2.4 and it is working now, but after upgrade failing to work.
You can use :
<!-- https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch -->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>5.6.6</version>
</dependency>
You can check here for further more dependency
I have a module which has Spark 2.1.0 and Presto 0.166.
Spark 2.1.0 requires com.faster.xml version 2.6.5 while Presto 0.166 requires 2.8.1 strictly. How Can I resolve the issue in the same pom.xml so that I can run them in the same module?
Simply specify the version of com-fasterxml-jackson in your pom file. The version specified here will override the versions in Spark 2.1.0 and Presto 0.166
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.8.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>com.facebook.presto</groupId>
<artifactId>presto...</artifactId>
<version>0.166</version>
</dependency>
Since, Spark 2.1.0 can use com.fasterxml.jackson 2.8.1, you won't need 2 different versions of it in your module.
Resources -
Introduction to the Dependency Mechanism
You cannot use multiple version(s) of same dependency in a single pom.xml, exclude-dependency com.faster.xml version either from Spark 2.1.0 or from Presto 0.166, for example:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.0</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</exclusion>
</exclusions>
</dependency>
Are you trying to write a plugin for Presto? If so, the Presto SPI explicitly depends on only jackson-annotations and not the implementation. There should be no problem with using the newer version of the annotations with an older version of Jackson within your plugin. The version of Jackson used by the Presto engine can and will be different from the one used by your plugin as plugins are loaded in a separate class loader.
The Presto plugin system is designed to have very minimal dependencies and allow you to use whatever versions of libraries you want (as that is often necessary when writing a connector to a random system that uses older versions of libraries).
I'm working on a project using Apache Camel and Elasticsearch and I was wondering which version of Elasticsearch does Camel support?
My pom.xml looks like this:
<dependencies>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>2.18.2</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-elasticsearch</artifactId>
<version>2.18.2</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-xmljson</artifactId>
<version>2.18.2</version>
</dependency>
<dependency>
<groupId>xom</groupId>
<artifactId>xom</artifactId>
<version>1.2.5</version>
</dependency>
But when I want to route a file to elasticsearch, I've got the following error:
java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [5.0.0]
I found that this exception is due to a node or a TransportClient using an old version. So I try to add the elasticsearch dependency :
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>5.1.2</version>
</dependency>
But it gives me a new error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/elasticsearch/action/WriteConsistencyLevel
So I'm wondering.. Which version of ES can I use with Apache Camel?
The code for trying to send data to elasticsearch:
XmlJsonDataFormat xmlJsonFormat = new XmlJsonDataFormat();
from("file://C:/Projects/?fileName=data.xml&charset=utf-8")
.marshal(xmlJsonFormat)
.to("elasticsearch://clusterES?transportAddresses=127.0.0.1:9300&operation=BULK_INDEX&indexName=xml&indexType=account");
I don't think you need to add any other pom except the camel-elasticsearch. It seems more likely that you have a TransportClient running on an older version. You need to find it and upgrade the TransportClient.
https://www.elastic.co/guide/en/elasticsearch/guide/current/_transport_client_versus_node_client.html
https://discuss.elastic.co/t/received-message-from-unsupported-version-2-0-0-minimal-compatible-version-is-5-0-0/64708
I'm seeing the following error :
java.lang.NoClassDefFoundError: com/google/common/util/concurrent/FutureFallback
when the cluster.connect() is called :
String hosts = CassandraClientUtil.getHost();
String localDC = CassandraClientUtil.getLocalDC();
Cluster cluster = null;
if (StringUtils.isNotEmpty(localDC))
{
cluster = Cluster.builder().addContactPoints(hosts.split(","))
.withCredentials(CassandraCopsComponentLogger.USER_NAME, CassandraCopsComponentLogger.AUTH_CODE)
.withQueryOptions(new QueryOptions().setConsistencyLevel(ConsistencyLevel.LOCAL_ONE))
.withLoadBalancingPolicy(new TokenAwarePolicy(DCAwareRoundRobinPolicy.builder().withLocalDc(localDC).build())).build();
}
else
{
cluster = Cluster.builder().addContactPoints(hosts.split(","))
.withCredentials(CassandraCopsComponentLogger.USER_NAME, CassandraCopsComponentLogger.AUTH_CODE)
.withQueryOptions(new QueryOptions().setConsistencyLevel(ConsistencyLevel.LOCAL_ONE)).build();
}
Session session = cluster.connect();
CassandraCopsComponentLogger.mappingManager = new MappingManager(session);
The pom.xml has the following dependencies :
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>16.0.1</version>
</dependency>
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>2.1.9</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
<version>3.9.0.Final</version>
</dependency>
<dependency>
<groupId>com.codahale.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-mapping</artifactId>
<version>2.1.9</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.3.1</version>
</dependency>
</dependencies>
I saw a post on stackoverflow here
where they recommended to upgrade the guava version to 16.0.1 but that did not help me solve my problem. Some directions from here will be really helpful as I'm new to cassandra. To add more background this thing works as a standalone project, when I include this project as a maven dependency to some other project it raises this runtime error.
com.google.common.util.concurrent.FutureFallback is deprecated in Guava 19.0 and removed since Guava 20.0.
Use Guava 19.0 and do not use Guava 20.0 or greater, until you upgrade the Cassandra driver.
I updated the Cassandra driver version to latest available and it should fix the issue.
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>3.5.0</version>
</dependency>
Don't add external guava version . whatever datastax-cassandra-core using only you can put that version . otherwise don't need of that .
If anyone like me didn't know, that there is a new version (4.x) out there with a new and different group id, take a look at the quickstart. This new version still uses Guava however it's shaded.
The driver now requires Java 8. It does not depend on Guava anymore (we still use it internally but it's shaded).
More information can be found in the upgrade guide.
What is the standard pom for adding the 1.2.1 JSTL taglib in a maven project. Any recommendations as to when/if can this be scoped as provided ? Any server peculiarities (interested in Jboss 7, Glassfish 4 and/or Tomcat 7)
EDIT: Added:
<dependency>
<groupId>org.glassfish.web</groupId>
<artifactId>javax.servlet.jsp.jstl</artifactId>
<version>1.2.1</version>
<scope>provided</scope>
</dependency>
This adds:
Notice it transitively adds the 1.2 api.
I am using provided as I am using Jboss which should provide it: Basic question complicated solution - Tomcat to JBoss. Still this is the 1.2 api apparently
$ find . -name *jstl*.jar
./modules/javax/servlet/jstl/api/main/jboss-jstl-api_1.2_spec-1.0.2.Final.jar
(contains implementation also). So would the correct way be to add the jstl jars to the pom (not in provided scope) and mark the servlet-api (I'm on 3 anyway) as provided somehow ?
Ended up using :
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>provided</scope>
</dependency>
<!-- Add the jstl-api dependency explicitly - otherwise jstl-api 1.2 is added -->
<dependency>
<groupId>javax.servlet.jsp.jstl</groupId>
<artifactId>javax.servlet.jsp.jstl-api</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.glassfish.web</groupId>
<artifactId>javax.servlet.jsp.jstl</artifactId>
<version>1.2.1</version>
<exclusions>
<!-- jstl-api was adding selvlet-api 2.5 and jsp-api-->
<exclusion>
<artifactId>jstl-api</artifactId>
<groupId>javax.servlet.jsp.jstl</groupId>
</exclusion>
</exclusions>
</dependency>
Not sure if I should add the jstl-api dependency - please comment (I would accept a more authoritative answer actually). Result:
See:
Autocomplete with JSP, JSTL 1.2.1 and Servlet 3.0 in Tomcat 7 with Eclipse Kepler
Fix maven JSTL 1.2.1 dependency so maven-war-plugin doesn't package JARs that offend Tomcat 7
JBoss 7.1 - Migration to JSF 2.2 - includes how to upgrate your jstl to 1.2.1
For 1.2:
Running JSTL 1.2 on Tomcat 7 using Maven