When I use the spatial index module in ignite1.6.0 , I found it depends 1.3.175 version of the H2, but I need to use the 1.4.X h2 version.
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.3.175</version>
<scope>compile</scope>
</dependency>
This method org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing # start will call org.h2.constant.SysProperties and org.h2.util.Utils, in front of the class in the 1.3.176 version of the above have been It does not exist, the latter class is missing serializer variables.
if (SysProperties.serializeJavaObject) {
U.warn(log, "Serialization of Java objects in H2 was enabled.");
SysProperties.serializeJavaObject = false;
}
if (Utils.serializer != null)
U.warn(log, "Custom H2 serialization is already configured, will override.");
Utils.serializer = h2Serializer();
Is there any way to solve it?
Ignite depends on H2 1.3.175 and you can't use any other version. If you already have some code that depends on 1.4, you should isolate Ignite-related code in a separate module in your project. This way different versions of H2 will coextist.
Related
I am trying to serialize/deserialize org.locationtech.jts.geom.Geometry(version 1.16.0) using com.bedatadriven.jackson.datatype.jts.JtsModule. It comes packaged with:
<dependency>
<groupId>com.graphhopper.external</groupId>
<artifactId>jackson-datatype-jts</artifactId>
<version>1.0-2.7</version>
</dependency>
I have registered it using spring configuration mechanism:
#Bean
public JtsModule jtsModule()
{
return new JtsModule();
}
But somehow com.bedatadriven.jackson.datatype.jts.serialization.GeometryDeserializer is not invoked.
Is there something else needed to be done?
GeometryDeserializer / GeometrySerializer worked for com.vividsolutions.jts.geom.Geometry
An issue is open about it here
You can create your custom serializer and deserializer for org.locationtech.jts.geom.Geometry using same code available.
Example for point
There is a PR for Upgrade to Locationtech jts but not merged yet
I tried to extend scaffolded quarkus demo, https://code.quarkus.io/, with polyglot code for GraalVM:
#GET
#Produces(MediaType.TEXT_PLAIN)
public String hello() {
String out = "From JS:";
try (Context context = Context.create()) {
Value function = context.eval("js", "x => x+1");
assert function.canExecute();
int x = function.execute(41).asInt();
out=out+x;
System.out.println(out);
}
return "hello";
}
I added dependencies to pom.xml as suggested here [https://stackoverflow.com/questions/54384499/illegalstateexception-no-language-and-polyglot-implementation-was-found-on-the]
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js</artifactId>
<version>20.1.0</version>
</dependency>
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js-scriptengine</artifactId>
<version>20.1.0</version>
</dependency>
<dependency>
<groupId>org.graalvm.truffle</groupId>
<artifactId>truffle-api</artifactId>
<version>20.1.0</version>
</dependency>
But when I run on cmd line
./mvnw clean package
Test fails with exception, which I do not understand.
2020-06-22 19:26:56,328 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler]
(executor-thread-1) HTTP Request to /hello failed, error id: 996b0479-d836-47a5-bbcb-67bd876f9277-1: org.jboss.resteasy.spi.UnhandledException:
java.lang.IllegalAccessError: superclass access check failed:
class com.oracle.truffle.polyglot.PolyglotImpl (in unnamed module #0x7bf61ba2) cannot access class org.graalvm.polyglot.impl.AbstractPolyglotImpl (in module org.graalvm.sdk)
because module org.graalvm.sdk does not export org.graalvm.polyglot.impl to unnamed module #0x7bf61ba2
UPDATE:
It looks like regression in quarkus, https://github.com/quarkusio/quarkus/issues/10226.
App test is passing when used with quarkus 1.2.1 (instead of 1.5.2).
Look into the mvn dependency:tree -- it turns out that org.graalvm.js:js:20.1.0 depends on org.graalvm.sdk:graal-sdk:19.3.1. I'd personally call that GraalVM JS bug.
If you add an explicit dependency on org.graalvm.sdk:graal-sdk:20.1.0, it should work.
(At least it did for me, but I was getting a different error than you, so not sure.)
EDIT: as I was warned about in the comment, it is not true that org.graalvm.js:js:20.1.0 depends on org.graalvm.sdk:graal-sdk:19.3.1. Instead, there must be something else that forces graal-sdk to 19.3.1, perhaps something from Quarkus. Explicitly managing it to 20.1.0 should still help.
are you maybe executing that on GraalVM 19.3.1? That is known to confuse the system. Our strong suggestion is to EITHER run on a GraalVM (which automatically includes a proper version of Graal.js, no further input needed), OR run on a Stock JDK and import the respective JARs from Maven. If you import (a different version of) our Jars from maven on a GraalVM, then you might run into conflicts like that.
I've implemented the code for scoring on a provided PMML file and a csv data file (Linear Regression) using Spark and Java. For this I've used jpmml-evaluator-spark and spark-mllib_2.11 maven artifacts, and it works fine.
Now, I'm looking at replacing jpmml-evaluator-spark library, which is AGPL licensed, to something similar may be bundled within org-apache-spark (or any other fully open source option)
I don't see Evaluators for scoring on a PMML available in org.apache.spark group of dependencies. Please confirm if this is correct and suggest some alternative.
https://github.com/jpmml/jpmml-evaluator-spark
This is the PMML evaluator library for the Apache Spark cluster computing system (http://spark.apache.org/) and is AGPL.
Also refer to: http://spark.apache.org/docs/latest/ml-guide.html
These suggest that whatever is packaged along with apache spark includes algorithms and model creation and training, but scoring on the model is not available here & has its dependencies included in the jpmml-evaluator-spark only.
import org.apache.spark.ml.Transformer;
import org.apache.spark.sql.Dataset;
import org.jpmml.evaluator.Evaluator;
import org.jpmml.evaluator.EvaluatorBuilder;
import org.jpmml.evaluator.LoadingModelEvaluatorBuilder;
import org.jpmml.evaluator.spark.TransformerBuilder;
...
...
...
EvaluatorBuilder evaluatorBuilder = new LoadingModelEvaluatorBuilder().setLocatable(false)
.setVisitors(new DefaultVisitorBattery()).load(pmmlInputStream);
Evaluator evaluator = evaluatorBuilder.build();
evaluator.verify();
TransformerBuilder pmmlTransformerBuilder = new TransformerBuilder(evaluator).withLabelCol("Predicted_SpeciesCategory").exploded(true);
Transformer pmmlTransformer = pmmlTransformerBuilder.build();
Dataset<?> resultDataset = pmmlTransformer.transform(csvDataset);
...
...
Maven dependencies:
<dependency>
<groupId>org.jpmml</groupId>
<artifactId>jpmml-evaluator-spark</artifactId>
<version>1.2.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.3</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.4.3</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.jpmml</groupId>
<artifactId>jpmml-sparkml</artifactId>
<version>1.5.4</version>
</dependency>
This code still has dependency on org.jpmml library, which I wish to remove. Looking for an alternative using org.apache.spark library to achieve similar results.
You could use the PMML4S-Spark to evaluate a PMML model against Spark, for example:
import org.pmml4s.spark.ScoreModel
val model = ScoreModel.fromInputStream(pmmlInputStream)
val resultDataset = model.transform(csvDataset)
If you want to use PMML4S-Spark in Java, it's also easy to use and similar as Scala, for example:
import org.pmml4s.spark.ScoreModel;
import org.apache.spark.sql.Dataset;
ScoreModel model = ScoreModel.fromInputStream(pmmlInputStream);
Dataset<?> resultDataset = model.transform(csvDataset);
BTW, PMML4S-Spark's license is APL 2.0.
My answer might be totally irrelevant to your question, but since I faced an issue and was coming to this question again and again - I don't want others to face it. I reached to the solution some how by going through many stackoverflows...
The problem
I was using spark in java using the old dependency maven code:
<dependency>
<groupId>org.jpmml</groupId>
<artifactId>pmml-evaluator-metro</artifactId>
<version>1.6.3</version>
</dependency>
Which I thought is perfect and will work. But that was not recognizing TransformerBuilder as one of its libraries.
The dependency code given below should solve your problem if your problem is related to TransformerBuilder:
<dependency>
<groupId>org.jpmml</groupId>
<artifactId>jpmml-evaluator-spark</artifactId>
<version>1.3.0</version>
</dependency>
That was it. You're welcome in advance 😉
I am trying to add some feature to my open daylight project (e.g. l2switch, dlux, rest,...).
I used to edit the features.xml and the pom.xml for add there features in Carbon release. I am currently using Nitrogen release, when adding these dependencies in my features pom.xml file, I am still unable to detect the features when I login to my karaf (using feature:install/list).
<dependency>
<groupId>org.opendaylight.netconf</groupId>
<artifactId>features-restconf</artifactId>
<classifier>features</classifier>
<version>${restconf.version}</version>
<type>xml</type>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.opendaylight.dluxapps</groupId>
<artifactId>features-dluxapps</artifactId>
<classifier>features</classifier>
<version>${dluxapps.version}</version>
<type>xml</type>
<scope>runtime</scope>
</dependency>
am I missing something else? when I try to add repositories,as I previously did in carbon-release. The feature.xml it automatically re-generated and all my editing is removed.
I am using Nitrogen release by defining and -DarchetypeVersion=1.4.0 when generating my maven artifact.
See the upstream configuration management tooling for running-code examples being used constantly in downstreams like OPNFV.
# Configuration of Karaf features to install
file { 'org.apache.karaf.features.cfg':
ensure => file,
path => '/opt/opendaylight/etc/org.apache.karaf.features.cfg',
# Set user:group owners
owner => 'odl',
group => 'odl',
}
$features_csv = join($opendaylight::features, ',')
file_line { 'featuresBoot':
path => '/opt/opendaylight/etc/org.apache.karaf.features.cfg',
line => "featuresBoot=${features_csv}",
match => '^featuresBoot=.*$',
}
puppet-opendaylight, manifests/config.pp, stable/nitrogen
So basically you shouldn't be editing the XML directly, you should edit the configuration that generates the XML. I'm surprised that worked in Carbon.
I recommend directly using upstream configuration management tooling, like puppet-opendaylight or ansible-opendaylight, vs trying to figure out the configuration knobs yourself, duplicating effort. If you're doing a more complex deployment, look at the OPNFV installer scenarios (that build on these ODL tools) vs trying to solve that very hard problem yourself.
I want to use flink-jdbc to get data from mysql。
I have seen an example on Apache flink website
// Read data from a relational database using the JDBC input format
DataSet<Tuple2<String, Integer> dbData =
env.createInput(
JDBCInputFormat.buildJDBCInputFormat()
.setDrivername("org.apache.derby.jdbc.EmbeddedDriver")
.setDBUrl("jdbc:derby:memory:persons")
.setQuery("select name, age from persons")
.setRowTypeInfo(new RowTypeInfo(BasicTypeInfo.STRING_TYPE_INFO, BasicTypeInfo.INT_TYPE_INFO))
.finish()
);
But when i try to write a demo， i can't find the method 'setRowTypeInfo'.
It was like this
import org.apache.flink.api.common.typeinfo.BasicTypeInfo
import org.apache.flink.api.java.ExecutionEnvironment
import org.apache.flink.api.java.io.jdbc.JDBCInputFormat
import org.apache.flink.api.scala._
/**
* Created by lulijun on 17/7/7.
*/
object FlinkJDBC {
def main(args:Array[String]): Unit = {
val env = ExecutionEnvironment.createLocalEnvironment()
val dbData = env.createInput(
JDBCInputFormat.buildJDBCInputFormat
.setDrivername("com.mysql.jdbc.Driver")
.setDBUrl("XXX")
.setUsername("xxx")
.setPassword("XXX")
.setQuery("select name, age from persons")
.setRowTypeInfo(new Nothing(BasicTypeInfo.STRING_TYPE_INFO, BasicTypeInfo.INT_TYPE_INFO))
.finish)
dbData.print()
env.execute()
}
}
The "setRowTypeInfo" method is always red, and the IDEA prompts
"cannot resolve symbol setRowTypeInfo"
The jar version of flink-jdbc i used is 1.0.0.
<dependencies>
<!-- Use this dependency if you are using the DataSet API -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-scala_2.10</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.10</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-jdbc</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.36</version>
</dependency>
</dependencies>
I have searched a lot, and most of the people use the method exactly like the official document, but on one mentioned this problem.
I doubt whether I used the wrong version of flink-jdbc, but I cannot get any information about the right way to use flink-jdbc.
If you know the problem, please teach me.Thank you.
I changed the flink-jdbc version from 1.0.0 to 1.3.0 and the problem solved.
But when I search flink-jdbc on maven websit
https://mvnrepository.com/search?q=flink-jdbc, I can't get the right information in the first few pages, It makes me thought the version of flink-jdbc do not need to be matched with other flink jars.
But the truth is flink-jdbc/1.1.3 use class RowTypeInfo of package api.table, but flink-jdbc/1.3.0 use class RowTypeInfo of package api.java.They have close ties with each other.
We must make sure the version is matched.