we have ambari cluster version - 2.5.0.3 ( with 3 masters nodes and 3 workers nodes )
we are in the middle of upgrade from HDP version 2.6.0.3 to 2.6.4 version
in the final stage installation failed
the errors that apears under - Finalize Upgrade Pre-Check are
The following components were found to have version mismatches. Finalize will not complete successfully:
master03.sys5dns.com: HDFS/ZKFC reports UNKNOWN
master01.sys5dns.com: HDFS/ZKFC reports UNKNOWN
also that:
{"hosts":["master01.sys5dns.com","master03.sys5dns.com"],"host_detail":{"master01.sys5dns.com":[{"service":"HDFS","component":"ZKFC","version":"UNKNOWN"}],"master03.sys5dns.com":[{"service":"HDFS","component":"ZKFC","version":"UNKNOWN"}]}}
we are stuck in this stage
please advice what is the meaning of - HDFS/ZKFC reports UNKNOWN ?
Seems to have been fixed in https://issues.apache.org/jira/browse/AMBARI-21526
Looks like ZKFC started advertising a version between HDP-2.3 and HDP-2.4. The suggestion would be to rollback to backup, upgrade Ambari to at least 2.5.2 and then upgrade stack
Or you may try looking up for a less drastic solution at https://community.hortonworks.com/index.html
Related
When I try us api-platform version 2.6.4 I am not able to run it when i build adn strat containers and check logs caddy is not working i get an error like this. Any idea? Caddy version is 2.3.0
caddy_1 | panic: proto: file "pb.proto" is already registered
caddy_1 | See https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
tureality_caddy_1 exited with code 2
Other people have reported having this bug and I had it too.
Fortunately, the bug as just been fixed by Dunglas itself. :)
https://github.com/api-platform/api-platform/issues/1881#issuecomment-822663193
The repair was done at the mercure level and not in the api platform source code itself so you can keep your current version.
You just have to docker-compose up and it will work.
I find similar error on https://issues.apache.org/jira/browse/KYLIN-2511
env:
hadoop-2.7.1
hbase-1.3.2
apache-hive-2.1.1-bin
apache-kylin-1.6.0-hbase1.x-bin
I've tried copy all the hive libs to kylin, but get another ERROR.
org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/typeinfo/TypeInfo
The missing class should be in hive-exec-.jar; Check and debug the "bin/find-hive-dependency.sh" to see why it wasn't able to locate this jar from your server. You can manually add it to the "hive_exec_path" variable.
BTW, Kylin 1.6 is quite old, try to upgrade to a 2.x version.
Why you just try the method mentioned in https://issues.apache.org/jira/browse/KYLIN-2511. You'd better prepare the env according to the document of v16. It is better for using the latest version of Kylin. It has more feature and fixes some bugs.
I've just setup a new hadoop 3.0 cluster with Hive 2.3.2 and Spark 2.3. When I want to run some queries on Hive tables, getting following error.
I know there were some bugs in Hive, but seems like it was fixed for 2.1.1, but not sure what's the situation with 2.3.2 version. Do you have any idea if that could be handled somehow?
Thanks
Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_151)
Type in expressions to have them evaluated.
Type :help for more information.
scala> import spark.sql
import spark.sql
scala> sql("show databases")
java.lang.NoSuchFieldError: HIVE_STATS_JDBC_TIMEOUT
at org.apache.spark.sql.hive.HiveUtils$.formatTimeVarsForHiveClient(HiveUtils.scala:205)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:286)
at org.apache.spark.sql.hive.HiveExternalCatalog.client$lzycompute(HiveExternalCatalog.scala:66)
at org.apache.spark.sql.hive.HiveExternalCatalog.client(HiveExternalCatalog.scala:65)
at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply$mcZ$sp(HiveExternalCatalog.scala:195)
at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$databaseExists$1.apply(HiveExternalCatalog.scala:195)
at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:97)
at org.apache.spark.sql.hive.HiveExternalCatalog.databaseExists(HiveExternalCatalog.scala:194)
at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)
at org.apache.spark.sql.internal.SharedState.externalCatalog(SharedState.scala:102)
at org.apache.spark.sql.hive.HiveSessionStateBuilder.externalCatalog(HiveSessionStateBuilder.scala:39)
at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog$lzycompute(HiveSessionStateBuilder.scala:54)
at org.apache.spark.sql.hive.HiveSessionStateBuilder.catalog(HiveSessionStateBuilder.scala:52)
at org.apache.spark.sql.hive.HiveSessionStateBuilder$$anon$1.<init>(HiveSessionStateBuilder.scala:69)
at org.apache.spark.sql.hive.HiveSessionStateBuilder.analyzer(HiveSessionStateBuilder.scala:69)
at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$2.apply(BaseSessionStateBuilder.scala:293)
at org.apache.spark.sql.internal.SessionState.analyzer$lzycompute(SessionState.scala:79)
at org.apache.spark.sql.internal.SessionState.analyzer(SessionState.scala:79)
at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57)
at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55)
at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:74)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:638)
... 49 elided
I am running spar 2.3 with Hive 2.3.2 and encounter similar issue.
The fix you mentioned is for Hive 2.1 as can be seen from the Spark Jira following：
https://issues.apache.org/jira/browse/SPARK-13446
You can see from the latest comment that people are getting exactly same error as yours.
Also, as this so question answered, the current Hive version supported by Spark is 2.1
Following the official guide of Titan DB here, and trying to run the command:
graph = TitanFactory.open('conf/titan-cassandra-es.properties')
I got this error:
Backend shorthand unknown: conf/titan-cassandra-es.properties
Obviously, the reason is the incorrect path to the
titan-cassandra-es.properties
file. So I changed it to:
graph = TitanFactory.open('../conf/titan-cassandra-es.properties')
and got this error:
Encountered unregistered class ID: 141.
The error happens in the following version:
titan-0.5.4-hadoop2
On titan-1.0.0-hadoop2 instead of this error message I get this one:
Invalid import definition: 'com.thinkaurelius.titan.hadoop.MapReduceIndexManagement'; reason: startup failed: script14747941661821834264593.groovy: 1: unable to resolve class com.thinkaurelius.titan.hadoop.MapReduceIndexManagement # line 1, column 1. import com.thinkaurelius.titan.hadoop.MapReduceIndexManagement ^
1 error
And on titan-1.0.0-hadoop2 I get this one:
The input line is too long.
The syntax of the command is incorrect.
Does anyone know how to handle this issue?
It seems like you have not even managed to get Titan 1 to start up yet.
I do not believe Titan 1 has been deployed to support Windows out of the box. I.e. the downloadable package will not just work with windows.
Saying that I have managed to get Titan DB 1 to work on windows. To do so, all you have to do is install Cassandra 2.x on Windows. This guide may help you out. Start cassandra and enable thrift connections.
With that done you should be able to get Titan doing basic operations on windows. From there you may find dealing with you current errors easier.
Side Note: Windows support for Titan 0.5.x may be more substantial. So you could look into that as well.
We are currently using the 2.0.0-M06 snapshot version of the neo4j jdbc driver and are trying to use the latest version available. We found the 2.1.4 version on the maven repository below,
https://m2.neo4j.org/content/repositories/releases/org/neo4j/neo4j-jdbc/
However, while trying to use this we see the below error..
Caused by: java.lang.IllegalStateException: Error during parsing
at org.neo4j.jdbc.rest.StreamingParser$ParserState.nextToken(StreamingParser.java:71)
at org.neo4j.jdbc.rest.StreamingParser.skipTo(StreamingParser.java:313)
at org.neo4j.jdbc.rest.StreamingParser.nextResult(StreamingParser.java:130)
at org.neo4j.jdbc.rest.StreamingParser$2.hasNext(StreamingParser.java:265)
at org.neo4j.jdbc.rest.StreamingParser$2$1.endReached(StreamingParser.java:269)
at org.neo4j.jdbc.rest.StreamingParser$1.hasNext(StreamingParser.java:201)
at org.neo4j.jdbc.IteratorResultSet.hasNext(IteratorResultSet.java:98)
at org.neo4j.jdbc.IteratorResultSet.next(IteratorResultSet.java:63)
at com.mchange.v2.c3p0.impl.NewProxyResultSet.next(NewProxyResultSet.java:2859)
... 92 more
Caused by: java.io.IOException: Stream closed
at sun.nio.cs.StreamDecoder.ensureOpen(StreamDecoder.java:46)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:148)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at org.codehaus.jackson.impl.ReaderBasedParser.loadMore(ReaderBasedParser.java:117)
at org.codehaus.jackson.impl.ReaderBasedParser._skipWSOrEnd(ReaderBasedParser.java:1476)
at org.codehaus.jackson.impl.ReaderBasedParser.nextToken(ReaderBasedParser.java:368)
at org.neo4j.jdbc.rest.StreamingParser$ParserState.nextToken(StreamingParser.java:67)
... 100 more
We found a reference that this is addressed in the 2.2 version of the driver and are therefore trying to download that. Can someone please point us in the right direction in getting this 2.2 binary for the neo4j-jdbc driver? Also, we currently use the neo4j 2.2 version for our db server.
Thx,
NN
I think version 2.2 is not released yet.
You can try to build your own binaries from the source code - https://github.com/neo4j-contrib/neo4j-jdbc