I use hbase client Scan to get data from remote hbase server cluster. When I set a Filter to the scan, the client will throw a exception:
org.apache.hadoop.ipc:RemoteException:IPC server unable to read call parameters:
Could not init org.apache.hadoop.hbase.util.Classes.
The server side's hbase log:
java.lang.NoClassDefFoundError:Could not initial class org.apache.hadoop.hbase.util.Classes
at org.apache.hadoop.hbase.client.Scan.readFields(Scan.java:590)
But it works well without a Filter. By the way, the filter is NOT a custom Filter.
My hbase version is 0.94.10 and hadoop is 1.2.1. I have copy hadoop-core.jar to the lib directory under hbase.
Don't forget to check JDK version. In IntelliJ DB for example I was having this issue because I had both JDK 15 and JDK 8 installed and the phoenix version I was using was intended to Java 8 so once I changed the JVM it worked.
In IntelliJ: You can change the Driver VM by going to Data Sources and Drivers -> NameYourCustomDriver -> Advanced (Tab) -> VM home path (field)
See this image example for reference: IntelliJ example
The org.apache.hadoop.hbase.util.Classes puts some initial code in its static block. So it will be initialized only once. When it initialize something the first time, if RunTimeException is thrown, it will not initialize anymore unless you restart your hbase cluster. In the initial block, it will create directory, and if it fails to create the directory, runtimeException will be thrown.
During initialization of org.apache.hadoop.hbase.util.Classes, if the configured value for "hbase.local.dir" doesn't exist, a runtime exception will be thrown and org.apache.hadoop.hbase.util.Classes will fail to initialize. That runtime exception causes a ClassDefNotFoundException to be thrown which is what is eventually reported in the logs.
Make sure these exist and are writeable by HBase:
hbase.local.dir (default: ${hbase.tmp.dir}/local)
hbase.tmp.dir (default: ${java.io.tmpdir}/hbase-${user.name})
Related
I'm using H2O init to specify the Snowflake JDBC driver as extra_classpath when connecting to external H2O instance however, getting the following error (H2O connects successfully to external instance), when attempting to access Snowflake DB:
H2OServerError: HTTP 500 Server Error:
Server error java.lang.RuntimeException:
Error: SQLException: No suitable driver found for jdbc:snowflake:..
It works fine when starting a standalone H2O instance with nothing else changed.
Here is the init code:
h2o.init(ip='<ip>',
port=54321,
username='**',
password='**',
extra_classpath = ["snowflake-jdbc-3.8.0.jar"])
H2O version: 3.22.1.1
Python 3
extra_classpath is for use when starting H2O from Python. When you are connecting to H2O that is running on another machine, it has to already be started. So it is up to whoever started it, to have given that extra classpath, as part of the java command, when starting it. (And if a cluster, you have to make sure every node of the cluster uses the exact same command.)
The snowflake jar has to be available, on the path you give, on the server. In fact, there is no need for it to be on the client, unless you are also using it directly from your Python script (i.e. outside of h2o).
BTW, see https://github.com/h2oai/h2o-3/blob/master/h2o-py/h2o/h2o.py#L147 for the code. If you search for uses of extra_classpath you see it is only used when starting the local server.
I'm intermittently getting the error message
DAG did not succeed due to VERTEX_FAILURE.
when running Hive queries via PyHive. Hive is running on an EMR cluster where hive.vectorized.execution.enabled is set to false in the hive-site.xml file for this reason.
I can set the above property through the configuration on the Hive connection and my query has run successfully every time I've executed it, however I want to confirm that this has fixed the issue and that it is definitely the case that hive-site.xml is being ignored.
Can anyone confirm if this is the expected behavior, or alternatively is there any way to inspect the Hive configuration via PyHive as I've not been able to find any way of doing this?
Thanks!
PyHive is a thin client that connects to HiveServer2, just like a Java or C client (via JDBC or ODBC). It does not use any Hadoop configuration files on your local machine. The HS2 session starts with whatever properties are set server-side.
Same goes for ImPyla BTW.
So it's your responsibility to set custom session properties from your Python code, e.g. execute this statement...
SET hive.vectorized.execution.enabled =False
... before running your SELECT.
I have a existing jmeter Master/Slave configuration which works fine. The existing jmeter is 3.0
I installed 3.3 version in the Master server and added the necessary library jar which enabled me to open the existing jmx file.
I copied the exisitng jmeter properties file to the new directory as well. But when I try to run that, I am getting the below error message
Error in rconfigure() method java.rmi.MarshalException: error marshalling arguments; nested exception is:
java.net.SocketException: Software caused connection abort: socket write error
Should I need to install my slave server with 3.3 as well? I tried upgrading the library files, but the jmeter version displayed remains 3.0 even after that. Hence not sure if that is the right way to upgrade it.
Yes, using the same JMeter version is amust in Remote Testing step 0:
Step 0: Configure the nodes
Make sure that all the nodes (client and servers) :
are running exactly the same version of JMeter.
are using the same version of Java on all systems. Using different versions of Java may work but is discouraged
You should use the download version in all computers there is no upgrade option in JMeter
I encountered an error while adding a new service (Service Type = HDFS) using Cloudera Manager (Free Edition). The error message says as follows:
Could not create process: com.cloudera.cmf.service.config.ConfigFileSpec$GenerateException: Unable to process template:couldn't find the template hadoop/topology.py.vm
I checked /var/log/cloudera-scm-server/cloudera-scm-server.log and found a line like below.
org.apache.velocity.exception.ResourceNotFoundException: Unable to find resource '/WEB-INF/templates/hadoop/topology.py.vm'
I guess that a certain war file does not contain hadoop-metrics.properties.vm (Velocity template file?) although it should do and that this might be related to WHIRR-370.
Could you help me to solve this problem, please?
May I ask which version of Cloudera Manager is being used? Does this error occurred just after you try to add add the service of after some time when the service is added?
Based on the error, it seems some of the configuration is missing that why service addition failed. So I would like to know how did you install Hadoop on this cluster?
If you download the virtual machine and compare from your installation, you can compare the folder for completeness and missing content. It does work for me always.
I am having some problems with an Hibernate Criteria query causing a outOfIndex error at driver level, I am pretty sure the problem is at the driver as I have being debugging and everything seems ok, but to be sure and be able to report the error I need to enable traces and I am not being able to do so.
I have added ojsbc5_g.jar as a new JDBC provider and created a new data source with it, renamed the jndi of the old one so the new debug data source uses the same and tried using -Doracle.jdbc.Trace=true -Djava.util.logging.config.file=ConfigFile.properties
at the server JVM configuration (using a absolute path for the file and a relative one).
The data source and the app works but no log appears, then I found the data source custom properties so I modified the level and the filename and now the file is there but empty.
The JVM Configuration should include:
-Doracle.jdbc.Trace=true -Djava.util.logging.config.file=/jdbc/OracleLog.properties
and the mentioned properties file should include something like:
.level=SEVERE
oracle.jdbc.level=ALL
oracle.jdbc.handlers=java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level=INFO
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
You can set the logging for following targets depending on the nature of your problem:
oracle.jdbc
oracle.jdbc.driver
oracle.jdbc.pool
oracle.jdbc.rowset
oracle.jdbc.xa
oracle.sql