I'm running Kafka Connect (Confluent) in distributed mode, and have recently added the db.timezone property to my JDBC Source Connector. After doing so I'm seeing this error when I load the connector:
java.lang.NoClassDefFoundError: Could not initialize class java.time.zone.ZoneRulesProvider\n\tat java.time.ZoneRegion.ofId(ZoneRegion.java:120)\n\tat java.time.ZoneId.of(ZoneId.java:411)
This is happening from this code in JDBCSourceConnectorConfig:
https://github.com/confluentinc/kafka-connect-jdbc/blob/master/src/main/java/io/confluent/connect/jdbc/source/JdbcSourceConnectorConfig.java#L807
If I log into my Kafka Connect box and run java -version I get:
openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-b10)
OpenJDK 64-Bit Server VM (build 25.262-b10, mixed mode)
If I create a small Java Program like the following and run it on that box, it works fine:
import java.time.ZoneId;
import java.time.zone.ZoneRulesProvider;
import java.util.TimeZone;
public class TestTime {
public static void main(String[] args) {
String dbTimeZone = "America/New_York";
System.out.println(TimeZone.getTimeZone(ZoneId.of(dbTimeZone)));
System.out.println(ZoneRulesProvider.getAvailableZoneIds());
}
}
So why is confluent/kafka connect breaking on it? Why would it not be able to find ZoneRulesProvider?
After doing a restart of kafka connect on my servers this issue appears to have gone away.
Related
Similar to https://answers.sap.com/questions/12675326/sap-dbtech-jdbc-708-receive-of-connect-fail.html
Connection to SAP Hana Cloud fails with the error "RTEException: -708 Receive of connect failed."
Steps to reproduce:
java -version
OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.20.04.1)
java -jar ngdbc.jar -V
package package com.sap.db.jdbc, Java Platform API Specification, version 1.4, SAP HANA JDBC Driver, SAP SE, 1.120.05-8c23c50e159e9883edab0e2ebdd4e02c5919cd08
java -jar ngdbc.jar -u DBADMIN,PASSWORD -n BIG-IDENTIFIER.hana.trial-us10.hanacloud.ondemand.com:443 -d test -o encrypt=true -o validatecertificate=false
(the certificate is imported, and the error occurs even without the parameters encrypt and validatecertificate)
Contents of trace log:
ClassLoader: jdk.internal.loader.ClassLoaders$AppClassLoader#55054057
Process-ID: 320850
package package com.sap.db.jdbc, Java Platform API Specification, version 1.4, SAP HANA JDBC Driver, SAP SE, 1.120.05-8c23c50e159e9883edab0e2ebdd4e02c5919cd08 on Java 11.0.15
---- Thread 1eb44e46 main Timestamp: 2022-09-29 09:50:03.162
new Connection 'jdbc:sap://BIG-IDENTIFIER.hana.trial-us10.hanacloud.ondemand.com:443'
password=****
databaseName=test
host=BIG-IDENTIFIER....
options=
cmd=Select top 1 42 as "connect test" fro...
user=DBADMIN
HOSTLIST: [BIG-IDENTIFIER.hana.trial-us10.hanacloud.ondemand.com:443,]
new RTEException: -708 Receive of connect failed.
whereAmIjava.lang.Throwable
at com.sap.db.util.Tracer.whereAmI(Tracer.java:280)
at com.sap.db.rte.comm.RTEException.(RTEException.java:51)
at com.sap.db.rte.comm.BasicSocketComm.receiveInfoRequest(BasicSocketComm.java:587)
at com.sap.db.rte.comm.BasicSocketComm.doInfoRequest(BasicSocketComm.java:84)
at com.sap.db.rte.comm.BasicSocketComm.connectDB(BasicSocketComm.java:187)
at com.sap.db.rte.comm.SocketComm$1.open(SocketComm.java:47)
at com.sap.db.jdbc.topology.Topology.getSession(Topology.java:88)
at com.sap.db.jdbc.Driver.openByURL(Driver.java:1216)
at com.sap.db.jdbc.Driver.connect(Driver.java:313)
at com.sap.db.jdbc.Driver.main(Driver.java:858)
using null
=> FAILED
any thoughts on the why and how to solve it?
Since this is HANA Cloud Trial, I don't think, that you should use -d to specify a tenant database name. Please try to remove this parameter as you may be trying to connect to a non-existing tenant database. You can also refer to this tutorial.
Second thing to consider is the allowlist for IP addresses. Per default connections are blocked if you have not added your client ip to the allowlist. More details can be found in this blog.
Last but not least, please check if your are using the latest version of ngdb.jar. It can be obtained on this site.
I am able to connect to the cluster, browse its hive catalog, see tables/views and columns/datatypes
Running a simple select statement from a view on a parquet file produces this error and no other results:
SQL Error [500540] [HY000]: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 180. Background thread ID: 223. Error caught: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available.
Standard Databricks cluster:
Standard_DS3_v2
JDBC URL:
jdbc:databricks://<reducted>.1.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/<reducted>/<reducted>;AuthMech=3;UID=token;PWD=<reducted>
Advanced Options Spark Config:
spark.databricks.cluster.profile singleNode
spark.databricks.io.directoryCommit.createSuccessFile false
spark.master local[*, 4]
spark.driver.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true
spark.hadoop.fs.azure.account.key.<reducted>.blob.core.windows.net <reducted>
spark.executor.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true
parquet.enable.summary-metadata false
My local machine:
Dbeaver Version 22.1.2.202207091909
MacOS version (M1 chip): Monterey 12.4
Java version:
java --version
openjdk 18.0.1 2022-04-19
OpenJDK Runtime Environment Homebrew (build 18.0.1+0)
OpenJDK 64-Bit Server VM Homebrew (build 18.0.1+0, mixed mode, sharing)
I am able to do the following with no errors (Databricks default test dataset):
CREATE TABLE diamonds USING CSV OPTIONS (path "/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv", header "true");
When I run this select color from diamonds; or this select * from diamonds;
I get this:
SQL Error [500618] [HY000]: [Databricks][DatabricksJDBCDriver](500618) Error occured while deserializing arrow data: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available
Hence, any select query on any object (parquet file or anything else) causes the error described above.
What could be the problem? Any recommendations how to resolve this error? Why am I able to connect and see the metadata of the schemas/tables/views/columns, but not query or view the data?
P.S. I followed this guide exactly: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/dbeaver#step-3-connect-dbeaver-to-your-azure-databricks-databases
Am trying to use netty jars, as part of pushy library(https://github.com/relayrides/pushy) to send apple push notifications. It runs fine in my local tomcat. When I try to deploy the same on IBM WAS and start my server, it gives me the below exception.
com.ibm.ws.ecs.internal.scan.context.impl.ScannerContextImpl scanJAR unable to open input stream for resource io/netty/util/internal/shaded/org/jctools/queues/package-info.class in archive WEB-INF/lib/netty-common-4.1.16.Final.jar
Pls find below the versions am using.
WAS - 8.5.5
Java - 1.7
Netty - 4.1.16
Pls help me understand the cause for the issue and how to fix the same. Thanks.
Looking at "netty-common-4.1.16.Final.jar" from
https://mvnrepository.com/artifact/io.netty/netty-common/4.1.16.Final
I'm seeing this resource:
netty-common-4.1.16.Final.jar/io/netty/util/internal/shaded/org/jctools/queues/package-info.class
That has been compiled using a java8 compiler. That seems incorrect -- none of the other class resources of the JAR are compiled using java8.
Data (internal reporting format) for "package-info":
/netty-common-4.1.16.Final.jar/io/netty/util/internal/shaded/org/jctools/queues/package-info.class
interface synthetic io.netty.util.internal.shaded.org.jctools.queues.package-info
extends java.lang.Object
Version [ 0x34 0x00 ] ( J2SE 8 )
The same data, for example, for "Log4JLoggerFactory":
/netty-common-4.1.16.Final.jar/io/netty/util/internal/logging/Log4JLoggerFactory.class
public io.netty.util.internal.logging.Log4JLoggerFactory
extends io.netty.util.internal.logging.InternalLoggerFactory
Version [ 0x32 0x00 ] ( J2SE 6.0 )
[F] public static final INSTANCE : [ Lio/netty/util/internal/logging/InternalLoggerFactory; ]
[M] public deprecated <init> : [ ()V ] ( void )
#java.lang.Deprecated
[M] public newInstance : [ (Ljava/lang/String;)Lio/netty/util/internal/logging/InternalLogger; ]
[M] static <clinit> : [ ()V ] ( void )
Can you try rerunning with that one resource removed? (Or, rebuilt to not use java8?) IBM WebSphere won't process java8 classes unless at a high enough service level (8.5.5.9 and higher). Or, try on a higher service level of WebSphere.
I am getting the following error on Line #4 of code below in IBM Websphere
Liberty Profile 16.0.0
InputStream keystoreStream = EncryptionUtility.class.getResourceAsStream(keyStoreLocation);
KeyStore keystore = KeyStore.getInstance("JCEKS");
keystore.load(keystoreStream, storePass.toCharArray());
Key key = keystore.getKey(alias, keyPass.toCharArray());
Which results in the following exception:
Caused by: java.security.UnrecoverableKeyException: com.ibm.crypto.provider.AESSecretKey
at com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:358)
at com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:133)
at java.security.KeyStore.getKey(KeyStore.java:804)
at com.comdata.base.helper.EncryptionUtility.initSymmetricKey(EncryptionUtility.java:134)
Any ideas why this is happening? Is anything need to be configured for cryptography?
I poked through the code of keyProtector.java in JDK 7 and UnrecoverableKeyException is triggered by ClassNotFoundException
com.ibm.crypto.provider.AESSecretKey
Do we need to install any feature via installUtility?
Any ideas why this is happening? Is anything need to be configured for cryptography?
The class not being found (com.ibm.crypto.provider.AESSecretKey) is from the IBM JDK.
It looks like your keystore was created using the IBM JDK and thus has a key packaged in it that uses the AESSecretKey from the IBM JDK.
At runtime, your Liberty server is probably using a non-IBM JDK, which would not have this IBM JDK specific class in it.
Do we need to install any feature via installUtility?
Nope. The missing class should be provided by the JDK, as opposed to a Liberty feature.
I know that spring automatically expose JMX beans. I was able to access it locally using VisualVM.
However on prod how I can connect to remotely to the app using it's JMX beans? Is there a default port or should I define anything in addition?
Thanks,
ray.
By default JMX is automatically accessible locally, so running jconsole locally would detect all your local java apps without port exposure.
To access an app via JMX remotely you have to specify an RMI Registry port. The thing to know is that when connecting, JMX initializes on that port and then establishes a data connection back on a random high port, which is a huge problem if you have a firewall in the middle. ("Hey sysadmins, just open up everything, mkay?").
To force JMX to connect back on the same port as you've established, you have a couple of the following options. Note: you can use different ports for JMX and RMI or you can use the same port.
Option 1: Command line
-Dcom.sun.management.jmxremote.port=$JMX_REGISTRY_PORT
-Dcom.sun.management.jmxremote.rmi.port=$RMI_SERVER_PORT
If you're using Spring Boot you can put this in your (appname).conf file that lives alongside your (appname).jar deployment.
Option 2: Tomcat/Tomee configuration
Configure a JmxRemoteLifecycleListener:
Maven Jar:
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-catalina-jmx-remote</artifactId>
<version>8.5.9</version>
<type>jar</type>
</dependency>
Configure your server.xml:
<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener"
rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002" />
Option 3: configure programmatically
#Configuration
public class ConfigureRMI {
#Value("${jmx.rmi.host:localhost}")
private String rmiHost;
#Value("${jmx.rmi.port:1099}")
private Integer rmiPort;
#Bean
public RmiRegistryFactoryBean rmiRegistry() {
final RmiRegistryFactoryBean rmiRegistryFactoryBean = new RmiRegistryFactoryBean();
rmiRegistryFactoryBean.setPort(rmiPort);
rmiRegistryFactoryBean.setAlwaysCreate(true);
return rmiRegistryFactoryBean;
}
#Bean
#DependsOn("rmiRegistry")
public ConnectorServerFactoryBean connectorServerFactoryBean() throws Exception {
final ConnectorServerFactoryBean connectorServerFactoryBean = new ConnectorServerFactoryBean();
connectorServerFactoryBean.setObjectName("connector:name=rmi");
connectorServerFactoryBean.setServiceUrl(String.format("service:jmx:rmi://%s:%s/jndi/rmi://%s:%s/jmxrmi", rmiHost, rmiPort, rmiHost, rmiPort));
return connectorServerFactoryBean;
}
}
The trick, you'll see, is the serviceUrl in which you specify both the jmx:rmi host/port and the jndi:rmi host/port. If you specify both, you won't get the random high "problem".
Edit: For JMX remoting to work, you'll need to make a decision about authenticating. It's better to do it in 3 distinct steps: 1) basic setup with -Dcom.sun.management.jmxremote.authenticate=false then 2) add a password file (-Dcom.sun.management.jmxremote.password.file). See here for instructions. + -Dcom.sun.management.jmxremote.ssl=false and then 3) set up SSL.
Add the following JVM Properties in "$JAVA_OPTS" (in your application):
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=<PORT_NUMBER> -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=<HOST'S_IP>
In the Jconsole/Visual VM use the following to connect:
service:jmx:rmi:///jndi/rmi://<HOST'S_IP>:<PORT_NUMBER>/jmxrmi
It doesn't enable security, but will help you to connect to the remote server.
A tested approach on Java 1.8.0_71 and Spring Boot(1.3.3.RELEASE).
Append below parameters to JVM arguments for monitored JVM.
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12348 -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.rmi.port=12349 -Dcom.sun.management.jmxremote.password.file=/somewhere/jmxremote.password -Dcom.sun.management.jmxremote.access.file=/somewhere/jmx/jmxremote.access
The com.sun.management.jmxremote.port is used to define the fixed RMI registry port, and the com.sun.management.jmxremote.rmi.port is used to instruct JVM to use fixed RMI port, but NOT use random one.
By setting this, I am able to connect JVM client from remote host to the monitored JVM via a firewall just opening 12348 and 12349 port.
I tested using java -jar cmdline-jmxclient-0.10.3.jar user:pwd hostip:12348 on a remote machine, which generates below output(shortened just for demonstration).
java.lang:type=Runtime
java.lang:name=PS Scavenge,type=GarbageCollector
Tomcat:J2EEApplication=none,J2EEServer=none,WebModule=//localhost/,j2eeType=Filter,name=requestContextFilter
java.nio:name=mapped,type=BufferPool
Tomcat:host=localhost,type=Host
java.lang:name=Compressed Class Space,type=MemoryPool
.......
The jar is downloaded from Here.
Another alternative
Reference for jmxremote.password and jmxremote.access files
import java.util.HashMap;
import java.util.Map;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.DependsOn;
import org.springframework.jmx.support.ConnectorServerFactoryBean;
import org.springframework.remoting.rmi.RmiRegistryFactoryBean;
#Configuration
public class ConfigureRMI {
#Value("${jmx.rmi.password.file:/tmp/jmxremote.password}")
private String passwordFile;
#Value("${jmx.rmi.access.file:/tmp/jmxremote.access}")
private String accessFile;
#Value("${jmx.rmi.port:19999}")
private Integer rmiPort;
#Bean
public RmiRegistryFactoryBean rmiRegistry() {
final RmiRegistryFactoryBean rmiRegistryFactoryBean = new RmiRegistryFactoryBean();
rmiRegistryFactoryBean.setPort(rmiPort);
rmiRegistryFactoryBean.setAlwaysCreate(true);
return rmiRegistryFactoryBean;
}
#Bean
#DependsOn("rmiRegistry")
public ConnectorServerFactoryBean connectorServerFactoryBean() throws Exception {
final ConnectorServerFactoryBean connectorServerFactoryBean = new ConnectorServerFactoryBean();
connectorServerFactoryBean.setObjectName("connector:name=rmi");
Map<String, Object> properties = new HashMap<>();
properties.put("jmx.remote.x.password.file", passwordFile);
properties.put("jmx.remote.x.access.file", accessFile);
connectorServerFactoryBean.setEnvironmentMap(properties);
connectorServerFactoryBean.setServiceUrl(String.format("service:jmx:rmi:///jndi/rmi://:%s/jmxrmi", rmiPort));
return connectorServerFactoryBean;
}
}