I am trying to use Zeppelin (v 0.7.0 java 1.8 on Windows 10; same with docker v .0.7.1) JDBC interpreter to query Oracle Database.
So far I've found papers like example. I try to use jdbc interpreter with:
common.max_count=100
default.driver=oracle.jdbc.pool.OracleDataSource
default.password:$password
default.user=$my_user_name
default.url=jdbc:oracle:thin:#$host:1521/$service_name>
zeppelin.jdbc.concurrent.max_connection=10
zeppelin.jdbc.concurrent.use=true
Connection looks to be established successfully*, but even simplest query (like sql% select 1 from dual) runs for ever.
Does anyone knows/have a clue on what is to be added?
what I see in terminal - repeating lines
sKey: shared_process for InterpreterSetting Id: 2CMBZV4T1, Name: jdbc
DEBUG [2017-06-15 16:03:28,203] ({Thread-90} InterpreterSetting.java[getInterpreterProcessKey]:143) - getInterpreterProcessKey: shared_process for InterpreterSetting Id: 2CMBZV4T1, Name: jdbc
DEBUG [2017-06-15 16:03:28,204] ({Thread-90} InterpreterSetting.java[getInterpreterProcessKey]:143) - getInterpreterProcessKey: shared_process for InterpreterSetting Id: 2CMBZV4T1, Name: jdbc
DEBUG [2017-06-15 16:03:28,204] ({Thread-90} InterpreterFactory.java[getInterpreterSessionKey]:1244) - Interpreter session key: shared_session, for note: 2CMEC9SS4, user: anonymous, InterpreterSetting Name: jdbc
DEBUG [2017-06-15 16:03:28,205] ({Thread-90} InterpreterSetting.java[getInterpreterProcessKey]:143) - getInterpreterProcessKey: shared_process for InterpreterSetting Id: 2CMBZV4T1, Name: jdbc
DEBUG [2017-06-15 16:03:28,205] ({Thread-90} InterpreterSetting.java[getInterpreterProcessKey]:143) - getInterpreterProcessKey: shared_process for InterpreterSetting Id: 2CMBZV4T1, Name: jdbc
In the same time I don't have an issue using spark interpretor with scala over jdbc same driver.
at least when I provide wrong password it prints meaningful response
Are you sure a query is running?
selec 1 from dual
is not valid SQL.
Check on Oracle side in gv$session for your query. If it is running, it will be there.
Related
I am able to connect to the cluster, browse its hive catalog, see tables/views and columns/datatypes
Running a simple select statement from a view on a parquet file produces this error and no other results:
SQL Error [500540] [HY000]: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 180. Background thread ID: 223. Error caught: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available.
Standard Databricks cluster:
Standard_DS3_v2
JDBC URL:
jdbc:databricks://<reducted>.1.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/<reducted>/<reducted>;AuthMech=3;UID=token;PWD=<reducted>
Advanced Options Spark Config:
spark.databricks.cluster.profile singleNode
spark.databricks.io.directoryCommit.createSuccessFile false
spark.master local[*, 4]
spark.driver.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true
spark.hadoop.fs.azure.account.key.<reducted>.blob.core.windows.net <reducted>
spark.executor.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true
parquet.enable.summary-metadata false
My local machine:
Dbeaver Version 22.1.2.202207091909
MacOS version (M1 chip): Monterey 12.4
Java version:
java --version
openjdk 18.0.1 2022-04-19
OpenJDK Runtime Environment Homebrew (build 18.0.1+0)
OpenJDK 64-Bit Server VM Homebrew (build 18.0.1+0, mixed mode, sharing)
I am able to do the following with no errors (Databricks default test dataset):
CREATE TABLE diamonds USING CSV OPTIONS (path "/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv", header "true");
When I run this select color from diamonds; or this select * from diamonds;
I get this:
SQL Error [500618] [HY000]: [Databricks][DatabricksJDBCDriver](500618) Error occured while deserializing arrow data: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available
Hence, any select query on any object (parquet file or anything else) causes the error described above.
What could be the problem? Any recommendations how to resolve this error? Why am I able to connect and see the metadata of the schemas/tables/views/columns, but not query or view the data?
P.S. I followed this guide exactly: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/dbeaver#step-3-connect-dbeaver-to-your-azure-databricks-databases
I want to create a SQL initialization file to populate test data every time on Spring startup:
databaseChangeLog:
- changeSet:
id: 0001
author: test
dbms: postgres
changes:
- sqlFile:
- relativeToChangelogFile: true
- path: data.sql
Do you know how I can configure it to be executed on every Spring startup?
You can use the following property. Now the specific changeSet will run every time
- changeSet:
runAlways: true
https://docs.liquibase.com/concepts/basic/changeset.html
runAlways Executes the changeset on every run, even if it has been run before.
according to https://www.testcontainers.org/modules/databases/jdbc/#database-containers-launched-via-jdbc-url-scheme , i'm trying to create an Oracle Container with Quarkus using jdbc url scheme.
After provided a valid docker image ("store/oracle/database-instantclient:12.2.0.1") and set this properties:
"%test":
quarkus:
datasource:
jdbc:
driver: org.testcontainers.jdbc.ContainerDatabaseDriver
url: jdbc:tc:oracle:///databasename
db-kind: other
i get this error:
Container is started (JDBC URL: jdbc:oracle:thin:system/oracle#localhost:32827:xe)
2020-11-09 17:33:06,719 INFO [🐳 .2.0.1]] (Agroal_13889837441) Container store/oracle/database-instantclient:12.2.0.1 started in PT4M7.8483772S
2020-11-09 17:33:06,738 WARN [io.agr.pool] (Agroal_13889837441) Datasource '<default>': Could not create new connection
2020-11-09 17:33:06,805 ERROR [io.qua.application] (main) Failed to start application (with profile test): org.flywaydb.core.internal.exception.FlywaySqlException:
Unable to obtain connection from database: Could not create new connection
--------------------------------------------------------------------------
SQL State : null
Error Code : 0
Message : Could not create new connection
at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:65)
at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:80)
at org.flywaydb.core.Flyway.execute(Flyway.java:453)
at org.flywaydb.core.Flyway.migrate(Flyway.java:158)
Can someone help me?
If I remember correctly, Flyway community editions (that ships with Quarkus) does not have support for Oracle. You need to use the Enterprise edition.
You need to replace the Quarkus Flyway dependency. Just exclude the one included and add the enterprise one (either in Maven or Gradle).
I am having this strange issue connection JDBC to Oracle 11g. It started happening suddenly this Monday without any settings changes that either me or the team were aware of. Reading in Stackoverflow and Oracle forums (see links at the bottom) I've learned that there has been an issue with the generation of random bytes on linux 64bit machines. It led my to try several things, which unfortunately didn't work. Here is a summary of all information I'm aware of.
The weird thing is that I can connect over the terminal using sqlplus, but not using JDBC.
Thank you for any ideas or help,
Amit
My Setup:
OS: Ubuntu 12.04
Java: Both 7 and 6_45 (Issue is the same)
JDBC: Downloaded from the Oracle official site (see further description down)
Connection behind a Cisco VPN
JDBC-MANIFESTS:
odbc7.jar:
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.1
Created-By: 20.12-b01 (Sun Microsystems Inc.)
Implementation-Vendor: Oracle Corporation
Implementation-Title: JDBC
Implementation-Version: 12.1.0.1.0
Repository-Id: JAVAVM_12.1.0.1.0_LINUX.X64_130403
Specification-Vendor: Sun Microsystems Inc.
Specification-Title: JDBC
Specification-Version: 4.0
Main-Class: oracle.jdbc.OracleDriver
sealed: true
Name: oracle/sql/converter/
Sealed: false
Name: oracle/sql/
Sealed: false
Name: oracle/sql/converter_xcharset/
Sealed: false
odbc6.jar:
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.1
Created-By: 20.12-b01 (Sun Microsystems Inc.)
Implementation-Vendor: Oracle Corporation
Implementation-Title: JDBC
Implementation-Version: 12.1.0.1.0
Repository-Id: JAVAVM_12.1.0.1.0_LINUX.X64_130403
Specification-Vendor: Sun Microsystems Inc.
Specification-Title: JDBC
Specification-Version: 4.0
Main-Class: oracle.jdbc.OracleDriver
sealed: true
Name: oracle/sql/converter/
Sealed: false
Name: oracle/sql/
Sealed: false
Name: oracle/sql/converter_xcharset/
Sealed: false
On Friday everything was still working:
I could start my application and have a Java connection to the Oracle DB.
Since Monday I'm having this issue
My old computer (Ubuntu 12.04) as well, I hadn't touched for almost a week started having this issue too, even though I left it in a working state.
Terminal connection using SQLPLUS is working:
amit#mymachine:/usr/lib/oracle$ sqlplus /#DB_HOST_IP:1521/DB_NAME
SQL*Plus: Release 12.1.0.1.0 Production on Tue Apr 22 11:35:58 2014
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Release 11.2.0.3.0 - 64bit Production
SQL>
But connection with JDBC is failing:
DriverManager.getConnection(
"jdbc:oracle:thin:#//DB_HOST_IP:1521/DB_NAME", username,
password);
Hangs for about 20 seconds and throws:
ava.sql.SQLRecoverableException: IO Error: Connection reset
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:682)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:711)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:385)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:30)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:558)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at OracleJDBC.main(OracleJDBC.java:32)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at oracle.net.ns.Packet.receive(Packet.java:311)
at oracle.net.ns.DataPacket.receive(DataPacket.java:105)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:305)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:249)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:171)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:89)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79)
at oracle.jdbc.driver.T4CMAREngineStream.unmarshalUB1(T4CMAREngineStream.java:426)
at oracle.jdbc.driver.T4C8TTIdty.receive(T4C8TTIdty.java:688)
at oracle.jdbc.driver.T4C8TTIdty.doRPC(T4C8TTIdty.java:595)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1439)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:486)
... 12 more
From the server's log:
Fatal NI connect error 12170.
VERSION INFORMATION:
TNS for Linux: Version 11.2.0.3.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version
11.2.0.3.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.3.0 - Production
Time: 22-APR-2014 09:57:45
Tracing not turned on.
Tns error struct:
ns main err code: 12535
TNS-12535: TNS:operation timed out
ns secondary err code: 12606
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=<my.ip.address>)(PORT=42738))
From the Oracle documentation on both server errors:
ORA-12535: TNS:operation timed out
Cause: The requested operation could not be completed within the time out period.
Action: Look at the documentation on the secondary errors for possible remedy. See SQLNET.LOG to find secondary error if not provided explicitly. Turn on tracing to gather more information.
ORA-12606: TNS: Application timeout occurred
Cause: A network session did not reach an application-defined stage within the allowed time interval.
Action: This is an error which does not normally appear at the high level. The action to take is application specific, and is detailed in the higher level error description.
Solutions I've tried:
Adding the following system property -Djava.security.egd=file:///dev/urandom as suggested in the Stackoverflow discussion below.
Set up rng-tools to 'help' the entropy generator of the OS. (see links)
Following code runs without a problem producing random byte arrays (regardless of the parameter -Djava.security.egd:
int a = 10000;
while (a-- > 0){
byte[] array = new byte[2048];
new SecureRandom().nextBytes(array);
System.out.println(a);
}
None of the above seem to have changed the error (time-out).
EDIT: new attemps:
Running my application I could recognize the exact query that causes me the live problem. This is when reading a CLOG column of 1-2kb in size.
SQL> select secure_params from session_token where id=11065073; <- query is stuck.
Restart the client,
increasingly updating this field in this specific row I could still read the field if it was 429 charactes in size, but then when I've increased this to 587 chars it got stuck again.
SQL> select length( secure_params) from session_token where id=11065073;
LENGTH(SECURE_PARAMS)
---------------------
587
SQL> select secure_params from session_token where id=11065073; <- query again stuck.
Attemps number 4 brings me to the thought that there might be an additional issue here, since the application is able to create an initial connection and get some short data from the DB, but only then fails when the results-set gets a bit fat.
END OF EDIT
Links:
Stackoverflow: Oracle JDBC intermittent Connection Issue
Rng entropy generator: http://www.howtoforge.com/helping-the-random-number-generator-to-gain-enough-entropy-with-rng-tools-debian-lenny
ORA-12170 indicates that the TCP connection to the database listener failed.
If this is an error that happens all the time, then check firewalls and network ACLs. Possibly something intermediary has changed?
Alternatively, it could be some other artifact impacting TCP connectivity, such as a IP address clash. I've recently had this problem myself, and have blogged about it at http://distracted-it.blogspot.co.nz/2014/04/ora-12170-tnsconnect-timeout-resolved.html
Hope it helps.
How connect to Sybase custdb.db using SwisSQL?
I work with standart Sybase sample.
I input appropriate command line:
"mlsrv12.exe -vcrs -zu+
-c "dsn=SQL Anywhere 12 CustDB;uid=ml_server;pwd=sql"
-x http(port=80)"
Database work fine - I test it from InteractiveSQL and from Android - client.
When I try to migrate from this database to Oracle I have trouble.
Host name: 127.0.0.1
Port number: 80
User name: ml_server
Password: sql
Service ID (SID): custdb
Shema(optional): empty
What I am doing wrong?
Any help appreciated!