phoenix jdbc doesn't work, no exceptions and stuck - hadoop

I'm new to phoenix and hbase.hbase table and phoenix view works well and i can fetch data through phoenix. when I access jdbc to phoenix,it stucks.
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class Phoenix {
private static String driver = "org.apache.phoenix.jdbc.PhoenixDriver";
public static void main(String[] args) throws SQLException {
try {
Class.forName(driver);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
Statement stmt = null;
ResultSet rs = null;
System.out.println("start...");
Connection con = DriverManager.getConnection("jdbc:phoenix:[my_cloud_server_ip]:2181");
System.out.println(con);
con.close();
}
}
(only one zookeeper server has public internet ip,so i write this ip there,does it matters?)
it prints "start..." and no response any more
BUT when when the url is "jdbc:phoenix:ip:2181" or "jdbc:phoenix:ip:2181/hbase"
i got no response
when i add some other words ,for example "jdbc:phoenix:ip:2181/balabala"
i got NULL POINTER EXCEPTION
why ?
I hope u can understand what i say:)

Reproducing your example, I am getting a java.net.SocketTimeoutException after 60 seconds, caused by java.net.UnknownHostException: unknown host: <hostname_of_my_zk_server>. But maybe that's what you called "no response" if you were in a hurry (or if you have a custom hbase-site.xml with a larger timeout client-side).
This error seems like the one explained in this article (section Zookeeper at the end) :
As in our case CDH was running in a test VM, we encountered this issue: http://stackoverflow.com/questions/18428722/hbase-java-client-unknown-host-localhost-localdomain
That was worked around by adding localhost.localdomain to the existing /etc/hosts entry for cluster1, which was already pointing to the right IP address.
This answer on SO summarizes the solution.
Basically, you need to add an entry to your /etc/hosts client-side :
<my_cloud_server_ip> <hostname_of_my_cloud_server_ip>
Moreover, you need to have an hbase-site.xml client-side (you can use the one on your server as a basis).

Related

Oracle Database JDBC driver cannot read wallet file from Spark

Objective
I'm trying to write to Oracle's ADWC (basically oracle database) from a Spark application running on Yarn. The only way to connect to this database is by using an Oracle Wallet file, which is basically a Java keystore.
Problem
The problem arises when the JDBC driver tries to read the wallet from HDFS. If I include the hdfs:// prefix the parser in the JDBC driver throws an error and if I don't then it cannot find the file.
Previous Attempts
including the directory in the connect string (prefixed and non) jdbc:oracle:thin:#luigi_low?TNS_ADMIN=/user/spark/wallet_LUIGI
including the directory as an spark.driver.extraJavaOptions with -Doracle.net.tns_admin and -Doracle.net.wallet_location
All the code is on GitHub, and specifically, the error messages are here https://github.com/sblack4/kafka-scala-jdbc/blob/master/ERROR.md
I've got a working example of the same connection here https://github.com/sblack4/scala-jdbc-adwc
help me StackOverflow. you are my only hope
If you need any more clarification don't hesitate :)
update (SparkFiles attempt)
the code is on a separate branch of the same repository, https://github.com/sblack4/kafka-scala-jdbc/tree/sparkfiles
This error message mystifies me as it seems my JDBC library has stopped trying to read the wallet files. It may be unrelated to the previous problem
Exception in thread "main" java.sql.SQLRecoverableException: IO Error: Invalid connection string format, a valid format is: "host:port:sid"
I've deleted the other JDBC libraries from my classpath through Ambari as this error could be related to spark picking up an older version of my JDBC library
Here's some code that will help diagnose what the issues is.
It checks and configures everything required to connect.
JDBC Driver version
JCE Installed
Classpath dependencies
Configures
tns_admin
ssl settings
trust/key stores
This is a slimmed down version of what's in sqldev/sqlcl
import java.security.NoSuchAlgorithmException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Collections;
import java.util.Properties;
import javax.crypto.Cipher;
import oracle.jdbc.OracleConnection;
public class JDBCTest {
public static void fail(String msg){
System.err.println(String.join("", Collections.nCopies(20, "*")));
System.err.println(msg);
System.err.println(String.join("", Collections.nCopies(20, "*")));
System.exit(1);
}
public static void main(String[] args) throws SQLException {
System.out.println("JDBC Driver Version:" + oracle.jdbc.OracleDriver.getDriverVersion());
// Check JDBC Driver Version
if (!oracle.jdbc.OracleDriver.getDriverVersion().startsWith("18.")) {
fail(" DRIVER TOOO OLD!!!");
}
// Check JCE Installed
int maxKeySize = 0;
try {
maxKeySize = Cipher.getMaxAllowedKeyLength("AES");
} catch (NoSuchAlgorithmException e) {
}
if (maxKeySize < 129 ) {
fail(" JCE Policy not unlimited!!!");
}
// Check Classpath
String cp = System.getProperty("java.class.path");
String[] cpFiles = {"ojdbc8.jar","oraclepki.jar","osdt_cert.jar","osdt_core.jar"};
for (String file:cpFiles){
if ( cp.indexOf(file) == -1 ){
fail("CLASSPATH Missing:" + file);
}
}
// Wallet unziped location
String unzippedWalletLocation = "/Users/klrice/workspace/12.2JDBC/wallet";
String conString = "jdbc:oracle:thin:#sqldev_medium";
Properties props = new Properties();
props.setProperty("oracle.net.wallet_location",unzippedWalletLocation);
props.setProperty(OracleConnection.CONNECTION_PROPERTY_THIN_NET_CONNECT_TIMEOUT, "2000");
// unzipped includes a tnsnames.ora
props.setProperty("oracle.net.tns_admin",unzippedWalletLocation);
props.setProperty("javax.net.ssl.trustStore","truststore.jks");
props.setProperty("javax.net.ssl.trustStorePassword","<password>");
props.setProperty("javax.net.ssl.keyStore","keystore.jks");
props.setProperty("javax.net.ssl.keyStorePassword","<password>");
props.setProperty("oracle.net.ssl_server_dn_match","true");
props.setProperty("oracle.net.ssl_version","1.2");
props.setProperty("user", "ADMIN");
props.setProperty("password", "<password>");
try {
// now Connect
Connection conn = DriverManager.getConnection(conString,props);
} catch (Exception e){
e.printStackTrace();
fail(e.getLocalizedMessage());
}
System.out.println("SUCCESS!!");
}
}
Are you using 18.3 JDBC drivers? Passing TNS_ADMIN as part of the connection URL requires 18.3 JDBC driver. Also, are you attempting to connect within the corporate network. In that case, you will need to pass HTTPS_PROXY and HTTPS_PROXY_PORT in the connection URL. Let us know. Happy to help with the problem.

Unable to connect to database through Eclipse

I have been trying to run this code for hours, to get no success. I have Oracle 11g Enterprise Edition. Does anyone know the code to get port number from sql * plus cmd prompt?
The code doesn't run and I am not able to view the output over eclipse. Please guide me.
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
public class Testt {
public static void main(String[] args) throws Exception {
try{
//Driver registration
Class.forName("oracle.jdbc.driver.OracleDriver");
//Create connection
Connection c =DriverManager.getConnection("jdbc:oracle:thin:
#localhost:1521:EE","scott","tiger");
//Create statement
Statement stmt=c.createStatement();
//Execute query
ResultSet rs=stmt.executeQuery("select * from emp;");
while(rs.next())
{
System.out.println(rs.getInt(1)+" "+rs.getString(2));
}
c.close();
}
catch(Exception e){e.printStackTrace();}
}
}
Output:
java.sql.SQLException: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

Connecting to Google cloud sql from Java stand alone application

I want to write a java stand alone application to connect to google cloud sql - MySql database. I could find samples for app client but not for a stand alone java application.
When I tried to do this, I get the following error.
Class.forName("com.mysql.jdbc.Driver");
Connection con = DriverManager.getConnection("jdbc:google:mysql://IP:Instance_name?user=user_name");
Statement st = con.createStatement();
ResultSet rs = con.createStatement().executeQuery("SELECT 1 + 1");
It fails with error:
No suitable driver found for jdbc:google:mysql:..
I see some answers where they have asked to enabled mysql connector. But it is the case with app engine project. How do I do it with stand alone application?
You can just use the standard mysql connector to connect to cloud sql instance: e.g:
DriverManager.getConnection("jdbc:mysql://IP:Instance_name?user=user_name");
You can check this link https://cloud.google.com/sql/docs/external for more information.
We can simply use the standard mysql java connector to connect to cloud sql form a standalone java application.
Follow the below code snippet. This should help you to connect to database seamlessly.
class.forName(com.mysql.jdbc.Driver);
String connectionUrl = "jdbc:mysql//<public-ip>/<database>?useSSL=false";
Connection conn = DriverManager.getConnection(connectionUrl, user, password);
Before you do this make sure you get your ipaddress from internet and add the same under the path : google cloud -> sql -> sqlinstance -> connections -> networking -> Authorized networks.
Click here to see the google cloud mysql configuration
Edit inside <> before running your code
import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.SQLNonTransientConnectionException;
import java.sql.Statement;
/**
* A sample app that connects to a Cloud SQL instance and lists all available tables
in a database.
*/
public class Cloud_sql {
public static void main(String[] args) throws SQLNonTransientConnectionException
,IOException, SQLException {
String instanceConnectionName = <Your instanceConnectionName>;
String databaseName = <Database name from which u want to list tables>;
String IP_of_instance = <IP address of the instance>;
String username = <mysql username>;
String password = <mysql password>;
String jdbcUrl = String.format(
"jdbc:mysql://%s/%s?cloudSqlInstance=%s"
+ "&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useSSL=false",
IP_of_instance,
databaseName,
instanceConnectionName);
Connection connection = DriverManager.getConnection(jdbcUrl, username, password);
try (Statement statement = connection.createStatement()) {
ResultSet resultSet = statement.executeQuery("SHOW TABLES");
while (resultSet.next()) {
System.out.println(resultSet.getString(1));
}
}catch(Exception e){
e.printStackTrace();
}
}
}

Route lines from file to persistent JMS queue: How to improve performance?

I need some help with performance tuning of a use case. In this use case the Camel route is tailing status lines in a log file and sends each line as a message to a JMS queue. I have implemented the use case like this:
package tests;
import java.io.File;
import java.net.URI;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.broker.BrokerFactory;
import org.apache.activemq.broker.BrokerService;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.sjms.SjmsComponent;
import org.apache.camel.main.Main;
public class LinesToQueue {
public static void main() throws Exception {
final File file = new File("data/log.txt");
final String uri = "tcp://127.0.0.1:61616";
final BrokerService jmsService = BrokerFactory.createBroker(new URI("broker:" + uri));
jmsService.start();
final SjmsComponent jmsComponent = new SjmsComponent();
jmsComponent.setConnectionFactory(new ActiveMQConnectionFactory(uri));
final Main main = new Main();
main.bind("jms", jmsComponent);
main.addRouteBuilder(new RouteBuilder() {
#Override
public void configure() throws Exception {
fromF("stream:file?fileName=%s&scanStream=true&scanStreamDelay=0", file.getAbsolutePath())
.routeId("LinesToQueue")
.to("jms:LogLines?synchronous=false");
}
});
main.enableHangupSupport();
main.run();
}
}
When I run this use case with a file already filled with 1.000.000 lines the overall performance I get in the route is about 313 lines/second. This means that it takes about 55 minutes to process the file.
As some sort of reference I also have created another use case. In this use case the Camel route is tailing status lines in a log file and sends each line as a document to an Elasticsearch index. I have implemented the use case like this:
package tests;
import java.io.File;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.main.Main;
public class LinesToIndex {
public static void main() throws Exception {
final File file = new File("data/log.txt");
final String uri = "local";
final Main main = new Main();
main.addRouteBuilder(new RouteBuilder() {
#Override
public void configure() throws Exception {
fromF("stream:file?fileName=%s&scanStream=true&scanStreamDelay=0", file.getAbsolutePath())
.routeId("LinesToIndex")
.bean(new LineConverter())
.toF("elasticsearch://%s?operation=INDEX&indexName=log&indexType=line", uri);
}
});
main.enableHangupSupport();
main.run();
}
}
When I run this use case with a file already filled with 1.000.000 lines the overall performance I get in the route is about 8333 lines/second. This means that it takes about 2 minutes to process the file.
I understand that there is a huge difference between a JMS queue and an Elasticsearch index but how can have the JMS use case above to perform better?
Update #1:
It seems to be the persistence in the JMS service that is the bottleneck in my first use case above. If I disable the persistence in the JMS service then the performance in the route is about 11111 lines/second. Which persistence storage for the JMS service will give me a better performance?
a couple of things to consider...
ActiveMQ producer connections are expensive, make sure you use a pooled connection factory...
consider using the VM transport for an in process ActiveMQ instance
consider using an external ActiveMQ broker over TCP (so it doesn't compete for resources with your test)
setup/tune KahaDB or LevelDB to optimize persistent storage for your use case

Does Oracle 11gR2 actually support IPV6?

Oracle has declared that 11g R2 has support IPV6 and ojdbc6.jar is the right one.
But when I test it, I get exception, have you solved that?
My test code is:
import java.sql.SQLException;
import java.util.Properties;
public class Test {
final static String sDBDriver = "oracle.jdbc.driver.OracleDriver";
/**
* #param args
* #throws SQLException
*/
public static void main(String[] args) throws SQLException {
// TODO Auto-generated method stub
java.sql.Connection conn=null;
String url="jdbc:oracle:thin:#(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=
[fe80::b056:5cff:fe78:b672])(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=fnstdb1))";
try
{
Class.forName(sDBDriver);
conn = DriverManager.getConnection(url,"scott","fnst1234");
}
catch (Exception e)
{
System.out.println("ERROR:"+e.getMessage());
}
finally
{
System.out.println("连接是否关闭:"+conn.isClosed());
conn.close();
}
}
}
and I use the following cmd:
java -cp ojdbc6.jar -Djava.net.preferIPv6Addresses=true Test
but the result is: ERROR:NL Exception was generated
What is wrong?
I have solved this problem .the following shows what have I done:
Procedure
edit $DB_HOME\NETWORK\ADMIN\listener.ora file to allow the oracle to listen on the ip and port. for example: (ADDRESS = (PROTOCOL = TCP)(HOST = [fe80::221:97ff:fe66:1fa9%4])(PORT= 1521))
Reatart the listener. Run: LSNRCTL stop/start
In some applications (javase, connection-pool, lookup)use ojdbc6.jar: jdbc:oracle:thin:#[fe80::221:97ff:fe66:1fa9]:1521:orcl.
The result is successful!!
Demo
the ip address in listener.ora must add "%4" ,or it will fail.
ipv6 address must in "["and "]",or it will fail.
the document of oracle declares that using ipv6 must set the jvm options
java.net.preferIPv6Addresses=true
but it does not matter whether it sets or not!

Resources