Run cmd :
./jsvc64/jsvc64 -pidfile ./log/jsvc.pid -outfile ./log/out.txt -errfile ./log/error.txt -Xmx512m -Djava.util.Arrays.useLegacyMergeSort=true -cp :./tools/lib/:./tools/ com.g2us.hbase.cmdlog.monitor.CmdLogHbase ./
SQL :
UPSERT INTO CMDLOG_20130818(game,roleid,otime,logtype,passport,subgame,cmdid,exception,moreinfo,pname_0,pname_1,pname_2) VALUES(?,?,?,?,?,?,?,?,?,?,?,?)
upsert 90000 row data,the exception occored.
How to solve it.
Exception in thread "Thread-0" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.reflect.Method.copy(Method.java:143)
at java.lang.reflect.ReflectAccess.copyMethod(ReflectAccess.java:118)
at sun.reflect.ReflectionFactory.copyMethod(ReflectionFactory.java:282)
at java.lang.Class.copyMethods(Class.java:2748)
at java.lang.Class.getMethods(Class.java:1410)
at org.apache.hadoop.hbase.ipc.Invocation.<init>(Invocation.java:67)
at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
at $Proxy8.getClosestRowBefore(Unknown Source)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1019)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:885)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:846)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:271)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:211)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:160)
at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:383)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:947)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1002)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:889)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:846)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:271)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:263)
at com.salesforce.phoenix.query.HTableFactory$HTableFactoryImpl.getTable(HTableFactory.java:60)
at com.salesforce.phoenix.query.ConnectionQueryServicesImpl.getTable(ConnectionQueryServicesImpl.java:133)
at com.salesforce.phoenix.execute.MutationState.commit(MutationState.java:227)
at com.salesforce.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:244)
at com.g2us.hbase.phoenix.HBaseHelper.executeUpdate(HBaseHelper.java:62)
at com.g2us.hbase.cmdlog.io.BaseLogPoster.upsertRow(BaseLogPoster.java:153)
I found the problem and fixed it.
the problem that 's preStat define as class field var ,so that call executeQuery() many time no close it ,then OutOfMemoryError.
error code:
public class F{
PreparedStatement preStat = null;
public ResultSet executeQuery(String sql, Object... args) throws Exception {
ResultSet rsResultSet = null;
Connection conn = null;
Statement stat = null;
try {
conn = HBaseUtility.getConnection();
preStat = conn.prepareStatement(sql);
if (args != null) {
for (int i = 0; i < args.length; i++) {
preStat.setObject(i + 1, args[i]);
}
}
rsResultSet = preStat.executeQuery();
} catch (Exception e) {
dispos(conn, stat);
Log.error(Log.DB, "queryerror|", e);
throw new RuntimeException("hbase query error");
} finally {
HBaseUtility.release(conn);
}
return rsResultSet;
}
}
fixed code:
public class F{
public ResultSet executeQuery(String sql, Object... args) throws Exception {
ResultSet rsResultSet = null;
Connection conn = null;
Statement stat = null;
try {
PreparedStatement preStat = null; //this var as a class var ,and no close every query .
conn = HBaseUtility.getConnection();
preStat = conn.prepareStatement(sql);
if (args != null) {
for (int i = 0; i < args.length; i++) {
preStat.setObject(i + 1, args[i]);
}
}
rsResultSet = preStat.executeQuery();
preStat.close(); //must be close.
} catch (Exception e) {
dispos(conn, stat);
Log.error(Log.DB, "queryerror|", e);
throw new RuntimeException("hbase query error");
} finally {
HBaseUtility.release(conn);
}
return rsResultSet;
}
}
Related
public void createCostRecord() throws Exception
{
Context ctx = null;
Connection conn = null;
CallableStatement ps = null;
ResultSet rs = null;
boolean spReturn = false;
try{
ctx = new InitialContext();
javax.sql.DataSource ds = (javax.sql.DataSource) ctx.lookup("CSMWebAppjndi");
conn = ds.getConnection();
conn.setAutoCommit(true);
String sp = "{call usp_CreateCostRecords(?,?)}";
ps = conn.prepareCall(sp);
ps.setInt(1, 1000);
ps.setInt(2, 2000);
for(int i=0;i<3;i++)
{
ps.executeQuery();
}
} catch (NamingException e)
{
log.error(e,e);
} catch (SQLException e)
{
log.error(e,e);
}
catch(Exception e)
{
log.error(e,e);
}
finally {
if(rs!=null){
rs.close();
}
if(ps!=null){
ps.close();
}
if(conn != null){
conn.close();
}
if(ctx != null ){
ctx.close();
}
}
}
while calling the above method the line number 23 executeQuery works fine for the first iteration of the for loop,
on second iteration of the for loop its getting struck at executeQuery and the procedure never completes execution.
But the weird thing is while i try the same procedure with same input from SQL developer its getting executed for any number of times without any struck.
Anyone help me to understand why the procedure from java is getting struck at second attempt and but its working fine in SQL developer.
I'm trying to get values from resulset, but it return nothing.
When i'm trying to do it through plain sql it return some values.
List<String> res = new ArrayList<String>();
try {
String query = "SELECT COLUMN_NAME FROM idoc.columns_to_show where user = ? "
+ DAO.DB2_UR_POSTFIX;
Connection connection = Properties.getDocsConnection();
try {
PreparedStatement pr = connection.prepareStatement(query);
try {
pr.setString(1, user.getDomainName());
ResultSet rs = pr.executeQuery();
try {
while (rs.next()) {
res.add(rs.getString("COLUMN_NAME"));
}
} finally {
rs.close();
}
} finally {
pr.close();
}
} finally {
connection.close();
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
return res;
Don't use column name "user" it is reserved name for DB2 database.
Therefore I couldn't find any result.
private void jButton1ActionPerformed(java.awt.event.ActionEvent evt) {
if (evt.getSource() == jButton1) {
String ab = jTextField1.getText();
String bc = jPasswordField1.getText().toString();
String cd = jTextField2.getText();
String de = jTextField3.getText();
PreparedStatement ps1 = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
Connection c = DriverManager.getConnection("jdbc:oracle:thin:#localhost:1521:xe", "system", "hr");
ps = c.prepareStatement("Select User_Name from AdminLogin where Password =?");
ps.setString(1, bc);
rs = ps.executeQuery();
rs.next();
if (ab.equals(rs.getString(1))) {
ps1 = c.prepareStatement("Update AdminLogin SET Date1=?,Time=? WHERE Password=?");
ps1.setString(1, cd);
ps1.setString(2, de);
ps1.setString(3, bc);
int e = ps1.executeUpdate();
JOptionPane.showMessageDialog(this, "Welcome", "Logged In", JOptionPane.INFORMATION_MESSAGE);
//MainMenuAAI mainMenuAAI = new MainMenuAAI();
//setVisible(false);}
} else if (!(ab.equals(rs.getString(1)))) {
JOptionPane.showMessageDialog(this, "<html>YOU ARE NOT A<br>ADMIN</br></html>", "ERROR", JOptionPane.ERROR_MESSAGE);
//AdminLogin admin=new AdminLogin();
//setVisible(false);
}
c.close();
} catch (Exception e) {
System.out.println(e);
}
}// TODO add your handling code here:
}
Everything is working fine in the code. It is executing the code inside
if(ab.equals(rs.getString(1)))
and showing "Welcome" but not d one inside
if(!(ab.equals(rs.getString(1))))
Whenever I enter wrong username or password it shows the error
java.sql.SQLException: Exhausted Resultset
That is correct, because if you enter wrong username or password, NO record will be returned. So, when you use the rs.next(); in this case, it is trying to access the first row of the empty result set! And that is where it is throwing the exception.
You could fix your code like this:
rs = ps.executeQuery();
//rs.next();
int counter=0;
while (rs.next()) {
counter++;
if (ab.equals(rs.getString(1))) {
ps1 = c.prepareStatement("Update AdminLogin SET Date1=?,Time=? WHERE Password=?");
ps1.setString(1, cd);
ps1.setString(2, de);
ps1.setString(3, bc);
int e = ps1.executeUpdate();
JOptionPane.showMessageDialog(this, "Welcome", "Logged In", JOptionPane.INFORMATION_MESSAGE);
//MainMenuAAI mainMenuAAI = new MainMenuAAI();
//setVisible(false);}
}
}
if(counter==0){
JOptionPane.showMessageDialog(this, "<html>YOU ARE NOT A<br>ADMIN</br></html>", "ERROR", JOptionPane.ERROR_MESSAGE);
}
My question is very much like Getting the return value of a PL/SQL function via Hibernate
I have a function which does some modifications internally and it returns a value.
The original idea was to do something like this:
protected Integer checkXXX(Long id, Long transId)
throws Exception {
final String sql = "SELECT MYSCHEMA.MYFUNC(" + id + ", "
+ transId + ") FROM DUAL";
final BigDecimal nr = (BigDecimal) this.getHibernateTemplate()
.getSessionFactory().getCurrentSession().createSQLQuery(sql)
.uniqueResult();
return nr.intValue();
}
Unfortunately this doesn't work with Oracle. What is the recommended way to do something like this?
Is there a way to extract declared variables from within my statement?
Hibernate Session provides a doWork() method that gives you direct access to java.sql.Connection. You can then create and use java.sql.CallableStatement to execute your function:
session.doWork(new Work() {
public void execute(Connection connection) throws SQLException {
CallableStatement call = connection.prepareCall("{ ? = call MYSCHEMA.MYFUNC(?,?) }");
call.registerOutParameter( 1, Types.INTEGER ); // or whatever it is
call.setLong(2, id);
call.setLong(3, transId);
call.execute();
int result = call.getInt(1); // propagate this back to enclosing class
}
});
You have the following options:
With a #NamedNativeQuery:
#org.hibernate.annotations.NamedNativeQuery(
name = "fn_my_func",
query = "{ ? = call MYSCHEMA.MYFUNC(?, ?) }",
callable = true,
resultClass = Integer.class
)
Integer result = (Integer) entityManager.createNamedQuery("fn_my_func")
.setParameter(1, 1)
.setParameter(2, 1)
.getSingleResult();
With JDBC API:
Session session = entityManager.unwrap( Session.class );
final AtomicReference<Integer> result =
new AtomicReference<>();
session.doWork( connection -> {
try (CallableStatement function = connection
.prepareCall(
"{ ? = call MYSCHEMA.MYFUNC(?, ?) }"
)
) {
function.registerOutParameter( 1, Types.INTEGER );
function.setInt( 2, 1 );
function.setInt( 3, 1 );
function.execute();
result.set( function.getInt( 1 ) );
}
} );
With a native Oracle query:
Integer result = (Integer) entityManager.createNativeQuery(
"SELECT MYSCHEMA.MYFUNC(:postId, :transId) FROM DUAL")
.setParameter("postId", 1)
.setParameter("transId", 1)
.getSingleResult();
Yes, you do need to use an out parameter. If you use the doWork() method, you'd do something like this:
session.doWork(new Work() {
public void execute(Connection conn) {
CallableStatement stmt = conn.prepareCall("? = call <some function name>(?)");
stmt.registerOutParameter(1, OracleTypes.INTEGER);
stmt.setInt(2, <some value>);
stmt.execute();
Integer outputValue = stmt.getInt(1);
// And then you'd do something with this outputValue
}
});
Alternative code :)
if you want to direct result you can use below code
int result = session.doReturningWork(new ReturningWork<Integer>() {
#Override
public Integer execute(Connection connection) throws SQLException {
CallableStatement call = connection.prepareCall("{ ? = call MYSCHEMA.MYFUNC(?,?) }");
call.registerOutParameter( 1, Types.INTEGER ); // or whatever it is
call.setLong(2, id);
call.setLong(3, transId);
call.execute();
return call.getInt(1); // propagate this back to enclosing class
}
});
http://keyurj.blogspot.com.tr/2012/12/dowork-in-hibernate.html
public static void getThroHibConnTest() throws Exception {
logger.debug("UsersActiion.getThroHibConnTest() | BEG ");
Transaction tx = null;
Connection conn = null;
CallableStatement cs = null;
Session session = HibernateUtil.getInstance().getCurrentSession();
try {
tx = session.beginTransaction();
conn = session.connection();
System.out.println("Connection = "+conn);
if (cs == null)
{
cs =
conn.prepareCall("{ ?=call P_TEST.FN_GETSUM(?,?) }");
}
cs.clearParameters();
cs.registerOutParameter(1,OracleTypes.INTEGER);
cs.setInt(2,1);
cs.setInt(3,2);
cs.execute();
int retInt=cs.getInt(1);
tx.commit();
}catch (Exception ex) {
logger.error("UsersActiion.getThroHibConnTest() | ERROR | " , ex);
if (tx != null && tx.isActive()) {
try {
// Second try catch as the rollback could fail as well
tx.rollback();
} catch (HibernateException e1) {
logger.debug("Error rolling back transaction");
}
// throw again the first exception
throw ex;
}
}finally{
try {
if (cs != null) {
cs.close();
cs = null;
}
if(conn!=null)conn.close();
} catch (Exception ex){;}
}
logger.debug("UsersActiion.getThroHibConnTest() | END ");
}
Google returns lots of people with deadlock issues in C3P0, but none of the solutions appear to apply (most people suggest setting maxStatements = 0 and maxStatementsPerConnection = 0, both of which we have).
I am using a ComboPooledDataSource from C3P0, initialised as;
cpds = new ComboPooledDataSource();
cpds.setDriverClass("org.postgresql.Driver");
cpds.setJdbcUrl("jdbc:postgresql://" + host + ":5432/" + database);
cpds.setUser(user);
cpds.setPassword(pass);
My query function looks like;
public static List<Map<String, Object>> query(String q) {
Connection c = null;
Statement s = null;
ResultSet r = null;
try {
c = cpds.getConnection();
s = c.createStatement();
s.executeQuery(q);
r = s.getResultSet();
/* parse result set into results List<Map> */
return results;
}
catch(Exception e) { MyUtils.logException(e); }
finally {
closeQuietly(r);
closeQuietly(s);
closeQuietly(c);
}
return null;
}
No queries are returning, despite the query() method reaching the return results; line. The issue is that the finally block is hanging. I have determined that the closeQuietly(s); is the line that is hanging indefinitely.
The closeQuietly() method in question is as you would expect;
public static void closeQuietly(Statement s) {
try { if(s != null) s.close(); }
catch(Exception e) { MyUtils.logException(e); }
}
Why would this method hang on s.close()? I guess it is something to do with the way I am using C3P0.
My complete C3P0 configuration (almost entirely defaults) can be viewed here -> http://pastebin.com/K8XDdiBg
MyUtils.logException(); looks something like;
public static void logException(Exception e) {
StackTraceElement ste[] = e.getStackTrace();
String message = " !ERROR!: ";
for(int i = 0; i < ste.length; i++) {
if(ste[i].getClassName().contains("packagename")) {
message += String.format("%s at %s:%d", e.toString(), ste[i].getFileName(), ste[i].getLineNumber());
break;
}
}
System.err.println(message);
}
Everything runs smoothly if I remove the closeQuietly(s); line. Both closing the ResultSet and Connection object work without problem - apart from Connection starvation of course.