We are using Hibernate 3 with WebSphere v6 and DB2. We are having strange issue with wrong SQL prepared statements issued by Hibernate and causing syntax issues.
Good query issued by Hibernate:
SELECT * FROM rocc.Transaction t WHERE t.status_cd = 0 and
t.application_cd = ? order by coalesce(t.priority,0) fetch first 25
rows only
Same query issued by Hibernate incorrectly sometimes:
SELECT * FROM rocc.Transaction t WHERE t.status_cd = ? AND
t.application_cd = ? ORDER BY COALESCE(t.priority,?) FETCH first
? ROWS ONLY
It is replacing values(zero) with question mark and changing font of the keywords to uppercase (it does not matter, but mentioning here). It also changed the number 25 in fetch clause with question mark.
What is causing with the wrong prepared statement issued by Hibernate sometimes?
Hibernate is calling WSJdbcPreparedStament.executeQuery method.
This is the normal behavior of the preparedStatement on JDBC, it substitutes the parameters on the query with ?, this also helps the database to construct a optimized access plan for your query. Hibernate is just automating this.
Jdbc Reference or how we used to to before we have hibernate.
Related
We are using hibernate named query as follows :
named_query : Select this_ from TableA this_ where this_.id in( select max(id) from TableA where COLA is null group by COLB ) and rownum=1
Query query = getNamedQuery("nq.select.DirtySubject.onMaxDirtySubjectRecId");
List<SomeObject> objectList = query.list();
The query has been flagged by the DBAs and the comments verbatim are as follows
These indicate the SQL statements that are consuming the most parsing
resources every time they execute.
The SQL statements that appear in
this report are probably being reparsed. Excessively-parsed SQL
statements should be optimized to reduce their parsing frequency. This
involves using bind variables and identical statement syntax and case,
in order to be able to reuse any previously-parsed statements in the
SQL cache.
Examine these queries to see if any SQL optimization is
possible and reasonable.
Other material facts:
This query is part of a polling logic and gets fired repeatedly.
Database:Oracle 11G
Technology stack : Java,Hibernate,Tomcat,Linux,Oracle 11G
Questions:
1: Behind the scene- Hibernate will be using Prepared statement - correct ?
2: What can we possibly do more from the application side - to avoid re-parsing of this query ?
3: Anything we can do on the Database server to avoid re-parsing ?
The comment talks about parsing resources (in contrast to execution time/resources), but the statement itself does not really look that difficult to parse. The most interesting information would be the execution plan. I assume that the query is always the same (but the comment contains the word "probably").
As for question 1:
If you want to make sure that prepared statements do help, you can try to run the same query without Hibernate - just plain java.sql, PreparedStatement.
If so, there is a Hibernate prepared statement cache, but I have never used it.
As for question 2:
If this is your hotspot, you can use "direct SQL" with Hibernate (if it and your connection pool allows the caching of prepared statements) or even without it: Then you have full control.
As for question 3: Have a look on Oracle 11g result cache.
My query that I put into a prepared statement is:
select *
from ( select seq, audit_ts, message_type
from log2
where 1 = 1
and message_type in ('SOURCE', 'DEST')
order by seq desc )
where ROWNUM <= ?
When I run the query in my application, I get:
java.sql.SQLSyntaxErrorException: ORA-00907: missing right parenthesis
EDIT: Here is the java executing the query. I am trying to return a set of search results, so the prefix contains the SELECT statement and then I can have any number of suffixes (in this excerpt "AUDIT_LOG_SEARCH2") which are the parameterized WHERE clauses based on the user search:
StringBuffer query = new StringBuffer(300);
query.append(dbAdapter.getQuery("AUDIT_LOG_ENTRY_PREFIX"));
query.append(dbAdapter.getQuery("AUDIT_LOG_SEARCH2"));
// Insert parameters to complete the sql prepared statement
PreparedStatement ps = _dbConn.prepareStatement(query.toString());
ResultSet rs = ps.executeQuery();
But the query runs fine when I run it separately in SQL Developer. The query was originally created for Postgres, then updated for Oracle. Any tips?
You need to set the variables into the preparedStatement before executing.
PreparedStatement ps = _dbConn.prepareStatement(query.toString());
ps.setInt(1, 10);
Please post what query.toString() gives you if that doesn't work. Not query, but query.toString()
What are you doing in your:
// Insert parameters to complete the sql prepared statement
Are you using correctly the methods ps.setString... or whatever? Or are you just replacing the question marks? the second might be corrupting your query.
Based on #AlexPoole and #EdGibbs comments, I decided to add a bunch more debug statements. It turns out the method was being recursively called with a different sql "suffix" later on in the program if certain conditions were met. The suffix was not updated with the necessary parenthesis for the new ROWNUM wrapping the statement. So although the ORA-00907 can be thrown for many different formatting problems, it was in fact a right parenthesis that was causing me problems :P
The prefix and suffix seems like a weird pattern in this code base for creating sql queries. I'm thinking of getting rid of this and refactoring so queries don't have to be built like that. Any advice??
So for anyone else who runs into this Oracle error, I would suggest logging the sql statement you are generating and play around with it in SQL Developer. If it works in there, but not in your application, your code is probably doing something funky :P
Nhibernate profiler shows lots of error messages about the query plan:
Different parameter sizes result in inefficient query plan cache usage
It also leads you to an explanation in http://nhprof.com/Learn/Alerts/UncachedQueryPlan and warns you about the use of prepare_sql = true parameter when building session. I do it that way with fluent:
.ExposeConfiguration(configuration => configuration
.SetProperty("current_session_context_class", "thread_static")
.SetProperty("prepare_sql", "true")
.SetProperty("generate_statistics", "true")
)
But it seems that it isn't working as error messages are still there. Is that a limitation on OracleClientConfiguration or am I doing it wrong?
Edit To provide with some more information about this...
In my repository I do this
session.Query<TEntity>.Where(predicate).ToList();
and this is the call
var value = ParameterRepository.First(p => (p.Pipeline.Id == pipelineId && p.Name == name));
For instance those are two SQL generated from this call and that nhibernate profiler shows as "DIfferent parameter sizes result in inefficient query plan cache usage"
select GUID1_12_,
PARAMETER2_12_,
PARAMETER3_12_,
GUID4_12_
from (select pipelineex0_.GUID_PIPELINE_EXEC_PARAMETER as GUID1_12_,
pipelineex0_.PARAMETER_NAME as PARAMETER2_12_,
pipelineex0_.PARAMETER_VALUE as PARAMETER3_12_,
pipelineex0_.GUID_PIPELINE_TRACKING as GUID4_12_
from FCT_PIPELINE_EXEC_PARAMETER pipelineex0_
where pipelineex0_.GUID_PIPELINE_TRACKING = 'A5916E73CF1E406DA26F65C24BFBF694' /* :p0 */
and pipelineex0_.PARAMETER_NAME = 'lid' /* :p1 */)
where rownum <= 1 /* :p2 */
and second
select GUID1_12_,
PARAMETER2_12_,
PARAMETER3_12_,
GUID4_12_
from (select pipelineex0_.GUID_PIPELINE_EXEC_PARAMETER as GUID1_12_,
pipelineex0_.PARAMETER_NAME as PARAMETER2_12_,
pipelineex0_.PARAMETER_VALUE as PARAMETER3_12_,
pipelineex0_.GUID_PIPELINE_TRACKING as GUID4_12_
from FCT_PIPELINE_EXEC_PARAMETER pipelineex0_
where pipelineex0_.GUID_PIPELINE_TRACKING = 'A5916E73CF1E406DA26F65C24BFBF694' /* :p0 */
and pipelineex0_.PARAMETER_NAME = 'period' /* :p1 */)
where rownum <= 1 /* :p2 */
IMHO is this PARAMETER_NAME with 'lid' and 'period' that is generating different query plans.
thanks in advance
To generate the same plan each time the parameter needs to be set to the same length regardless of the parameter value.
You can customise the driver implementation to set the query parameter length to the field length specified in your mapping.
public class CustomOracleClientDriver : OracleClientDriver
{
protected override void InitializeParameter(IDbDataParameter dbParam, string name, SqlType sqlType)
{
base.InitializeParameter(dbParam, name, sqlType);
if (sqlType.LengthDefined)
dbParam.Size = sqlType.Length;
}
}
(NOTE: Inherit from OracleDataClientDriver if you are using ODP.Net)
If you are using Fluent NHibernate you register your driver implementation like this:
Fluently.Configure()
.Database(
OracleDataClientConfiguration.Oracle10
.ConnectionString(c => c.FromAppSetting("ConnectionString"))
.Driver<CustomOracleClientDriver>())
I tested this with an overridden OracleClientDriver (using the old Microsoft Oracle driver, not ODP.NET), similar to the code in the answer from mattk, and I didn't see any differences in the Oracle execution, although string parameters now had a common size.
Here's my post on Stackexchange DBA.
Oracle Enterprise Manager showed no duplicate queries for my NHibernate generated SQL, and in both versions, each call caused a parse (up to some 1000 for long testing), but very few hard parses, with no differences between variable and fixed parameter length.
In fact, Oracle created duplicate query plans only for queries without bind parameters, but with values concatenated into the SQL string (something to avoid in coded SQL). So it now seems to me that parameter size doesn't matter for Oracle, when it comes to reuse query plans (or, in Oracle terms, cursor sharing).
Oracle probably only compares the SQL string for plan matching, while SQL Server also checks the parameter definitions. You can also see a difference when looking at the dynamic SQL commands EXECUTE IMMEDIATE (Oracle) and sp_executesql (SQL Server): sp_executesql also gets a string with parameter definitions (in a string, not as parameters for the sp_executesql call itself!). I know NHibernate/ADO.NET uses sp_executesql when sending parameterized queries to SQL Server, so it likely has a different handling under SQL Server. Also, when connecting to SQL Server via NHibernate, all string parameters have unique sizes (from NHibernate mapping or default max length), so the problem has likely been fixed where relevant. Correct me if I'm wrong!
Using Prepare/prepare_sql in ADO.NET/NHibernate has some disadvantages: depending on implementation, before any SQL is executed, a Prepare request has to be sent to the database, the application has to keep a handle for the prepared statement, and it can be used only for one connection. Meaning: new handles must be created often. When I tested with Oracle and ODP.NET, it was somewhat slower than the non-prepared version, although querying by handle itself is (little) more performant than by parameterized SQL, matched on database by string equality. Likely, Prepare is good if the application uses many times the same query within the same DB connection or NHibernate session.
yesterday I've been trying to make this code work inspite the fact it's just working fine with nhibernate and SQL server but when it come to oracle it generate wrong sql
UnitOfWork.Current.CreateCriteria<Bill>().Add(Restrictions.IsEmpty("ReqId"))
.SetMaxResults(BatchSize).SetLockMode(LockMode.Upgrade).List<Bill>();
the generated SQL will something like
Select * from
(Select bill_0.id,bill_0.BillNo ...... from Bill bill_0 where bill_0.reqId is Not null )
where ROWNUM < 10 for UPDATE of bill_0.ID
so i wont run because the allies bill_o is defined inside the inner sql statement so who got the solution ?
the correct sql would be something like this which i tried and worked on oracle db
Select bill_0.id,bill_0.BillNo ...... from Bill bill_0
where bill_0.reqId is Not null and ROWNUM < 10 for UPDATE of bill_0.ID
Since, as you say, NHibernate is generating invalid Oracle SQL, I suggest you file a bug with the NHibernate people. The SQL would work if the in-line view had been assigned an alias of "bill_0", or if the FOR UPDATE clause didn't use a table alias ("for UPDATE of ID"). Whether you can modify your NHibernate calls to make either of these happen I'm afraid I have no idea.
does statement object contain the session id the database returns for the current session? What does a resultset
contain?
To the best of my knowledge, no, Statements do not have session IDs. It seems like the Java API specifications for the Statement class backs that up. Basically, Statements are used to execute SQL statements by specifying a SQL query through the execute method.
A ResultSet is used to retrieve results which are returned by executing a query via a Statement or PreparedStatement.
The JDBC(TM) Database Access trail of The Java Tutorials contains some information on these topics. The following sections may be of interest:
Lesson: JDBC Basics
Updating Tables
Retrieving Values from Result Sets