Nhibernate profiler shows lots of error messages about the query plan:
Different parameter sizes result in inefficient query plan cache usage
It also leads you to an explanation in http://nhprof.com/Learn/Alerts/UncachedQueryPlan and warns you about the use of prepare_sql = true parameter when building session. I do it that way with fluent:
.ExposeConfiguration(configuration => configuration
.SetProperty("current_session_context_class", "thread_static")
.SetProperty("prepare_sql", "true")
.SetProperty("generate_statistics", "true")
)
But it seems that it isn't working as error messages are still there. Is that a limitation on OracleClientConfiguration or am I doing it wrong?
Edit To provide with some more information about this...
In my repository I do this
session.Query<TEntity>.Where(predicate).ToList();
and this is the call
var value = ParameterRepository.First(p => (p.Pipeline.Id == pipelineId && p.Name == name));
For instance those are two SQL generated from this call and that nhibernate profiler shows as "DIfferent parameter sizes result in inefficient query plan cache usage"
select GUID1_12_,
PARAMETER2_12_,
PARAMETER3_12_,
GUID4_12_
from (select pipelineex0_.GUID_PIPELINE_EXEC_PARAMETER as GUID1_12_,
pipelineex0_.PARAMETER_NAME as PARAMETER2_12_,
pipelineex0_.PARAMETER_VALUE as PARAMETER3_12_,
pipelineex0_.GUID_PIPELINE_TRACKING as GUID4_12_
from FCT_PIPELINE_EXEC_PARAMETER pipelineex0_
where pipelineex0_.GUID_PIPELINE_TRACKING = 'A5916E73CF1E406DA26F65C24BFBF694' /* :p0 */
and pipelineex0_.PARAMETER_NAME = 'lid' /* :p1 */)
where rownum <= 1 /* :p2 */
and second
select GUID1_12_,
PARAMETER2_12_,
PARAMETER3_12_,
GUID4_12_
from (select pipelineex0_.GUID_PIPELINE_EXEC_PARAMETER as GUID1_12_,
pipelineex0_.PARAMETER_NAME as PARAMETER2_12_,
pipelineex0_.PARAMETER_VALUE as PARAMETER3_12_,
pipelineex0_.GUID_PIPELINE_TRACKING as GUID4_12_
from FCT_PIPELINE_EXEC_PARAMETER pipelineex0_
where pipelineex0_.GUID_PIPELINE_TRACKING = 'A5916E73CF1E406DA26F65C24BFBF694' /* :p0 */
and pipelineex0_.PARAMETER_NAME = 'period' /* :p1 */)
where rownum <= 1 /* :p2 */
IMHO is this PARAMETER_NAME with 'lid' and 'period' that is generating different query plans.
thanks in advance
To generate the same plan each time the parameter needs to be set to the same length regardless of the parameter value.
You can customise the driver implementation to set the query parameter length to the field length specified in your mapping.
public class CustomOracleClientDriver : OracleClientDriver
{
protected override void InitializeParameter(IDbDataParameter dbParam, string name, SqlType sqlType)
{
base.InitializeParameter(dbParam, name, sqlType);
if (sqlType.LengthDefined)
dbParam.Size = sqlType.Length;
}
}
(NOTE: Inherit from OracleDataClientDriver if you are using ODP.Net)
If you are using Fluent NHibernate you register your driver implementation like this:
Fluently.Configure()
.Database(
OracleDataClientConfiguration.Oracle10
.ConnectionString(c => c.FromAppSetting("ConnectionString"))
.Driver<CustomOracleClientDriver>())
I tested this with an overridden OracleClientDriver (using the old Microsoft Oracle driver, not ODP.NET), similar to the code in the answer from mattk, and I didn't see any differences in the Oracle execution, although string parameters now had a common size.
Here's my post on Stackexchange DBA.
Oracle Enterprise Manager showed no duplicate queries for my NHibernate generated SQL, and in both versions, each call caused a parse (up to some 1000 for long testing), but very few hard parses, with no differences between variable and fixed parameter length.
In fact, Oracle created duplicate query plans only for queries without bind parameters, but with values concatenated into the SQL string (something to avoid in coded SQL). So it now seems to me that parameter size doesn't matter for Oracle, when it comes to reuse query plans (or, in Oracle terms, cursor sharing).
Oracle probably only compares the SQL string for plan matching, while SQL Server also checks the parameter definitions. You can also see a difference when looking at the dynamic SQL commands EXECUTE IMMEDIATE (Oracle) and sp_executesql (SQL Server): sp_executesql also gets a string with parameter definitions (in a string, not as parameters for the sp_executesql call itself!). I know NHibernate/ADO.NET uses sp_executesql when sending parameterized queries to SQL Server, so it likely has a different handling under SQL Server. Also, when connecting to SQL Server via NHibernate, all string parameters have unique sizes (from NHibernate mapping or default max length), so the problem has likely been fixed where relevant. Correct me if I'm wrong!
Using Prepare/prepare_sql in ADO.NET/NHibernate has some disadvantages: depending on implementation, before any SQL is executed, a Prepare request has to be sent to the database, the application has to keep a handle for the prepared statement, and it can be used only for one connection. Meaning: new handles must be created often. When I tested with Oracle and ODP.NET, it was somewhat slower than the non-prepared version, although querying by handle itself is (little) more performant than by parameterized SQL, matched on database by string equality. Likely, Prepare is good if the application uses many times the same query within the same DB connection or NHibernate session.
Related
The REST call is sending the branchId and emplId to this exec-sql-file method. I am passing these as a parameter. I am not able to execute the SQL statement when I pass branch_id = #branchid and empl_id = #emplid. But when I hardcode the branch_id = 'BR101' and empl_id = 123456 then it is working. Any suggestion how to get the branch_Id and empl_Id in my some-statements.sql?
(defn exec-sql-file
[branchid emplid]
(sql/with-db-connection (db-conn)
(sql/db-do-prepared conn
[branchid emplid (slurp (resource "sql/some-statements.sql"))])))
some-statements.sql have this query
DELETE from customer where branch_id = #branchid and empl_id = #emplid;
I am executing this from REPL as
(exec-sql-file "BR101" 123456)
I grab the code snippet from the below post.
Is it possible to patch load SQL statements from a file using clojure.java.jdbc?
There is no simple way to do this as your approach requires that you have to provide parameters to multiple SQL statements in one run. Another issue is that Java's PreparedStatement (used under the hood by clojure.java.jdbc) doesn't support named parameters, so even if parameters to multiple SQL statements done using a single prepared statement would have to be provided for every placeholder (?).
I would suggest following solutions:
use multiple prepared statements (so separate clojure.java.jdbc/execute! calls) for each of the SQL statement you want to execute wrapped in a single transaction (each SQL could be read from a separate file). You could also use some helper library like YeSQL to make loading your SQL statements from external files and exposing them as functions you could call as ordinary Clojure functions. It would be simple but if you change the number of statements you would like to execute, then you need to change your code
create a stored procedure and call them from Clojure providing the parameters - this will define an interface for some DB logic which will be defined on the DB side. Unless you change the interface of your stored procedure you can modify its implementation without changing your Clojure code or redeployment
implement your own logic of interpolating named parameters into your "multistatement" SQL file. The issue is to appropriately escape parameters' values so your code is not vulnerable to SQL injection. I would discourage this solution.
I'd like to generate the following SQL with rails / arel:
SELECT * FROM GROUPS
WHERE id = 10
CONNECT BY PARENT_ID = ID
I don't want to use plain SQL except for the last statement which is oracle specific (the real query is much more complex and I don't want to perform endless string concatenations).
What I've tried so far:
Group.where(id: 10).join('CONNECT BY PARENT_ID=ID')
This does not work as it places the custom SQL before the WHERE statement (as it assumes it's a join).
So the actual question is, how to add a custom bit of SQL to a query after the WHERE statements?
My query that I put into a prepared statement is:
select *
from ( select seq, audit_ts, message_type
from log2
where 1 = 1
and message_type in ('SOURCE', 'DEST')
order by seq desc )
where ROWNUM <= ?
When I run the query in my application, I get:
java.sql.SQLSyntaxErrorException: ORA-00907: missing right parenthesis
EDIT: Here is the java executing the query. I am trying to return a set of search results, so the prefix contains the SELECT statement and then I can have any number of suffixes (in this excerpt "AUDIT_LOG_SEARCH2") which are the parameterized WHERE clauses based on the user search:
StringBuffer query = new StringBuffer(300);
query.append(dbAdapter.getQuery("AUDIT_LOG_ENTRY_PREFIX"));
query.append(dbAdapter.getQuery("AUDIT_LOG_SEARCH2"));
// Insert parameters to complete the sql prepared statement
PreparedStatement ps = _dbConn.prepareStatement(query.toString());
ResultSet rs = ps.executeQuery();
But the query runs fine when I run it separately in SQL Developer. The query was originally created for Postgres, then updated for Oracle. Any tips?
You need to set the variables into the preparedStatement before executing.
PreparedStatement ps = _dbConn.prepareStatement(query.toString());
ps.setInt(1, 10);
Please post what query.toString() gives you if that doesn't work. Not query, but query.toString()
What are you doing in your:
// Insert parameters to complete the sql prepared statement
Are you using correctly the methods ps.setString... or whatever? Or are you just replacing the question marks? the second might be corrupting your query.
Based on #AlexPoole and #EdGibbs comments, I decided to add a bunch more debug statements. It turns out the method was being recursively called with a different sql "suffix" later on in the program if certain conditions were met. The suffix was not updated with the necessary parenthesis for the new ROWNUM wrapping the statement. So although the ORA-00907 can be thrown for many different formatting problems, it was in fact a right parenthesis that was causing me problems :P
The prefix and suffix seems like a weird pattern in this code base for creating sql queries. I'm thinking of getting rid of this and refactoring so queries don't have to be built like that. Any advice??
So for anyone else who runs into this Oracle error, I would suggest logging the sql statement you are generating and play around with it in SQL Developer. If it works in there, but not in your application, your code is probably doing something funky :P
I have a table like this one in Oracle:
create table suppliers(name varchar2(100));
With its corresponding index on upper(name):
create index supplier_name_upper_idx on suppliers(upper(name));
I would like to populate an autocomplete through AJAX, getting the information from a Servlet running JDBC queries.
This works:
PreparedStatement ps =
conn.prepareStatement(
"select * from suppliers where upper(name) like ?"
);
ps.setString(1, 'something%');
The problem is, as far as I know, PreparedStatement won't use the index because it can't know, at statement compile time, whether the parameter is going to be 'something%' (being able to get performance advantage from index) or '%something%' (not being able to get performance advantage from index).
So, my question is:
Should I use a Statement instead? If so, what's the best way to escape the input parameter (since it will come from an AJAX request)
Is there something I can use to make the PreparedStatement to use the index?
I think prepared statement is better because you will have fewer parses on server side, and as a result fewer 'latches : library cache'.
SELECT /*+ INDEX suppliers(supplier_name_upper_idx) */ * from suppliers .... should make statement use index "supplier_name_upper_idx".
I have a query with multiple calls in the same line(see code below)
The Statement object is automatically created by JDBCTemplate.
The JDBCTemplate is instantiated as new JDBCTemplate() and the datasource is looked up from the weblogic server pool.
class A implements org.springframework.jdbc.core.StatementCallback {
public Object doInStatement(Statement stmt) throws Exception {
String sql = "select * from a where pk = 'test';select * from b where pk = 'test';select * from c where pk = 'test'";
Statement stmt =
ResultSet rs = stmt.executeQuery(sql);
rs = stmt.getResultSet();
...
rs = stmt.getMoreResults();
...
rs = stmt.getMoreResults();
...
}
Using IBM's Type 2 driver, the above worked perfectly fine. We had to change the driver to Oracle's Type 4 JDBC driver and when we did that, the above broke. It does not work anymore and I get the error below:
[DAO.exec] ERROR :
java.sql.SQLException: [OWLS][DB2 JDBC Driver][DB2]ILLEGAL SYMBOL
select * from a where pk ; VALID SYMBOLS ARE BEGIN-OF-STATEMENT
Does anyone know why a Type 4 driver would not support the above? Is there a different delimeter I need to use in the statement, different way to make it work?
Note: THis code worked without any problems with the IBM type 2 JDBC driver, it failed when we switched to Oracle's Type 4 driver.
We use weblogic as the application server and DB2 database.
You need to execute multiple statements separately not compounded into one string. Most drivers will only allow a single statement to be executed. There are only a few drivers which allow multiple statements to be executed in on statement/execute.
In general you should not depend on this behavior, as it is not explicitly defined in the JDBC spec. If you read between the lines of the JDBC spec, then IMHO executing multiple statements in on statement/execute should not be supported at all, but that is debatable.
Why drivers don't support this: some database systems simply don't support preparing, executing and retrieving results of multiple statements in one statement/execute. So to support it, the driver would actually need to jump through all kinds of hoops to get it to work, even though it is not a requirement of JDBC (or IMHO: not allowed by JDBC).
To substantiate my claim that only one statement should be executed in a single statement/execute:
The object used for executing a static SQL statement (from http://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html )
In some (uncommon) situations, a single SQL statement may return multiple result sets and/or update counts http://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#execute(java.lang.String)
Similarly, the JDBC spec always talks about 'a query' when talking about statements