When generating an explain plan for a query my system executes, I notice that if I leave filters in their parameterized form (e.g. "somecolumn=:param1") the explain plan is different when I replace the parameter with a real value (e.g. "somecolumn='real_value'). In my case, the explain plan with the database parameters ends up ignoring indexes and does full table scans on massive tables leading to high cost, bytes and estimated rows.
Why do the plans differ? Is Oracle using the less than desirable plan because the system uses bind parameters?
Related
In our application we have several queries that take a long time to parse (> 500 ms, sometimes much more) but are fast to execute (< 50 ms). This is due to complicated views and generally not a problem, if the parsed Query is cached by Oracle.
We now run into the problem that some Queries are parsed every time they are executed: These Queries select from a View in one Schema (SCHEMA1) and use a Function from a Package in a different Schema (SCHEMA2) in the Select clause.
When we execute this Query, it is parsed on every execution. In V$SQLAREA the VERSION_COUNT is equal to the number of executions. Every execution takes the long time.
If we warp the call to the Function from SCHEMA1 in a local Function in SCHEMA2 and use the new Function in the Query, only the first execution leads to a parse. All subsequent executions are much faster. In V$SQLAREA we see a VERSION_COUNT of 1 (or some number much lower than the number of executions).
Unfortunaly, wrapping the functions in local functions are highly impractical in our use case, because there are many functions in SCHEMA2 and they are used with Views von several other Schemas.
The Query doesn’t contain parameters and the circumstances of execution are exactly the same every time.
The effect dos not depend on the code in the function: if we replace the actual function with a test function that just returns a constant value, we get the same effect:
It doesn’t make a different whether we execute the Query from SCHEMA1, SCHEMA2 or any third Schema, except when we execute it as SYSDBA. In this case, subsequent executions don’t lead to new parses.
We use Oracle 12c Release 12.1.0.2.0.
Update: V$SQL_SHARED_CURSOR displays Y at AUTH_CHECK_MISMATCH for these queries. I am not sure yet what this means.
This can be caused by various reasons.
Different sql execution environment. For example caller uses different datatypes for bind variables. Check the view V$SQL_SHARED_CURSOR.
Different optimizer goal
Bind variable peeking. Check v$sql.is_bind_aware.
Some bug in the database caused by cursor_sharing parameter
Maybe you should generate AWR and check wait events, and also the size of Library cache. Possibly Oracle might have problems to find some free space in Library cache for your query(is it's really complex).
I'm in the need to rewrite some query made by OBIEE
My need is to change all the literal value in bind variable but I didn't find how to use bind variables
Someone can help me?
Thanks
First, a statement:
If you are utilizing ADF (Oracle Application Development Framework) with OBIEE, there is a setting in the ADF layer which allows us to specify how the View Criteria and the WHERE clauses used in the queries fired on the VOs should be handled.
By default the setting useBindVarsForViewCriteriaLiterals is set to "False" in adf-config.xml. If this setting is False, then ADF will generate SQLs with literals for the view criteria and this can cause contention in the shared pool area of the database.
If we change the setting in adf-config.xml to "True" ADF generates SQL with bind variables for all view criterias.
However this setting should not be changed for BI (OBIEE), as BI does not support Bind variables in the queries. If we see any queries (or its related logs) which are using Bind variables in OBIEE queries/reports, then its possible that it is due to the above setting.
For BI, this setting should be left to the default value, i.e. "False"
Quick answer: It would seem that OBIEE is not capable of using bind variables in place of literals.
Now, the reasoning behind the restriction on bind variables:
In a data warehouses, instead of running say 1,000 statements
per second, they do something like take an average of 100 seconds to run a single query.
In these systems, the queries are few but big (they ask large questions). Here, the
overhead of the parse time is a tiny fraction of the overall execution time. Even if you
have thousands of users, they are not waiting behind each other to parse queries, but
rather are waiting for the queries to finish getting the answer.
In these systems, using bind variables may be counterproductive. Here, the runtimes for
the queries are lengthy-in seconds, minutes, hours, or more. The goal is to get the best
query optimization plan possible to reduce the runtime, not to execute as many of OLTP,
one-tenth-second queries as possible. Since the optimizer's goal is different, the rules
change.
Sometimes using a bind variable forces the optimizer to come up with the best generic plan, which actually might be the worst plan for the specific query. In a system where the queries take considerable time to execute, bind variables remove information the optimizer could have used to come up with a superior plan. In fact, some data warehouse-specific features are defeated by using bind variables. For example, Oracle supports a star transformation feature for data warehouses that can greatly reduce the time a query takes. However, one restriction that precludes star transformation is having queries that contain bind variables.
I have a vey huge query. It is rather large, so i will not post it here(it has 6 levels of nested queries with ordering and grouping). Query has 2 parameters that are passed to it via PreparedStatement.setString(index, value). When I execute my query through SQL Developer(replacing query parameters to actual values before it by hand) the query runs about 10 seconds and return approximately 15000 rows. But when I try to run it through java program using PreparedStament with varibales it fails with ORA-01652(unable to extend temp segment). I have tried to use simple Statement from java program - it works fine. Also when I use preparedStatement without variables(don't use setString(), but specify parameters by hand) it works fine too.
So, I suspect that problem is in PreparedStatemnt parameters.
How does the mechanism of that parameters work? Why simple statement works fine but prepared one fails?
You're probably running into issues with bind variable peeking.
For the same query, the best plan can be significantly different depending on the actual bind variables. In 10g, Oracle builds the execution plan based on the first set of bind variables used. 11g mostly fixed this problem with adaptive cursor sharing, a feature that creates multiple plans for different bind variables.
Here are some ideas for solving this problem:
Use literals This isn't always as bad as people assume. If the good version of your query runs in 10 seconds, the overhead of hard-parsing the query will be negligible. But you may need to be careful to avoid SQL injection.
Force a hard-parse There are a few ways to force Oracle to hard-parse every query. One method is to call DBMS_STATS with NO_INVALIDATE=>FALSE on one of the tables in the query.
Disable bind-variable peeking / hints You can do this by removing the relevant histograms, or using one of the parameters in the link provided by OldProgrammer. This will stabilize your plan, but will not necessarily pick the correct plan. You may also need to use hints to pick the right plan. But then you may not have the right plan for every combination of inputs.
Upgrade to 11g This may not be an option, but this issue is another good reason to start planning an upgrade.
I'm trying to make sure I have a good understanding regarding the relationship between CURSOR_SHARING, bind variables, bind variable peeking and histograms as most sources cover these topics is different sections.
Ok so here's what I've gathered so far, feel free too correct me if I got anything wrong:
CURSOR_SHARING
1. = EXACT (default)
1.1. if SQL statement uses literals: the optimizer will generate a new execution plan for every combination of literals - optimizer will not replace literals with binds. A new parent cursor is generated for every literal combination.
1.2. if SQL statement uses bind variables: first time the statment is run, the optimizer will peek at the value of the bind variables and use those specific values to generate an execution plan - all future statements with those bind variables will use that same plan (even if the plan is suboptimal for other values of the bind variable).
2. = FORCE
2.1. optimizer will replace all literals with binds - and will basically use the same algorithm as scenario 1.2
3. = SIMILAR
3.1. no histogram: optimizer replaces all literals with binds -> same final effect as with 1.2 and 2.1
3.2. with histogram: optmizer replaces all literals with binds, but peeks at the bind variable EVERY time the statement is run (as opposed to just on the first run through) to see if there is a more optimal execution plan for that specific value of the bind variable (based on histogram statistics). Therefore, a new child cursor is effectively created for every distinct value of the bind variable that the optimizer encounters.
Questions:
From my understanding, isn't using CUSOR_SHARING = EXACT + writing SQL statments with bind-variables (1.2) lead to the exact same outcome as setting CURSOR_SHARING = FORCE (2.1)? In both cases, the optimizer will only peek at the bind variable on the first run to generate the execution plan and then reuse that plan no matter what the values of the bind variables on subsequent runs? If so, then why do most sources recommend using bind variables? this seems like it could have a significant impact on performance.
Is the histogram used in the initial bind variable peek for 1.2 and 2.1? As in, the first time that SQL statment is run and the optimizer peeks at the bind variable, does it use the histogram (if there is one) to determine if full-table scan or index scan is used? "Oracle Database 11g, Performance Tuning Recipes" seems to indicate that histograms are relevant only when CURSOR_SHARING = SIMILAR but some other sources indicate that the histogram is used in all the other CURSOR_SHARING settings as well.
In case 1.1, would the optimizer make use the histogram to determine the best execution plan? Basically I just want to know when the histogram is used. Is it only when CURSOR_SHARING = SIMILAR or for other CURSOR_SHARING settings are well?
Adpative Cursor Sharing - this feature will only take place if there are bind variables (either from user query or system-generated (by literal replacements)). Therefore it only takes place in 1.2, 2.1, 3.1 and 3.2? but since SIMILAR has been deprecated, does this mean that ACS only occurs in 1.2 and 2.1?
Hopefully, I'm not too far off base right now but if I made any mistakes please do correct me
Thanks!
Edited by: BYS2 on Dec 20, 2011 12:11 PM
The difference between (a) using FORCE and (b) using EXACT and coding bind variable yourself, is that in the latter case you control when bind variables are used. So if you can see that in a particular case bind variables are hurting performance, or aren't necessary, you can change that query. With FORCE, you're stuck. The reason that bind variables are recommended for OLTP type queries is that parsing is a highly serialized process that can become a big bottleneck. In OLTP systems you tend to see lots of queries that should always be using the same execution plan run with different values, so re-parsing them all the time is a waste. Any good source will also recommend that you consider when to not use bind variables -- for instance, if you only have a few possible values that can appear at a particular position in a query, and one or more of those values might benefit from a different execution plan, it may be better overall to use literals since you can parse each variant once and then reuse the cached plan.
(Another benefit to using bind variables is that it leaves you less open to SQL injection.)
2 & 3. Histograms are used in general when creating execution plans for queries, and in more ways than are obvious. Yes, in the case of a standard bind variable peek with EXACT setting, the histogram is (or at least, may be) used by the optimizer in determining the execution plan. This can be a good thing or a bad thing, depending on the skew and what particular value you have for the bind. I think the point your source is making about histograms and the SIMILAR setting is that in that case, the presence of the histogram is one of the triggers that will cause a new execution plan to be created.
(I would highly recommend Jonathan Lewis's "Cost-Based Oracle Fundamentals" for all the information you could want about when and how histograms are used.)
4.. I believe that Adaptive Cursor Sharing is essentially an enhanced version of the logic that was previously implemented for CURSOR_SHARING=SIMILAR. The optimizer will consider creating new plans based on bind variable peeking, in all circumstances. SIMILAR appears to still exist as an option. This post may provide some further helpful info.
How oracle CACHE a Query (SQL), Query execution contains the following steps.
1. PARSE
2. Execute
3. Fetch
in First step oracle checks if query exists in CACHE (Shared Pool) or not (It will be exists if query is identical and based on LRU), if exists then PARSING will be skipped and execution will start.
So to make query performance intensive we must use bind variables and use the identical sql queries.
But in Parsing oracle verifies authentication also (User Access), if multiple users are using the same query, how oracle skip/use parsing?
The parsing of a query is not tied to a user, it is dependent on the query. Note that an exact character for character match is required. Blanks and comments in a query will cause it to miss the Shared Pool match.
The parse-tree is then used in the generation of an execution plan. If the same schema is used in the new query as the matched query then the existing execution plan is used.
You can test this by creating multiple schema, one with a small amount of data and one with a large
amount. Then analyze all the tables. Take a look at the execution plans for the same query with vastly different amounts of data. This will show the different execution plans for the same query.
Now run the query a large amount of times and check the amount of time that it takes for the first
and then subsequent executions. Use Oracle Trace and look in the left hand pain for the "Re-Parse"
frequency. This can also be gleaned from some of the dictionary tables.
Take a look at The Oracle documentation on using Oracle Trace
First step oracle checks if query exists in CACHE (Shared Pool) or not (It will be exists if query is identical and based on LRU), if exists then PARSING will be skipped and execution will start. So to make query performance intensive we must use bind variables and use the identical sql queries.
This is actual process when you execute a query on Oracle:
Parsing Steps
Syntax Check
Semantic Analysis
Has the query been executed in some other session?
Hard Parse
Parse
Optimize
Generate the plan for the query.
If the answer to #1.3 is yes - Oracle skips the hard parse portion, and uses the existing query plan.
For more info:
* AskTom: Difference between soft parse and hard parse
* Bind variables - The key to application performance
Usual practice in Oracle is to create stored procedures with definer rights which means that the queries are executed with privileges of their definer, despite of who calls them. That's why cache works well.
If you create a procedure or package with invoker rights (authid current_user), then the queries will be parsed for each invoker separately.
See Invoker Rights Versus Definer Rights for details.