I'm writting some stored functions in Oracle. One of these is a really basic function who take a string as parameter and return an another string. Here is my function:
CREATE OR REPLACE
FUNCTION get_mail_custcode (
custcodeParam IN customer_table.custcode%TYPE)
RETURN VARCHAR2
IS
mail_rc contact_table.email%TYPE;
BEGIN
SELECT cc.email
INTO mail_rc
FROM contact_table cc, customer_table cu
WHERE cu.customer_id = cc.customer_id
AND cu.custcode like custcodeParam ;
RETURN mail_rc ;
END;
So it's not working.. The function seems to work well but is executed without any end.. The function is working time and time, I manually cancel the operation after 2 or 3 minutes (this query give normally instant result).
After writing the query again and again I finally (and randomly) change the cu.custcode like custcodeParam into a cu.custcode = custcodeParam and it is working!!
So my question is why? Why I can't use a like comparator in a stored function? Why this makes no error but the function run indefinitly.
Thanks.
Cursors are all treated identically in Oracle. A query in a function will be treated exactly the same as a query you enter manually through SQL*Plus.
However, what may differ in your example is how Oracle works with variables. The following two queries are fundamentally different to the optimizer:
SELECT * FROM tab WHERE code LIKE 'FOO%';
and
variable v_code VARCHAR2(4)
EXEC :v_code := 'FOO%';
SELECT * FROM tab WHERE code LIKE :v_code;
In the first case the optimizer looks at the constant FOO% and can instantly tell that an index on code is perfectly suited to retrieve the rows rapidly via an index RANGE SCAN.
In the second case, the optimizer has to consider that :V_CODE is not constant. The purpose of the optimizer is to determine a plan for a query that will be shared by successive executions of the same query (because computing a plan is expensive).
The behaviour of the optimizer will depend upon your version of Oracle:
In old Oracle versions (9i and before), the value of the variable was ignored to build the plan. In effect Oracle had to build a plan that would be efficiently, whatever value was passed to it. In your case this likely would result in a full scan because Oracle had to take the least risky option and consider that FOO% was as likely a value as %FOO (the latter can't be accessed efficiently via an index range scan).
In 10g Oracle introduced bind peeking: now the optimizer can access the value of the variable and produce a suitable plan. The main problem is that in most cases a query can only have one plan, which means that the value of the first variable ever passed to the function will force the execution plan for all further executions. If the first value to be passed is %FOO, a FULL SCAN will likely be chosen.
In 11g, Oracle has "intelligent cursor sharing": a single query can share more than one plan, in the example above the value FOO% would use a RANGE SCAN while %FOO would probably use a FULL SCAN.
What version of Oracle are you using?
update
In 10g and before if this function is used often without wildcards you should rewrite it to acknowledge the optimizer behaviour:
BEGIN
IF instr(custcodeParam, '%') > 0 OR instr(custcodeParam, '_') > 0 THEN
SELECT cc.email
INTO mail_rc
FROM contact_table cc, customer_table cu
WHERE cu.customer_id = cc.customer_id
AND cu.custcode LIKE custcodeParam;
ELSE
SELECT cc.email
INTO mail_rc
FROM contact_table cc, customer_table cu
WHERE cu.customer_id = cc.customer_id
AND cu.custcode = custcodeParam;
END IF;
RETURN mail_rc;
END;
Related
I have a performance problem.
First PL/SQL (most time never ends and OS database process is always over 90%):
DECLARE
myId nvarchar2(10) := '0;WF21izb0';
BEGIN
insert into MY_TABLE (select * from MY_VIEW where ID = myId);
END;
Second PL/SQL (ends with successfull result in 50s):
BEGIN
insert into MY_TABLE (select * from MY_VIEW where ID = '0;WF21izb0');
END;
select count(*) from MY_VIEW
is also a not ending call, there are a lot of table joins behind this view.
select count(*) from MY_VIEW where ID = '0;WF21izb0'
ends in 50s with count=60000.
Can somebody explain me the reason why my first PL/SQL is not finishing after 50s? What is the difference between using static string and declared parameter?
It boils down to what the DB engine knows about your data and your query, when preparing the query execution plan.
When a literal is placed in your query, it is a part of your query, so is known to the engine responsible for preparing the plan. It can take that literal value into account and decide on an execution plan, that is suitable, e.g. based on the DB data statistics (e.g. that this value is rare).
When you are using a PL/SQL variable, the actual query, for which the plan is determined, is different. It's something like:
insert into MY_TABLE (select * from MY_VIEW where ID = :param)
As you can see, the DB engine has now no information on the value, which will be used, when the query gets executed. So the best plan for such a scenario, is to prepare something, which is averagely good for most of the probable values (i.e. see what values in the DB will match this place most often, i.e. values that are prevalent).
If your data is unbalanced, and the '0;WF21izb0' value is rare (or even non-existent) in your data, a selective index may be used to narrow down, what needs to be processed, relatively soon in the critical parts of the execution plan. This plan will however backfire, when you'll use a value, which is all over the place - use of the index will be counter-productive. A better plan for such case may be a full table scan. Possibly the same one, which is used when executing select count(*) from MY_VIEW.
If you are faced with a scenario, where you do not know the filtering value upfront, you'll have to analyze the view code, and try to adjust it so it can be effectively used also for less "selective" values. You could try applying some optimizer hints to the query. You could also resign from using a view, and try your luck with a tabular function, where you can push your filtering predicates to the exact spots of the query, where they can be used most effectively.
Edit:
All in all, follow the advises from the question comments, and examine your execution plans and execution profile data. You should be able to find the culprit. From there it may not be obvious, what the solution is, but still, you know your data and relations much better than us.
I was checking some traces, but after reading the comment of APC and the answer of Hilarion i end up in this solution:
declare
sql_stmt VARCHAR2(200);
id VARCHAR2(10) := '0;WF21izb0';
BEGIN
sql_stmt := 'insert into MY_TABLE (select * from MY_VIEW where ID = :1)';
EXECUTE IMMEDIATE sql_stmt using id;
END;
This is done in 50s, and id can be now a function/procedure parameter.
Thanks for the comments.
First, I want to make it clear that the question is not about the materialized views feature.
Suppose, I have a table function that returns a pre-defined set of columns.
When a function call is submitted as
SELECT col1, col2, col3
FROM TABLE(my_tfn(:p1))
WHERE col4 = 'X';
I can evaluate the parameter and choose what queries to execute.
I can either open one of the pre-defined cursors, or I can assemble my query dynamically.
What if instead of evaluating the parameter I want to evaluate the text of the requesting query?
For example, if my function returns 20 columns but the query is only requesting 4,
I can assign NULLs to remaining 16 clumns of the return type, and execute fewer joins.
Or I can push the filter down to my dynamic query.
Is there a way to make this happen?
More generally, is there a way to look at the requesting query before exuting the function?
There is no robust way to identify the SQL that called a PL/SQL object.
Below is a not-so-robust way to identify the calling SQL. I've used code like this before, but only in special circumstances where I knew that the PL/SQL would never run concurrently.
This seems like it should be so simple. The data dictionary tracks all sessions and running SQL. You can find the current session with sys_context('userenv', 'sid'), match that to GV$SESSION, and then get either SQL_ID and PREV_SQL_ID. But neither of those contain the calling SQL. There's even a CURRENT_SQL in SYS_CONTEXT, but it's only for fine-grained auditing.
Instead, the calling SQL must be found by a string search. Using a unique name for the PL/SQL object will help filter out unrelated statements. To prevent re-running for old statements, the SQL must be individually purged from the shared pool as soon as it is found. This could lead to race conditions so this approach will only work if it's never called concurrently.
--Create simple test type for function.
create or replace type clob_table is table of clob;
--Table function that returns the SQL that called it.
--This requires elevated privileges to run.
--To simplify the code, run this as SYS:
-- "grant execute on sys.dbms_shared_pool to your_user;"
--(If you don't want to do that, convert this to invoker's rights and use dynamic SQL.)
create or replace function my_tfn return clob_table is
v_my_type clob_table;
type string_table is table of varchar2(4000);
v_addresses string_table;
v_hash_values string_table;
begin
--Get calling SQL based on the SQL text.
select sql_fulltext, address, hash_value
bulk collect into v_my_type, v_addresses, v_hash_values
from gv$sql
--Make sure there is something unique in the query.
where sql_fulltext like '%my_tfn%'
--But don't include this query!
--(Normally creating a quine is a challenge, but in V$SQL it's more of
-- a challenge to avoid quines.)
and sql_fulltext not like '%quine%';
--Flush the SQL statements immediately, so they won't show up in next run.
for i in 1 .. v_addresses.count loop
sys.dbms_shared_pool.purge(v_addresses(i)||', '||v_hash_values(i), 'C');
end loop;
--Return the SQL statement(s).
return v_my_type;
end;
/
Now queries like these will return themselves, demonstrating that the PL/SQL code was reading the SQL that called it:
SELECT * FROM TABLE(my_tfn) where 1=1;
SELECT * FROM TABLE(my_tfn) where 2=2;
But even if you go through all this trouble - what are you going to do with the results? Parsing SQL is insanely difficult unless you can ensure that everyone always follows strict syntax rules.
I have a question regarding how queries executed through
'execute immediate' is treated in the library cache (We use Oracle 11).
Let's say I have a function like this:
FUNCTION get_meta_map_value (
getfield IN VARCHAR2,
searchfield IN VARCHAR2,
searchvalue IN VARCHAR2
) RETURN VARCHAR2 IS
v_outvalue VARCHAR2(32767);
sql_stmt VARCHAR2(2000) := 'SELECT '||getfield||' FROM field_mapping, metadata '||
'WHERE field_mapping.metadataid = metadata.metadataid AND rownum = 1 AND '||searchfield||' = :1';
BEGIN
EXECUTE IMMEDIATE sql_stmt INTO v_outvalue USING searchvalue;
...
The getfield and searchfield are in one installation always the same (but has other values in another installation, so that is why we use dynamic sql)
So this leaves us with an sql that only differs in the searchvalue (which is a parameter).
This function is called in a loop that executes x times, from inside another stored procedure.
The stored procedure is executed y times during the connection life time, through ODBC connection.
And there are z connections, but each of them uses the same database login.
Now let us also assume that the searchvalue changes b times during one loop.
Question 1:
When calculating how many copies of the sql will be kept in the library cache,
can we disregard the different values the searchvalue can have (b), as the value is sent as a parameter to execute immediate?
Question 2:
Will the loop cause a hard parse of the query x times (query will be created in library cache x times), or can Oracle reuse the query?
(We assume that the searchvalue is the same for all calls in this question here, for simplicity)
Question 3:
Does the y (number of times the stored procedure is called from odbc during the lifetime of one connection)
also multiply the amount of copies of the query that are kept in library cache?
Question 4:
Does the z (number of simultaneous connections with same db login)
multiply the amount of copies of the query that are kept in library cache?
Main question:
What behaviour should I expect here?
Is the behaviour configurable?
The cause for this question is that we have had this code is production for 4 years, and now one of our customer gets back to us and says "This query fills our whole SGA, and Oracle says it's your fault".
The number of different combinations of getfield and searchfield should determine how many "copies" there will be. I use teh word "copies" cautiously because Oracke will treat each variation as distinct. Since you are using a bind variable for searchvalue so however many values you have for this will not add to the query count.
In short, it looks like your code is OK.
Number of connections should not increase the hard parses.
Ask for a AWR report to see exactly how many of these queries are in the SGA, and how many hard parses are being triggered.
I will disagree that the number of connections will not increase the hard parse count for the posted code because the last I knew dynamic SQL cannot be shared between sessions. Since the generated SQL uses a bind variable it should generate a reusable statement by the session, but it will not be sharable between user sessions. As a general rule dynamic SQL should be used only for infrequently executed statements. You may want to refer to the following:
- -
Designing applications for performance and scalability An Oracle White Paper July 2005
https://www.oracle.com/technetwork/database/performance/designing-applications-for-performa-131870.pdf
- -
enter code here
First off, my background is in SQL Server. Using CTEs (Common Table Expressions) is a breeze and converting it to a stored procedure with variables doesn't require any changes to the structure of the SQL other than replacing entered values with variable names.
In Oracle PL/SQL however, it is a completely different matter. My CTEs work fine as straight SQL, but once I try to wrap them as PL/SQL I run into a host of issues. From my understanding, a SELECT now needs an INTO which will only hold the results of a single record. However, I am wanting the entire recordset of multiple values.
My apologies if I am missing the obvious here. I'm thinking that 99% of my problem is the paradigm shift I need to make.
Given the following example:
NOTE: I am greatly over simplifying the SQL here. I do know the below example can be done in a single SQL statement. The actual SQL is much more complex. It's the fundamentals I am looking for here.
WITH A as (SELECT * FROM EMPLOYEES WHERE DEPARTMENT = 200),
B as (SELECT * FROM A WHERE EMPLOYEE_START_DATE > date '2014-02-01'),
C as (SELECT * FROM B WHERE EMPLOYEE_TYPE = 'SALARY')
SELECT 'COUNTS' as Total,
(SELECT COUNT(*) FROM A) as 'DEPT_TOTAL',
(SELECT COUNT(*) FROM B) as 'NEW_EMPLOYEES',
(SELECT COUNT(*) FROM C) as 'NEW_SALARIED'
FROM A
WHERE rowcount = 1;
Now if I want to make this into PL/SQL with variables that are passed in or predefined at the top, it's not a simple matter of declaring the variables, popping values into them, and changing my hard-coded values into variables and running it. NOTE: I do know that I can simply change the hard-coded values to variables like :Department, :StartDate, and :Type, but again, I am oversimplifying the example.
There are three issues I am facing here that I am trying to wrap my head around:
1) What would be the best way to rewrite this using PL/SQL with declared variables? The CTEs now have to go INTO something. But then I am dealing with one row at a time as opposed to the entire table. So CTE 'A' is a single row at a time, and CTE B will only see the single row as opposed to all of the data results of A, etc. I do know that I will most likely have to use CURSORS to traverse the records, which somehow seems to over complicate this.
2) The output now has to use DBMS_OUTPUT. For multiple records, I will have to use a CURSOR with FETCH (or a FOR...LOOP). Yes?
3) Is there going to a big performance issue with this vs. straight SQL in regards to speed and resources used?
Thanks in advance and again, my apologies if I am missing something really obvious here!
First, this has nothing to do with CTEs. This behavior would be the same with a simple select * from table query. The difference is that with T-SQL, the query goes into an implicit cursor which is returned to the caller. When executing the SP from Management Studio this is convenient. The result set appears in the data window as if we had executed the query directly. But this is actually non-standard behavior. Oracle has the more standard behavior which might be stated as "the result set of any query that isn't directed into a cursor must be directed to variables." When directed into variables, then the query must return only one row.
To duplicate the behavior of T-SQL, you just have to explicitly declare and return the cursor. Then the calling code fetches from the cursor the entire result set but one row at a time. You don't get the convenience of Sql Developer or PL/SQL Developer diverting the result set to the data display window, but you can't have everything.
However, as we don't generally write SPs just to be called from the IDE, it is easier to work with Oracle's explicit cursors than SQL Server's implicit ones. Just google "oracle return ref cursor to caller" to get a whole lot of good material.
Simplest way is to wrap it into an implicit for loop
begin
for i in (select object_id, object_name
from user_objects
where rownum = 1) loop
-- Do something with the resultset
dbms_output.put_line (i.object_id || ' ' || i.object_name);
end loop;
end;
Single row query without the need to predefine the variables.
Consider a deterministic function like:
CREATE OR REPLACE FUNCTION SCHEMA.GET_NAME(ss_id nvarchar2
) RETURN nvarchar2 DETERMINISTIC IS
tmpVar nvarchar2(500);
BEGIN
select name into tmpvar from logistics.organization_items
where id = ss_id ;
return tmpvar ;
END ss_name;
Using Toad I called the SCHEMA.GET_NAME(1) and it returns A. I then changed the value from the table from A to B and recalling the SCHEMA.GET_NAME(1) returned B.
It is a good result. But I'm afraid of the value not being updated according to this page in the documentation, which said:
When Oracle Database encounters a deterministic function in one of these contexts, it attempts to use previously calculated results when possible rather than reexecuting the function. If you subsequently change the semantics of the function, you must manually rebuild all dependent function-based indexes and materialized views.
In what situations would the value of GET_NAME(1) return an old cached value (A instead of B)?
If you select from a table then the results of your function are not deterministic. A deterministic system is one that will always produce the same output, given the same initial conditions.
It is possible to alter the information in a table, therefore a function that selects from a table is not deterministic. To quote from the PL/SQL Language Reference:
Do not specify this clause to define a function that uses package variables or that accesses the database in any way that might affect the return result of the function. The results of doing so are not captured if the database chooses not to reexecute the function.
In other words, Oracle does not guarantee that the results of the function will be accurate (they just might be). If your table is static, and unlikely to ever change, then it should be okay but this is not something I'd ever like to rely on. To answer your question, do not assume that Oracle will return anything other than the cached value within the same transaction/session.
If you need to speed this up there are two ways. Firstly, check that you have an index on ID!
Simply JOIN to this table. If your function is only this then there is no need for the function to exist.
Use scalar sub-query caching (not necessarily possible but worth the try).
select ( select get_name(:id) from dual )
from your_table
Oracle will create an in-memory hash of the results of the function, like a result cache. If you're executing the same function multiple times then Oracle will hit the cache rather than the function.
Ben's answer sums it up nicely, and I would just like to add that the way you used DETERMINISTIC keyword inside your function is not right - keeping in view that you are reading the value from a table and then returning the same to the user.
A deterministic function should be used in cases, where you are evaluating an expression over a fixed input, for example, when you need to return a substring, or upper/lower case for the input string. Programatically, you know that for the same input the lowercase function will always return the same value, and so you would like to cache the result (using deterministic keyword).
When you read a value from a table, Oracle has no way to know that the value in the column has not changed, and so it prefers to rexecute the function and not depend on the cached result (which makes sense)
Can you add a timestamp parameter to your function? Then pass in sysdate to the function from wherever you're calling it.
This way, you're effectively caching the result and you avoid running the function over and over when it generally returns the same value within a given transaction.
The remark of Erez is the answer I was looking for.
Before executing the query or plsql-unit you can with this solution force to execute the function again after you reseted the ret-values of the function (e.g. changing a package var).
I use this for:
select ...
from big_table_vw;
where
create view big_table_vw
as
select ... (analytical functions)
from big_table
where last_mutated >= get_date();
In my case the big_table_vw contains window-functions that prevents Oracle to push the predicate into the view.
This is a late follow-up to a long-answered question, but I just wanted to add that Oracle does provide a caching mechanism for functions with mutable dependencies. RESULT_CACHE is an alternative to DETERMINISTIC that allows Oracle to abandon cached function results any time a referenced object is modified.
This way one can cache costly calculations against rarely-updated objects with confidence that cached results will not return incorrect results.
Here's an example using mythological monsters:
CREATE TABLE MONSTER (
MONSTER_NAME VARCHAR2(100) NOT NULL PRIMARY KEY
);
INSERT INTO MONSTER VALUES ('Chthulu');
INSERT INTO MONSTER VALUES ('Grendel');
INSERT INTO MONSTER VALUES ('Scylla');
INSERT INTO MONSTER VALUES ('Nue');
COMMIT;
CREATE OR REPLACE PACKAGE MONSTER_PKG
IS
FUNCTION IS_THIS_A_MONSTER(P_MONSTER_NAME IN VARCHAR2)
RETURN BOOLEAN RESULT_CACHE;
END MONSTER_PKG;
/
CREATE OR REPLACE PACKAGE BODY MONSTER_PKG
IS
FUNCTION IS_THIS_A_MONSTER(P_MONSTER_NAME IN VARCHAR2)
RETURN BOOLEAN
RESULT_CACHE RELIES_ON (MONSTER)
IS
V_MONSTER_COUNT NUMBER(1, 0) := 0;
BEGIN
SELECT COUNT(*)
INTO V_MONSTER_COUNT
FROM MONSTER
WHERE MONSTER_NAME = P_MONSTER_NAME;
RETURN (V_MONSTER_COUNT > 0);
END;
END MONSTER_PKG;
/
When a scenario like the below occurs, any existing cache is invalidated and a new cache can then be rebuilt.
BEGIN
DBMS_OUTPUT.PUT_LINE('Is Kraken initially a monster?');
IF MONSTER_PKG.IS_THIS_A_MONSTER('Kraken')
THEN
DBMS_OUTPUT.PUT_LINE('Kraken is initially a monster');
ELSE
DBMS_OUTPUT.PUT_LINE('Kraken is not initially a monster');
END IF;
INSERT INTO MONSTER VALUES ('Kraken');
COMMIT;
DBMS_OUTPUT.PUT_LINE('Is Kraken a monster after update?');
IF MONSTER_PKG.IS_THIS_A_MONSTER('Kraken')
THEN
DBMS_OUTPUT.PUT_LINE('Kraken is now a monster');
ELSE
DBMS_OUTPUT.PUT_LINE('Kraken is not now a monster');
END IF;
END;
/
Is Kraken initially a monster?
Kraken is not initially a monster
Is Kraken a monster after update?
Kraken is now a monster