When does a deterministic function use the previous calculated value? - oracle

Consider a deterministic function like:
CREATE OR REPLACE FUNCTION SCHEMA.GET_NAME(ss_id nvarchar2
) RETURN nvarchar2 DETERMINISTIC IS
tmpVar nvarchar2(500);
BEGIN
select name into tmpvar from logistics.organization_items
where id = ss_id ;
return tmpvar ;
END ss_name;
Using Toad I called the SCHEMA.GET_NAME(1) and it returns A. I then changed the value from the table from A to B and recalling the SCHEMA.GET_NAME(1) returned B.
It is a good result. But I'm afraid of the value not being updated according to this page in the documentation, which said:
When Oracle Database encounters a deterministic function in one of these contexts, it attempts to use previously calculated results when possible rather than reexecuting the function. If you subsequently change the semantics of the function, you must manually rebuild all dependent function-based indexes and materialized views.
In what situations would the value of GET_NAME(1) return an old cached value (A instead of B)?

If you select from a table then the results of your function are not deterministic. A deterministic system is one that will always produce the same output, given the same initial conditions.
It is possible to alter the information in a table, therefore a function that selects from a table is not deterministic. To quote from the PL/SQL Language Reference:
Do not specify this clause to define a function that uses package variables or that accesses the database in any way that might affect the return result of the function. The results of doing so are not captured if the database chooses not to reexecute the function.
In other words, Oracle does not guarantee that the results of the function will be accurate (they just might be). If your table is static, and unlikely to ever change, then it should be okay but this is not something I'd ever like to rely on. To answer your question, do not assume that Oracle will return anything other than the cached value within the same transaction/session.
If you need to speed this up there are two ways. Firstly, check that you have an index on ID!
Simply JOIN to this table. If your function is only this then there is no need for the function to exist.
Use scalar sub-query caching (not necessarily possible but worth the try).
select ( select get_name(:id) from dual )
from your_table
Oracle will create an in-memory hash of the results of the function, like a result cache. If you're executing the same function multiple times then Oracle will hit the cache rather than the function.

Ben's answer sums it up nicely, and I would just like to add that the way you used DETERMINISTIC keyword inside your function is not right - keeping in view that you are reading the value from a table and then returning the same to the user.
A deterministic function should be used in cases, where you are evaluating an expression over a fixed input, for example, when you need to return a substring, or upper/lower case for the input string. Programatically, you know that for the same input the lowercase function will always return the same value, and so you would like to cache the result (using deterministic keyword).
When you read a value from a table, Oracle has no way to know that the value in the column has not changed, and so it prefers to rexecute the function and not depend on the cached result (which makes sense)

Can you add a timestamp parameter to your function? Then pass in sysdate to the function from wherever you're calling it.
This way, you're effectively caching the result and you avoid running the function over and over when it generally returns the same value within a given transaction.

The remark of Erez is the answer I was looking for.
Before executing the query or plsql-unit you can with this solution force to execute the function again after you reseted the ret-values of the function (e.g. changing a package var).
I use this for:
select ...
from big_table_vw;
where
create view big_table_vw
as
select ... (analytical functions)
from big_table
where last_mutated >= get_date();
In my case the big_table_vw contains window-functions that prevents Oracle to push the predicate into the view.

This is a late follow-up to a long-answered question, but I just wanted to add that Oracle does provide a caching mechanism for functions with mutable dependencies. RESULT_CACHE is an alternative to DETERMINISTIC that allows Oracle to abandon cached function results any time a referenced object is modified.
This way one can cache costly calculations against rarely-updated objects with confidence that cached results will not return incorrect results.
Here's an example using mythological monsters:
CREATE TABLE MONSTER (
MONSTER_NAME VARCHAR2(100) NOT NULL PRIMARY KEY
);
INSERT INTO MONSTER VALUES ('Chthulu');
INSERT INTO MONSTER VALUES ('Grendel');
INSERT INTO MONSTER VALUES ('Scylla');
INSERT INTO MONSTER VALUES ('Nue');
COMMIT;
CREATE OR REPLACE PACKAGE MONSTER_PKG
IS
FUNCTION IS_THIS_A_MONSTER(P_MONSTER_NAME IN VARCHAR2)
RETURN BOOLEAN RESULT_CACHE;
END MONSTER_PKG;
/
CREATE OR REPLACE PACKAGE BODY MONSTER_PKG
IS
FUNCTION IS_THIS_A_MONSTER(P_MONSTER_NAME IN VARCHAR2)
RETURN BOOLEAN
RESULT_CACHE RELIES_ON (MONSTER)
IS
V_MONSTER_COUNT NUMBER(1, 0) := 0;
BEGIN
SELECT COUNT(*)
INTO V_MONSTER_COUNT
FROM MONSTER
WHERE MONSTER_NAME = P_MONSTER_NAME;
RETURN (V_MONSTER_COUNT > 0);
END;
END MONSTER_PKG;
/
When a scenario like the below occurs, any existing cache is invalidated and a new cache can then be rebuilt.
BEGIN
DBMS_OUTPUT.PUT_LINE('Is Kraken initially a monster?');
IF MONSTER_PKG.IS_THIS_A_MONSTER('Kraken')
THEN
DBMS_OUTPUT.PUT_LINE('Kraken is initially a monster');
ELSE
DBMS_OUTPUT.PUT_LINE('Kraken is not initially a monster');
END IF;
INSERT INTO MONSTER VALUES ('Kraken');
COMMIT;
DBMS_OUTPUT.PUT_LINE('Is Kraken a monster after update?');
IF MONSTER_PKG.IS_THIS_A_MONSTER('Kraken')
THEN
DBMS_OUTPUT.PUT_LINE('Kraken is now a monster');
ELSE
DBMS_OUTPUT.PUT_LINE('Kraken is not now a monster');
END IF;
END;
/
Is Kraken initially a monster?
Kraken is not initially a monster
Is Kraken a monster after update?
Kraken is now a monster

Related

Writing a function that reads through a specific table and fills another based on the table that got read

What am trying to do is read through a table that is called transit and based on that table, I want to fill ord & ord 2 and if the function went ahead and executed with no problems, it would return true
CREATE OR REPLACE FUNCTION new_func
RETURN boolean
IS
BEGIN
INSERT INTO ORD(ordernum,cnum,snum,rec,ship,typ)
SELECT ordernum,cnum,snum,rec,ship,typ
FROM TRANSIT;
INSERT INTO ORD2(ordernum,cnum,snum,rec,ship,typ)
SELECT ordernum,cnum,snum,rec,ship,typ
FROM TRANSIT
return true;
END;
Oracle provides a convenient statement for this type action: INSERT ALL. This allows inserting rows into several tables with just one statement: (see example fiddle here)
insert all
into ord(ordernum,cnum,snum,rec,ship,typ)
values(ordernum,cnum,snum,rec,ship,typ)
into ord2(ordernum,cnum,snum,rec,ship,typ)
values(ordernum,cnum,snum,rec,ship,typ)
select ordernum,cnum,snum,rec,ship,typ
from transit;
In the above every row would be copied into both tables. This however is not required. The fiddle referenced above splits the tables between even and odd ordernum, but the split could be any on expression which eventuates true or false.
Initially my reply was just the above. But reading the comments it became apparent that OP may benefit from seeing examples of why this should be a procedure and not a function. (As could whoever required a function.)
As a developer I automatically assume a function can be called directly form SQL. In this case that is not true. This function can only be called form a plsql block. There are (at least) 2 reasons for this:
There is no boolean data type in SQL. Boolean data type is
strictly a plsql data type.
A function called by SQL cannot perform DML.
The referenced fiddle above also contains a example of each of these and what happens when called when called via SQL and by plsql.

What is the purpose of this package?

I am new to Oracle PL/SQL. I have found this package and it is called from a trigger. I just cannot figure exactly what this simple looking package code is doing.
It is called from a trigger as below:
IF INSERTING THEN
i := STATE_PKG_OVERRIDE_CN.AffectedRows.COUNT+1;
STATE_PKG_OVERRIDE_CN.AffectedRows(i).IDU := :new.IDU;
STATE_PKG_OVERRIDE_CN.AffectedRows(i).cn := :new.cn;
This is the package. Can somebody please explain the basics of what is it doing? does it return a value? change a value? what is AffectedRows RIDARRAY or EMPTY ?
create or replace PACKAGE STATE_PKG_OVERRIDE_CN
AS
TYPE rowid_cn IS RECORD
(
idu dirxml.USR.IDU%TYPE,
cn dirxml.USR.CN%TYPE
);
TYPE RIDARRAY IS TABLE OF rowid_cn INDEX BY BINARY_INTEGER;
AffectedRows RIDARRAY;
EMPTY RIDARRAY;
END;
I have googled EMPTY but found nothing, i believe it is creating a table of type record. The trigger is passing in a value of cn or IDU am i am familiar with these two values. But what is the package doing? or returning ? im confused.
Cheers
This is a bespoke package belonging to your organisation. (That's why Google wasn't helpful for you.) We cannot tell you for sure what it does or how it's used. But we can guess.
The package has no procedures or functions, it just defines array variables of a bespoke type, which can be used by other program units such as triggers. The trigger you posted assigns values to the array AffectedRows. Presumably this trigger fires FOR EACH ROW. Likely there is another trigger on the same table firing AFTER STATEMENT which reads that array and does some processing, then assigns AffectedRows := EMPTY to reset the array.
The purpose of this infrastructure is to pass state across trigger actions. A common reason for doing this is work around a mutating table exception. They are risky because state cannot be guaranteed; for instance if a insert fails before the AFTER STATEMENT trigger fires the AffectedRows array is not re-initialised, so subsequent processing will be incorrect (or will fail).
Since 11g Oracle provides compound triggers which remove the need for this sort of package. Find out more.
To investigate further, first you want to check USER_TRIGGERS to find other triggers on the table which owns the trigger you mentioned. If that doesn't help or you want to see whether other tables also use this package run this query:
select *
from user_dependencies
where referenced_type = 'PACKAGE'
and referenced_name = 'STATE_PKG_OVERRIDE_CN'
It begins with the package which
declares a type ROWID_CN as a record that contains two values: IDU and CN
syntax requires to create another type (RIDARRAY) which is based on previously declared ROWID_CN
affectedrows and empty are arrays whose type is RIDARRAY. Basically, you can imagine them as a table that has two columns, IDU and CN
Now, a trigger: piece of code you posted says that those 3 lines are executed when someone inserts a row into a table the trigger is based on. For example,
create or replace trigger trg_biu_emp
before insert or update on emp
for each row
declare
i number;
begin
if inserting then ...
Apparently, there's some code that is executed when updating, or even deleting rows.
Anyway:
i := ... line counts number of elements in affectedrows array (which is declared in the package) and adds 1 to that number. For example, if there were 3 elements, i would be 4.
...affectedrows(i).idu := :new.idu enters a new row into the array. Its ordinal number is i (4 in our example). As you're inserting a row into the table, trigger knows IDU column's :new value and puts it into the array. For example, if you used
insert into emp (idu, cn) values (100, 'A')
then affectedrows(4).idu = 100, while affectedrows(4).cn = 'A'
It is, probably, something similar with EMPTY array. Google can't return anything useful, it is just a custom-made array.

PL/SQL Creating Associative Array With Employee Id as Key

I am trying to create a hash with employee_id (NUMBER(6,0)) as the key and salary (NUMBER(8,2)) as the value.
For that I have created a INDEX-OF table(associative array) in PL/SQL (Oracle 11g) using the following definition:
TYPE emp_title_hash IS TABLE OF employees.salary%type
INDEX BY employees.employee_id%type;
I am getting the following compilation error:
Error(22,28): PLS-00315: Implementation restriction: unsupported table index type
I am aware in this case that the only supported type for the index is STRING or PLS_INTEGER. This seems to really restrictive. Why exactly has this been imposed in Oracle ? Is there a work around to get the above done ?
Appreciate your comments / suggestions.
As someone already pointed out, you can use "index by pls_integer", since pls_integer can contain any number(6,0) value.
Surely it would be nice to be able to use any possible type to index a pl/sql associative array, but I have always managed a way of writing a function that builds a string that identifies the object instance I want to use as index value.
So, instead of writing:
type TMyAssociativeArray is table of MyDataType index by MyIndexType;
I write:
Type TMyAssociativeArray is table of MyDataType index by varchar2(30);
then I write a function that calculates a unique string from MyIndexType:
function GetHash(obj MyIndexType) return varchar2;
and, having written this function I can use it to simulate an associative array indexed by MyIndexType object:
so I can write
declare
arr TMyAssociativeArray;
obj TMyDataType;
idx TMyDataType;
begin
....
arr(GetHash(idx)) := obj;
end;
Here I am getting out of the strict question you asked, and giving you an advice about a possible other way of obtaining a quick customer->salary lookup cache... This is the final purpose of your associative-array question, as I can read from your comment, so maybe this could be useful:
If you are using associative arrays to build a fast look-up mechanism, and if you can use the oracle 11g R2 new features, a easier way of obtaining this caching is to rely on the native RESULT_CACHE feature which has been introduced both for queries (by using the RESULT_CACHE hint) and for pl/sql functions.
for pl/sql functions you can create a function whose result value is cached like this one:
create or replace function cached_employee_salary(employee number)
return number RESULT_CACHE is
result number;
begin
select salary
into result
from employees e
where e.code = employee;
return result;
end;
the RESULT_CACHE keyword instructs oracle to keep an in-memory cache of the result values and to reuse them on subsequent calls (when the function is called with the same parameters, of course).
this cache has these advantages, compared to the use of associative arrays:
it is shared among all sessions: cached data is not kept in the private memory allocated for each session, so it wastes less memory
oracle is smart enough to detect that the function calculates its results by accessing the employees table and automatically invalidates the cached results if the table data is being modified.
Of course I suggest you to run some tests to see if this optimization, in your case gives some tangible results. it mostly depends on how complex is the calculation you make in your function.
You can also rely on an analogous feature introduced for SQL queries, triggered by the /+RESULT_CACHE/ hint:
select /*+RESULT_CACHE*/
salary
from employees e
where e.code = employee;
return result
this hint instructs oracle to store and reuse (on subsequent executions) the result set of the query. it will be stored too in memory.
actually this hint has also the advantage that, being the hint -syntactically speaking- a special comment, this query will continue working without any modifications even on servers < 11gR2, whereas, for the function cache version, you should use some "conditional compilation" magic to make it compile also with previous server versions (for which it would be a normal function without any result caching)
I Hope this helps.

why can not I use Like in stored functions?

I'm writting some stored functions in Oracle. One of these is a really basic function who take a string as parameter and return an another string. Here is my function:
CREATE OR REPLACE
FUNCTION get_mail_custcode (
custcodeParam IN customer_table.custcode%TYPE)
RETURN VARCHAR2
IS
mail_rc contact_table.email%TYPE;
BEGIN
SELECT cc.email
INTO mail_rc
FROM contact_table cc, customer_table cu
WHERE cu.customer_id = cc.customer_id
AND cu.custcode like custcodeParam ;
RETURN mail_rc ;
END;
So it's not working.. The function seems to work well but is executed without any end.. The function is working time and time, I manually cancel the operation after 2 or 3 minutes (this query give normally instant result).
After writing the query again and again I finally (and randomly) change the cu.custcode like custcodeParam into a cu.custcode = custcodeParam and it is working!!
So my question is why? Why I can't use a like comparator in a stored function? Why this makes no error but the function run indefinitly.
Thanks.
Cursors are all treated identically in Oracle. A query in a function will be treated exactly the same as a query you enter manually through SQL*Plus.
However, what may differ in your example is how Oracle works with variables. The following two queries are fundamentally different to the optimizer:
SELECT * FROM tab WHERE code LIKE 'FOO%';
and
variable v_code VARCHAR2(4)
EXEC :v_code := 'FOO%';
SELECT * FROM tab WHERE code LIKE :v_code;
In the first case the optimizer looks at the constant FOO% and can instantly tell that an index on code is perfectly suited to retrieve the rows rapidly via an index RANGE SCAN.
In the second case, the optimizer has to consider that :V_CODE is not constant. The purpose of the optimizer is to determine a plan for a query that will be shared by successive executions of the same query (because computing a plan is expensive).
The behaviour of the optimizer will depend upon your version of Oracle:
In old Oracle versions (9i and before), the value of the variable was ignored to build the plan. In effect Oracle had to build a plan that would be efficiently, whatever value was passed to it. In your case this likely would result in a full scan because Oracle had to take the least risky option and consider that FOO% was as likely a value as %FOO (the latter can't be accessed efficiently via an index range scan).
In 10g Oracle introduced bind peeking: now the optimizer can access the value of the variable and produce a suitable plan. The main problem is that in most cases a query can only have one plan, which means that the value of the first variable ever passed to the function will force the execution plan for all further executions. If the first value to be passed is %FOO, a FULL SCAN will likely be chosen.
In 11g, Oracle has "intelligent cursor sharing": a single query can share more than one plan, in the example above the value FOO% would use a RANGE SCAN while %FOO would probably use a FULL SCAN.
What version of Oracle are you using?
update
In 10g and before if this function is used often without wildcards you should rewrite it to acknowledge the optimizer behaviour:
BEGIN
IF instr(custcodeParam, '%') > 0 OR instr(custcodeParam, '_') > 0 THEN
SELECT cc.email
INTO mail_rc
FROM contact_table cc, customer_table cu
WHERE cu.customer_id = cc.customer_id
AND cu.custcode LIKE custcodeParam;
ELSE
SELECT cc.email
INTO mail_rc
FROM contact_table cc, customer_table cu
WHERE cu.customer_id = cc.customer_id
AND cu.custcode = custcodeParam;
END IF;
RETURN mail_rc;
END;

How to optimize usage of pipelined, weakly typed ref cursor

I am having some trouble with a procedure; when run for “big” sets (800+ parents, 1300+ children), it is very slow (30 - 60 secs).
Basic idea is to fetch all parent records (and their respective children) fitting a certain search criteria, along with 3 additional pieces of information that will have to be computed.
My approach to the problem was
to create a custom record type with additional fields for the computed values.
A reference to this record type can then be passed around to each function, controlled by a main processing function.
As a value is computed for each parent record, tack it onto the record.
Each procedure GET_PARENT_RECORDS and GET_CHILD_RECORDS are called once per search, and each computing functions are run N times (where N is the number of parent and/or child records).
Question 1: Is this the right approach? (weakly typed cursors, pipelined functions) If not, then how should I have approached the problem, assuming I can have a re-do?
Question 2: Barring a complete rewrite, is there anything obvious that can be improved in the code provided?
Question 3: Or could something else be wrong, as I notice that the same slow query returned in 20 secs when I've run the procedures a few times?
Package definition
create or replace
PACKAGE THIS_PKG AS
Type parentCursor IS REF CURSOR;
Type childCursor IS REF CURSOR;
Type ParentRecordType IS RECORD (
other_columns,
Extra_column_A,
Extra_column_B,
Extra_column_C,
Row_num);
--associative array
TYPE ParentArray IS TABLE OF ParentRecordType;
FUNCTION processParents(
p IN THIS_PKG. parentCursor
) RETURN ParentArray
PIPELINED
;
FUNCTION countSomething(some params…)
RETURN INT;
FUNCTION checkCondX (SomeParent IN ParentRecordType)
RETURN VARCHAR2;
FUNCTION checkCondY (SomeParent IN ParentRecordType)
RETURN VARCHAR2;
PROCEDURE GET_PARENT_RECORDS( other_parameters, Parents OUT THIS_PKG.parentCursor);
PROCEDURE GET_CHILD_RECORDS( other_parameters, Children OUT THIS_PKG.childCursor);
END THIS_PKG;
Package Body
-- omitted
FUNCTION processParents(
p IN THIS_PKG.parentCursor
) RETURN ParentArray
PIPELINED
IS
out_rec ParentArray;
someParent ParentRecordType;
BEGIN
LOOP
FETCH p BULK COLLECT INTO out_rec LIMIT 100;
FOR i IN 1 .. out_rec.COUNT
LOOP
out_rec(i).extra_column_A := countSomething (out_rec(i).field1, out_rec(i).field2);
out_rec(i).extra_column_B := checkCondX(out_rec(i));
out_rec(i).extra_column_C := checkCondY(out_rec(i));
pipe row(out_rec(i));
END LOOP;
EXIT WHEN p%NOTFOUND;
END LOOP;
RETURN;
END processParents;
PROCEDURE GET_PARENT_RECORDS(
some_columns,
Parents OUT THIS_PKG. parentCursor) IS
BEGIN
OPEN Parents FOR
SELECT *
FROM TABLE(processParents (CURSOR(
SELECT *
FROM (
--some select statement with quite a few where clause
--to simulate dynamic search (from pre-canned search options)
)
))) abc
WHERE abc.extra_column_C like '%xyz%' --(xyz is a user given value)
;
END GET_PARENT_RECORDS;
Update
Did some exploring yesterday and came across the Quest Batch SQL Optimizer (from Toad). I plugged in the package and here's what I got.
Batch Optimizer results
Complex query
Problematic query
What is happening in the row processing section? A lot of time may be spent in these countSomething, checkCondX/Y functions. Are they also making SQL calls? I'd check the performance of the table function without the additional predicates first. It may be better to simply create a query which does this all in SQL rather than functions - if you can do this it will be very much quicker than calling out to a function for every row.
out_rec(i).extra_column_A := countSomething (out_rec(i).field1, out_rec(i).field2);
out_rec(i).extra_column_B := checkCondX(out_rec(i));
out_rec(i).extra_column_C := checkCondY(out_rec(i));
Also the explain plan you provided is interesting as the optimiser believes there is only 1 row being returned from all tables (cardinality 1). If this isn't the case then the query plan will not be optimal. It may be necessary to gather statistics, use dynamic sampling or cardinality hints on the table function.
Finally, look into DBMS_SQLTUNE.REPORT_SQL_MONITOR which provides a detailed report on your sql. Unless the query is dynamically identified as needing monitoring your need to add the /*+ MONITOR */ hint. This provides more details like number of rows returned, execution count and other interesting tidbits not available in an explain plan.
SELECT /*+ MONITOR */
FROM slow_query;
-- then run sqltune to get a report
SELECT *
FROM TABLE(DBMS_SQLTUNE.REPORT_SQL_MONITOR());
Quest Batch SQL Optimizer (from Toad) or any other tool will be unable to help you taking into consideration that they don't understand what you do inside the functions. The problem is in "FETCH p BULK COLLECT INTO out_rec LIMIT 100;". The quality of the query that is passed into p actually defines the final execution plan and run times. Pipelining is not the reason for slowness. When you run your procedure several times, Oracle uses cached data. My best advice is: use Java instead of PL/SQL for this particular purpose, it will be more simpler for understanding.

Resources