I need to find a safe and correct way to implement this: I am storing an operation log on the database with a sequential EVENT_ID numeric column. I need to be able to delete the last row at anytime and the value in EVENT_ID cannot skip any numbers therefore I can't use a sequence.
I wrote a PL/SQL function that returns MAX(EVENT_ID)+1
CREATE OR REPLACE FUNCTION NEW_ID
(OP_TYPE IN LOG_TABLE.OP_TYPE_ID%TYPE,
APP_ID IN LOG_TABLE.APPLICATION_ID%TYPE)
RETURN LOG_TABLE.EVENT_ID%TYPE IS
MYRES LOG_TABLE.EVENT_ID%TYPE;
BEGIN
LOCK TABLE LOG_TABLE IN EXCLUSIVE MODE NOWAIT;
SELECT MAX(LOG_TABLE.EVENT_ID) INTO MYRES
FROM
LOG_TABLE
WHERE
LOG_TABLE.OP_TYPE_ID = OP_TYPE
AND
LOG_TABLE.APPLICATION_ID = APP_ID;
IF MYRES IS NULL THEN
MYRES := 1;
ELSE
MYRES := MYRES + 1;
END IF;
RETURN MYRES;
END NEW_ID;
/
However there are multiple clients running and reporting on the same database. This means I could end up with multiple equal EVENT_ID values if two or more clients call the function in the exact same moment.
AFAIK locking a table doesn't prevent it being read from other sessions - is there a way to do it? Using a cursor maybe?
It is very rare that there is a real business requirement for a gapless sequence, so I think you should still consider using a sequence. If you do insist that you must have no gaps in the generated ID, it is better to use a separate 1-row table, which you can exclusively lock without preventing reads on LOG_TABLE, where you would maintain the next available EVENT_ID value.
Related
Is there a hint to generate execution plan ignoring the existing one from the shared pool?
There is not a hint to create an execution plan that ignores plans in the shared pool. A more common way of phrasing this question is: how do I get Oracle to always perform a hard parse?
There are a few weird situations where this behavior is required. It would be helpful to fully explain your reason for needing this, as the solution varies depending why you need it.
Strange performance problem. Oracle performs some dynamic re-optimization of SQL statements after the first run, like adaptive cursor sharing and cardinality feedback. In the rare case when those features backfire you might want to disable them.
Dynamic query. You have a dynamic query that used Oracle data cartridge to fetch data in the parse step, but Oracle won't execute the parse step because the query looks static to Oracle.
Misunderstanding. Something has gone wrong and this is an XY problem.
Solutions
The simplest way to solve this problem are by using Thorsten Kettner's solution of changing the query each time.
If that's not an option, the second simplest solution is to flush the query from the shared pool, like this:
--This only works one node at a time.
begin
for statements in
(
select distinct address, hash_value
from gv$sql
where sql_id = '33t9pk44udr4x'
order by 1,2
) loop
sys.dbms_shared_pool.purge(statements.address||','||statements.hash_value, 'C');
end loop;
end;
/
If you have no control over the SQL, and need to fix the problem using a side-effect style solution, Jonathan Lewis and Randolf Geist have a solution using Virtual Private Database, that adds a unique predicate to each SQL statement on a specific table. You asked for something weird, here's a weird solution. Buckle up.
-- Create a random predicate for each query on a specific table.
create table hard_parse_test_rand as
select * from all_objects
where rownum <= 1000;
begin
dbms_stats.gather_table_stats(null, 'hard_parse_test_rand');
end;
/
create or replace package pkg_rls_force_hard_parse_rand is
function force_hard_parse (in_schema varchar2, in_object varchar2) return varchar2;
end pkg_rls_force_hard_parse_rand;
/
create or replace package body pkg_rls_force_hard_parse_rand is
function force_hard_parse (in_schema varchar2, in_object varchar2) return varchar2
is
s_predicate varchar2(100);
n_random pls_integer;
begin
n_random := round(dbms_random.value(1, 1000000));
-- s_predicate := '1 = 1';
s_predicate := to_char(n_random, 'TM') || ' = ' || to_char(n_random, 'TM');
-- s_predicate := 'object_type = ''TABLE''';
return s_predicate;
end force_hard_parse;
end pkg_rls_force_hard_parse_rand;
/
begin
DBMS_RLS.ADD_POLICY (USER, 'hard_parse_test_rand', 'hard_parse_policy', USER, 'pkg_rls_force_hard_parse_rand.force_hard_parse', 'select');
end;
/
alter system flush shared_pool;
You can see the hard-parsing in action by running the same query multiple times:
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
Now there are three entries in GV$SQL for each execution. There's some odd behavior in Virtual Private Database that parses the query multiple times, even though the final text looks the same.
select *
from gv$sql
where sql_text like '%hard_parse_test_rand%'
and sql_text not like '%quine%'
order by 1;
I think there is no hint indicating that Oracle shall find a new execution plan everytime it runs the query.
This is something we'd want for select * from mytable where is_active = :active, with is_active being 1 for very few rows and 0 for maybe billions of other rows. We'd want an index access for :active = 1 and a full table scan for :active = 0 then. Two different plans.
As far as I know, Oracle uses bind variable peeking in later versions, so with a look at the statistics it really comes up with different execution plans for different bind varibale content. But in older versions it did not, and thus we'd want some hint saying "make a new plan" there.
Oracle only re-used an execution plan for exactly the same query. It sufficed to add a mere blank to get a new plan. Hence a solution might be to generate the query everytime you want to run it with a random number included in a comment:
select /* 1234567 */ * from mytable where is_active = :active;
Or just don't use bind variables, if this is the problem you want to address:
select * from mytable where is_active = 0;
select * from mytable where is_active = 1;
I have the following function, which returns the next available client ID from the Client table:
CREATE OR REPLACE FUNCTION getNextClientID RETURN INT AS
ctr INT;
BEGIN
SELECT MAX(NUM) INTO ctr FROM Client;
IF SQL%NOTFOUND THEN
RETURN 1;
ELSIF SQL%FOUND THEN
-- RETURN SQL%ROWCOUNT;
RAISE_APPLICATION_ERROR(-20010, 'ROWS FOUND!');
-- RETURN ctr + 1;
END IF;
END;
But when calling this function,
BEGIN
DBMS_OUTPUT.PUT_LINE(getNextClientID());
END;
I get the following result:
which I found a bit odd, since the Client table contains no data:
Also, if I comment out RAISE_APPLICATION_ERROR(-20010, 'ROWS FOUND!'); & log the value of SQL%ROWCOUNT to the console, I get 1 as a result.
On the other hand, when changing
SELECT MAX(NUM) INTO ctr FROM Client;
to
SELECT NUM INTO ctr FROM Client;
The execution went as expected. What is the reason behind this behavior ?
Aggregate functions will always return a result:
All aggregate functions except COUNT(*), GROUPING, and GROUPING_ID
ignore nulls. You can use the NVL function in the argument to an
aggregate function to substitute a value for a null. COUNT and
REGR_COUNT never return null, but return either a number or zero. For
all the remaining aggregate functions, if the data set contains no
rows, or contains only rows with nulls as arguments to the aggregate
function, then the function returns null.
You can change your query to:
SELECT COALESCE(MAX(num), 1) INTO ctr FROM Client;
and remove the conditionals altogether. Be careful about concurrency issues though if you do not use SELECT FOR UPDATE.
Query with any aggregate function and without GROUP BY clause always returns 1 row. If you want no_data_found exception on empty table, add GROUP BY clause or remove max:
SQL> create table t (id number, client_id number);
Table created.
SQL> select nvl(max(id), 0) from t;
NVL(MAX(ID),0)
--------------
0
SQL> select nvl(max(id), 0) from t group by client_id;
no rows selected
Usually queries like yours (with max and without group by) are used to avoid no_data_found.
Agregate functions like MAX will always return a row. It will return one row with a null value if no row is found.
By the way SELECT NUM INTO ctr FROM Client; will raise an exception where there's more than one row in the table.
You should instead check whether or not ctr is null.
Others have already explained the reason why your code isn't "working", so I'm not going to be doing that.
You seem to be instituting an identity column of some description yourself, probably in order to support a surrogate key. Doing this yourself is dangerous and could cause large issues in your application.
You don't need to implement identity columns yourself. From Oracle 12c onwards Oracle has native support for identity columns, these are implemented using sequences, which are available in 12c and previous versions.
A sequence is a database object that is guaranteed to provide a new, unique, number when called, no matter the number of concurrent sessions requesting values. Your current approach is extremely vulnerable to collision when used by multiple sessions. Imagine 2 sessions simultaneously finding the largest value in the table; they then both add one to this value and try to write this new value back. Only one can be correct.
See How to create id with AUTO_INCREMENT on Oracle?
Basically, if you use a sequence then you don't need any of this code.
As a secondary note your statement at the top is incorrect:
I have the following function, which returns the next available client ID from the Client table
Your function returns the maximum ID + 1. If there's a gap in the IDs, i.e. 1, 2, 3, 5 then the "missing" number (4 in this case) will not be returned. A gap can occur for any number of reasons (deletion of a row for example) and does not have a negative impact on your database in any way at all - don't worry about them.
I'm here because i could not finde anywhere else if there is a way to return the previous value in a loop (Cursor) to compare with the current value, for instance..
Cursor.Value = Cursor-1.Value;
It's bacause i have several contract numbers that i need to send by mail to the Business sector, but, in order to resume all the rows i want to compare if the current contract number are the same as the last contract number and validate it to dont send duplicated contract numbers.
Exemple of Record that i to skip in order to send no duplicate "Order Numbers": (Order_Number is my Key, not a sequencial numeric id):
cCursor.Value = cCursor-1.Value
cCursor.(111) = cCursor-1.(111)
Exemple of Record that i want to save in order to send as a processed "Order Number": (Order_Number is my Key, not a sequencial numeric id):
cCursor.Value = cCursor-1.Value
cCursor.(132) = cCursor-1.(111)
My Regards.
You cant reference backwards. Th easiest alternative is to store the key value (contract_id) in a variable and have logic like:
DECLARE
CURSOR c1 IS .....;
vLastContractID NUMBER := 0;
BEGIN
FOR r1 IN c1 LOOP
IF vLastContractID != r1.CONTRACT_ID THEN
-- do something
vLastContractID := r1.CONTRACT_ID;
END IF;
END LOOP;
END;
It's not entirely clear what you are asking.
A cursor is a forward-only structure. You cannot fetch a prior row, just the next row (or set of rows). Your query, however, can certainly include data from prior rows using the lag function. For example, this will show you the ename for the prior row in your result
SELECT empno, ename, lag(ename) over (order by empno) prior_ename
FROM emp
ORDER BY empno
In a PL/SQL loop, you can also obviously have a local variable that has the data from the previous row that was fetched and use that to compare against the data from the most current row.
Please use ANALYTICAL function to check for prior or post rows. LEAD,LAG functions are best way to do this.
I have a procedure which returns ref cursor as output parameter. I need to find a way to get the count of no.of records in the cursor. Currently I have count fetched by repeating the same select query which is hindering the performance.
ex:
create or replace package temp
TYPE metacur IS REF CURSOR;
PROCEDURE prcSumm (
pStartDate IN DATE,
pEndDate IN DATE,
pKey IN NUMBER,
pCursor OUT metacur
) ;
package body temp is
procedure prcSumm(
pStartDate IN DATE,
pEndDate IN DATE,
pKey IN NUMBER,
pCursor OUT metacur
)
IS
vCount NUMBER;
BEGIN
vCount := 0;
select count(*) into vCount
from customer c, program p, custprog cp
where c.custno = cp.custno
and cp.programid = p.programid
and p.programid = pKey
and c.lastupdate >= pStartDate
and c.lastupdate < pEndDate;
OPEN pCursor for SELECT
c.custno, p.programid, c.fname, c.lname, c.address1, c.address2, cp.plan
from customer c, program p, custprog cp
where c.custno = cp.custno
and cp.programid = p.programid
and p.programid = pKey
and c.lastupdate >= pStartDate
and c.lastupdate < pEndDate;
end prcSumm;
Is there a way to get the no.of rows in the out cursor into vCount.
Thanks!
Oracle does not, in general, know how many rows will be fetched from a cursor until the last fetch finds no more rows to return. Since Oracle doesn't know how many rows will be returned, you can't either without fetching all the rows (as you're doing here when you re-run the query).
Unless you are using a single-user system or you are using a non-default transaction isolation level (which would introduce additional complications), there is no guarantee that the number of rows that your cursor will return and the count(*) the second query returns would match. It is entirely possible that another session committed a change between the time that you opened the cursor and the time that you ran the count(*).
If you are really determined to produce an accurate count, you could add a cnt column defined as count(*) over () to the query you're using to open the cursor. Every row in the cursor would then have a column cnt which would tell you the total number of rows that will be returned. Oracle has to do more work to generate the cnt but it's less work than running the same query twice.
Architecturally, though, it doesn't make sense to return a result and a count from the same piece of code. Determining the count is something that the caller should be responsible for since the caller has to be able to iterate through the results. Every caller should be able to handle the obvious boundary cases (i.e. the query returns 0 rows) without needing a separate count. And every caller should be able to iterate through the results without needing to know how many results there will be. Every single time I've seen someone try to follow the pattern of returning a cursor and a count, the correct answer has been to redesign the procedure and fix whatever error on the caller prompted the design.
I am working on a data warehousing project, and therefore, I have been implementing some ETL Functions in Packages. I first encountered a problem on my developing laptop and thought it had something to do with my oracle installation, but now it has "spread" over to the production servers.
Two functions "sometimes" become incredible slow. We have implemented a logging system, giving us output on a logging table each x rows. When the function usually needs like 10 seconds per chunk, "sometimes" the functions needs up to 3 minutes. After rebuilding some indexes and restarting the function, it is as quick again as it used to be.
Unfortunately, I can't tell which index it is exactly, since restarting the function and building up the cursor it uses for its work takes some time and we do not have the time to check each index on its own, so I just rebuild all indexes that are potentially used by the function and restart it.
The functions that have the problem use a cursor to select data from a table with about 50 million to 200 million entries, joined by a small table with about 50-500 entries. The join condition is a string comparison. We then use the primary key from the small table we get from the join to update a foreign key on the main table. The update process is done by a forall loop, this has proven to save loads of time.
Here is a simplified version of the table structure of both tables:
CREATE TABLE "maintable"
( "pkmid" NUMBER(11,0) NOT NULL ENABLE,
"fkid" NUMBER(11,0),
"fkstring" NVARCHAR2(4) NOT NULL ENABLE,
CONSTRAINT "PK_MAINTABLE" PRIMARY KEY ("pkmid");
CREATE TABLE "smalltable"
( "pksid" NUMBER(11,0) NOT NULL ENABLE,
"pkstring" NVARCHAR2(4) NOT NULL ENABLE,
CONSTRAINT "PK_SMALLTABLE" PRIMARY KEY ("pksid");
Both tables have indexes on their string columns. Adding the primary keys, I therefore rebuild 4 indexes each time the problem happens.
We get our data in a way, that we only have the fkstring in the maintable available and the fkid is set to null. In a first step, we populate the small table. This only takes minutes and is done the following way:
INSERT INTO smalltable (pksid, pkstring)
SELECT SEQ_SMALLTABLE.NEXTVAL, fkstring
FROM
(
SELECT DISTINCT mt.fkstring
FROM maintable mt
MINUS
SELECT st.pkstring
FROM smalltable st
);
commit;
This function never causes any trouble.
The following function does (it is a simplified version of the function - I have removed logging and exception handling and renamed some variables):
function f_set_fkid return varchar2 is
cursor lCursor_MAINTABLE is
SELECT MT.PKmID, st.pksid
FROM maintable mt
JOIN smalltable st ON (mt.fkstring = st.pkstring)
WHERE mt.fkid IS NULL;
lIndex number := 0;
lExitLoop boolean := false;
type lCursorType is table of lCursor_MAINTABLE%rowtype index by pls_integer;
lCurrentRow lCursor_MAINTABLE%rowtype;
lTempDataArray lCursorType;
lCommitEvery constant number := 1000;
begin
open lCursor_MAINTABLE;
loop
-- get next row, set exit condition
fetch lCursor_MAINTABLE into lCurrentRow;
if (lCursor_MAINTABLE%notfound) then
lExitLoop := true;
end if;
-- in case of cache being full, flush cache
if ((lTempDataArray.count > 0) AND (lIndex >= lCommitEvery OR lExitLoop)) then
forall lIndex2 in lTempDataArray.FIRST..lTempDataArray.LAST
UPDATE maintable mt
set fkid = lTempDataArray(lIndex2).pksid
WHERE mt.pkmid = lTempDataArray(lIndex2).pkmid;
commit;
lTempDataArray.delete;
lIndex := 0;
end if;
-- data handling, fill cache
if (lExitLoop = false) then
lIndex := lIndex + 1;
lTempDataArray(lIndex). := lCurrentRow;
end if;
exit when lExitLoop;
end loop;
close lCursor_MAINTABLE;
return null;
end;
I would be very thankful for any help.
P.S. I do know that bulk collect into would speed up the function and probably also ease up the code a bit too, but at the moment we are content with the speed of the function it usually has. Changing the function to use bulk collect is on our plan for next year, but at the moment it is not an option (and I doubt it would solve this index problem).
If you have a table where the number of rows fluctuates wildly (as in when doing ETL loads) I would use the statistics of the fully loaded table throughout the load process.
So, generate statistics when your table is fully loaded and then use those statistics for subsequent loads.
If you use statistics from when the table is half-loaded the optimizer may be tricked into not using indexes or not using the fastest index. This is especially true if data is loaded in order so that low value, high value and density are skewed.
In your case, the statistics for columns fkstring and fkid are extra important since those two columns are heavily involved in the procedure that has performance issues.
function f_set_fkid return varchar2 is
cursor lCursor_MAINTABLE is
SELECT MT.PKmID, st.pksid
FROM maintable mt
JOIN smalltable st ON (mt.fkstring = st.pkstring)
WHERE mt.fkid IS NULL;
commit_every INTGER := 1000000;
commit_counter INTEGER :=0;
begin
for c in lCursor_MAINTABLE
loop
UPDATE maintable mt
set fkid = c.pksid
WHERE mt.pkmid = c.pkmid;
commit_counter := commit_counter+1;
if mod(commit_every,commit_counter) = 0
then
commit;
commit_counter := 0;
end if;
end loop;
return null;
end;