Inserting Row Number based on existing value in the table - oracle

I have a requirement that I need to insert row number in a table based on value already present in the table. For example, the max row_nbr record in the current table is something like this:
+----------+----------+------------+---------+
| FST_NAME | LST_NAME | STATE_CODE | ROW_NBR |
+----------+----------+------------+---------+
| John | Doe | 13 | 123 |
+----------+----------+------------+---------+
Now, I need to insert more records, with given FST_NAME and LST_NAME values. ROW_NBR needs to be generated while inserting the data into table with values auto-incrementing from 123.
I can't use a sequence, as my loading process is not the only process that inserts data into this table. And I can't use a cursor as well, as due to high volume of data the TEMP space gets filled up quickly. And I'm inserting data as given below:
insert into final_table
( fst_name,lst_name,state_code)
(select * from staging_table
where state_code=13);
Any ideas how to implement this?

It sounds like other processes are finding the current maximum row_nbr value and incrementing it as they do single-row inserts in a cursor loop.
You could do something functionally similar, either finding the maximum in advance and incrementing it (if you're already running this in a PL/SQL block):
insert into final_table (fst_name, lst_name, state_code, row_nbr)
select st.*, variable_holding_maximum + rownum
from staging_table st
where st.state_code=13;
or by querying the table as part of the query, which doesn't need PL/SQL:
insert into final_table (fst_name, lst_name, state_code, row_nbr)
select st.*, (select max(row_nbr) from final_table) + rownum
from staging_table st
where st.state_code=13;
db<>fiddle
But this isn't a good solution because it doesn't prevent clashes from different processes and sessions trying to insert at the same time; but neither would the cursor loop approach, unless it is catching unique constraint errors and re-attempting with a new value, perhaps.
It would be better to use a sequence, which would be an auto-increment column but you said you can't change the table structure; and you need to let the other processes continue to work without modification. You can still do that with a sequence and trigger approach, having the trigger always set the row_nbr value form the sequence, regardless of whether the insert statement supplied a value.
If you create a sequence that starts from the current maximum, with something like:
create sequence final_seq start with <current max + 1>
or without manually finding it:
declare
start_with pls_integer;
begin
select nvl(max(row_nbr), 0) + 1 into start_with from final_table;
execute immediate 'create sequence final_seq start with ' || start_with;
end;
/
then your trigger could just be:
create trigger final_trig
before insert on final_table
for each row
begin
:new.row_nbr := final_seq.nextval;
end;
/
Then your insert ... select statement doesn't need to supply or even think about the row_nbr value, so you can leave it as you have it now (except I'd avoid select * even in that construct, and list the staging table columns explicitly); and any existing inserts that do supply the row_nbr don't need to be modified and the value they supply will just be overwritten from the sequence.
db<>fiddle showing inserts with and withouth row_nbr specified.

Related

View Performance

I have requirement of performing some calculation on a column of a table with large date set ( 300 GB). and return that value.
Basically I need to create a View on that table. Table has data of 21 years and It is partitioned on date column (Daily). We can not put date condition on View's query and User will put filter on runtime while execution of the view.
For example:
Create view v_view as
select * from table;
Noe I want to query View like
Select * v_view where ts_date between '1-Jan-19' and '1-Jan-20'
How Internally Oracle execute above statement? Will it execute view query first and then put date filter on that?
If so will there not be performance issue ? and how to resolve this?
oracle first generates the view and then applies the filter. you can create a function that input may inserted by user. the function results a create query and if yo run the query then the view will be created. just run:
create or replace function fnc_x(where_condition in varchar2)
return varchar2
as
begin
return ' CREATE OR REPLACE VIEW sup_orders AS
SELECT suppliers.supplier_id, orders.quantity, orders.price
FROM suppliers
INNER JOIN orders
ON suppliers.supplier_id = orders.supplier_id
'||where_condition||' ';
end fnc_x;
this function should be run. input the function is a string like this:
''WHERE suppliers.supplier_name = Microsoft''
then you should run a block like this to run the function's result:
cl scr
set SERVEROUTPUT ON
declare
szSql varchar2(3000);
crte_vw varchar2(3000);
begin
szSql := 'select fnc_x(''WHERE suppliers.supplier_name = Microsoft'') from dual';
dbms_output.put_line(szSql);
execute immediate szSql into crte_vw; -- generate 'create view' command that is depended on user's where_condition
dbms_output.put_line(crte_vw);
execute immediate crte_vw ; -- create the view
end;
In this manner, you just need received where_condition from user.
Oracle can "push" the predicates inside simple views and can then use those predicates to enable partition pruning for optimal performance. You almost never need to worry about what Oracle will run first - it will figure out the optimal order for you. Oracle does not need to mindlessly build the first step of a query, and then send all of the results to the second step. The below sample schema and queries demonstrate how only the minimal amount of partitions are used when a view on a partitioned table is queried.
--drop table table1;
--Create a daily-partitioned table.
create table table1(id number, ts_date date)
partition by range(ts_date)
interval (numtodsinterval(1, 'day'))
(
partition p1 values less than (date '2000-01-01')
);
--Insert 1000 values, each in a separate day and partition.
insert into table1
select level, date '2000-01-01' + level
from dual
connect by level <= 1000;
--Create a simple view on the partitioned table.
create or replace view v_view as select * from table1;
The following explain plan shows "Pstart" and "Pstop" set to 3 and 4, which means that only 2 of the many partitions are used for this query.
--Generate an explain plan for a simple query on the view.
explain plan for
select * from v_view where ts_date between date '2000-01-02' and date '2000-01-03';
--Show the explain plan.
select * from table(dbms_xplan.display(format => 'basic +partition'));
Plan hash value: 434062308
-----------------------------------------------------------
| Id | Operation | Name | Pstart| Pstop |
-----------------------------------------------------------
| 0 | SELECT STATEMENT | | | |
| 1 | PARTITION RANGE ITERATOR| | 3 | 4 |
| 2 | TABLE ACCESS FULL | TABLE1 | 3 | 4 |
-----------------------------------------------------------
However, partition pruning and predicate pushing do not always work when we may think they should. One thing we can do to help the optimizer is to use date literals instead of strings that look like dates. For example, replace
'1-Jan-19' with date '2019-01-01'. When we use ANSI date literals, there is no ambiguity and Oracle is more likely to use partition pruning.

Inserting Record Type

I have a source table, and a history table. They are identical, except the history table has a notes column as the last column.
The table is large, 55 columns, which I do not want to list all the columns. We do not want to use a trigger on this, but just create the history entry in the code itself.
If I simply do an INSERT INTO <history> SELECT * FROM <source> WHERE...... I will get "not enough values".
I'm hoping to do something of this nature: (Note this is just an anonymous block)
DECLARE
v_old_rec company_table%ROWTYPE;
BEGIN
SELECT * INTO v_old_rec
FROM company_table
WHERE company_id = 32789;
INSERT INTO company_table_hist
VALUES v_old_rec || ',MONTHLY UPDATE';
END;
Anything like this possible, so I do not have to list 55 columns?
Thank you.
It's quite simple actually:
INSERT INTO company_table_hist
(SELECT t.*, MONTHLY_UPD
FROM company_table t
WHERE company_id = 32789);
Of course monthly_upd must be bound somehow

Oracle finding last row inserted

Let say I have table my table has values(which they are varchar):
values
a
o
g
t
And I have insert a new value called V
values
V
a
o
g
t
Is there a way or query that can specify what is the last value was insert in the column ? the desired query : select * from dual where rown_num = count(*) -- just an example and the result will be V
Rows in a table have no inherent order. rownum is a pseudocolumn that's part of the select so it isn't useful here. There is no way to tell where in the storage a new row will physically be placed, so you can't rely on rowid, for example.
The only way to do this reliably is to have a timestamp column (maybe set by a trigger so you don't have to worry about it). That would let you order the rows by timestamp and find the row with the highest (most recent) timestamp.
You are still restricted by the precision of the timestamp, as I discovered creating a SQL Fiddle demo; without forcing a small gap between the inserts the timestamps were all the same, but then it only seems to support `timestamp(3). That probably won't be a significant issue in the real world, unless you're doing bulk inserts, but then the last row inserted is still a bit of an arbitrary concept.
As quite correctly pointed out in comments, if the actual time doesn't need to be know, a numeric field populated by a sequence would be more reliable and performant; another SQL Fiddle demo here, and this is the gist:
create table t42(data varchar2(10), id number);
create sequence seq_t42;
create trigger bi_t42
before insert on t42
for each row
begin
:new.id := seq_t42.nextval;
end;
/
insert into t42(data) values ('a');
insert into t42(data) values ('o');
insert into t42(data) values ('g');
insert into t42(data) values ('t');
insert into t42(data) values ('V');
select data from (
select data, row_number() over (order by id desc) as rn
from t42
)
where rn = 1;

oracle - sequences without a sequence

I want to populate a column with a sequential number but a single sequence is not sufficient. This column will behave somewhat like a 'sub id' if you will; an incrementing id for groups of records in the table.
The plan is to get the 'next number in the sequence' when inserting using a trigger, much like a normal sequence may be used. However, rather than just the 'next number', it needs to be the 'next number' in a given result set.
Consider the following example data where the display_id column is the sequence I need help managing and it is dependent on the record's value for group_name..
id | group_name | display_id
------------------------------
5 | GroupA | 3
4 | GroupA | 2
3 | GroupA | 1
2 | GroupB | 2
1 | GroupB | 1
I'm thinking of a query like this query to get the 'next number' for GroupA:
select max(record_id) + 1
from my_records
where group_name = 'GroupA'
For GroupA it returns 4, but for GroupB it returns 3.
We could, of course, use the above query but would lose the atomic benefits of a sequence. Is there any way to manage such a sequence confidently?
We are not concerned about potentially skipping numbers (as sequences may).
Edit:
We are comfortable with a number or two being missed due to rollbacks and the like (as with sequences). However, our requirement is still that the display_id column maintain multiple sequences.
Although I would strongly recommend against it (preferring to use a single sequence and just accept that there will be larger than expected gaps), you can build your own pseudo-sequence table
CREATE TABLE my_sequences (
sequence_name VARCHAR2(30) PRIMARY KEY,
sequence_val NUMBER
);
insert a couple of rows
INSERT INTO my_sequences( sequence_name, sequence_val )
VALUES( 'GroupA', 1 );
INSERT INTO my_sequences( sequence_name, sequence_val )
VALUES( 'GroupB', 1 );
and then write a function to get the next sequence value
CREATE FUNCTION get_nextval( p_sequence_name IN VARCHAR2 )
RETURN NUMBER
IS
l_val NUMBER;
BEGIN
SELECT sequence_val
INTO l_val
FROM my_sequences
WHERE sequence_name = p_sequence_name
FOR UPDATE;
UPDATE my_sequences
SET sequence_val = sequence_val + 1
WHERE sequence_name = p_sequence_name;
RETURN l_val;
END;
This will lock the row in the table for the particular sequence until the transaction that retrieved the next row either commits or rolls back. This will radically decrease the scalability of your application compared to using Oracle sequences by ensuring that only one session can be inserting a row for a particular group_name at a time-- the others will block waiting for the sequence. If you have a system with a relatively small number of concurrent users (or a relatively large number of group_name values), that may be acceptable to you. But in general it is a poor practice. Depending on the Oracle version, you may be able to use autonomous transactions to increase concurrency but that just adds one more bit of complexity to the solution. At the point that you're really worried about scalability, you'd really want to push back on the whole design and just use an Oracle sequence.
Create an unique composite index on group_name + display_id columns.
Then use your code and in case exception is thrown - re-run the next value generation.
PS: personally I don't like it, but it's likely in this case there is no good alternatives

Pattern to substitute for MERGE INTO Oracle syntax when not allowed

I have an application that uses the Oracle MERGE INTO... DML statement to update table A to correspond with some of the changes in another table B (table A is a summary of selected parts of table B along with some other info). In a typical merge operation, 5-6 rows (out of 10's of thousands) might be inserted in table B and 2-3 rows updated.
It turns out that the application is to be deployed in an environment that has a security policy on the target tables. The MERGE INTO... statement can't be used with these tables (ORA-28132: Merge into syntax does not support security policies)
So we have to change the MERGE INTO... logic to use regular inserts and updates instead. Is this a problem anyone else has run into? Is there a best-practice pattern for converting the WHEN MATCHED/WHEN NOT MATCHED logic in the merge statement into INSERT and UPDATE statements? The merge is within a stored procedure, so it's fine for the solution to use PL/SQL in addition to the DML if that is required.
Another way to do this (other than Merge) would be using two sql statements one for insert and one for update. The "WHEN MATCHED" and "WHEN NOT MATCHED" can be handled using joins or "in" Clause.
If you decide to take the below approach, it is better to run the update first (sine it only runs for the matching records) and then insert the non-Matching records. The Data sets would be the same either way, it just updates less number of records with the order below.
Also, Similar to the Merge, this update statement updates the Name Column even if the names in Source and Target match. If you dont want that, add that condition to the where as well.
create table src_table(
id number primary key,
name varchar2(20) not null
);
create table tgt_table(
id number primary key,
name varchar2(20) not null
);
insert into src_table values (1, 'abc');
insert into src_table values (2, 'def');
insert into src_table values (3, 'ghi');
insert into tgt_table values (1, 'abc');
insert into tgt_table values (2,'xyz');
SQL> select * from Src_Table;
ID NAME
---------- --------------------
1 abc
2 def
3 ghi
SQL> select * from Tgt_Table;
ID NAME
---------- --------------------
2 xyz
1 abc
Update tgt_Table tgt
set Tgt.Name =
(select Src.Name
from Src_Table Src
where Src.id = Tgt.id
);
2 rows updated. --Notice that ID 1 is updated even though value did not change
select * from Tgt_Table;
ID NAME
----- --------------------
2 def
1 abc
insert into tgt_Table
select src.*
from Src_Table src,
tgt_Table tgt
where src.id = tgt.id(+)
and tgt.id is null;
1 row created.
SQL> select * from tgt_Table;
ID NAME
---------- --------------------
2 def
1 abc
3 ghi
commit;
There could be better ways to do this, but this seems simple and SQL-oriented. If the Data set is Large, then a PL/SQL solution won't be as performant.
There are at least two options I can think of aside from digging into the security policy, which I don't know much about.
Process the records to merge row by row. Attempt to do the update, if it fails to update then insert, or vise versa, depending on whether you expect most records to need updating or inserting (ie optimize for the most common case that will reduce the number of SQL statements fired), eg:
begin
for row in (select ... from source_table) loop
update table_to_be_merged
if sql%rowcount = 0 then -- no row matched, so need to insert
insert ...
end if;
end loop;
end;
Another option may be to bulk collect the records you want to merge into an array, and then attempted to bulk insert them, catching all the primary key exceptions (I cannot recall the syntax for this right now, but you can get a bulk insert to place all the rows that fail to insert into another array and then process them).
Logically a merge statement has to check for the presence of each records behind the scenes anyway, and I think it is processed quite similarly to the code I posted above. However, merge will always be more efficient than coding it in PLSQL as it will be only 1 SQL call instead of many.

Resources