Unique constraint violated message on inserting code below - oracle

I am writing this code which throws an error in primary key:
DECLARE CURSOR A1 AS
SELECT
TRANS_DET_ID,
(SELECT MAX (NVL(TRANS_DET_DET_ID, 0) + 1)
FROM PROD_OPERATIONS_RATE) DET_ID,
OPER_CODE, ART_CODE, RATE, FROM_DATE, CLOSE_IND
FROM
PROD_OPERATIONS_RATE
WHERE
TRANS_DET_ID = 1
AND OPER_CODE = 1
AND RATE = 2.3005;
a1_var A1%ROWTYPE;
BEGIN
OPEN A1;
LOOP
FETCH A1
INTO a1_var;
EXIT WHEN A1%NOTFOUND;
INSERT INTO PROD_OPERATIONS_RATE (
TRANS_DET_ID,TRANS_DET_DET_ID,OPER_CODE,ART_CODE,RATE,FROM_DATE,CLOSE_IND)
VALUES (1,a1_var.DET_ID,1,a1_var.ART_CODE,2.50,DATE '2022-05-01','N');
END LOOP;
CLOSE A1;
COMMIT;
END;
I want to insert data same table throw conditions met, it throws an error of unique constraint on column TRANS_DET_DET_ID which is the primary key. What am I doing wrong? Please can anyone help me with this? Regards

Unique (primary) key value which is calculated as MAX + 1 is almost always wrong. Switch to a sequence.
Find MAX trans_det_det_id value:
SELECT MAX (trans_det_det_id) max_id FROM PROD_OPERATIONS_RATE;
Create sequence as max_id + 1 (I put a dummy value of 1000; you'd use what query actually returns):
CREATE SEQUENCE seq START WITH 1000;
Now, use the sequence in your PL/SQL script:
DECLARE
CURSOR A1 IS
SELECT TRANS_DET_ID,
--(SELECT MAX (NVL (TRANS_DET_DET_ID, 0) + 1)
-- FROM PROD_OPERATIONS_RATE) DET_ID,
OPER_CODE,
ART_CODE,
RATE,
FROM_DATE,
CLOSE_IND
FROM PROD_OPERATIONS_RATE
WHERE TRANS_DET_ID = 1
AND OPER_CODE = 1
AND RATE = 2.3005;
a1_var A1%ROWTYPE;
BEGIN
OPEN A1;
LOOP
FETCH A1 INTO a1_var;
EXIT WHEN A1%NOTFOUND;
INSERT INTO PROD_OPERATIONS_RATE (TRANS_DET_ID,
TRANS_DET_DET_ID,
OPER_CODE,
ART_CODE,
RATE,
FROM_DATE,
CLOSE_IND)
VALUES (1,
seq.NEXTVAL, -- a1_var.DET_ID,
1,
a1_var.ART_CODE,
2.50,
DATE '2022-05-01',
'N');
END LOOP;
CLOSE A1;
COMMIT;
END;
By the way, if there's no particular reason for doing it slowly in a loop, use an ordinary INSERT INTO statement (SQL, not PL/SQL), it'll be much faster:
INSERT INTO prod_operations_rate (trans_det_id,
trans_det_det_id,
oper_code,
art_code,
rate,
from_date,
close_ind)
SELECT trans_det_id,
seq.NEXTVAL,
oper_code,
art_code,
rate,
from_date,
close_ind
FROM prod_operations_rate
WHERE trans_det_id = 1
AND oper_code = 1
AND rate = 2.3005;

Related

Oracle Cursors : Invalid Identifier

I need to write code that increases the salary of an employee who is over 40 years old.
Here is my code:
DECLARE
CURSOR kurs IS SELECT ID_PRACOWNIKA , pensja_BR, wiek FROM PRACOWNICY p , OSOBY o ;
ID_PRACOWNIKA decimal(2):=0;
pensja DECIMAL(8,2);
wiek DECIMAL(2);
BEGIN
OPEN kurs;
LOOP
IF wiek > 40
THEN
UPDATE PRACOWNICY
SET pensja = PENSJA_BR * 1.02
WHERE ID_PRACOWNIKA = ID_PRACOWNIKA;
dbms_OUTPUT.put_line( ID_PRACOWNIKA|| '-'||pensja);
END IF;
ID_PRACOWNIKA := ID_PRACOWNIKA+1;
EXIT WHEN ID_PRACOWNIKA=6;
END LOOP;
CLOSE kurs;
END;
Unfortunately I have SQL error
SQL Error [6550] [65000]: ORA-06550: line 14, column 12:
PL/SQL: ORA-00904: "PENSJA": invalid identifier
ORA-06550: line 13, column 7:
PL/SQL: SQL Statement ignored
Osoby table strucuture:
Id_osoby NUMBER CONSTRAINT osoby_pk PRIMARY KEY,
Imie VARCHAR2(15) NOT NULL,
Nazwisko VARCHAR2(30) NOT NULL,
Wiek NUMBER NOT NULL CONSTRAINT ch_wiek CHECK((Wiek>=0) AND (Wiek<=125)),
Stan_cywilny VARCHAR2(12) NOT NULL,
Telefon VARCHAR2(20),
Pesel CHAR(11) NOT NULL CONSTRAINT osoba_uni UNIQUE,
Id_adresu NUMBER NOT NULL,
CONSTRAINT os_ad_fk FOREIGN KEY (Id_adresu) REFERENCES Adresy(Id_adresu)
Pracownicy table structure:
Id_pracownika NUMBER CONSTRAINT pracownik_pk PRIMARY KEY,
Id_osoby NUMBER NOT NULL CONSTRAINT pr_unique UNIQUE,
Id_stanowiska NUMBER NOT NULL,
Staz NUMBER NOT NULL CONSTRAINT ch_staz CHECK((Staz>=0) AND (Staz<=45)),
Pensja_br NUMBER NOT NULL CONSTRAINT pen_staz CHECK(Pensja_br>=1226),
CONSTRAINT pr_os_fk FOREIGN KEY (Id_osoby) REFERENCES Osoby(Id_osoby),
CONSTRAINT pr_st_fk FOREIGN KEY (Id_stanowiska) REFERENCES Stanowiska(Id_stanowiska)
I think you can use the single update statement but as you want to print the details also. You can go with FOR loop as follows:
BEGIN
FOR I IN (
SELECT P.ID_PRACOWNIKA,
P.PENSJA_BR * 1.02 AS PENSJA_BR
FROM PRACOWNICY P
JOIN OSOBY O
ON P.ID_OSOBY = O.ID_OSOBY
WHERE O.WIEK > 40
) LOOP
UPDATE PRACOWNICY P
SET P.PENSJA_BR = I.PENSJA_BR
WHERE P.ID_PRACOWNIKA = I.ID_PRACOWNIKA;
DBMS_OUTPUT.PUT_LINE(I.ID_PRACOWNIKA || '-' || I.PENSJA_BR);
END LOOP;
END;
If you just want to update the data with a single query then you can use the following update SQL:
UPDATE PRACOWNICY P
SET P.PENSJA_BR = P.PENSJA_BR * 1.02
WHERE EXISTS (
SELECT 1
FROM OSOBY O
WHERE P.ID_OSOBY = O.ID_OSOBY
AND O.WIEK > 40
);
I am not quite sure what the expected result would be but I hava question. If you already have the columns ID_PRACOWNIKA , pensja_BR, wiek retrieved from a table why do you create some variables with the same name?
Now, what I understood from your problem is that you try to change the PENSJA_BR Column to have the value of PENSJA_BR * 1.02 for each row which has the value >40 in the column WIEK.
I think that the following code might help you with your Problem. I have tested it within my testing environment and it updates the column PENSJA_BR accordingly. Nevertheless, you will have to update it to your needs and add what you need extra.
DECLARE
check_stauts NUMBER;
CURSOR kurs IS
SELECT id_pracownika, pensja_br, wiek
FROM pracownicy;
BEGIN
FOR i IN kurs LOOP
if i.wiek > 40 THEN
UPDATE pracownicy
SET
pensja_br = i.pensja_br * 1.02
WHERE
id_pracownika = i.id_pracownika;
END IF;
END LOOP;
END;
PS: when trying to assign a value to a variable use:
pensja := PESNJA_BR * 1.02
I believe it's UPDATE clause causing the issue (and the error description say the error is on line 14 of your script)
SET pensja = PENSJA_BR * 1.02
As I see in the beginning, there is a column named pensja_BR in the table PRACOWNICY but you are trying to update the "pensija" column. Which presumably does not exist in the table.
Another things to mention here is there is a cartesian join in your cursor because you're joining two tables without any join/where conditions
UPD: the loop will probably not work here as well because you did not fetched data from. Opening a cursor will not fetch data automatically. You have to either fetch it explicitly every time in the loop
open kurs;
loop
fetch kurs into some_variable
...
end loop;
or to use another for..loop statement in order to loop through the cursor
for k in kurs loop
...
end loop;
So, you need to update cursor definition with following where clause
WHERE p.id_osoby = o.id_osoby and wiek > 40
Then remove the IF > 40 statement, you don't need it anymore.
Declare a rowtype variable in DECLARE:
kurs_l kurs%rowtype;
in PLSQL:
open kurs;
loop
fetch kurs into kurs_l;
exit when kurs%notfound;
... do your stuff ...
end loop;
close kurs;

How to update each row in a table Oracle PLSQL

How can I create a procedure to verify each row of a table and update a field according to the established statement?
I have 4 fields that I work with
SYS_UPDATE_PING
UPDATE_PING
PING_STATUS
TIME_OUT_PING
Here is my code:
create or replace procedure SP_DASHBOARD_PINGSTATUS is
begin
declare
sp_ping_final number;
BEGIN
Update rsmes.tb_op_pc_monitoring_v4 t
set t.sys_update_ping = sysdate;
for c in (select ((SYS_UPDATE_PING - UPDATE_PING)*60*60*24) as PING_RESULT into sp_ping_final from TB_OP_PC_MONITORING_V4)
loop
sp_ping_final := c.ping_result;
if c.ping_result <= 5 then
Update rsmes.tb_op_pc_monitoring_v4 tg
set tg.ping_status = 'OK',
tg.time_out_ping = sp_ping_final;
else
Update rsmes.tb_op_pc_monitoring_v4 tn
set tn.ping_status = 'NG',
tn.time_out_ping = sp_ping_final;
end if;
end loop;
commit;
END;
end SP_DASHBOARD_PINGSTATUS;
I make a time difference between SYS_UPDATE_PING and UPDATE_PING, if the result in seconds is less than 5 it must update the PING_STATUS field to OK and put the seconds difference in TIME_OUT_PING, otherwise it will update NG and put the seconds difference in TIME_OUT_PING,
I want it to compare on each row but it updates me on all fields instead of one by one.
I know I'm forgetting something but could you help me find my fault and know the solution?
Thank you
That's because UPDATE statements miss the WHERE clause. Without it, you're always updating all rows in the table.
It means that cursor should contain some kind of an ID, which you'll the reuse in UPDATE. For example:
for c in (select id, --> this
((sys_update_ping - ...)
) loop
...
update tb_op_pc_monitoring_v4 tg set
tg.ping_status = 'OK'
where tg.id = c.id; --> this
...
end loop;
Also, you don't SELECT INTO in cursor. Remove the sp_ping_final entirely.
Finally, I'd say that you don't need PL/SQL (loop especially) at all. The whole code you wrote can be rewritten into a single
UPDATE tb_op_pc_monitoring_v4 tg
SET tg.ping_status =
CASE
WHEN (SYSDATE - tg.update_ping) * 60 * 60 * 24 <= 5 THEN 'OK'
ELSE 'NG'
END,
tg.time_out_ping = (SYSDATE - tg.update_ping) * 60 * 60 * 24;
which should work way faster than your row-by-row processing.

Sequential vs parallel solution

I will try to present my problem as simplified as possible.
Assume that we have 3 tables in Oracle 11g.
Persons (person_id, name, surname, status, etc )
Actions (action_id, person_id, action_value, action_date, calculated_flag)
Calculations (calculation_id, person_id,computed_value,computed_date)
What I want is for each person that meets certain criteria (let's say status=3)
I should get the sum of action_values from the Actions table where calculated_flag=0. (something like this select sum(action_value) from Actions where calculated_flag=0 and person_id=current_id).
Then I shall use that sum in a some kind of formula and update the Calculations table for that specific person_id.
update Calculations set computed_value=newvalue, computed_date=sysdate
where person_id=current_id
After that calculated_flag for participated rows will be set to 1.
update Actions set calculated_flag=1
where calculated_flag=0 and person_id=current_id
Now this can be easily done sequentially, by creating a cursor that will run through Persons table and then execute each action needed for the specific person.
(I don't provide the code for the sequential solution as the above is just an example that resembles my real-world setup.)
The problem is that we are talking about quite big amount of data and sequential approach seems like a waste of computational time.
It seems to me that this task could be performed in parallel for number of person_ids.
So the question is:
Can this kind of task be performed using parallelization in PL/SQL?
What would the solution look like? That is, what special packages (e.g. DBMS_PARALLEL_EXECUTE), keywords (e.g. bulk collect), methods should be used and in what manner?
Also, should I have any concerns about partial failure of parallel updates?
Note that I am not quite familiar with parallel programming with PL/SQL.
Thanks.
Edit 1.
Here my pseudo code for my sequential solution
procedure sequential_solution is
cursor persons_of_interest is
select person_id from persons
where status = 3;
tempvalue number;
newvalue number;
begin
for person in persons_of_interest
loop
begin
savepoint personsp;
--step 1
select sum(action_value) into tempvalue
from actions
where calculated_flag = 0
and person_id = person.person_id;
newvalue := dosomemorecalculations(tempvalue);
--step 2
update calculations set computed_value = newvalue, computed_date = sysdate
where person_id = person.person_id;
--step 3
update actions set calculated_flag = 1;
where calculated_flag = 0 and person_id = person.person_id;
--step 4 (didn't mention this step before - sorry)
insert into actions
( person_id, action_value, action_date, calculated_flag )
values
( person.person_id, 100, sysdate, 0 );
exception
when others then
rollback to personsp;
-- this call is defined with pragma AUTONOMOUS_TRANSACTION:
log_failure(person_id);
end;
end loop;
end;
Now, how would I speed up the above either with forall and bulk colletct or with parallel programming Under the following constrains:
proper memory management (taking into consideration large amount of data)
For a single person if one part of the step sequence fails - all steps should be rolled back and the failure logged.
I can propose the following. Let's say you have 1 000 000 rows in persons table, and you want to process 10 000 persons per iteration. So you can do it in this way:
declare
id_from persons.person_id%type;
id_to persons.person_id%type;
calc_date date := sysdate;
begin
for i in 1 .. 100 loop
id_from := (i - 1) * 10000;
id_to := i * 10000;
-- Updating Calculations table, errors are logged into err$_calculations table
merge into Calculations c
using (select p.person_id, sum(action_value) newvalue
from Actions a join persons p on p.person_id = a.person_id
where a.calculated_flag = 0
and p.status = 3
and p.person_id between id_from and id_to
group by p.person_id) s
on (s.person_id = c.person_id)
when matched then update
set c.computed_value = s.newvalue,
c.computed_date = calc_date
log errors into err$_calculations reject limit unlimited;
-- updating actions table only for those person_id which had no errors:
merge into actions a
using (select distinct p.person_id
from persons p join Calculations c on p.person_id = c.person_id
where c.computed_date = calc_date
and p.person_id between id_from and id_to)
on (c.person_id = p.person_id)
when matched then update
set a.calculated_flag = 1;
-- inserting list of persons for who calculations were successful
insert into actions (person_id, action_value, action_date, calculated_flag)
select distinct p.person_id, 100, calc_date, 0
from persons p join Calculations c on p.person_id = c.person_id
where c.computed_date = calc_date
and p.person_id between id_from and id_to;
commit;
end loop;
end;
How it works:
You split the data in persons table into chunks about 10000 rows (depends on gaps in numbers of ID's, max value of i * 10000 should be knowingly more than maximal person_id)
You make a calculation in the MERGE statement and update the Calculations table
LOG ERRORS clause prevents exceptions. If an error occurs, the row with the error will not be updated, but it will be inserted into a table for errors logging. The execution will not be interrupted. To create this table, execute:
begin
DBMS_ERRLOG.CREATE_ERROR_LOG('CALCULATIONS');
end;
The table err$_calculations will be created. More information about DBMS_ERRLOG package see in the documentation.
The second MERGE statement sets calculated_flag = 1 only for rows, where no errors occured. INSERT statement inserts the these rows into actions table. These rows could be found just with the select from Calculations table.
Also, I added variables id_from and id_to to calculate ID's range to update, and the variable calc_date to make sure that all rows updated in first MERGE statement could be found later by date.

insert from one table to another table use procedure

I want to insert from one table to other with some edits in values. source table has 20,000,000 records and insert and commit them impossible . so i write a procedure to commit each 1000 insert in a loop. but it does not work. what is the problem?
CREATE OR REPLACE PROCEDURE CONVERT_INTO_K2 IS
batch_size number;
row_num number;
CURSOR trans IS
select rownum,KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRPAN),
t.TRPRRN, t.TRSIDE,t.switchcode,t.kcb_switchcode,
t.TERMID, t.TRTERMID,t.TRPOSCCOD,t.TRCAID,
t.TRMTI,t.TRPRCOD,t.TR87RSPCOD,t.TRMSGRSN,
t.TRREVFLG,t.TRSETTMD,t.TRCURCOD,t.TRAMNT,
t.TRORGAMNT,t.TRCHBLCUR,t.TRFEEAMNT,t.TRCHBLAMNT,
t.TRFRWIIC,t.TRACQIIC,t.TRRECIIC,t.PTRTRCNO,
t.BTRRN, t.TRTRACEDT, t.TRRSPDT, t.TRLDATE,
t.TRLTIME, t.BAID, t.BADATE,t.TRACNTID1,
KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRACNTID2),
t.ATJHNO,t.ATJHDAT, t.TRORGDTELM,t.TRADDATA,
KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRACNTID2),
Trn_ID.Nextval, Trn_diag.Nextval,
1, 1, 1, 1, 1, 1, 1, 1, 1
from P912.KCTRNS t
where t.trprcod != 47
and rownum < 200;
BEGIN
batch_size := 0;
row_num:=0;
FOR rec IN trans LOOP
batch_size := batch_size + 1;
row_num := row_num + 1;
if MOD( row_num, 1000 ) != 0 then
insert into P912.KCTRNS2
(srcpan,rfrnnum, trnsid,swchcod, prswchcod,intrtrmid,
xtrntrmid,trmcod, aptrid,msgtypidnt,trntyp,
rspcod, msqrsn,rvrsflg,sttlsts,currcod,
amt,origamt,crdhldrcurrcod,feeamt,
crdhldrdiscamt, isurcrdinstid,acqrcrdinstid,
rcvrcrdinstid,trcnum,intrrfrnnum,
rcvdt,rspdt, prrcvdtsctn,prrcvtmsctn,
btchid, btchiopendt,firsacctnum,
scndacctnum,docnum,docdt, origdtelmt,
dditdat,dstpan, id, diag, mngcod,
funccod, sttlcod,trnres, custno,
crdlesstrcno,accttyp1,accttyp2,chnltyp)
Values
(KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRPAN),
t.TRPRRN, t.TRSIDE,t.switchcode,t.kcb_switchcode,
t.TERMID, t.TRTERMID,t.TRPOSCCOD,t.TRCAID,
t.TRMTI,t.TRPRCOD,t.TR87RSPCOD,t.TRMSGRSN,
t.TRREVFLG,t.TRSETTMD,t.TRCURCOD,t.TRAMNT,
t.TRORGAMNT,t.TRCHBLCUR,t.TRFEEAMNT,t.TRCHBLAMNT,
t.TRFRWIIC,t.TRACQIIC,t.TRRECIIC,t.PTRTRCNO,
t.BTRRN, t.TRTRACEDT, t.TRRSPDT, t.TRLDATE,
t.TRLTIME, t.BAID, t.BADATE,t.TRACNTID1,
KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRACNTID2),
t.ATJHNO,t.ATJHDAT, t.TRORGDTELM,t.TRADDATA,
KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRACNTID2),
Trn_ID.Nextval, Trn_diag.Nextval,
1, 1, 0, 1, 1, 1, 1, 1, 1);
else
insert into P912.KCTRNS2
(srcpan,rfrnnum, trnsid,swchcod, prswchcod,intrtrmid,
xtrntrmid,trmcod, aptrid,msgtypidnt,trntyp,
rspcod, msqrsn,rvrsflg,sttlsts,currcod,
amt,origamt,crdhldrcurrcod,feeamt,
crdhldrdiscamt, isurcrdinstid,acqrcrdinstid,
rcvrcrdinstid,trcnum,intrrfrnnum,
rcvdt,rspdt, prrcvdtsctn,prrcvtmsctn,
btchid, btchiopendt,firsacctnum,
scndacctnum,docnum,docdt, origdtelmt,
dditdat,dstpan, id, diag, mngcod,
funccod, sttlcod,trnres, custno,
crdlesstrcno,accttyp1,accttyp2,chnltyp)
Values
(KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRPAN),
t.TRPRRN, t.TRSIDE,t.switchcode,t.kcb_switchcode,
t.TERMID, t.TRTERMID,t.TRPOSCCOD,t.TRCAID,
t.TRMTI,t.TRPRCOD,t.TR87RSPCOD,t.TRMSGRSN,
t.TRREVFLG,t.TRSETTMD,t.TRCURCOD,t.TRAMNT,
t.TRORGAMNT,t.TRCHBLCUR,t.TRFEEAMNT,t.TRCHBLAMNT,
t.TRFRWIIC,t.TRACQIIC,t.TRRECIIC,t.PTRTRCNO,
t.BTRRN, t.TRTRACEDT, t.TRRSPDT, t.TRLDATE,
t.TRLTIME, t.BAID, t.BADATE,t.TRACNTID1,
KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRACNTID2),
t.ATJHNO,t.ATJHDAT, t.TRORGDTELM,t.TRADDATA,
KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRACNTID2),
Trn_ID.Nextval, Trn_diag.Nextval,
1, 1, sttl_cod.Nextval, 1, 1, 1, 1, 1, 1);
end if;
IF batch_size = 10 THEN
begin
COMMIT;
end;
batch_size := 0;
end if;
END loop;
EXCEPTION
WHEN others THEN
ROLLBACK;
END CONVERT_INTO_K2;
The Values expression references t instead of rec the actual name of the cursor record. Try changing those t (only those in the Values expression, not those in the select) into rec.
Also make an alias for KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRPAN) it will be easier. Otherwise you will have to double quote (") and upper case the column name in the Values clause.
Finally and most importantly at first sight your need would be better served by using a insert into … from (select …) syntax. Does this not work?
Failling that, a more robust approach would be to add some form of state column to your source (PSAM952.KCTRNS) table that would indicate that the entry has already been inserted into the destination (PSAM961.KCTRNS2) table. Say 0 for the default value, 1 meaning it will be part of next batch, and 2 to say it has been copied.
In your example, you are doing a mod() of row_num to provide some conditional logic. But you do not have an order-by on you cursor, so the row to which the row_num is applied is not deterministic. Of course one you apply and order-by, then you need to rethink rownum, since that is applied prior to an order-by. So you really need to re-evaluate the logic of what you are trying to do.
Having said that, you can perform conditional logic during the insert as shown below.
insert /*+ APPEND */ into PSAM961.KCTRNS2
select KWP_Trns_Utils.Get_Acnt_From_TRACNTID(t.TRPAN)
, t.TRPRRN
,etc
...
case when xxx=0 then stt1_cod.nextval else 0 end
,1
,1
,etc
from PSAM952.KCTRNS t
where t.trprcod != 47
and rownum < 200;

How to efficiently transform narrow table into wide in Oracle?

I have a narrow table with the following columns:
<Customer ID> <Field ID> <Value>, all of them are numbers.
I want to reshape this table into the wide format:
<Customer ID> <Field1> <Field2> <Field3> ...
I have a separate dictionary table DIC_FIELDS that translates Field ID into Field Name.
I work on EXADATA server. The narrow table has 2.5 billion records, and we have about 200 fields.
The obvious simple solution below badly fills up all temporary space on our EXADATA server.
create table WIDE_ADS as (
CUSTOMERID
,max(case when FIELDID = 1 then VALUE end) as GENDER
,max(case when FIELDID = 2 then VALUE end) as AGE
,max(case when FIELDID = 3 then VALUE end) as EDUCATION
from NARROW_ADS
group by CUSTOMERID
);
We tried also a cleverer and manual method:
create index index1
on SZEROKI_ADS(CUSTOMERID);
DECLARE
rowidWide rowid;
type tColNames is table of STRING(32000) index by pls_integer ;
arrColNames tColNames;
x_CustomerID number;
strColName varchar2(32);
strColvalue varchar2(32000);
strSQL varchar2(200);
lngCounter pls_integer;
lngFieldID pls_integer;
BEGIN
lngCounter := 0;
-- we pre-load the dictionary arrColNames to speedup lookup.
for DIC_EL in (select * from DIC_FIELDS order by FIELDID) LOOP
lngFieldID := to_number(DIC_EL.FIELDID);
arrColNames(lngFieldID) := DIC_EL.FIELDNAME;
END LOOP;
FOR NARROW_REC IN (SELECT * FROM NARROW_ADS where VALUE is not null ) LOOP
strColName := arrColNames(NARROW_REC.FIELDID);
strColvalue := NARROW_REC.VALUE;
x_IDKlienta := NARROW_REC.CUSTOMERID;
BEGIN
select rowid into rowidWide from WIDE_ADS
where CUSTOMERID = NARROW_REC.CUSTOMERID;
strSQL := 'update :1 set :2 = :3 where rowid = :4';
execute immediate strSQL using WIDE_ADS, strColName, strColvalue, rowidWide;
EXCEPTION
WHEN NO_DATA_FOUND THEN
strSQL :=
'insert into '|| WIDE_ADS ||' (CUSTOMERID, '|| strColName ||')
values
(:1, :2)';
execute immediate strSQL using x_CustomerID, to_number(strColvalue) ;
END;
IF lngCounter=10000 THEN
COMMIT;
lngCounter:=0;
dbms_output.put_line('Clik...');
ELSE
lngCounter:=lngCounter+1;
END IF;
END LOOP;
END;
Although it doesn't take a temp, it fails miserably performance-wise; it processes 10 000 records in 50 sec - that is about 1000 times slower, then expected.
What can we do to speed up the process?
As Lalit comments, try to do it in chunks based on CUSTOMERID.
First, create a index on CUSTOMERID (if it does not exist):
CREATE INDEX INDNARROWADS ON NARROW_ADS(CUSTOMERID);
Second, we are going to create an auxiliary table to compute buckets based on CUSTOMERID (in this example we create 1000 buckets, 1 bucket will represent 1 block insert statement):
CREATE TABLE BUCKETS(MINCUSTOMER, MAXCUSTOMER, BUCKETNUM) AS
SELECT MIN(CUSTOMERID), MAX(CUSTOMERID), BUCKET
FROM (SELECT CUSTOMERID,
WIDTH_BUCKET(CUSTOMERID,
(SELECT MIN(CUSTOMERID) FROM NARROW_ADS),
(SELECT MAX(CUSTOMERID) FROM NARROW_ADS),
1000) BUCKET
FROM NARROW_ADS)
GROUP BY BUCKET;
You can use more/less buckets modifying the fourth argument of WIDTH_BUCKET function.
Third, create the WIDE_ADS table (the structure with no data). You should do it manually (with special attention on storage parameters) but you can also use your own query with a WHERE false condition:
create table WIDE_ADS as select
CUSTOMERID
,max(case when FIELDID = 1 then VALUE end) as GENDER
,max(case when FIELDID = 2 then VALUE end) as AGE
,max(case when FIELDID = 3 then VALUE end) as EDUCATION
from NARROW_ADS
where 1=0;
Fourth, execute your query over each bucket (1 bucket means 1 insert statement):
BEGIN
FOR B IN (SELECT * FROM BUCKETS ORDER BY BUCKETNUM) LOOP
INSERT INTO WIDE_ADS
SELECT
CUSTOMERID
,max(case when FIELDID = 1 then VALUE end) as GENDER
,max(case when FIELDID = 2 then VALUE end) as AGE
,max(case when FIELDID = 3 then VALUE end) as EDUCATION
FROM NARROW_ADS
WHERE CUSTOMERID BETWEEN B.MINCUSTOMER AND B.MAXCUSTOMER
GROUP by CUSTOMERID;
COMMIT;
END LOOP;
END;
And finally, drop auxiliary table (and index if it is not necessary).
Oracle optimizer should use the index on CUSTOMERID to perform an "index range scan" over NARROW_ADS. So, each INSERT should find efficiently its corresponding interval.
Note that WIDTH_BUCKETS creates buckets based on uniform divisions over the specified interval on CUSTOMERID (from min to max values). It does not create buckets based on uniform number of rows. And also note that NARROW_ADS must not be modified while this process is being executed.
As the PL/SQL block executes a COMMIT on each iteration and the loop iterates over buckets using the BUCKETNUM order, you can see how WIDE_ADS grows and which bucket is being processed (retrieving the max CUSTOMERID from WIDE_ADS and find its corresponding bucket on BUCKETS table).
If temporary space usage is to high, then increase the number of buckets (each insert will be smaller).

Resources