We created a materialized view with fast refresh on commit. We have seven base tables for MV and corresponding logs created too. We have Golden Gate configured for replication on base tables. Due to load on MV Golden Gate job on base tables getting abended.
create materialized view MV_SPON_PERMIT_RES
refresh fast on commit
as
SELECT
p.rowid p_rowid,
pd.rowid pd_rowid,
ep.rowid ep_rowid,
tl.rowid tl_rowid,
la.rowid la_rowid,
ip.rowid ip_rowid,
pp.rowid pp_rowid,
on tableA,tableB,TableC
union all
p.rowid p_rowid,
pd.rowid pd_rowid,
ep.rowid ep_rowid,
tl.rowid tl_rowid,
la.rowid la_rowid,
ip.rowid ip_rowid,
pp.rowid pp_rowid
on tableX,tableY,tableZ
Problem: fast refresh on commit is making Golden Gate job abend:
DELETE FROM "VISION_INTG"."MV_SPON_PERMIT_RES" SNA$
WHERE "PP_ROWID" IN (SELECT/*+ NO_MERGE HASH_SJ */ *
FROM (SELECT CHARTOROWID ("MAS$"."M_ROW$$") RID$
FROM "VISION_INTG"."MLOG$test" "MAS$"
WHERE "MAS$".SNAPTIME$$ > :B_ST0) MAS$)
AND "FILE_TYPE" = 'Permit'
Please help me in this.
If we are using Golden gate on replicating the data between layers then we cant use MV with refresh fast commit. we can go MV on demand which should refresh data in incremental way. If incremental way refresh is taking time then we need to check the system parameter.
_mv_refresh_use_stats
As per oracle documentation it should be set to false
alter system set "_mv_refresh_use_stats"=FALSE;
Related
Problem: A significant performance degradation during complete refresh when indexes are active. I'm not really sure why having indexes active during a complete refresh would cause a significant difference in performance. At the moment our data warehouse has an issue with over indexing, but I was astonished to see a huge performance degradation with even just one active index vs. no active index on a complete refresh.
Oracle Version 12c
Research:
Materialized view refresh terrible performance degradation
I found this on SO, but it doesn't necessarily answer my question why the indexes would cause a reduction in performance. I may proceed with the suggestion to drop indexes and rebuild after the complete refresh, but I'm still trying figure out the WHY.
Example Performance Test:
I have many MVs, but this is an example of how I tested the MV and the costs associated. I've tested about 10 MVs and they all show the same pattern. Please note, I modified the code to remove all of the object names
With all indexes active:
exec dbms_mview.refresh('MY_MV_TEST','C');
Real time exec as reported from SQL Developer: ~153s
Getting performance:
SELECT elapsed_time, log_purge_time
FROM dba_mvref_stats
....
elapsed_time = 151
log_purge_time = 1
ALTER INDEX IX_MY_MV_TEST_1 UNUSABLE;
....
ALTER INDEX IX_MY_MV_TEST_13 UNUSABLE;
Re-run complete refresh:
exec dbms_mview.refresh('MY_MV_TEST','C');
Get stats from dba_mvref_stats:
elapsed_time = 27
log_purge_time = 1
It was a bit astonished, so I tried 1 by 1, with only 1 index active at a time. For each index it was reported an elapsed_time of 33 and log_purge_time of 2 (I thought it was a bit weird they all reported the same time too). There are a few other MVs that go from 300s to 40s as well. I've only tested on a small subset of our data warehouse so far, and I'm going to assume some of our larger MVs will show the same results. The rebuild of the indexes only takes 11s as reported by SQL developer.
MV DDL:
It would take some time to rename all objects, but if necessary I will if needed. For right now, this is the general over view of this particular MV defintion. In the SELECT clause there are only columns, a couple case statements, and a couple substr(), and cast().
CREATE MATERIALIZED VIEW MY_MV_TEST
BUILD DEFERRED
USING INDEX REFRESH FORCE ON DEMAND
USING DEFAULT LOCAL ROLLBACK SEGMENT
USING ENFORCED CONSTRAINTS AS
SELECT column1, column2, CASE..., SUBSTR(..), CAST()...
FROM mv1, mv2, mv3
WHERE mv1.column1 = mv2.column1
AND mv1.column1 = mv3.column1
AND ... (other simple conditions using the equality operator)
Also Note that all MVs that I've tested are REFRESH FAST capable. The DBMS_MVIEW.EXPLAIN_MVIEW shows that they are REFRESH FAST capable. I'm using the COMPLETE REFRESH just for testing.
Please check if it helps to run the refresh parallel:
ALTER SESSION ENABLE PARALLEL DML;
Furthermore, switch the refresh to non-atomic:
EXEC dbms_mview.refresh(list=>'MY_MV_TEST', method=>'C', atomic_refresh=>false);
Then Oracle will disable the indexes, refresh the data and rebuild the indexes automatically, which is in most cases faster.
I'm looking for a best performance solution to detect if data in an Oracle table has changed. This will be used to kick start a calculation that uses lots of data from the same tables. It would be too expensive to poll the data to track changes. The changes happen rarely.
I have analyzed the following solutions to the problem.
ORA_ROWSCN : Too slow, will do a full table scan.
Oracle Audit : Not possible to set up in my environment.
DBMS_ALERT : Writers are not able to signal.
Then I came up with the following simple idea. Add a trigger to the tables that increments a sequence on insert, update or delete. I know this will be materialized if rolled back but I can afford some false positives. My calculations service then polls the sequence current value to detect possible changes (= very cheap query)
CREATE OR REPLACE TRIGGER trg_change_tracker
before insert or update or delete on mytable
declare
dummy number;
begin
select seq_event_seqno.nextval into dummy from dual;
end;
How does this sound? Any pitfalls?
EDIT: Yes there is a major pitfall: When the writer holds back the commit, and the reader sees the new sequence value and query for changes before the writer has committed.
I can suggest you some addition to your trigger soulution:
When your polling see changes of the sequence, you can check opened transactions on interested table, if it exists, then skip current recalc.
UPD:
Also, you can save min SCN of opened transactions, and in case of frequently table changes you will not freeze
UPD2:
It is some heuristic improvement, not full problem solution. If you will skip recal every time when you see opened transaction, then you can freeze much time in case of frequently (and may be long) DML on table.
I mean, when seq is changed and the polling see opened transactions on your table, you can store min(start_scn) from v$transaction, and when the polling see opened transactions at next time you can compare current min(start_scn) with sotred min(start_scn), if current is greater then it is some chance that it is time to recalc.
You can use ( for particular XTABLE ):
SELECT * FROM dba_tab_modifications WHERE TABLE_NAME = 'XTABLE';
it will allow you ( among other data ) to see:
INSERTS UPDATES DELETES
43,708 1,845 0
.. there are issues ( bugs in some Oracle versions ), but if you will test thoroughly in your environment - you may find this useful.
.. as this question is pretty old - it would be nice to know how you implemented the change detection mechanism.
I have declared MV logs on 4 source tables (database A ) and then I have created 4 Materialized Views on this tables (Database B using dblinks ) with FAST Refresh which works fine. But I need to use this Materialized view which would be refreshed every 5 minutes and I want to extract recent changes i.e Delta only and then apply to 4-5 different target tables. This are big tables and target tables needs to refreshed instantly how can I get just delta ?
May seem obvious to some but I am not able to design reliable process/mechanism. Any help/suggestions ?
Add column to your MV with default sysdate, for example "date_delta". And then use this case: WHERE date_delta >= sysdate - 5/1440 (5/1440 - it's 5 minutes)
Hi I have created 5 MViews for tables in Remote DB 2 days back in production environment. Out of that one Mview, at the time of creation it worked fine. But 2 days after(today) the MView's last refresh date is still showing the day before yesterday's date.
I checked on the DBA_JOBS:
LAST_DATE,NEXT_DATE,BROKEN,INTERVAL,FAILURES,WHAT
NULL,"23-JUL-13 01:00:00", N,"trunc(SYSDATE+1)+1/24",12,"dbms_refresh.refresh('"OCCSS_ENTMT_HK"."MV_SCI_STD_CODE_VALUE"');"
MView create statement:
CREATE MATERIALIZED VIEW MV_SCI_STD_CODE_VALUE BUILD IMMEDIATE USING INDEX REFRESH FORCE ON DEMAND START WITH SYSDATE+0 NEXT TRUNC(SYSDATE+1)+1/24 WITH PRIMARY KEY USING DEFAULT LOCAL ROLLBACK SEGMENT USING ENFORCED CONSTRAINTS DISABLE QUERY REWRITE AS SELECT STV_STD_CODE_NUM STANDARD_CODE,
STV_STD_CODE_VALUE STANDARD_CODE_VALUE,
STV_STD_CODE_VALUE_DESC DESCRIPTION,
stv_prnt_std_code_value parent_code FROM SCIADMIN.P03_STD_CODE_VALUE#SCI_LINK02.HK.BT.COM P03 WHERE STV_STD_CODE_NUM IN ('19','31','54','5','6','300') AND p03.update_status_ind <> 'D';
COMMENT ON MATERIALIZED VIEW MV_SCI_STD_CODE_VALUE IS 'snapshot table for snapshot MV_SCI_STD_CODE_VALUE';
The Mview is scheduled to run once in one day. But the failures shown is 12.
I couldn't find any jobs for the above in DBMS_JOBS_RUNNING.
I need to investigate this issue. What would be the possible issues which could have caused this failure?
I have a Materialized view in Oracle that contains a LEFT JOIN which takes a very long time to update. When I update the underlying table it takes 63914.765 s to run (yes that is almost 17 hours).
I am using a LEFT JOIN on the same table, because I want to pivot the data from rows to columns. The pivot command is not available in this Oracle version, and using a GROUP BY + CASE is not allowed on a FAST REFRESH Materialized View.
The Materialized View Log looks like this:
CREATE MATERIALIZED VIEW LOG ON Programmes_Titles
WITH PRIMARY KEY, rowid
INCLUDING NEW Values;
The Materialized View itself looks like this (it contains 700000 rows, the Programmes_Titles table contains 900000 rows):
CREATE MATERIALIZED VIEW Mv_Web_Programmes
REFRESH FAST ON COMMIT
AS
SELECT
t1.ProgrammeId,
t1.Title as MainTitle,
t2.Title as SecondaryTitle,
--Primary key
t1.Title_Id as t1_titleId,
t2.Title_Id as t2_titleId,
t1.rowid as t1_rowid,
t2.rowid as t2_rowid
FROM
Programmes_Titles t1,
Programmes_Titles t2
WHERE
t1.Titles_Group_Type = 'mainTitle'
AND t1.Programme_Id = t2.Programme_Id(+) AND t2.Titles_Group_Type(+) = 'secondaryTitle'
The UPDATE statement I use is this:
UPDATE Programmes_Titles
SET Title = 'New title'
WHERE rowid = 'AAAL4cAAEAAAftTABB'
This UPDATE statement takes 17 hours.
When using an INNER JOIN (remove the (+)'s) it takes milliseconds.
I also tried adding INDEXES on the Mv_Web_Programmes Materialized View, but that did not seem to help either. (It still runs for more than a minute, which is way to slow, I am not waiting 17 hours after every change, so it might improved the UPDATE)
So my question is: Why does is take such a long time to UPDATE the underlying table? How can I improve this?
I've managed to reproduce your problem on a 10.2.0.3 instance. The self- and outer-join seems to be the major problem (although with indexes on every column of the MV it finally did update in under a minute).
At first I thought you could use an aggregate MV:
SQL> CREATE MATERIALIZED VIEW LOG ON Programmes_Titles
2 WITH PRIMARY KEY, ROWID (programmeId, Titles_Group_Type, title)
3 INCLUDING NEW Values;
Materialized view log created
SQL> CREATE MATERIALIZED VIEW Mv_Web_Programmes
2 REFRESH FAST ON COMMIT
3 AS
4 SELECT ProgrammeId,
5 MAX(decode(t1.Titles_Group_Type, 'mainTitle', t1.Title)) MainTl,
6 MAX(decode(t1.Titles_Group_Type, 'secondaryTitle', t1.Title)) SecTl
7 FROM Programmes_Titles t1
8 GROUP BY ProgrammeId;
Materialized view created
Unfortunately, as you have noticed, as of 10g a MV that contains MIN or MAX can only be fast-refreshed on commit after insert (so called insert-only MV). The above solution would not work for update/delete (the MV would have to be refreshed manually).
You could trace your session and open the trace file to see what SQL query gets executed so that you can find if you can optimize it via indexes.
We too faced this issue recently on Oracle 11.2.0.3
In our case, it was unavoidable to to remove an 'OUTER JOIN' due to functional impact.
On investigation, it was found that Oracle was adding a nasty HASH_SH (Hash Semi Join) hint with MV refresh DML.
Nothing worked including things mentioned in following blog-
http://www.adellera.it/blog/2010/03/11/fast-refresh-of-join-only-mvs-_mv_refresh_use_stats-and-locking-log-stats/#comment-2975
In the end, a hidden hint worked...(though in general, it should be avoided by making change in application if possible)
Oracle Doc ID 1949537.1 suggests that setting the hidden _mv_refresh_use_hash_sj parameter to FALSE should prevent it using that hint.
alter session set "_mv_refresh_use_hash_sj"=FALSE;
That stopped CBO using the HASH_SJ hint.
Posting it here in the interests of others.