Update materialized view when urderlying tables change - oracle

I have a materialized view defined this way:
CREATE MATERIALIZED VIEW M_FOO
REFRESH COMPLETE ON COMMIT
AS
SELECT FOO_ID, BAR
FROM FOO
WHERE BAR IS NOT NULL
GROUP BY FOO_ID, BAR
/
COMMENT ON MATERIALIZED VIEW M_FOO IS 'Foo-Bar pairs';
I wrote as a sort of cache: the source table is huge but the number of different pairs is fairly small. I need those pairs to get them JOINed with other tables. So far so good: it absolutely speeds queries.
But I want to make sure that the view does not contain obsolete data. The underlying table is modified four or five times per month but I don't necessarily know when. I understand that a materialized view can be defined so it updates when the source tables change. However, the docs get pretty complicate.
What's the exact syntax I need to
use?
Do I need to create a materialized
view log?
What's the difference between fast
and complete refresh?

To take your questions in reverse order
A FAST refresh is also known as an incremental refresh. That should give you a clue as to the difference. A COMPLETE refresh rebuilds the entire MVIEW from scratch, whereas a FAST refresh applies just the changes from DML executed against the feeder table(s).
In order to do execute FAST refreshes you need the appropriate MVIEW LOG. This tracks changes to the data of the underlying tables, which allows Oracle to efficiently apply a delta to the materialized view, rather than querying the whole table.
As for the syntax, here are the basics:
SQL> create materialized view log on emp
2 with rowid, primary key, sequence (deptno, job)
3 including new values
4 /
Materialized view log created.
SQL> create materialized view emp_mv
2 refresh fast on commit
3 as
4 select deptno, job from emp
5 group by deptno, job
6 /
Materialized view created.
SQL>
The ON COMMIT clause means that the MVIEW is refreshed transactionally (as opposed to ON DEMAND which is regular refresh in bulk). The REFRESH clauses specifies whether to apply incremental or complete refreshes. There are some categories of query which force the use of COMPLETE refresh, although these seem to diminish with each new version of Oracle.
A quick test to see that it works ...
SQL> select * from emp_mv
2 order by deptno, job
3 /
DEPTNO JOB
---------- ---------
10 MANAGER
10 PRESIDENT
10 SALES
20 ANALYST
20 CLERK
20 MANAGER
30 CLERK
30 MANAGER
30 SALESMAN
40 CLERK
40 DOGSBODY
11 rows selected.
SQL>
How about a new record?
SQL> insert into emp (empno, ename, deptno, job)
2 values (6666, 'GADGET', 40, 'INSPECTOR')
3 /
1 row created.
SQL> commit
2 /
Commit complete.
SQL> select * from emp_mv
2 order by deptno, job
3 /
DEPTNO JOB
---------- ---------
10 MANAGER
10 PRESIDENT
10 SALES
20 ANALYST
20 CLERK
20 MANAGER
30 CLERK
30 MANAGER
30 SALESMAN
40 CLERK
40 DOGSBODY
40 INSPECTOR
12 rows selected.
SQL>
You can find more details on the syntax in the SQL Reference. It's also worth reading the Materialized View chapter in the Data Warehousing Guide.
Despite the concerns of the commenters below this does work as advertised. Unfortunately the usual places for publishing demos (SQL Fiddle, db<>fiddle) do not allow materialized views. I have published something on Oracle SQL Live (free Oracle account required): I am awaiting Oracle approval for it and will update this question when it arrives.

A fast refresh will only insert/update/delete changed data into the materialized view. A complete refresh will empty the materialized view and then copy over all rows.
The "on commit" means the materialized view will be refreshed whenever a change is committed in the master table. So your current syntax is going to be extremely inefficient. Every time somebody changes any row in foo, m_foo will be truncated and then every row in foo table will be inserted.
You can do better with fast refreshes, where only the modified rows in foo will be sent to m_foo. That gives you consistency without lots of overhead.
create materialized view log on foo with primary key; -- assuming you have a primary key, you should
create materialized view m_foo refresh fast on commit as \;
There are some additional subtleties with grants and synonyms if you're using db links, or the schema that owns foo isn't the one that owns m_foo.

Related

Oracle More than one materialized view on a materialized log

I would like to have more than one materialized view with refresh fast on commit.
For "refresh fast on commit" you need a materialized view log. Obviously a refresh fast on commit needs the log. The question is can I have more than one materialized view accessing the log.
Obviously I still need the log to satisfy the normal prerequisites (across all views):
http://docs.oracle.com/cd/B19306_01/server.102/b14223/basicmv.htm
Cheers for any help in advance.
Absolutely.
create table data_table as (
select *
from dba_users
);
alter table data_table
add constraint data_table_test_pk PRIMARY KEY (user_id);
select * from data_table;
So now we have a table that looks like dba_users with a PRIMARY KEY constraint.
create materialized view log on data_table;
create materialized view mat_view_one
refresh fast on commit
as
select username, user_id
from data_table
;
create materialized view mat_view_two
refresh fast on commit
as
select user_id, username, account_status
from data_table;
There's our log and the 2 views create successfully.
SYS record in the test table:
select * from mat_view_one
where user_id=0;
USERNAME USER_ID
------------------------------ ----------
SYS 0
select * from mat_view_two
where user_id=0;
USER_ID USERNAME ACCOUNT_STATUS
---------- ------------------------------ --------------------------------
0 SYS OPEN
Now lets update the SYS's name and commit and see what our views show:
update data_table
set username='WALTERWHITE'
WHERE USER_ID=0
;
COMMIT;
USERNAME USER_ID
------------------------------ ----------
WALTERWHITE 0
USER_ID USERNAME ACCOUNT_STATUS
---------- ------------------------------ --------------------------------
0 WALTERWHITE OPEN
So yes, absolutely. 1 materialized view log can serve as many materialized views as you need so long as the proper constraints are held.
Yes. A materialized view log can support as many materialized views as you'd like.
Supporting multiple materialized views may cause the log to be larger than it would be if it was supporting a single materialized view though that probably isn't terribly significant here since your logs probably won't store the data long. If you're replicating data to remote databases (which by definition you can't do with a fast refreshable materialized view), it can be a bit tricky if one of the remote materialized views stops refreshing or goes away without the source being aware since it can cause data to queue up indefinitely in the log. But, again, that doesn't sound like the situation you're facing.

Fast Refresh on commit of materialized view

I just created tables DEPT and EMP like follow :
create table DEPT
( dept_no number , dept_name varchar(32) , dept_desc varchar(32),
CONSTRAINT dept_pk Primary Key (dept_no) );
create table EMP
( emp_no number, dept_no number, CONSTRAINT emp_pk Primary Key (emp_no,dept_no));
insert into dept values (10,'it','desc1');
insert into dept values (20,'hr','desc2');
insert into emp values (1,10);
insert into emp values (2,20);
I created materialized view logs on these tables with rowid and materialized views as follows:
create materialized view log on emp with rowid;
create materialized view log on dept with rowid;
create materialized view empdept_mv refresh fast on commit as
select a.rowid dept_rowid, b.rowid emp_rowid, a.dept_no,b.emp_no
from dept a, emp b
where a.dept_no=b.dept_no ;
select * from emp;
EMP_NO DEPT_NO
---------- ----------
1 10
2 20
3 30
select * from dept;
DEPT_NO DEPT_NAME DEPT_DESC
---------- -------------------------------- --------------------------------
10 it desc1
20 hr desc2
30 it desc3
select * from empdept_mv;
DEPT_ROWID EMP_ROWID DEPT_NO EMP_NO
------------------ ------------------ ---------- ----------
AAAli5AABAAAPZ6AAA AAAli7AABAAAQs6AAA 10 1
AAAli5AABAAAPZ6AAB AAAli7AABAAAQs6AAB 20 2
I inserted a new record and did COMMIT; ..but still when i check the materialized view, the new record is not shown in the materialized view.
insert into dept values (30,'it','desc3');
commit;
insert into emp values (3,30);
commit;
select * from empdept_mv;
DEPT_ROWID EMP_ROWID DEPT_NO EMP_NO
------------------ ------------------ ---------- ----------
AAAli5AABAAAPZ6AAA AAAli7AABAAAQs6AAA 10 1
AAAli5AABAAAPZ6AAB AAAli7AABAAAQs6AAB 20 2
Now, when I run the procedure for Fast and complete refresh as per, The Fast refresh does not update the Mview but the complete refresh does. ( Note: But the Mview is still REFRESH ON COMMIT)
execute DBMS_MVIEW.REFRESH('empdept_mv', 'F', '', TRUE, FALSE, 0,0,0,FALSE, FALSE);
PL/SQL procedure successfully completed.
DEPT_ROWID EMP_ROWID DEPT_NO EMP_NO
------------------ ------------------ ---------- ----------
AAAli5AABAAAPZ6AAA AAAli7AABAAAQs6AAA 10 1
AAAli5AABAAAPZ6AAB AAAli7AABAAAQs6AAB 20 2
execute DBMS_MVIEW.REFRESH('test_mview2', 'C', '', TRUE, FALSE, 0,0,0,FALSE, FALSE);
PL/SQL procedure successfully completed.
DEPT_ROWID EMP_ROWID DEPT_NO EMP_NO
------------------ ------------------ ---------- ----------
AAAli5AABAAAPZ6AAA AAAli7AABAAAQs6AAA 10 1
AAAli5AABAAAPZ6AAB AAAli7AABAAAQs6AAB 20 2
AAAli5AABAAAPZ6AAC AAAli7AABAAAQs6AAC 30 3
The DBMS_MVIEW.EXPLAIN_MVIEW output is as shown : (capability_name --Possible-- msgtxt)
PCT --N--
REFRESH_COMPLETE --Y--
REFRESH_FAST --Y--
REWRITE --N--
PCT_TABLE --N-- Oracle error: see RELATED_NUM and RELATED_TEXT for
details
REFRESH_FAST_AFTER_INSERT --Y--
REFRESH_FAST_AFTER_ONETAB_DML --Y--
REFRESH_FAST_AFTER_ANY_DML --Y--
REFRESH_FAST_PCT --N-- PCT is not possible on any of the detail
tables in the mater
REWRITE_FULL_TEXT_MATCH --N-- Oracle error: see RELATED_NUM and
RELATED_TEXT for details
REWRITE_FULL_TEXT_MATCH --N-- query rewrite is disabled on the
materialized view
REWRITE_PARTIAL_TEXT_MATCH --N-- materialized view cannot support
any type of query rewrite
REWRITE_PARTIAL_TEXT_MATCH --N-- query rewrite is disabled on the
materialized view
REWRITE_GENERAL --N-- materialized view cannot support any type of
query rewrite
REWRITE_GENERAL --N-- query rewrite is disabled on the materialized
view
REWRITE_PCT --N-- general rewrite is not possible or PCT is not
possible on an
PCT_TABLE_REWRITE --N-- Oracle error: see RELATED_NUM and
RELATED_TEXT for details
How can I achieve Fast Refresh On Commit ?
The Oracle Version details are as follows:
NLSRTL 10.2.0.4.0 Production
Oracle Database 10g 10.2.0.4.0 64bit Production
PL/SQL 10.2.0.4.0 Production
TNS for Linux: 10.2.0.4.0 Production
I don't know if the problem still persists, but as I took a look at the artice you provided, I noticed something (which might just be the solution here):
ON COMMIT Refresh
A materialized view can be refreshed automatically using the ON COMMIT method. Therefore, whenever a transaction commits which has updated the tables on which a materialized view is defined, those changes are automatically reflected in the materialized view. The advantage of using this approach is you never have to remember to refresh the materialized view. The only disadvantage is the time required to complete the commit will be slightly longer because of the extra processing involved. However, in a data warehouse, this should not be an issue because there is unlikely to be concurrent processes trying to update the same table.
Notice the bold line.
Then we have:
Table 7-1 ON DEMAND Refresh Methods
Refresh Option Parameter Description
COMPLETE C Refreshes by recalculating the defining query of the materialized view.
FAST F Refreshes by incrementally applying changes to the materialized view. For local materialized views, it chooses the refresh method which is estimated by optimizer to be most efficient. The refresh methods considered are log-based FAST and FAST_PCT.
FAST_PCT P Refreshes by recomputing the rows in the materialized view affected by changed partitions in the detail tables.
FORCE ? Attempts a fast refresh. If that is not possible, it does a complete refresh.
For local materialized views, it chooses the refresh method which is estimated by optimizer to be most efficient. The refresh methods considered are log based FAST, FAST_PCT, and COMPLETE.
Notice the bold lines.
I personally prefer the FORCE Option.
Could you please tell, if this occurs again after some time (depending of the parameters of the DB and the machine it runs on, so I can't even hint you how much)?
When Fast Refresh is Possible
Not all materialized views may be fast refreshable. Therefore, use the package DBMS_MVIEW.EXPLAIN_MVIEW to determine what refresh methods are available for a materialized view.
If you are not sure how to make a materialized view fast refreshable, you can use the DBMS_ADVISOR.TUNE_MVIEW procedure, which provides a script containing the statements required to create a fast refreshable materialized view.
Cheers
I see that you created the materialized view logs with ROWID, which is not really required as both tables have a primary key so you could try without the ROWID.
create materialized view log on emp;
create materialized view log on dept;
Additionally, if you create the materialized view log with ROWID you should create the materialized view with rowid.
create materialized view empdept_mv refresh fast on commit WITH ROWID as
select a.rowid dept_rowid, b.rowid emp_rowid, a.dept_no,b.emp_no
from dept a, emp b
where a.dept_no=b.dept_no ;
You could try those changes and see if the materialized views fast refresh on commit.

Oracle materialized view computational cost

Is the computational cost of updating a stored procedure materialized view, in Oracle, based on the query execution or the result set? More specifically, does Oracle store the results of the query in such a way that contributes significantly to the time required to refresh the view?
Of course, queries which take very long to execute as well as incredibly large or small result sets make this impossible to answer ubiquitously.
The question is more about how the view actually stores the result set (in memory, on disk) so I can think about how frequently to rebuild materialized views.
Materialized view is basically a table combined with an algorithm to update it.
01:37:23 HR#sandbox> create materialized view mv_dual as select dummy from dual;
Materialized view created.
Elapsed: 00:00:00.52
01:37:56 HR#sandbox> select object_name, object_type from user_objects where object_name = 'MV_DUAL';
OBJECT_NAME OBJECT_TYPE
--------------- -------------------
MV_DUAL TABLE
MV_DUAL MATERIALIZED VIEW
Elapsed: 00:00:00.01
You can also create materialized views on prebuilt tables.
If we talk about refresh - there are two options: fast refresh and complete refresh.
Complete refresh just re-executes MV query, while fast refresh performs incremental updates.
http://docs.oracle.com/cd/E16338_01/server.112/e10706/repmview.htm#i29858
there are two types of mviews
Complete refresh mview - the entier mview will be rebuild every refresh. similar to delete and insert (notice: if you specify atomic = F or have version < 9 it will be truncate / insert append).
Fast refresh mview - oracle will create a table to store incremental changes. when refreshing, the changes stored in the side table will be applied to the mview.
fast refresh is faster on refresh but slows down dml operations on the base table.
when you consider your refresh strategy you should consider how much changes are applied to the base table and how often you need to refresh the mview.

Why does a ORA-12054 error occur when creating this simple materialized view example?

ALTER TABLE RECORDINGS ADD PRIMARY KEY (ID);
CREATE MATERIALIZED VIEW LOG ON RECORDINGS TABLESPACE USERS NOLOGGING;
DROP MATERIALIZED VIEW REC_SEARCH_TEST;
CREATE MATERIALIZED VIEW REC_SEARCH_TEST
REFRESH COMPLETE ON COMMIT
AS (
SELECT DISTINCT ID, TITLE FROM RECORDINGS
);
ORA-12054: cannot set the ON COMMIT refresh attribute for the materialized view
Cannot understand what is wrong here, I know that if I take out the DISTINCT clause it works, but why can I not use 'DISTINCT' if I specify 'REFRESH COMPLETE ON COMMIT' which is required.
If I use DISTINCT and REFRESH on demand there is no problem, but these are not the requirements.
Seems like with the addition of the DISTINCT, you've made your view's underlying SQL ineligible for fast refresh, and therefore not able to be used with ON COMMIT (even tho you specify refresh complete instead of refresh fast). From Oracle docs:
The two refresh execution modes are ON COMMIT and ON DEMAND. Depending
on the materialized view you create, some of the options may not be
available. Table 8-4 describes the refresh modes.
Table 8-4 Refresh Modes
ON COMMIT
Refresh occurs automatically when a transaction that modified one of
the materialized view's detail tables commits. This can be specified
as long as the materialized view is fast refreshable (in other words,
not complex). The ON COMMIT privilege is necessary to use this mode.
ON DEMAND
Refresh occurs when a user manually executes one of the available
refresh procedures contained in the DBMS_MVIEW package (REFRESH,
REFRESH_ALL_MVIEWS, REFRESH_DEPENDENT).
The same document link has a list of restrictions for fast refresh as well.
"Perhaps the example isn't the best, because I want to expand the view
to a more complicated query that will require a distinct keyword,
right now I am just trying to get it working on a basic level. "
The DISTINCT is the cause of the ORA-12054.
SQL> CREATE MATERIALIZED VIEW REC_SEARCH_TEST
REFRESH COMPLETE ON COMMIT
AS (
SELECT DISTINCT empno, ename FROM emp
)
/
2 3 4 5 6
SELECT DISTINCT empno, ename FROM emp
*
ERROR at line 4:
ORA-12054: cannot set the ON COMMIT refresh attribute for the materialized view
Elapsed: 00:00:01.14
SQL> SQL>
SQL> CREATE MATERIALIZED VIEW REC_SEARCH_TEST
REFRESH COMPLETE ON COMMIT
AS (
SELECT empno, ename FROM emp
)
/
2 3 4 5 6
Materialized view created.
Elapsed: 00:00:02.33
SQL>
Why not start with a something that works? Remove the DISTINCT. Get your MView working. Then complicate it later when it becomes necessary.
Although, as you already know you cannot use a DISTINCT you will have to revise either your query's logic or your refresh strategy.
The important thing to note about the question is that it is not about fast refresh, but complete refresh. Thus, there is no logical reason that the usual restrictions for fast refresh on commit should apply, except the one that all referenced objects must be local.
The "refresh complete on commit is a relatively new feature, so the best answer to the "why" question is probably "Oracle has not yet implemented this fully, please check future versions of Oracle Database". Not very useful, but true...

Data loading in Oracle

I am facing problem in loading data. I have to copy 800,000 rows from one table to another in Oracle database.
I tried for 10,000 rows first but the time it took is not satisfactory. I tried using the "BULK COLLECT" and "INSERT INTO SELECT" clause but for both the cases response time is around 35 minutes. This is not the desired response I'm looking for.
Does anyone have any suggestions?
Anirban,
Using an "INSERT INTO SELECT" is the fastest way to populate your table. You may want to extend it with one or two of these hints:
APPEND: to use direct path loading, circumventing the buffer cache
PARALLEL: to use parallel processing if your system has multiple cpu's and this is a one-time operation or an operation that takes place at a time when it doesn't matter that one "selfish" process consumes more resources.
Just using the append hint on my laptop copies 800,000 very small rows below 5 seconds:
SQL> create table one_table (id,name)
2 as
3 select level, 'name' || to_char(level)
4 from dual
5 connect by level <= 800000
6 /
Tabel is aangemaakt.
SQL> create table another_table as select * from one_table where 1=0
2 /
Tabel is aangemaakt.
SQL> select count(*) from another_table
2 /
COUNT(*)
----------
0
1 rij is geselecteerd.
SQL> set timing on
SQL> insert /*+ append */ into another_table select * from one_table
2 /
800000 rijen zijn aangemaakt.
Verstreken: 00:00:04.76
You mention that this operation takes 35 minutes in your case. Can you post some more details, so we can see what exactly is taking 35 minutes?
Regards,
Rob.
I would agree with Rob. Insert into () select is the fastest way to do this.
What exactly do you need to do? If you're trying to do a table rename by copying to a new table and then deleting the old, you might be better off doing a table rename:
alter table
table
rename to
someothertable;
INSERT INTO SELECT is the fastest way to do it.
If possible/necessary, disable all indexes on the target table first.
If you have no existing data in the target table, you can also try CREATE AS SELECT.
As with the above, I would recommend the Insert INTO ... AS select .... or CREATE TABLE ... AS SELECT ... as the fastest way to copy a large volume of data between two tables.
You want to look up the direct-load insert in your oracle documentation. This adds two items to your statements: parallel and nologging. Repeat the tests but do the following:
CREATE TABLE Table2 AS SELECT * FROM Table1 where 1=2;
ALTER TABLE Table2 NOLOGGING;
ALTER TABLE TABLE2 PARALLEL (10);
ALTER TABLE TABLE1 PARALLEL (10);
ALTER SESSION ENABLE PARALLEL DML;
INSERT INTO TABLE2 SELECT * FROM Table 1;
COMMIT;
ALTER TABLE 2 LOGGING:
This turns off the rollback logging for inserts into the table. If the system crashes, there's not recovery and you can't do a rollback on the transaction. The PARALLEL uses N worker thread to copy the data in blocks. You'll have to experiment with the number of parallel worker threads to get best results on your system.
Is the table you are copying to the same structure as the other table? Does it have data or are you creating a new one? Can you use exp/imp? Exp can be give a query to limit what it exports and then imported into the db. What is the total size of the table you are copying from? If you are copying most of the data from one table to a second, can you instead copy the full table using exp/imp and then remove the unwanted rows which would be less than copying.
try to drop all indexes/constraints on your destination table and then re-create them after data load.
use /*+NOLOGGING*/ hint in case you use NOARCHIVELOG mode, or consider to do the backup right after the operation.

Resources