Imagine I have an oracle database with data that is constantly changing, a strategy to capture changes on this db is to work with materialized view and materialized view log.
Let's consider table T as an example:
describe T
Name Null? Type
-------------------------------------------- -------- ------------------------------
KEY NOT NULL NUMBER
VAL VARCHAR2(5)
Let's create a mv for table t with fast refresh:
create materialized view mv
REFRESH FAST
as select * from t
;
View the content of the mv:
select key, val, rowid
from mv ;
KEY VAL ROWID
---------- ----- ------------------
1 a AAAWm+AAEAAAAaMAAA
2 b AAAWm+AAEAAAAaMAAB
3 c AAAWm+AAEAAAAaMAAC
4 AAAWm+AAEAAAAaMAAD
Now I create a mv log to track changes on the base table:
create materialized view log on t ;
describe MLOG$_T
Name Null? Type
-------------------------------------------- -------- ------------------------------
KEY NUMBER
SNAPTIME$$ DATE
DMLTYPE$$ VARCHAR2(1)
OLD_NEW$$ VARCHAR2(1)
CHANGE_VECTOR$$ RAW(255)
At this point the the mv log is empty because no change has been applied:
select * from MLOG$_T ;
no rows selected
Let's apply changes:
UPDATE t set val = upper( val ) where KEY = 1 ;
INSERT into t ( KEY, val ) values ( 5, 'e' );
select key, dmltype$$ from MLOG$_T ;
KEY DMLTYPE$$
---------- ----------
1 U
5 I
An update and insert has been recorded as we can see above. If we refresh the mv the changes will be applied to the mv, and the log will be deleted.
Now let's imagine I have another table t2 that is the same table as mv and t before changes have been applied to t. How can I update t2 only from reading the logs of the mv log ? I ask this because in reality t2 will be a table in another database (staging server) which I will updating and applying some ETL as the changes in the mv log happen. Therefore applying the changes in t2 the materialized view log is crucial. I'm also open to new strategies because I believe this causes the oracle databse some overhead.
Related
Is it possible to create a materialized view where it is only incremental?
I would like the old data already inserted not to be updated, only new insertions should be included in the view.
If possible, how could I do it?
Is there any documentation or any place I can use as a guide?
If you want to see every row at the time of insert in an MV the short answer is:
You can't.
A materialized view stores the result of a query as it exists now, not some time in the past. So after updating a row you can either see its current values or exclude it from the MV.
If you want to preserve/view the state of data at insert you have a few options:
Make the table insert-only (possibly storing change history within the table)
Capture the data when its added (e.g. via triggers) into another table
Use Flashback Data Archive to store change history and view it with Flashback Query
Which of these is most appropriate depends on why you need view the data at insert.
Technically it is possible to view data at the time of insert - up to a point.
With Flashback Version Query you can see changes to the table over time. So you can do something like this:
create table t (
c1 int, c2 int,
insert_date timestamp,
update_date timestamp
);
exec dbms_session.sleep(10);
insert into t values ( 1, 1, systimestamp, systimestamp );
insert into t values ( 2, 2, systimestamp, systimestamp );
commit;
create materialized view mv
as
select t.*
from t
versions between scn minvalue
and maxvalue
where versions_operation = 'I';
exec dbms_session.sleep(10);
update t
set c2 = 9999,
update_date = systimestamp
where c1 = 2;
insert into t values ( 3, 3, systimestamp, systimestamp );
commit;
exec dbms_mview.refresh ( 'mv' );
select * from t;
C1 C2 INSERT_DATE UPDATE_DATE
1 1 26-JUL-2021 13.49.51.666954000 26-JUL-2021 13.49.51.666954000
2 9999 26-JUL-2021 13.49.51.712259000 26-JUL-2021 13.50.08.421872000
3 3 26-JUL-2021 13.50.08.462300000 26-JUL-2021 13.50.08.462300000
select *
from mv;
C1 C2 INSERT_DATE UPDATE_DATE
3 3 26-JUL-2021 13.50.08.462300000 26-JUL-2021 13.50.08.462300000
2 2 26-JUL-2021 13.49.51.712259000 26-JUL-2021 13.49.51.712259000
1 1 26-JUL-2021 13.49.51.666954000 26-JUL-2021 13.49.51.666954000
This uses undo to reconstruct history. Eventually older changes will drop off and you're back to only seeing the current state. By default you only get 15 minutes worth of changes!
If you want to store history for long period of time, Flashback Data Archive is the way to go.
all!
Version: Oracle 12.2
Mat.view fast refresh fails when in WHERE clause
there is column , starting with double quotes and underscore, like that:
"_ID".
Do you know workaround, other than rename column in base tabl
Don't rename column in base table; rename column in materialized view:
SQL> desc test
Name Null? Type
----------------------------------------- -------- ----------------------------
_ID VARCHAR2(20)
SOME_COL VARCHAR2(3)
SQL> create materialized view mv_test as
2 select "_ID" as id, --> this
3 some_col
4 from test;
Materialized view created.
SQL> desc mv_test
Name Null? Type
----------------------------------------- -------- ----------------------------
ID VARCHAR2(20)
SOME_COL VARCHAR2(3)
SQL>
I tried as you said and i have created a small demo as per my understanding from your question.
please find below observation,
--drop table test_mv;
create table test_mv("_ID" number primary key,NAME varchar2(100));
--data
insert into test_mv values (1,'first row');
insert into test_mv values (2,'second row');
commit;
--trying to select without ""
select *
from test_mv
where _ID = 1;
--result
--Error report -
--SQL Error: ORA-00911: invalid character
--00911. 00000 - "invalid character"
--trying to select with ""
select *
from test_mv
where "_ID" = 1;
--result
_ID NAME
--- -----
1 first row
--mat view log
create materialized view log on test_mv;
--output: Materialized view log TEST_MV created.
--drop materialized view mv_test_mv;
--create materialized view giving "_id" in where clause as you described and `on commit`
create materialized view mv_test_mv
refresh fast on commit
as
select *
from test_mv
where "_ID" = 1;
--output: Materialized view MV_TEST_MV created.
--selecting from mat view
select * from mv_test_mv;
--output:
_id NAME
--- -----
1 first row
--update the base table record
update test_mv set name = 'modified first row' where "_ID" = 1;
commit;
--output
--1 row updated.
--Commit complete.
--selecting again from mat view to check if the changes propagated to the mat view or not
select * from mv_test_mv;
output:
_ID NAME
--- ------------------
1 modified first row
It all worked fine even though I have a where clause with "_ID".
Is this what you are saying or something else which I didn't understand ?
This may be impossible, but I'd hoped to see if there is an approachable way to run an online replacement of a table with a view.
For online table restructuring like partitioning, etc. DBMS_REDEFINITION works great. But I'd like to replace a table with a (materialized) view, so DBMS_REDEFINITION appears to be unsuitable.
I have no constraints, dependencies or mutating dml, etc. to worry about during the rename; I would only like to keep the target SELECTable when replacing the table with a view. A trumped-up example is below.
CREATE TABLE SCI_FI_MOVIE (
SCI_FI_MOVIE_ID NUMBER(10, 0) NOT NULL PRIMARY KEY,
NAME VARCHAR2(100) UNIQUE NOT NULL,
DIRECTOR VARCHAR2(100) NOT NULL,
REVIEW_SCORE NUMBER(1, 0) CHECK ( REVIEW_SCORE IN (1, 2, 3, 4, 5))
);
CREATE TABLE NO_SCORES_SCI_FI_MOVIE (
SCI_FI_MOVIE_ID NUMBER(10, 0) NOT NULL PRIMARY KEY,
NAME VARCHAR2(100) UNIQUE NOT NULL,
DIRECTOR VARCHAR2(100) NOT NULL
);
CREATE MATERIALIZED VIEW KUBRICK_SABOTAGE
(SCI_FI_MOVIE_ID, NAME, DIRECTOR, REVIEW_SCORE)
REFRESH COMPLETE ON COMMIT
AS
SELECT
SCI_FI_MOVIE_ID,
NAME,
DIRECTOR,
CASE WHEN DIRECTOR = 'KUBRICK'
THEN 5
ELSE 2 END AS REVIEW_SCORE
FROM NO_SCORES_SCI_FI_MOVIE;
INSERT INTO SCI_FI_MOVIE VALUES (1, 'Apollo 13', 'HOWARD', 5);
INSERT INTO SCI_FI_MOVIE VALUES (2, '2001: A Space Odyssey', 'KUBRICK', 4);
INSERT INTO NO_SCORES_SCI_FI_MOVIE VALUES (1, 'Apollo 13', 'HOWARD');
INSERT INTO NO_SCORES_SCI_FI_MOVIE VALUES (2, '2001: A Space Odyssey', 'KUBRICK');
COMMIT;
-- THEN WHAT STEPS TO REPLACE TABLE WITH VIEW?
In this example I'd like to end up with the MV named SCI_FI_MOVIE and the TABLE SCI_FI_MOVIE renamed to SCI_FI_MOVIE_TEMP or whatever pending its removal. There isn't any requirement for the MV to exist prior to replacing the original table, if the replacement can be done atomically
I'd like to avoid any interruption or compromise to object name resolution (CREATE PUBLIC SYNONYM then renaming original won't work here)
Is there a clean no-downtime way to do this?
I am free to disable logging, read-only etc. anything as needed; the only goal is to prevent "ORA-00942: table or view does not exist" during the dictionary switch. I'm on 11gR2 but would welcome in 12c solutions as well.
Thanks so much for your thoughts
You don't need to do on-line redefinition or any clever; what you're trying to do is built in via the ON PREBUILT TABLE clause:
The ON PREBUILT TABLE clause lets you register an existing table as a preinitialized materialized view. This clause is particularly useful for registering large materialized views in a data warehousing environment. The table must have the same name and be in the same schema as the resulting materialized view.
If the materialized view is dropped, then the preexisting table reverts to its identity as a table.
So you can just do:
CREATE MATERIALIZED VIEW SCI_FI_MOVIE
(SCI_FI_MOVIE_ID, NAME, DIRECTOR, REVIEW_SCORE)
ON PREBUILT TABLE
REFRESH COMPLETE ON COMMIT
AS
SELECT
SCI_FI_MOVIE_ID,
NAME,
DIRECTOR,
CAST(CASE WHEN DIRECTOR = 'KUBRICK'
THEN 5
ELSE 2 END AS NUMBER(1,0)) AS REVIEW_SCORE
FROM NO_SCORES_SCI_FI_MOVIE;
Materialized view SCI_FI_MOVIE created.
The CAST(... AS NUMBER(1,0)) is needed to make the generated data type match the underlying table.
The table is locked while the MV is built (which is pretty much instant anyway as there is no data collection or creation) so queries against it while that is happening will just block briefly.
The view will have the original table values:
select * from SCI_FI_MOVIE;
SCI_FI_MOVIE_ID NAME DIRECTOR REVIEW_SCORE
--------------- ------------------------------ ---------- ------------
1 Apollo 13 HOWARD 5
2 2001: A Space Odyssey KUBRICK 4
... until it's refreshed, which (in this example) will be on next commit:
INSERT INTO NO_SCORES_SCI_FI_MOVIE VALUES (3, 'Star Wars', 'LUCAS');
1 row inserted.
COMMIT;
select * from SCI_FI_MOVIE;
SCI_FI_MOVIE_ID NAME DIRECTOR REVIEW_SCORE
--------------- ------------------------------ ---------- ------------
1 Apollo 13 HOWARD 2
2 2001: A Space Odyssey KUBRICK 5
3 Star Wars LUCAS 2
For a normal (non-materialized view) you could do a bit of shuffling to achieve the same thing, as long as you can create a public synonym:
CREATE TABLE SCI_FI_MOVIE_TMP_TAB AS SELECT * FROM SCI_FI_MOVIE;
CREATE VIEW SCI_FI_MOVIE_TMP_VIEW AS SELECT * FROM SCI_FI_MOVIE_TMP_TAB;
CREATE PUBLIC SYNONYM SCI_FI_MOVIE FOR SCI_FI_MOVIE_TMP_VIEW;
ALTER TABLE SCI_FI_MOVIE RENAME TO SCI_FI_MOVIE_OLD;
CREATE VIEW SCI_FI_MOVIE AS
SELECT
SCI_FI_MOVIE_ID,
NAME,
DIRECTOR,
CAST(CASE WHEN DIRECTOR = 'KUBRICK'
THEN 5
ELSE 2 END AS NUMBER(1,0)) AS REVIEW_SCORE
FROM NO_SCORES_SCI_FI_MOVIE;
DROP PUBLIC SYNONYM SCI_FI_MOVIE;
DROP VIEW SCI_FI_MOVIE_TMP_VIEW;
DROP TABLE SCI_FI_MOVIE_TMP_TAB;
DROP TABLE SCI_FI_MOVIE_OLD;
This relies on how Oracle resolves schema object references. When you rename the original table it can no longer find an object with that name in the current schema (namespace), and looks for a public synonym, and happily uses that. When the view is created that takes precedence over the public synonym again.
I need to extract the unique values of a column which is part of the primary key from a table into a materialized view. I can create the materialized view if using "refresh complete" but with no luck when trying to use "refresh fast on commit". Can anyone point out whether I missed anything or Oracle does not support such action.
The example output is listed below. Thanks.
SQL> create table TEST( col1 number, col2 number, col3 varchar(32), CONSTRAINT test_pk Primary Key (col1, col2));
Table created.
SQL> create materialized view test_mv build immediate refresh fast on commit as select distinct col2 from test;
create materialized view test_mv build immediate refresh fast on commit as select distinct col2 from test
*
ERROR at line 1:
ORA-12054: cannot set the ON COMMIT refresh attribute for the materialized view
SQL> create materialized view test_mv build immediate refresh complete as select distinct col2 from test;
Materialized view created.
SQL> drop materialized view test_mv;
Materialized view dropped.
SQL> create materialized view log on test;
Materialized view log created.
SQL> create materialized view test_mv build immediate refresh fast on commit as select distinct col2 from test;
create materialized view test_mv build immediate refresh fast on commit as select distinct col2 from test
*
ERROR at line 1:
ORA-12054: cannot set the ON COMMIT refresh attribute for the materialized view
Main issue of your view is the DISTINCT clause. On commit fast refresh is super sensitive to underlying query. There exist many rules that must be fulfilled for a materialized view to support fast refresh. DISTINCT prevents it.
You can check the capabilities of a materialized view using DBMS_MVIEW.EXPLAIN_MVIEW procedure:
DECLARE
result SYS.EXPLAINMVARRAYTYPE := SYS.EXPLAINMVARRAYTYPE();
BEGIN
DBMS_MVIEW.EXPLAIN_MVIEW('TEST_MV', result);
FOR i IN result.FIRST..result.LAST LOOP
DBMS_OUTPUT.PUT_LINE(result(i).CAPABILITY_NAME || ': ' || CASE WHEN result(i).POSSIBLE = 'T' THEN 'Yes' ELSE 'No' || CASE WHEN result(i).RELATED_TEXT IS NOT NULL THEN ' because of ' || result(i).RELATED_TEXT END || '; ' || result(i).MSGTXT END);
END LOOP;
END;
You find more information in documentation http://docs.oracle.com/cd/B28359_01/server.111/b28313/basicmv.htm#i1007007
Fast refresh views are picky. This solution requires a materialized view log with specific properties, and a materialized view with a few extra features and a different syntax.
DISTINCT alone does not appear to be supported. But there are aggregate materialized views that support GROUP BY. If that materialized view is created with ENABLE QUERY REWRITE, Oracle can use it in a DISTINCT query. There is also an extra COUNT(*) because "COUNT(*) must always be present to guarantee all types of fast refresh."
Create table, materialized view log, and materialized view.
create table test(col1 number, col2 number, col3 varchar(32)
,constraint test_pk primary key (col1, col2));
create materialized view log on test with rowid (col2) including new values;
create materialized view test_mv
build immediate
refresh fast on commit
enable query rewrite as
select col2, count(*) total from test group by col2;
Queries can use the materialized view.
These explain plans show that the materialized view works for both a GROUP BY and a DISTINCT query.
explain plan for select col2 from test group by col2;
select * from table(dbms_xplan.display);
explain plan for select distinct col2 from test;
select * from table(dbms_xplan.display);
Plan hash value: 1627509066
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 13 | 2 (0)| 00:00:01 |
| 1 | MAT_VIEW REWRITE ACCESS FULL| TEST_MV | 1 | 13 | 2 (0)| 00:00:01 |
----------------------------------------------------------------------------------------
I have a data mart mastered from our OLTP Oracle database using basic Materialized Views with on demand fast refresh capability. Refresh is working fine. What I am interested in adding are some statistics about the refresh of each Materialized View, such as the number of inserts, updates, and deletes that were applied to the master table since the last refresh like that data I can find in user_tab_modifications. Is this possible for Materialized Views?
Prior to doing the refresh, you could query the materialized view log to see what sort of change vectors it stores. Those will be the change vectors that need to be applied to the materialized view during the refresh process (assuming that there is just one materialized view that depends on this materialized view log).
For example, if I create my table, my materialized view log, and my materialized view.
SQL> create table foo( col1 number primary key);
Table created.
SQL> create materialized view log on foo;
Materialized view log created.
SQL> ed
Wrote file afiedt.buf
1 create materialized view mv_foo
2 refresh fast on demand
3 as
4 select *
5* from foo
SQL> /
Materialized view created.
SQL> insert into foo values( 1 );
1 row created.
SQL> insert into foo values( 2 );
1 row created.
SQL> commit;
Commit complete.
Now, I refresh the materialized view and verify that the table and the materialized view are in sync
SQL> exec dbms_mview.refresh( 'MV_FOO' );
PL/SQL procedure successfully completed.
SQL> select * from user_tab_modifications where table_name = 'MV_FOO';
no rows selected
SQL> select * from foo;
COL1
----------
1
2
SQL> select * from mv_foo;
COL1
----------
1
2
Since the two objects are in sync, the materialized view log is empty (the materialized view log will be named MLOG$_<<table name>>
SQL> select * from mlog$_foo;
no rows selected
Now, if I insert a new row into the table, I'll see a row in the materialized view log with a DMLTYPE$$ of I indicating an INSERT
SQL> insert into foo values( 3 );
1 row created.
SQL> select * from mlog$_foo;
COL1 SNAPTIME$ D O
---------- --------- - -
CHANGE_VECTOR$$
--------------------------------------------------------------------------------
XID$$
----------
3 01-JAN-00 I N
FE
2.2519E+15
So you could do something like this to get the number of pending inserts, updates, and deletes.
SELECT SUM( CASE WHEN dmltype$$ = 'I' THEN 1 ELSE 0 END ) num_pending_inserts,
SUM( CASE WHEN dmltype$$ = 'U' THEN 1 ELSE 0 END ) num_pending_updates,
SUM( CASE WHEN dmltype$$ = 'D' THEN 1 ELSE 0 END ) num_pending_deletes
FROM mlog$_foo
Once you refresh the materialized view log, however, this information is gone.
On the other hand, USER_TAB_MODIFICATIONS should track the approximate number of changes that have been made to the materialized view since the last time that statistics were gathered on it just as it would track the information for a table. You'll almost certainly need to call DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO to force the data to be made visible if you want to capture the data before and after the refresh of the materialized view.
SELECT inserts, updates, deletes
INTO l_starting_inserts,
l_starting_updates,
l_starting_deletes
FROM user_tab_modifications
WHERE table_name = 'MV_FOO';
dbms_mview.refresh( 'MV_FOO' );
dbms_stats.flush_database_monitoring_info;
SELECT inserts, updates, deletes
INTO l_ending_inserts,
l_ending_updates,
l_ending_deletes
FROM user_tab_modifications
WHERE table_name = 'MV_FOO';
l_incremental_inserts := l_ending_inserts - l_starting_inserts;
l_incremental_updates := l_ending_updates - l_starting_updates;
l_incremental_deletes := l_ending_deletes - l_starting_deletes;