I'd like to use materialized view on prebuilt table to keep a table in sync before a migration. The data is changing constantly, so the changes between export start and import finish need to be tracked. It goes without saying the table is huge so a complete refresh is too slow.
Steps id like to perform:
Create table on new db.
Create mv log on old db table.
Import data from old db to new db.
Create materialized view on the new db on the prebuild table and keep refresh it from the point when the mv log was created.
The problem is that the moment the materialized view is created the mv log on the old table is purged.
Old DB:
create table kvrtest (id number, cat number);
alter table kvrtest add ( constraint pkkvrtest primary key (id) using index);
insert into kvrtest (id, cat) values (1, 1);
commit;
CREATE MATERIALIZED VIEW LOG ON kvrtest WITH PRIMARY KEY;
insert into kvrtest (id, cat) values (2, 1);
insert into kvrtest (id, cat) values (3, 2);
commit;
select * from MLOG$_KVRTEST; --Yields 2, these should be caught by a fast refresh.
New DB:
create table kvrtest (id number, cat number);
alter table kvrtest add ( constraint pkkvrtest primary key (id) using index);
insert into kvrtest (id, cat) values (1, 1); --Simulate import.
commit;
CREATE MATERIALIZED VIEW kvrtest
ON PREBUILT TABLE WITHOUT REDUCED PRECISION
USING INDEX
REFRESH FORCE ON DEMAND
AS
select * from kvrtest#oldDb;
At this point the mv log is purged
select * from MLOG$_KVRTEST; --Yields 0, a fast refresh from here does not catch these records.
Any suggestions?
when you create new MATERIALIZED VIEW with REFRESH tag log table is clearing, because our view is actually after creating.
CREATE MATERIALIZED VIEW kvrtest
ON PREBUILT TABLE WITHOUT REDUCED PRECISION
USING INDEX
REFRESH FORCE ON DEMAND
AS
if don't want to clear your log table use NEVER REFRESH and then change to REFRESH ON DEMAND like this:
CREATE MATERIALIZED VIEW kvrtest
ON PREBUILT TABLE WITHOUT REDUCED PRECISION
USING INDEX
NEVER REFRESH
AS
select * from kvrtest#oldDb;
let's see our log table
select * from MLOG$_KVRTEST;
we have two rows, nice! and then
ALTER MATERIALIZED VIEW kvrtest
REFRESH ON DEMAND;
then after refresh view log table will clear again.
Related
Suppose I have two tables job and batch:
CREATE TABLE batch
(
batch_id NUMBER(20) PRIMARY KEY,
batch_type NUMBER(20),
[some other values] ...
);
CREATE TABLE job
(
job_id NUMBER(20) PRIMARY KEY,
job_batch_id NUMBER(20),
job_usr_id NUMBER(20),
job_date DATE,
[some other values] ...
CONSTRAINT fk_job_batch
FOREIGN KEY (job_batch_id) REFERENCES batch(batch_id),
CONSTRAINT fk_job_usr
FOREIGN KEY (job_usr_id) REFERENCES client(usr_id)
);
And suppose they each contain a considerable amount of data (many millions of rows). What I want to do is create a Materialized View to reflect, for each usr_id, what the first and last jobs were run for a particular type of batch. For example:
CREATE MATERIALIZED VIEW client_first_last_job
(usr_id, first_job_date, last_job_date)
AS
(
SELECT
job_usr_id AS usr_id,
MIN(job_date) AS first_job_date,
MAX(job_date) AS last_job_date
FROM job, batch
WHERE job_batch_id=batch_id
AND batch_type IN (1,3,5,9)
GROUP BY job_usr_id
);
This is all well and good, but because there are so many records it takes a very long time to build this materialized view (far longer than is acceptable to deal with each time it needs to refresh). My immediate thought would be to use Materialized View Logs to have incremental updates. These are easy enough to create. But when I try to build the MV to use REFRESH FAST ON DEMAND, that gets me a ORA-12015: cannot create a fast refresh materialized view from a complex query error, which from some Googling I am guessing is due to the coexistence of a join and aggregate functions.
Is there another way to do this? Note that de-normalization or other alterations to the parent tables are not feasible.
You can nest your mviews which you can read about from the docs:
CREATE MATERIALIZED VIEW joinmview
(usr_id, job_date)
REFRESH FORCE ON DEMAND
AS
(
SELECT
job_usr_id AS usr_id,
job_date
FROM job, batch
WHERE job_batch_id=batch_id
AND batch_type IN (1,3,5,9)
);
CREATE MATERIALIZED VIEW LOG ON JOINMVIEW
WITH ROWID (usr_id, JOB_DATE) including new values;
CREATE MATERIALIZED VIEW client_first_last_job
(usr_id, first_job_date, last_job_date)
REFRESH FORCE ON DEMAND
AS
(
SELECT
usr_id,
MIN(job_date) AS first_job_date,
MAX(job_date) AS last_job_date
FROM joinmview
GROUP BY usr_id
);
Verify that both mviews can fast refresh:
exec dbms_mview.refresh('JOINMVIEW', 'C');
exec dbms_mview.refresh('JOINMVIEW', 'F');
exec dbms_mview.refresh('CLIENT_FIRST_LAST_JOB', 'C');
exec dbms_mview.refresh('CLIENT_FIRST_LAST_JOB', 'F');
You can put both mviews into the same refresh group (docs), just be sure to add them in the order of their dependency. In other words, in this example, add JOINMVIEW before you add CLIENT_FIRST_LAST_JOB to the refresh group.
I have problem with Oracle DB - domain index returns zero rows after search by CONTAINS() on materialized view. I see that materialized view is filled with data and I also used procedure ctx_ddl.sync_index() for domain index synchronization.
What works good:
CREATE TABLE
INSERT DATA
CREATE DOMAIN INDEX
SYNC DOMAIN INDEX
FIND ROWS BY CONTAINS - RETURN ROWS
What is not working:
CREATE TABLE
INSERT DATA
CREATE MATERIALIZED VIEW
REFRESH MATERIALIZED VIEW
CREATE DOMAIN INDEX IN MATERIALIZED VIEW
SYNC DOMAIN INDEX IN MATERIALIZED VIEW
FIND ROWS BY CONTAINS IN MATERIALIZED VIEW - RETURN ZERO ROWS (LIKE %TERM% WORKS)
Why everything works fine without materialized view?
Here is my queries (you can copy-paste and try it in your oracle db):
--create table
CREATE TABLE "PCOUNTERPARTY" ( "ID_COUNTERPARTY" NUMBER(10,0), "TXT_SEARCH_FULL_NAME" NVARCHAR2(260), CONSTRAINT "PCOUNTERPARTY_PK" PRIMARY KEY ("ID_COUNTERPARTY"));
--INSERT DATA.
Insert into PCOUNTERPARTY (ID_COUNTERPARTY,TXT_SEARCH_FULL_NAME) values (1184,'MARTINKO3');
--create materialized view
CREATE MATERIALIZED VIEW m_pcounterparty
AS
SELECT c.ID_COUNTERPARTY, CAST( c.TXT_SEARCH_FULL_NAME AS varchar2(260 CHAR) ) as txt_search_full_name_all
FROM PCOUNTERPARTY c;
--create domain index
create index IDXM_1_pcounterparty on m_pcounterparty(TXT_SEARCH_FULL_NAME_ALL) indextype is ctxsys.context PARAMETERS ('SYNC ( ON COMMIT)');
--refresh of materialized view
EXECUTE DBMS_MVIEW.REFRESH('M_PCOUNTERPARTY');
--refresh of index
exec ctx_ddl.sync_index('IDXM_1_pcounterparty');
--search in materialized view
SELECT
TXT_SEARCH_FULL_NAME_ALL
from
M_PCOUNTERPARTY c
WHERE
CONTAINS(c.TXT_SEARCH_FULL_NAME_ALL, 'martin', 1) > 0; --return ZERO and THIS IS PROBLEM
--c.TXT_SEARCH_FULL_NAME_ALL LIKE '%MARTIN%'; -- return rows but we want search thru CONTAINS
Oracle Text indexes normally search words, not strings.
A wildcard is not necessary to search for "martin" in "Martin Luther King Jr." But a wildcard is necessary to search for "martin" in "MARTINKO3".
Change the CONTAINS predicate to this:
CONTAINS(c.TXT_SEARCH_FULL_NAME_ALL, 'martin%', 1) > 0;
I ran tests on Oracle 12.2 and could not find any differences in behavior between using a table or a materialized view. Perhaps there was a test mistake or a very specific bug that caused the text indexes to act differently on your system.
So I'm pretty sure Oracle supports this, so I have no idea what I'm doing wrong. This code works:
CREATE MATERIALIZED VIEW MV_Test
NOLOGGING
CACHE
BUILD IMMEDIATE
REFRESH FAST ON COMMIT
AS
SELECT V.* FROM TPM_PROJECTVERSION V;
If I add in a JOIN, it breaks:
CREATE MATERIALIZED VIEW MV_Test
NOLOGGING
CACHE
BUILD IMMEDIATE
REFRESH FAST ON COMMIT
AS
SELECT V.*, P.* FROM TPM_PROJECTVERSION V
INNER JOIN TPM_PROJECT P ON P.PROJECTID = V.PROJECTID
Now I get the error:
ORA-12054: cannot set the ON COMMIT refresh attribute for the materialized view
I've created materialized view logs on both TPM_PROJECT and TPM_PROJECTVERSION. TPM_PROJECT has a primary key of PROJECTID and TPM_PROJECTVERSION has a compound primary key of (PROJECTID,VERSIONID). What's the trick to this? I've been digging through Oracle manuals to no avail. Thanks!
To start with, from the Oracle Database Data Warehousing Guide:
Restrictions on Fast Refresh on Materialized Views with Joins Only
...
Rowids of all the tables in the FROM list must appear in the SELECT
list of the query.
This means that your statement will need to look something like this:
CREATE MATERIALIZED VIEW MV_Test
NOLOGGING
CACHE
BUILD IMMEDIATE
REFRESH FAST ON COMMIT
AS
SELECT V.*, P.*, V.ROWID as V_ROWID, P.ROWID as P_ROWID
FROM TPM_PROJECTVERSION V,
TPM_PROJECT P
WHERE P.PROJECTID = V.PROJECTID
Another key aspect to note is that your materialized view logs must be created as with rowid.
Below is a functional test scenario:
CREATE TABLE foo(foo NUMBER, CONSTRAINT foo_pk PRIMARY KEY(foo));
CREATE MATERIALIZED VIEW LOG ON foo WITH ROWID;
CREATE TABLE bar(foo NUMBER, bar NUMBER, CONSTRAINT bar_pk PRIMARY KEY(foo, bar));
CREATE MATERIALIZED VIEW LOG ON bar WITH ROWID;
CREATE MATERIALIZED VIEW foo_bar
NOLOGGING
CACHE
BUILD IMMEDIATE
REFRESH FAST ON COMMIT AS SELECT foo.foo,
bar.bar,
foo.ROWID AS foo_rowid,
bar.ROWID AS bar_rowid
FROM foo, bar
WHERE foo.foo = bar.foo;
Have you tried it without the ANSI join ?
CREATE MATERIALIZED VIEW MV_Test
NOLOGGING
CACHE
BUILD IMMEDIATE
REFRESH FAST ON COMMIT
AS
SELECT V.*, P.* FROM TPM_PROJECTVERSION V,TPM_PROJECT P
WHERE P.PROJECTID = V.PROJECTID
You will get the error on REFRESH_FAST, if you do not create materialized view logs for the master table(s) the query is referring to. If anyone is not familiar with materialized views or using it for the first time, the better way is to use oracle sqldeveloper and graphically put in the options, and the errors also provide much better sense.
The key checks for FAST REFRESH includes the following:
1) An Oracle materialized view log must be present for each base table.
2) The RowIDs of all the base tables must appear in the SELECT list of the MVIEW query definition.
3) If there are outer joins, unique constraints must be placed on the join columns of the inner table.
No 3 is easy to miss and worth highlighting here
USE THIS CODE
CREATE MATERIALIZED VIEW MV_ptbl_Category2
BUILD IMMEDIATE
REFRESH FORCE
ON COMMIT
AS
SELECT *
FROM ptbl_Category2;
Note- MV_ptbl_Category2 is the Materialized view name
Ptbl is the table name.
Say that you have two Oracle databases, DB_A and DB_B. There is a table named TAB1 in DB_A with a materialized view log, and a materialized view named SNAP_TAB1 in DB_B created with
CREATE SNAPSHOT SNAP_TAB1
REFRESH FAST
AS SELECT * FROM TAB1#DB_A;
Is there a way to query in DB_B the changes made to SNAP_TAB1 after each call to fast-refresh the materialized view ?
DBMS_SNAPSHOT.REFRESH( 'SNAP_TAB1', 'F' );
In DB_A, prior to the refresh, you can query the materialized view log table, MLOG$_TAB1, to see which rows have been changed in TAB1. I'm looking for a way to query in DB_B, after each refresh, which rows have been refreshed in SNAP_TAB1.
Thanks!
I think the lines below work with prebuilt table:
You can add a column in the table SNAP_TAB1.
For inserts You can put it on default sysdate => for every insert you'll have the timestamp of the insert.
For updates you can use a trigger. Because the column is not involved in the Materialized View, updating the column with the trigger won't be a problem.
Probaly better, with the trigger you can use an unique id to store in that column, incremented before every new refresh.(Obtaining the unique id may have different aproaches.)
Obviously, you can't track deletes with this idea.
This question already has an answer here:
Closed 11 years ago.
Possible Duplicate:
adding primary key to sql view
I'm working with a software that requires a primary key in a Oracle view. There is possible to add a Primary key in a Oracle view? If yes, how? I can't google information about this.
The SQL standard unfortunately only permits UNIQUE and PRIMARY KEY constraints on base tables, not views. Oracle permits unique indexes on materialized views but not on views generally.
The only thing that comes in my mind is using a materialized view and then create a unique index on it:
drop materialized view tq84_mat_view;
drop table tq84_table;
create table tq84_table (
a number,
b number
);
create materialized view tq84_mat_view
refresh on commit as
select
a,
sum(b) sum_b
from
tq84_table
group by
a;
create unique index tq84_mat_view_uix on tq84_mat_view (sum_b);
insert into tq84_table values (1, 1);
insert into tq84_table values (2, 2);
insert into tq84_table values (1, 4);
commit;
insert into tq84_table values (2, 3);
commit;
--> ORA-12008: error in materialized view refresh path
--> ORA-00001: unique constraint (SPEZMDBA.TQ84_MAT_VIEW_UIX) violated
While this might be useful, it must be kept in mind that the materialized view, as opposed to a "normal" view occupies space in a tablespace. And of course, the index needs space, too.
This is the way by which you can add a primary key in your view.
CREATE OR REPLACE FORCE VIEW VU_NAME
(
PRIMARY_KEY, NAME_ID, ADDRESS_ID
)
AS
SELECT DISTINCT ROWNUM AS PRIMARY_KEY,
NAME.ID UNIT_ID,
ADDRESS_ID
from table1;