I know Oracle materialized views cannot be fast refreshed with "not exists" clause.
Is there a work around. I tried using left outer join and (+) but these 2 options too didnt seem to work. Any help is appreciated
create materialized view mv_myview refresh fast as
select a.*
from tableA a
where
not exists (select * from tableB b where a.my_id = b.my_id);
Enabling fast refresh is tricky, there are many strange restrictions and unhelpful error messages. In this case, you need to create a materialized view log WITH ROWID, use the (+) join syntax, and add a ROWID for each table.
create table tablea(my_id number primary key, a number);
create table tableb(my_id number primary key, b number);
create materialized view log on tablea with rowid;
create materialized view log on tableb with rowid;
create materialized view mv_myview refresh fast on commit as
select a.my_id, a.a, b.b, a.rowid a_rowid, b.rowid b_rowid
from tableA a, tableB b
where a.my_id = b.my_id(+)
and b.My_id IS NULL;
insert into tablea values(1, 1);
commit;
select * from mv_myview;
MY_ID A B A_ROWID B_ROWID
----- - - ------- -------
1 1 AAAUH3AAEAAC+t0AAA
Executing your query under oracle 11, I've got the following error:
Using a LEFT JOIN, I had the same problem:
create materialized view mv_myview refresh fast as
select a.*
from tableA a LEFT JOIN tableB b ON a.my_id = b.my_id
where
b.id IS NULL;
Same problem using NOT IN...
create materialized view mv_myview refresh fast as
select a.*
from tableA a
where
a.my_id not in (select b.my_id from tableB b);
First aid informations are quite clear:
ORA-12015: cannot create a fast refresh materialized view from a
complex query Cause: Neither ROWIDs and nor primary key constraints
are supported for complex queries. Action: Reissue the command with
the REFRESH FORCE or REFRESH COMPLETE option or create a simple
materialized view.
The problem seems impossible. You'll have to change the view type.
I can't think of a complete workaround. If the antijoin resulting from the not exists is inefficient for some reason then you could create a fast refresh MV based on optimising that:
select my_id, count(*)
from tab
group by my_id
Antijoins are usually pretty efficient though. You're not just missing an index are you?
Related
full code :
create materialized view log on TABLEAU.GW_STATISTICS
WITH ROWID,
SEQUENCE(AD_ID,day)including new values;
create materialized view log on TABLEAU.GW_CLIENTS
WITH ROWID,
SEQUENCE(id,NAME)including new values;
CREATE MATERIALIZED VIEW MV_CREATIVE_DCO_ADWORDS_2
NOLOGGING
NOCOMPRESS
CACHE
NOPARALLEL
REFRESH FAST ON DEMAND
as
select TABLEAU.GW_STATISTICS.ROWID,
ACC.ROWID,
ACC.NAME,
TABLEAU.GW_STATISTICS.AD_ID,
min(TABLEAU.GW_STATISTICS.DAY),
TABLEAU.GW_STATISTICS.DAY
FROM TABLEAU.GW_STATISTICS,
TABLEAU.GW_CLIENTS ACC
WHERE TABLEAU.GW_STATISTICS.CLIENTID=ACC.ID(+) and TABLEAU.GW_STATISTICS.DAY>TO_DATE('20200531','yyyymmdd')
group by TABLEAU.GW_STATISTICS.ROWID,
ACC.ROWID,
ACC.NAME,
TABLEAU.GW_STATISTICS.AD_ID,
TABLEAU.GW_STATISTICS.DAY;
working create materialized view
WHERE TABLEAU.GW_STATISTICS.CLIENTID=ACC.ID(+)
not working create materialized view
WHERE TABLEAU.GW_STATISTICS.CLIENTID=ACC.ID(+)
AND TABLEAU.GW_STATISTICS.DAY>TO_DATE('20200531','yyyymmdd')
00000 - "cannot fast refresh materialized view %s.%s"
*Cause: Either ROWIDs of certain tables were missing in the definition or
the inner table of an outer join did not have UNIQUE constraints on
join columns.
*Action: Specify the FORCE or COMPLETE option. If this error is got
during creation, the materialized view definition may have be
changed. Refer to the documentation on materialized views.
Why cant i add a filter by date ? How to get around this limitation ?
Cause: Either ROWIDs of certain tables were missing in the definition...
For starters, you need to include the ROWID from each table in your select:
CREATE MATERIALIZED VIEW MV_CREATIVE_DCO_ADWORDS
NOLOGGING
NOCOMPRESS
CACHE
NOPARALLEL
REFRESH FAST ON DEMAND
as
select TABLEAU.GW_STATISTICS.ROWID,
ACC.ROWID,
ACC.NAME,
TABLEAU.GW_STATISTICS.AD_ID,
min(TABLEAU.GW_STATISTICS.DAY),
TABLEAU.GW_STATISTICS.DAY
FROM TABLEAU.GW_STATISTICS,
TABLEAU.GW_CLIENTS ACC
WHERE TABLEAU.GW_STATISTICS.CLIENTID=ACC.ID(+)
AND TABLEAU.GW_STATISTICS.DAY>TO_DATE('20200531','yyyymmdd')
group by TABLEAU.GW_STATISTICS.ROWID,
ACC.ROWID,
ACC.NAME,
TABLEAU.GW_STATISTICS.AD_ID,
TABLEAU.GW_STATISTICS.DAY;
or the inner table of an outer join did not have UNIQUE constraints on
join columns.
Also, TABLEAU.GW_STATISTICS.CLIENTID needs to have a unique constraint.
Maybe you need to use alias:
select
TABLEAU.GW_STATISTICS.ROWID as GW_STATISTICS_ROWID,
ACC.ROWID AS ACC_ROWID,
ACC.NAME,
TABLEAU.GW_STATISTICS.AD_ID,
min(TABLEAU.GW_STATISTICS.DAY) AS MIN_DAY,
TABLEAU.GW_STATISTICS.DAY
or try
FROM (SELECT * FROM TABLEAU.GW_STATISTICS WHERE DAY > DATE '2020-05-31') STATS,
TABLEAU.GW_CLIENTS ACC
WHERE STATS.CLIENTID = ACC.ID(+)
Should be the same, I think
For performance needs I want to create a materialized view on commit refresh option using the following script:
CREATE TABLE DEVDV
(DEVDV_ID INTEGER PRIMARY kEY,
DEVDV_SRC_DVISE_ID INTEGER,
DEVDV_CIB_DVISE_ID INTEGER);
CREATE TABLE CONDV
(CONDV_ID INtEgEr PRiMARY KEY,
CONDV_DEVDV_iD INTEGER,
CONDV_TX NUMbeR,
CONDV_DATE_DEB datE,
CONDV_DATE_FIN DATE);
CREATE MATERIALIZED VIEW LOG ON DEVDV WITH ROWID;
CREATE MATERIALIZED VIEW LOG ON CONDV WITH ROWID;
CREATE MATERIALIZED VIEW MV_DEVDV_TYP_2
REFRESH FAST
ON COMMIT
AS
SELECT DEVDV.ROWID CROWID,
CONDV.ROWID DROWID,
DEVDV_ID,
1 AS MARKER,
DEVDV_SRC_DVISE_ID,
DEVDV_CIB_DVISE_ID,
CONDV_TX,
CONDV_DATE_DEB,
CONDV_DATE_FIN
FROM
DEVDV INNER JOIN CONDV ON DEVDV_ID = CONDV_DEVDV_ID
UNION ALL
SELECT DEVDV.ROWID CROWID,
CONDV.ROWID DROW_ID,
DEVDV_ID,
2 AS MARKER,
DEVDV_CIB_DVISE_ID,
DEVDV_SRC_DVISE_ID,
1/CONDV_TX,
CONDV_DATE_DEB,
CONDV_DATE_FIN
FROM
DEVDV INNER JOIN CONDV ON DEVDV_ID = CONDV_DEVDV_ID;
Oracle says that it's a complex query and it doesn't meet the fast refresh requirements,
Could you please tell me which rule I've broken?
I don't know why, but - for materialized views in oracle - you have to use old syntax for joins. So put all tables in FROM separating them with commas, and join condition put in where clause (use "(+)" for outer joins).
That works for me:
CREATE MATERIALIZED VIEW MV_DEVDV_TYP_2
REFRESH FAST
ON COMMIT
AS
SELECT DEVDV.ROWID CROWID,
CONDV.ROWID DROWID,
DEVDV_ID,
1 AS MARKER,
DEVDV_SRC_DVISE_ID,
DEVDV_CIB_DVISE_ID,
CONDV_TX,
CONDV_DATE_DEB,
CONDV_DATE_FIN
FROM
DEVDV, CONDV
WHERE DEVDV_ID = CONDV_DEVDV_ID
UNION ALL
SELECT DEVDV.ROWID CROWID,
CONDV.ROWID DROW_ID,
DEVDV_ID,
2 AS MARKER,
DEVDV_CIB_DVISE_ID,
DEVDV_SRC_DVISE_ID,
1/CONDV_TX,
CONDV_DATE_DEB,
CONDV_DATE_FIN
FROM
DEVDV, CONDV
WHERE DEVDV_ID = CONDV_DEVDV_ID;
Materialized view MV_DEVDV_TYP_2 created.
I need to create materialized view test without data then I will create a script to insert data into this materialized view for the first time. After this I will run materialized view refresh to refresh the view every night.
As I am not expert in materialized views can anyone help me here.
At present I have script to create materialized view which is running for 2 hours for 20 million rows.
create materialize view
If I understand the question correctly, you want to break up the MV creation into separate steps:
Create an empty table / materialized view.
Populate it.
Schedule a nightly refresh process.
For this you can use the on prebuilt table clause to change a normal table into a materialized view.
Demo source table:
create table demo_source (id, name) as
select 1, 'Red' from dual union all
select 2, 'Yellow' from dual union all
select 3, 'Orange' from dual union all
select 4, 'Blue' from dual;
New table which is going to be our MV (you could also populate it with the create table as select, or you could create it using explicit column names, datatypes, constraints, partitioning etc like any normal table):
create table demo_mv as
select * from demo_source s
where 1 = 2;
Populate it using a separate insert step:
insert into demo_mv
select * from demo_source;
Now we convert it from a regular table into an MV:
create materialized view demo_mv on prebuilt table
as
select * from demo_source;
Now DEMO_MV is a materialized view.
If I were you, I'd create the materialized view "as is" (i.e. no restrictions you mentioned).
Anyway: the simplest option is to include the false condition in the WHERE clause which creates the object without data, such as
SQL> create materialized view mv_dept as
2 select * from dept
3 where 1 = 2; --> this
Materialized view created.
SQL> select * from mv_dept;
no rows selected
SQL> desc mv_dept;
Name Null? Type
----------------------------- -------- --------------------
DEPTNO NOT NULL NUMBER(2)
DNAME VARCHAR2(14)
LOC VARCHAR2(13)
SQL>
I know this question was asked specifically about Oracle, but I got here looking for the same question about Postgres.
Luckily, Postgres has a 'WITH NO DATA' clause at the end of the materialized view statement that just creates the view but does not populate data into it. It can still be refreshed on-demand the same way after that.
https://www.postgresql.org/docs/current/sql-creatematerializedview.html
I have the same problem. At deploy time I don't want that the refresh takes to0 much time. So here I think is a better solution.
drop materialized view test_mv;
create materialized view test_mv
as select * from all_objects
where 1 = ( select count(*) from user_tables where table_name = 'TEST_MV' ) ;
select * From test_mv
=> null
exec DBMS_MVIEW.REFRESH('TEST_MV', method => 'C', atomic_refresh => FALSE, out_of_place => false , PARALLELISM => 4);
select * From test_mv
=> result is now of all objects
We've got a highly (perhaps over?) normalized table that keeps track of versioned values. It's insert only, no updates.
Example Data:
"ID" "Version" "Value"
1 0 "A_1"
2 0 "B_1"
1 1 "A_2"
3 0 "C_1"
We frequently run queries to pull only the latest values for each ID. As we're hitting millions of rows, we're starting to encounter performance problems. I've been able to prototype improvements using Materialized Views, but have not been able to create them in such a way that they self-refresh "ON COMMIT"
What I've got so far is this (Revised below)
CREATE MATERIALIZED VIEW TABLE_LATEST
BUILD IMMEDIATE
REFRESH FAST
ON COMMIT AS
SELECT T.ID
,T.LAST_VERSION
FROM (
SELECT ID
,MAX(VERSION) OVER (PARTITION BY ID) LAST_VERSION
FROM TABLE
) T
GROUP BY T.ID, T.LAST_VERSION;
Which is now revised, due to feedback:
CREATE MATERIALIZED VIEW TABLE_LATEST
BUILD IMMEDIATE
REFRESH FAST
ON COMMIT AS
SELECT ID
,MAX(VERSION)
FROM TABLE
GROUP BY T.ID;
Which fails with:
ORA-12033: cannot use filter columns from materialized view log on
"SCHEMA"."TABLE"
*Cause: The materialized view log either did not have filter columns
logged, or the timestamp associated with the filter columns was
more recent than the last refresh time.
*Action: A complete refresh is required before the next fast refresh.
Add filter columns to the materialized view log, if required.
It will only 'work' if I change Refresh to Force and remove On Commit. I can't tell if this falls under the 'No Analytics' rule for Materialized Views or if perhaps I've created the Log incorrectly in the first place?
CREATE MATERIALIZED VIEW LOG ON TABLE
LOGGING
WITH SEQUENCE, ROWID, (VALUE)
INCLUDING NEW VALUES;
Table Schema:
CREATE TABLE "TABLE"
(
ID NUMBER(10, 0) NOT NULL
, VERSION NUMBER(10, 0) NOT NULL
, VALUE VARCHAR2(4000 CHAR)
, CONSTRAINT MASTERRECORDFIELDVALUES_PK PRIMARY KEY
(
ID
, VERSION
)
USING INDEX
(
CREATE UNIQUE INDEX TABLE_PK ON TABLE(ID ASC, VERSION ASC)
LOGGING
...
)
ENABLE
)
LOGGING
Am I even on the right track? Would there be a better performing way to pre-calculate latest versions? Or do I just need to get the Log & View settings dialed in?
If you don't need the value associated with the latest version, then you can simply do:
CREATE MATERIALIZED VIEW LOG ON t1
LOGGING
WITH SEQUENCE, ROWID, (val)
INCLUDING NEW VALUES;
create materialized view t1_latest
refresh fast on commit
as
select id,
max(version) latest_version
from t1
group by id;
The test case for this can be found over at Oracle LiveSQL.
Otherwise, you need to create three separate MVs (because you can't have a fast refreshable on commit materialized view that involves keep dense_rank first/last) - as per http://www.sqlsnippets.com/en/topic-12926.html - like so:
Materialized view log on the main table:
CREATE MATERIALIZED VIEW LOG ON t1
LOGGING
WITH SEQUENCE, ROWID, (val)
INCLUDING NEW VALUES;
First materialized view:
create materialized view t1_sub_mv1
refresh fast on commit
as
select id,
max(version) latest_version,
count(version) cnt_version,
count(*) cnt_all
from t1
group by id;
Materialized view log on the first materialized view:
create materialized view log on t1_sub_mv1
with rowid, sequence (id, latest_version, cnt_version, cnt_all)
including new values;
Second materialized view:
create materialized view t1_sub_mv2
refresh fast on commit
as
select id,
version,
max(val) max_val_per_id_version,
count(*) cnt_all
from t1
group by id,
version;
Materialized view log on the first materialized view:
create materialized view log on t1_sub_mv2
with rowid, sequence (id, max_val_per_id_version, cnt_all)
including new values;
Third and final materialized view:
create materialized view t1_main_mv
refresh fast on commit
as
select mv1.id,
mv1.latest_version,
mv2.max_val_per_id_version val_of_latest_version,
mv1.rowid mv1_rowid,
mv2.rowid mv2_rowid
from t1_sub_mv1 mv1,
t1_sub_mv2 mv2
where mv1.id = mv2.id
and mv1.latest_version = mv2.version;
The supporting test case for this can be found over at Oracle LiveSQL.
I am sorry but I can't give you an answer straight away. One reason might be the use of analytic function which are not that well supported by MVs. To analyze the problem you will need to take a look at the capabilities of the materialized view.
DECLARE
-- Local variables here
--
v_sql VARCHAR2(32000) := 'SELECT T.ID
,T.LAST_VERSION
FROM (SELECT ID
,MAX(VERSION) OVER (PARTITION BY ID) LAST_VERSION
FROM TABLE) T
GROUP BY T.ID
,T.LAST_VERSION';
v_msg_arrray SYS.EXPLAINMVARRAYTYPE;
msg SYS.ExplainMVMessage;
BEGIN
-- Test statements here
dbms_mview.explain_mview(mv => v_sql, msg_array => v_msg_arrray);
FOR i IN v_msg_arrray.FIRST..v_msg_arrray.LAST LOOP
msg := v_msg_arrray(i);
DBMS_OUTPUT.put_line('MVOWNER:' || msg.MVOWNER);
DBMS_OUTPUT.put_line('MVNAME:' || msg.MVNAME);
DBMS_OUTPUT.put_line('CAPABILITY_NAME:' || msg.CAPABILITY_NAME);
DBMS_OUTPUT.put_line('POSSIBLE:' || msg.POSSIBLE);
DBMS_OUTPUT.put_line('RELATED_TEXT:' || msg.RELATED_TEXT);
DBMS_OUTPUT.put_line('RELATED_NUM:' || msg.RELATED_NUM);
DBMS_OUTPUT.put_line('MSGNO:' || msg.MSGNO);
DBMS_OUTPUT.put_line('MSGTXT:' || msg.MSGTXT);
DBMS_OUTPUT.put_line('SEQ:' || msg.SEQ);
DBMS_OUTPUT.put_line('----------------------------------------');
END LOOP;
END;
BTW: You can write your query far simpler:
SELECT t.id,
MAX(t.version) AS last_version
FROM table t
GROUP BY t.id;
What's wrong with this fast refreshable view definition in Oracle 10.2:
create table A
(
ID number(19,0) not null constraint A_PK primary key using index
, C number(9,0) not null
);
create table B
(
ID number(19,0) not null constraint B_PK primary key using index
, A_ID number(19,0) not null constraint A_FK references A(ID) on delete cascade
, D number(9,0) not null
);
create index B_FK_IDX on B(A_ID);
create materialized view log on A
with primary key, rowid, sequence (C) including new values;
create materialized view log on B
with primary key, rowid, sequence (A_ID, D) including new values;
create materialized
view X
refresh fast with primary key
as
select A.ID as A_ID
, A.ROWID as A_ROWID
, B.ID as B_ID
, B.ROWID as B_ROWID
, A.C
, B.D
from A
inner join
B
on B.A_ID = A.ID;
When the script is executed I get:
table A created.
table B created.
index B_FK_IDX created.
materialized view LOG created.
materialized view LOG created.
...[view definition and local error message left out]
SQL-Error: ORA-12015: cannot create a fast refresh materialized view from a complex query
12015. 00000 - "cannot create a fast refresh materialized view from a complex query"
*Cause: Neither ROWIDs and nor primary key constraints are supported for
complex queries.
*Action: Reissue the command with the REFRESH FORCE or REFRESH COMPLETE
option or create a simple materialized view.
I can not see a violation of any of the restrictions for materialized views as defined in Oracle's support document 179466.1.
You must not use ANSI join syntax, use the old Oracle join syntax. It is a bug in Oracle.
Long time ago I opened a case for this, however Oracle considers this only as lack of documentation!
From the docs:
Restrictions on Fast Refresh on Materialized Views with Joins Only
Defining queries for materialized views with joins only and no aggregates have the following restrictions on fast refresh:
All restrictions from "General Restrictions on Fast Refresh".
They cannot have GROUP BY clauses or aggregates.
Rowids of all the tables in the FROM list must appear in the SELECT list of the query.
Materialized view logs must exist with rowids for all the base tables in the FROM list of the query.