I have some questions about optimizing materialized views (MVs):
In the following tutorial it is recommended to set PCTFREE and PCTUSED to 0 or 99:
Note: If a materialized view is complete refreshed, then set it's
PCTFREE to 0 and PCTUSED to 99 for maximum efficiency.
--> https://satya-dba.blogspot.com/2009/07/materialized-views-oracle.html
First question:
Is this recommendation correct? Should the two parameters PCTFREE 0 and PCTUSED 99 be set accordingly?
How is that exact procedure when the MV is updated?
The MV is created ( PCTFREE 0 and PCTUSED 99 are set accordingly):
CREATE MATERIALIZED VIEW "SYSK85"."CMVF_01_SK85_BESTAND_2019" ("BRNR", "LEAT", "TLRTAL", "LEATSO", "STAND_DATUM", "VOAT", "AQEQ", "ELAT", "ELDT", "VSKN", "VTKZ", "ANZAHL")
PCTFREE 0 PCTUSED 99
NOCOMPRESS LOGGING
NO INMEMORY
BUILD IMMEDIATE
USING INDEX
REFRESH FORCE ON DEMAND
ENABLE QUERY REWRITE
AS (
select BRNR, LEAT, TLRTAL, LEATSO, STAND_DATUM, VOAT, AQEQ, ELAT, ELDT, VSKN, VTKZ, count(*) as ANZAHL
from CVF_SK85_BESTAND_2019 GROUP BY BRNR, LEAT, TLRTAL, LEATSO, STAND_DATUM, VOAT, AQEQ, ELAT, ELDT, VSKN, VTKZ
);
Second question:
Should the Create of the MV be set the two parameters PCTFREE 0 and PCTUSED 99?
But what if the MV is updated?
EXEC DBMS_MVIEW.REFRESH('CMVF_01_SK85_BESTAND_2019', 'C', atomic_refresh=>FALSE);
Should the two parameters PCTFREE and PCTUSED be reset again explicitly after the update (so that the database can reorganize the data - if necessary)? Or is not that necessary?
Thank you very much
Kind regards
George
The imporatant point is the limitation to the complete refresh of the MV.
Such MV are refreshed either by truncate and insert or delete + insert, so the recommendation boils down to a trivial statement that you should not reserve space for future updates that would newer happen in complete refresh (PCTFREE = 0).
This will lead to a decrease of the size of the MV.
PCTUSED plays IMO no role in this scenario as there are no subsequent inserts.
Both parameters remain valid until they are explicitely changed with ALTER TABLE.
Summary
If you are unsure about the refresh strategy, I'd never touch those parameters.
If you know that you only make full refressh and you make a lot of full tables scans on the MV - you may see an optimizing effect. While doing normal OLTP index access and nested loops joins on the MV you'll hardly see a difference.
Related
I'm trying to introduce new audit tables for our project. In the deployment process we use redgate's Schema Compare for Oracle (version 4.0.8.420).
The table looks something like this:
create table MY_SCHEMA.MY_TEST_AUDT (
TEST_ID NUMBER(19),
-- all sorts of business fields, omitted for clarity
AUDT_CRT_DTM TIMESTAMP DEFAULT SYSTIMESTAMP,
AUDT_ACTN_CODE VARCHAR2(1),
AUDT_CRT_USR_NM VARCHAR2(128) DEFAULT USER,
AUDT_CLIENT_IDENTIFIER VARCHAR2(256),
AUDT_CLIENT_INFO VARCHAR2(256)
)
TABLESPACE MY_TABLESPACE
PCTFREE 0
INITRANS 10
STORAGE (
INITIAL 64K
NEXT 1M
MINEXTENTS 1
MAXEXTENTS UNLIMITED
PCTINCREASE 0
BUFFER_POOL DEFAULT
)
COMPRESS FOR OLTP
NOCACHE
PARTITION BY RANGE (AUDT_CRT_DTM)
INTERVAL(interval '1' month)
(
PARTITION P0 VALUES LESS THAN (date '2018-11-01')
PCTFREE 0
INITRANS 10
)
/
The first time I ran it I got an error concerning the storage clause
Parsing failed with message SyntaxError. Unexpected token 'K'
When I got rid of the storage clause (since I can use the defaults) it started complaining about the partitioning clause and that's where I am not very happy with the software.
Parsing failed with message SyntaxError. Unexpected token 'PARTITION' (Line 35, Col 1) symbol Id
I tried turning all the storage options on and off, nothing worked. I tried the latest version 5.2 with a simple compare of files and it didn't work either. I tried to post it on the redgate forums and my post has been stuck as drafted for two days now.
I'm using the scripts folder comparison, the above mentioned file for source and no file for the target, Oracle 11g scripts.
I have managed to get it working without the partition. I had to replace the slash with a semicolon and switch the 8K and 1M to the full values. But I'm still not able to create partitions.
create table MY_SCHEMA.MY_TEST_AUDT (
TEST_ID NUMBER(19),
-- all sorts of business fields, omitted for clarity
AUDT_CRT_DTM TIMESTAMP DEFAULT SYSTIMESTAMP,
AUDT_ACTN_CODE VARCHAR2(1),
AUDT_CRT_USR_NM VARCHAR2(128) DEFAULT USER,
AUDT_CLIENT_IDENTIFIER VARCHAR2(256),
AUDT_CLIENT_INFO VARCHAR2(256)
)
TABLESPACE MY_TABLESPACE
PCTFREE 0
INITRANS 10
STORAGE (
INITIAL 65536
NEXT 1048576
MINEXTENTS 1
MAXEXTENTS UNLIMITED
PCTINCREASE 0
BUFFER_POOL DEFAULT
)
COMPRESS FOR OLTP
NOCACHE;
Any help is very much appreciated.
Alain
For completeness here's my DatabaseInformation.xml file
<?xml version="1.0" encoding="utf-16" standalone="yes"?>
<ScriptsFolderInformation version="2" type="ScriptsFolderInformation">
<DatabaseVersion>ElevenG</DatabaseVersion>
</ScriptsFolderInformation>
I asked the redgate support and got a reply.
It seems to work just fine when you work with the schemas directly (haven't tried it myself). The problem only happens when you do the scripts folder to scripts folder comparison.
To get the partition working you have to drop the NOCACHE keyword. Then everything is working.
Redgate now has a bug report for the support of those keywords (OC-1026)
Here's the version that works:
create table MY_SCHEMA.MY_TEST_AUDT (
TEST_ID NUMBER(19),
-- all sorts of business fields, omitted for clarity
AUDT_CRT_DTM TIMESTAMP DEFAULT SYSTIMESTAMP,
AUDT_ACTN_CODE VARCHAR2(1),
AUDT_CRT_USR_NM VARCHAR2(128) DEFAULT USER,
AUDT_CLIENT_IDENTIFIER VARCHAR2(256),
AUDT_CLIENT_INFO VARCHAR2(256)
)
TABLESPACE MY_TABLESPACE
PCTFREE 0
INITRANS 10
STORAGE (
INITIAL 65536
NEXT 1048576
MINEXTENTS 1
MAXEXTENTS UNLIMITED
PCTINCREASE 0
BUFFER_POOL DEFAULT
)
COMPRESS FOR OLTP;
I have read quite a lot for past few hours about refreshing MV in Oracle, but I cannot still find an answer to my question. Imagine I have a MV view on top of a table with change logs. So that there are three records in this MV:
COL_ID, COL1
1, "OLD"
2, "OLD"
3, "OLD"
Now let's say the value for COL1 has changed to "EDITED" for record 1 in the table used to create MV. I want to perform fast, in-place refresh to update MV as fast as possible. In real life example with around 50M records, this would take around 3 minutes to refresh.
Imagine situation.
Refresh process is still running (there are still records that have not been amended in the MV).
Neverthelss, record with ID = 1 has been already processed so that it has value of "EDITED" in MV.
In another session, a query to MV is executed to get the value of record with ID = 1.
As the result will record with value "OLD" or "EDITED" be given?
I understand that because this is an in-place refresh after this record is processed by refresh mechanism the value in the materialized view will reflect the value in origin table ("EDITED"). But is there any mechanism (like undo logs) that would make the whole refresh atomic? And by this I mean that unless all amendments are done to the materialized views (unless the refresh process is done) if user queries for rows that have been modified in the process of undergoing refresh, he/she will be presented to an old, cached value - before the change.
I presume that this bevahiour is true for out-of-place refresh, but since the former seems to be working way more efficient in terms of time consumption I was curious whether this is also true for the in-place transformation. If not by default, is there any way to force this atomic behaviour?
----- [EDIT]
I run the test following test to see if during the refresh process the results I get from the materialized view will change gradually.
-- create table
create table MV_REFRESH_ATOMICITY_TEST
(
id NUMBER,
value NUMBER
)
-- populate initial data
-- delete from MV_REFRESH_ATOMICITY_TEST
declare
begin
for i in 1..10000000 loop
insert into MV_REFRESH_ATOMICITY_TEST values(i, 0);
end loop;
end;
-- check if equal zero and 1M
select sum(value) from MV_REFRESH_ATOMICITY_TEST
select to_char(count(*),'999,999,999') as COUNT from MV_REFRESH_ATOMICITY_TEST
-- create mv logs on the table
-- drop materialized view log on MV_REFRESH_ATOMICITY_TEST;
create materialized view log on MV_REFRESH_ATOMICITY_TEST with rowid;
-- create mv on top
-- drop materialized view MV_REFRESH_ATOMICITY_TEST_MV
create materialized view MV_REFRESH_ATOMICITY_TEST_MV
refresh fast on demand with rowid
as
select
fact.*,
fact.ROWID "FACT_ROWID"
from
MV_REFRESH_ATOMICITY_TEST fact
-- check if equals zero and 10M
select sum(value) from MV_REFRESH_ATOMICITY_TEST_MV
select to_char(count(*),'999,999,999') as COUNT from MV_REFRESH_ATOMICITY_TEST_MV
-- change value for first million records, 1 milion records in the middle, last milion of records
update MV_REFRESH_ATOMICITY_TEST set value = 1 where id between 1 and 1000000
update MV_REFRESH_ATOMICITY_TEST set value = 1 where id between 5000001 and 6000000
update MV_REFRESH_ATOMICITY_TEST set value = 1 where id between 9000001 and 10000000
-- check if equals 3.000.000
select to_char(sum(value),'999,999,999') as "SUM" from MV_REFRESH_ATOMICITY_TEST
-- check if equals 3.000.000
select to_char(count(*),'999,999,999') from MLOG$_MV_REFRESH_ATOMICITY;
--select * from MLOG$_MV_REFRESH_ATOMICITY;
-- while refreshing mv
-- exec dbms_mview.refresh('MV_REFRESH_ATOMICITY_TEST_MV', 'F');
-- below sum should be equal 0
select
( select sum(value) from MV_REFRESH_ATOMICITY_TEST_MV ) "SUM",
( select count(*) from MV_REFRESH_ATOMICITY_TEST_MV ) "NUMBER OF RECORDS"
from dual
So what I discovered by constantly executing the last select statement was that the only time that the SUM value changed, it already changed by 3M, meaning all the records have been changed in one go - atomically.
Nevertheless, I am not 100% percent sure I can trust this experiment as at some point it took around 40s to execute these select queries. The whole refresh statement took 911s to execute.
[EDIT]
This question has been marked as a possible duplicate of this thread. The other thread does respond to a similar problem, but for a complete-refresh which to my understanding is performed in very different way than fast-refresh which is the case here. Therefore I am not sure whether the same explanation can be applied here.
As far I can see from Oracle documentation (http://docs.oracle.com/cd/B19306_01/server.102/b14226/repmview.htm#i31171) - all refresh is done in atomic way:
A materialized view's data does not necessarily match the current data of its master table or master materialized view at all times. A materialized view is a transactionally (read) consistent reflection of its master as the data existed at a specific point in time (that is, at creation or when a refresh occurs).
Oracle provides even greater read consistency with materialized view groups:
To preserve referential integrity and transactional (read) consistency among multiple materialized views, Oracle has the ability to refresh individual materialized views as part of a refresh group. After refreshing all of the materialized views in a refresh group, the data of all materialized views in the group correspond to the same transactionally consistent point in time.
I would like to use parallel execution on LET_TESTATE_LETTURE without forcing the full table scan, I want to use force the parallelism on index.
How can I solve?
alter session enable parallel dml;
CREATE TABLE netatemp.let_testate_letture1
AS
SELECT /* parallel(tele 32) full(tele) */
tele.TELE_DATA_LETTURA,
tele.tele_storico_id
FROM let_testate_letture tele
WHERE tele.prov_provenienza_lettura_id = '*1ENI01BCAMBIO'
AND tele.spwkf_stato_pubblico_id != '*1UNICOANN';
Size 56,1 GB
Number Extents 1.081
OWNER SIUMETERING
TABLE_NAME LET_TESTATE_LETTURE
TABLESPACE_NAME SIUMETERING_DATITD
CLUSTER_NAME
IOT_NAME
STATUS VALID
PCT_FREE 10
PCT_USED
INI_TRANS 30
MAX_TRANS 255
INITIAL_EXTENT 80 KB
NEXT_EXTENT 1 MB
MIN_EXTENTS 1
MAX_EXTENTS 2.147.483.645
PCT_INCREASE
FREELISTS
FREELIST_GROUPS
LOGGING YES
BACKED_UP N
NUM_ROWS 456.635.338
BLOCKS 3.340.120
EMPTY_BLOCKS 0
AVG_SPACE 0
CHAIN_CNT 0
AVG_ROW_LEN 385
AVG_SPACE_FREELIST_BLOCKS 0
NUM_FREELIST_BLOCKS 0
DEGREE 1
INSTANCES 1
CACHE N
TABLE_LOCK ENABLED
SAMPLE_SIZE 456.635.338
LAST_ANALYZED 29/12/2012 13:03:15
PARTITIONED NO
IOT_TYPE
TEMPORARY N
SECONDARY N
NESTED NO
BUFFER_POOL DEFAULT
FLASH_CACHE DEFAULT
CELL_FLASH_CACHE DEFAULT
ROW_MOVEMENT DISABLED
GLOBAL_STATS YES
USER_STATS NO
DURATION
SKIP_CORRUPT DISABLED
MONITORING YES
CLUSTER_OWNER
DEPENDENCIES DISABLED
COMPRESSION ENABLED
COMPRESS_FOR OLTP
DROPPED NO
READ_ONLY NO
SEGMENT_CREATED YES
RESULT_CACHE DEFAULT
you have to alter the index to parallel. ie
alter index xxx parallel;
or
alter index xxx parallel <n>;
as the parallel hint only applies to tables.
Try
/*+ parallel_index(tele, let_tele_letb_prov_fk_idx, 32) */
Notice the "+" after the asterisk. Without it Oracle will ignore the hint.
Also, you may want to create the table in parallel as well depending on the nr of rows returned, like:
CREATE TABLE netatemp.let_testate_letture1 parallel 32 as
select /*+ ...
You posted a lot of useful information, which is very refreshing. So in addition to answering your question, I can provide
other advice for improving performance:
Statement-level hint.
Since you are on 11gR2 (based on the existence of the column SEGMENT_CREATED), you should use a
statement-level parallel hint, instead of an object-level. Use /*+ parallel(32) */, and then Oracle will parallelize everything in the query, regardless of the access method or the alias names.
Old stats or old data?
Last Analyzed on 2012-12-29 seems kind of old. If your table is very active, then you should re-gather statistics. If it really doesn't change very often, you may want to consider re-creating it, ordered by prov_provenienza_lettura_id. That may significantly improve the performance of your index. Although it could decrease performance of other indexes.
Compression.
Your table uses compression, but is your index also compressed? If you really have 9 million entries for the same value, index compression could save a huge amount of space, and make index reads much faster. Also, a bitmap index may be a good fit here.
Full table scans.
The optimizer thinks you're going to read about 2% of your rows. Even 2% may be enough to warrant a full table scan, depending on things such as the clustering factor. You may not want to try to force a specific access method - first let Oracle try to pick. If Oracle gets it wrong, then you should try to help it by providing more useful information, such as better statistics and maybe a histogram.
The advice from DazzaL and Ronnis should also be helpful.
I've traced a bug in my Java EE application to the Oracle database: there is a materialized view which is not refreshing correctly. If I do a query against the MV, it gives me foreign keys which are bad and appear to be old.
So how can I fix or replace this materialized view? Any thoughts are welcome.
I tried refreshing manually, like this:
DBMS_MVIEW.REFRESH('PRODUCTDESCRIPTIONS', 'C');
I got the error "ORA-00942: table or view does not exist". I don't understand this, because when I run the MV's subquery by hand, it looks fine.
The Apex Web interface indicates that the MV has not refreshed for over a year, so this is not a new problem.
I looked for any logging from the refresh process, but couldn't find the file refresh.log.
I've tried replacing the materialized view with a simple query, but it's too slow. I'd be happy to rewrite/reconfigure/reinstall the MV somehow.
Database and OS version:
Oracle Database 10g Express Edition Release 10.2.0.1.0 - Product
PL/SQL Release 10.2.0.1.0 - Production
uname -a:
Linux <server name> 2.6.9-78.0.22.ELsmp #1 SMP Thu Apr 30 19:14:39 EDT 2009 i686 i686 i386 GNU/Linux
Source code for the materialized view:
CREATE MATERIALIZED VIEW "PRODUCTDESCRIPTIONS"
ORGANIZATION HEAP PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT)
TABLESPACE "USERS"
BUILD IMMEDIATE
USING INDEX
REFRESH COMPLETE ON DEMAND
USING DEFAULT LOCAL ROLLBACK SEGMENT
DISABLE QUERY REWRITE
AS SELECT prdcts.primarykey AS product,
prdcts.upcid AS productupcid,
prdcts.description AS productdescription,
prdctctgrs.primarykey AS productcategory,
prdctctgrs.id AS productcategoryid,
prdctctgrs.name AS productcategoryname,
prdctpkgs.primarykey AS productpackage,
prdctpkgs.name AS productpackagename FROM prdctctgrs,
prdcts,
prdctpkgs,
prdctctgrstoprdcts,
prdctstoprdctpkgs
WHERE
prdctctgrstoprdcts.productcategory = prdctctgrs.primarykey
AND prdctctgrstoprdcts.product = prdcts.primarykey
AND prdctstoprdctpkgs.product = prdcts.primarykey
AND prdctstoprdctpkgs.productpackage = prdctpkgs.primarykey
AND bitand(prdctctgrs.metaflags, 1)+0 = 0
AND bitand(prdcts.metaflags, 1)+0 = 0
AND bitand(prdctpkgs.metaflags, 1)+0 = 0
AND bitand(prdctctgrstoprdcts.metaflags, 1)+0 = 0
AND bitand(prdctstoprdctpkgs.metaflags, 1)+0 = 0
/
When you run the refresh procedure, are you executing it as the owner of the tables you're selecting from? Are all of the tables you're accessing directly granted to you? If the tables are granted to you via roles, then the refresh procedure won't be able to see them.
Just to confirm my comment on the original question: dropping and recreating the MV fixed the problem.
I have an Oracle DB package that is routinely causing what I believe is an ITL (Interested Transaction List) deadlock. The relevant portion of a trace file is below.
Deadlock graph:
---------Blocker(s)-------- ---------Waiter(s)---------
Resource Name process session holds waits process session holds waits
TM-0000cb52-00000000 22 131 S 23 143 SS
TM-0000ceec-00000000 23 143 SX 32 138 SX SSX
TM-0000cb52-00000000 30 138 SX 22 131 S
session 131: DID 0001-0016-00000D1C session 143: DID 0001-0017-000055D5
session 143: DID 0001-0017-000055D5 session 138: DID 0001-001E-000067A0
session 138: DID 0001-001E-000067A0 session 131: DID 0001-0016-00000D1C
Rows waited on:
Session 143: no row
Session 138: no row
Session 131: no row
There are no bit-map indexes on this table, so that's not the cause. As far as I can tell, the lack of "Rows waited on" plus the "S" in the Waiter waits column likely indicates that this is an ITL deadlock. Also, the table is written to quite often (roughly 8 insert or updates concurrently, as often as 240 times a minute), so an ITL deadlock seems like a strong possibility.
I've increased the INITRANS parameter of the table and it's indexes to 100 and increased the PCT_FREE on the table from 10 to 20 (then rebuilt the indexes), but the deadlocks are still occurring. The deadlock seems to happen most often during an update, but that could just be a coincidence, as I've only traced it a couple of times.
My questions are two-fold:
1) Is this actually an ITL deadlock?
2) If it is an ITL deadlock, what else can be done to avoid it?
It turns out that this was not an ITL deadlock issue at all, but rather an issue with un-indexed foreign keys. I discovered this thanks to dpbradley's answer, which clued me into the fact that it wasn't an ITL issue and prompted me to find out what the other causes of a deadlock with "no rows" might be.
The best indication of ITL pressure is from the performance views:
select event, total_waits, time_waited, average_wait
from v$system_event
where event like 'enq: TX%'
order by 2 desc;
shows TX contention waits, and
select OBJECT_NAME, SUBOBJECT_NAME, TABLESPACE_NAME,
OBJECT_TYPE, STATISTIC_NAME, VALUE
from v$segment_statistics
where statistic_name = 'ITL waits'
and value > 0
order by value desc;
shows the tables and indexes involved.
(Like all v$ views, the results are from the point in time when the instance was started.)
If this shows that you do indeed have ITL waits, then the INITRANS and PCTFREE parameters are the main knobs to turn (but INITRANS = 100 sounds pretty high to me and these do cost space).
If ITL waits are not a problem, then the application code needs to be examined.
The best option is to increase it as needed (start from default 10 and increment by 10). If you see reduction in ITL waits, you're set. Usually these related parameters are adjusted by trial and error both in Oracle and SQL Server. Adjusting these parameters in real-time won't be that much of an issue, unless the resource is extremely busy. You can use the following query to see after each increment, if the ITL waits either go away or is highly reduced:
SELECT t.OWNER, t.OBJECT_NAME, t.OBJECT_TYPE, t.STATISTIC_NAME, t.VALUE
FROM v$segment_statistics t
WHERE t.STATISTIC_NAME = 'ITL waits' AND t.VALUE > 0
ORDER BY t.value desc;
We have carried out several tunings for Oracle deadlock scenarios due to ITL waits using this method. NOTE: Make sure the index is rebuilt, if the initrans is modified for indexes. Also ensure that statistics are not stale.
For a quick check SQL Tuning Advisor can be utilized to see the full state of the query/index and statistics.