I have following query which has select query that returns data in 5sec. But when I add create materialized view command infront it takes ever for the query to create materialized view.
When you create a materialized view, you actually create a copy of the data that Oracle takes care to keep synchronized (and it makes those views somewhat like indexes). If your view operates over a big amount of data or over data from other servers, it's natural that the creating this view can take time.
From docs.oracle.com:
A materialized view is a replica of a target master from a single
point in time.
Just for "yuks", try
create table temp_tab nologging as select ...
I've seen cases where MV creation is long for some reason, probably logging.
Also, query development tools sometimes begin returning the data to the screen right away, but if you "paged" to the last row, you would find out how long it really takes to get all the data.
You should profile the select statement with explain plan and understand the table cardinality, indexes, waits states when running, ... in order to see if the query needs tuning.
Related
I am looking for performance improvement of SQL (ORACLE)
Based on few examples I tried to compare execution time between simple join between two tables v/s same query with MatrializedView.
Both execution time is almost same.
TableA Join TableB
V/s
CREATE MATERIALIZED VIEW emp_mv
BUILD IMMEDIATE
REFRESH FORCE
ON DEMAND
AS (QUERY TableA Join TableB)
both sqls are running for 7m for 1000 records.
total we have 14k Records in Table A and 50 recoreds in Table B , final output with 14K records
Is there anything which I am missing regarding performance of query execution?
Why would you expect the same query to act differently?
Materialized view's benefit might come later, when you actually start using data it contains because you already pre-processed it and prepared for future use. You could use the same query (with the join) over and over again and it'll take more or less the same time (disregard caching). But, if you store that query's data into a materialized view (and properly index it), data retrieval might/should be faster.
That's kind of "opposite" of creating an ordinary view which doesn't contain any data - it is just a stored query and it retrieves data every time you select from it, performing the same join all over again.
Materialized view contains data, just as if it were a table. It helps a lot if data is stored in tables you access over database links - that might be, and usually is, slow. But, if you create a materialized view (during night/off hours), you have data available to you much faster. It won't help much if data in tables change frequently because you'll have to refresh the MV frequently as well (usually ON COMMIT), but - if tables are really large, you have a complex query, then refreshing might also take some (a lot of?) time.
I am migrating Oracle objects to Snowflake. I have materialized view in Oracle that fetches data from multiple tables but in Snowflake a materialized view can be created on single table only. Can I use Oracle materialized view script and use it as a simple view to load into a temporary table and then use this temporary table to create a materialized view on top of it?
Can I use Oracle materialized view script and use it as a simple view to load into a temporary table and then use this temporary table to create a materialized view on top of it?
No, this won't work. A materialized view in Snowflake cannot be based on another view. But don't despair, just because you needed a materialised view in Oracle does not mean that you will need one in Snowflake ! On the contrary, it is typical in scenarios where a materialized view was needed in traditional RDBMS, that no special handling is required in Snowflake due to it's superior performance. Snowflake recommends the following considerations when deciding to use a materialized or regular view:
Create a materialized view when all of the following are true:
The query results from the view don’t change often. This almost always means that the underlying/base table for the view doesn’t change often, or at least that the subset of base table rows used in the materialized view don’t change often.
The results of the view are used often (typically significantly more often than the query results change).
The query consumes a lot of resources. Typically, this means that the query consumes a lot of processing time or credits, but it could also mean that the query consumes a lot of storage space for intermediate results.
Create a regular view when any of the following are true:
The results of the view change often.
The results are not used often (relative to the rate at which the results change).
The query is not resource intensive so it is not costly to re-run it.
When materialized views are created in Oracle, do they store indices or do they store actual table values?
I am asking this as creating index on table and using views on that table and using materialized views (created with refresh complete start with (sysdate) next (sysdate+1) with rowid as) on unindexed table gives similar performance.
Where as I would expect materialized views to be far more faster.
Update
I slightly modified the content/title. My current concern after discussion is if materialized views are actual real tables or virtual tables with some optimization.
Materialized views create a copy of the data. To all intents and purposes they are actual tables. In fact we can create a materialized view from an existing table using the PREBUILT clause. The only difference is how the data is mastered - a materialized view doesn't own its data, a table does.
As to your performance conundrum:
When you say "on unindexed table" do you literally mean one table? If so, we wouldn't expect any difference in the time to query a view, a materialized view or the actual data: they all execute a full table scan on the same volume of data.
Consider the case where views have expecting select * from <table> where <condition>.
We would a SELECT against a materialised view built on that query to execute quicker than the same SELECT against the actual table, provided the WHERE clause restricts the data to a significantly smaller subset of the original data. Simply because a full table scan over a small table (materialised view) takes less time than a full table scan over a big table. Same applies if the materialised view's projection has fewer columns than the base table.
Indexing is a different matter. Unless the query selects a very small subset of the data it's not going to be more efficient than a full table scan and a filter.
To sum up: the only universal tuning heuristic is: it takes less time to do less work. Beyond that it is impossible to generalise. We can't discuss some vague "consider the case where views have select * from <table> where <condition>." It's all about the specifics.
Fundamentally, a materialized view is just a table with an associated query to populate it.
Given static data, one would generally expect the performance of a SELECT * from the materialized view (with no WHERE clause) to be at least as fast as running the query that underlies the materialized view, regardless of indexing.
If we add a WHERE clause to a SELECT * against the mview, however, that query could perform significantly slower than running the query that underlies the mview with the same WHERE clause. That's because the tables referenced in the query underlying the mview could have indexes to support the conditions in the WHERE clause, where as the mview might not have such indexes.
Having a structure where there is a base table, then a materialized view base_mv that aggregates sending the result TO an AggregatedMergeTree table base_agg_by_id. Then we have a view over this final table. base_unique. Similarly as in this blog post](https://www.altinity.com/blog/clickhouse-continues-to-crush-time-series).
However, if I delete from base, I would expect the base_mv would trigger the mutation and then act on it, and reflected on the base_agg_by_id, but it doesn't.
Is this the expected behaviour? How to DELETE in such a schema?
I've seen here that in MVs that keep data you can act on .inner tables. However in this case, since the table is from an AggregatedMergeTree and its fields are defined as functions (e.g. AggregateFunction(argMax, String, DateTime) ), I cannot apply a deletion via a value such as ALTER base_agg_by_id DELETE WHERE field = 'myval'.
Note. For the record, we have these tables in a replicated environment using Replicated* engine: base_d, base_agg_by_id_d, base_unique_d
Mutations are not propagated to materialized views.
The reason is very simple: it not possible in common case. And even in cases when it is theoretically possible it can be very expensive operation.
For example, let's say you're deleting one record from the table which references some userid. And your materialized view contains uniqState( userid ). Data structures used for calculating uniqState don't support 'remove' operation; but even if they would - the is no way to decide if that userid should be removed or not without rereading whole data for the partition again because that userid could be seen in other records too.
So in general case, you need to refill the whole partition for your AggregatedMergeTree.
I.e. something like (daily partitioning case):
ALTER amt_table DROP PARTITION '2019-03-01';
-- use same select as in your materialized view
INSERT INTO amt_table SELECT ... WHERE date = '2019-03-01';
I have a very complex Oracle view based on other materialized views, regular views as well as some tables (I can't "fast refresh" it). Most of the time, existing records in this view are based on a date and are "stable", with new record sets having new dates.
Occasionally, I receive back-dates. I know what those are and how to deal with them if I were maintaining a table, but I would like to keep this a "view". A complete refresh would take around 30 minutes, but it only takes 25 seconds for any given date.
Can I specify that only a portion of a materialized view should be updated (i.e. the affected dates)?
Do I have to scrap the view and use a table and a procedure to populate or refresh a given date in that table?
Partition by date as in answer 3 (skaffman).
You could just do the refresh of a normal mv(table_refreshed below) and than use the exchange keyword i.e.
ALTER TABLE all_partitions
EXCHANGE PARTITION to_calculate
WITH TABLE table_refreshed
WITHOUT VALIDATION
UPDATE GLOBAL INDEXES;
After more reading and judging by the lack of answers to this question, I come come to the conclusion that it is not possible to refresh a single partition of a materialized view.
If you can give a syntax example that proves otherwise, I will happily mark your answer the accepted one.
To others who might find this questions useful in the future: you might also want to know that in Oracle 10g, refreshing a partition (or any mview) will cause Oracle to issue DELETE, followed by INSERT.
If this is giving you performance problems (like me), there is an option to use atomic_refresh => false, which will TRUNCATE, then INSERT /*+APPEND*/.
I have been able to refresh a single partition of a materialized view with partition change tracking.
It seems to require that the view is created with REFRESH FAST WITH ROWID option and DBMS_MVIEW.REFRESH is called with 'P' method.
You can partition materialized views just as you can with normal tables. Partition your mview by date, and then you can refresh only the required partition.