How to find out if changes made at known time in underlying table are already available in fast refreshing materialized view?
So far:
all_mview_analysis.LAST_REFRESH_DATE - this seems to be the start time of the refresh not end time? Is this value written when refresh is started or when it is finished?
all_mview_analysis.INCREFRESHTIM - docs say this is "Approximate refresh time, in seconds, for fast refresh". Approximate is not enough.
In other words: I have changed data at time x. I need to query if those changes made at time x are already included in materialized view or not. I want to wait as much as needed but as little as possible.
Database: Oracle 11g.
Related
In our oracle query, I used materialized view to improve the performance and saw a drastic change in performance time of query, response time is in .17 seconds now rather 30 seconds so its working in case we are working on historical data set and need to have one time refresh of materialized view but now requirement is if base table of materialized view would be updating frequently in that case it would be working as the refresh activity of materialized view would be consuming time as well, currently the base table has around 170000 records but in future definitely going to increase so is there any solution or option to resolve this. As i have tried to apply Indexes as well on the table but not able to achieve response time within .17 sec or around the last response time based on index creation was 18 sec.
My manager at work asks me to create 50 Materialized views and set the refresh every 15 minutes, my question is: will refreshing all 50 MVs at the same time every 15 minutes affect DB performance?
thank you
Refreshing 50 at the same time is likely to stress some part of your system, generally you only make materialized views because the work required to execute the query is longer than a user would otherwise be happy waiting.
If you are using fast refreshable materialized views (which you should be striving for if you want to refresh that frequently) the amount of work required is far far less - you only need to work with the data that has changed since the last refresh. Oracle will calculate everything that requires changing and change it in a small number of SQL statements. For a 15 minute change window, there's probably not a lot of actual changes that get made - and these changes will be processed using batch operations much more efficiently.
However... even when you have made the refresh process as fast as possible, having 50 things all working at once (even for a short period) is still a lot. If they all need to do CPU work then that's 50 busy CPUs that would need to be scheduled by the OS, and then it also needs to schedule in all the remaining user sessions that are trying to do work (Oracle CPU licencing for this would be super expensive). This is why you would typically limit the number of jobs that can run at any one time using the job_queue_processes parameter (although recent versions will also help out by having Resource Manager restrict number of jobs at any one time) - set it to a sensible number depending on how much work you can physically be processing in the background at once. If you use fast refreshable materialized views then refreshing 50 should take no time even if Oracle processes them one at a time.
Another benefit of having fast refreshable materialized views (if you have upgraded to at least 12.2) is that you can have Real Time Materialized Views, this means even if it hasn't been refreshed, your users can still see completely up to date data all the time without having to do the work of the query - very clever stuff! This could allow you to have less frequent refreshes for some of your materialized views but have the data completely in sync at all times.
Looking into getting near-realtime analytics data from bigquery, and considering costs vs. accuracy, it seems like using materialized view might be a great win.
Considering "near real time" will change to a minimum of 1 minute refresh_interval_minutes, my main concern is that while from documentation it seems that the query will be only on the delta data, the billing will be on the "standard" minimum of 10MB per table.
As I see it, if this minimum is being forced, it dismiss using materialized view as a valid solution for near-realtime.
I would have used "standard" caching over the queries, but caching does not work when querying table with "buffered data", as far as I understand it.
Please advise,
Shushu
The following are key characteristics of BigQuery Materialized Views:
Zero maintenance: A materialized view is recomputed in background once the base table has changed. All incremental data changes from the base tables are automatically added to the materialized views. No user inputs are required.
In other words it means that incremental changes like streaming data into, is automatically added to the materialized view. That means you don't need to set 1 minute refresh period. The maximum refresh frequency cap is 7 days. You could set it to 7 days, or even disable manual refresh as you don't have deletes.
We have to create a materialised view in our database from a remote database which is currently in production. The view has about 5 crore records and is taking a long time. In between , the connection has snapped once and not even a single record persisted in our database. Since the remote database is production server, we get a very limited window to create the view.
My question is, can we have something like auto commit /auto start from where left at last time while the view is created so that we don't have to do the entire operation in one go ?
We are tying to make the query such that records can be fetched in smaller numbers as a alternate. But the data is read only for us and doesn't really have a where clause at this point which we can use.
Due to sensitive nature of data, I cannot post the view structure or the query.
No, you can not commit during the process of creating the view. But why don't you import the data to a table instead of a view? This would provide the possibility to commit in between. Furthermore this might provide the possibility, to load only the delta of changes maybe on daily basis - this would reduce the required time dramatically.
I am creating an oracle view basically as a read-only summary of data from other tables. It is not necessary to update the view every time if there is any updates to associated tables otherwise the workload will be huge and a waste. It is better to refresh the view every 5 minutes and then the front end can receive the latest data that will be all.
However, how to setup the oracle view refreshing interval? Do we have to settle this constraints in PL/SQL?