So I have a process to design where there are few base tables on which I have to create a view and from that view I will extract and derive data and insert into an another table.
we would use this last table for extracting reports
This process will run daily, since the data on the base tables have updates have everyday.
he whole purpose of this process is to make things "faster" for reporting,
So my concern here is with using the view. I am wondering if a materialized view would be better to use here in terms of performance instead the view or view itself would be the better.
Has anyone had to deal with such situation and found an approach that performs well and works efficiently??
Any leads on this would be highly appreciated
Related
My case is that a third party prepares a table in our schema domain on which we run different spring batch jobs that look for mutations (diff between the given third party table and our own tables). This table will contain about 200k records on average.
My question is simply: does generating a material view up front provide any benefits vs running the query at runtime?
Since the third party table will be populated on our command (basically it's a db boolean field that is set to 1, after which a scheduler picks it up to populate the table. Don't ask me why it's done this way), the query needs to run anyway.
Obviously from an application point of view, it seems more performant to query a flat material view. However, I'm not sure if there is any real performance benefit, since the material view needs to be built on db level.
Thanks.
The benefit of a materialized view here is if you are running the multiple times (more so if the query is expensive and / or there is a big drop in cardinality).
If you are only hitting the query once then you there isn't going to be a huge amount in it. You are running the same query either way and you have the overhead of inserting into the materialized view but you also have the benefit that you can tune this a lot easier than you could querying via JPA and could apply things like compression so less data is transferred back to the application but for 200k rows any difference is likely to be small.
All in all, unless you are running the same query multiple times then I wouldn't bother.
Update
One other thing to consider is coupling. Referencing a materialized view directly in JPA would allow you to update any logic without updating the application but the flip side of this is that logic is hidden outside the application which can make debugging a pain.
Also if you are just referencing a materialized view directly and not using any query rewrite or rollup features then am simple table created via CTAS would actually be better as you still have the precomputed data without the (small) overhead of maintaining the materialized view.
I'm trying to make faster dashboards in Tableau by creating views of my calculations directly in BigQuery.
Based on my understating if the gcloud documentation here, the view will re-execute the query once it is accessed, so it kinda defeats my goal.*
*My goal is to eliminate calculations on the fly, be it in Tableau or BigQuery.
Is it possible to "save" these views, by way of scheduled scripts or workflows?
Thanks,
A view is best thought of as a way to reformat a table to make it look more convenient to further queries. The query still has to run on BigQuery so the benefits will be that the view may look simpler to Tableau than the raw table (particularly convenient if the view uses some complex SQL to create some of its columns). But it won't save calculation time.
But, if your view is doing some complex consolidation of a larger table then it might be worth saving the results as a new table instead of creating a view. This is OK if your underlying table doesn't change frequently (rule of thumb if you use the results every day and the table changes weekly, it is probably worthwhile and certainly so if the changes are monthly). Then Tableau will be querying pre-consolidated results rather than the much larger raw table. BigQuery storage and processing is cheap so this is often a reasonable solution.
Another alternative is to use a Tableau extract to bring the data into your local drive or server. This is only practical if the table is small enough to fit locally and will only work really well for speed if it fits into local memory (which can be a lot more than you might think). But extracts, at least on Tableau server, can be set to refresh on a schedule, making much faster user interaction and absolving you of having to remember to manually update the consolidated table.
I'm learning about Oracle Views and I got the concept of views but little confused about performance.
I was watching video and there I listen that oracle view can increase the performance. Suppose I have created view like below.
CREATE VIEW SALES_MAN
AS
SELECT * FROM EMP WHERE JOB='SALESMAN';
Ok now I have executed query to get SALES_MAN detail.
SELECT * FROM SALES_MAN
Ok now confusion start.
I listened in video that once the above query SELECT * FROM SALES_MAN will be executed the DATA/RECORD will be placed into cache memory after hitting the oracle DB. and if I will execute same query( IN Current Session/Login ) Oracle Engine will not hit to Database and will give you record from CACHED-MEMORY is it right?
But I have read on many websites that View add nothing to SQL performance more
Another Reference that says view not help in performance. Here
So Views increase performance too or not?
A view is just a kind of virtual table defined by a query. It has no proper data and performance will depend on the underlying table(s) and the query definition.
But you also have Materialized View wich stores the results from the view query definition. Data is synchronized automatically, and you can add indexes to the view. In this case, you can achieve better performances. The cost to pay is
more space (the view contains data dupplicated),
no access to the up to date data from the underlying tables
You (and perhaps the creator of the un-cited video) are confusing two different concepts. The reason the data that was in the cache was used on the second execution was NOT because it was the second execution of the view. As long as data remains in the cache, it is available for ANY query that needs it. The very fact that the first execution of the view had to get the data from disk should be a strong clue. And what if your second use of the view wasn't until hours or days later when the data was no longer in the cache? The answer remains. A view does NOT improve performance.
I am new to oracle and i wanted to know if materialized views can be created on top of views. I also need a column in the mview that has a complex calculation for which I am thinking of writing a function.
There are materialised views, and then again there are materialised views.
By which I mean that the capabilities of the materialised view can vary greatly with respect to query rewrite and refreshability according to the exact nature of the tables (or views) that it references, and according to their properties.
So although you might be able to create an MV that references a view, whether it is useful to you or not is a matter that depends on a great many other issues.
I find that the documentation can sometimes be unclear, particularly in edge-cases, and the best approach is generally to create the MV and then to test its capabilities using Oracle built-in procedures -- here's a link to DBMS_MView for 10.2 http://docs.oracle.com/cd/B19306_01/appdev.102/b14258/d_mview.htm#CEGGEHHC
IF a create thousands of view, Does it hamper the database performance. I mean is there any problem with creating thousands of view in oracle. Please explain as I am new in this area...I am using oracle...
The simple existence of these views shouldn't harm performance at all. However, once those views start being used it's possible that there will be some negative performance impact. Oracle tries to "remember" the plan for each statement that it sees, but it compares statements by comparing the source code (the SQL). Your thousands of views will all be named differently since you can't have multiple views with the same name, and thus each time one of them is used Oracle is going to have to do a full parse of the SQL, even if it's something as basic as
SELECT * FROM VIEW_1;
and
SELECT * FROM VIEW_2;
All these re-parses will certainly take some time.
What's different about each of these views? I think it might be a good idea to step back and consider other possibilities. Questions I'd ask include
What is to be accomplished here?
Why are thousands of different views needed?
Is there some other way to accomplish what needs to be done without creating all these views?
I don't know the answers to 1 and 2, but I'm reasonably sure that the answer to #3 is "Yes".
Good luck.
Oracle views are an encapsulation of a
complex query and must be used with
care. Here are the key facts to
remember:
Views are not intended to improve SQL
performance. When you need to
encapsulate SQL, you should place it
inside a stored procedure rather than
use a view. Views hide the complexity
of the underlying query, making it
easier for inexperienced programmers
and end users to formulate queries.
Views can be used to tune queries with
hints, provided that the view is
always used in the proper context.
source: Guard against performance issues when using Oracle hints and views
View is as heavy to run as the select that creates it but Oracle loads balance and single select can't harm the DB. If you have thousands concurrent selects going then you might have a problem. The amount of views is not important but how heavy they are and how much you use them.
You would actually need to show the views code and tell what you are actually trying to do.
View should not affect performance, if optimizer is smart enough. I remember cases with other DB engines when Views do harm performance. As in many performance cases - I suggest to measure your particular case.