When the VIEW table calculate or change the value? - spring

I create a View table, like
CREATE VIEW a AS SELECT b.kcu_id, sum(b.price), FROM b GROUP BY b.kcu_id
I create a view because my table contains too many rows, like 10000 or more. And it is too costly if we must sum that many rows every time get operation is called.
I use spring data jpa to get the data from the view. What I want to ask is, when I use the getPrice method to get the sum of prices, it is calculate the sum when I use the get method or the database calculate the sum when there are a change in column performance in b table in database?
For your info, price column is rarely change in my case.

If it's just a "regular" view like you have in your example, the data will be calculated anew every time you query it. A view, after all, is just a slightly modified view of the data in the table at any given point.
You can have what they call "materialised views", which are more like a physical table that's updated from the underlying table periodically but you generally have to do that differently that with a normal "create view" command.
With PostgreSQL, the commands you're looking for are:
create materialized view
refresh materialized view
The former creates a materialised view in pretty much the same way as your create view and also populates the view with data (unless you've used the with no data clause). It also remembers the underlying query used to create the view (like any view does) so that you can update the data at some later point (which is what the refresh command above does).
By way of example, the following PostgreSQL code:
create table below (val integer);
insert into below values (42);
create materialized view above as select * from below;
insert into below values (99);
select * from below;
select * from above;
refresh materialized view above;
select * from above;
will materialise the view when the table contains only the 42 and later refresh it to include the 99 as well:
Underlying table 'below' with both data items:
| val |
|-----|
| 42 |
| 99 |
Materialised view 'above', created before the insert of 99:
| val |
|-----|
| 42 |
Materialised view 'above' after refreshing the view:
| val |
|-----|
| 42 |
| 99 |
Provided you're willing to live with the possibility that the data may be a little out of date, that's probably the best way to do it. Given your comment that the "price column is rarely change[d]", that may not be an issue.
However, I'm actually quite surprised that 10,000 rows is causing you an problem, it's not really that big a table. Hence you may want to look at other possible fixes, such as ensuring you have an index on the kcu_id column.

Related

Create a dynamic View based on changable partitions table

I have an application that reads from a view in oracle, that view reads from a big table, and the view included functions and joins with other tables.
the view takes a while to run because the table becomes bigger for each month.
I try to partitions the table by year and become faster than before.
my problem is how to create a view based on the changeable partition (by year).
Assuming that PARTITION_COL is a date column that is your partitioning key, you could do this:
create or replace
view THIS_CURRENT_YEAR as
select *
from MY_PARTITIONED_TABLE
where PARTITION_COL >= trunc(sysdate,'YYYY')
and PARTITION_COL < add_months(trunc(sysdate,'YYYY'),12)
in this way you'll get partition pruning where possible.

Adding a sequence to a large Oracle table

I have queries that take an existing large table and build tables off of them for reporting. The problem is that the source tables are 60-80MM+ records and it takes a long time to recreate. I'd like to be able to identify which records are new so I can build just add the new records to the reporting tables.
To me, the best way to identify this is to have an identity column. Is there any significant cost to creating this and adding it to the table?
Separately, is it possible to create a materialized view that takes data from one of these tables but add a sequence as part of the materialized view? That is, something like
create materialized view some_materialized_view as
select somesequence.nextval, source_table.*
from source_table?
You can add a sequence based column to your table, but as Gary suggests I wouldn't do that.
The task you are about to solve is so common that other solutions have been already implemented.
The first built-in option that comes to mind is the system change number SCN, a kind of Oracle internal clock. By default, tables are set up to record the SCN of the whole (usually 8K) block, containing usually many rows, but you can set a table to keep a record of the SCN that changed every row. Then you can track the columns that are new or change and have not been copied to your reporting tables.
CREATE TABLE t (c1 NUMBER) ROWDEPENDENCIES;
INSERT INTO t VALUES (1);
COMMIT;
SELECT c1, ora_rowscn FROM t;
Secondly, I would think of adding a date column. With 60-80 mio rows I wouldn't do this with ALTER TABLE xxx ADD (d DATE DEFAULT SYSDATE), but with rename, create as select, drop:
CREATE TABLE t AS SELECT * FROM all_objects;
RENAME t TO told;
CREATE TABLE t AS SELECT sysdate AS d, told.* FROM told;
ALTER TABLE t MODIFY d DATE DEFAULT SYSDATE;
DROP TABLE told;
Thirdly, I would read up on materialized views. I never had the chance to use this a work, but in theory, you should be able to set up a materialized view log on your 80 m table that records changes and updates dependent materialized views.
And forthly, I'd look into partitioning your large table on the (newly introduced) date column, so that identifying the new rows will become faster. That sadly depends on your version and Oracle license, though.

Are materialized views virtual tables or real tables with real data?

When materialized views are created in Oracle, do they store indices or do they store actual table values?
I am asking this as creating index on table and using views on that table and using materialized views (created with refresh complete start with (sysdate) next (sysdate+1) with rowid as) on unindexed table gives similar performance.
Where as I would expect materialized views to be far more faster.
Update
I slightly modified the content/title. My current concern after discussion is if materialized views are actual real tables or virtual tables with some optimization.
Materialized views create a copy of the data. To all intents and purposes they are actual tables. In fact we can create a materialized view from an existing table using the PREBUILT clause. The only difference is how the data is mastered - a materialized view doesn't own its data, a table does.
As to your performance conundrum:
When you say "on unindexed table" do you literally mean one table? If so, we wouldn't expect any difference in the time to query a view, a materialized view or the actual data: they all execute a full table scan on the same volume of data.
Consider the case where views have expecting select * from <table> where <condition>.
We would a SELECT against a materialised view built on that query to execute quicker than the same SELECT against the actual table, provided the WHERE clause restricts the data to a significantly smaller subset of the original data. Simply because a full table scan over a small table (materialised view) takes less time than a full table scan over a big table. Same applies if the materialised view's projection has fewer columns than the base table.
Indexing is a different matter. Unless the query selects a very small subset of the data it's not going to be more efficient than a full table scan and a filter.
To sum up: the only universal tuning heuristic is: it takes less time to do less work. Beyond that it is impossible to generalise. We can't discuss some vague "consider the case where views have select * from <table> where <condition>." It's all about the specifics.
Fundamentally, a materialized view is just a table with an associated query to populate it.
Given static data, one would generally expect the performance of a SELECT * from the materialized view (with no WHERE clause) to be at least as fast as running the query that underlies the materialized view, regardless of indexing.
If we add a WHERE clause to a SELECT * against the mview, however, that query could perform significantly slower than running the query that underlies the mview with the same WHERE clause. That's because the tables referenced in the query underlying the mview could have indexes to support the conditions in the WHERE clause, where as the mview might not have such indexes.

ClickHouse: How to delete on *AggregatingMergeTree tables from a materialized view

Having a structure where there is a base table, then a materialized view base_mv that aggregates sending the result TO an AggregatedMergeTree table base_agg_by_id. Then we have a view over this final table. base_unique. Similarly as in this blog post](https://www.altinity.com/blog/clickhouse-continues-to-crush-time-series).
However, if I delete from base, I would expect the base_mv would trigger the mutation and then act on it, and reflected on the base_agg_by_id, but it doesn't.
Is this the expected behaviour? How to DELETE in such a schema?
I've seen here that in MVs that keep data you can act on .inner tables. However in this case, since the table is from an AggregatedMergeTree and its fields are defined as functions (e.g. AggregateFunction(argMax, String, DateTime) ), I cannot apply a deletion via a value such as ALTER base_agg_by_id DELETE WHERE field = 'myval'.
Note. For the record, we have these tables in a replicated environment using Replicated* engine: base_d, base_agg_by_id_d, base_unique_d
Mutations are not propagated to materialized views.
The reason is very simple: it not possible in common case. And even in cases when it is theoretically possible it can be very expensive operation.
For example, let's say you're deleting one record from the table which references some userid. And your materialized view contains uniqState( userid ). Data structures used for calculating uniqState don't support 'remove' operation; but even if they would - the is no way to decide if that userid should be removed or not without rereading whole data for the partition again because that userid could be seen in other records too.
So in general case, you need to refill the whole partition for your AggregatedMergeTree.
I.e. something like (daily partitioning case):
ALTER amt_table DROP PARTITION '2019-03-01';
-- use same select as in your materialized view
INSERT INTO amt_table SELECT ... WHERE date = '2019-03-01';

Oracle Apex, Master detail with adjustable sequence

Oracle 11g
Apex 4.2
HI guys
I have a master / detail table, however I’m trying to apply a sequence to the detail section.
Master table contains
M_ID, DATE, PER_ID
1, 01-May-15, 1
2, 01-May-15, 2
Detail
D_ID, M_ID(FK), SEQ, ACCOUNT_ID, TASK_ID, HRS, TRAVEL_HRS
The sequence needs to increase each time a job is requested/created, however have the ability to change its order. For Each job this must start from 1.
Once the SEQ is set. I will then need to go update all current records retrospectively .
so tables would look something like
As you can see, the SEQ increases for each detail record.
The user can update the detail table via different methods. However the primary way is via a tabular form. So when a user creates a new row within the detail form this needs to increment the SEQ by one. However the user may wish to change the SEQ or order.
Hope this makes sense.

Resources