I have a table, demo_fact in Oracle 11g and it has several virtual columns defined as such:
ALTER TABLE demo_fact ADD (demo_measure_from_virtual NUMBER GENERATED ALWAYS AS
(CASE WHEN demo_category_column = 20 THEN demo_numericdata_column ELSE 0 END)
VIRTUAL VISIBLE);
Then I have a materialized view defined as
CREATE MATERIALIZED VIEW demo_agg_mv
REFRESH FORCE ON DEMAND
ENABLE QUERY REWRITE
AS
SELECT
demo_dim_one,
demo_dim_two,
SUM(demo_measure_from_virtual) demo_measure_from_virtual
FROM demo_fact
GROUP BY demo_dim_one, demo_dim_two
Now I want Query Rewrite to kick in on the following query:
SELECT demo_dim_one, SUM(demo_measure_from_virtual)
FROM demo_fact
GROUP BY demo_dim_one
but it doesn't. I ran EXPLAIN_REWRITE on and here is the output:
QSM-01150: query did not rewrite
QSM-01102: materialized view, DEMO_AGG_MV, requires join back to table,
DEMO_FACT, on column, DEMO_MEASURE_FROM_VIRTUAL
QSM-01082: Joining materialized view, DEMO_AGG_MV, with table, DEMO_FACT,
not possible
QSM-01102: materialized view, DEMO_AGG_MV, requires join back to table,
DEMO_FACT, on column, DEMO_NUMERICDATA_COLUMN
Backstory: I'm doing this with 70M rows and 50 virtual columns (all of them have the same structure, the simple case statement above, but with a different comparison column and a different result column)
This problem seems to only manifest when the fact table has virtual columns, but changing them to non-virtual would consume too much diskspace. Why isn't Oracle rewriting the query? What can I do to fix it?
I don't know how helpful this is for you but Oracle requires all columns that the materialzied view grouped on to be included in the statement to be rewritten. (edit at least in conjunction with virtual columns. This is probably "not by design"...)
If you try to explain_rewrite on
select
demo_dim_one,
sum(s)
from (
select
demo_dim_one,
sum(demo_measure_from_virtual) s
from
demo_fact
group by
demo_dim_one,
demo_dim_two
)
group by demo_dim_one
it should tell you that it has rewritten the query.
This can be demonstrated like so:
A table to on which the virtual column will be defined:
create table tq84_virt_col (
a varchar2(2),
b varchar2(2),
c number,
d number
);
insert into tq84_virt_col values ('A', 'X', 1, 1);
insert into tq84_virt_col values ('A', 'X', 2, 1);
insert into tq84_virt_col values ('A', 'Y', 3, 0);
insert into tq84_virt_col values ('A', 'Y', 4, 1);
insert into tq84_virt_col values ('B', 'Y', 11, 1);
insert into tq84_virt_col values ('B', 'X', 12, 0);
insert into tq84_virt_col values ('B', 'X', 13, 1);
The definition of the virtual column:
alter table tq84_virt_col add (
virt_col number generated always as (
case when d = 1 then c else 0 end
)
virtual visible
);
The materialized view. Note: it groups on columns a and b:
create materialized view tq84_mat_view
refresh force on demand
enable query rewrite
as
select
a, b,
sum(virt_col) sum_virt_col
from
tq84_virt_col
group by
a,b
The materialized view will not be used, as you have observed:
begin
dbms_mview.explain_rewrite(
'select a, sum(virt_col) from tq84_virt_col group by a'
);
end;
/
select message
from rewrite_table;
QSM-01150: query did not rewrite
QSM-01102: materialized view, TQ84_MAT_VIEW, requires join back to table, TQ84_VIRT_COL, on column, VIRT_COL
QSM-01082: Joining materialized view, TQ84_MAT_VIEW, with table, TQ84_VIRT_COL, not possible
QSM-01102: materialized view, TQ84_MAT_VIEW, requires join back to table, TQ84_VIRT_COL, on column, C
Now, both columns a and b are selected and grouped on (with an outer query to ensure the same result set):
truncate table rewrite_table;
begin
dbms_mview.explain_rewrite(
'select a, sum(s) from (select a, sum(virt_col) s from tq84_virt_col group by a, b) group by a'
);
end;
/
select message
from rewrite_table;
QSM-01151: query was rewritten
QSM-01209: query rewritten with materialized view, TQ84_MAT_VIEW, using text match algorithm
QSM-01219: no suitable materialized view found to rewrite this query
Related
Could you please tell me how to compare differences between table and my select query and insert those results in separate table? My plan is to create one base table (name RESULT) by using select statement and populate it with current result set. Then next day I would like to create procedure which will going to compare same select with RESULT table, and insert differences into another table called DIFFERENCES.
Any ideas?
Thanks!
You can create the RESULT_TABLE using CTAS as follows:
CREATE TABLE RESULT_TABLE
AS SELECT ... -- YOUR QUERY
Then you can use the following procedure which calculates the difference between your query and data from RESULT_TABLE:
CREATE OR REPLACE PROCEDURE FIND_DIFF
AS
BEGIN
INSERT INTO DIFFERENCES
--data present in the query but not in RESULT_TABLE
(SELECT ... -- YOUR QUERY
MINUS
SELECT * FROM RESULT_TABLE)
UNION
--data present in the RESULT_TABLE but not in the query
(SELECT * FROM RESULT_TABLE
MINUS
SELECT ... );-- YOUR QUERY
END;
/
I have used the UNION and the difference between both of them in a different order using MINUS to insert the deleted data also in the DIFFERENCES table. If this is not the requirement then remove the query after/before the UNION according to your requirement.
-- Create a table with results from the query, and ID as primary key
create table result_t as
select id, col_1, col_2, col_3
from <some-query>;
-- Create a table with new rows, deleted rows or updated rows
create table differences_t as
select id
-- Old values
,b.col_1 as old_col_1
,b.col_2 as old_col_2
,b.col_3 as old_col_3
-- New values
,a.col_1 as new_col_1
,a.col_2 as new_col_2
,a.col_3 as new_col_3
-- Execute the query once again
from <some-query> a
-- Outer join to detect also detect new/deleted rows
full join result_t b using(id)
-- Null aware comparison
where decode(a.col_1, b.col_1, 1, 0) = 0
or decode(a.col_2, b.col_2, 1, 0) = 0
or decode(a.col_3, b.col_3, 1, 0) = 0;
I have two tables, one with values (FACT table), that I'm trying to merge one column into a key/value table (FACT_ATTRIBUTE table). The key is a number, and the value is a varchar2. The FACT_ATTRIBUTE table already has some entries in it. I am trying to write a package procedure that merges in new entries. We are trying to do away with triggers and sequences. I have a bit of SQL that selects entries not already in the table, but I am struggling with how to create the index key value. The key should just be one greater than the current max.
For example, say the FACT_ATTRIBUTE table already has entries up to key 5. A new entry "value 6" comes in. The insert for the key/value would be (6, "value 6") into FACT_ATTRIBUTE.
Here is where I am at:
INSERT INTO FACT_ATTRIBUTE (
KEY, VALUE
)
VALUES(
(SELECT (MAX(KEY) + 1) FROM FACT_ATTRIBUTE),
(SELECT DISTINCT ENTRY
FROM FACT fact_table
WHERE NOT EXISTS (
SELECT 1 FROM FACT_ATTRIBUTE fa
WHERE fa.VALUE = fact_table.ENTRY
))
);
The problem is obvious, the inner select grabs multiple elements, while the insert is single value. One constraint is that it cannot crash on null (if there are no new values). Is there a good way to accomplish this? My SQL skills are not the sharpest. Thanks in advance for any help!
One approach that might solve your problem is converting your insert statement from values to select. Try this -
INSERT INTO FACT_ATTRIBUTE (
KEY, VALUE
)
SELECT max(key) over (partition by value) + 1 as key_max, f.entry
FROM (
SELECT f.entry, fa.value
FROM fact_table f
LEFT OUTER JOIN fact_attribute fa
ON f.entry = fa.value)
WHERE fa.value is null;
It might need a little syntax polish cause I haven't had a server to check this on, but the gist seems right to me - using a left outer join to query all of the entries not existing in FACT_ATTRIBUTE and attaching the current maximum value in the table + 1.
This is the final solution that got me to what I needed.
DECLARE
max_id NUMBER;
BEGIN
--If no value, start with 1
SELECT NVL(max(fa.FACT_ATTRIBUTE), 1)
INTO max_id
FROM FACT_ATTRIBUTE fa;
INSERT INTO FACT_ATTRIBUTE (KEY, VALUE)
SELECT max_id + ROWNUM, DATA
FROM (
SELECT DISTINCT f.DATA
FROM FACT f
WHERE NOT EXISTS (
SELECT 1
FROM FACT_ATTRIBUTE fa
WHERE fa.VALUE = f.DATA
)
AND f.DATA IS NOT NULL
);
END;
When I updated a view which was created using a base table, the updation affected the base table as well. How is that possible? If view is considered as just a 'window' through which we can see a set of data of the base table then how can the base table change when I try to change the data inside a view.
You can make changes to the state of underlying table using the view as long as the you are targeting the change in single table.
View is a security layer on top of table object and allows most of the DML operation as long as you do not violet the base rule.
Example:
CREATE TABLE T1
(ID INT IDENTITY(1,1), [Value] NVARCHAR(50))
CREATE TABLE T2
(ID INT IDENTITY(1,1), [Value] NVARCHAR(50))
--Dummy Insert
INSERT INTO T1 VALUES ('TestT1')
INSERT INTO T2 VALUES ('TestT2')
--Create View
CREATE VIEW V1
AS
SELECT T1.ID AS T1ID, T2.ID AS T2ID, T1.Value AS T1Value, T2.Value AS T2Value FROM T1 INNER JOIN T2
ON T2.ID = T1.ID
--Check the result
SELECT * FROM V1
--Insert is possible via view as long as it affects only one table
INSERT INTO V1 (T1Value) VALUES
('TestT1_T1')
INSERT INTO V1 (T2Value) VALUES
('TestT2_T2')
--Change is possible only if target is only one table
UPDATE V1
SET T1Value = 'Changed'--**
WHERE T2ID = 1
--This is not allowed
INSERT INTO V1 (T1Value, T2Value) VALUES
('TestT1_T1','TestT2_T2')
--Msg 4405, Level 16, State 1, Line 1
--View or function 'V1' is not updatable because the modification affects multiple base tables.
--Check T1 and T2 with each statement to see how it gets affected
--
In some databases it's possible to update the source table(s) for a view if there is a one-to-one relationship between the rows in the view and the rows in the underlying table, that is, you cant have derived columns, aggregate functions or a distinct clause in your view for example.
In Oracle, even if a view is not inherently updatable, updates may be allowed if an INSTEAD OF DML trigger is defined.
If you use mysql, you can read a detailed description about this feature Updatable and insertable views.
" If view is considered as just a 'window' through which we can see a set of data of the base table "
- Where did you get this definition?
What oracle says about views:
A view is a logical representation of another table or combination of
tables. A view derives its data from the tables on which it is based.
These tables are called base tables. Base tables might in turn be
actual tables or might be views themselves. All operations performed
on a view actually affect the base table of the view. You can use
views in almost the same way as tables. You can query, update, insert
into, and delete from views, just as you can standard tables.
Such a view into which you can update or insert are fondly named as "Updatable and Insertable Views". Oracle documentation about them is here.
Also, this is how the purpose of an "insert" statement is defined by Oracle:
Use the INSERT statement to add rows to a table, the base table of a
view, a partition of a partitioned table or a subpartition of a
composite-partitioned table, or an object table or the base table of
an object view.
Yes we can achieve the DML Operation in Views like belows:
Create or replace view emp_dept_join as Select d.department_id,
d.department_name, e.first_name, e.last_name from employees
e, departments d where e.department_id = d.department_id;
SQL>CREATE OR REPLACE TRIGGER insert_emp_dept
INSTEAD OF INSERT ON emp_dept_join DECLARE v_department_id departments.department_id%TYPE;
BEGIN
BEGIN
SELECT department_id INTO v_department_id
FROM departments
WHERE department_id = :new.department_id;
EXCEPTION
WHEN NO_DATA_FOUND THEN
INSERT INTO departments (department_id, department_name)
VALUES (dept_sequence.nextval, :new.department_name)
RETURNING ID INTO v_department_id;
END;
INSERT INTO employees (employee_id, first_name, last_name, department_id)
VALUES(emp_sequence.nextval, :new.first_name, :new.last_name, v_department_id);
END insert_emp_dept;
/
if the viwe is defined through a simple query involving single base relation and either containing primary key or candidate key, so there will be change in base relation if changing the view. ( however there is restriction)
And updates are not allowed through view if there is multiple base relations or grouping operations.
In tableA I have an ID field that I want to be a value from tableB when inserting into tableA. It should however cycle through all values in tableB for each record entered. So if I have id's in tableB of 0, 1, 2, 3 then on first insert to tableA it would assign tableB value 0, on 2nd insert it should use tableB 1, then 2, then 3, then back to 0, and just loop in a cycle like this.
I'm guessing I'd need a table to track which one should be the next value and then in my trigger use that value then get the next value to use for the next insert trigger. Would there be any other ways that someone can think about doing this outside of a stored proc?
From what I have understand is you trying to load the data from TableB into TableA looping through IDs from TableB.
If the data in TableB is historical data that you want to load it on to TableA, then you need stored procedure with cursor inside so that you can do initial delta load from B to A. you dont need another table here in that case since you are using Cursor in Stored Procedure.
If you want TableA to be populated when there is an insert or update on TableB then you can use trigger on TableB.
Let say I have table my table has values(which they are varchar):
values
a
o
g
t
And I have insert a new value called V
values
V
a
o
g
t
Is there a way or query that can specify what is the last value was insert in the column ? the desired query : select * from dual where rown_num = count(*) -- just an example and the result will be V
Rows in a table have no inherent order. rownum is a pseudocolumn that's part of the select so it isn't useful here. There is no way to tell where in the storage a new row will physically be placed, so you can't rely on rowid, for example.
The only way to do this reliably is to have a timestamp column (maybe set by a trigger so you don't have to worry about it). That would let you order the rows by timestamp and find the row with the highest (most recent) timestamp.
You are still restricted by the precision of the timestamp, as I discovered creating a SQL Fiddle demo; without forcing a small gap between the inserts the timestamps were all the same, but then it only seems to support `timestamp(3). That probably won't be a significant issue in the real world, unless you're doing bulk inserts, but then the last row inserted is still a bit of an arbitrary concept.
As quite correctly pointed out in comments, if the actual time doesn't need to be know, a numeric field populated by a sequence would be more reliable and performant; another SQL Fiddle demo here, and this is the gist:
create table t42(data varchar2(10), id number);
create sequence seq_t42;
create trigger bi_t42
before insert on t42
for each row
begin
:new.id := seq_t42.nextval;
end;
/
insert into t42(data) values ('a');
insert into t42(data) values ('o');
insert into t42(data) values ('g');
insert into t42(data) values ('t');
insert into t42(data) values ('V');
select data from (
select data, row_number() over (order by id desc) as rn
from t42
)
where rn = 1;