oracle query with pivot and row totals - oracle

Here is my entire requirement along with create and insert commands.
create table sample
( schedule_id varchar2(256),
schedule_details varchar2(256),
pod_name varchar2(256),
milestone varchar2(256),
metric_column varchar2(256),
metric_value varchar2(4000));
insert into sample values ('23456','details','ABCD','M1','status','DONE');
insert into sample values ('123456','details','ABCD','M1','progress','GREEN');
insert into sample values ('123456','details','ABCD','M1','last_reported','2016-03-18 04:18:56');
insert into sample values ('123456','det','ABCD','M2','status','DONE');
insert into sample values ('123456','det','ABCD','M2','progress','RED');
insert into sample values ('123456','det','ABCD','M2','last_reported','2016-03-19 04:18:56');
insert into sample values ('123456','det','ABCD','M3','status','SKIP');
insert into sample values ('123456','det','ABCD','M3','progress','YELLOW');
insert into sample values ('123456','det','ABCD','M3','last_reported','2016-03-20 04:18:56');
insert into sample values ('123456','det''EFGH','M1','status','DONE');
insert into sample values ('123456','det','EFGH','M1','progress','GREEN');
insert into sample values ('123456','det','EFGH','M1','last_reported','2016-03-10 04:18:56');
insert into sample values ('123456','det','EFGH','M2','status','DONE');
insert into sample values ('123456','det','EFGH','M2','progress','RED');
insert into sample values ('123456','det','EFGH','M2','last_reported','2016-03-12 04:18:56');
insert into sample values ('123456','det','EFGH','M3','status','SKIP');
insert into sample values ('123456','det','EFGH','M3','progress','YELLOW');
insert into sample values ('123456','det','EFGH','M3','last_reported','2016-03-11 04:18:56');
insert into sample values ('7890','det','ABCD','M1','status','DONE');
insert into sample values ('7890','det','ABCD','M1','progress','GREEN');
insert into sample values ('7890','det','ABCD','M1','last_reported','2016-04-18 04:18:56');
insert into sample values ('7890','det','ABCD','M2','status','DONE');
insert into sample values ('7890','det','ABCD','M2','progress','RED');
insert into sample values ('7890','det','ABCD','M2','last_reported','2016-04-19 04:18:56');
insert into sample values ('7890','det','ABCD','M3','status','FAILED');
insert into sample values ('7890','det','ABCD','M3','progress','GREEN');
insert into sample values ('7890','det','ABCD','M3','last_reported','2016-04-20 04:18:56');
insert into sample values ('7890','det','PQRS','M1','status','DONE');
insert into sample values ('7890','det','PQRS','M1','progress','GREEN');
insert into sample values ('7890','det','PQRS','M1','last_reported','2016-04-10 04:18:56');
insert into sample values ('7890','det','PQRS','M2','status','DONE');
insert into sample values ('7890','det','PQRS','M2','progress','RED');
insert into sample values ('7890','det','PQRS','M2','last_reported','2016-04-11 04:18:56');
insert into sample values ('7890','det','PQRS','M3','status','SKIP');
insert into sample values ('7890','det','PQRS','M3','progress','GREEN');
insert into sample values ('7890','det','PQRS','M3','last_reported','2016-04-12 04:18:56');
my output is
SCHEDULE_ID | STATUS_DONE | STATUS_SKIP | STATUS_FAILED | STATUS_TOTAL | PROGRESS_RED | PROGRESS_GREEN | PROGRESS_YELLOW | PROGRESS_TOTAL
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
123456 | 1 | 1 | 0 | 2 | 1 | 0 | 1 | 2
7890 | 0 | 2 | 0 | 2 | 0 | 1 | 1 | 2
The query needs to fetch the distinct number of pod_name for each schedule_id. For each pod it needs to take the metric_column->status and metric_column->progress of the milestone which has latest metric_column->last_reported. So the value of STATUS_DONE,is the number of POD_NAME that have the latest milestone (based on last_reported) as status DONE.
For eg for schedule id 123456, the pod ABCD, the status and progress of milestone M3 needs to be taken as M3 has the latest last_reported compared to M1 and M2.
I have the query with distinct pod name without taking into consideration the latest milestone.
select *
from (SELECT SCHEDULE_ID,SCHEDULE_DETAILS,POD_NAME,
CASE WHEN GROUPING(value) = 0
THEN value
ELSE CASE WHEN METRIC_COLUMN='status'
THEN 'STATUS_TOTAL'
ELSE 'PROGRESS_TOTAL'
END
END value
FROM sample
WHERE metric_column IN ('status','progress' )
GROUP BY SCHEDULE_ID,SCHEDULE_DETAILS,POD_NAME, METRIC_COLUMN, ROLLUP(value))
PIVOT (COUNT( DISTINCT POD_NAME) FOR VALUE in ('DONE' AS "DONE",'SKIP' AS "SKIP",'FAILED' AS "FAILED",'STATUS_TOTAL' AS "STATUS_TOTAL",'GREEN' AS GREEN,'RED' AS "RED",'YELLOW' AS "YELLOW", 'PROGRESS_TOTAL' AS "PROGRESS_TOTAL"));

Related

View Performance

I have requirement of performing some calculation on a column of a table with large date set ( 300 GB). and return that value.
Basically I need to create a View on that table. Table has data of 21 years and It is partitioned on date column (Daily). We can not put date condition on View's query and User will put filter on runtime while execution of the view.
For example:
Create view v_view as
select * from table;
Noe I want to query View like
Select * v_view where ts_date between '1-Jan-19' and '1-Jan-20'
How Internally Oracle execute above statement? Will it execute view query first and then put date filter on that?
If so will there not be performance issue ? and how to resolve this?
oracle first generates the view and then applies the filter. you can create a function that input may inserted by user. the function results a create query and if yo run the query then the view will be created. just run:
create or replace function fnc_x(where_condition in varchar2)
return varchar2
as
begin
return ' CREATE OR REPLACE VIEW sup_orders AS
SELECT suppliers.supplier_id, orders.quantity, orders.price
FROM suppliers
INNER JOIN orders
ON suppliers.supplier_id = orders.supplier_id
'||where_condition||' ';
end fnc_x;
this function should be run. input the function is a string like this:
''WHERE suppliers.supplier_name = Microsoft''
then you should run a block like this to run the function's result:
cl scr
set SERVEROUTPUT ON
declare
szSql varchar2(3000);
crte_vw varchar2(3000);
begin
szSql := 'select fnc_x(''WHERE suppliers.supplier_name = Microsoft'') from dual';
dbms_output.put_line(szSql);
execute immediate szSql into crte_vw; -- generate 'create view' command that is depended on user's where_condition
dbms_output.put_line(crte_vw);
execute immediate crte_vw ; -- create the view
end;
In this manner, you just need received where_condition from user.
Oracle can "push" the predicates inside simple views and can then use those predicates to enable partition pruning for optimal performance. You almost never need to worry about what Oracle will run first - it will figure out the optimal order for you. Oracle does not need to mindlessly build the first step of a query, and then send all of the results to the second step. The below sample schema and queries demonstrate how only the minimal amount of partitions are used when a view on a partitioned table is queried.
--drop table table1;
--Create a daily-partitioned table.
create table table1(id number, ts_date date)
partition by range(ts_date)
interval (numtodsinterval(1, 'day'))
(
partition p1 values less than (date '2000-01-01')
);
--Insert 1000 values, each in a separate day and partition.
insert into table1
select level, date '2000-01-01' + level
from dual
connect by level <= 1000;
--Create a simple view on the partitioned table.
create or replace view v_view as select * from table1;
The following explain plan shows "Pstart" and "Pstop" set to 3 and 4, which means that only 2 of the many partitions are used for this query.
--Generate an explain plan for a simple query on the view.
explain plan for
select * from v_view where ts_date between date '2000-01-02' and date '2000-01-03';
--Show the explain plan.
select * from table(dbms_xplan.display(format => 'basic +partition'));
Plan hash value: 434062308
-----------------------------------------------------------
| Id | Operation | Name | Pstart| Pstop |
-----------------------------------------------------------
| 0 | SELECT STATEMENT | | | |
| 1 | PARTITION RANGE ITERATOR| | 3 | 4 |
| 2 | TABLE ACCESS FULL | TABLE1 | 3 | 4 |
-----------------------------------------------------------
However, partition pruning and predicate pushing do not always work when we may think they should. One thing we can do to help the optimizer is to use date literals instead of strings that look like dates. For example, replace
'1-Jan-19' with date '2019-01-01'. When we use ANSI date literals, there is no ambiguity and Oracle is more likely to use partition pruning.

How to call Oracle stored procedure using muliple param values from table

I have requirement to call stored procedure for which input parameters are multiples values are coming from table.
For example:
sp(p1,p2,p2,...);
Table looks like this:
Col1 | Col2
-----+-------
p1 | val1
p2 | Val2
p3 | val3
How we can achieve this in PL/SQL block?

How to create KSQL table from a topic with composite key?

I have some topic data with the fields stringA stringB and I am just trying to use that as the key when creating a KSQL table from a topic.
Here's an example. First, I'll create and populate a test stream
ksql> CREATE STREAM TEST (STRINGA VARCHAR,
STRINGB VARCHAR,
COL3 INT)
WITH (KAFKA_TOPIC='TEST',
PARTITIONS=1,
VALUE_FORMAT='JSON');
Message
----------------
Stream created
----------------
ksql> INSERT INTO TEST (STRINGA, STRINGB, COL3) VALUES ('A','B',1);
ksql> INSERT INTO TEST (STRINGA, STRINGB, COL3) VALUES ('A','B',2);
ksql> INSERT INTO TEST (STRINGA, STRINGB, COL3) VALUES ('C','D',3);
ksql>
ksql> SET 'auto.offset.reset' = 'earliest';
Successfully changed local property 'auto.offset.reset' to 'earliest'. Use the UNSET command to revert your change.
ksql> SELECT * FROM TEST EMIT CHANGES LIMIT 3;
+--------------+--------+---------+----------+------+
|ROWTIME |ROWKEY |STRINGA |STRINGB |COL3 |
+--------------+--------+---------+----------+------+
|1578569329184 |null |A |B |1 |
|1578569331653 |null |A |B |2 |
|1578569339177 |null |C |D |3 |
Note that ROWKEY is null.
Now I'll create a new stream, populated from the first, and create the composite column set it as the key. I'm also including the original fields themselves, but this is optional if you don't need them:
ksql> CREATE STREAM TEST_REKEY AS
SELECT STRINGA + STRINGB AS MY_COMPOSITE_KEY,
STRINGA,
STRINGB,
COL3
FROM TEST
PARTITION BY MY_COMPOSITE_KEY ;
Message
------------------------------------------------------------------------------------------
Stream TEST_REKEY created and running. Created by query with query ID: CSAS_TEST_REKEY_9
------------------------------------------------------------------------------------------
Now you have a stream of data with the key set to your composite key:
ksql> SELECT ROWKEY , COL3 FROM TEST_REKEY EMIT CHANGES LIMIT 3;
+---------+-------+
|ROWKEY |COL3 |
+---------+-------+
|AB |1 |
|AB |2 |
|CD |3 |
Limit Reached
Query terminated
You can also inspect the underlying Kafka topic to verify the key:
ksql> PRINT TEST_REKEY LIMIT 3;
Format:JSON
{"ROWTIME":1578569329184,"ROWKEY":"AB","MY_COMPOSITE_KEY":"AB","STRINGA":"A","STRINGB":"B","COL3":1}
{"ROWTIME":1578569331653,"ROWKEY":"AB","MY_COMPOSITE_KEY":"AB","STRINGA":"A","STRINGB":"B","COL3":2}
{"ROWTIME":1578569339177,"ROWKEY":"CD","MY_COMPOSITE_KEY":"CD","STRINGA":"C","STRINGB":"D","COL3":3}
ksql>
With this done, we can now declare a table on top of the re-keyed topic:
CREATE TABLE TEST_TABLE (ROWKEY VARCHAR KEY,
COL3 INT)
WITH (KAFKA_TOPIC='TEST_REKEY', VALUE_FORMAT='JSON');
From this table we can query the state. Note that the composite key AB only shows the latest value, which is part of the semantics of a table (compare to the stream above, in which you see both values - both stream and table are the same Kafka topic):
ksql> SELECT * FROM TEST_TABLE EMIT CHANGES;
+----------------+---------+------+
|ROWTIME |ROWKEY |COL3 |
+----------------+---------+------+
|1578569331653 |AB |2 |
|1578569339177 |CD |3 |
Just an update to #Robin Moffat..
Use the below
CREATE STREAM TEST_REKEY AS
SELECT STRINGA + STRINGB AS MY_COMPOSITE_KEY,
STRINGA,
STRINGB,
COL3
FROM TEST
PARTITION BY STRINGA + STRINGB ;
instead of
CREATE STREAM TEST_REKEY AS
SELECT STRINGA + STRINGB AS MY_COMPOSITE_KEY,
STRINGA,
STRINGB,
COL3
FROM TEST
PARTITION BY MY_COMPOSITE_KEY ;
NOTE: column ordering matters
Worked for me! (CLI v0.10.1, Server v0.10.1)

Oracle partition key

I have many tables with large amount of data. The PK is the column (TAB_ID) which has data type RAW(16). I created the hash partitions with partition key having the TAB_ID column.
My issue is: the SQL statement (select * from my_table where tab_id = 'aas1df') does not use partition pruning. If I change the column datatype to varchar2(32), partition pruning works.
Why does not partition pruning work with partition key which have datatype RAW(16)?
I'm just guessing: try select * from my_table where 'aas1df' = tab_id.
Probably the datatype conversion works other way that expected. Anyway you should use the function UTL_RAW.CAST_TO_RAW.
Edited:
Is your table partitioned by TAB_ID? If yes, then there is something wrong with your design, you usually partition table by some useful business value, NOT by surrogate key.
If you know the PK value you do not need partition pruning at all. When Oracle traverses the PK index it gets ROWID value. This ROWID contains file-number, block ID and also row number within the block. So Oracle can access directly the row.
HEXTORAW enables partition pruning.
In the sample below the Pstart and Pstop are literal numbers, implying partition pruning occurs.
create table my_table
(
TAB_ID raw(16),
a number,
constraint my_table_pk primary key (tab_id)
)
partition by hash(tab_id) partitions 16;
explain plan for
select *
from my_table
where tab_id = hextoraw('1BCDB0E06E7C498CBE42B72A1758B432');
select * from table(dbms_xplan.display(format => 'basic +partition'));
Plan hash value: 1204448714
--------------------------------------------------------------------------
| Id | Operation | Name | Pstart| Pstop |
--------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | |
| 1 | TABLE ACCESS BY GLOBAL INDEX ROWID| MY_TABLE | 2 | 2 |
| 2 | INDEX UNIQUE SCAN | MY_TABLE_PK | | |
--------------------------------------------------------------------------

two tables with the same sequence

Is possible to have two tables with the same incrementing sequence?
I was trying to do a tree with ID, NAME, ParentID and i have to join two tables.
If i have different id the tree scheme of ID - ParentId will not work.
Table A Table B
ID | NAME | PID ID | NAME | PID
1 | xpto | 0 1 | xpto | 1
If you are doing both inserts at the same time, you can use SEQUENCE.NEXTVAL for the insert into the first table to get a new ID, and then SEQUENCE.CURRVAL for the insert into the second table to reuse the same ID.
I found the answer: "Sequence numbers are generated independently of tables, so the same sequence can be used for one or for multiple tables."
http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_6015.htm
Tanks for your help.
You could have a master table that is nothing but the sequence PK/FK and then have two child tables. Insert a row in the master just to get the sequence and then use that sequence as the PK in the child tables. If the child tables have the same sequence then why is not one table?
create sequence v_seq
INCREMENT by 1
minvalue 1
maxvalue 10;
Sample Image
create table v_t_s_emp(v_id number,vname varchar2(10));
insert into v_t_s_emp values(v_seq.nextval,'krishna');
create table v_t_s_emp1(v_id number,vname varchar2(10));
insert into v_t_s_emp1 values(v_seq.nextval,'RAMesh');
commit;
select * from v_t_s_emp
union all
select * from v_t_s_emp1;

Resources