How to use SQL query as source in ODI 12c - etl

I need to use Select Query results ( having Sub query) as source. This results needs to join with another source table.
Hi All,
I need to use Select Query results ( having Sub query) as source. This results needs to join with another source table.
Ex: Select A,B from (Select t1.a as A, t1.b as B from table1 t1,table t2 where t1.a=t2.b)
In 11g, we've yellow interface where we can write custom queries and invoke that into blue interface for joining with another table.
I'm looking for similar options in 12c

Related

How to create a temporary table/view inside an Oracle procedure?

I am making a new procedure making queries to a huge table.
The structure of my procedure is as follows:
{
open cursor for
QUERY 1
UNION
QUERY 2
UNION
QUERY 3
}
The structure of QUERY 1 is INNER JOIN 2 ([INNER JOIN 1 (TABLE) x (TABLE) ] x TABLE )
The structure of QUERY 2 is INNER JOIN 3 ([INNER JOIN 1 (TABLE) x (TABLE) ] x TABLE )
Is there a way to store [INNER JOIN 1 (TABLE) x (TABLE) ] somewhere so that I don't have to do it twice?
EDIT: Forgot to add that I cannot create a table outside of the procedure because multiple instances of this procedure will run in parallel. They will just block each other from running by inserting in the same table. Also, I don't know how many instances will run in parallel so I cannot create as many tables as instances.
Don't create any tables from PL/SQL. It is possible (hint: dynamic SQL), but that's not how Oracle works.
If you need a table, then create it BEFORE running this procedure, either using CREATE TABLE (and name all columns you need), or using CTAS (Create Table As Select) which would - basically - be your current query.
That table can be "normal" or "global (or private, depending on database version) temporary table" (GTT). If you use a GTT, only you can see data stored within. If it is a "normal" table, everyone sees data so you might need to pay attention to who sees & uses what.
Another option is to use the CTE (Common Table Expression, a.k.a. the WITH factoring clause) which can be used directly in the procedure as
with your_view as
(select ...
from table1 join table2 on ...
join table3 on ...
)
select whatever
from some_other_table join your_view
where ...
union
select whatever_else
from yet_another_table join your_View
where ...
[EDIT, after seeing your edit]
If you don't want to use a CTE for some reason, then a GTT might be your choice. Why? See my 3rd paragraph ("everyone sees only their own data").
You could always use a Global Temporary table:
https://oracle-base.com/articles/misc/temporary-tables#temporary-tables

inserting records from two different tables into a single table in oracle

I want to insert data from two different tables (say table A and table B ) into a third table (table C) in oracle.
I have written two different cursors for fetching data from table A and B separately, and populated two collections based on these two tables.
Now, i want to insert the data in those two collections into the third table (table C), how can i get this done.
Now there are two common columns that are present in both the columns, say for example ID and YEARMONTH, these two columns are there in all tables (A, B and C).
I have tried doing a merge based on these two fields.
but i am looking for an efficient and more convenient way to do this.
You didn't provide code you wrote, so I'll guess: cursors mean PL/SQL. If you're doing it in a loop, row-by-row, it'll be slow-by-slow.
As there are common columns in both tables (A and B), I'd suggest doing it in pure SQL: join those two tables and insert the result into C. Something like
insert into c (id, yearmonth, ...)
select a.id, a.yearmonth, ...
from a join b on a.id = b.id;
Make sure that indexes exist on columns you use to join tables. Or, even better, compare explain plans in both cases (with and without indexes) and choose an option which seems to be the best.
insert into tableC
select * from tableA where ...
union
select * from tableB where ...

How to populate columns of a new hive table from multiple existing tables?

I have created a new table in hive (T1) with columns c1,c2,c3,c4. I want to populate data into this table by querying from other existing tables(T2,T3).
E.g c1 and c2 come from a query run on T2 & the other columns c3 and c4 come from a query run on T3.
Is this possible in hive ? I have done immense research but still am unable to find a solution to this
Didn't something like this work?
create table T1 as
select t2.c1, t2.c2, t3.c3, t3.c4 from (some query against T2) t2 JOIN (some query against T3) t3
Obviously replace JOIN with whatever is needed. I assume some join between T2 and T3 is possible or else you wouldn't be putting their columns alongside each other in T1.
According to the hive documentation, you can use the following syntax to insert data:
INSERT INTO TABLE tablename1 [PARTITION (partcol1=val1, partcol2=val2 ...)] select_statement1 FROM from_statement;
Be careful that:
Values must be provided for every column in the table. The standard SQL syntax that allows the user to insert values into only some columns is not yet supported. To mimic the standard SQL, nulls can be provided for columns the user does not wish to assign a value to.
So, I would make a JOIN between the two existing table, and then insert only the needed values in the target table playing around with SELECT. Or maybe creating a temporary table would allow you to have more control over the data. Just remember to handle the problem with NULL, as stated in the official documentation. This is just an idea, I guess there are other ways to achieve what you need, but could be a good place to start from.

How to use Oracle Materialzed View in a Dimensional Model

I have a dimensional model with a large fact table (millions of rows) which is range partitioned by date and smaller dimensional tables that are not partitioned. I came across materialized views which is often used in these scenarios to improve query performance.
Now, I want to know which way is better of the following two to utilize these materialized views to get aggregated reports.
A. Create one with the by joining the whole fact table with each of the dimension tables required.
create materialized view my_mview execute immediate query rewrite
select
fact.col1, dim1.col2, dim2.col3, sum(fact.col4)
from
my_fact fact
inner join
my_dim1 dim1
on fact.dim1_key = dim1.dim1_key
inner join
my_dim2 dim2
on fact.dim2_key = dim2.dim2_key group by fact.col1, dim1.col2, dim2.col3
This seems like the most basic way of using them. But it seems
rather limiting and I would require a new materialzed view for each
variation of the query I want to create.
B. Create it over the aggregation of the fact table and utilize the query rewrite when doing a dimensional join back.
create materialized view my_mview execute immediate query rewrite
select
col1, dim1.dim2_key, dim2.dim_key, sum(fact.col4)
from
my_fact fact
And do the join as above in case A, which will use this aggregated materialzed view for the join and not the whole fact table.
Can anyone tell me when I would use each case or the other?
Your first example works exactly as you described.
For the second example the query should be:
create materialized view my_mview execute immediate query rewrite
select
col1, fact.dim2_key, fact.dim_key, sum(fact.col4)
from
my_fact fact
group by
col1, fact.dim2_key, fact.dim_key
This will automatically speed up aggregates such as
select sum(fact.col4)
from fact
select fact.dim_key,sum(fact.col4)
from fact
group by fact.dim_key
select fact.dim2_key,sum(fact.col4)
from fact
group by fact.dim2_key
I don't think Oracle will rewrite your first type of query to this MV automatically because in the MV the join columns are already grouped by (They also should be grouped in your second example). It never happened for us. This however may also depend on if there are relationships defined between dim and fact table and the value of QUERY_REWRITE_INTEGRITY parameter, so there is still some room for testing here.
You may still get a performance gain by writing a query in a specific way
WITH preaggr as (
select
col1, fact.dim2_key, fact.dim_key, sum(fact.col4)
from
my_fact fact
group by
col1, fact.dim2_key, fact.dim_key
)
select
dim2.col1,
sum(preaggr.col4)
from
preaggr
join
dim2
on
preaggr.dim2_key = fact.dim2_key
group by
dim2.col1

How to select row data as column in Oracle

I have two tables like bellow shows figures
I need to select records as bellow shown figure. with AH_ID need to join in second table and ATT_ID will be the column header and ATT_DTL_STR_VALUE need to get as that column relevant value
Required output
Sounds like you have an Entity-Attribute-Value data model which relational DBs aren't the best at modeling. You may want to look into a key-value store.
However, as Justin suggested, if you're using 11g you can use th pivot clause as follows:
SELECT *
FROM (
SELECT T1.AH_ID, T1.AH_DESCRIPTION, T2.ATT_ID, T2.ATT_DTL_STR_VALUE
FROM T1
LEFT OUTER JOIN T2 ON T1.AH_ID = T2.AH_ID
)
PIVOT (MAX(ATT_DTL_STR_VALUE) FOR (ATT_ID) IN (1));
This statement requires you to hard-code in ATT_ID however there are ways to do it dynamically. More info can be found here.

Resources