Why inline views are used..??
There are many different reasons for using inline views. Some things can't be done without inline views, for example:
1) Filtering on the results of an analytic function:
select ename from
( select ename, rank() over (order by sal desc) rnk
from emp
)
where rnk < 4;
2) Using ROWNUM on ordered results:
select ename, ROWNUM from
( select ename
from emp
order by ename
);
Other times they just make it easier to write the SQL you want to write.
The inline view is a construct in Oracle SQL where you can place a query in the SQL FROM, clause, just as if the query was a table name.
Inline views provide
Bind variables can be introduced inside the statement to limit the data
Better control over the tuning
Visibility into the code
To get top N ordered rows.
SELECT name, salary,
FROM (SELECT name, salary
FROM emp
ORDER BY salary DESC)
WHERE rownum <= 10;
An inline view can be regarded as an intermediate result set that contributes to the required data set in some way. Sometimes it is entirely a matter of improving maintainability of the code, and sometimes it is logically neccessary.
From the Oracle Database Concepts document there are the inline view concept definition:
An inline view is not a schema object.
It is a subquery with an alias
(correlation name) that you can use
like a view within a SQL statement.
About the subqueries look in Using Subqueries from the Oracle SQL Reference manual. It have a very nice pedagogic information.
Anyway, today is preferred to use the Subquery Factoring Clause that is a more powerfull way of use inline views.
As an example of all together:
WITH
dept_costs AS (
SELECT department_name, SUM(salary) dept_total
FROM employees e, departments d
WHERE e.department_id = d.department_id
GROUP BY department_name),
avg_cost AS
SELECT * FROM dept_costs
WHERE dept_total >
(SELECT avg FROM (SELECT SUM(dept_total)/COUNT(*) avg
FROM dept_costs)
)
ORDER BY department_name;
There you can see one of all:
An inline view query: SELECT SUM...
A correlated subquery: SELECT avg FROM...
A subquery factoring: dept_costs AS (...
What are they used for?:
To avoid creating an intermediate view object: CREATE VIEW ...
To simplify some queries that a view cannot be helpfull. For instance, when the view need to filter from the main query.
You will often use inline views to break your query up into logical parts which helps both readability and makes writing more complex queries a bit easier.
Jva and Tony Andrews provided some good examples of simple cases where this is useful such as Top-N or Pagination queries where you may want to perform a query and order its results before using that as a part of a larger query which in turn might feed a query doing some other processing, where the logic for these individual queries would be difficult to achieve in a single query.
Another case they can be very useful is if you are writing a query that joins various tables together and want to perform aggregation on some of the tables, separating group functions and the processing into different inline views before performing the joins makes managing cardinality a lot easier. If you want some examples, I would be happy to provide them to make it more clear.
Factored subqueries (where you list your queries in the WITH clause at the start of the query) and inline views also often bring performance benefits. If you need to access the results of the subquery multiple times, you only need to run it once and it can be materialized as a Global Temporary Table (how the optimizer acts isn't totally black and white so I won't go into it here but you can do your own research - for example, see http://jonathanlewis.wordpress.com/2007/07/26/subquery-factoring-2/)
Related
I was working on task about optimization queries. One of the improvement ways was using WITH clause. I notice that it did very good job, and it lead to shorter time of execution, but i am not sure now, when should I use WITH clause and is there any risk of using it?
Here is one of the queries that I am working on :
WITH MY_TABLE AS
( SELECT PROD_KY,
sum(GROUPISPRIVATE) AS ISPRIVATE,
sum(GROUPISSHARED) AS ISSHARED
FROM
(
SELECT GRP_PROD_CUSTOMER.PROD_KY,
1 as ISPRIVATE,
0 as ISSHARED
FROM CUSTOMER
JOIN GRP_CUSTOMER ON GRP_CUSTOMER.CUST_KY = CUSTOMER.CUST_KY
JOIN GRP_PROD_CUSTOMER ON GRP_PROD_CUSTOMER.GRP_KY = GRP_CUSTOMER.GRP_KY
GROUP BY GRP_PROD_CUSTOMER.PROD_KY
)
GROUP BY PROD_KY
)
SELECT * FROM MY_TABLE;
is there any risk of using it?
Yes. Oracle may decide to materialize the subquery, which means writing its result set to disk and then reading it back (except it might not mean that in 12cR2 or later). That unexpected I/O could be a performance hit. Not always, and usually we can trust the optimizer to make the correct choice. However, Oracle has provided us with hints to tell the optimizer how to handle the result set: /*+ materialize */ to um materialize it and /*+ inline */ to keep it in memory.
I start with this potential downside because I think it's important to understand that the WITH clause is not a silver bullet and it won't improve every single query, and may even degrade performance. For instance I share the scepticism of the other commenters that the query you posted is in any way faster because you re-wrote it as a common table expression.
Generally, the use cases for the WITH clause are:
We want to use the result set from the subquery multiple times
with cte as
( select blah from meh )
select *
from t1
join t2 on t1.id = t2.id
where t1.col1 in ( select blah from cte )
and t2.col2 not in ( select blah from cte)
We want to be build a cascade of subqueries:
with cte as
( select id, blah from meh )
, cte2 as
( select t2.*, cte.blah
from cte
join t2 on t2.id = cte.id)
, cte3 as
( select t3.*, cte2.*
from cte2
join t3 on t3.col2 = cte2.something )
….
This second approach is beguiling and can be useful for implementing complex business logic in pure SQL. But it can lead to a procedural mindset and lose the power sets and joins. This too is a risk.
We want to use recursive WITH clause. This allows us to replace Oracle's own CONNECT BY syntax with a more standard approach. Find out more
In 12c and later we can write user-defined functions in the WITH clause. This is a powerful feature, especially for users who need to implement some logic in PL/SQL but only have SELECT access to the database. Find out more
For the record I have seen some very successful and highly performative uses of the second type of WITH clause. However I have also seen uses of WITH when it would have been just as easy to write an inline view. For instance, this is just using the WITH clause as syntactic sugar ...
with cte as
( select id, blah from meh )
select t2.*, cte.blah
from t2
join cte on cte.id = t2.id
… and would be clearer as ...
select t2.*, cte.blah
from t2
join ( select id, blah from meh ) cte on cte.id = t2.id
WITH clause is introduced in oracle to match SQL-99 standard.
The main purpose is to reduce the complexity and repetitive code.
Lets say you need to find the average salary of one department and then need to fetch all the department(d1) with more than average salary of that department(d1).
This can make multiple references to the subquery more efficient and readable.
The MATERIALIZE and INLINE optimizer hints can be used to influence the decision. The undocumented MATERIALIZE hint tells the optimizer to resolve the subquery as a global temporary table, while the INLINE hint tells it to process the query inline. Decision to use the hint is purely depends on logic that we are going to implement in query.
In oracle 12c, declaration of PL/SQL Block in WITH clause is introduced.
You must refer it from oracle documents.
Cheers!!
Your query is rather useless in terms of WITH statement (aka Common Table Expression, CTE)
Anyway, using the WITH clause brings several benefits:
The query is better readable (in my opinion)
You can use the same subquery several times in the main query. You can even cascade them.
Oracle can materialize the subquery, i.e. Oracle may create a temporary table and stores result of the subquery in it. This can give better performance.
The WITH clause may be processed as an inline view or resolved as a temporary table. The SQL WITH clause is very similar to the use of Global temporary tables. This technique is often used to improve query speed for complex subqueries and enables the Oracle optimizer to push the necessary predicates into the views.
The advantage of the latter is that repeated references to the subquery may be more efficient as the data is easily retrieved from the temporary table, rather than being requeried by each reference. You should assess the performance implications of the WITH clause on a case-by-case basis.
You can read more here:
http://www.dba-oracle.com/t_with_clause.htm
https://oracle-base.com/articles/misc/with-clause
one point to consider is, that different RDBMS handle the with clause - aka common table expressions (CTE) aka subquery factoring - differently:
Oracle may use a materialization or an inlining (as already explained in the answer provided by APC)
postgres always uses a materialization in releases up to 11 (so here a CTE is an optimization fence). In postgres 12 the behaviour changes and is similar to Oracles approach: https://info.crunchydata.com/blog/with-queries-present-future-common-table-expressions. You even have something that almost looks like a hint (though it is known that postgres does not use hints...)
in SQL Server currently a CTE is always inlined, as explained in https://erikdarlingdata.com/2019/08/what-would-materialized-ctes-look-like-in-sql-server/
So depending on the RDBMS you use and its version your mileage may vary.
There are set of sql queries which I want to convert into procedure without intermediary tables creation and dropping. I have included the sample queries below. what could be best possible way for doing that? Since I am new to procedures can advice me how to do that.
Can we use cursor in procedures for doing that? In the last line of the query im joining 2 intermediary tables. instead of that Can we join two cursor? If so how can we do that? Is there any way to do that. Please suggest me.
CREATE TABLE A AS SELECT ID_LOAN FROM master_copy WHERE ZERO_BAL_CODE IN (1);
CREATE TABLE B AS SELECT master_copy.ID_LOAN,LOAN_AGE,master_copy.vintage,DELINQ_STATUS,ZERO_BAL_CODE,master_copy.ACTUAL_LOSS,current_upb FROM master_copy INNER JOIN A ON master_copy.ID_LOAN= A.ID_LOAN;
CREATE TABLE prepaidData AS SELECT ID_LOAN,max(to_number(DELINQ_STATUS)) as DELINQ_STATUS,max(loan_age) AS LOAN_AGE,max(ZERO_BAL_CODE) as ZERO_BAL_CODE,max(vintage) as vintage,
min(ACTUAL_LOSS) as actual_loss,MIN(NULLIF(current_upb,0)) as current_upb FROM B group by id_loan;
alter table prepaiddata add loan_type varchar2(255) default 'prepaid';
drop table a;
drop table b;
CREATE TABLE A AS SELECT ID_LOAN FROM master_copy WHERE ZERO_BAL_CODE IN (3);
CREATE TABLE B AS SELECT master_copy.ID_LOAN,LOAN_AGE,master_copy.vintage,DELINQ_STATUS,ZERO_BAL_CODE,master_copy.ACTUAL_LOSS,current_upb FROM master_copy INNER JOIN A ON master_copy.ID_LOAN= A.ID_LOAN;
CREATE TABLE DEFAULT_FORECLOSURE AS SELECT ID_LOAN,max(to_number(DELINQ_STATUS)) as DELINQ_STATUS,max(loan_age) AS LOAN_AGE,max(ZERO_BAL_CODE) as ZERO_BAL_CODE,max(vintage) as vintage,
min(ACTUAL_LOSS) as actual_loss,MIN(NULLIF(current_upb,0)) as current_upb FROM B group by id_loan;
alter table DEFAULT_FORECLOSURE add loan_type varchar2(255) default 'default_foreclosure';
drop table a;
drop table b;
create table aa_loan_type as(select * from prepaiddata union select * from DEFAULT_FORECLOSURE);
The best way to avoid intermediate tables is to use inline views. Your queries can be rewritten into this one SQL statement:
--aa_loan_type
--
--prepaidData
SELECT ID_LOAN,max(to_number(DELINQ_STATUS)) as DELINQ_STATUS,max(loan_age) AS LOAN_AGE,max(ZERO_BAL_CODE) as ZERO_BAL_CODE,max(vintage) as vintage, min(ACTUAL_LOSS) as actual_loss,MIN(NULLIF(current_upb,0)) as current_upb, 'PREPAID' LOAN_TYPE
FROM
(
--B
SELECT
master_copy.ID_LOAN,LOAN_AGE,master_copy.vintage,DELINQ_STATUS,ZERO_BAL_CODE,master_copy.ACTUAL_LOSS,current_upb
FROM master_copy
INNER JOIN
(
--A
SELECT ID_LOAN FROM master_copy WHERE ZERO_BAL_CODE IN (1)
) A
ON master_copy.ID_LOAN= A.ID_LOAN;
) B
GROUP BY ID_LOAN
union
--DEFAULT_FORECLOSURE
SELECT ID_LOAN,max(to_number(DELINQ_STATUS)) as DELINQ_STATUS,max(loan_age) AS LOAN_AGE,max(ZERO_BAL_CODE) as ZERO_BAL_CODE,max(vintage) as vintage, min(ACTUAL_LOSS) as actual_loss,MIN(NULLIF(current_upb,0)) as current_upb, 'default_foreclosure' loan_type
FROM
(
--B
SELECT master_copy.ID_LOAN,LOAN_AGE,master_copy.vintage,DELINQ_STATUS,ZERO_BAL_CODE,master_copy.ACTUAL_LOSS,current_upb
FROM master_copy
INNER JOIN
(
--A
SELECT ID_LOAN FROM master_copy WHERE ZERO_BAL_CODE IN (3)
) A
ON master_copy.ID_LOAN= A.ID_LOAN
) B
group by id_loan;
When built correctly, one large SQL statement is often much better than multiple small SQL statements. The overall code will be simpler (less objects, easier to debug in an IDE) and potentially much faster (no need to write data, gives the optimizer more chances to do something clever).
Building large SQL "correctly" is subjective, but it boils down to treating each inline view as a miniature program:
Keep each inline view simple, combine them in simple steps, and repeat.
Use good names and comments for each inline view. You'll probably want something better than "A" and "B".
Use Allman style bracket matching, with parentheses on each line. Inline views are important and deserve extra whitespace and aligned parentheses. Except for beginner developers, we don't need to align keywords like SELECT and FROM. We need to align important boundaries, like the parentheses of each inline view. This will help you debug by quickly highlighting and running code in an IDE.
Keep the subquery interfaces simple by using inline views instead of correlated subqueries or common table expressions. The beauty of inline views is their simplicity - relational data in, relational data out.
Here is the example SQL in question; The SQL should run on any Oracle DBMS (I'm running 11.2.0.2.0).
Note how the UUID values are different (one has 898 the other has 899) in the resultset despite being built from within the inline view/with clause. Further below you can see how DBMS_RANDOM.RANDOM() does not have this side effect.
SQL:
WITH data AS (SELECT SYS_GUID () uuid FROM DUAL)
SELECT uuid, uuid
FROM data
Output:
UUID UUID_1
F8FCA4B4D8982B55E0440000BEA88F11 F8FCA4B4D8992B55E0440000BEA88F11
In Contrast DBMS_RANDOM the results are the same
SQL:
WITH data AS (SELECT DBMS_RANDOM.RANDOM() rand FROM DUAL)
SELECT rand, rand
FROM data
Output:
RAND RAND_1
92518726 92518726
Even more interesting is I can change the behavior / stabilize sys_guid by including calls to DBMS_RANDOM.RANDOM:
WITH data AS (
SELECT SYS_GUID () uuid,
DBMS_RANDOM.random () rand
FROM DUAL)
SELECT uuid a,
uuid b,
rand c,
rand d
FROM data
SQL Fiddle That Stabilizes SYS_GUID:
http://sqlfiddle.com/#!4/d41d8/29409
SQL Fiddle That shows the odd SYS_GUID behavior:
http://sqlfiddle.com/#!4/d41d8/29411
The documentation gives a reason as to why you may see a discrepancy (emphasis mine):
Caution:
Because SQL is a declarative language, rather than an imperative (or procedural) one, you cannot know how many times a function invoked by a SQL statement will run—even if the function is written in PL/SQL, an imperative language.
If your application requires that a function be executed a certain number of times, do not invoke that function from a SQL statement. Use a cursor instead.
For example, if your application requires that a function be called for each selected row, then open a cursor, select rows from the cursor, and call the function for each row. This technique guarantees that the number of calls to the function is the number of rows fetched from the cursor.
Basically, Oracle doesn't specify how many times a function will be called inside a sql statement: it may be dependent upon the release, the environment, the access path among other factors.
However, there are ways to limit query rewrite as explained in the chapter Unnesting of Nested Subqueries:
Subquery unnesting unnests and merges the body of the subquery into the body of the statement that contains it, allowing the optimizer to consider them together when evaluating access paths and joins. The optimizer can unnest most subqueries, with some exceptions. Those exceptions include hierarchical subqueries and subqueries that contain a ROWNUM pseudocolumn, one of the set operators, a nested aggregate function, or a correlated reference to a query block that is not the immediate outer query block of the subquery.
As explained above, you can use ROWNUM pseudo-column to prevent Oracle from unnesting a subquery:
SQL> WITH data AS (SELECT SYS_GUID() uuid FROM DUAL WHERE ROWNUM >= 1)
2 SELECT uuid, uuid FROM data;
UUID UUID
-------------------------------- --------------------------------
1ADF387E847F472494A869B033C2661A 1ADF387E847F472494A869B033C2661A
The NO_MERGE hint "fixes" it. Prevents Oracle from re-writing the inline view.
WITH data AS (SELECT /*+ NO_MERGE */
SYS_GUID () uuid FROM DUAL)
SELECT uuid, uuid
FROM data
From the docs:
The NO_MERGE hint instructs the optimizer not to combine the outer
query and any inline view queries into a single query.This hint lets
you have more influence over the way in which the view is accessed.
SQL Fiddle with the NO_MERGE hint applied:
I'm still struggling to understand/articulate how the query is being re-written in such a way that sys_guid() would be called twice. Perhaps it is a bug; but I tend to assume it is a bug in my own thoughts/code.
Very interesting.
We can use the materialize hint to fix it to.
WITH data AS (SELECT /*+materialize*/SYS_GUID () uuid FROM DUAL)
SELECT uuid, uuid
FROM data;
1 F9440E2613761EC8E0431206460A934C F9440E2613761EC8E0431206460A934C
From my point of view, if we can change the result of a query just by adding a hint, there is an Oracle bug.
Maybe we have to ask metalink to check it...
I have a page that pulls together aggregate data from two different tables. I would like to perform these queries in parallel to reduce the latency without having to introduce a stored procedure that would do both.
For example, I currently have this:
ViewBag.TotalUsers = DB.Users.Count();
ViewBag.TotalPosts = DB.Posts.Count();
// Page displays both values but has two trips to the DB server
I'd like something akin to:
var info = DB.Select(db => new {
TotalUsers = db.Users.Count(),
TotalPosts = db.Posts.Count());
// Page displays both values using one trip to DB server.
that would generate a query like this
SELECT (SELECT COUNT(*) FROM Users) AS TotalUsers,
(SELECT COUNT(*) FROM Posts) AS TotalPosts
Thus, I'm looking for a single query to hit the DB server. I'm not asking how to parallelize two separate queries using Tasks or Threads
Obviously I could create a stored procedure that got back both values in a single trip, but I'd like to avoid that if possible as it's easier to add additional stats purely in code rather than having to keep refreshing the DB import.
Am I missing something? Is there a nice pattern in EF to say that you'd like several disparate values that can all be fetched in parallel?
This will return the counts using a single select statement, but there is an important caveat. You'll notice that the EF-generated sql uses cross joins, so there must be a table (not necessarily one of the ones you are counting), that is guaranteed to have rows in it, otherwise the query will return no results. This isn't an ideal solution, but I don't know that it's possible to generate the sql in your example since it doesn't have a from clause in the outer query.
The following code counts records in the Addresses and People tables in the Adventure Works database, and relies on StateProvinces to have at least 1 record:
var r = from x in StateProvinces.Top("1")
let ac = Addresses.Count()
let pc = People.Count()
select new { AddressCount = ac, PeopleCount = pc };
and this is the SQL that is produced:
SELECT
1 AS [C1],
[GroupBy1].[A1] AS [C2],
[GroupBy2].[A1] AS [C3]
FROM
(
SELECT TOP (1) [c].[StateProvinceID] AS [StateProvinceID]
FROM [Person].[StateProvince] AS [c]
) AS [Limit1]
CROSS JOIN
(
SELECT COUNT(1) AS [A1]
FROM [Person].[Address] AS [Extent2]
) AS [GroupBy1]
CROSS JOIN
(
SELECT COUNT(1) AS [A1]
FROM [Person].[Person] AS [Extent3]
) AS [GroupBy2]
and the results from the query when it's run in SSMS:
C1 C2 C3
----------- ----------- -----------
1 19614 19972
You should be able to accomplish what you want with Parallel LINQ (PLINQ). You can find an introduction here.
It seems like there's no good way to do this (yet) in EF4. You can either:
Use the technique described by adrift which will generate a slightly awkward query.
Use the ExecuteStoreQuery where T is some dummy class that you create with property getters/setters matching the name of the columns from the query. The disadvantage of this approach is that you can't directly use your entity model and have to resort to SQL. In addition, you have to create these dummy entities.
Use the a MultiQuery class that combines several queries into one. This is similar to NHibernate's futures hinted at by StanK in the comments. This is a little hack-ish and it doesn't seem to support scalar valued queries (yet).
When joining across tables (as in the examples below), is there an efficiency difference between joining on the tables or joining subqueries containing only the needed columns?
In other words, is there a difference in efficiency between these two tables?
SELECT result
FROM result_tbl
JOIN test_tbl USING (test_id)
JOIN sample_tbl USING (sample_id)
JOIN (SELECT request_id
FROM request_tbl
WHERE request_status='A') USING(request_id)
vs
SELECT result
FROM (SELECT result, test_id FROM result_tbl)
JOIN (SELECT test_id, sample_id FROM test_tbl) USING(test_id)
JOIN (SELECT sample_id FROM sample_tbl) USING(sample_id)
JOIN (SELECT request_id
FROM request_tbl
WHERE request_status='A') USING(request_id)
The only way to find out for sure is to run both with tracing turned on and then look at the trace file. But in all probability they will be treated the same: the optimizer will merge all the inline views into the main statement and come up with the same query plan.
It doesn't matter. It may actually be WORSE since you are taking control away from the optimizer which generally knows best.
However, remember if you are doing a JOIN and only including a column from one of the tables that it is QUITE OFTEN better to re-write it as a series of EXISTS statements -- because that's what you really mean. JOINs (with some exceptions) will join matching rows which is a lot more work for the optimizer to do.
e.g.
SELECT t1.id1
FROM table1 t1
INNER JOIN table2 ON something = something
should almost always be
SELECT id1
FROM table1 t1
WHERE EXISTS( SELECT *
FROM table2
WHERE something = something )
For simple queries the optimizer may reduce the query plans into identical ones. Check it out on your DBMS.
Also this is a code smell and probably should be changed:
JOIN (SELECT request_id
FROM request_tbl
WHERE request_status='A')
to
SELECT result
FROM request
WHERE EXISTS(...)
AND request_status = 'A'
No difference.
You can tell by running EXPLAIN PLAN on both those statements - Oracle knows that all you want is the "result" column, so it only does the minimum necessary to get the data it needs - you should find that the plans will be identical.
The Oracle optimiser does, sometimes, "materialize" a subquery (i.e. run the subquery and keep the results in memory for later reuse), but this is rare and only occurs when the optimiser believes this will result in a performance improvement; in any case, Oracle will do this "materialization" whether you specified the columns in the subqueries or not.
Obviously if the only place the "results" column is stored is in the blocks (along with the rest of the data), Oracle has to visit those blocks - but it will only keep the relevant info (the "result" column and other relevant columns, e.g. "test_id") in memory when processing the query.