Can an STA work on a query that is not even run once in the database? I mean, if it's not there in the cache, can SQL tuning advisor get some result for us?
SQL Tuning Advisor can work on queries that have never been run before. I'm not sure what interface you're using, but the DBMS_SQLTUNE package makes it easy to pass in the SQL text.
For example, let's create a simple table with no indexes:
--Create a simple table with no indexes.
create table table_missing_index(a number);
insert into table_missing_index
select level from dual connect by level <= 100000;
begin
dbms_stats.gather_table_stats(user, 'TABLE_MISSING_INDEX');
end;
/
Then pass in a query that could obviously benefit from an index:
--Create, execute, and display the tuning task.
declare
v_task varchar2(64);
begin
v_task := dbms_sqltune.create_tuning_task
(
sql_text => 'select a from table_missing_index where a = 1'
);
dbms_sqltune.execute_tuning_task(task_name => v_task);
dbms_output.put_line('Task name: '||v_task);
end;
/
Grab the task name from the output, and plug it into this query to see the results:
select dbms_sqltune.report_tuning_task('TASK_362') from dual;
The output should look something like this:
GENERAL INFORMATION SECTION
-------------------------------------------------------------------------------
Tuning Task Name : TASK_362
Tuning Task Owner : JHELLER
Workload Type : Single SQL Statement
Scope : COMPREHENSIVE
Time Limit(seconds): 1800
Completion Status : COMPLETED
Started at : 04/04/2021 14:26:15
Completed at : 04/04/2021 14:26:15
-------------------------------------------------------------------------------
Schema Name : JHELLER
Container Name: ORCLPDB
SQL ID : 0g6v1x1c7kcjt
SQL Text : select a from table_missing_index where a = 1
-------------------------------------------------------------------------------
FINDINGS SECTION (1 finding)
-------------------------------------------------------------------------------
1- Index Finding (see explain plans section below)
--------------------------------------------------
The execution plan of this statement can be improved by creating one or more
indices.
Recommendation (estimated benefit: 98.56%)
------------------------------------------
- Consider running the Access Advisor to improve the physical schema design
or creating the recommended index.
create index JHELLER.IDX$$_016A0001 on JHELLER.TABLE_MISSING_INDEX("A");
... [removed rest of the large report] ...
Related
Is there a hint to generate execution plan ignoring the existing one from the shared pool?
There is not a hint to create an execution plan that ignores plans in the shared pool. A more common way of phrasing this question is: how do I get Oracle to always perform a hard parse?
There are a few weird situations where this behavior is required. It would be helpful to fully explain your reason for needing this, as the solution varies depending why you need it.
Strange performance problem. Oracle performs some dynamic re-optimization of SQL statements after the first run, like adaptive cursor sharing and cardinality feedback. In the rare case when those features backfire you might want to disable them.
Dynamic query. You have a dynamic query that used Oracle data cartridge to fetch data in the parse step, but Oracle won't execute the parse step because the query looks static to Oracle.
Misunderstanding. Something has gone wrong and this is an XY problem.
Solutions
The simplest way to solve this problem are by using Thorsten Kettner's solution of changing the query each time.
If that's not an option, the second simplest solution is to flush the query from the shared pool, like this:
--This only works one node at a time.
begin
for statements in
(
select distinct address, hash_value
from gv$sql
where sql_id = '33t9pk44udr4x'
order by 1,2
) loop
sys.dbms_shared_pool.purge(statements.address||','||statements.hash_value, 'C');
end loop;
end;
/
If you have no control over the SQL, and need to fix the problem using a side-effect style solution, Jonathan Lewis and Randolf Geist have a solution using Virtual Private Database, that adds a unique predicate to each SQL statement on a specific table. You asked for something weird, here's a weird solution. Buckle up.
-- Create a random predicate for each query on a specific table.
create table hard_parse_test_rand as
select * from all_objects
where rownum <= 1000;
begin
dbms_stats.gather_table_stats(null, 'hard_parse_test_rand');
end;
/
create or replace package pkg_rls_force_hard_parse_rand is
function force_hard_parse (in_schema varchar2, in_object varchar2) return varchar2;
end pkg_rls_force_hard_parse_rand;
/
create or replace package body pkg_rls_force_hard_parse_rand is
function force_hard_parse (in_schema varchar2, in_object varchar2) return varchar2
is
s_predicate varchar2(100);
n_random pls_integer;
begin
n_random := round(dbms_random.value(1, 1000000));
-- s_predicate := '1 = 1';
s_predicate := to_char(n_random, 'TM') || ' = ' || to_char(n_random, 'TM');
-- s_predicate := 'object_type = ''TABLE''';
return s_predicate;
end force_hard_parse;
end pkg_rls_force_hard_parse_rand;
/
begin
DBMS_RLS.ADD_POLICY (USER, 'hard_parse_test_rand', 'hard_parse_policy', USER, 'pkg_rls_force_hard_parse_rand.force_hard_parse', 'select');
end;
/
alter system flush shared_pool;
You can see the hard-parsing in action by running the same query multiple times:
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
select * from hard_parse_test_rand;
Now there are three entries in GV$SQL for each execution. There's some odd behavior in Virtual Private Database that parses the query multiple times, even though the final text looks the same.
select *
from gv$sql
where sql_text like '%hard_parse_test_rand%'
and sql_text not like '%quine%'
order by 1;
I think there is no hint indicating that Oracle shall find a new execution plan everytime it runs the query.
This is something we'd want for select * from mytable where is_active = :active, with is_active being 1 for very few rows and 0 for maybe billions of other rows. We'd want an index access for :active = 1 and a full table scan for :active = 0 then. Two different plans.
As far as I know, Oracle uses bind variable peeking in later versions, so with a look at the statistics it really comes up with different execution plans for different bind varibale content. But in older versions it did not, and thus we'd want some hint saying "make a new plan" there.
Oracle only re-used an execution plan for exactly the same query. It sufficed to add a mere blank to get a new plan. Hence a solution might be to generate the query everytime you want to run it with a random number included in a comment:
select /* 1234567 */ * from mytable where is_active = :active;
Or just don't use bind variables, if this is the problem you want to address:
select * from mytable where is_active = 0;
select * from mytable where is_active = 1;
Each time an ETL job finished I have to collect the actual used space for each table (including its index spaces), as well as the row count.
The bytes column from user_segments and user_extents only indicates the space allocated. So I used the following to get each table size
Dbms_Space.object_space_usage (
object_owner => v_owner,
object_name => c.name,
object_type => c.typ,
sample_control => NULL,
space_used => v_space_used_out,
space_allocated => v_space_allocated_out,
chain_pcent => v_chain_pcent_out);
v_space_used := v_space_used+v_space_used_out;
v_space_allocated := v_space_allocated+v_space_allocated_out;
looping through each indexes and the table itself.
My question is before I run the above code is it necessary to run?
EXEC DBMS_STATS.GATHER_TABLE_STATS(v_owner,v_table_name);
Why and/or why no?
Thanks,
It is not necessary to run EXEC DBMS_STATS.GATHER_TABLE_STATS(v_owner,v_table_name) ahead of running DBMS_SPACE.OBJECT_SPACE_USAGE
I ran Dbms_Space.object_space_usage, got the results. And then inserted a lot of data into a table, and ran Dbms_Space.object_space_usage a second time.
The results were updated to the new, larger values, without running GATHER_TABLE_STATS.
We know from Oracle DBMS_SPACE documentation that this package requires the analyze privilege. However, merely running OBJECT_SPACE_USAGE on a table, does not update the last_analyzed column of all_tables.
So from observation, DBMS_SPACE.OBJECT_SPACE_USAGE appears to do its own analysis of the table, getting correct space usage. It does not rely on updated stats being collected by DBMS_STATS nor does it update the stats.
Just to further prove the point:
SQL> EXEC DBMS_STATS.delete_table_stats('SOMEOWNER','SOMETABLE');
PL/SQL procedure successfully completed.
SELECT LAST_ANALYZED
FROM ALL_TABLES
WHERE OWNER = 'SOMEOWNER' AND TABLE_NAME = 'SOMETABLE';
LAST_ANALYZED
------------------
SQL>
DECLARE
su NUMBER;
sa NUMBER;
cp NUMBER;
BEGIN
dbms_space.object_space_usage('SOMEOWNER', 'SOMETABLE', 'TABLE',
NULL, su, sa, cp);
dbms_output.put_line('Space Used: ' || TO_CHAR(su));
dbms_output.put_line('Space Allocated: ' || TO_CHAR(sa));
dbms_output.put_line('Chained Percentage: ' || TO_CHAR(cp));
END;
/
Space Used: 1055374677
Space Allocated: 1073741824
Chained Percentage: 0
I would like to have your advise how to implement the plsql. Below is the situation that i want to do..
select * from table A
loop - get each records from #1 step, and execute the store procedure, processMe(a.field1,a.field2,a.field3 || "test",a.field4);
i dont have any idea how to implement something like this. Below is sample parameter for processMe
processMe(
number_name IN VARCHAR,
location IN VARCHAR,
name_test IN VARCHAR,
gender IN VARCHAR )
Begin
select objId into obj_Id from tableUser where name = number_name ;
select locId into loc_Id from tableLoc where loc = location;
insert into tableOther(obj_id,loc_id,name_test,gender)
values (obj_Id ,loc_Id, name_test, gender)
End;
FOR rec IN (SELECT *
FROM table a)
LOOP
processMe( rec.field1,
rec.field2,
rec.field3 || 'test',
rec.field4 );
END LOOP;
does what you ask. You probably want to explicitly list the columns you actually want in the SELECT list rather than doing a SELECT * (particularly if there is an index on the four columns you actually want that could be used rather than doing a table scan or if there are columns you don't need that contain a large amount of data). Depending on the data volume, it would probably be more efficient if a version of processMe were defined that could accept collections rather than processing data on a row-by-row bases as well.
i just add some process. but this is just a sample. By the way, why
you said that this is not a good idea using loop? i interested to know
Performance wise, If you can avoid looping through a result set executing some other DMLs inside a loop, do it.
There is PL/SQL engine and there is SQL engine. Every time PL/SQL engine stumbles upon a SQL statement, whether it's a select, insert, or any other DML statement, it has to send it to the SQL engine for the execution. It calls context switching. Placing DML statement inside a loop will cause the switch(for each DML statement if there are more than one of them) as many times as many times the body of a loop has to be executed. It can be a cause of a serious performance degradation. if you have to loop, say, through a collection, use foreach loop, it minimizes context switching by executing DML statements in batches.
Luckily, your code can be rewritten as a single SQL statement, avoiding for loop entirely:
insert into tableOther(obj_id,loc_id,name_test,gender)
select tu.objId
, tl.locid
, concat(a.field3, 'test')
, a.field4
from table a
join tableUser tu
on (a.field1 = tu.name)
join tableLoc tl
on (tu.field2 = tl.loc)
You can put that insert statement into a procedure, if you want. PL/SQL will have to sent this SQL statement to the SQL engine anyway, but it will only be one call.
You can use a variable declared using a cursor rowtype. Something like this:
declare
cursor my_cursor is
select * from table;
reg my_cursor%rowtype;
begin
for reg in my_cursor loop
--
processMe(reg.field1, reg.field2, reg.field3 || "test", reg.field4);
--
end loop;
end;
I want to measure the RAM usage of an sql statement (e.g. a simple create or insert statement) in an oracle 11g database environment.
I tried to get it by using dbms_space, but it seems like that only gets the disk space.
I also found this site:
http://www.dba-oracle.com/m_memory_usage_percent.htm
But the statement
select
*
from
v$sql
where sql_text like {my table}
dont return the create statement.
See comment above:
select operation,
options,
object_name name,
trunc(bytes/1024/1024) "input(MB)",
trunc(last_memory_used/1024) last_mem,
trunc(estimated_optimal_size/1024) opt_mem,
trunc(estimated_onepass_size/1024) onepass_mem,
decode(optimal_executions, null, null,
optimal_executions||'/'||onepass_executions||'/'||
multipasses_executions) "O/1/M"
from v$sql_plan p
, v$sql_workarea w
where p.address=w.address(+)
and p.hash_value=w.hash_value(+)
and p.id=w.operation_id(+)
and p.address= ( select address
from v$sql
where sql_text like '%my_table%' )
We just converted our sql server stored procedures to oracle procedures. Sql Server SP's were highly dependent on session tables (INSERT INTO #table1...) these tables got converted as global temporary tables in oracle. We ended up with aroun 500 GTT's for our 400 SP's
Now we are finding out that working with GTT's in oracle is considered a last option because of performance and other issues.
what other alternatives are there? Collections? Cursors?
Our typical use of GTT's is like so:
Insert into GTT
INSERT INTO some_gtt_1
(column_a,
column_b,
column_c)
(SELECT someA,
someB,
someC
FROM TABLE_A
WHERE condition_1 = 'YN756'
AND type_cd = 'P'
AND TO_NUMBER(TO_CHAR(m_date, 'MM')) = '12'
AND (lname LIKE (v_LnameUpper || '%') OR
lname LIKE (v_searchLnameLower || '%'))
AND (e_flag = 'Y' OR
it_flag = 'Y' OR
fit_flag = 'Y'));
Update the GTT
UPDATE some_gtt_1 a
SET column_a = (SELECT b.data_a FROM some_table_b b
WHERE a.column_b = b.data_b AND a.column_c = 'C')
WHERE column_a IS NULL OR column_a = ' ';
and later on get the data out of the GTT. These are just sample queries, in actuality the queries are really complext with lot of joins and subqueries.
I have a three part question:
Can someone show how to transform
the above sample queries to
collections and/or cursors?
Since
with GTT's you can work natively
with SQL...why go away from the
GTTs? are they really that bad.
What should be the guidelines on
When to use and When to avoid GTT's
Let's answer the second question first:
"why go away from the GTTs? are they
really that bad."
A couple of days ago I was knocking up a proof of concept which loaded a largish XML file (~18MB) into an XMLType. Because I didn't want to store the XMLType permanently I tried loading it into a PL/SQL variable (session memory) and a temporary table. Loading it into a temporary table took five times as long as loading it into an XMLType variable (5 seconds compared to 1 second). The difference is because temporary tables are not memory structures: they are written to disk (specifically your nominated temporary tablespace).
If you want to cache a lot of data then storing it in memory will stress the PGA, which is not good if you have lots of sessions. So it's a trade-off between RAM and time.
To the first question:
"Can someone show how to transform the
above sample queries to collections
and/or cursors?"
The queries you post can be merged into a single statement:
SELECT case when a.column_a IS NULL OR a.column_a = ' '
then b.data_a
else column_a end AS someA,
a.someB,
a.someC
FROM TABLE_A a
left outer join TABLE_B b
on ( a.column_b = b.data_b AND a.column_c = 'C' )
WHERE condition_1 = 'YN756'
AND type_cd = 'P'
AND TO_NUMBER(TO_CHAR(m_date, 'MM')) = '12'
AND (lname LIKE (v_LnameUpper || '%') OR
lname LIKE (v_searchLnameLower || '%'))
AND (e_flag = 'Y' OR
it_flag = 'Y' OR
fit_flag = 'Y'));
(I have simply transposed your logic but that case() statement could be replaced with a neater nvl2(trim(a.column_a), a.column_a, b.data_a) ).
I know you say your queries are more complicated but your first port of call should be to consider rewriting them. I know how seductive it is to break a gnarly query into lots of baby SQLs stitched together with PL/SQL but pure SQL is way more efficient.
To use a collection it is best to define the types in SQL, because it gives us the flexibility to use them in SQL statements as well as PL/SQL.
create or replace type tab_a_row as object
(col_a number
, col_b varchar2(23)
, col_c date);
/
create or replace type tab_a_nt as table of tab_a_row;
/
Here's a sample function, which returns a result set:
create or replace function get_table_a
(p_arg in number)
return sys_refcursor
is
tab_a_recs tab_a_nt;
rv sys_refcursor;
begin
select tab_a_row(col_a, col_b, col_c)
bulk collect into tab_a_recs
from table_a
where col_a = p_arg;
for i in tab_a_recs.first()..tab_a_recs.last()
loop
if tab_a_recs(i).col_b is null
then
tab_a_recs(i).col_b := 'something';
end if;
end loop;
open rv for select * from table(tab_a_recs);
return rv;
end;
/
And here it is in action:
SQL> select * from table_a
2 /
COL_A COL_B COL_C
---------- ----------------------- ---------
1 whatever 13-JUN-10
1 12-JUN-10
SQL> var rc refcursor
SQL> exec :rc := get_table_a(1)
PL/SQL procedure successfully completed.
SQL> print rc
COL_A COL_B COL_C
---------- ----------------------- ---------
1 whatever 13-JUN-10
1 something 12-JUN-10
SQL>
In the function it is necessary to instantiate the type with the columns, in order to avoid the ORA-00947 exception. This is not necessary when populating a PL/SQL table type:
SQL> create or replace procedure pop_table_a
2 (p_arg in number)
3 is
4 type table_a_nt is table of table_a%rowtype;
5 tab_a_recs table_a_nt;
6 begin
7 select *
8 bulk collect into tab_a_recs
9 from table_a
10 where col_a = p_arg;
11 end;
12 /
Procedure created.
SQL>
Finally, guidelines
"What should be the guidelines on When
to use and When to avoid GTT's"
Global temp tables are very good when we need share cached data between different program units in the same session. For instance if we have a generic report structure generated by a single function feeding off a GTT which is populated by one of several procedures. (Although even that could also be implemented with dynamic ref cursors ...)
Global temporary tables are also good if we have a lot of intermediate processing which is just too complicated to be solved with a single SQL query. Especially if that processing must be applied to subsets of the retrieved rows.
But in general the presumption should be that we don't need to use a temporary table. So
Do it in SQL unless it is too hard it which case ...
... Do it in PL/SQL variables (usually collections) unless it takes too much memory it which case ...
... Do it with a Global Temporary Table
Generally I'd use a PL/SQL collection for storing small volumes of data (maybe a thousand rows). If the data volumes were much larger, I'd use a GTT so that they don't overload the process memory.
So I might select a few hundred rows from the database into a PL/SQL collection, then loop through them to do some calculation/delete a few or whatever, then insert that collection into another table.
If I was dealing with hundreds of thousands of rows, I would try to push as much of the 'heavy lifting' processing into large SQL statements. That may or may not require GTT.
You can use SQL level collection objects as something that translates quite easily between SQL and PL/SQL
create type typ_car is object (make varchar2(10), model varchar2(20), year number(4));
/
create type typ_coll_car is table of typ_car;
/
select * from table (typ_coll_car(typ_car('a','b',1999), typ_car('A','Z',2000)));
MAKE MODEL YEAR
---------- -------------------- ---------------
a b 1,999.00
A Z 2,000.00
declare
v_car1 typ_car := typ_car('a','b',1999);
v_car2 typ_car := typ_car('A','Z',2000);
t_car typ_coll_car := typ_coll_car();
begin
t_car := typ_coll_car(v_car1, v_car2);
FOR i in (SELECT * from table(t_car)) LOOP
dbms_output.put_line(i.year);
END LOOP;
end;
/