I am trying to find data in a database with hundreds of tables. I am just going through and writing simple select * queries for all of the tables checking them one by one. It is taking forever! I was wondering if there is a way to write a query to filter all of the tables with at least 1 row in the database. Is this possible?
with
function f(tabname in varchar2) return int as
res int;
begin
execute immediate 'select count(*) cnt from "'||tabname||'" where rownum=1' into res;
return res;
end;
select table_name, f(table_name) chk
from user_tables;
Results:
TABLE_NAME CHK
------------------------------ ----------
STRVALS 1
MY_EMP 1
TBL_3 1
TBL_1 1
TLOCK 1
DATES 1
B 1
A 1
C 1
T0 0
Or select count(*) from dual where exists(...):
with
function f(tabname in varchar2) return int as
res int;
begin
execute immediate 'select count(*) cnt from dual where exists(select 1 from "'||tabname||'")' into res;
return res;
end;
select table_name, f(table_name) chk
from user_tables
If your database is being properly maintained, you'll have optimizer statistics.
If you tables are missing stats, you can collect them thusly:
BEGIN
dbms_stats.gather_schema_stats(
ownname => 'HR'
, estimate_percent => X -- note the higher you go here, the more work will be done
);
END;
Then, QUERY THE STATISTICS, not your tables.
SELECT num_rows
, table_name
FROM sys.dba_all_tables
WHERE owner = 'HR'
You can then QUICKLY see what tables are empty or not - based on how accurate your stats are.
If you use dynamic SQL to run a SELECT COUNT(*) on every table in your schema, and your schema and/or tables are HUGE, you are going to have a very bad experience, and perhaps cause performance issues for your database.
Related
I have a table "TEST_TABLE" with two columns TABLE_NAME and RECORD_COUNT.
enter image description here
We want to update the column RECORD_COUNT by taking the total records in table specified in TABLE_NAME.
You could do this with dynamic SQL but why be complicated?
Create a view
Create view my_tables as
Select
'table_1' as "table name",
count(*) as "rows"
From table_1
Add the following for each table
Union all
Select 'table_2',count(*) From table_2
You can then use the view like a table:
Select * from my_tables;
OK, but - why wouldn't you use information Oracle already provides for you? You should regularly gather statistics anyway, so:
SQL> execute dbms_stats.gather_schema_stats(ownname => 'SCOTT', estimate_percent => null);
PL/SQL procedure successfully completed.
and then fetch num_rows from user_tables:
SQL> select table_name, num_rows from user_tables where rownum <= 10;
TABLE_NAME NUM_ROWS
-------------------- ----------
EMP 14
DEPT 4
BONUS 0
SALGRADE 5
DUMMY 1
TBL_ERROR 1
AUDIT_TAB 2
SOURCE_DET 3
EXAMPLE 1
FLIGHT 0
10 rows selected.
SQL>
It can't be much worse than your attempt (due to possible frequent changes to tables' contents; inserts and deletes) because you'd collect your own data periodically (how often?) anyway.
If it has to be your way, then you'd use dynamic SQL, looping through all tables in your schema and actually count number of rows:
SQL> create table test_table
2 (table_name varchar2(30),
3 num_rows number);
Table created.
SQL> create or replace procedure p_test as
2 l_cnt number;
3 begin
4 execute immediate 'truncate table test_table';
5 for cur_R in (select table_name from user_tables) loop
6 execute immediate 'select count(*) from ' ||cur_R.table_name into l_Cnt;
7 insert into test_table (table_name, num_rows) values (cur_r.table_name, l_cnt);
8 end loop;
9 end;
10 /
Procedure created.
Running the procedure and viewing the result:
SQL> exec p_test;
PL/SQL procedure successfully completed.
SQL> select * From test_Table where rownum <= 10;
TABLE_NAME NUM_ROWS
-------------------- ----------
EMP 14
DEPT 4
BONUS 0
SALGRADE 5
DUMMY 1
TBL_ERROR 1
AUDIT_TAB 2
SOURCE_DET 3
EXAMPLE 1
FLIGHT 0
10 rows selected.
SQL>
Note that performance will suffer as number of tables and number of rows stored in them grows.
If I were you, I'd go for the first option.
In the following minimal PL/SQL query example, I can't get the query to work, in order to filter records from a table based on a certain date, which is stored in a variable.
The goal obviously is to end up with different query results, by changing the date variable to some other moment, different from sysdate.
This query does succeed in printing the calculation date variable's content, but the select query afterwards fails with ORA-06550: line 6, column 3 (so on 'select *' below) : PLS-00428: an INTO clause is expected in this SELECT statement.
DECLARE
v_calculation_date date := sysdate;
BEGIN
dbms_output.put_line(v_calculation_date);
select *
from t
WHERE to_number(to_char(trunc(t.some_date), 'YYYYMM'))
BETWEEN to_number(to_char(add_months(v_calculation_date,-24), 'YYYYMM'))
AND to_number(to_char(add_months(v_calculation_date,-1), 'YYYYMM'));
END;
OK, with a SELECT INTO then... This again prints the calculation date variable's content just fine, but the select query afterwards again fails, with the same error:
DECLARE
v_calculation_date date;
BEGIN
select sysdate into v_calculation_date from dual; -- dual is a dummy table with only one row and one column, just what we need here.
dbms_output.put_line(v_calculation_date);
select *
from t
WHERE to_number(to_char(trunc(t.some_date), 'YYYYMM'))
BETWEEN to_number(to_char(add_months(v_calculation_date,-24), 'YYYYMM'))
AND to_number(to_char(add_months(v_calculation_date,-1), 'YYYYMM'));
END;
I already have a SELECT INTO, why is the normal SELECT, which follows afterwards, then refused?
For the sake of completeness: this is a minimal query example for easy reproduction of the problem. My actual query is more complex, I eventually want something like this to work:
DECLARE
v_calculation_date date;
BEGIN
DROP TABLE t1;
CREATE TABLE t1 as (
WITH intermediary1 AS (
-- The actual query from above, as a subquery
SELECT *
FROM t
WHERE to_number(to_char(trunc(t.some_date), 'YYYYMM'))
BETWEEN to_number(to_char(add_months(v_calculation_date,-24), 'YYYYMM'))
AND to_number(to_char(add_months(v_calculation_date,-1), 'YYYYMM'));
)
)
/* other drops, creates, inserts ... */
END
What is wrong in my query?
Execution info: the SQL dialect is PL/SQL, the IDE is PL/SQL Developer version 13.0, the database server runs Oracle Database 12c Enterprise Edition Release 12.2.
In PL/SQL, every SELECT (that isn't part of a cursor) must have an INTO clause. Therefore, it's not enough that one of your SELECTs has it.
As of your final code (the one that contains CREATE TABLE): that won't work. DDL can't be executed from PL/SQL, unless you use dynamic SQL (execute immediate). Therefore, all your "other drops, creates, ..." will also have to be dynamic.
Note that we usually do not create objects dynamically; create table at SQL level. Then, if you want to populate it with different data, do so (either from SQL or PL/SQL). First delete "old" data - you can use delete or truncate (but it is DDL so - as you already know by now - it requires execute immediate), and then re-populate the table.
Dynamic SQL is difficult to debug. Don't use it if you don't have to.
If you're concerned about your own data so that other users wouldn't interfere, consider creating a global temporary table (or private; depending on database version you use).
[EDIT, based on your comments]
I'm afraid you're misinterpreting reality. DDL can't be executed in PL/SQL unless it is dynamic SQL. Your "final" code is an anonymous PL/SQL block (I'm talking about the last code you posted, the one that begins with the DECLARE, contains DROP and CREATE table t1) so it can not execute those statements as you posted.
Here's a demo. I don't have your tables so I'll mimic what you're doing, using Scott's sample DEPT table instead on my 11gXE database.
Does the WITH factoring clause work? Yes.
SQL> with intermediary1 as (
2 -- The actual query from above, as a subquery
3 select *
4 from dept
5 where 1 = 1
6 )
7 select * from intermediary1;
DEPTNO DNAME LOC
---------- -------------- -------------
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
Does it work with the CREATE TABLE? Not on my 11gXE:
SQL> create table t1 as (
2 with intermediary1 as (
3 -- The actual query from above, as a subquery
4 select *
5 from dept
6 where 1 = 1
7 )
8 select * from intermediary1
9 );
)
*
ERROR at line 9:
ORA-32034: unsupported use of WITH clause
I tried it on apex.oracle.com which runs 18cEE - doesn't work either.
Maybe your database version supports it, can't tell as I don't have any higher database version to try it on.
But OK, doesn't matter, that's easily fixed by moving a CTE into a subquery:
SQL> create table t1 as
2 select *
3 from (-- The actual query from above, as a subquery
4 select *
5 from dept
6 where 1 = 1
7 );
Table created.
So, everything is OK with the query itself, it won't cause my PL/SQL block to fail because of it.
OK, so - let's try it, the way you say you use frequently.
SQL> declare
2 v_calculation_date date;
3 begin
4 drop table t1;
5
6 create table t1 as
7 select *
8 from (-- The actual query from above, as a subquery
9 select *
10 from dept
11 where 1 = 1
12 );
13 /* other drops, creates, inserts ... */
14 end;
15 /
drop table t1;
*
ERROR at line 4:
ORA-06550: line 4, column 3:
PLS-00103: Encountered the symbol "DROP" when expecting one of the following:
( begin case declare exit for goto if loop mod null pragma
raise return select update while with <an identifier>
<a double-quoted delimited-identifier> <a bind variable> <<
continue close current delete fetch lock insert open rollback
savepoint set sql execute commit forall merge pipe purge
SQL>
Whooops! There's an error on DROP. Let's remove it and leave CREATE TABLE:
SQL> declare
2 v_calculation_date date;
3 begin
4 -- drop table t1;
5
6 create table t1 as
7 select *
8 from (-- The actual query from above, as a subquery
9 select *
10 from dept
11 where 1 = 1
12 );
13 /* other drops, creates, inserts ... */
14 end;
15 /
create table t1 as
*
ERROR at line 6:
ORA-06550: line 6, column 3:
PLS-00103: Encountered the symbol "CREATE" when expecting one of the following:
( begin case declare exit for goto if loop mod null pragma
raise return select update while with <an identifier>
<a double-quoted delimited-identifier> <a bind variable> <<
continue close current delete fetch lock insert open rollback
savepoint set sql execute commit forall merge pipe purge
SQL>
Whoops #2! Error on CREATE TABLE. Looks like your way doesn't really work. Let's try it my way, using dynamic SQL:
SQL> declare
2 v_calculation_date date;
3 begin
4 execute immediate 'drop table t1';
5
6 execute immediate (
7 'create table t1 as
8 select *
9 from (-- The actual query from above, as a subquery
10 select *
11 from dept
12 where 1 = 1
13 )'
14 );
15 /* other drops, creates, inserts ... */
16 end;
17 /
PL/SQL procedure successfully completed.
SQL> select * from t1;
DEPTNO DNAME LOC
---------- -------------- -------------
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
SQL>
Now it works.
I'd be more than happy if you demonstrate the way you're doing it without dynamic SQL. Please, post your own SQL*Plus session (just like I did). Or, if you use some GUI, post a screenshot, no problem.
You do not need a PL/SQL block for your query and can simplify it to:
SELECT *
FROM t
WHERE ADD_MONTHS(TRUNC(SYSDATE, 'MM'), -24) <= some_date
AND some_date < TRUNC(SYSDATE, 'MM');
If you want it in a PL/SQL block then you want to use SELECT ... INTO ... if the query is going to return a single row:
DECLARE
v_calculation_date DATE := SYSDATE;
v_col1 T.COL1%TYPE;
v_col2 T.COL2%TYPE;
v_some_date T.SOME_DATE%TYPE;
BEGIN
SELECT col1, col2, some_date
INTO v_col1, v_col2, v_some_date
FROM t
WHERE ADD_MONTHS(TRUNC(v_calculation_date, 'MM'), -24) <= some_date
AND some_date < TRUNC(v_calculation_date, 'MM');
-- Do something with the variables.
DBMS_OUTPUT.PUT_LINE( v_col1 || ', ' || v_col2 || ', ' || v_some_date );
END;
/
If you can return multiple rows then use SELECT ... BULK COLLECT INTO ... or use a cursor:
DECLARE
v_calculation_date DATE := SYSDATE;
BEGIN
FOR cur IN (
SELECT *
FROM t
WHERE ADD_MONTHS(TRUNC(v_calculation_date, 'MM'), -24) <= some_date
AND some_date < TRUNC(v_calculation_date, 'MM')
)
LOOP
-- Do something with the cursor.
DBMS_OUTPUT.PUT_LINE( cur.col1 || ', ' || cur.col2 || ', ' || cur.some_date );
END LOOP;
END;
/
db<>fiddle here
Again, for your more complicated query, you do not need PL/SQL (and cannot have DDL statements in a PL/SQL context):
DROP TABLE t1;
CREATE TABLE t1 as
SELECT *
FROM t
WHERE ADD_MONTHS(TRUNC(SYSDATE, 'MM'), -24) <= some_date
AND some_date < TRUNC(SYSDATE, 'MM');
Or, if you want to use PL/SQL then do the DDL statements in the SQL context and then switch to PL/SQL for the DML statements:
DROP TABLE t1;
CREATE TABLE t1 AS
SELECT * FROM t WHERE 1 = 0;
-- or
-- TRUNCATE TABLE t1;
DECLARE
v_calculation_date DATE := SYSDATE;
BEGIN
INSERT INTO t1
SELECT *
FROM t
WHERE ADD_MONTHS(TRUNC(v_calculation_date, 'MM'), -24) <= some_date
AND some_date < TRUNC(v_calculation_date, 'MM');
END;
/
I have a procedure which receives as parameter a where clause (i.e. where col1 = 1). I am using this clause to search in some tables using an EXECUTE IMMEDIATE statement and the result to be inserted into a nested table, and than be displayed.
The procedure works fine if any data is found but in case no data is found, then the above error is thrown.
Can someone explain what cause this error, please?
Here is the procedure:
create or replace procedure prc_checks(pi_where varchar2) as
cursor c_tables is
select object_name,
case object_name
when 'XP_IMPORT_MW' THEN 99999999
when 'XP_IMPORT_MW_ARCH' THEN 99999998
else TO_NUMBER(SUBSTR(object_name, -8, 8))
end to_order
from dba_objects
where object_type = 'TABLE'
and object_name IN ('XP_IMPORT_MW', 'XP_IMPORT_MW_ARCH')
or REGEXP_LIKE (object_name, 'XP_IMPORT_MW_ARCH_201(5|6|7)[0-9]{4}') order by 2 desc;
type t_result is table of xp_import_mw%rowtype;
v_result t_result;
v_sql varchar2(300);
BEGIN
for i in c_tables
loop
v_sql := 'select * from ' || i.object_name || ' ' || pi_where;
execute immediate v_sql bulk collect into v_result;
if v_result.count > 0
then
for j in v_result.first .. v_result.last
loop
dbms_output.put_line(v_result(j).art_nr);
end loop;
dbms_output.put_line('... the required information was found on table name ' || upper(i.object_name));
exit;
end if;
end loop;
END prc_checks;
You'll get this is one of the tables being found by the cursor has fewer columns than xp_import_mw. For example:
create table xp_import_mw (col1 number, art_nr number, dummy number);
create table xp_import_mw_arch_20160102 (col1 number, art_nr number, dummy number);
create table xp_import_mw_arch_20160101 (col1 number, art_nr number);
insert into xp_import_mw_arch_20160101 values (1, 42);
So the main xp_import_mw table has three columns but no matching data. One of the old archive tables has one fewer columns.
I added a dbms_output.put_line(v_sql) to the procedure to see which table it fails against, then ran it:
set serveroutput on
exec prc_checks('where col1 = 1');
which got output:
select * from XP_IMPORT_MW where col1 = 1
select * from XP_IMPORT_MW_ARCH_20160102 where col1 = 1
select * from XP_IMPORT_MW_ARCH_20160101 where col1 = 1
Error starting at line : 49 in command -
BEGIN prc_checks('where col1 = 1'); END;
Error report -
ORA-01007: variable not in select list
ORA-06512: at "MY_SCHEMA.PRC_CHECKS", line 25
ORA-06512: at line 1
01007. 00000 - "variable not in select list"
*Cause:
*Action:
So the problem isn't that there is no data found; the problem is that there is matching data in a table which has the wrong structure.
You could construct the select list based on the xp_import_mw table's structure, instead of using *; that won't stop it failing, but would at least give you a slightly more helpful error message - in this case ORA-00904: "DUMMY": invalid identifier instead of ORA-01007.
You could do a quick and crude check for discrepancies with something like:
select table_name, count(column_id) as column_count,
listagg(column_name, ',') within group (order by column_id) as columns
from dba_tab_columns
where table_name IN ('XP_IMPORT_MW', 'XP_IMPORT_MW_ARCH')
or REGEXP_LIKE (table_name, 'XP_IMPORT_MW_ARCH_201(5|6|7)[0-9]{4}')
group by table_name
having count(column_id) != (
select count(column_id) from dba_tab_columns where table_name = 'XP_IMPORT_MW'
);
... although if you're using dba_* or all_* view you should really be including the owner, here and in your procedure.
I have a table with a few hundred partitions and I am generally interested on the latest 35.
Accordingly I am trying to create views which would access these dynamically. i.e. always use the latest in case ones are created.
The query:
select PARTITION_NAME,
PARTITION_POSITION,
NUM_ROWS,
AVG_ROW_LEN
from all_tab_partitions
where
table_name = 'MY_TABLE'
AND PARTITION_NAME <> 'P_LAST'
AND PARTITION_POSITION < (SELECT MAX(PARTITION_POSITION)
FROM all_tab_partitions) - 35
order by 2 DESC
;
Seems to return me the partition names I'm interested, however, I don't manage to use it's results to select the partitions. e.g.:
CREATE OR REPLACE VIEW MY_VIIEW AS
WITH t AS ( [Above query] )
SELECT * FROM
MY_TABLE PARTITION (SELECT /*+ FIRST_ROWS(1) */ PARTITION_NAME
from t);
(not the actual view, just an example)
So how do I do that? How do I create a view which will acess always the latest partition (execpt of "MAX")?
I am using Oracle 10g
thanks
You can do it using PL/SQL only
create or replace package my_table_ is
type t_records is table of my_table%rowtype;
function getpart(c_parts sys_refcursor) return t_records pipelined;
end;
create or replace package body my_table_ is
function getpart(c_parts sys_refcursor) return t_records pipelined is
v_partition all_tab_partitions.partition_name%type;
v_row my_table%rowtype;
c_mytab sys_refcursor;
begin
loop
fetch c_parts into v_partition;
exit when c_parts%notfound;
open c_mytab for 'select * from my_table partition ('||v_partition||')';
loop
fetch c_mytab into v_row;
exit when c_mytab%notfound;
pipe row (v_row);
end loop;
end loop;
end;
end;
Now you can
select * from table(my_table_.getpart(cursor(<QUERY_RETURNING_PARTITION_NAMES>)));
May be you can construct view's query using batch of union all statements with partition name in each statement, e.g.
create view p as
select * from my_table partition (part1)
union all
select * from my_table partition (part1)
...
union all
select * from my_table partition (part35)
Ok... I don't think your can use the Partition-Names, but you can use the Starting-Values of the Partitions to select the Data matching these Partitions...
So you View would look like this:
SELECT * FROM my_table WHERE date_col > get_part_limit( 'my_table', 35 ):
Where date_col is the column you use for partitioning - and get_part_limit is a stored function you write like this:
...
BEGIN
SELECT high_value FROM all_tab_partitions
INTO local_var
WHERE table_name = parameter_name
AND PARTITION_POSITION = MAX... - 35
EXECUTE IMMEDIATE 'SELECT '||local_var||' FROM DUAL' INTO local_return_value;
RETURN local_return_value;
END;
partitions are designed to be transparent for the data, so when you write a query, you simply don't know how your data is stored.
I see only one possibility to hit a particular partition: your WHERE clause should match values to the partitioned columns of latest (or latest 5) partition.
Next question is to build this WHERE clause on the fly. You already know that there is plenty of information in oracle dictionary. So you will read that and create a constructor to convert metadata conditions back into SQL.
irl we do exactly the same thing and use falco's solution like.
Here is my code:
create or replace function longToDate( myOwner varchar2,
mytable_name in varchar2,
mypartition_name in varchar2
) return date
as
cDate date;
cvar varchar2(1024);
rq varchar2(1024);
infiniteValue EXCEPTION;
PRAGMA EXCEPTION_INIT(infiniteValue, -00904);
begin
select high_value into cvar FROM dba_tab_partitions t where t.table_owner=myOwner and table_name=mytable_name and partition_name=mypartition_name;
rq:='select '||cvar||' from dual';
execute immediate rq into cDate;
return cdate;
EXCEPTION
WHEN infiniteValue
then return'01 jan 3000';
when others
then return null;
end longToDate;
Ant the view is something like this
create or replace view last_35 as
with maxdate as
(select longToDate(p.table_owner,p.table_name,p.partition_name) mydate,
rank()over(order by p.partition_position desc) mypos,
p.* from all_tab_partitions p
where p.table_name='MY_TABLE'
)
select /*+full(a)*/* from MY_TABLE a, maxdate
where MY_TABLE.partition_name>maxdate.mydate
and maxdate.mypos=35
I have a table with 2 varchar2 columns. I have added new number column to existing table to make this column primary key. This table now includes 3 columns. I gave a try to use anonymous block as following:
declare
cnt number;
begin
select nvl(count(*),0) into cnt from sometable;
for i in 1..cnt
loop
update sometable set id=i where i=rownum;
end loop;
end;
Using this anonymous block updates this table unexpectedly.
My solution was to use the following statement:
create table sometablecopy as select row_number() over(order by sometable.col1) as id, sometable.* from sometable;
Nevertheless I am curios why doesn't anonymous block produce expected primary key values with the help of rownum pseudocolumn? It must be rownum related issue.
Rownum is a pseudocolumn. Its assigned to rows as they are returned from the select. So you can't say "select * from my_table where rownum = 42" since the row with rownum=42 hasn't been defined yet, it will vary depending on your select and predicate (and "select * from my_table where rownum = 1" will return a single row, not the "first" row, whatever that would be). You could do something like (untested):
declare
cursor sel_cur is
select rowid as row_id
from my_table
order by orderby_col;
v_ctr pls_integer := 0;
begin
for rec in sel_cur
loop
v_ctr := v_ctr + 1;
update my_table set pk_col = v_ctr where rowid = rec.row_id;
end loop;
commit;
exception
when others then
rollback;
raise;
end;
This assumes you have sufficient rollback to update the entire table.
Hope that helps.
You cannot use ROWNUM like that (see ROWNUM in SQL).
What you could have done is this:
UPDATE sometable SET id = ROWNUM;