I am looking for an SQL query to give me that kind of output:
table_owner table_name partition_name data (bytes) indexes (bytes)
MY_OWNER MY_TABLE P01 12345678 23456789
MY_OWNER MY_TABLE P02 34567890 45678901
MY_OWNER MY_TABLE P03 56789012 67890123
...
I visited How do I calculate tables size in Oracle but didn't find what I am looking for, because it is always per table, per owner/schema but never per partition.
I know that the amount in bytes may not really be representative of the reality since it could be "real bytes in disk" and/or "bytes preallocated" (and real data could only use 1% of this preallocated space), but I am open to have this value even if it is the preallocated or the real amount of bytes used.
Notes:
Using Oracle 18c Enterprise
We do NOT care about system tables, ORACLE tables, maintenance tables, etc. just focusing on tables created by myself for the data
Tablespace name is ALWAYS the same for all partitions in all tables in the same schema, and of course each tablespace name is different for each schema
No need to round in Kb, Mb, Gb or even Tb, if possible I would prefer it in bytes.
No need of pourcentage, no need of max space available
I only use partitions, NOT any subpartitions
I can use a PL/SQL block if we need to loop on each schema, table and partition
Guessing I can run this query by any user
Any idea?
Hi something like this? We use a similar query that I've modified. You would have to change your ORDER BY clause as you wish, and maybe limit it to one owner as it could take long over the whole database. For that reason we don't join to segments for size, but only look at rows and blocks.
select "OWNER", "OBJECT", "TYPE", "PARTITION_NAME", "NUM_ROWS", "GB", "PARTITION_POSITION", "DUMMY"
from ((select tp.table_owner owner, tp.TABLE_NAME object, 'TABLE' type, tp.PARTITION_NAME,ROUND(s.bytes/1024/1024/1024, 2) GB, tp.NUM_ROWS,tp.PARTITION_POSITION, 1 dummy
from dba_tab_partitions tp, dba_tables t, dba_segments s
where t.table_name=tp.table_name and t.table_name=s.segment_name )
union
(select ip.index_owner owner,ip.INDEX_NAME object, 'INDEX' type, ip.PARTITION_NAME,ROUND(s.bytes/1024/1024/1024, 2) GB,ip.NUM_ROWS, ip.PARTITION_POSITION, 2 dummy
from dba_ind_partitions ip, dba_indexes i, dba_tables t, dba_segments s
where i.index_name=ip.index_name and i.table_name=t.table_name and i.index_name=s.segment_name )
union
(select lp.table_owner owner,lp.LOB_NAME object, 'LOB' type, lp.PARTITION_NAME, 0, ROUND(s.bytes/1024/1024/1024, 2) GB,lp.PARTITION_POSITION, 3 dummy
from dba_lob_partitions lp, dba_tables t, dba_segments s
where t.table_name=lp.table_name and t.table_name=s.segment_name ))
order by 8,1,2,7;
Related
I have the following table
Create table my_source(
ID number(15) not null,
Col_1 Varchar 2(3000),
Col_2 Varchar 2(3000),
Col_3 Varchar 2(3000),
Col_4 Varchar 2(3000),
Col_5 Varchar 2(3000),
...
Col_90 Varchar 2(3000)
);
This table have 6,926,220 rows.
Now I am going to create two table based on this table.
Target1
Create table el_temp as
select
id,
Col_1,
Col_2,
Col_3,
Col_4,
Col_5,
Col_6,
Col_7,
Col_8,
Col_9,
Col_10,
Col_11,
Col_12
from
my_source;
Target2:
Create table el_temp2 as
select DISTINCT
id,
Col_1,
Col_2,
Col_3,
Col_4,
Col_5,
Col_6,
Col_7,
Col_8,
Col_9,
Col_10,
Col_11,
Col_12
from
my_source;
select count(*) from el_temp; -- 6926220
select count(*) from el_temp2; --6880832
The only difference between el_temp and el_temp2 is the "distinct" operator.
Now I got the following result from SQL Developer
It is a surprise result to me that EL_TEMP, the one with more rows have a smaller size, while the el_temp2 have less row but a bigger size.
Could anyone share me any reason and how to avoid this happen?
Thanks in advance!
The most likely cause is that the table has undergone some updates to existing rows over its lifetime.
By default, when you create a table, we reserve 10% of the space in each block for rows to grow (due to updates). As updates occur, that space is used up, so your blocks might be (on average) around 95% full.
When you do "create table as select" from that table to another, we will take those blocks and pad them out again to 10% free space, thus making it slightly larger.
If PCTFREE etc is unfamiliar to you, I've also got a tutorial video to get you started here
https://youtu.be/aOZMp5mncqA
I'm trying to insert data from PURCHASE_HIST_D to PURCHASE_HIST.
(different schemas in different servers, with DBLINK).
The target table has a lot of deleted data blocks.
This is how I'm checking the segments vs the used blocks:
-- result : 199.8743480481207370758056640625 GB
select (AVG_ROW_LEN*NUM_ROWS)/1024/1024/1024 from DBA_TABLES where TABLE_NAME='PURCHASE_HIST';
-- result: 250.7939453125 GB
select SUM(BYTES)/1024/1024/1024 from DBA_SEGMENTS where SEGMENT_NAME='PURCHASE_HIST';
which means that there are 50 GB of used blocks which can be reuse for the new data.
I'm query the same for the source table:
-- result: 21.8079682849347591400146484375
select (AVG_ROW_LEN*NUM_ROWS)/1024/1024/1024 from DBA_TABLES where TABLE_NAME='PURCHASE_HIST_D';
-- result: 27.447265625
select SUM(BYTES)/1024/1024/1024 from DBA_SEGMENTS where SEGMENT_NAME='PURCHASE_HIST_D';
The source is only 27 GB so It's looks like I dont need to add more space for the tablespace.
This is the free tablespace information:
-- result: 1889477 (Used MB) 4923 (Free MB) 1894400 (Total MB)
select
fs.tablespace_name "Tablespace",
(df.totalspace - fs.freespace) "Used MB",
fs.freespace "Free MB",
df.totalspace "Total MB",
round(100 * (fs.freespace / df.totalspace)) "Pct. Free"
from
(select
tablespace_name,
round(sum(bytes) / 1048576) TotalSpace
from
dba_data_files
group by
tablespace_name
) df,
(select
tablespace_name,
round(sum(bytes) / 1048576) FreeSpace
from
dba_free_space
group by
tablespace_name
) fs
WHERE
DF.TABLESPACE_NAME = FS.TABLESPACE_NAME
and df.TABLESPACE_NAME = 'TS_DWHDATA';
So why when I'm execute the insert (even with NOAPPEND hint) I get an error that there is no enough space in tablesapce?
-- examole of the Insert
INSERT
/*+ monitor NOAPPEND parallel(64) statement_queuing */
INTO DWH.PURCHASE_HIST
SELECT *
FROM DWH_MIG.PURCHASE_HIST_D#DWH_MIG ;
The exception:
ORA-01653: unable to extend table DWH.PURCHASE_HIST by 8192 in tablespace TS_DWHDATA
You are getting the free space (using query) from the table but the high watermark is still at the point from where you can not insert data below that.
You need to reclaim that space using the following:
alter table your_table enable row movement;
alter table your_table shrink space cascade;
Also, You will need to rebuild the indexes of the table after such an operation.
Refer Oracle documentation for reclaiming waste space.
Cheers!!
Table myfirst3 have 4 columns and 1.2 million records.
Table mtl_object_genealogy has over 10 million records.
Running the below code takes very long time. How to tune this code using with options?
WITH level1 as (
SELECT mln_parent.lot_number,
mln_parent.inventory_item_id,
gen.lot_num ,--fg_lot,
gen.segment1,
gen.rcv_date.
FROM mtl_lot_numbers mln_parent,
(SELECT MOG1.parent_object_id,
p.segment1,
p.lot_num,
p.rcv_date
FROM mtl_object_genealogy MOG1 ,
myfirst3 p
START WITH MOG1.object_id = p.gen_object_id
AND (MOG1.end_date_active IS NULL OR MOG1.end_date_active > SYSDATE)
CONNECT BY nocycle PRIOR MOG1.parent_object_id = MOG1.object_id
AND (MOG1.end_date_active IS NULL OR MOG1.end_date_active > SYSDATE)
UNION all
SELECT p1.gen_object_id,
p1.segment1,
p1.lot_num,
p1.rcv_date
FROM myfirst3 p1 ) gen
WHERE mln_parent.gen_object_id = gen.parent_object_id )
select /*+ NO_CPU_COSTING */ *
from level1;
execution plan
CREATE TABLE APPS.MYFIRST3
(
TO_ORGANIZATION_ID NUMBER,
LOT_NUM VARCHAR2(80 BYTE),
ITEM_ID NUMBER,
FROM_ORGANIZATION_ID NUMBER,
GEN_OBJECT_ID NUMBER,
SEGMENT1 VARCHAR2(40 BYTE),
RCV_DATE DATE
);
CREATE TABLE INV.MTL_OBJECT_GENEALOGY
(
OBJECT_ID NUMBER NOT NULL,
OBJECT_TYPE NUMBER NOT NULL,
PARENT_OBJECT_ID NUMBER NOT NULL,
START_DATE_ACTIVE DATE NOT NULL,
END_DATE_ACTIVE DATE,
GENEALOGY_ORIGIN NUMBER,
ORIGIN_TXN_ID NUMBER,
GENEALOGY_TYPE NUMBER,
);
CREATE INDEX INV.MTL_OBJECT_GENEALOGY_N1 ON INV.MTL_OBJECT_GENEALOGY(OBJECT_ID);
CREATE INDEX INV.MTL_OBJECT_GENEALOGY_N2 ON INV.MTL_OBJECT_GENEALOGY(PARENT_OBJECT_ID);
Your explain plan shows some very big numbers. The optimizer reckons the final result set will be about 3227,000,000,000 rows. Just returning that many rows will take some time.
All table accesses are Full Table Scans. As you have big tables that will eat time too.
As for improvements, it's pretty hard to for us understand the logic of your query. This is your data model, you business rules, your data. You haven't explained anything so all we can do is guess.
Why are you using the WITH clause? You only use the level result set once, so just have a regular FROM clause.
Why are you using UNION ALL? That operation just duplicates the records retrieved from myfirst3 ( all those values are already included as rows where MOG1.object_id = p.gen_object_id.
The MERGE JOIN CARTESIAN operation is interesting. Oracle uses it to implement transitive closure. It is an expensive operation but that's because treewalking a hierarchy is an expensive thing to do. It is unfortunate for you that you are generating all the parent-child relationships for a table with 27 million records. That's bad.
The full table scans aren't the problem. There are no filters on myfirst3 so obviously the database has to get all the records. If there is one parent for each myfirst3 record that's 10% of the contents mtl_object_genealogy so a full table scan would be efficient; but you're rolling up the entire hierarchy so it's like you're looking at a much greater chunk of the table.
Your indexes are irrelevant in the face of such numbers. What might help is a composite index on mtl_object_genealogy(OBJECT_ID, PARENT_OBJECT_ID, END_DATE_ACTIVE).
You want all the levels of PARENT_OBJECT_ID for the records in myfirst3. If you run this query often and mtl_object_genealogy is a slowly changing table you should consider materializing the transitive closure into a table which just has records for all the permutations of leaf records and parents.
To sum up:
Ditch the WITH clause
Drop the UNION ALL
Tune the tree-walk with a composite index (or materializing it)
I have 5 development schemas. And each of them have partitioned tables. We also have scripts to dynamically create partition tables (Monthly/Yearly). We have to go to DBA everytime for gathering the details over the parition tables. Our real problem is we do have a parition table with 9 partitions. Every day after a delta load operation (Updates/Deletes using a PL/SQL) also some APPEND load using SQL*Loader. This operation happens when database has the peak load. We do have some performace issues over this table.(SELECT queries)
When reported to DBA, they would say the table statistics are stale and after they do "gathering stats", magically the query works faster. I searched about this and identified some information about dynamic performance views.
So, now , I have the following Questions.
1) Can the developer generate a the list of all partitionon tables, partition name, no of records available without going to DBA?
2) Shall we identify the last analysed date of every parition
3) Also the status of the parition(index) if it usable or unusable.
Normally there is no need to identify objects that need statistics gathered. Oracle automatically gathers statistics for stale objects, unless the task has been
manually disabled. This is usually good enough for OLTP systems. Use this query to find the status of the task:
select status
from dba_autotask_client
where client_name = 'auto optimizer stats collection';
STATUS
------
ENABLED
For data warehouse systems there is also not much need to query the data dictionary for stale stats. In a data warehouse statistics need to be considered after almost
every operation. Developers need to get in the habit of always thinking about statistics after a truncate, insert, swap, etc. Eventually they will "just know" when to gather statistics.
But if you still want to see how Oracle determines if statistics are stale, look at DBA_TAB_STATISTICS and DBA_TAB_MODIFICATIONS.
Here is an example of an initial load with statistics gathering. The table and partitions are not stale.
create table test1(a number, b number) partition by list(a)
(
partition p1 values (1),
partition p2 values (2)
);
insert into test1 select 1, level from dual connect by level <= 50000;
begin
dbms_stats.gather_table_stats(user, 'test1');
dbms_stats.flush_database_monitoring_info;
end;
/
select table_name, partition_name, num_rows, last_analyzed, stale_stats
from user_tab_statistics
where table_name = 'TEST1'
order by 1, 2;
TABLE_NAME PARTITION_NAME NUM_ROWS LAST_ANALYZED STALE_STATS
---------- -------------- -------- ------------- -----------
TEST1 P1 50000 2014-01-22 NO
TEST1 P2 0 2014-01-22 NO
TEST1 50000 2014-01-22 NO
Now add a large number of rows and the statistics are stale.
begin
insert into test1 select 2, level from dual connect by level <= 25000;
commit;
dbms_stats.flush_database_monitoring_info;
end;
/
select table_name, partition_name, num_rows, last_analyzed, stale_stats
from user_tab_statistics
where table_name = 'TEST1'
order by 1, 2;
TABLE_NAME PARTITION_NAME NUM_ROWS LAST_ANALYZED STALE_STATS
---------- -------------- -------- ------------- -----------
TEST1 P1 50000 2014-01-22 NO
TEST1 P2 0 2014-01-22 YES
TEST1 50000 2014-01-22 YES
USER_TAB_MODIFICATIONS gives more specific information on table staleness.
--Stale statistics.
select user_tables.table_name, user_tab_modifications.partition_name
,inserts+updates+deletes modified_rows, num_rows, last_analyzed
,case when num_rows = 0 then null
else (inserts+updates+deletes) / num_rows * 100 end percent_modified
from user_tab_modifications
join user_tables
on user_tab_modifications.table_name = user_tables.table_name
where user_tables.table_name = 'TEST1';
TABLE_NAME PARTITION_NAME MODIFIED_ROWS NUM_ROWS LAST_ANALYZED PERCENT_MODIFIED
---------- -------------- ------------- -------- ------------- ----------------
TEST1 P2 25000 50000 2014-01-22 50
TEST1 25000 50000 2014-01-22 50
Yes, you can generate a list of partitioned tables, and a lot of related data which you would like to see, by using ALL_PART_TABLES or USER_PART_TABLES (provided you have access).
ALL_TAB_PARTITIONS can be used to get number of rows per partition, alongwith other details.
Check other views Oracle has for gathering details about partitioned tables.
I would suggest that you should analyze the tables, and possibly rebuild the indexes, every day after your data load. If your data load is affecting a lot of records in the table, and is going to affect the existing indexes, it's a good idea to proactively update the statistics for the table and index.
You can use on the system views to get this information (Check http://docs.oracle.com/cd/E18283_01/server.112/e16541/part_admin005.htm)
I had a some what similar problem and I solved it by gathering stats on stale partitions only using 11g new INCREMENTAL option.
It's the reverse approach to your problem but it might worth investigating (specifically - how oracle determines what's a "stale" partition is).
dbms_stats.set_table_prefs('DWH','FACT_TABLE','INCREMENTAL','TRUE')
I always prefer the pro active approach - meaning, gather stats on stale partition at the last step of my etl, rather then giving the developer stronger privs.
I used to query all_ tables mentioned below.
The statistics and histogram details you mention will be updated in a frequency automatically by Oracle. But when the database is busy with many loads, I have seen these operations needs to be triggered manually. We faced similar situation, so we used to force the Analyze operation after our load for critical tables. You need to have privilege for the id you use to load the table.
ANALYZE TABLE table_name PARTITION (partition_name) COMPUTE STATISTICS;
EDIT: ANALYZE no longer gather CBO stats as mentioned here
So, DBMS_STATS package has to be used.
DBMS_STATS.GATHER_TABLE_STATS (
ownname VARCHAR2,
tabname VARCHAR2,
partname VARCHAR2 DEFAULT NULL,
estimate_percent NUMBER DEFAULT to_estimate_percent_type
(get_param('ESTIMATE_PERCENT')),
block_sample BOOLEAN DEFAULT FALSE,
method_opt VARCHAR2 DEFAULT get_param('METHOD_OPT'),
degree NUMBER DEFAULT to_degree_type(get_param('DEGREE')),
granularity VARCHAR2 DEFAULT GET_PARAM('GRANULARITY'),
cascade BOOLEAN DEFAULT to_cascade_type(get_param('CASCADE')),
stattab VARCHAR2 DEFAULT NULL,
statid VARCHAR2 DEFAULT NULL,
statown VARCHAR2 DEFAULT NULL,
no_invalidate BOOLEAN DEFAULT to_no_invalidate_type (
get_param('NO_INVALIDATE')),
force BOOLEAN DEFAULT FALSE);
And until the analyze is complete, the view tables below may not produce the accurate results (Especially the last_analyzed and num_rows columns)
Note: Try replace all_ as dba_ in table names, if you have access to it, you can try them.
You can also try to get SELECT_CATALOG_ROLE for your development id you use, so that you can SELECT the data dictionary views, and this reduces the dependency over DBA over such queries.(Still DBA are the right persons for few issues!!)
Query to identify the partition table, partition name, number of rows and last Analysed date!
select
all_part.owner as schema_name,
all_part.table_name,
NVL(all_tab.partition_name,'N/A'),
all_tab.num_rows,
all_tab.last_analyzed
from
all_part_tables all_part,
all_tab_partitions all_tab
where all_part.table_name = all_tab.table_name and
all_tab.partition_name = all_tab.partition_name and
all_part.owner=all_tab.table_owner and
all_part.owner in ('SCHEMA1','SCHEMA2','SCHEMA3')
order by all_part.table_name,all_tab.partition_name;
The Below Query returns the index/table name that are UNUSABLE
SELECT INDEX_NAME,
TABLE_NAME,
STATUS
FROM ALL_INDEXES
WHERE status NOT IN ('VALID','N/A');
The Below Query returns the index/table (PARTITION) name that are UNUSABLE
SELECT INDEX_NAME,
PARTITION_NAME,
STATUS ,
GLOBAL_STATS
FROM ALL_IND_PARTITIONS
WHERE status != 'USABLE';
I have a table with 200 hash partitions, I like to monitor daily size growth (rows and size in MB) for each of the partition. For more information, another process loads / updates rows on this table on daily basis and I like to know the growth pattern.
I am not looking for overall table size script, but size of each partition.
You can check the size of your partitions with this statement:
SELECT partition_name, bytes/1024/1024 "MB"
FROM dba_segments
WHERE segment_name = 'MYTABLE'
AND segment_type = 'TABLE PARTITION';