The nature of my application involves daily deleting and bulk inserting of large datasets into an Oracle 12c database. My tables are interval-partitioned by a date field and partitioned-indexed. I use a stored procedure to gather statistics for the affected partitions after each run. Lately, I found that the runs have been slowing down considerably and was wondering if this was due to the increasing size of the database.
I have searched for how to calculate the total disk space that my tables use and usually arrive at this:
select sum(bytes)/1024/1024/1024
from dba_segments
where owner='SCHEMA' and segment_name in ('TABLE_A', 'TABLE_B');
However, the numbers were huge and do not reflect the actual data volume used. When we exported the tables for restoration to another database, the file was much smaller than that query suggests. I dug deeper and arrived at this query instead:
select partition_name,
blocks*8/1024 size_m,
num_rows*avg_row_len/1024/1024 occ_m,
blocks*8/1024 - num_rows*avg_row_len/1024/1024 wast_m
from dba_tab_partitions
where table_name='TABLE_A';
This query suggests that there is a "wasted" space concept where after performing bulk inserts and deleting the data before it is replaced again, the space used is not reclaimed.
Thus I have the following questions:
Does the "wasted" space contribute to performance degradation when I
perform delete from table where ..?
Is there a difference between
performing a delete from table where .. as compared to dropping
the partitions with regard to "wasted" space?
Is performing table reorganization / defragmentation on a regular basis to reclaim table space a recommended practice?
Does the "wasted" space contribute to performance degradation when I perform delete from table where ..?
Yes, you are deleting from table Oracle has to to perform Full Tabl Scan/Index Range Scan(Index leaf node may lead to empty blocks) on the underlying table up to High Water Mark, which makes your delete slow.
Is there a difference between performing a delete from table where .. as compared to dropping the partitions with regard to "wasted" space?
Deleting is a slow process. It has to create before images(undo), update indexes, write redo logs and remove the data. Since DDL(Drop) doesnt generate redo/undo(Generate tiny bit of undo/redo for meta data) it would be faster than DML(delete).
Is performing table reorganization/defragmentation on a regular basis to reclaim table space a recommended practice?
Objects with fragmented free space can result in much wasted space, and can impact database performance. The preferred way to defragment and reclaim this space is to perform an online segment shrink.
For details:Reclaiming Unused Space
The following blog post demostrate the performance impact during DML becuase of wasted space and how to get rid of it.
Defragmentation Can Degrade Query Performance
If you're doing deletes or updates your space is getting fragmented. You can read about it in documentation.
To improve your process you can either perform some cleaning operations like shrink or just recreate tables on some big inserts. I mean instead of doing delete and insert do create table as select from old where rows not to delete and then insert new set into new table. After that just swap names and drop old table.
With your second question I think answer is here. Dropping partition will reduce HWM and delete will not.
This query suggests that there is a "wasted" space concept where after performing bulk inserts and deleting the data before it is replaced again, the space used is not reclaimed.
This is correct.
A direct path insert uses space above the high water mark for the segment. Subsequent deletes remove rows, but do not reset the high water mark.
It would be best to be able to truncate the segment prior to performing another direct path insert, as this resets the high water mark as well as removing all the rows.
Related
newb here, We have an old Oracle 10g instance that they have to keep alive until it is replaced. The nightly jobs have been very slow causing some issues. Every other Week there is a large process that does large amounts of DML (deletes, inserts, updates). Some of these tables have 2+ million rows. I noticed that some of the tables the HWM is higher than expected and in Toad I ran a database advisor check that recommended shrinking some tables, but I am concerned that the tables may need the space for DML operations or will shrinking them make the process faster or slower?
We cannot add cpu due to licensing costs
If you are accessing the tables with full scans and have a lot of empty space below the HWM, then yes, definitely reorg those (alter table move). There is no downside, only benefit. But if your slow jobs are using indexes, then the benefit will be minimal.
Don't assume that your slow jobs are due to space fragmentation. Use ASH (v$active_session_history) and SQL monitor (v$sql_plan_monitor) data or a graphical tool that utilizes this data to explore exactly what your queries are doing. Understand how to read execution plans and determine whether the correct plan is being used for your data. Tuning is unfortunately not a simple thing that can be addressed with a question on this forum.
In general, shrinking tables or rebuilding indexes should speed up reads of the table, or anything that does full table scans. It should not affect other DML operations.
When selecting or searching data, all of the empty blocks in the table and any indexes used by the query must still be read, so rebuilding them to reduce empty space and lower the high water mark will generally improve performance. This is especially true in indexes, where space lost to deleted rows is not recovered for reuse.
I am very new to oracle and I have below query.
I have one table which has almost 6 L records.
In daily batch i need to delete almost 5.7 L record and insert it again in from another table. Note that i can not use truncate table because 30000 records are constant one which I should not delete.
Issue here is if I delete daily 5.67 L record, it may cause for High WaterMark issue.
So my query is can Gather Stats helps to reduce HWM?
I can do Oracle Gather Stats Daily.
You can use the SHRINK command to recover the space and reset the high water mark:
alter table your_table shrink space;
However, you should only do this if you need to. In your case it seems likely you need only do this if you are inserting your 567,000 records using the /* APPEND */ hint; this hint tells the optimiser to insert records above the HWM, which in your scenario would cause the table to grow, with vast amounts of empty space. Shrinking would definitely be useful here.
If you're just inserting records without the hint then they will mainly reuse the empty space vacated by the prior deletion, so you don't need to concern yourself with the HWM.
Incidentally, deleting and re-inserting 5.67L records every day sounds rather poor practice. There are probably better solutions (such as MERGE), depending on the underlying business rules you're trying to satisfy.
If you reinsert the same amount of data, the table should be roughly the same size. Therefore, I would not worry about the high water mark too much.
Having said that, it is good practice to gather statistics if the data has changed substantially, so I would recommend that.
I have a problem with hard disk space lack.
so I delete all records from a big table in my oracle database.
But my hard disk space is not changed.
How can I fix it?
Try ALTER TABLE table_name SHRINK SPACE CASCADE; and rebuild your indexes.
Deleting all rows from a table does not reset the high water mark and does not free up any space. This is because with DML operations Oracle assumes the space will be re-used later and doesn't bother to release it.
You probably want to run a TRUNCATE statement instead. It's also much faster than a DELETE as it doesn't generate any REDO or UNDO. But beware that truncating a table is a DDL statement that cannot be rolled back.
TRUNCATEing tables releases space to the tablespace and that space is available to other objects. But it still doesn't reduce the amount of operating system space used. Lowering disk space requires shrinking datafiles.
Shrinking datafiles can be tricky. Datafiles can only be shrunk to the last extent. It's possible for a datafile to have a ton of empty space but not be shrinkable, if a single block of data is at the end of the file. In that case the datafile must be defragmented. That can be done by Oracle Enterprise Manager and you can find many other scripts to do it. They usually involve moving or rebuilding all the objects.
The simplest way to save some space is to try to shrink every file. This won't free up all available space, but in practice most of the free space is is usually after the last extent and can be reclaimed.
Run the below script, it will generate a bunch of alter statements to try shrink the datafiles by a large amount. Most of the statements will fail but that's OK. Cut the number in half and try again, repeatedly. This is a "dumb" way to do it, but it can usually free up some space quickly.
with decrease_size as (select 64 gb from dual)
select 'alter database datafile '''||file_name||''' resize '||to_char(round(bytes/1024/1024/1024)-(select gb from decrease_size))||'g;'
from dba_data_files
where bytes > (select gb from decrease_size) * 1024*1024*1024
order by 1 desc;
I have a table in my oracle db with 100 columns. 50 columns in this table are not used by the program accessing this table. (i.e. the select queries only select the relevant columns and NOT using '*')
My question is this :
If I recreate the same table with only the columns I need will it improve queries performance using the same query I used with the original table (remember that only the relevant columns are selected)
It is well worth mentioning the the program makes these queries a reasonable amount of times per second!
P.S. :
This is an existing project I am working on and the table design was made a long time ago for other products as well (thats why we have unused columns now)
So the effect of this will be that the average row will be smaller, if the extra columns have got data that will no longer be in the table. Therefore the table can be smaller, and not only will it use less space on disk it will use less memory space in the SGA, and caching will be more efficient.
Therefore, if you access the table via a full table scan then it will be faster to read the segment, but if you use index-based access mechanisms then the only performance improvement is likely to be through an improved chance of fetching the block from cache.
[Edited]
This SO thread suggests "it always pulls a tuple...". Hence, you are likely to see some performance improvement, not sure major or minor, as already mentioned.
In my application we are uploaded huge data.if any wrongly uploaded data then we can delete the data.So,whatever data deleted that space when will reclaim the data?.is there any impact on performance Is there anyway to reclaim the space after a large delete in oracle?.How to reclaim the space?
In Oracle, deleting data does not automatically reclaim disk space. The database will retain its storage until you do something administrative to the tablespace. The expectation of this behavior is that if you needed the storage at one time, you will likely need it again and it would therefore be more efficient to simply keep the allocation.
As far as performance impact, less data to process will generally make queries go faster. :)
To reclaim the space, you have a few choices. This article from ORACLE-BASE has a pretty comprehensive look at this situation.
Also, why would you insert data, then determine it is "bad" to then delete it? Wouldn't you be better off avoiding putting the data in from the beginning?
You need to research the High Water Mark (HWM).
Basically it is the maximum number of blocks that has ever been used by a specific table, if you load a large volume of data them you may well increase the HWM, deleting those records does not then reduce the HWM.
Here is a great article on how to adjust the HWM and if, once you understand it, you think it may be affecting your environment then use the tips included to reduce your HWM.
Hope it helps...
There is no way to reclaim space after delete. but if after the delete the size of the remaining data is relatively small then create a new table with the data, drop the old table and rename the new table to the old name
I know it is an old question, but for future references:
DELETE empty the table but space is already "locked" for future fillings
TRUNCATE TABLE do the trick (you need to disable all foreign keys referencing the table in order to work).