In Oracle the space allocated for data after INSERT INTO operation is not cleaned up when deleting rows from the table. Instead, after DELETE FROM operation some "waste space" is left. So what happens when I do INSERT INTO after DELETE FROM - does it span this "waste space" or allocates new space again?
See the Oracle Concepts manual:
By default, a table is organized as a heap, which means that the
database places rows where they fit best rather than in a
user-specified order. Thus, a heap-organized table is an unordered
collection of rows. As users add rows, the database places the rows in
the first available free space in the data segment. Rows are not
guaranteed to be retrieved in the order in which they were inserted.
It is the high water mark (HWM) concept.
high water mark (HWM)
The boundary between used and unused space in a segment.
Very well explained in detail by Thomas Kyte here.
Standard Oracle table is a heap-organized table. It is a table with rows stored in no particular order.
If you want to claim the free space, then you need to reorganize the table.
Related
I am very new to oracle and I have below query.
I have one table which has almost 6 L records.
In daily batch i need to delete almost 5.7 L record and insert it again in from another table. Note that i can not use truncate table because 30000 records are constant one which I should not delete.
Issue here is if I delete daily 5.67 L record, it may cause for High WaterMark issue.
So my query is can Gather Stats helps to reduce HWM?
I can do Oracle Gather Stats Daily.
You can use the SHRINK command to recover the space and reset the high water mark:
alter table your_table shrink space;
However, you should only do this if you need to. In your case it seems likely you need only do this if you are inserting your 567,000 records using the /* APPEND */ hint; this hint tells the optimiser to insert records above the HWM, which in your scenario would cause the table to grow, with vast amounts of empty space. Shrinking would definitely be useful here.
If you're just inserting records without the hint then they will mainly reuse the empty space vacated by the prior deletion, so you don't need to concern yourself with the HWM.
Incidentally, deleting and re-inserting 5.67L records every day sounds rather poor practice. There are probably better solutions (such as MERGE), depending on the underlying business rules you're trying to satisfy.
If you reinsert the same amount of data, the table should be roughly the same size. Therefore, I would not worry about the high water mark too much.
Having said that, it is good practice to gather statistics if the data has changed substantially, so I would recommend that.
The nature of my application involves daily deleting and bulk inserting of large datasets into an Oracle 12c database. My tables are interval-partitioned by a date field and partitioned-indexed. I use a stored procedure to gather statistics for the affected partitions after each run. Lately, I found that the runs have been slowing down considerably and was wondering if this was due to the increasing size of the database.
I have searched for how to calculate the total disk space that my tables use and usually arrive at this:
select sum(bytes)/1024/1024/1024
from dba_segments
where owner='SCHEMA' and segment_name in ('TABLE_A', 'TABLE_B');
However, the numbers were huge and do not reflect the actual data volume used. When we exported the tables for restoration to another database, the file was much smaller than that query suggests. I dug deeper and arrived at this query instead:
select partition_name,
blocks*8/1024 size_m,
num_rows*avg_row_len/1024/1024 occ_m,
blocks*8/1024 - num_rows*avg_row_len/1024/1024 wast_m
from dba_tab_partitions
where table_name='TABLE_A';
This query suggests that there is a "wasted" space concept where after performing bulk inserts and deleting the data before it is replaced again, the space used is not reclaimed.
Thus I have the following questions:
Does the "wasted" space contribute to performance degradation when I
perform delete from table where ..?
Is there a difference between
performing a delete from table where .. as compared to dropping
the partitions with regard to "wasted" space?
Is performing table reorganization / defragmentation on a regular basis to reclaim table space a recommended practice?
Does the "wasted" space contribute to performance degradation when I perform delete from table where ..?
Yes, you are deleting from table Oracle has to to perform Full Tabl Scan/Index Range Scan(Index leaf node may lead to empty blocks) on the underlying table up to High Water Mark, which makes your delete slow.
Is there a difference between performing a delete from table where .. as compared to dropping the partitions with regard to "wasted" space?
Deleting is a slow process. It has to create before images(undo), update indexes, write redo logs and remove the data. Since DDL(Drop) doesnt generate redo/undo(Generate tiny bit of undo/redo for meta data) it would be faster than DML(delete).
Is performing table reorganization/defragmentation on a regular basis to reclaim table space a recommended practice?
Objects with fragmented free space can result in much wasted space, and can impact database performance. The preferred way to defragment and reclaim this space is to perform an online segment shrink.
For details:Reclaiming Unused Space
The following blog post demostrate the performance impact during DML becuase of wasted space and how to get rid of it.
Defragmentation Can Degrade Query Performance
If you're doing deletes or updates your space is getting fragmented. You can read about it in documentation.
To improve your process you can either perform some cleaning operations like shrink or just recreate tables on some big inserts. I mean instead of doing delete and insert do create table as select from old where rows not to delete and then insert new set into new table. After that just swap names and drop old table.
With your second question I think answer is here. Dropping partition will reduce HWM and delete will not.
This query suggests that there is a "wasted" space concept where after performing bulk inserts and deleting the data before it is replaced again, the space used is not reclaimed.
This is correct.
A direct path insert uses space above the high water mark for the segment. Subsequent deletes remove rows, but do not reset the high water mark.
It would be best to be able to truncate the segment prior to performing another direct path insert, as this resets the high water mark as well as removing all the rows.
If I have an Oracle table with a VARCHAR2 column with length 4000, and I have inserted a 4000 character string into the table, and then update the row with a 1 character string, do I need to do anything to make the other 3999 characters of space available for reuse, or is it automatically available for reuse?
After the update, 3999 bytes of space (assuming that 1 character = 1 byte in your database character set) is freed up in the block in which the row resides. That space will be immediately available if other rows in that block need to expand in size or if other columns in that row need to expand in size. Of course, since most databases use 8k blocks and the largest block size is 32k, it is likely that there are relatively few rows in this particular block since the original row was so large.
Oracle also tracks how full blocks are and uses that information to make them available for subsequent insert operations. The mechanics of this depend on the type of tablespace (locally or dictionary managed), the segment space management policy (automatic or manual), and table-level parameters like pctused. At a high level, though, freeing up 4k of space within a single data block will almost certainly cause the data block to be made available for future insert operations (or for update operations that cause rows in other blocks to need to be migrated to a new block or chained across multiple blocks). So the space will almost certainly be available to be reused.
As part of maintenance, we remove a considerable amount of records from one table and wish to dispose of this space for other table. The thing is that I'm checking the size of the table and it shows the same original size before the delete. What am I missing here?
To check the size, I'm looking at the dba_segments.
There are some concepts you need to know about tables before you can really understand space usage. I will try and give you the 5 second tour ...
A table is made up of extents, these can vary in size, but lets just assume they are 1MB chunks here.
When you create a table, normally 1 extent is added to it, so even though the table has no rows, it will occupy 1MB.
When you insert rows, Oracle has a internal pointer for the table known as the high water mark (HWM). Below the HWM, there are formatted data block, and above it there are unformated blocks. It starts at zero.
If you take a new table as an example, and start inserting rows, the HWM will move up, giving more and more of that first extent to be used by the table. When the extent is all used up, another one will be allocated and the process repeats.
Lets says you fill a three 1MB extents and then delete all the rows - the table is empty, but Oracle does not move the HWM back down, or free those used extents. So the table will take 3MB on disk even though it is empty, but there is 3MB of free space to be reused in it. New inserts will go into that space below the HWM until it is filled up again before the HWM is move again.
To recover the space, if your table is using ASSM, you can use the command:
alter table t1 shrink space;
If you are not using ASSM, the you need to think about a table reorg to recover the space.
If you want to "reclaim" the space, the easiest method is:
ALTER TABLE table MOVE TABLESPACE different_tablespace;
ALTER TABLE table MOVE TABLESPACE original_tablespace;
Providing you have:
Some downtime in which to do it
A second tablespace with enough space to transfer the table into.
Take a look at this site
about the table size after deleting rows.
Space is effectively reused when you delete. Your database will not show any new free
space in dba_free_space -- it will have more blocks on freelists and more empty holes in
index structures.
SELECT SUM(BYTES)
FROM DBA_SEGMENTS
WHERE SEGMENT_NAME LIKE 'YOUR TABLE NAME%';
You will get the right answer.
Scenario:
I have a table with roughly 24 million records. The table has pricing history related to individual customers and is computed daily. There are on average 6 million records for each day. Every morning a the price list is generated and a merge statement is ran to reflect the changes in their pricing.
The merge statement begins with the previous day's previous data being inserted into a variable table, that table is then merged into the actual table. The main problem is that the merge statement takes pretty long.
My real question centers around the performance of using a variable table vs physical table vs temp table. What is the best practice for large merges like this?
Thoughts
I'd consider a temp table: these have statistics which will help. A table variable is always assumed to have one row. Also, the IO can be shunted onto separate drives (assuming you have tempdb separately)
If a single transaction is not required, I'd split the MERGE too into a DELETE, UPDATE, INSERT sequence to reduce the amount of work needed in each action (which reduces the amount of rollback info needed and the amount of locking etc
Temp tables often perform better than table variables for large data sets. Additionally you can put the data into the temp table and then index it.
Check if you indexes on the tables. Indexes would be updated every time you add/delete records on that table.
Try removing the indexes before merging the records and then re-create it again after the merge.