We are using a one-time merge query in oracle 12c which fetches around "22633334" records of data and updates it in the target table, but every time when the query run's it throws "ORA-01652: unable to extend temp segment by 64 in tablespace TEMP" issue, the DBA has already extended it to 60 GB. Can anyone tell me how this can be resolved or what would be the ideal temp space to be allocated for this volume of data?
Oracle uses that space to build temporary result sets and perform sorts. The amount of space required depends on the precise explain plain of your query (check it out and it should give you some clues), but could be several times the actual amount of data in the tables if there are joins. The number of records matters not at all; it's about the size of the records and what Oracle is trying to do with them. You want to avoid full table scans, unnecessary sorts, and that sort of thing. Make sure your table statistics are up to date. In the end, it needs what it needs and all you can do is increase your TEMP tablespace size until it works.
Related
I have a same query running on two different DB servers with almost identical config. Query is doing Full Table scan(FTS) on one table
SELECT COUNT (1) FROM tax_proposal_dtl WHERE tax_proposal_no = :b1 AND taxid != :b2 AND INSTR(:b3 , ',' || STATUS || ',' ) > 0 ;
While on 1st DB I get result in less than 3 secs with 0 disk read while on 2nd DB disk read is high and elapsed time is approx 9 secs
Only difference between the table config on two DBs is that on 1st Table has Cache = 'Y' while on 2nd Cache = 'N'. As per my understanding is that in case of FTS cache wont be used and direct path read will be used. so, why is the performance of same query is impacted by cache/nocache(Because that is the only difference between two envs and even the execution plan is same).
As suggested by Jon and after doing further research on this topic(Specially with regards to _SMALL_TABLE_THRESHOLD), I am adding more details.
Current version: Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit
Details of 2nd DB:
Total block count of table from DBA_SEGMENTS = 196736
Details of 1st DB:
Total block count of table from DBA_SEGMENTS = 172288
Execution plan on both the DBs are same but there are two major differences :
a) On 2nd DB cache option is false on the table(I tried alter table cache but still no impact on performance)
b) On 2nd DB because _STT parameter is 23920 so as per 5*_STT rule table will not be qualified as medium sized table while on 1st DB _STT parameter is 48496 so as per 5*_STT rue table will be qualified as medium sized table.
Below is a chart based on my research till now on _STT an Cache parameter of how system will behave for different table size.
Please let me know if my understanding is correct in assuming that Cache option will have no impact on Medium or Large sized table but it will help in retaining small sized table longer in LRU. So based on above assumptions and chart presented I am concluding that in the case of 2nd DB Table is classified as Large sized table and hence DPR and more elapsed time while in the case of 1st it is classified as medium sized table and hence cache read and less elapsed time.
As per this link I have set the _STT parameter on session on 2nd DB
alter session set "_small_table_threshold"=300000;
So, performance has improved considerably and almost same as 1st DB with 0 disk reads, as this implies that table will be considered Small sized.
I have used following articles in my research.
https://jonathanlewis.wordpress.com/2011/03/24/small-tables/
https://hoopercharles.wordpress.com/2010/06/17/_small_table_threshold-parameter-and-buffer-cache-what-is-wrong-with-this-quote/?unapproved=43522&moderation-hash=be8d35c5530411ff0ca96388a6fa8099#comment-43522
https://dioncho.wordpress.com/tag/full-table-scan/
https://mikesmithers.wordpress.com/2016/06/23/oracle-pinning-table-data-in-the-buffer-cache/
http://afatkulin.blogspot.com/2012/07/serial-direct-path-reads-in-11gr2-and.html
http://afatkulin.blogspot.com/2009/01/11g-adaptive-direct-path-reads-what-is.html
The keywords CACHE and NOCACHE are a bit misleading - they don't simply enable or disable caching, they only make cache reads more or less likely by changing how the data is stored in the cache. Like most memory systems, the Oracle buffer cache is constantly adding new data and aging out old data. The default, NOCACHE, will still add table data from full table scans to the buffer cache, but it will mark it as the first piece of data to age out.
According to the SQL Language Reference:
CACHE
For data that is accessed frequently, this clause indicates
that the blocks retrieved for this table are placed at the most
recently used end of the least recently used (LRU) list in the buffer
cache when a full table scan is performed. This attribute is useful
for small lookup tables.
...
NOCACHE
For data that is not accessed
frequently, this clause indicates that the blocks retrieved for this
table are placed at the least recently used end of the LRU list in the
buffer cache when a full table scan is performed. NOCACHE is the
default for LOB storage.
The real behavior can be much more complicated. The in-memory option, result caching, OS and SAN caching, direct path reads (usually for parallelism), the small table threshold (where Oracle doesn't cache the whole table if it exceeds a threshold), and probably other features I can't think of may affect how data is cached and read.
Edit: I'm not sure if I can add much to your analysis. There's not a lot of official documentation around these thresholds and table scan types. Looks like you know as much about the subject as anyone else.
I would caution that this kind of full table scan optimization should only be needed in rare situations. Why is a query frequently doing a full table scan of a 1GB table? Isn't there an index or a materialized view that could help instead? Or maybe you just need to add more memory if you need the development environment to match production.
Another option, instead of changing the small table threshold, is to change the perceived size of the table. Modify the statistics so that Oracle thinks the table is small. This way no other tables are affected.
begin
dbms_stats.set_table_stats(ownname => user, tabname => 'TAX_PROPOSAL_DTL', numblks => 999);
dbms_stats.lock_table_stats(ownname => user, tabname => 'TAX_PROPOSAL_DTL');
end;
/
I have a friend who has a website and asked me for help.
I often use MySQL databases but never Oracle databases.
And unfortunately he has an Oracle database, so I can't find a solution.
The available disk space is slowly decreasing... I delete a lot of lines from the table but that doesn't solve his problem.
The database continues to take up disk space slowly.
I read that LOB files do not return disk space, even if you delete data.
How can I reorganize LOB files easily with a simple request?
(or/and) How can I recover disk space on Oracle?
SELECT DISTINCT VERSION FROM PRODUCT_COMPONENT_VERSION
12.1.0.1.0
The BLOB column exists within the table blocks along with data even after deletion. It is only marked as unused. You can use the following command to free up space from the BLOB table:
ALTER TABLE <YOUR_TABLE_NAME> MODIFY
LOB <LOB_COLUMN_NAME>
( SHRINK SPACE );
Now, Table must have released some space and it is now available to be used within the tablespace.
Further, you can just alter the data file and reduce the size of the data file accordingly to free up space from Disk. (Note: Space allocated to the data file will not be automatically reduced. It must be done manually)
Cheers!!
In my business case, I need insert one row and can't use batch insert . So I want to know what the throughput can made by Oracle. I try these ways:
Effective way
I use multi-thread, each thread owns one connection to insert data
I use ssd to store oracle datafile
Ineffective way
I use multi table to store data in one schema
I use table partition
I use multi schema to store data
Turn up data file block size
Use append hint in insert SQL
In the end the best TPS is 1w/s+
Other:
Oracle 11g
Single insert data size 1k
CPU i7, 64GB memory
Oracle is highly optimized for anything from one row inserts to batches of hundreds of rows. You do not mention whether you are having performance problems with this one row insert nor how long the insert takes. For such a simple operation, you don't need to worry about any of those details. If you have thousands of web-based users inserting one row into a table every minute, no problem. If you are committing your work at the appropriate time, and you don't have a huge number of indexes, a single row insert should not take more than a few milliseconds.
In SQL*Plus try the commands
set autotrace on explain statistics
set timing on
and run your insert statement.
Edit your question to include the results of the explain plan. And be sure to indent the results 4 spaces.
When I do a bulk insert in an oracle db, (e.g 1 million rows), before starting the insert ,does oracle calculate how much space is required for the new rows and somehow 'block' that space so no other process can insert anything in that space?
No. You'll run into ORA-01653: unable to extend table xxx by yyy in tablespace zzz
You'd have to compute this manually and create large enough initial and/or next extents using the storage clause.
Refer to the Oracle documentation of Conventional and Direct Path Loads
Answer discussed by ASKTOM: A Must Read
I need to perform a query 2.5 million times. This query generates some rows which I need to AVG(column) and then use this AVG to filter the table from all values below average. I then need to INSERT these filtered results into a table.
The only way to do such a thing with reasonable efficiency, seems to be by creating a TEMPORARY TABLE for each query-postmaster python-thread. I am just hoping these TEMPORARY TABLEs will not be persisted to hard drive (at all) and will remain in memory (RAM), unless they are out of working memory, of course.
I would like to know if a TEMPORARY TABLE will incur disk writes (which would interfere with the INSERTS, i.e. slow to whole process down)
Please note that, in Postgres, the default behaviour for temporary tables is that they are not automatically dropped, and data is persisted on commit. See ON COMMIT.
Temporary table are, however, dropped at the end of a database session:
Temporary tables are automatically dropped at the end of a session, or
optionally at the end of the current transaction.
There are multiple considerations you have to take into account:
If you do want to explicitly DROP a temporary table at the end of a transaction, create it with the CREATE TEMPORARY TABLE ... ON COMMIT DROP syntax.
In the presence of connection pooling, a database session may span multiple client sessions; to avoid clashes in CREATE, you should drop your temporary tables -- either prior to returning a connection to the pool (e.g. by doing everything inside a transaction and using the ON COMMIT DROP creation syntax), or on an as-needed basis (by preceding any CREATE TEMPORARY TABLE statement with a corresponding DROP TABLE IF EXISTS, which has the advantage of also working outside transactions e.g. if the connection is used in auto-commit mode.)
While the temporary table is in use, how much of it will fit in memory before overflowing on to disk? See the temp_buffers option in postgresql.conf
Anything else I should worry about when working often with temp tables? A vacuum is recommended after you have DROPped temporary tables, to clean up any dead tuples from the catalog. Postgres will automatically vacuum every 3 minutes or so for you when using the default settings (auto_vacuum).
Also, unrelated to your question (but possibly related to your project): keep in mind that, if you have to run queries against a temp table after you have populated it, then it is a good idea to create appropriate indices and issue an ANALYZE on the temp table in question after you're done inserting into it. By default, the cost based optimizer will assume that a newly created the temp table has ~1000 rows and this may result in poor performance should the temp table actually contain millions of rows.
Temporary tables provide only one guarantee - they are dropped at the end of the session. For a small table you'll probably have most of your data in the backing store. For a large table I guarantee that data will be flushed to disk periodically as the database engine needs more working space for other requests.
EDIT:
If you're absolutely in need of RAM-only temporary tables you can create a table space for your database on a RAM disk (/dev/shm works). This reduces the amount of disk IO, but beware that it is currently not possible to do this without a physical disk write; the DB engine will flush the table list to stable storage when you create the temporary table.