I am using Postgres 8.4
I need to execute an ALTER statement on a running database with ~4M data on the relevant table. My sql is like:
ALTER TABLE some_table ALTER a_row bigint;
Now, relevant row type is int
But what i wonder is data consistency, Nearly 3-4 records are written to that table and some more are being read per second.
What i need to do for avoiding data consistency and such other problems.
When you execute and ALTER TABLE sql, table will be locked and you shouldn't have any problems except some possible performance issues in INSERT sqls in your case. But if you are going to do this once, there is no reason to hesitate.
Related
Forgive me to ask a silly question.
Would a temporary table be dropped automatically in Oracle (12c)?
Yesterday I have executed the following DDL to create a temporary table:
Create global temporary table my_1st_t_table on commit preserve rows as
select
*
from
other_table
where
selected_col = 'T';
After that I have executed following statements:
commit;
select count(*) from my_1st_t_table;
Yesterday, the last select statement returned 2000 rows.
After that I disconnected my VPN and also switched off my client laptop.
Today I rerun the last select statement after restarted my computer and reconnected to the VPN.
It returned 0 rows. So this means the table was still there but just all rows being deleted after my session.
However, may I ask when will my temporary table be dropped?
Thanks in advance!
A temporary table in Oracle is much different than a temp table in other database platforms such as MS SQL Server, and the "temporary" nomenclature invariably leads to confusion.
In Oracle, a temporary table just like other tables, and does not get "dropped". However, the rows in the table only exist within the context of the session that inserted the rows. Once the session is terminated, assuming the session did not delete the rows, Oracle will delete the rows in the table for that session.
So bottom line, the data is temporary, the table structure is permanent, until the table is dropped.
I have a requirement to populate an audit column with current timestamp only if there are any updates to the table. Here is the trigger. Trigger works fine
create or replace TRIGGER test.Audit_Trigger
BEFORE UPDATE ON test.TEST_TABLE
FOR EACH ROW
BEGIN
:NEW.column_dtm := current_timestamp;
END;
Instead of adding same trigger for every table (around 1000 tables means 1000 triggers) with only change in table name, is there any other better way to accomplish this task?
It would be nice if you could write a schema level trigger to do this, but unfortunately Oracle only supports schema level triggers for DDL, not for DML.
You could generate triggers on each table quite easily using dynamic SQL, but assuming your DB version is reasonably recent (9i or later I think), a better alternative might be to talk to your DBA about turning on fine grained auditing for table updates.
https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_4007.htm
I need to delete a large amount of data from my database on a regular basis. The process generates huge volume of archive logs. We had a database crash at one point because there was no storage space available on archive destination. How can I avoid generation of logs while I delete data?
The data to be deleted is already marked as inactive in the database. Application code ignores inactive data. I do not need the ability to rollback the operation.
I cannot partition the data in such a way that inactive data falls in one partition that can be dropped. I have to delete the data with delete statements.
I can ask DBAs to set certain configuration at table level/schema level/tablespace level/server level if needed.
I am using Oracle 11g.
What proportion of the data on the table would be deleted, what volume? Are there any referential integrity constraints to manage or is this table childless?
Depending on the answers , you might consider:
"CREATE TABLE keep_data UNRECOVERABLE AS SELECT * FROM ... WHERE
[keep condition]"
Then drop the original table
Then rename keep_table to original table
Rebuild the indexes (again with unrecoverable to prevent redo),constraints etc.
The problem with this approach is it's a multi-step DDL, process, which you will have a job to make fault tolerant and reversible.
A safer option might be to use data-pump to:
Data-pump expdp to extract the "Keep" data
TRUNCATE the table
Data-pump impdp import of data from step 1, with direct-path
At this point I suggest you read the Oracle manual on Data Pump, particularly the section on Direct Path Loads to be sure this will work for you.
MY preferred option would be partitioning.
Of course, the best way would be TenG solution (CTAS, drop and rename table) but it seems it's impossible for you.
Your only problem is the amount of archive logs and database crash problem. In this case, maybe you could partition your delete statement (for example per 10.000 rows).
Something like:
declare
e number;
i number
begin
select count(*) from myTable where [delete condition];
f :=trunc(e/10000)+1;
for i in 1.. f
loop
delete from myTable where [delete condition] and rownum<=10000;
commit;
dbms_lock.sleep(600); -- purge old archive if it's possible
end loop;
end;
After this operation, you should reorganize your table which is surely fragmented.
Alter the table to set NOLOGGING, delete the rows, then turn logging back on.
I have read that one should not analyze a temp table, as it screws up the table statistics for others. What about an index? If I put an index on the table for the duration of my program, can other programs using the table be affected by that index?
Does an index affect my process, and all other processes using the table?
or Does it affect my process alone?
None of the responses have been authoritative, so I am offering said bribe.
Does an index effect my process, and all other processes using the table? or Does it effect my process alone?
I'm assuming we are talking of GLOBAL TEMPORARY tables.
Think of a temporary table as of multiple tables that are created and dropped by each process on the fly from a template stored in the system dictionary.
In Oracle, DML of a temporary table affects all processes, while data contained in the table will affect only one process that uses them.
Data in a temporary table is visible only inside the session scope. It uses TEMPORARY TABLESPACE to store both data and possible indexes.
DML for a temporary table (i. e. its layout, including column names and indexes) is visible to everybody with sufficient privileges.
This means that existence of the index will affect your process as well as other processes using the table in sense that any process that modifies data in the temporary table will also have to modify the index.
Data contained in the table (and in the index too), on the contrary, will affect only the process that created them, and will not even be visible to other processes.
IF you want one process to use the index and another one not to use it, do the following:
Create two temporary tables with same column layout
Index on one of them
Use indexed or non-indexed table depending on the process
I assume you're referring to true Oracle temporary tables and not just a regular table created temporarily and then dropped. Yes, it is safe to create indexes on the temp tables and they will be used according to the same rules as a regular tables and indexes.
[Edit]
I see you've refined your question, and here's a somewhat refined answer:
From:
Oracle® Database Administrator's Guide
10g Release 2 (10.2)
Part Number B14231-02
"Indexes can be created on temporary tables. They are also temporary and the data in the index has the same session or transaction scope as the data in the underlying table."
If you need the index for efficient processing during the scope of the transaction then I would imagine you'll have to explicitly hint it in the query because the statistics will show no rows for the table.
You're asking about two different things, indexes and statistics.
For indexes, yes, you can create indexes on the temp tables, they will be maintained as per usual.
For statistics, I recommend that you explicitly set the stats of the table to represent the average size of the table when queried. If you just let oracle gather stats by itself, the stats process isn't going to find anything in the tables (since by definition, the data in the table is local to your transaction), so it will return inaccurate results.
e.g. you can do:
exec dbms_stats.set_table_stats(user, 'my_temp_table', numrows=>10, numblks=>4)
Another tip is that if the size of the temporary table varies greatly, and within your transaction, you know how many rows are in the temp table, you can help out the optimizer by giving it that information. I find this helps out a lot if you are joining from the temp table to regular tables.
e.g., if you know the temp table has about 100 rows in it, you can:
SELECT /*+ CARDINALITY(my_temp_table 100) */ * FROM my_temp_table
Well, I tried it out and the index was visible and used by the second session. Creating a new global temporary table for your data would be safer if you really need an index.
You are also unable to create an index while any other session is accessing the table.
Here's the test case I ran:
--first session
create global temporary table index_test (val number(15))
on commit preserve rows;
create unique index idx_val on index_test(val);
--second session
insert into index_test select rownum from all_tables;
select * from index_test where val=1;
You can also use the dynamic sampling hint (10g):
select /*+ DYNAMIC_SAMPLING (3) */ val
from index_test
where val = 1;
See Ask Tom
You cannot create an index on a temporary table while it is used by another session, so answer is: No, it cannot affect any other process, because it is not possible.
An existing Index affects only your current session, because for any other session the temporary table appears empty, so it cannot access any index values.
Session 1:
SQL> create global temporary table index_test (val number(15)) on commit preserve rows;
Table created.
SQL> insert into index_test values (1);
1 row created.
SQL> commit;
Commit complete.
SQL>
Session 2 (while session 1 is still connected):
SQL> create unique index idx_val on index_test(val);
create unique index idx_val on index_test(val)
*
ERROR at line 1:
ORA-14452: attempt to create, alter or drop an index on temporary table already in use
SQL>
Back to session 1:
SQL> delete from index_test;
1 row deleted.
SQL> commit;
Commit complete.
SQL>
Session 2:
SQL> create unique index idx_val on index_test(val);
create unique index idx_val on index_test(val)
*
ERROR at line 1:
ORA-14452: attempt to create, alter or drop an index on temporary table already in use
SQL>
still failing, you first have to disconnect session 1 or table has to be truncated.
Session 1:
SQL> truncate table index_test;
Table truncated.
SQL>
Now you can create the index in Session 2:
SQL> create unique index idx_val on index_test(val);
Index created.
SQL>
This index of course will be used by any session.
I am writing a data conversion in PL/SQL that processes data and loads it into a table. According to the PL/SQL Profiler, one of the slowest parts of the conversion is the actual insert into the target table. The table has a single index.
To prepare the data for load, I populate a variable using the rowtype of the table, then insert it into the table like this:
insert into mytable values r_myRow;
It seems that I could gain performance by doing the following:
Turn logging off during the insert
Insert multiple records at once
Are these methods advisable? If so, what is the syntax?
It's much better to insert a few hundred rows at a time, using PL/SQL tables and FORALL to bind into insert statement. For details on this see here.
Also be careful with how you construct the PL/SQL tables. If at all possible, prefer to instead do all your transforms directly in SQL using "INSERT INTO t1 SELECT ..." as doing row-by-row operations in PL/SQL will still be slower than SQL.
In either case, you can also use direct-path inserts by using INSERT /*+APPEND*/, which basically bypasses the DB cache and directly allocates and writes new blocks to data files. This can also reduce the amount of logging, depending on how you use it. This also has some implications, so please read the fine manual first.
Finally, if you are truncating and rebuilding the table it may be worthwhile to first drop (or mark unusable) and later rebuild indexes.
Regular insert statements are the slowest way to get data in a table and not meant for bulk inserts. The following article references a lot of different techniques for improving performance: http://www.dba-oracle.com/oracle_tips_data_load.htm
Drop the index, then insert the rows, then re-create the index.
If dropping the index doesn't speed things up enough, you need the Oracle SQL*Loader:
http://www.oracle.com/technology/products/database/utilities/htdocs/sql_loader_overview.html
Suppose you have taken eid,ename,sal,job. So create a table first as:
SQL>create table tablename(eid number, ename varchar2(20),sal number,job char(10));
Now insert data:-
SQL>insert into tablename values(&eid,'&ename',&sal,'&job');
Check this link
http://www.dba-oracle.com/t_optimize_insert_sql_performance.htm
main points to consider for your
case is to use Append hint as this
will directly append into the table
instead of using freelist. If you can afford to turn off logging than use append with nologging hint to do it
Use a bulk insert instead instead of iterating in PL/SQL
Use sqlloaded to load the data directly into the table if you are getting data from a file feed
Here are my recommendations on fast insert.
Trigger - Disable any triggers associated with a table. Enable after Inserts are complete.
Index - Drop Index and re-create it after your Inserts are complete.
Stale stats - Re-analyze table and index stats.
Index de-fragmentation - Rebuild Index if needed
Use No Logging -Insert using INSERT APPEND (Oracle only). This approach is very risky approach, no redo logs are generated therefore you can’t do a rollback - make a backup of table before you start and don't try on live tables. Check if your db has similar option
Parallel Insert: Running parallel insert will get the job faster.
Use Bulk Insert
Constraints - Not much overhead during inserts but still a good idea to check, if it is still slow after even after step 1
You can learn more on http://www.dbarepublic.com/2014/04/slow-insert.html
Maybe one of your best option is to avoid Oracle as much as possible actually.
I've been baffled by this myself, but very often a Java process can outperform many of the Oracle's utilities which either use OCI (read: SQL Plus) or will take up so much of your time to get right (read: SQL*Loader).
This doesn't prevent you to use specific hints either (like /APPEND/).
I've been pleasantly surprised each time I've turned to that kind of solution.
Cheers,
Rollo