Populating a database in PostgreSQL - performance

The following link on the PostgreSQL documentation manual http://www.postgresql.org/docs/8.3/interactive/populate.html says that to disable autocommit in postgreSQL you can simply place all insert statements within BEGIN; and COMMIT;
However I have difficulty in capturing any exceptions that may happen between the BEGIN; COMMIT; and if an error occurs (like trying to insert a duplicate PK) I have no way to explicitly call the ROLLBACK or COMMIT commands. Although all insert statements are automatically rolled back, PostgreSQL still expects an explicit call to either the COMMIT or ROLLBACK commands before it can consider the transaction to be terminated. Otherwise, the script has to wait for the transaction to time out and any statements executed thereafter will raise an error.
In a stored procedure you can use the EXCEPTION clause to do this but the same does not apply in my circumstance of performing bulk inserts. I have tried it and the exception block did not work for me because the next statement/s executed after the error takes place fails to execute with the error:
ERROR: current transaction is aborted, commands ignored until end of transaction block
The transaction remains open as it has not been explicitly finalised with a call to COMMIT or ROLLBACK;
Here is a sample of the code I used to test this:
BEGIN;
SET search_path TO testing;
INSERT INTO friends (id, name) VALUES (1, 'asd');
INSERT INTO friends (id, name) VALUES (2, 'abcd');
INSERT INTO friends (id, nsame) VALUES (2, 'abcd'); /*note the deliberate mistake in attribute name and also the deliberately repeated pk value number 2*/
EXCEPTION /* this part does not work for me */
WHEN OTHERS THEN
ROLLBACK;
COMMIT;
When using such technique do I really have to guarantee that all statements will succeed? Why is this so? Isn't there a way to trap errors and explicitly call a rollback?
Thank you

if you do it between begin and commit then everything is automatically rolled back in case of an exception.
Excerpt from the url you posted:
"An additional benefit of doing all insertions in one transaction is that if the insertion of one row were to fail then the insertion of all rows inserted up to that point would be rolled back, so you won't be stuck with partially loaded data."

When I initialize databases, i.e. create a series of tables/views/functions/triggers/etc. and/or loading in the initial data, I always use psql and it's Variables to control the flow. I always add:
\set ON_ERROR_STOP
to the top of my scripts, so whenever I hit any exception, psql will abort. It looks like this might help in your case too.
And in cases when I need to do some exception handling, I use anonymous code blocks like this:
DO $$DECLARE _rec record;
BEGIN
FOR _rec IN SELECT * FROM schema WHERE schema_name != 'master' LOOP
EXECUTE 'DROP SCHEMA '||_rec.schema_name||' CASCADE';
END LOOP;
EXCEPTION WHEN others THEN
NULL;
END;$$;
DROP SCHEMA master CASCADE;

Related

Oracle pl/sql: executing dynamic delete within a transaction

I need to delete one or more row from list of tables stored in a table, and commit only if all deletion succeed.
So I wrote something like this (as part of a bigger procedure):
BEGIN
SAVEPOINT sp;
FOR cur_table IN (SELECT * FROM TABLE_OF_TABLES)
LOOP
EXECUTE IMMEDIATE 'DELETE FROM ' || cur_table.TABNAME || ' WHERE ID = :id_bind'
USING id;
END LOOP;
EXCEPTION
WHEN OTHERS THEN
ROLLBACK TO SAVEPOINT sp;
END;
I know this couldn't work, because of the "execute immediate".
So, what is the correct way to do that?
Dynamic SQL (Execute Immediate) doesn't commit the transaction. You have to commit explicitly. Your code is fine but it doesn't record/log the errors in case if they occur.
Log the errors in the exception handler section.
Honestly, this sounds like a bad idea. Since you have the tables stored in the database, why not just list them out in your procedure? Surely, you're not adding tables that often that this procedure would need to be updated often?
As pointed out in the comments, this will work fine, since EXECUTE IMMEDIATE does not automatically commit.
Don't forget to add a RAISE at the end of your exception block, or you'll never know that an error happened.

Re-run PL/SQL script command

I have a simple insert script that I want to expand upon.
DECLARE
i varchar2(3000) := dbms_random.string('A',8);
BEGIN
INSERT INTO BUYERS
(USER_ID,BUYER_CD,BUYER_ENG_NM,REG_DT)
VALUES
(i,'tes','test','test');
EXCEPTION WHEN OTHER
THEN
(this is where I need help)
end;
We have dynamic replication going on between two DB's. However, for some odd reason we have to run a script twice for the changes to commit to both DB's for that reason I am creating a script that will attempt to do a insert amongst all tables. As of now I'm only working on one table. Within the exception handler how do I make the script run again when the initial insert fails? Any help is appreciated.
If a problem happens with the insert then the best approach is to find out what the error is and raise the error. This is best accomplished by an autonomous logging procedure that will record the what, where, when and then RAISE the error again so processing stops. You do not want to take a chance of inserting records once, twice or not at all which could happen if the errors are not raised again.
The LOG_ERROR procedure below can be created from the answers to your previous questions about error handling.
DECLARE
i varchar2(3000) := dbms_random.string('A',8);
BEGIN
INSERT INTO BUYERS
(USER_ID,BUYER_CD,BUYER_ENG_NM,REG_DT)
VALUES
(i,'tes','test','test');
EXCEPTION WHEN OTHER
THEN
--by the time you got here there is no point in trying to insert again
LOG_ERROR(SQLERRM, LOCATION);
RAISE;
end;

How to create and use an Oracle transaction

I want to write a stored proc that delete data from two tables.
Should either of the deletes fail I want to make sure that no data was deleted.
This should be a simple task, but I never worked in Oracle before.
I'm not if I should be using TRY/CATCH, TRANSACTIONS or SAVEPOINTS.
Any guidance would be appreciated.
Currently I have:
CREATE OR REPLACE PROCEDURE SP_DELETE_STUFF
(
GROUPNAME IN VARCHAR2
) AS
BEGIN
SAVEPOINT Original_Start;
-- First delete all permissions for a given group
DELETE FROM my_table_1
WHERE group_name = GROUPNAME;
-- Second delete the group
DELETE FROM my_table_2
WHERE group_name = GROUPNAME;
EXCEPTION
WHEN OTHERS THEN
BEGIN
ROLLBACK TO SAVEPOINT Original_Start;
COMMIT;
END;
END
If your goal is just to rollback the changes that a particular call of the stored procedure has made if there is an error, you'd use a savepoint and a rollback to savepoint like you are doing here.
I would question your use of a commit after your rollback to savepoint. That will commit the transaction that the caller of your stored procedure had started. It seems unlikely that you want to commit the caller's changes when your procedure encounters an error. So I would expect that you want to remove the commit.
Not related to transaction scoping, I would also expect that you would want to at least re-raise the exception that you caught so that the caller is aware that there was a failure. I would expect that you would want something like
EXCEPTION
WHEN OTHERS THEN
BEGIN
ROLLBACK TO SAVEPOINT Original_Start;
RAISE;
END;

BEGIN - END block atomic transactions in PL/SQL

This information should be easy to find, but I haven't had any luck.
When I have a BEGIN - END block in a PL/SQL, does it behave as an atomic transaction, that will try to commit on hitting the END block and if anything goes wrong rolls back the changes?
If not, how do I make sure that the code inside the BEGIN - END block behaves like an atomic transaction and how does the block behave "by default"?
EDIT: I am running from a stored procedure and I am using an implicit block, I think.
Firstly, BEGIN..END are merely syntactic elements, and have nothing to do with transactions.
Secondly, in Oracle all individual DML statements are atomic (i.e. they either succeed in full, or rollback any intermediate changes on the first failure) (unless you use the EXCEPTIONS INTO option, which I won't go into here).
If you wish a group of statements to be treated as a single atomic transaction, you'd do something like this:
BEGIN
SAVEPOINT start_tran;
INSERT INTO .... ; -- first DML
UPDATE .... ; -- second DML
BEGIN ... END; -- some other work
UPDATE .... ; -- final DML
EXCEPTION
WHEN OTHERS THEN
ROLLBACK TO start_tran;
RAISE;
END;
That way, any exception will cause the statements in this block to be rolled back, but any statements that were run prior to this block will not be rolled back.
Note that I don't include a COMMIT - usually I prefer the calling process to issue the commit.
It is true that a BEGIN..END block with no exception handler will automatically handle this for you:
BEGIN
INSERT INTO .... ; -- first DML
UPDATE .... ; -- second DML
BEGIN ... END; -- some other work
UPDATE .... ; -- final DML
END;
If an exception is raised, all the inserts and updates will be rolled back; but as soon as you want to add an exception handler, it won't rollback. So I prefer the explicit method using savepoints.
BEGIN-END blocks are the building blocks of PL/SQL, and each PL/SQL unit is contained within at least one such block. Nesting BEGIN-END blocks within PL/SQL blocks is usually done to trap certain exceptions and handle that special exception and then raise unrelated exceptions. Nevertheless, in PL/SQL you (the client) must always issue a commit or rollback for the transaction.
If you wish to have atomic transactions within a PL/SQL containing transaction, you need to declare a PRAGMA AUTONOMOUS_TRANSACTION in the declaration block. This will ensure that any DML within that block can be committed or rolledback independently of the containing transaction.
However, you cannot declare this pragma for nested blocks. You can only declare this for:
Top-level (not nested) anonymous PL/SQL blocks
List item
Local, standalone, and packaged functions and procedures
Methods of a SQL object type
Database triggers
Reference: Oracle
You don't mention if this is an anonymous PL/SQL block or a declarative one ie. Package, Procedure or Function.
However, in PL/SQL a COMMIT must be explicitly made to save your transaction(s) to the database. The COMMIT actually saves all unsaved transactions to the database from your current user's session.
If an error occurs the transaction implicitly does a ROLLBACK.
This is the default behaviour for PL/SQL.
The default behavior of Commit PL/SQL block:
You should explicitly commit or roll back every transaction. Whether you issue the commit or rollback in your PL/SQL program or from a client program depends on the application logic. If you do not commit or roll back a transaction explicitly, the client environment determines its final state.
For example, in the SQLPlus environment, if your PL/SQL block does
not include a COMMIT or ROLLBACK statement, the final state of your
transaction depends on what you do after running the block. If you
execute a data definition, data control, or COMMIT statement or if you
issue the EXIT, DISCONNECT, or QUIT command, Oracle commits the
transaction. If you execute a ROLLBACK statement or abort the SQLPlus
session, Oracle rolls back the transaction.
https://docs.oracle.com/cd/B19306_01/appdev.102/b14261/sqloperations.htm#i7105

exception in stored procedure

I have following procedure:
procedure mayFailProc() as
begin
insert into t1 (id, val) values (1, '123');
insert into t1 (id, val) values (2, '123');
insert into t1 (id, val) values (3, '123'); //fails, i.e. due to pk uniqueness error
end;
this exception thrown in mayFailProc is a normal thing and it is handled by its caller. So the transaction is not rolled back and execution continues as if there was no exception in mayFailProc. I wonder what will happen to first two successfully executed inserts? Will they be retained or not?
See here for Oracle's explanation. You can jump to the How Oracle Does Implicit Rollbacks section to start.
Before executing an INSERT, UPDATE, or DELETE statement, Oracle marks
an implicit savepoint (unavailable to you). If the statement fails,
Oracle rolls back to the savepoint. Usually, just the failed SQL
statement is rolled back, not the whole transaction. If the statement
raises an unhandled exception, the host environment determines what is
rolled back.
More:
You should explicitly commit or roll back every transaction. Whether
you issue the commit or rollback in your PL/SQL program or from a
client program depends on the application logic. If you do not commit
or roll back a transaction explicitly, the client environment
determines its final state.
For example, in the SQL*Plus environment, if your PL/SQL block does
not include a COMMIT or ROLLBACK statement, the final state of your
transaction depends on what you do after running the block. If you
execute a data definition, data control, or COMMIT statement or if you
issue the EXIT, DISCONNECT, or QUIT command, Oracle commits the
transaction. If you execute a ROLLBACK statement or abort the SQL*Plus
session, Oracle rolls back the transaction.
Just done a quick test... the rows in my test were not retained when the third insert caused a "unique constraint ... violated" error

Resources