we have application where database contains large parts of business logic in triggers, with a update subsequently firing triggers on several other tables. I want to refactor the mess and wanted to start by extracting procedures from triggers, but can't find any reliable tool to do this. Using "Extract procedure" in both SQL Developer and Toad failed to properly handle :new and :old trigger variables.
If you had similar problem with triggers, did you find a way around it?
EDIT: Ideally, only columns that are referenced by extracted code would be sent as in/out parameters, like:
Example of original code to be extracted from trigger:
.....
if :new.col1 = some_var then
:new.col1 := :old.col1
end if
.....
would become :
procedure proc(in old_col1 varchar2, in out new_col1 varchar2, some_var varchar2) is
begin
if new_col1 = some_var then
new_col1 := old_col1
end if;
end;
......
proc(:old.col1,:new.col1, some_var);
It sounds like you want to carry out transformations on PL/SQL source. To do this reliably, you need a tool that can parse PL/SQL to some kind of compiler data structure, pattern-match against that structure and make directed changes, and then regenerate the modified PL/SQL code.
The DMS Software Reengineering Toolkit is such a tool. It is parameterized by the programming language being translated; it has off-the-shelf front ends for many languages, including C, C++, C#, Java, COBOL and ... PL/SQL.
This is not exactly the answer. I have not enough reputation to edit original question, obviously.
The issue is that it is not a "refactoring" as we usually think of. Even when you'll create bunch of procedures from triggers, you'll need to make a proper framework to run them in order to achieve original functionality. I suspect that this will be a challenge as well.
As a solution proposal, I'd go with one python script, based on state machine (see http://www.ibm.com/developerworks/library/l-python-state.html for example). If you put strict definition of what should be translated and how, it will be easy to implement.
Related
Executive summary: PostgreSQL is amazing, but we are facing many issues at work due to the fact that it postpones many checks on PL/pgSQL code until runtime. Is there a way to make it more like Oracle's PL/SQL in this respect?
For example...
Try executing this in any Oracle DB:
create function foo return number as
begin
select a from dual;
return a;
end;
Oracle will immediately (i.e. at compile-time!) respond with:
[Error] ORA-00904: invalid identifier
Now try the semantically equivalent thing in PostgreSQL:
CREATE OR REPLACE FUNCTION public.foo ()
RETURNS integer AS
$body$
BEGIN
select a;
return a;
END;
$body$
LANGUAGE plpgsql;
You will see it - unfortunately! - execute fine ... No error is reported.
But when you then try to call this function (i.e. at runtime) you will get:
ERROR: column "a" does not exist
LINE 1: select a
Is there a way to force PostgreSQL to perform syntax analysis and checking at function definition time - not at run-time? We have tons of legacy PL/SQL code at work, which we are porting to PostgreSQL - but the lack of compile-time checks is very painful, forcing us to do manual work - i.e. writing code to test all code paths in all functions/procedures - that was otherwise automated in Oracle.
Yes, this is a known issue.
PL/pgSQL (like any other function, except on SQL) is a “black box” for the PostgreSQL, therefore it is not really possible to detect errors except in runtime.
You can do several things:
wrap your function calling SQL queries into BEGIN / COMMIT statements in order to have better control over errors;
add EXCEPTION blocks to your code to catch and track errors. Note, though, that this will affect function performance;
use plpgsql_check extension, developed by the Pavel Stěhule, who is one of the main contributors to PL/pgSQL development. I suppose eventually this extension will make it into the core of the PostgreSQL, but it'll take some time (now we're in 9.4beta3 state);
You might also look into this related question: postgresql syntax check without running the query
And it really looks like you're in a huge need of a unit testing framework.
Plpgsql language is designed without semantics checking at compile-time. I am not sure if this feature was an intention or a side effect of old plpgsql implementation, but over time we found some advantages to it (but also disadvantages as you mentioned).
Plus :
there are less issues with dependency between functions and other database objects. It's a simple solution to cyclic dependency problem. Deployment of plpgsql functions is easier, because you don't need to respect dependency.
Some patterns with temporary tables are possible using lazy dependency checking. It's necessary, because Postgres doesn't support global temporary tables.
Example:
BEGIN
CREATE TEMP TABLE xx(a int);
INSERT INTO xx VALUES(10); -- isn't possible with compile-time dependency checks
END;
Minus:
Compile-time deep checking is not possible (identifiers checking), although it's sometimes possible.
For some bigger projects a mix of solutions should be used:
regress and unit tests - it is fundamental, because some situations cannot be checked statically - dynamic SQL for example.
plpgsql_check - it is an external but supported project used by some bigger companies and bigger plpgsql users. It can enforce a static check of SQL identifiers validity. You can enforce this check by DDL triggers.
With TSQL I'm used to putting some repeatable tests in for my stored procs. Typically this may include putting the db in a particular state, runnings the sproc, validating the state and rolling back. And contrived example might something like this"
BEGIN TRAN
--input for test case
DECLARE #TestName VARCHAR(10) = 'bob'
--insert test row
INSERT INTO tbl (data) values (#TestName)
--display initial state of target row
SELECT * FROM tbl WHERE data = #TestName
--do some useful test
EXEC MyProc
--display the final state of the target row
SELECT * FROM tbl WHERE data = #TestName
--put the db back where it started
ROLLBACK TRAN
Now I'm working with Oracle and PL/SQL and I'm trying to use a some similar pattern to test my work and not finding it obvious to me quite how to do that. I believe there are a few different ways I might accomplish it but haven't gotten anything to actually work. Ideally I would have a single script in which I could run multiple test cases and inspect the result.
I am trying to work in PL/SQL Developer at this point and understand that might have some differences from how it might work in Oracle SQL Developer or elsewhere.
In Oracle, using tools like SQL*Plus and GUI tools like SQL Developer, you have many options :
To execute the statements and procedures in a single session in an order, i.e. using procedural method of PL/SQL, write an anonymous plsql block and execute it as a script.
Most of the GUI based tools have an option like Execute as script or Test Window to execute your scripts individually or embedded in an anonymous block.
Using DBMS_SCHEDULER also you could achieve the same task.
As you are interested in PL/SQL Developer tool product of Allround Automations, you could simply use the test window to test individual objects.
I have documented few useful features of the PL/SQL Developer tool in my blog, please read http://lalitkumarb.wordpress.com/2014/08/14/plsql-developer-settings/
Is there any "best practice" on how to call a stored procedure from APEX instead of just using a simple DB-Link?
Calling PL/SQL Stored Procedures Within Oracle APEX
There's a great deal of detail involved with developing robust, maintainable and efficient solutions by paring PL/SQL stored procs with the Application Express web based frame work. Not knowing the current skill level of the author of the OP, I assume that the most helpful explanation is a simple example developed from scratch.
Getting Started: The APEX environment has a lot of tools that can get you started. If you don't already have an Oracle database environment with APEX installed, consider signing up for a free, hosted trial account through the APEX home page on Oracle.com.
Application Design Decisions
PL/SQL stored procedures are not necessary to develop applications on Oracle APEX, however they do supply a greater amount of flexibility and customization during the development process.
This example will use the popular schema object: EMP typically available as an optional part of each Oracle database installation. In case you don't have it, here is the DDL and DML source for building the example table:
The EMP Table (A Copy Aliased as: LAB01_SAMPLE_EMP)
CREATE TABLE "EMP"
( "EMPNO" NUMBER(4,0) NOT NULL ENABLE,
"ENAME" VARCHAR2(10),
"JOB" VARCHAR2(9),
"MGR" NUMBER(4,0),
"HIREDATE" DATE,
"SAL" NUMBER(7,2),
"COMM" NUMBER(7,2),
"DEPTNO" NUMBER(2,0),
PRIMARY KEY ("EMPNO") ENABLE
)
/
ALTER TABLE "EMP" ADD FOREIGN KEY ("MGR")
REFERENCES "EMP" ("EMPNO") ENABLE
/
If you would like some test data, this is what I had to work with:
BUILD a SQL-based DML update statement that will change the SAL (Salary) value for one employee at a time based on their ENAME.
WRAP the UPDATE DML statement into a reusable, compiled PL/SQL stored procedure. Include parameter arguments for the two data input values for "Name" and "Amount of Increase".
TEST the stored procedure and verify it works as required.
DESIGN and CODE an APEX Application Page which will:
(a) Show the current contents of the employee entity table.
(b) Accept input values to pass into the PL/SQL Stored Procedure.
(c) Utilize native APEX Error and Success message settings to provide more feedback on the outcome of the procedure call.
TEST the APEX page and verify it works as specified.
REVIEW the discussion and conclusions section at the end for additional comments and more tips to keep you moving on your own path to developing Oracle skills.
Programming a SQL DML Process
This is the initial example created to accomplish this task.
UPDATE LAB01_SAMPLE_EMP
SET SAL = SAL + 1250
WHERE ENAME = 'KING';
A quick revision reveals how we can parametrize the approach to make this statement
UPDATE LAB01_SAMPLE_EMP
SET SAL = SAL + p_salary_increase
WHERE ENAME = p_ename;
If you are not sure where to go next, this is where a lesson on "best-practices" is available thanks to APEX. Navigate to the OBJECT BROWSER and CREATE a Procedure Object. The application will walk through every step to set up a PL/SQL stored proc.
The Working PL/SQL Procedure Source Code:
After walking through the wizard setup, this is the cleaned-up stored procedure. There were some additional changes after debugging some compile-time error warnings:
create or replace procedure "PROC_UPDATE_SALARY"
(p_ename IN VARCHAR2, p_salary_increase IN VARCHAR2)
is
v_new_salary lab01_sample_emp.sal%TYPE;
begin
UPDATE LAB01_SAMPLE_EMP
SET SAL = SAL + p_salary_increase
WHERE ENAME = p_ename
RETURNING SAL INTO v_new_salary;
commit;
dbms_output.put_line('INCREASED SALARY FOR: ' || p_ename ||
' TO THE NEW AMOUNT OF: $ ' || to_char(v_new_salary));
end;
Best Practices, a Quick Aside and Discusssion: You just have to keep doing it... coding that is. There is just no way around it. Look for examples at work or in life and try your hand at developing schemas and designs to satisfy made-up but realistic requirements. For beginning developers, the PL/SQL stored procedure above may already show some "unfamiliar" or odd coding syntax and commands.
That is only the tip of what is possible out there. Coding style is also only a part of it as you get deeper into things, you may notice a few things:
ORGANIZATION is important. Learn quickly some conventions in naming and notation to use in the code. This will keep things organized and easy to find or reference elsewhere.
RECYCLE and REUSE means your code should be developed with reuse in mind. Common routines and processes should be bundled together to avoid redundancy.
ALWAYS TEST your work suggests that less frustration is found when initial, fundamental steps in your process or application have been carefully tested first before proceeding.
TESTING the Oracle PL/SQL Procedure
I used the built-in APEX scripting engine found in the SQL WORKSHOP section of the environment. Below is a screenshot of the output logs of my testing script.
Bringing it Together: Designing an APEX Application Page with a Procedure Call
Create or open up an APEX Application and start out by making a new page. There are system wizard processes that will help you get started if you haven't done this before.
Select the option to BUILD a FORM on top of a STORED PROCEDURE. There will be prompts for permission to build the page items needed for input parameters. Follow the wizard to completion to make sure all the dependent page design elements are included.
My near-finalized design is below:
There are a few extras added, such as the REPORT region to provide immediate visibility to the table and any applied changes to its data.
The Form in Action: Testing Data Input and Results
The SUCCESS alert message is a feature available for certain page elements to inform the user of any significant events conducted on the database.
Closing Comments and Discussion
The immediate answer to the question of the OP is YES, there are best practices. It is such a huge subject that the only realistic way of handling it is by walking through different examples to see the ways that these practices are "commonly" applied.
There were a few shortcuts involved in this solution (based on several assumptions) and it might be helpful to bring them up as a parting discussion on the possibility of revisiting this walk-through to make an improved, EMP-2.0 project.
The procedure works on the EMP table based on searches by ENAME. What is the REAL key of this table? Are there any risks involved with this approach- possibly with respect to larger data sets?
Most Oracle PL/SQL Objects used in production ready environments have some level of error trapping or exception handling through a PL/SQL EXCEPTION BLOCK. What kind of errors would you want to trap? How?
Don't underestimate the resources available within the APEX tool. There are lots of wizards that walk developers through the process of creating different, functioning modules of code. These automated guides provide solutions that can also be reverse-engineered to understand how the code was generated and also what general design approaches make compliant design patterns.
Recently, I encountered a BEFORE INSERT OR UPDATE trigger on a table. In this trigger, the author relies on the INSERTING and UPDATING functions (both return a BOOLEAN) of the DBMS_STANDARD package to determine if the trigger was fired before an insert or before an update.
For example:
CREATE OR REPLACE TRIGGER CUSTOMER_TRIGGER
BEFORE INSERT OR UPDATE ON CUSTOMER
FOR EACH ROW
BEGIN
IF INSERTING THEN
/* Some code */
END IF;
IF UPDATING THEN
/* Some other code */
END IF;
END;
Yes, I know that two, individual triggers could have been written to handle the two events separately. That's not the point of this question.
After troubleshooting an error being received by these functions, we received word (from Oracle Support) that "dbms_standard routines are not really meant to be called by user programs". Is this true?
I find this a little strange, considering other procedures (such as RAISE_APPLICATION_ERROR and COMMIT) are commonly used in PL/SQL code.
The functions INSERTING, UPDATING and DELETING are expressly provided for use in writing trigger code (see trigger documentation), so there is absolutely no proscription against using those. Similarly, RAISE_APPLICATION_ERROR is documented to be intended for use by developers.
Having just DESCribed DBMS_STANDARD there are certainly some functions in there I don't know about and that perhaps shouldn't be used in your own code (for all I know) such as GRANTEE.
Generally, if you can find Oracle official documentation telling you how to use something, then it is OK to use it.
I have a situation where I have an Oracle procedure that is being called from at least 3 or 4 different places. I need to be able to be able to call custom-code depending on some data. The custom-code is customer-specific - so, customer A might want to do A-B-C where customer B might want to do 6-7-8 and customer C doesn't need to do anything extra. When customers D...Z come along, I don't want to have to modify my existing procedure.
I'd like to be able to enter the customer-specific procedure into a table. In this existing procedure, check that database table if a custom-code procedure exists and if so, execute it. Each of the customer-code procedures would have the same parameters.
For instance:
My application (3+ places) calls this "delete" procedure
In this delete procedure, look up the name of a child-procedure to call (if one exists at all)
If one exists, execute that delete procedure (passing the parameters in)
I know I can do this with building a string that contains the call to the stored procedure. But, I'd like to know if Oracle 10g has anything built in for doing this kind of thing?
Do each of your customers have their own database? If so the best option would be to use conditional compilation. This has the advantage of not requiring dynamic SQL. Have the main program always call the custom procedure, and use CC flags to vary the code it contains.
Otherwise, Oracle does have a Rule Engine but it is not really intended for our use.
The final solution that we went with was to store the name of a procedure in a database table. We then build the SQL call and use an EXECUTE statement.
Agree with APC's answer and just to expand on it, in this white paper if you look for "Component based installation" it describes a similar problem solved by using conditional compilation.
Your solution seems reasonable given the requirements, so I voted it up.
Another option would be to loop through the results from your table look-up and put calls to the procedures inside a big case statement. It would be more code, but it would have the advantage of making the dependency chain visible so you could more easily catch missing permissions and invalid procedures.