Recently, I have read about UDF Pragma optimization method in Oracle Database 12.
I'm very interested in how exactly it works. I've only found very short description in the Oracle documentation.
As I understand, every Pragma in PL/SQL is some kind of compiler directive (I could be wrong here) similar to C++ Pragma Directives.
Maybe someone can explain to me in more details (or provide links :) ) how the internal mechanism of UDF Pragma works?
The internals are not made available to us, but essentially the pragma is an instruction to the compiler to focus on reducing the cost of the PL/SQL => SQL context switch overhead. If I had to hypothesise I would say that it takes advantage of a different 12c feature which allow PL/SQL directly within a CTE (common table expression), e.g.:
WITH
function myfunction(x int) return int as
begin
...
return ...
end;
select myfunction(col) from my_table
PL/SQL functions coded like the above exist solely for the duration of the SQL execution and also execute with improved performance characteristics over that of standalone functions. So I suspect the UDF pragma allows invocation of a standalone function via the same code path as the inline example above.
(But all just hypothesis on my part)
Related
I occasionally encounter examples where SELECT...INTO...FROM DUAL is used to call a function - e.g.:
SELECT some_function INTO a_variable FROM DUAL;
is used, instead of
a_variable := some_function;
My take on this is that it's not good practice because A) it makes it unclear that a function is being invoked, and B) it's inefficient in that it forces a transition from the PL/SQL engine to the SQL engine (perhaps less of an issue today).
Can anyone explain why this might have been done, e.g. was this necessary in early PL/SQL coding in order to invoke a function? The code I'm looking at may date from as early as Oracle 8.
Any insights appreciated.
This practice dates from before PLSQL and Oracle 7. As already mentioned assignment was possible (and of course Best Practice) in Oracle7.
Before Oracle 7 there were two widely used Tools that needed the use of Select ... into var from dual;
On the one hand there used to be an Oracle Tool called RPT, some kind of report generator. RPT could be used to create batch processes. It had two kinds of macros, that could be combined to achieve what we use PLSQL for today. My first Oracle job involved debugging PLSQL that was generated by a program that took RPT batches and converted them automatically to PLSQL. I threw away my only RPT handbook sometime shortly after 2000.
On the other hand there was Oracle Forms 2.x and its Menu component. Context switching in Oracle Menu was often done with a Select ... from dual; I still remember how proud I was when I discovered that an untractable Bug was caused by a total of 6 records in table Dual.
I am sorry to say that I can not proof any of this, but it is the time of year to think back to the old times and really fun to have the answer.
Executive summary: PostgreSQL is amazing, but we are facing many issues at work due to the fact that it postpones many checks on PL/pgSQL code until runtime. Is there a way to make it more like Oracle's PL/SQL in this respect?
For example...
Try executing this in any Oracle DB:
create function foo return number as
begin
select a from dual;
return a;
end;
Oracle will immediately (i.e. at compile-time!) respond with:
[Error] ORA-00904: invalid identifier
Now try the semantically equivalent thing in PostgreSQL:
CREATE OR REPLACE FUNCTION public.foo ()
RETURNS integer AS
$body$
BEGIN
select a;
return a;
END;
$body$
LANGUAGE plpgsql;
You will see it - unfortunately! - execute fine ... No error is reported.
But when you then try to call this function (i.e. at runtime) you will get:
ERROR: column "a" does not exist
LINE 1: select a
Is there a way to force PostgreSQL to perform syntax analysis and checking at function definition time - not at run-time? We have tons of legacy PL/SQL code at work, which we are porting to PostgreSQL - but the lack of compile-time checks is very painful, forcing us to do manual work - i.e. writing code to test all code paths in all functions/procedures - that was otherwise automated in Oracle.
Yes, this is a known issue.
PL/pgSQL (like any other function, except on SQL) is a “black box” for the PostgreSQL, therefore it is not really possible to detect errors except in runtime.
You can do several things:
wrap your function calling SQL queries into BEGIN / COMMIT statements in order to have better control over errors;
add EXCEPTION blocks to your code to catch and track errors. Note, though, that this will affect function performance;
use plpgsql_check extension, developed by the Pavel Stěhule, who is one of the main contributors to PL/pgSQL development. I suppose eventually this extension will make it into the core of the PostgreSQL, but it'll take some time (now we're in 9.4beta3 state);
You might also look into this related question: postgresql syntax check without running the query
And it really looks like you're in a huge need of a unit testing framework.
Plpgsql language is designed without semantics checking at compile-time. I am not sure if this feature was an intention or a side effect of old plpgsql implementation, but over time we found some advantages to it (but also disadvantages as you mentioned).
Plus :
there are less issues with dependency between functions and other database objects. It's a simple solution to cyclic dependency problem. Deployment of plpgsql functions is easier, because you don't need to respect dependency.
Some patterns with temporary tables are possible using lazy dependency checking. It's necessary, because Postgres doesn't support global temporary tables.
Example:
BEGIN
CREATE TEMP TABLE xx(a int);
INSERT INTO xx VALUES(10); -- isn't possible with compile-time dependency checks
END;
Minus:
Compile-time deep checking is not possible (identifiers checking), although it's sometimes possible.
For some bigger projects a mix of solutions should be used:
regress and unit tests - it is fundamental, because some situations cannot be checked statically - dynamic SQL for example.
plpgsql_check - it is an external but supported project used by some bigger companies and bigger plpgsql users. It can enforce a static check of SQL identifiers validity. You can enforce this check by DDL triggers.
In an Oracle PL/SQL block, why is dynamic sql allowed
begin
execute immediate 'drop table table_name';
end;
but static is not?
begin
drop table table_name;
end;
I hope the answer is more insightful than "because that's how the language works".
The answer is PL/SQL does not support dynamic polymorphism. it only supports static polymorphism because
All PL/SQL generats a "DIANA" -> Descriptive Intermediate Attributed Notation for Ada , a tree-structured intermediate language. DIANA is used internally by compilers.
At compile time, PL/SQL source code is translated into system code and generates corresponding DIANA. Now think if there were a DDL statement like create table statement which at the compile time does not exists it will be created after running the program. how would your PL/SQL engine generate a DIANA then ????
The DIANA is plays an important role in PL/SQL to check/validate that the sub program. this is required because as we know that a sub-program can use database objects such as Tables,Views,Synonyms or other stored procs. it could be possible that the the objects may have changed/removed/droped when next time you run the program. For ex : some one might have droped the table, the stored proc or function singnature may have changed.
Thats why generally PL/SQL is used to manipulate the data within database structure, but not to manipulate those structures.
but there are ways to manipulate using dynamic SQL and DBMS_SQL package but theses methodlogy are again should be used cautiously. For example if you are creating a Table you should check first if this table is already exists or not using data dictionary views.
Probably because otherwise some code would be like:
begin
create table tmp (n number);
insert into tmp values (1);
end;
And we would expect the compiler to know that at time of the insert, the table exists. The compilation of the block would me much more difficult. Here it is a very simple case, but we can easily imagine some conditional branching, and complex blabla.
But, since we need to put the DDL in an execute immediate block, the limitation maybe somehow easier to understand.
Just an idea...
Before execution/compilation, oracle checks all access permissions,validity and dependencies of all schema objects like tables,views,stored procs etc referenced inside a pl/sql block. But problem with DDL statement is that it
can create, alter or drop schema object and as a result, it can change the object dependencies.So it is possible that we are referring one object which has been dropped using DDL. To prevent such situation I guess plsql block does not allow direct DDL statement.
But we can still include DDL in PL/SQL block using Dynamic query. As in this case the actual query is not known until run-time, basically we can hide the DDL statement in the form of Dynamic SQL and include DDL inside PL/SQL block.
You can refer my blog to understand the concept with example:
Why oracle does not allow direct DDL statements inside the procedure (PLSQL BLOCK)
I have recently had some input from a colleague regarding committing in a stored function. Whether we use procedures or functions to execute offline / batch logic in an Oracle database is mostly a matter of taste in our application. In both cases, we return a code either as function result, or as procedure OUT parameter. We usually require those offline / batch routines to be called from PL/SQL, not from SQL:
-- good
declare
rc number(7);
begin
rc := our_function(1, 2, 3);
end;
-- less good
select our_function(1, 2, 3) from dual;
The reason why the latter is less good is because our_function may commit the transaction for performance reasons. This is ok for a batch routine.
The question is: Are there any best practices around this topic, or some special keywords that prevent such functions from being used in SQL statements on a compiler-level? Or should we avoid functions for batch operations and only use procedures?
You can use RESTRICT_REFERENCES to indicate that a function won't read/write package or database state.
CREATE PACKAGE t_pkg AS
FUNCTION showup (msg VARCHAR2) RETURN VARCHAR2;
PRAGMA RESTRICT_REFERENCES(showup, WNDS, RNDS);
END t_pkg;
/
-- create the package body
CREATE OR REPLACE PACKAGE BODY t_pkg AS
FUNCTION showup (msg VARCHAR2) RETURN VARCHAR2 IS
v_val varchar2(1);
BEGIN
select dummy into v_val from dual;
RETURN v_val;
END;
END t_pkg;
/
It used to be the case that SQL wouldn't allow you to call a function unless it made such a promise, but that restriction got dropped.
I'd prefer to make it a differentiator between a procedure and a function. It's worth bearing in mind that if a PL/SQL function raises a NO_DATA_FOUND exception, a calling SQL statement does not fail (as no data found isn't an SQL error). So I prefer to use procedures unless the object is specifically designed to be called from SQL.
Are there any best practices around this topic, or some special
keywords that prevent such functions from being used in SQL statements
on a compiler-level?
If you use a function that requires a transaction (and therefore a commit), AFAIK you will not be able to call it from a SELECT, unless the function uses an AUTONOMOUS TRANSACTION (otherwise you get a ORA-14551 cannot perform a DML operation inside a query).
See also: ORA-14551: cannot perform a DML operation inside a query
So, having a function that requires a transaction itself should prevent it from being called from a SELECT.
From my point of view there is no way to accomplish this.Although you can avoid runtime errors like "ORA-14551" by using PRAGMA RESTRICT_REFERENCES in "our_function(1, 2, 3)" to be sure that it is safe to use it the SQL query,but you can't prevent it from using in sql at the compiler-level
we have application where database contains large parts of business logic in triggers, with a update subsequently firing triggers on several other tables. I want to refactor the mess and wanted to start by extracting procedures from triggers, but can't find any reliable tool to do this. Using "Extract procedure" in both SQL Developer and Toad failed to properly handle :new and :old trigger variables.
If you had similar problem with triggers, did you find a way around it?
EDIT: Ideally, only columns that are referenced by extracted code would be sent as in/out parameters, like:
Example of original code to be extracted from trigger:
.....
if :new.col1 = some_var then
:new.col1 := :old.col1
end if
.....
would become :
procedure proc(in old_col1 varchar2, in out new_col1 varchar2, some_var varchar2) is
begin
if new_col1 = some_var then
new_col1 := old_col1
end if;
end;
......
proc(:old.col1,:new.col1, some_var);
It sounds like you want to carry out transformations on PL/SQL source. To do this reliably, you need a tool that can parse PL/SQL to some kind of compiler data structure, pattern-match against that structure and make directed changes, and then regenerate the modified PL/SQL code.
The DMS Software Reengineering Toolkit is such a tool. It is parameterized by the programming language being translated; it has off-the-shelf front ends for many languages, including C, C++, C#, Java, COBOL and ... PL/SQL.
This is not exactly the answer. I have not enough reputation to edit original question, obviously.
The issue is that it is not a "refactoring" as we usually think of. Even when you'll create bunch of procedures from triggers, you'll need to make a proper framework to run them in order to achieve original functionality. I suspect that this will be a challenge as well.
As a solution proposal, I'd go with one python script, based on state machine (see http://www.ibm.com/developerworks/library/l-python-state.html for example). If you put strict definition of what should be translated and how, it will be easy to implement.