How do I access the AST (abstract syntax tree) for a PL/SQL stored procedure? - oracle

When Oracle compiles a stored procedure, it stores the AST for the procedure in DIANA format.
how can I access this AST?
are there built-in tools for processing this AST?

There is an undocumented package DUMPDIANA that is meant to dump the Diana in a human-readable format.
The file $ORACLE_HOME\rdbms\admin\dumpdian.sql says "Documentation is available in /vobs/plsql/notes/dumpdiana.txt". I cannot find that file, and without it we can only guess at the meaning of some parameters. Basic usage of DUMPDIANA is as follows:
SQL> show user
USER is "SYS"
SQL> #?\rdbms\admin\dumpdian
Library created.
Package created.
Package body created.
create or replace procedure hello_world
2 as
3 begin
4 dbms_output.put_line('hello world');
5* end;
Procedure created.
SQL> set serveroutput on
SQL> execute sys.DUMPDIANA.dump('HELLO_WORLD');
user: SYS
PL/SQL procedure successfully completed.
At this point a pair of files should have been created in the folder
$ORACLE_BASE/diag/rdbms/orcl12c/orcl12c/trace. The two files seem to follow the naming convention:
orcl12c_ora_{PROCESS}.trc
orcl12c_ora_{PROCESS.trm
Where the trc file is a human readable version of the corresponding trm file, and {PROCESS} is the operating system process ID. To find this use the following query from the same session:
select p.spid from v$session s,v$process p
where s.paddr = p.addr
and s.sid = sys_context('USERENV','SID');
For example if the session ID was 8861 then from a bash shell you can view the results using:
vim $ORACLE_BASE/diag/rdbms/orcl12c/orcl12c/trace/orcl12c_ora_8861.trc
The result is interesting... if not particularly intuitive! For example here is a snippet of the file produced. Note the HELLO_WORLD string literal.
PD1(2):D_COMP_U [
L_SRCPOS : row 1 col 1
A_CONTEX :
PD2(2): D_CONTEX [
L_SRCPOS : row 1 col 1
AS_LIST : < >
]
A_UNIT_B :
PD3(2): D_S_BODY [
L_SRCPOS : row 1 col 1
A_D_ :
PD4(2): DI_PROC [
L_SRCPOS : row 1 col 11
L_SYMREP : HELLO_WORLD,
S_SPEC : PD5^(2),
S_BODY : PD8^(2),
A couple of notes. I've run this as SYS, which as we know is not a good practice, this is no reason I know of why you shouldn't grant privileges on DUMPDIANA to a normal user. All the procedures you dump go into the same file - if you delete that file, it stops working, and you'll need to start a new session. If it stops working, starting a new session sometimes seems to fix the problem.

Here is an excellent tutorial on DIANA and IDL in the PDF How to unwrap PL/SQL by Pete Finnigan, principal consultant at Siemens at the time of the writting, specializing in researching and securing Oracle databases.
Among other very interesting things you will learn that:
DIANA is written down as IDL (Interface Definition Language).
The 4 tables the IDL is stored in (IDL_CHAR$, IDL_SB4$, IDL_UB1$ and IDL_UB2$)
Wrapper PL/SQL is simply DIANA written down as IDL.
Dumpdiana is not installed by default, you need to ensure DIANA, PIDL, and DIUTIL PL/SQL packages are installed as well and you need to run it as SYS.
How to dump the DIANA tree and understand it.
How to reconstruct the PL/SQL source from DIANA.
How to write a PL/SQL un-wrapper.
Limitations of a PL/SQL API based un-wrapper.
Limitations of the PL/SQL API itself.
How to enumerate DIANA nodes and attributes.
A proof of concept un-wrapper.
You can find his website here. There is so much content there. You will find awesome papers about Oracle security and also a lot of useful security tools developed not only by him but other authors as well.
Best of all, you can get in touch with him if after the reading you still have questions.

Related

Altering Stored Procedures in ASE Sybase 15.7

I am new to ASE Sybase 15.7 but do have some background in other RDBMS systems. So i assumed there would be an equivalent of CREATE OR REPLACE for Stored procedures in ASE Sybase 15.7.
But I dont seem to see any way to do this. Most people i have asked suggest dropping and creating with the newer version of the stored procedure but that gives me a challenge of managing the permissions on the stored procedure which are different across environments depending on the users in each.
So My ask is below:
Suppose I have a stored procedure as so:
ENV1
CREATE Procedure test (
as
begin
SELECT getdate()
end
grant execute on test to group1
go
grant execute on test to group2
go
ENV2 has :
CREATE Procedure test (
as
begin
SELECT getdate()
end
grant execute on test to group1
go
grant execute on test to group2
go
grant execute on test to group3
go
I want to update this stored proc to give me 2 dates instead of 1 so new proc should be
ENV1:
CREATE Procedure test (
as
begin
SELECT getdate(), getdate()
end
grant execute on test to group1
go
grant execute on test to group2
go
ENV2:
CREATE Procedure test (
as
begin
SELECT getdate(), getdate()
end
grant execute on test to group1
go
grant execute on test to group2
go
grant execute on test to group3
go
Above is a very simplistic example ofcourse. Is there a way to deploy the changes to just modify the stored procedure body preserving the permissions?
CREATE or REPLACE and ALTER PROCEDURE dont seem to work and dropping and creating the stored procedure would mean additional logic for each environment to figure out the permissions to be granted.
Is there a way to do this kind of deployment in an optimum way considering we have 20 plus different user environments?
Thanks!
While ASE does support create or replace, this is only available with ASE 16.x (ie, you'd need to upgrade to ASE 16.x).
Assuming you're looking to build some sort of scripted solution, I'd recommend taking a look at the ddlgen utility to assist with extracting the current permissions for a stored proc.
One (very simple) example of using ddlgen to pull the DDL for a stored proc:
$ ddlgen -SmyASE -Umylogin -Pmypassword -TP -Nsybsystemprocs.dbo.sp_help -Osp_help.ddl.out
$ cat sp_help.ddl.out
-- Sybase Adaptive Server Enterprise DDL Generator Utility/1 ...snip...
...snip...
use sybsystemprocs
go
...snip...
create procedure sp_help
...snip...
Grant Execute on dbo.sp_help to public Granted by dbo
go
sp_procxmode 'sp_help', anymode
go
From here you could grep out the desired grant, revoke and/or sp_procxmode lines, to be (re)executed once you've dropped/created the replacement stored proc.
If you don't have access to ddlgen (I know it's included in the ASE installation software but don't recall if it's provided in the SDK/client software installation) you have a few alternatives:
have the DBA run the ddlgen commands for you and provide you with the results (yeah, I'm sure the DBA will love that idea)
get ddlgen installed on your 'client' machine (eg, install the ASE installation package; or copy over just the needed files from an ASE installation - easier said than done, and would be a PITA when it comes to upgrading the software)
run sp_helprotect <proc_name> (and sp_procxmode <proc_name>) and parse the output for the desired grant, revoke and/or sp_procxmode commands
And one alternative on the 'run-and-parse sp_helprotect/sp_procxmode output' ... look at the source code for these procs and roll your own SQL code to extract the desired data in a format that's easier for you process to handle.

APEX Call external stored function or stored procedure

Is there any "best practice" on how to call a stored procedure from APEX instead of just using a simple DB-Link?
Calling PL/SQL Stored Procedures Within Oracle APEX
There's a great deal of detail involved with developing robust, maintainable and efficient solutions by paring PL/SQL stored procs with the Application Express web based frame work. Not knowing the current skill level of the author of the OP, I assume that the most helpful explanation is a simple example developed from scratch.
Getting Started: The APEX environment has a lot of tools that can get you started. If you don't already have an Oracle database environment with APEX installed, consider signing up for a free, hosted trial account through the APEX home page on Oracle.com.
Application Design Decisions
PL/SQL stored procedures are not necessary to develop applications on Oracle APEX, however they do supply a greater amount of flexibility and customization during the development process.
This example will use the popular schema object: EMP typically available as an optional part of each Oracle database installation. In case you don't have it, here is the DDL and DML source for building the example table:
The EMP Table (A Copy Aliased as: LAB01_SAMPLE_EMP)
CREATE TABLE "EMP"
( "EMPNO" NUMBER(4,0) NOT NULL ENABLE,
"ENAME" VARCHAR2(10),
"JOB" VARCHAR2(9),
"MGR" NUMBER(4,0),
"HIREDATE" DATE,
"SAL" NUMBER(7,2),
"COMM" NUMBER(7,2),
"DEPTNO" NUMBER(2,0),
PRIMARY KEY ("EMPNO") ENABLE
)
/
ALTER TABLE "EMP" ADD FOREIGN KEY ("MGR")
REFERENCES "EMP" ("EMPNO") ENABLE
/
If you would like some test data, this is what I had to work with:
BUILD a SQL-based DML update statement that will change the SAL (Salary) value for one employee at a time based on their ENAME.
WRAP the UPDATE DML statement into a reusable, compiled PL/SQL stored procedure. Include parameter arguments for the two data input values for "Name" and "Amount of Increase".
TEST the stored procedure and verify it works as required.
DESIGN and CODE an APEX Application Page which will:
(a) Show the current contents of the employee entity table.
(b) Accept input values to pass into the PL/SQL Stored Procedure.
(c) Utilize native APEX Error and Success message settings to provide more feedback on the outcome of the procedure call.
TEST the APEX page and verify it works as specified.
REVIEW the discussion and conclusions section at the end for additional comments and more tips to keep you moving on your own path to developing Oracle skills.
Programming a SQL DML Process
This is the initial example created to accomplish this task.
UPDATE LAB01_SAMPLE_EMP
SET SAL = SAL + 1250
WHERE ENAME = 'KING';
A quick revision reveals how we can parametrize the approach to make this statement
UPDATE LAB01_SAMPLE_EMP
SET SAL = SAL + p_salary_increase
WHERE ENAME = p_ename;
If you are not sure where to go next, this is where a lesson on "best-practices" is available thanks to APEX. Navigate to the OBJECT BROWSER and CREATE a Procedure Object. The application will walk through every step to set up a PL/SQL stored proc.
The Working PL/SQL Procedure Source Code:
After walking through the wizard setup, this is the cleaned-up stored procedure. There were some additional changes after debugging some compile-time error warnings:
create or replace procedure "PROC_UPDATE_SALARY"
(p_ename IN VARCHAR2, p_salary_increase IN VARCHAR2)
is
v_new_salary lab01_sample_emp.sal%TYPE;
begin
UPDATE LAB01_SAMPLE_EMP
SET SAL = SAL + p_salary_increase
WHERE ENAME = p_ename
RETURNING SAL INTO v_new_salary;
commit;
dbms_output.put_line('INCREASED SALARY FOR: ' || p_ename ||
' TO THE NEW AMOUNT OF: $ ' || to_char(v_new_salary));
end;
Best Practices, a Quick Aside and Discusssion: You just have to keep doing it... coding that is. There is just no way around it. Look for examples at work or in life and try your hand at developing schemas and designs to satisfy made-up but realistic requirements. For beginning developers, the PL/SQL stored procedure above may already show some "unfamiliar" or odd coding syntax and commands.
That is only the tip of what is possible out there. Coding style is also only a part of it as you get deeper into things, you may notice a few things:
ORGANIZATION is important. Learn quickly some conventions in naming and notation to use in the code. This will keep things organized and easy to find or reference elsewhere.
RECYCLE and REUSE means your code should be developed with reuse in mind. Common routines and processes should be bundled together to avoid redundancy.
ALWAYS TEST your work suggests that less frustration is found when initial, fundamental steps in your process or application have been carefully tested first before proceeding.
TESTING the Oracle PL/SQL Procedure
I used the built-in APEX scripting engine found in the SQL WORKSHOP section of the environment. Below is a screenshot of the output logs of my testing script.
Bringing it Together: Designing an APEX Application Page with a Procedure Call
Create or open up an APEX Application and start out by making a new page. There are system wizard processes that will help you get started if you haven't done this before.
Select the option to BUILD a FORM on top of a STORED PROCEDURE. There will be prompts for permission to build the page items needed for input parameters. Follow the wizard to completion to make sure all the dependent page design elements are included.
My near-finalized design is below:
There are a few extras added, such as the REPORT region to provide immediate visibility to the table and any applied changes to its data.
The Form in Action: Testing Data Input and Results
The SUCCESS alert message is a feature available for certain page elements to inform the user of any significant events conducted on the database.
Closing Comments and Discussion
The immediate answer to the question of the OP is YES, there are best practices. It is such a huge subject that the only realistic way of handling it is by walking through different examples to see the ways that these practices are "commonly" applied.
There were a few shortcuts involved in this solution (based on several assumptions) and it might be helpful to bring them up as a parting discussion on the possibility of revisiting this walk-through to make an improved, EMP-2.0 project.
The procedure works on the EMP table based on searches by ENAME. What is the REAL key of this table? Are there any risks involved with this approach- possibly with respect to larger data sets?
Most Oracle PL/SQL Objects used in production ready environments have some level of error trapping or exception handling through a PL/SQL EXCEPTION BLOCK. What kind of errors would you want to trap? How?
Don't underestimate the resources available within the APEX tool. There are lots of wizards that walk developers through the process of creating different, functioning modules of code. These automated guides provide solutions that can also be reverse-engineered to understand how the code was generated and also what general design approaches make compliant design patterns.

Is there a way to access private plsql procedures for testing purposes?

I'm working on a project with a lot of plsql code and would like to add more specific unit-tests to our codebase. Some of the procedures/functions I like to test aren't in the package spec and I have no means to change that.
Is there a way to access these 'private' plsql procedures without adding them to the spec?
The only Idea I had so far, was to compile a special package spec to the DB before the tests, that specifies the procedures under test. I gues that would work, but I wonder if there is a simpler way, some evil secret oracle hack maybe ;-)
I'm testing from Java with JUnit/DBUnit.
BR
Frank
There is a way to do this, providing you are on 10g or higher. It's called Conditional Compilation. This is a highly neat feature which provides special syntax so we can change our PL/SQL code at compilation time.
As it happens I have been using this feature precisely to expose private packages in a spec so I can run UTPLSQL tests against them.
Here is the special syntax:
create or replace package my_pkg
as
$IF $$dev_env_test $THEN
PROCEDURE private_proc;
$END
FUNCTION public_function return date;
end my_pkg;
/
That variable with the double-dollar sign is a Conditional Compilation flag.
If I describe the package we can only see the public package:
SQL> desc my_pkg
FUNCTION PUBLIC_FUNCTION RETURNS DATE
SQL>
Now I set the conditional flag and re-compile the package, and as if by magic ...
SQL> alter session set plsql_ccflags='dev_env_test:true'
2 /
Session altered.
SQL> alter package my_pkg compile
2 /
Package altered.
SQL> desc my_pkg
PROCEDURE PRIVATE_PROC
FUNCTION PUBLIC_FUNCTION RETURNS DATE
SQL>
Privatising the functions is as simple as you think it would be:
SQL> alter session set plsql_ccflags='dev_env_test:false'
2 /
Session altered.
SQL> alter package my_pkg compile
2 /
Package altered.
SQL> desc my_pkg
FUNCTION PUBLIC_FUNCTION RETURNS DATE
SQL>
We can do lots more with conditional compilation. It's covered in the docs. Find out more.
I would be surprised if such a thing existed. The whole purpose of private procedures, functions and variables is that they are not visible to applications outside the package.
As #Robert said, it shouldn't be possible to access anything that is declared only in the package body outside of that package. Furthermore, creating a "special" spec for the purpose of running unit tests may not work either: if the body contains forward declarations (statements like those in the spec, usually found at the beginning of the body), then the "special" spec will conflict with those declarations and the package won't compile.
You can use pl/sql developer for testing the pl/sql procedures.
1) Go to the packages--> procedures/functions
2) Right click and select "test"
3) Enter the input parameters and click execute/run button and verify the results.
4) you can run with varieties of data sets and check in target tables.
5) Run with invalid data and check for the expected errors.
you can get more details at
http://www.handyinsight.com/2016/06/database-testing.html
temruzinn

Quick-n-dirty results: View results of Procedure OUT cursor in SQL Worksheet?

Platform: Oracle
Language: PL/SQL
Issue: Want to output a procedure OUT cursor into the SQLDeveloper SQLWosksheet.
Anyone know how to use the Oracle "Select * from Table( PipelinedFunction( Param ) ) " to check procedure code output cursors?
I am using Crsytal Reports off of an Oracle stored procedure. Crystal requires that a procedure return a cursor, which it fetchs and reads.
The procedure code I have is currently working, but I want to find the easiest way to view the effects of changes to the procedure code. I have SQLDeveloper available, and I'm doing my creation and sql testing in that. I would like to get a quick result visible in the SQL Developer Query Result window ("SQL Worksheet").
Is there a (simple) way to use a Function to read the cursor from the procedure? (and pipe that out to the Table function?)
Convoluted, I know, but I deal best when I can just see the results of code changes. If I can view the record results directly, it will speed up development of the report.
I know of the Table function and a little about pipelining in Oracle. I know a little about cursors in general and sys_refcursor. I know diddly about types and why I need them. (Isn't sys_regCursor supposed to get us away from that?)
The current procedure does an adequate but ungraceful series of queries, inserts to global temp tables (GTT), joins from GTT and original tables, more inserts, and more self-joins and then SELECTS the results into the OUT cursor. I might be able to do better relying on just cursors and such, but the current method is good enough to get results to the report.
I think I can handle SQL pretty well (for our purposes), but I am not an Oracle-specific developer... but I need help.
Anybody run across this? The whole idea was to speed my development for the procedure code, but I've spent a couple of days looking for a way to just get at the output... not what I had in mind.
Update:
I have tried some hare-brained schemes based on slivers that I've seen on the web... such as
Create or replace FUNCTION GET_BACKPLANE (
Node VARCHAR2 ) RETURN SYS_REFCURSOR
AS
RESULTS SYS_REFCURSOR;
BEGIN
Open Results for
Select Backplane(Results, Node) from Dual ;
... etc.
and
Create or replace Function GET_BACKPLANE (
NODE VARCHAR2 ) RETURN My_Table_Stru%ROWTYPE PIPELINED
AS
BEGIN ...
I don't think that Oracle is even considering letting me re-reference the output cursor from the procedure ("Results" is a sys_refcursor that holds the results of the last SELECT in the procedure). I don't know how to define it, open it, and reference it from the procedure.
I never got to the place where I could try
SELECT * FROM TABLE(GET_BACKPLANE( ... etc )
Sorry for any typos and bad Oracle Grammar... it's been a long several days.
SQL Developer allows us to use SQL*Plus commands in the Worksheet. So all you need to do is define a variable to hold the output of the ref cursor.
I may have misinterpreted the actual code you want to run but I'm assuming your actual program is a procedure Backplane(Results, Node) where results is an OUT parameter of datatype sys_refcursor and node is some input parameter.
var rc refcursor
exec Backplane(results=>:rc, Node=>42)
print rc
The output of the print statement is written to the Script Output pane.
Note that the use of SQL*Plus commands means we have to use the Run Script option F5 rather than execute statement.
Thanks for the help. In the end, I wound up brute-force-ing it...
Step by step:
Make a query, test a query,
create a global temp table from the structure,
add code to make another query off of that GTT, test the query,
create a global temp table from the structure,
etc.
In the end, I wound up running (anonymous block) scripts and checking the GTT contents at every stage.
The last part was to use the same last query from the original procedure, stuffing everything into the Cursor that crystal likes...
tomorrow, I test that.
But, I'll just force it through for the next procedure, and get it done in a day and a half instead of 2+ weeks (embarrassed).
Thanks,
Marc

Can I recover older Oracle pl/sql source code from a package body after i have replaced with newer code

I had created an Oracle PL/SQL package with a header and a body with lots of code.
Later, I ended up accidentally erasing the code from that body after reran the CREATE OR REPLACE PACKAGE BODY... statement with different source code (which actually I intended to save under a different package name).
Is there any way I can recover my older replaced source code from the package?
You might be able to get it back by using a flashback query on all_source.
e.g. my package body is currently at version 2, executing this query as a standard user:
SQL> select text
2 from all_source
3 where name = 'CARPENTERI_TEST'
4 and type = 'PACKAGE BODY';
TEXT
package body carpenteri_test
is
procedure do_stuff
is
begin
dbms_output.put_line('version 2');
end do_stuff;
end carpenteri_test;
10 rows selected.
I know I changed this around 9:30 this evening so after connecting as a SYSDBA user I ran this query:
SQL> select text
2 from all_source
3 as of timestamp
4 to_timestamp('04-JUN-2010 21:30:00', 'DD-MON-YYYY HH24:MI:SS')
5 where name = 'CARPENTERI_TEST'
6 and type = 'PACKAGE BODY';
TEXT
----------------------------------------------------------------------------
package body carpenteri_test
is
procedure do_stuff
is
begin
dbms_output.put_line('version 1');
end do_stuff;
end carpenteri_test;
10 rows selected.
More information on flashback can be found here. Tom Kyte also demostrates how to use flashback with all_source here.
Unless you have logging/auditing of DDL commands enabled, or a backup of the database, then the answer is almost certainly not
Database definitions, including stored procedures, should always be treated like source code, and maintained in a code repository

Resources