My team is migrating an Oracle Form and Reports-based system into Java/Grails system. Since the destination programming language is very different from the source language, we cannot change the fact that there is a 100% recoding to be done. Some stored procedures are short, converting them is easy.
I stumble across a very important module with the name "End of Day (EOD) Batch Process". Inside this oracle form is a huge package with 4369 lines of pure PL/SQL code. I am becoming anxious on converting this batch process. Is there any way to move this package to our database? Aside from the actual batch process underlying this package, there are also form-specific functions inside that are needed to be considered on the conversion (set_application_property(), :b_main.ctrl_something := 'X' and ADD_GROUP_COLUMN()).
Does removing this form specific functions enough for the conversion? Or is it really not possible to convert this form package into database package?
Related
I need your help to one strategic question in my programing for database.
I work in Delphi 2010 and SQL Server 2012. Components Sdac for access to database. I quite often use a stored procedure for very many often absolutely different tasks. Fortunately SQL Server allows different result sets from one stored procedure.
I have a parameter #Wmode int (and several others too) and each value of wmode is separate task. Some of this are heavy other not, some have quite a lot of code some very few.
I use this approach because I try to put as much logic of application in code of stored procedures because at least for me its much easier to make any modifications, to test, debug, to set logs or to execute separately of my application.
Tell me please how much this approach can affect a performance this requests to database and how i can improve it. Is it better for me to use such approach only to seldomly executing parts of application. Can you give some links read on this subject.
I am generating code for Oracle Stored Procedure (SP) based on a dependency graph. In order to reduce recompilation unit size I have organised them in Oracle Packages. But this is resulting in large number of packages (250+). The number of procedures are 1000+.
My question: Will this large number of package create any performance issues with Oracle 11gR2+ ? Or will there be any deployment/management related issues ? Can somebody share their experience on working with large number of Oracle packages ?
In one of the products that I've worked on, the schema had many thousands of stored procedures, functions and packages, totalling almost half a million lines of code. Oracle shouldn't have any issues with this at all. The biggest headache was maintenance and version control of the objects.
We stored each package header and body in separate files so that we could version them independently (the header typically changes much less frequently than the body), and we used an editor that supported ctags to make navigation within a package more manageable. When you have a hundred or more procedures and functions within a package, finding the right place to actually make changes takes as much time as actually doing the work! Another great tool was OpenGrok, which indexes the entire code base and makes searching for things super quick.
Deployment wise, we just used a simple script that wrapped SQL*Plus to load the files and log any issues with compilation or connectivity. There are more advanced tools that sit on top of your source control system and "manage" deployment and dependencies, but we never found that it was necessary.
The purpose of writing packages in oracle to implement the concept of modular methodology which explained as follows:
Consolidate logical procedure and functional under one package
There is way to define member variable in global and can be accessed with in package or outside packages
The program units defined in package will be loaded at once in memory for processing and reduces context switching time
More details provided under link:
https://oracle-concepts-learning.blogspot.com/
I have to use an PL/SQL package as API for importing data into an Oracle database. I'm doing this within an Java application with the latest ojdbc driver. All statements (of cause PreparedStatements) I'm using during the import are initialized only one time and reused for every set to import.
Now I'm facing following problem: The first call of an procedure of the package takes over 90% of the time for one set. I have to call about 10 procedures during the import and the first one takes about 4 seconds the rest about 0.4 seconds. It doesn't matter if it's the 10th or 100,000th set to import the first procedure call allways takes that time.
Important to know is, if I'm calling another procedure on first position this on takes the 90%. So, may be I'm wrong, it is something about the package initialization? But if I'm (re-)using prepared statements, shouldn't that happen only at first call?
The PL/SQL package has about 10,000 lines of code and also calls several other packages during the import.
So now my questions are:
What are possible reasons for this problem? And what are potential solutions?
Are there any tools I can use to identify the causer?
EDIT: I could identify the cause of the slow import. It had nothing to do with wrong code or something. The reason was simply the kind of data I used in my test scenario. My mistake was importing allways the same data.
If thread one made an update on a data-set in the first procedure it was holding an lock on this row until the commit after the complete import. Thread two to n were trying to update exactly the same row. The result is effectivly a synchronization of all threads.
First of all, this is not normal. So there is definitely something awry with your code. But without being able to see your source there's no way we're going to be able to spot the problem. And frankly I don't want to debug 10000 LOC, not even mine let alone yours. Sorry.
So the best we can do is give you some pointers.
One:
"The first call of an procedure of the package takes over 90% of the
time for one set. .... if I'm calling another procedure on first
position this on takes the 90%"
Perhaps there is some common piece of coding which every procedure executes that behaves differently depending on whether the calling procedure is the first one to execute it in any given run. You need to locate that rogue code.
Two:
" I've used the profiler in pl/sql developer. The execution is very
fast there. "
Your program behaves differently depending on whether you call it from PL/SQL Developer of JDBC. So there is a strong possiblity that the problem lies not in the PL/SQL code but in the JDBC code. Acquiring database connections is definitely one potential source of pain. Depnding on your architecture, network traffic may be another problem: are you returning lots of data to the Java program which is then used in subsequent procedural calls?
In short: you either need to identify something common in your PL/SQL code which can cause the same outcome in different proocedural calls or identify what happens differently when you call the program in PL/SQL Developer and JDBC.
We're discussing the performance impact of putting a common function/procedure in a separate package or using a local copy in each package.
My thinking is that it would be cleaner to have the common code in a package, but others worry about the performance overhead.
Thoughts/experiences?
Put it in one place and call it from many - that's basic code re-use. Any overhead in calling one package from another will be minuscule. If they still doubt it, get them to demonstrate the performance difference.
The worriers are perfectly at liberty to prove the validity of their concerns by demonstrating a performance overhead. that ought to be trivial.
Meanwhile they should consider the memory usage and maintenance overhead in repeating code in multiple places.
Common code goes in one package.
Unless you are calling a procedure in a package situated on a different data base over a DB link, the overhead of calling a procedure in another package is negligible.
There are some performance concerns, as well as memory concerns, but they are rare and far between. Besides, they fall into "Oracle black magic" category. For example, check this link. If you can clearly understand what that is about, consider yourself an accomplished Oracle professional. If not - don't worry, because it's really hardcore stuff.
What you should consider, however, is the question of dependencies.
Oracle package consists of 2 parts: spec and body:
Spec is a header, where public procedures and functions (that is, visible outside the package) are declared.
Body is their implementation.
Although closely connected, they are 2 separate database objects.
Oracle uses package status to indicate if the package is VALID or INVALID. If a package becomes invalid, then all the other packages
that depend on it become invalid too.
For example, If you programme calls a procedure in package A, which calls a procedure in package B, that means that
you programme depends on package A, and package A depends on package B. In Oracle this relation is transitive and that means that
your programme depends on package B. Hence, if package B is broken, your programme also brakes (terminates with error).
That should be obvious. But less obvious is that Oracle also tracks dependencies during the compile time via package specs.
Let's assume that the specs and bodies for both of your package A and package B are successfully compiled and valid.
Then you go and make a change to the package body of package B. Because you only changed the body, but not the spec,
Oracle assumes that the way package B is called have not changed and doesn't do anything.
But if along with the body you change the package B's spec, then Oracle suspects that you might have changed some
procedure's parameters or something like that, and marks the whole chain as invalid (that is, package B and A and your programme).
Please note that Oracle doesn't check if the spec is really changed, it just checks the timestemp. So, it's enough just to recomplie the spec to invalidate everything.
If invalidation happens, next time you run you programme it will fail.
But if you run it one more time after that, Oracle will recompile everything automatically and execute it successfully.
I know it's confusing. That's Oracle. Don't try to wrap your brains too much around it.
You only need to remember a couple of things:
Avoid complex inter-package dependencies if possible. If one thing depends on the other thing, which depends on one more thing and so on,
then the probability of invalidating everything by recompiling just one database object is extremely high.
One of the worst cases is "circular" dependencies, when package A calls a procedure in package B, and package B calls procedure in package A.
It that case it is almost impossible to compile one without braking another.
Keep package spec and package body in separate source files. And if you need to change the body only, don't touch the spec!
I am not a DBA. However, I work on a web application that lives entirely in an Oracle database (Yes, it uses PL/SQL procedures to write HTML to clobs and then vomits the clob at your browser. No, it wasn't my idea. Yes, I'll wait while you go cry.).
We're having some performance issues, and I've been assigned to find some bottlenecks and remove them. How do I go about measuring Oracle performance and finding these bottlenecks? Our unhelpful sysadmin says that Grid Control wasn't helpful, and that he had to rely on "his experience" and queries against the data dictionary and "v$" views.
I'd like to run some tests against my local Oracle instance and see if I can replicate the problems he found so I can make sure my changes are actually improving things. Could someone please point me in the direction of learning how to do this?
Not too surprising there are entire books written on this topic.
Really what you need to do is divide and conquer.
First thing is to just ask yourself some standard common sense questions. Has performance slowly degraded or was there a big drop in performance recently is an example.
After the obvious a good starting point for you would be to narrow down where to spend your time - top queries is a decent start for you. This will give you particular queries which run for a long time.
If you know specifically what screens in you front-end are slow and you know what stored procedures go with that, I'd put some logging. Simple DBMS_OUTPUT.put_lines with some wall clock information at key points. Then I'd run those interactively in SQLNavigator to see what part of the stored procedure is going slow.
Once you start narrowing it down you can look to evaluate why a particular query is going slow. EXPLAIN_PLAN will be your best friend to start with.
It can be overwhelming to analyze database performance with Grid Control, and I would suggest starting with the simplier AWR report - you can find the scripts to generate them in $ORACLE_HOME/rdbms/admin on the db host. This report will rank the SQL seen in the database by various categories (e.g. CPU time, disk i/o, elapsed time) and give you an idea where the bottlenecks are on the database side.
One advantage of the AWR report is that it is a SQL*Plus script and can be run from any client - it will spool HTML or text files to your client.
edit:
There's a package called DBMS_PROFILER that lets you do what you want, I think. I found out my IDE will profile PL/SQL code as I would guess many other IDE's do. They probably use this package.
http://www.dba-oracle.com/t_dbms_profiler.htm
http://www.databasejournal.com/features/oracle/article.php/2197231/Oracles-DBMSPROFILER-PLSQL-Performance-Tuning.htm
edit 2:
I just tried the Profiler out in PL/SQL Developer. It creates a report on the total time and occurrences of snippets of code during runtime and gives code location as unit name and line number.
original:
I'm in the same boat as you, as far as the crazy PL/SQL generated pages go.
I work in a small office with no programmer particularly versed in advanced features of Oracle. We don't have any established methods of measuring and improving performance. But the best bet I'd guess is to try out different PL/SQL IDE's.
I use PL/SQL Developer by Allaround Automations. It's got a testing functionality that lets you debug your PL/SQL code and that may have some benchmarking feature I haven't used yet.
Hope you find a better answer. I'd like to know too. :)
"I work on a web application that
lives entirely in an Oracle database
(Yes, it uses PL/SQL procedures to
write HTML to clobs and then vomits
the clob at your browser"
Is it the Apex product ? That's the web application environment now included as standard part of the Oracle database (although technically it doesn't spit out CLOBs).
If so there is a whole bunch of instrumentation already built in to the product/environment (eg it keeps a rolling two-week history of activity).