I have an understanding that it might be the DBA's job to push the changes, much like a refresh, but without the DML (Data Manipulation).
Any comments/suggestions will be great!
There is no simple answer for this, but essentially the task of deploying code is similar for any computer language, with the main differences for the database component being
We can't drop and rebuild tables because we need to keep their contents.
Our code compiles in the database, so there are no binaries to deploy.
The first rule for PL/SQL development is that you should always, always work from source-controlled files, for example the code for mypackage would be in a source file named mypackage.pck (or whatever file extension works best with your chosen IDE, as long as it's not .sql). Don't edit database source code directly except for trivial testing when you don't care about keeping the changes.
Some sites only ever do incremental deployments, while others use a full teardown and rebuild from a release branch for major releases, which takes a bit more thought but is ultimately cleaner. Then deployment consists of running the scripts, recompiling the schema and perhaps running some tests and checks to ensure it has worked. You'll need a branching strategy, perhaps some kind of 'run everything in this folder' script, and ideally some tools such as Team City or Jenkins to automate as much as possible, though I don't think there is as much out there ready-made for PL/SQL as there is for more mainstream languages such as Java.
The deployment process is usually performed by an application support team as it does not require any DBA privileges unless you are creating schemas or roles etc, although some sites may organise support roles differently.
Yes. Partially it will involve DBA. But, as the developer, you probably need to provide the entire DDL script by exporting (probably in a file) to the DBA to deploy.
Check 'Importing and exporting SQL scripts' part of this link for exporting db scripts: https://docs.oracle.com/cd/B25329_01/doc/appdev.102/b25309/sql_rep.htm#BABBHEHA
Related
I need somehow compare local version of pl sql package with the one that is stored in database. Are there any "easy" way to do it?
Currently I download package from database, save it in some file and perform diff using some diff tool. That is a bit cumbersome, so I would like to have such feature in ide (pl sql developer is preferable).
Typically developers use versioning software to check differences, maintain version history, and help coordinate team development of code. PL/SQL code should be no different. There's many ways to handle code releases, the following is just what I've seen done, not necessarily the "best" way.
In the Oracle environments I've seen, CVS or SVN is used. Most approaches involve checking out latest code from repository, editing (tagging/branching), and checking in. When code is tested (development instance) and release is ready, the DBAs either grab the release scripts from repository and apply, or one individual is tasked with handing over the correct release scripts to DBAs (more common from my experience). Note that the database here is typically a user acceptance instance that mirrors the production instance. If app testing passes, the code is promoted to production.
If you want to sync directly between the database and your IDE, the one option I've seen is Toad Team Coding. Toad is not free, and this option will require additional objects installed on the database (metadata/tracking tables, etc). A good overview is found here, and good setup article is found here.
Team coding is very cool, but I would only install on a development environment. How you promote the code from there through your system is up to you.
It's fairly easy to automate the procedure you described with a sqlplus command file. Select from all_source, spool it to a file. Invoke the diff command as "host diff ..." or "!diff ..." within the command file. The -b and -B options to diff will ignore whitespace and blank lines respectively.
I usually create a solution folder in Visual Studio and put my DB scripts in them. I always use at least this set of scripts:
Drop model
Create model script
User functions
Stored procedures
Static data (lookup tables)
Test data (not deployed)
Then I simply combine them and run against an SQL Server so I'm able to recreate the whole DB in a single step (by combining these scripts into a single one and executing it).
Anyway. I've never used projects in either:
Visual Studio or
SQL Management Studio
I've tried creating SQL Server 2008 Database Project in Visual Studio 2010, but I'm somehow overwhelmed by all the possible server settings (which I prefer to stay default as set on the server anyway). So I'm a bit confused: Should I use this project template or should I just do the same thing I always did?
What do you use and why? What are advantages I may benefit from by using either?
If I were you I would continue to do it the way you are doing it. In fact I do! The advantages of having the actual .sql files right there in a folder for you to use/edit/look at in my opinion are far better than the advantages you get by using a DB project. DB Project would be used if you were doing something like Storage Reports, were you have to communicate with like 8 databases and compare then to 8 different databases and save result sets etc... Now don't get my wrong there are advantages of Database Projects, I just don't think they are actually doing much help when you have such a simple setup that works already.
Advantages of the SQL Server 2008 Database Project in VS10:
Not having to switch back and forth
from your current client you use to
communicate with your SQL server.
Decent Data and Schema compare tools.
Gives you a one-click way to reverse
engineer a database into source
control, and keep it up to date.
You can compare projects to physical
databases and vice-versa. (This makes it pretty easy to keep your database up to date, no matter where you make change it: file system database project, or in the physical database itself)
If the current tool your using is not specifically tailored to SQL Server, this one is.
Extremely helpful if you need to do
unit tests directly on the database
without using abstractions.
If you're looking for something a little less complicated, you might want to try SQL Source Control. This won't even require you to maintain scripts, as it doesn't this for you behind the scenes. It will, however, only work as a solution for you if you use either TFS or SVN. And it costs $295...
It has a 28-day trial period, so if you're happy to try it out, I'd be interested in your feedback.
I've just implemented build and deploy process which consists of java files, ant script and cmd files. In the process, a release manager will have to check out source, hit the build.cmd button and then carry a zip file over to a server.
I am wondering if it is worthwhile to make a GUI for it? So that the release manager does not need to check out source manually for example?
How do I start? I have quite limited knowledge of javax, but I very much like to learn.
Thanks,
Sarah
This sounds like something that could be handled by Hudson. It can check out source, run Ant scripts, etc., saving you the trouble of maintaining a GUI. I'd give that a shot before rolling your own.
I have helped develop the build process at my current company. The way we currently do it is with a script file. It checks out the latest code from the stable branch of our repository, performs some steps to get some data from a database (such as static SQL data that needs to be loaded at deployment), then compresses everything. The file is then distributed to our production servers and then the setup routine is executed. Everything is automatic and the script is written in Python. Python is great for these types of things because of the sheer number of libraries it has to help the developer.
Perhaps it may be useful to build a GUI for your deployment procedure -- typically this would be useful if the deployment requires user interaction to make decisions, such as "Which server shall I deploy to?", etc. But, if it's just a matter of doing things automatically, then a script file's the way to go. Choose your favourite language and dive in -- I of course recommend Python.
If you'd like to learn how to make a simple GUI in Java (since that seems to be what your company is familiar with), you should check out the stuff at this site:
http://java.sun.com/docs/books/tutorial/uiswing/index.html
I learned everything I know about Java from that site. The section on GUI programming is great.
Best of luck!
Shad
I have a Delphi application that has many dependencies, and it would be difficult to refactor it to use DUnit (it's huge), so I was thinking about using something like AutomatedQA's TestComplete to do the testing from the front-end UI.
My main problem is that a bugfix or new feature sometimes breaks old code that was previously tested (manually), and used to work.
I have setup the application to use command-line switches to open-up a specific form that could be tested, and I can create a set of values and clicks needed to be done.
But I have a few questions before I do anything drastic... (and before purchasing anything)
Is it worth it?
Would this be a good way to test?
The result of the test should in my database (Oracle), is there an easy way in testcomplete to check these values (multiple fields in multiple tables)?
I would need to setup a test database to do all the automated testing, would there be an easy way to automate re-setting the test db? Other than drop user cascade, create user,..., impdp.
Is there a way in testcomplete to specify command-line parameters for an exe?
Does anybody have any similar experiences.
I would suggest you plan to use both DUnit and something like TestComplete, as they each serve a different purpose.
DUnit is great for Unit Testing, but is difficult to use for overall application testing and UI testing.
TestComplete is one of the few automated testing products that actually has support for Delphi, and our QA Engineer tells me that their support is very good.
Be aware though that setting up automated testing is a large and time-consuming job. If you rigourously apply unit testing and auomated UI testing, you could easily end up with more test code than production code.
With a large (existing) application you're in a difficult situation with regards to implementing automated testing.
My recommendation is to set up Unit Testing first, in conjunction with an automated build server. Every time someone checks anything in to source control, the Unit Tests get run automatically. DO NOT try to set up unit tests for everything straight up - it's just too big an effort for an existing application. Just remember to create unit tests whenever you are adding new functionality, and whenever you are about to make changes. I also strongly suggest that whenever a bug is reported that you create a unit test that reproduces the bug BEFORE you fix it.
I'm in a similar situation. (Large app with lots of dependencies). There is almost no automated testing. But there is a big wish to fix this issue. And that's why we are going to tackle some of the problems with each new release.
We are about to release the first version of the new product. And the first signs are good. But it was a lot of work. So next release we sure need some way to automate the test process. That's why I'm already introducing unit tests. Although due to the dependencies, these are no real unit tests, but you have to start somewhere.
Things we have done:
Introduced a more OO approach, because a big part the code was still procedural.
Moved stuff between files.
Eliminated dependencies where possible.
But there is far more on the list of requirements, ensuring enough work for the entire team until retirement.
And maybe I'm a bit strange, but cleaning up code can be fun. Refactoring without unit tests is a dangerous undertaking, especially if there are a lot of side effects. We used pair programming to avoid stupid mistakes. And lots of test sessions. But in the end we have cleaner code, and the amount of new bugs introduced was extremely low.
Oh, and be sure that you know this is an expensive process. It takes lots of time. And you have to fight the tendency to tackle more than one problem in a row.
I can't answer everything, as I've never used testcomplete, but I can answer some of those.
1 - Yes. Regression testing is worth it. It's quite embarrassing to you as a developer when the client comes back to you when you've broken something that used to work. Always a good idea to make sure everything that used to work, still does.
4 - Oracle has something called Flashback which lets you create a restore point in the database. After you've done your testing you can just jump back to this restore point. You can write scripts to use it too, FLASHBACK DATABASE TO TIMESTAMP (FEB-12-2009, 00:00:00);, etc
We're looking at using VMWare to isolate some of our testing.
You can start from a saved snapshot, so you always have a consistent environment and local database state.
VMWare actions can be scripted, so you can automatically install your latest build from a network location, launch your tests and shut down afterwards.
Is it worth it?
Probably. Setting up and maintaining tests can be a big job, but when you have them, tests can be executed very easily and consistently. If your project is evolving, some kind of test suite is very helpful.
Would this be a good way to test?
I would say that proper DUnit test suite is better first step. However if you have large codebase which is not engineered for testing, setting up functional tests is even bigger pain than setting up GUI tests.
The result of the test should in my database (Oracle), is there an easy
way in testcomplete to check these values (multiple fields in multiple tables)?
TestComplete has ADO and BDE interface. Or you can use OLE interface in VBScript to access everything that's available.
Is there a way in testcomplete to specify command-line parameters for
an exe?
Yes.
One way to introduse unittesting in an (old) application could be to have a "Start database" (like the "Flashback" feature described by Rich Adams).
The program som unittest using DUnit to control the GUI.
Se the "GUI testing with DUnit" on http://delphixtreme.com/wordpress/?p=181
Every time the test is started by restoring to "Start database", because, then a known set of data, can be used.
I would need to setup a test database to do all the automated
testing, would there be an easy way to
automate re-setting the test db?
Use transactions: perform a rollback when the test completed. This should revert everything to the initial state.
Recommended reading:
http://xunitpatterns.com/
I'm working in a multi-developer environment in Oracle with a large package. We have a DEV => TST => PRD promotion pattern. Currently, all package edits are made directly in TOAD and then compiled into the DEV package.
We run into two problems:
Concurrent changes need to be promoted on different schedules. For instance, developer A makes a change that needs to be promoted tomorrow while developer B is concurrently working on a change that won't be promoted for another two weeks. When it comes promotion time, we find ourselves manually commenting out stuff that isn't being promoted yet and then uncommenting it afterwards...yuck!!!
If two developers are making changes at the same exact time and one of them compiles, it wipes out the other developer's changes. There isn't a nice merge; instead the latest compile wins.
What strategies would you recommend to get around this? We are using TFS for our source-control but haven't yet utilized this with our Oracle packages.
P.S. I've seen this posting, but it doesn't fully answer my question.
The key is to adopt a practice of only deploying code from the source control system. I'm not familiar with TSF, but it must implement the concepts of branches, tags, etc. The question of what to deploy then falls out of the build and release tagging in the source control system.
Additional tips (for Oracle):
it works best if you split the package spec and body into different files that use a consistent file pattern for each (e.g. ".pks" for package spec, and ".pkb" for package body). If you use an automated build process that can process file patterns then you can build all of the specs and then the bodies. This also minimizes object invalidations if you are only deploying a package body.
put the time in to configure an automated build process that is driven from a release or build state of your source control system. If you have even a moderate number of db code objects it will pay to be able to build the code into a reference system and compare it to your qa or production system.
See my answer about Tools to work with stored procedures in Oracle, in a team (which I have just retagged).
Bottom line : don't modify procedures directly with TOAD. Store the source as files, that you will store in source control, modify then execute.
Plus, I would highly recommend that each developer works on its own copy of the database (use Oracle Express, which is free). You can do that if you store all the scripts to create the database in source control. More insight can be found here.
To avoid 2 developers working on the same package at the same time:
1) Use your version control system as the source of the package code. To work on a package, the developer must first check out the package from version control; nobody else can check the package out until this developer checks it back in.
2) Don't work directly on the package code in Toad or any other IDE. You have no clue whether the code you are working on there is correct or has been modified by one or more other developers. Work on the code in the script you have checked out from version control, and run that into the database to compile the package. My preference is to use a nice text editor (TextPad) and SQL Plus, but you can do this in Toad too.
3) When you have finished, check the script back into version control. Do not copy and paste code out of the database into your script (see point 2 again).
The downside (if it is one) of this controlled approach is that only one developer at a time can work on a package. This shouldn't be a major problem as long as:
You keep packages down to a reasonable size (in terms of WHAT they do, not how many lines of code or number of procedures in them). Don't have one big package that holds all the code.
Developers are encouraged to check out code only when ready to work on it, and to check it back in as soon as they have finished making and testing their changes.
We use Oracle Developer Tools for Visual Studio.NET...plugs right into TFS
we do it with a Dev database for every stream, and labels for the different streams.
Our Oracle licensing gives us unlimited dev/test instances, but we are an ISV, you may have a different licensing option
You can use the Oracle developer tools for VS or you can use sql developer. SQL developer integrates with Subversion and CVS and you can download it for free. See here: http://www.oracle.com/technology/products/database/sql_developer/files/what_is_sqldev.html
We use Toad for Oracle with the TFS MSSCCI provider against TFS 2008. We use a Custom Tool that pulls database checkins from source control and packages them for release.
To my knowledge Oracle Developer Tools for Visual Studio.Net doesn't have any real source control integration with TFS or otherwise.
You might consider Toad Extensions for Visual Studio though it's not cheap, maybe $4k I think.
Another option is the Oracle Change Management Pack but believe it requires the Enterprise edition of Oracle which is much more pricey.
You may be interested in Gitora www.gitora.com. It helps managing Oracle database objects with Git.
This article about collaborative development with the Oracle database can also be helpful: http://blog.gitora.com/plsql-how-to-develop-two-features-simultaneously-but-deploy-only-one/
Full disclosure: I am the developer and author of the article.