I'm actually working in a situation where a .NET stack is managing an Oracle Database. Now, because the legacy code is consistently based on PL SQL stored procedures that deal with the majority of the work, the correct choice of driver to connect to the database is of primary importance.
For this reason, knowing that Oracle provides a large number of driver for the most known programming languages, I was trying to find a documented benchmark (even with all the problem and the influence of the context in which the tests are made) that could compare the different Oracle drivers for the different programming languages, just to support the hypothesis that the best choice in terms of performance for an isolated test microservice would be to use the Java driver in combination with Scala (Java is now property of Oracle now, after all).
Are there any on the internet that could support (or not) this hypothesis?
EDIT
I didn’t provide all the information. What I’m trying to achieve is develop a series of microservices focused on fetching data from the database and convert them into json to be consumed by the front end. .NET driver behaves seamlessly until the numbers become really overwhelming (> 1000 rows).
That’s why I was wondering if it could make any sense using JDBC to increase performance, having heard that, for instance, .NET driver for SQL server, both made by the same Company, performs 5 times better than the oracle one when it comes to gathering data from a cursor.
Your choice of drives may not give you the milage if most of the work is in PL/SQL or stored procedures. Having said that, jdbc is a good choice. However do not fight if developers are more familiar with other drivers like Oracle ODBC, oracle provider for .NET, ADO,etc.. all protocols have a Oracle drive to access Oracle DB.
You don’t have to convert to json. Oracle DB can convert it. Your network latency is more impacted by how big the pipe is ,array size,and packet size than protocols.
Related
Does PreparedStatement.setFetchDirection(ResultSet.FETCH_FORWARD) make a difference for an Oracle DB?
I'm on project to upgrade the database and application server for the power facilities and line maintenance system of a large electrical utility company. We are testing the new set up now (Oracle 12c and JBoss 7).
Sometimes the queries to the db are taking minutes to get an answer. I've suggested doing PreparedStatement.setFetchSize(int rows), where rows being the number of rows of information the user wants to see on his screen. Many posters here in Stackoverflow and elsewhere have said setFetchSize() is important to getting the network connection optimized.
But I noticed the PreparedStatement also has setFetchDirection(). There's FETCH_FORWARD, FETCH_REVERSE and FETCH_UNKNOWN. I've seen this mentioned many times, but I've not seen anyone saying how much of a benefit it is.
If it helps to understand the situation - the power company services 30 million customers, with decades of records (from nuclear power plants down to utility poles). The users of the service will be maintenance crews, office staff, both from with the company and from subcontractors. So, the computer skills and patience of the users may not tolerate a 3 minute wait to find the locations of the transformers they need to repair.
Oracle says
"The JDBC 2.0 standard allows the ability to pre-specify the
direction, known as the fetch direction, for use in processing a
result set. This allows the JDBC driver to optimize its processing."
However Oracle also says "the Oracle JDBC drivers support only the forward preset value" ResultSet.FETCH_FORWARD
"The values ResultSet.FETCH_REVERSE and ResultSet.FETCH_UNKNOWN are
not supported."
So, my guess is that if you are using JDBC 2.0 standard drivers, then you can optimize fetch by saying setFetchDirection(ResultSet.FETCH_REVERSE) when wanting to find overdue maintenance work. But if you are using Oracle JDBC drivers you are out of luck.
(Downvote me if I'm wrong - I know that improves your geek creed.)
Am looking for a solution to a problem that I have - I have some applications that connect to Oracle databases and we need to filter out the query results that is returned based on the user role and the row content of the Oracle database. RBAC and Oracle VPD (LBAC) is not an option as this is considered too unwieldy and difficult to maintain, even though LBAC is already implemented.
Am looking at application and database-agnostic solutions such as IBM DataPower and Websphere WTX that are more used in the SOA space as enterprise application brokers. I am aware of the challenge in using these or other solutions at the network layer which involves deconstructing Oracle TNS packets and not being able to package them back. Am wondering if there is a solution to this problem that I have using DataPower or any other similar?
Or are there any alternate solutions to solve this problem?
Our company is in the process of creating an ASP.NET service to accept XML data sent from ERP systems such as Oracle. We have no experience (at all) with Oracle, so please excuse the simplicity of this question.
I see online that Oracle has a tool called JDeveloper that can hook up to WCF Services that use a DataContract/WSDL to send/receive data with relative ease.
Can anyone advise about the situation regarding WebAPI's, where no WSDL or DataContracts exist? Is it simple to craft a POST in Oracle to send to a WebAPI, or is the former option better/easier to work with?
Thanks in advance.
It's simple enough to call web service directly from Oracle:
There are a good support of XML/XSLT/XQuery to construct requests and parse responses (XML DB)
Oracle have an API to work with HTTP/HTTPS requests (UTL_HTTP package).
So if you decide to call web service from Oracle - it's possible and relatively simple for SOAP and REST web-services.
You can find example code in this answer on StackOverflow.
Update - answer on a comment
To make it clear, example above isn't work at a "database query level" because it's implemented on PL/SQL. Oracle Database engine natively incorporates support for two different languages:
traditional SQL, which is ANSI SQL standards implementation (traditionally with some incompatibilities and extensions);
PL/SQL, which is procedural programming language tightly integrated with a traditional SQL.
This two things are really different. Even there are a common questions about performance affected by switching context between SQL and PL/SQL engines and mostly caused by improper procedures design.
PL/SQL as a procedural language can access a rich set of APIs, provided by Oracle as a set of built-in packages. Among others there are a number of packages directly related to network communication protocols and standards: UTL_TCP, UTL_URL, UTL_SMTP, UTL_MAIL, UTL_INADDR, UTL_HTTP, HTP, HTF, DBMS_LDAP.
Needs to be said, that there are a set of APIs, provided to support publishing of PL/SQL code on the web. Set of OWA_xxxx packages supports access through mod_plsql. Another thing is a support for publishing SOAP web services in Oracle XML DB.
If you need to unload data from Oracle to web service on a schedule then look at DBMS_SCHEDULE and DBMS_JOB packages to start unload procedures periodically.
Most of this system packages implemented on Java and it's possible to write your own Java extensions callable from PL/SQL.
P.S. There are a UTL_DBWS package dedicated for implementation of calls to SOAP services from Oracle Database, but seems that it produces more problems than solves and I can't find reference to it in 11g documentation (10g only).
P.P.S. Some statements may be slightly inaccurate or contain exaggerations, but that should be enough to understand the overall picture.
I've seen a couple of SQLite performance questions here on Stackoverflow, but the focus was on websites, and I'm considering using this DB in a client-server scenario:
I expect 1-10 clients for one server for now, could go up to 50 or more in the future.
slightly more reads than writes
the DB would sit behind a server process (i.e: not using direct DB access through a network)
Would using SQLite make the app less responsive as opposed to using PostgreSQL? My intuition tells me that it should be ok for these loads, but maybe someone has some practical experience with this kind of scenario.
I did use SQLite for a major client/server product used with ~10 concurrent users and I deeply regret that decision. In my opinion - PostgreSQL is much more suitable for client/server scenarios than SQLite due to its fine locking granularity.
You simply can't get very far when the entire database is locked whenever someone needs to write something ..
I like SQLite very much (I even wrote a commercial utility for comparing SQLite databases - SQLite Compare but I don't think it fits the bill when you have client/server scenarios.
Even SQLite's author says that it should be used as a replacement for custom file formats and not as a full blown database server. I wish I took his advice more seriously..
You didn't mention what operating system and Postgres versions you are using. However, before considering change of database engine, try to do some logging and benchmarking your current database with typical usage, then optimize "heaviest" questions. And maybe your backend processing load makes DB question time irrelevant? As SQLite is a file-based DBMS, concurrent access from multiple processes will degrade performance when client number grows up (edited after comment)
Following question may be helpful: How Scalable is SQLite?
I would confirm to S.Lott's answer.
I dont know how SQLite performs in comparison to PostgreSQL, since I don't know any newer meassurements, but my own experience with SQLite in a rather similar environment is rather good.
The only thing that might cause troubles in my view is that you have rather many writes. But it all depends on the total number per second I would say.
Also your setting to have one server process is optimal for SQLite in my opinion -- so you circumvent its weakness in multi-tasking.
We have a client that has Oracle Standard, and a project that would be ten times easier addressed using OLAP. However, Oracle only supports OLAP in the Enterprise version.
Migration to enterprise is not possible
I'm thinking of doing some manual simulation of OLAP, creating relational tables to simulate the technology.
Do you know of some other way I could do this? Maybe an open-source tool for OLAP? Any ideas?
You can simulate OLAP functionality using client side tools pointed at a relational database.
Personally I think the best tool for the job is probably Tableau Desktop. This is an amazingly sophisticated front end analytics tool that will make your relational data look multidimensional without much effort, and the tool itself is really mind blowing. They have a free trial so you can take it for a spin. We use Tableau heavily for our own analysis and have been very impressed. Of course, this tool also works with multidimensional databases as well, so if you end up with some cubes at the end of the day you can continue to use the Tableau front end.
As for open source, you could try out Palo - an open source MOLAP server and Excel front end.
If you are interesting in building your own reporting front end and use .NET there are a number of components (such as the DevExpress PivotGrid or the several tools from RadarSoft) that will do the same thing, but will require some elbow grease to get wired together.
I find that it's the schema that causes most of the issues people have with querying a database. OLAP forces you to either a flat table or a Star/snowflake schema which is easy to query and comparably faster to the source oltp tables. So if you ETL your source to a flat table or star schema you should get 80% of what you get from OLAP, the 20% being MDX and analytic functions and performance.
Note that you should get a perf boost with a star schema in relational database as well and Oracle probably has analytic functions in PL/SQL anyways.
Try an open-source OLAP server called 'Mondrian'. IIRC the XMLA API on this is sufficiently compatible with AS to fool Pivot Table Services, which would allow you to use it with ProClarity or Excel.
IIRC it was originally designed to work over Oracle - it is a HOLAP architecture using base tables in the underlying relational store and caching aggregates. You can also make use of materialised views and query rewrite in the underlying Oracle database to do aggregates.
A few more thoughts on this topic:
Actually, Oracle Standard does have an OLAP facility based on a descendent of Express embedded in the database engine and storing its internal data structures in BLOBs in the main tablespaces. Using this is technically possible but not necessarily advisable for the following reasons:
It uses a highly non-standard OLAP query engine with very little third party tool support (AFAIK ArcPlan is the only third-party OLAP front-end supporting 10g+ OLAP), poor documentation for the query language and almost no third party literature describing it. This will work with B.I. Beans if you feel like writing a JSP front-end. It is not compatible with MDX at all. As of early 2006 the best Oracle could do when asked about drillthrough (this functionality was not supported in Discoverer 'Drake') was to recommend building a JSP apllication using B.I. Beans.
The reason that there is no migration path from Standard to Enterprise is that Enterprise is actually what used to be Siebel Analytics. Standard is the old Oracle OLAP/Express descendant which Oracle partners recommended avoiding even before Oracle bought out Seibel. Oracle has not even attempted to support migrating.
From this point of view, Mondrian is actually the most cost-effective OLAP solution for an Oracle Standard Edition shop. You can get a supported version from an outfit called Pentaho1. The next cheapest is Analysis Services, which comes with SQL Server. Following that you are into the likes of Hyperion Essbase, which will be an order of magnitude more expensive than SQL Server or any supported verion of Mondrian.
Whilst MS SQL Server offers OLAP, you'll need an Enterprise licence to use a cube in a live environment that is web-facing.
You might want as well to give a try to www.icCube.com - we're quite flexible on the data-source used to populate the cube and are quite cost effective compared to the big actors of the market.