We have a client that has Oracle Standard, and a project that would be ten times easier addressed using OLAP. However, Oracle only supports OLAP in the Enterprise version.
Migration to enterprise is not possible
I'm thinking of doing some manual simulation of OLAP, creating relational tables to simulate the technology.
Do you know of some other way I could do this? Maybe an open-source tool for OLAP? Any ideas?
You can simulate OLAP functionality using client side tools pointed at a relational database.
Personally I think the best tool for the job is probably Tableau Desktop. This is an amazingly sophisticated front end analytics tool that will make your relational data look multidimensional without much effort, and the tool itself is really mind blowing. They have a free trial so you can take it for a spin. We use Tableau heavily for our own analysis and have been very impressed. Of course, this tool also works with multidimensional databases as well, so if you end up with some cubes at the end of the day you can continue to use the Tableau front end.
As for open source, you could try out Palo - an open source MOLAP server and Excel front end.
If you are interesting in building your own reporting front end and use .NET there are a number of components (such as the DevExpress PivotGrid or the several tools from RadarSoft) that will do the same thing, but will require some elbow grease to get wired together.
I find that it's the schema that causes most of the issues people have with querying a database. OLAP forces you to either a flat table or a Star/snowflake schema which is easy to query and comparably faster to the source oltp tables. So if you ETL your source to a flat table or star schema you should get 80% of what you get from OLAP, the 20% being MDX and analytic functions and performance.
Note that you should get a perf boost with a star schema in relational database as well and Oracle probably has analytic functions in PL/SQL anyways.
Try an open-source OLAP server called 'Mondrian'. IIRC the XMLA API on this is sufficiently compatible with AS to fool Pivot Table Services, which would allow you to use it with ProClarity or Excel.
IIRC it was originally designed to work over Oracle - it is a HOLAP architecture using base tables in the underlying relational store and caching aggregates. You can also make use of materialised views and query rewrite in the underlying Oracle database to do aggregates.
A few more thoughts on this topic:
Actually, Oracle Standard does have an OLAP facility based on a descendent of Express embedded in the database engine and storing its internal data structures in BLOBs in the main tablespaces. Using this is technically possible but not necessarily advisable for the following reasons:
It uses a highly non-standard OLAP query engine with very little third party tool support (AFAIK ArcPlan is the only third-party OLAP front-end supporting 10g+ OLAP), poor documentation for the query language and almost no third party literature describing it. This will work with B.I. Beans if you feel like writing a JSP front-end. It is not compatible with MDX at all. As of early 2006 the best Oracle could do when asked about drillthrough (this functionality was not supported in Discoverer 'Drake') was to recommend building a JSP apllication using B.I. Beans.
The reason that there is no migration path from Standard to Enterprise is that Enterprise is actually what used to be Siebel Analytics. Standard is the old Oracle OLAP/Express descendant which Oracle partners recommended avoiding even before Oracle bought out Seibel. Oracle has not even attempted to support migrating.
From this point of view, Mondrian is actually the most cost-effective OLAP solution for an Oracle Standard Edition shop. You can get a supported version from an outfit called Pentaho1. The next cheapest is Analysis Services, which comes with SQL Server. Following that you are into the likes of Hyperion Essbase, which will be an order of magnitude more expensive than SQL Server or any supported verion of Mondrian.
Whilst MS SQL Server offers OLAP, you'll need an Enterprise licence to use a cube in a live environment that is web-facing.
You might want as well to give a try to www.icCube.com - we're quite flexible on the data-source used to populate the cube and are quite cost effective compared to the big actors of the market.
Related
We're considering Snowflake and want to understand how we could use it, and possibly other tools, to overcome one of our main problems - ETL! We currently use a legacy DWH with an ETL process consisting of SSIS and some views. This has all the common pitfalls of this methodology - most notably that it takes ages!
I was under the assumption that we'd move to an ELT model in Snowflake, I started to research tools to do the 'T' part of it, however, I'm just listening to this podcast: https://www.dataengineeringpodcast.com/snowflakedb-cloud-data-warehouse-episode-110/
And it's suggesting that just slapping a SQL View over something and exposing it in say PowerBI or Tableau is enough for the T part of things!...
Just wondering what people's experience was here?
- Do you do transformations just by writing a view in Snowflake?
- Do you use a third party tool specifically to address this need?
Secondary to this, for the Extraction and Loading, do you:
- Do this using Snowflake only
- Use a third party tool
I'm specifically interested if you do this to create some kind of timeseries in Snowflake from a non timeseries source. That's something we'd be keen to do.
This question is hard to answer without sounding opinionated, especially not knowing your use case. For what it's worth here is what I think:
Don't stick views on top of your tables and expose to a reporting tool unless you have a very very simple setup. If you're considering a tool like Snowflake then you will probably want to go for something more sustainable, this approach can become prohibitive in terms of cost and complexity in your views.
Use a third-party tool to manage your ELT process. Your choice of tool will depend on your internal skills and cloud strategy, have a look at the tools out there like Stich, Fivetran etc. If you don't mind having on-premise technologies why not stick with SSIS or use something like Apache Airflow (requires up-skilling)
Snowflake will not help you with the E of ELT, you will need to use a third-party tool to manage the extract of data from your other systems like SSIS. It will help with the L part, for this you can use Snowpipe or COPY commands which are available within the Snowflake ecosystem. Snowflake will also help you share your data with external parties which is really nice.
My organization has created a fairly complicated dimensional model in Snowflake using layers of SQL views, against which we can point our reporting tools. We use a separate replication tool for extraction from source systems and loading into Snowflake. Using views simplifies our approach in that we don't need to use an additional tool. It also makes managing the code easier than something like SSIS. For instance, we can search for code using the Snowflake interface or our version control tool instead of having to open individual SSIS packages.
What would be better if need to migrate from Oracle Forms 10g , Either Apex or ADF
If Apex then Why and Why not ADF
If Adf then Why and Why not Apex
Obviously, this will be an opinion-based answer.
They say "Forms is dead" for at least two decades now, but - Forms is still alive. Guys in India still develop applications in 6i, I'm not sure they switched to web yet (how do I know? By reading questions on different Oracle forums).
ADF looked like a great choice for everyone who wanted to abandon Forms. ADF looks pretty. ADF is Java. I don't like Java (but that's irrelevant). ADF is not a simple tool and people who don't speak Java can expect certain problems. You probably know how simple it is to create a master-detail form (for example, based on Scott's EMP and DEPT tables) using Forms? Matter of minutes. I don't know about ADF, but I've watched a Java developer who spent hours (literally) to do the same task (while, at the same time, another person created it in Delphi even faster than in Forms).
APEX is way simpler than ADF (and Forms, to be honest). All you need is a web-browser. No complex IAS or Web logic installation, JInitiator and stuff ... OK - embedded listener does its job if it must, but you'd better switch to ORDS. If you have fair (PL/)SQL background, creating an Apex application is matter of several hours/days, and you'll have a nice-looking application. How robust is it? Quite, as far as I can tell. What about you, as a developer? Lucky you, you came when Apex 19.x is out. In pre-dynamic actions era (3.x), you'd suffer in pain because Forms does everything with triggers - there are none in Apex. But now, now you have dynamic actions which are the closest thing to triggers so far.
If you keep the logic in the database (stored procedures, packages) and let APEX take care about nice presentation to users, you shouldn't expect too many problems. I've heard (just yesterday) that ADF developer I know does the same - keeps everything in the database and uses ADF to create a nice application. Users don't like Forms applications because they look ugly. But hey, what do they know? Forms applications - if well written - are really good, robust, reliable piece of software.
I've heard (on one of OUG conferences) that - on OpenWorld - there were zillion APEX presentations and zero ADF ones. Apex is in nowadays, ADF seems to be out.
Would I want to migrate my Forms applications to anything? No, I wouldn't. I sympathize with you.
Depends on the experience and knowledge of your developers and what language you want to develop in.
ADF is actually 2 products (well, really 4...) ADF Business Components (ADF BC) is the back-end server ORM framework and ADF Faces (JSF) is for front end server generated Web Pages.
ADF is a full JEE Solution using servlets, JSP/JSF, ORM and sophisticated concurrency and data management. It is for those who want a Java solution for their Apps. That being said, most ppl are leaving ADF Faces (Front End) for JET (JS) or APEX (PL/SQL) and are choosing ADF BC for its excellent ability to create RESTFul web services declaratively, which can be extended with custom Java code.
ADF BC can work with many different data sources and databases - basically anything that has a JDBC driver. APEX is Oracle DB ONLY.
The migration from Forms to APEX is much more direct and there are migration tools in APEX to help with this. APEX is all PL/SQL vs ADF which is Java.
If it were me I would use APEX if I want a Web App on top of an Oracle DB or are migrating from Froms. Knowing Java well, I would use ADF BC if I had to create RESTFul Web Services and need to integrate with different servers and sources. While I can create RESTFul web services in APEX/ORDS - it is a SQL/PL/SQL solution only, ADF BC allows for the use of Java across any data source.
I'm actually working in a situation where a .NET stack is managing an Oracle Database. Now, because the legacy code is consistently based on PL SQL stored procedures that deal with the majority of the work, the correct choice of driver to connect to the database is of primary importance.
For this reason, knowing that Oracle provides a large number of driver for the most known programming languages, I was trying to find a documented benchmark (even with all the problem and the influence of the context in which the tests are made) that could compare the different Oracle drivers for the different programming languages, just to support the hypothesis that the best choice in terms of performance for an isolated test microservice would be to use the Java driver in combination with Scala (Java is now property of Oracle now, after all).
Are there any on the internet that could support (or not) this hypothesis?
EDIT
I didn’t provide all the information. What I’m trying to achieve is develop a series of microservices focused on fetching data from the database and convert them into json to be consumed by the front end. .NET driver behaves seamlessly until the numbers become really overwhelming (> 1000 rows).
That’s why I was wondering if it could make any sense using JDBC to increase performance, having heard that, for instance, .NET driver for SQL server, both made by the same Company, performs 5 times better than the oracle one when it comes to gathering data from a cursor.
Your choice of drives may not give you the milage if most of the work is in PL/SQL or stored procedures. Having said that, jdbc is a good choice. However do not fight if developers are more familiar with other drivers like Oracle ODBC, oracle provider for .NET, ADO,etc.. all protocols have a Oracle drive to access Oracle DB.
You don’t have to convert to json. Oracle DB can convert it. Your network latency is more impacted by how big the pipe is ,array size,and packet size than protocols.
I want to ask what are the performance gains of pulling reports in Sharepoint 2010 directly from an Oracle DW using an ODBC Connection as opposed to building an OLAP Layer using SSAS and accessing the data that way.
It sounds like you're confusing terminologies and technologies. Comparing ODBC to OLAP and SSAS is like comparing apples to oranges. They are very different things used for very different purposes. The commonality is that they both deal with data... and apples and oranges are both fruit.
But, trying to read between the lines, OLAP, if configured correctly, will deliver actionable information (biz intel etc.) much more quickly and readily than data aggregated from a standard RDBMS. After all, that's what OLAP is designed for:
In computing, online analytical processing, or OLAP (play /ˈoʊlæp/),
is an approach to swiftly answer multi-dimensional analytical (MDA)
queries.1 OLAP is part of the broader category of business
intelligence, which also encompasses relational reporting and data
mining.[2]
Now that SQL Server 2012 is released, you may want to look into building a BISM (using SSAS) over your Oracle data and then displaying the results in SharePoint.
Although you can use the 'PowerPivot on SharePoint' mode to be able to host (render) the BISM using SharePoint resources, you'd rather get a separate server to host SSAS, so that effort of rendering (presumably) large reports doesn't slow down your SharePoint server.
Of course it is still possible build an OLAP cube using SSAS, but unless you need one or more of the features not yet available in the BISM model, you'd rather just build a BISM than an OLAP cube, as the BISM leverages Vertipaq (so it's simpler to build, i.e. requires less ETL from Oracle). For more check out my deck on slideshare on the BISM
I need to develop a program that must delete and insert data into an Oracle database. Is it possible to use LINQ to SQL with Oracle?
For development I use MS SQL server but it will be a Oracle database in production. What do you recommend?
Officially No. Linq to SQL was originally build with the ability to swap out the data provider to allow connections to other databases, but they disabled this functionality in the released versions to encourage people to use more stable and supported data access layers (like EF). The recommended approach is to use Entity Framework if you want to switch between SQL and Oracle.
Also, Patrick is very right, make sure you are developing and testing against the same database platform you are going to use in production, there is a world of difference in how they operate. Sure, you should be able to abstract it away to not care about whether you are using SQL or Oracle, but that is almost never really the case.
No, you can't. Although LINQ to SQL was initialy designed with multi-database support in mind (you can see this when looking at the code using .NET Reflector) using a provider model, this model was never made public and Microsoft has no intensions in adding multi-database support to LINQ to SQL.
If you need multi-database support, please use Entity Framework.
No, LINQ-to-SQL doesn't support Oracle. Internally, the project had support for multiple back-ends, but this never made it into the final public release. I believe LINQ-to-Entities supports other databases.