I have 2 databases, Oracle and SQlite. And I want to create exact copies of some of the Oracle tables in SQLite in one of my applications. Most of these tables contains more than 10,000 rows so copying each table by going through each row programmatically is not efficient. Also the table structure may change in the future so I want to achieve this using a generic way without hard-coding the SQL statements. Is there a any way to do this?
p.s. - This application is being developed using Qt framework. All the queries and databases are represented by QtSql module objects.
Can't help with Qt framework, but for large amounts of data is is usually better to use bulk-copy operations.
Export data from Oracle
http://download.oracle.com/docs/cd/B25329_01/doc/admin.102/b25107/impexp.htm#BCEGAFAB
Import data into SQLite
http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles
IHTH
What you probably really want to use is the Oracle Database Mobile Server, which can automatically synchronize a SQLite and an Oracle Database.
The recent release of the Oracle Database Mobile Server (formally called Oracle Database Lite Mobile Server) supports synchronization between an Oracle Database and a SQLite or a Berkeley DB database running on the client. It supports both synchronous and asynchronous data exchange, as well as secure communications between client and server. You can configure the Mobile Server to synchronize based on several options without the need to modify the application that is accessing the database.
You can also find an excellent discussion forum for questions from developers and implementers using the Mobile Server.
Related
We have a legacy process that runs on SSIS 2016 on Windows Server 2016, executes custom queries against databases on remote servers, pulls the results (thousands or sometimes millions of records) and stores them in a local SQL Server database. These other databases are on DB2 and Oracle 19c.
This process has always connected using an OLE DB driver and a data flow with OLE DB source and destination components. It also has always been slow.
Because of some article we read recently talking about how OLE DB transfers only 1 record at a time, but with ADO.NET this network transfer could be done in batches (is this even true?), we decided to try to use an ADO.NET driver to connect to DB2 and replace the OLE DB source and destination components by ADO.NET components.
The transfer we were using as test case, which involved 46 million records, basically flew and we could see it bring down around 10K records at a time. Something that used to run in 13 hours ran in 24 minutes with no other changes. Some small tweaks in the query allowed us to bring that time even lower to 11 minutes.
This is obviously major and we want to be able to replicate it with our Oracle data sources. Network bandwidth seems to have been the main issue, so we want to be able to transfer data from Oracle 19c to our SQL Server 2016 databases using SSIS in batches, but want to ask the experts what the best/fastest way to do this is.
Is Microsoft Connector for Oracle the way to go as far as driver? Since we're not on SQL Server 2019, this article says we also need to install the Oracle Client and Microsoft Connector Version 4.0 for Oracle by Attunity. What exactly is the Oracle Client? Is it one of these? If so, which one, based on our setup?
Also, should we use ADO.NET components in the data flow just like we did with DB2? In other words, is the single record vs. record batches difference driven by the driver used to connect, the type of components in the data flow or both need to go hand in hand for this to work?
Thanks in advance for your responses!
OLEDB connections are not slow by themselves - it's a matter or what features the driver has available to it. It sounds like the ADO.NET driver for DB2 allows bulk insert and the OLEDB one does not.
Regarding Oracle, the attunity driver is the way to go. You'll need to install the oracle driver as well. The links that you have look correct to me but I don't have access to test.
Also, please note that dataflows will batch data by default in increments of the buffer size. 10k rows for example.
What are the supported clients and client versions for Oracle Autonomous Database?
Any connection type supported by Oracle Net Services can be used to connect to Autonomous Database.
Oracle Call Interface (OCI) connections require installing the Oracle client software. For these connections, Oracle Database Client 11.2.0.4 (or higher) or the Oracle Instant Client 12.1.0.2 (or higher) are supported by Autonomous Database.
For JDBC Thin connections, the only requirement is to have JDK8 (or higher) installed.
You can also connect to Autonomous Database using the Oracle supplied tools such as Oracle SQL Developer (version 18.2 or higher recommended)and Oracle SQLcl.
Some of these clients and tools such as JDBC, SQL Developer, and SQlcl support TLS authentication without using any wallet; hence making life much easier as opposed to mTLS authentication, which requires a wallet. I definitely recommend checking out the TLS option without a wallet in ADB since it has several advantages as described in this blog post.
Another great option is Oracle Database Actions if you don't want to worry about supported versions, installing a client, whether to use a wallet or not. Database Actions is a web-based interface that uses Oracle REST Data Services to provide executing your SQL statements and scripts, creating Data Modeler diagrams, developing RESTful web services, managing JSON collections, and using the Data Load, Catalog, Data Insights, Business Models, and Data Transforms tools to load data from local and remote sources, view data in your tables and views, view objects in your data dictionary, and organize, analyze, and transform your data. You can access Database Actions from the ADB details page of your instance in Oracle Cloud Infrastructure console:
I also recommend referring to the documentation for more details on supported clients, client versions, and connection methods.
Disclaimer: I’m a Product Manager at Oracle.
I have an ERP application running with Oracle forms and Oracle database. Now I am planning to migrate this application to a java based enterprise application. will it be a good idea to keep the existing oracle database as back end and developing a web application with certain level of changes/additions in the DB design.
There are two facts to know before answer your questions:
has your database schema some oracle forms special structures or is it in 3rd normal form and simply stores data using keys and enforced referential integrity?
How much stored code contains your database?
Ad 1. Oracle forms don't have specific schema reqiurements. They work best if your schema if based on 3rd normal form. If your schema is like this, use it for new Java application. We have both forms and Java EE applications on same database schemas and is works fine.
Advantage is, if you have keys (primary, unique, foreign) in your schema. Use them when generating Java app.
Probably you will have to add #Version columns for optimistic locking (see https://docs.oracle.com/javaee/6/api/javax/persistence/Version.html). But there is not reason to build new schema for it.
Ad 2. Your will have to overwrite bigger part of database stored code (triggers, procedures, functions) to Java. In most cases this does not have dramatic impact to schema structure, but deal with it.
So - if your database schema is not tailored to some UI client needs AND you want only use a new client, use your schema. If not, create a new one.
We have two divisions in our company, one uses E1 on Oracle 11g the other uses SAP on Oracle 11g.
We also have a SQL Server system we use to data warehouse information once a night from both system to run our report server against.
The question I have is for pooled tables in SAP, such as A016, how would I get that information out of SAP?
Currently we have SSIS's setup with a linked server to the two Oracle servers which pull the data we need I just don't have the knowledge of SAP to find the Pooled tables.
if I can't pull the pooled tables because they don't physically exist is there a tool I can use in SAP to find out what tables the pooled table is getting it's information from? This way I can rebuild that table in SQL using a open query and some fun Joins.
Thanks
You have to access those tables using the application server. They can't be accessed directly from the database.
You'll probably want to write an ABAP program to extract the data you need go from there.
working on an Drupal architecture where I'd like to have a stand alone Drupal/DrupalCommerce installation with product definition based on content coming from an apart Oracle DB.
Does anybody have experience in integration between Drupal and ORacle.
I'm not speaking about replacing MySQL by ORacle but by using ORacle content in specific modules.
What is the best connections options : XML-RPC, REST, WebServices/SOAP ...
Did you try to define the Oracle database as a secondary database in the Drupal settings file?
http://www.php.net/manual/en/book.oci8.php
http://drupal.org/project/oracle
If you manage to connect, you could then write a custom module to switch to the secondary database, retrieve the data from the Oracle database and bring it into the Drupal context.
I would like to find out if you found other solutions.