I have an ETL project running on-premises using ODI.
This ETL is quite customized with python scripts to perform tasks such as download/uploading files to Sharepoint, reading and parsing Excel files etc...
If I would want to migrate this to OCI, what would be the best service(s) for it? Is it a simple migration or a redo using a new approach?
I see that we have Data Integration workspaces but it seems not to be able to handle this type of customization...
Not sure about Data Flows. I am definitely not familiar with all Oracle stack, so I need some help :)
Thank you
Related
I am trying to get my Oracle DB content into GCP - BigQuery specifically, and then keep both in synch. I have not been able to find a standard way of doing this using GCP tools without using third-party software. Has anyone tried this? Any recommendations?
I need some example where we can version control our db creation for google cloud datastore.
I can see example for the google cloud mysql https://medium.com/#ramzi.maalej/micronaut-using-google-cloud-sql-9b2fc466c01a but is there any documentation by google cloud to work with the google cloud datastore and liquibase.
Any help or example on how to setup liquibase with cloud datastore will be really appreciated
Even datastore does not offer a versioning system, maybe this could be created from the scratch using the export tool and some code in the back.
https://cloud.google.com/datastore/docs/export-import-entities#rest
For example Visual Studio has a tool to make migrations between the structure of the database, this is aimed to Entity Framework that maps relational databases,
But the most important approach is the thing that NoSQL databases maybe doest have context in each change.
https://guptakumartanuj.wordpress.com/2017/09/04/approaches-to-support-versioning-through-nosql-database-like-document-db/
I'm working on a project to create hybrid SQL database per tenant model. While we were able to replicate the on-premise databases to databases in Azure. I'm not able to find a way to continuously sync both on-prem DB and cloud DB (We cannot use data sync agent or transaction replication). We are looking for any other alternatives that we can try to achieve our purpose.
Also, how does synchronization works when the internet is down and cannot sync with cloud?
Sorry for my ignorance since I'm new to this field.
Thanks.
why can't you use data sync agent? It can be installed in other machine that has access to the database and internet.
We are using a Java Endeca API to perform ETL on various datasources on our production servers. It appears that Oracle Endeca Integrator Studio (Clover ETL) has the ability to perform many of these tasks. But it would not be possible to run Endeca Integrator Studio on production servers which lack GUI features. Is there a way to have Integrator export scripts that could be run in command mode on a Linux server?
Take a look at the Oracle Endeca Integrator Server product. It is a long-running, headless service with a web management interface. You can then execute Integrator projects and ETL graphs on the remote server, or schedule them to run periodically.
Endeca 3.0 has Integrator web Administration console as well but I am not sure to what extent it would be useful to you.
I've recently installed the latest version of SQL Server (2008 R2 x64) and I'm now trying get my first steps to create an ETL.
Is it possible to create an ETL with SQL Server 2008 R2 Express, and if not is there some tool that provides that for free?
What I plan to do is collect data from multiple sources like a web services, rss feeds, a parsed url request, a file or maybe other database engine. Transform this collected data for an understandable schema (for my purposes) and store this in my SQL Server database.
If you are looking for a rather simple, yet very flexible and more code-based approach as an alternative to the SSIS (Integration Services), I'd recommend you have a good look at the Rhino ETL code library by Ayende Rahien.
Check out these resources:
Rhino ETL 2.0
Object-oriented ETL using Rhino ETL
Write ETL jobs in pure C#
Rhino ETL is a code-based, programmer-oriented approach - it gives you nice, usable base classes, from which you can inherit and do just about anything - with very little effort.
Integration Services, which is the product of SQLSERVER for this kind of tasks is not supported by the Express Edition.
Although, you can take the long way of making the integrations using plain TSQL and features of Express Edition like linked server (for other databases, access and excel), sqlcmd (for file system) and XML support(for web services).