We are using a Java Endeca API to perform ETL on various datasources on our production servers. It appears that Oracle Endeca Integrator Studio (Clover ETL) has the ability to perform many of these tasks. But it would not be possible to run Endeca Integrator Studio on production servers which lack GUI features. Is there a way to have Integrator export scripts that could be run in command mode on a Linux server?
Take a look at the Oracle Endeca Integrator Server product. It is a long-running, headless service with a web management interface. You can then execute Integrator projects and ETL graphs on the remote server, or schedule them to run periodically.
Endeca 3.0 has Integrator web Administration console as well but I am not sure to what extent it would be useful to you.
Related
I have an ETL project running on-premises using ODI.
This ETL is quite customized with python scripts to perform tasks such as download/uploading files to Sharepoint, reading and parsing Excel files etc...
If I would want to migrate this to OCI, what would be the best service(s) for it? Is it a simple migration or a redo using a new approach?
I see that we have Data Integration workspaces but it seems not to be able to handle this type of customization...
Not sure about Data Flows. I am definitely not familiar with all Oracle stack, so I need some help :)
Thank you
As my team is planning to install SonarQube on a central platform where it will be used by projects across organization please suggest which database is recommended for handling multiple projects in Sonarqube MYSQL or Oracle ?
Also, does it matter if the db is installed on the same server on which SOnarQube will be installed (considering RHEL platform) in terms of performance or should it be recommended to use external DB with SonarQube .
For MySQL, the MXJ connector makes it very easy to launch a managed MySQL instance.
I know that Oracle provides Oracle XE for quick setup, but I've only found an RPM distribution that needs to be installed. Is there a neatly packaged jar that I can just drop in the classpath and start up by calling a specific JDBC url, a la HSQLDB or MXJ?
I'm interested in having developers use this locally for running tests, as well as on our continuous integration server.
The short answer is No. Oracle is a big meaty chunk of database. Amongst other things, it generally expects itself to be run by its own special user rather than the client user.
For simplicity, your best bet is a separate DB server with each of your developers having their own username/password (and hence their own independent schema) in the database.
Although Oracle does not provide an embedded database, spinning up a local Docker container running Oracle XE might be an ideal way to accommodate Oracle-specific local integration tests. Since Docker containers are ephemeral in nature by design, the database could also be completely torn down as desired providing clean sandboxing.
The alexeiled/docker-oracle-xe-11g image on DockerHub I found has particularly clear setup and documentation instructions: https://hub.docker.com/r/alexeiled/docker-oracle-xe-11g/
After spinning up the Docker container, be sure to:
First connect to the APEX web console, login, as per the instructions
Then open Oracle SQL Developer and select Reset Password... first. Otherwise the following error may be thrown java.lang.ArithmeticException when attempting to get connection in Oracle 11.2.0.2.0 (64 bit)
As the documentation describes, the docker run command can also be designed to automatically run SQL scripts on the container's startup, which could also be very valuable in the CI/integration testing workflow.
Hope this helps!
I've recently installed the latest version of SQL Server (2008 R2 x64) and I'm now trying get my first steps to create an ETL.
Is it possible to create an ETL with SQL Server 2008 R2 Express, and if not is there some tool that provides that for free?
What I plan to do is collect data from multiple sources like a web services, rss feeds, a parsed url request, a file or maybe other database engine. Transform this collected data for an understandable schema (for my purposes) and store this in my SQL Server database.
If you are looking for a rather simple, yet very flexible and more code-based approach as an alternative to the SSIS (Integration Services), I'd recommend you have a good look at the Rhino ETL code library by Ayende Rahien.
Check out these resources:
Rhino ETL 2.0
Object-oriented ETL using Rhino ETL
Write ETL jobs in pure C#
Rhino ETL is a code-based, programmer-oriented approach - it gives you nice, usable base classes, from which you can inherit and do just about anything - with very little effort.
Integration Services, which is the product of SQLSERVER for this kind of tasks is not supported by the Express Edition.
Although, you can take the long way of making the integrations using plain TSQL and features of Express Edition like linked server (for other databases, access and excel), sqlcmd (for file system) and XML support(for web services).
I have tried SQLite in Java, but the speed is slow due to the JDBC driver. Then I tried HSQLDB and thought the speed is good, but I cannot find a good management tool for HSQLDB such as phpMyAdmin for MySQL or SQLite Manager for SQLite.
I'd like to use the manager tool to prepare the test data for unit tests, or use the manager tool to navigate the data after doing some small experiments.
Is there any good tool?
Here are a couple other suggestions you might checkout:
Squirrel SQL http://squirrel-sql.sourceforge.net/
Execute Query http://executequery.org/
Razor SQL (paid) http://www.razorsql.com/
Razor has the best feature set, but is paid. The others are good at different things and worth checking into.
This would only have meaning if you are running in HSQLDB server mode. If you are running in memory or file mode, then you either can't access the DB from another process or doing so would lock it.
In Server mode you could use any universal client. JDBC driver is the hsqldb.jar itself.
Actually HSQL brings its own management tool (which is not super). See http://hsqldb.org/doc/guide/apf.html
I've used Squirrel SQL. It's a universal client for any JDBC database.
See: http://squirrel-sql.sourceforge.net/