I am using Talend Open Studio for MDM and I have a requirement to do version control on customer records.
When using an Oracle database, I can use tOracleSCD to capture the changes. Likewise, for MySQL, I can use tMysqlSCD.
But in Talend Open Studio for MDM, the only supported database is H2 and so I am storing all master records in a H2 database.
In this case, how can I achieve version control as there is no component available in Talend
for H2 database?
The SCD components just set up triggers on the watched tables and provide an easy interface into reading the trigger output tables.
You could set the triggers up manually on the H2 database by recreating the database in MySQL and then using the MySQL SCD components to work out what it's doing and work out how to read the data back in and then recreate those steps with H2 components as part of a data integration task.
That said, Talend MDM has the concept of a journal which stores all of the changes made to a data record. The Talend Open Studio for MDM documentation has some more detailed information about how to view the journal. All changes made through the MDM interface should make an entry in the journal automatically.
Related
How can I create a script of inserts for my sybase to oracle Migration? The Migration wizard only gives me the option to migrate procedures and triggers and such. But there is no select for just tables. When I try to migrate tables offline and move data. the datamove/ folder is empty. I would also want to only migrate specific tables (ones with long identifiers) because i was able to migrate the rest with Copy to Oracle.
I must also note that i do not want to upgrade to an new version of oracle. Currently on ~12.1 so i need to limit the identifiers.
How can I get the offline scripts for table inserts?
You (probably!) don't want INSERTs for offline migration scripts. If you're just running INSERTs, then the online method would probably suffice.
The point of the Offline strategy is to take the data from your Sybase instance to flat, delimited text files (using BCP), which we can THEN use to load back into an Oracle Database using SQLLDR or External Tables which will be EXPONENTIALLY faster than using INSERT scripts.
Take a look at this whitepaper where I go into offline Sybase migrations in detail.
You can consider DCO-based Sybase-to-Oracle replication via the Sybase Rep Server. This way, not only will you have all data moved, but you will also be able to have DML updates propagated online, which will make your system switchable live.
I need to use an embedded DB (H2, Apache Derby) with talend open studio. I saw that it's possible with talend MDM, but couldn't find any tutorial on how to embed this in talend open studio.
I have a big amount of data, from different tables that are processed the first time, stored locally before a second step of transformations. But can't use cache memory or files (csv) as middle storage.
Any ideas ? help please
Talend MDM uses an internal Database for the purpose of the product (data repository). Talend Open Studio (DI) does not use it (there are no internal informations needed with Talend Open Studio for Data Integration).
If you want to connect and extract data from your DB2 database, you can use DB2 components inside your talend jobs.
I wish to use MongoDB as source and target, and perform ETL using Informatica PowerCenter 9.1.
I currently use Oracle 11g as the primary database and wish to migrate to MongoDB to see if MongoDB could be a better database. I also have MongoDB as the database for a few UI tools and wish to migrate the data to Oracle for reporting needs.
I have fair knowledge of Informatica. I would like to use MongoDB as a source and target.
Is this possible and if someone could guide me with some information.
It should be possible as long as you can create the sources in Mapping Designer => Source Analyzer, Sources top-bar menu. I'm not sure if you can access it by ODBC defined on your client machine, or if you need a dedicated connector that should be listed in the menu I've mentioned.
Once you import the source (and target respectively, in Target Designer), you should be able to create a copy of your mapping and replace the source and target with the new ones. Then you just need to create a workflow and set proper connections using server-defined ones.
This is my first time with MVC. When I am trying to add an ADO.net entity model then while creating a new connection I couldn't see my oracle database in the list of data source name. Only different type of Microsoft server is listed. What should I do to get the oracle database listed there too. When using simple database connection in the desktop application I could see the oracle database listed but not in this case.
The main problem is I couldn't connect entity data model to oracle database
I think you need to install the Oracle Developer Tools, which is a surprisingly complicated process.
This is a rather old article, but I think this is still useful
http://blogs.msdn.com/b/kaevans/archive/2009/07/18/connecting-to-oracle-from-visual-studio.aspx
When I am publishing my site using Web Deploy in Visual Studio I see something like:
MySql.Data.MySqlClient is not supported for incremental database publishing.
I want to know what exactly would this technology do if it is supported?
Incremental database publishing is a deployment feature where the source code for the schema (new version) is compared with the live database schema (current version) to create a change script to upgrade the current version to the new version.
For example, if your schema has a table with a few columns and in the source code you add a new column definition, when publishing it would create a script to add the new column to the table.
It's not surprising that a database engine besides SQL Server isn't supported, since supporting other database engines would be a significant amount of work. All it really means is that you'll have to synchronize database changes yourself. MySQL may provide tools to help with this, but Visual Studio doesn't.
Incremental database publishing is used for versioning purposes. SVN can't really handle versioning for database schema like tfs can support.
If you're using incremental database publishing you can set up to keep the current database and just do the update, or to always recreate database.
Let's assume you want to declare sql variables like InitData, SeedData.
You can do this and you have access at some before and post publishing script. If your variable SeedData is true, you can seed some data in the post publishing script.
More info here: http://www.asp.net/web-forms/tutorials/deployment/web-deployment-in-the-enterprise/deploying-database-projects
EDIT: it's just the schema that's updated if you choose not to recreate your database when you publish