we were using Oracle Workflow Builder for a long time for the plsql packages execution, scheduling the jobs etc. The product is already discontinued so I am looking for some reasonable alternatives. Alternatives as Informatica Workflow scheduler, etc.
Thanks
I'd look at Oracle Data Integrator and/or Oracle BPEL Process Manager. I think these products are the successors to to Oracle Workflow in the spaces you are looking for.
Related
I am planning to use SQL developer scheduled jobs for our daily extract.
Assuming that I already have a scheduled job (SQL query), where or how can I see the extracted data?
please following the link for tutorial create job sql developer.
scott
https://www.foxinfotech.in/2018/10/how-to-schedule-a-job-oracle-sql-developer.html
I am situation where I have to check and confirm whether SSAS partitions queries are running parallel or not while processing the SSAS cube using SSIS job. SSIS job/package using 'Analysis Services Processing Task' to process cube by selecting each object(dimensions and partitions) in it instead of selecting direct SSAS DB.
Can any one please guide how to check parallelism using sql profiler?
Also if anyone can point out why cube processing using above way is taking longer than the cube processing by SSIS job in which 'Analysis Services Processing Task' selecting ssas db name directly.
please help with any comments/ suggestions.
Many Thanks
Regards,
Update: My end db from which partitions will fetch the data is Oracle
I think there is an easier way than using SQL Profiler, you can benefit from the amazing stored procedure sp_whoisactive to check what are the current query running on the server (Data Source SQL Database Engine) while processing the Analysis Services Processing Task.
Just download the stored procedure and create it on your master database.
sp_whoisactive homepage
How to Log Activity Using sp_whoisactive
Hint: In SQL Server Management Studio, go to data source properties and check the maximum allowed connections property, since it may prevent queries parallel execution
If you are looking for an answer using SQL Profiler, you can simply create a trace to monitor the SQL Server that contains the data sources used by partitions. And while the partitions are processed if many SQL queries are executed in parallel then parallelism is achieved.
If you are new to SQL Profiler you can refer to the following links to learn how to monitor only T-SQL commands:
How to monitor just t-sql commands in SQL Profiler?
Use SQL Server Profiler to trace database calls from third party applications
But if you are looking for a simpler solution, then the other answer is what you are looking for.
We want to take tables from Oracle to Cassandra every day. Because tables is updated in Oracle everyday. So when i searched this , i find these options:
Extract oracle tables as a file , then write Cassandra
Using sqoop to get tables from oracle, write Map Reduce job and insert into Cassandra ?
I am not sure which way is the appropriate ? Also is there another options ?
Thank you.
Option 1
Extracting oracle tables as a file and then writing to Cassandra manually everyday can be tiresome process unless if you are scheduling a cron job. I have tried this before, but if the process fails then logging it might be an issue. If you are using this process and exporting to CSV and trying to write to cassandra then I would suggest using cassandra bulk loader (https://github.com/brianmhess/cassandra-loader)
Option 2
I haven't worked with this, so can't speak about this.
Option 3 (I use this)
I use an open source tool, Pentaho Data Integration (Spoon) (https://community.hitachivantara.com/docs/DOC-1009855-data-integration-kettle) to solve this problem. It's fairly a simple process
spoon. You can automate this process by using a carte server (spoon server) which has logging capabilities as well as automatic restarting if the process failed in between.
Let me know if you found any other solution that worked for you.
I am new to ETL migration. I have worked with Talend, but not yet faced the task to migrate large ETL project from one tool to another(IBM Data Manager to Informatica PowerCenter or Informatica Developer).
I am looking to general guidlines for migrate jobs from one tool to another one, and of course for my specific case.
I will be more clear:
The Databases Sources and Targes will be the same, what I have to migrate is the ETL part itself.
The approach will be the parallel run as suggested at this blog :
Parallel Run
In my case I have not to migrate the all DWH instead only the ETL as the old software will become a legacy one and the new one is from another Vendor(luckly both of them can export XML ).
I am looking for the pratical approch for parallel run, indeed I am been suggested to copy the Sources and Targes Tables in the orginal Database schema, but it does not look to me the best way to go(even not pratical when a schema has many tables).
The DWH I am woking of course has several DBS instance in Oracle and some in SQL Server, a Test server and a Production one, as well as for each, a Staging, Storage and a Data Mart area.
As from this related question and its answer, I am thinking to copy each schema on the go for each project.
Staging in ETL: Best Practices
Looking to have guidlines references, but my specific case is the migration from IBM Data Manager to Informatica PowerCenter
The approach depends on various criteria and personal preferences. Either way you will need to either duplicate parts or all of the source and destination systems. On one extreme you can use two instances of the entire system. If you have complex upstream processes that are part of the test, or you have massive numbers of tables and processes, and you have the bandwidth and resources to duplicate your system then this approach may be optimal.
At the other extreme, if any complex processes occur within the ETL tool itself, or you are simply loading tables and need to check they are loaded correctly, then making copies of the tables and pointing your new or old tool to the table copies may be the way to go. This method is very simple and easy to setup.
Keep in mind this forum is not meant to replace blogs and in-depth tech articles on those techniques.
Now I'm too keen to know how can I export dmp file with Oracle jobs? It's because of I'm very new to Oracle and don't know how to backup Oracle with jobs like MsSQL bachup with schedule. That's why I want to know what I asked.
You can fairly easy setup a backup scheduled by the database. Best approach for this is to install the Oracle Scheduler Remote Job Agent - local to the database - and configure that agent in the database that holds your backup schedule. This can be the database itself, it can also be a central backup schedule database, all a matter of taste.
Oracle Scheduler is very powerful and can execute tasks in the local database, in remote database[s], on the local server and on remote servers. If using OS type of jobs, best is to use the 11g Remote Scheduler Agent. Don't use the old fashioned 10g style External Jobs. Use remote jobs with defined credentials.
For help look at my blog where you also find pointers to docu.
After you installed and configured the job agent to be a valid target for the database that performs the scheduling, easiest is to use dbconsole to define the jobs. If you configure the dbconsole, it also gives an option to generate auto backup jobs. Maybe this is already enough. You asked for export and there expdp with the Oracle Scheduler does a wonderful job.
You can run an OS process from Oracle Job using Java Stored Procedure or a C program.
See this blog entry,
Instead of exporting dump using the old imp/exp utilities to generate dmp files, look at Oracle Datapump, especially since from the tags I infer you're using Oracle 11g.
Data pump supports table-level, tablespace-level, schema-level & full export modes and is kown to be consdirably faster than the previous imp/emp tools.
Further reading:
Export/import with Oracle Data pump
Oracle Documentation on Data pump
know how can I export dmp file with Oracle jobs?
That's not possible. The emp tools runs outside the database, while jobs run within. if you want scheduled exports perhaps you could use a cronjob/scheduled task.