I've recently installed the latest version of SQL Server (2008 R2 x64) and I'm now trying get my first steps to create an ETL.
Is it possible to create an ETL with SQL Server 2008 R2 Express, and if not is there some tool that provides that for free?
What I plan to do is collect data from multiple sources like a web services, rss feeds, a parsed url request, a file or maybe other database engine. Transform this collected data for an understandable schema (for my purposes) and store this in my SQL Server database.
If you are looking for a rather simple, yet very flexible and more code-based approach as an alternative to the SSIS (Integration Services), I'd recommend you have a good look at the Rhino ETL code library by Ayende Rahien.
Check out these resources:
Rhino ETL 2.0
Object-oriented ETL using Rhino ETL
Write ETL jobs in pure C#
Rhino ETL is a code-based, programmer-oriented approach - it gives you nice, usable base classes, from which you can inherit and do just about anything - with very little effort.
Integration Services, which is the product of SQLSERVER for this kind of tasks is not supported by the Express Edition.
Although, you can take the long way of making the integrations using plain TSQL and features of Express Edition like linked server (for other databases, access and excel), sqlcmd (for file system) and XML support(for web services).
Related
I have an ETL project running on-premises using ODI.
This ETL is quite customized with python scripts to perform tasks such as download/uploading files to Sharepoint, reading and parsing Excel files etc...
If I would want to migrate this to OCI, what would be the best service(s) for it? Is it a simple migration or a redo using a new approach?
I see that we have Data Integration workspaces but it seems not to be able to handle this type of customization...
Not sure about Data Flows. I am definitely not familiar with all Oracle stack, so I need some help :)
Thank you
I would like to export the data from oracle to salesforce . Is there a way to do this?
can't find a way to connect directly.
Thank you.
Salesforce exposes only REST & SOAP API access, there's no ODBC/JDBC/... direct connector to underlying database.
In SQL Server world there's a package that pretends SF is a remote server or whatever is the term (https://www.cdata.com/dbamp/), bunch of stored procedures etc that hide the API access. For Oracle world... look into https://sesamesoftware.com/salesforce/ ? (I'm not affiliated with any of these)
ETL and ESB tools (Informatica, Azure Data Factory, Talend, MuleSoft) have SF connectors too and there's paid extra OData-compatible API if that's something you'd prefer.
If you want to roll something out yourself - Salesforce Data Loader and SFDX (tools your SF Admin / Developer probably already use) can import/export CSV files, can be scripted from commandline.
Or write a program in Java / .Net / PHP / Python. Either craft the REST / SOAP messages (there are WSDLs to speed you up) or there are ready libraries & examples.
Whatever you're comfortable with really.
You can use 3rd party ODBC and JDBC connectors, such as the ones we offer at DataDirect. https://www.progress.com/connectors/salesforce
We also offer a data gateway which can provide access to Salesforce as a relational source, while also providing a way to OData-enable existing sources (think MSSQL, Oracle, Rest, etc) for use with Salesforce Connect - their external data feature to access data where it lives.
https://www.progress.com/connectors/cloud-data-integration
Full disclosure - I am an SE with DataDirect.
We are using a Java Endeca API to perform ETL on various datasources on our production servers. It appears that Oracle Endeca Integrator Studio (Clover ETL) has the ability to perform many of these tasks. But it would not be possible to run Endeca Integrator Studio on production servers which lack GUI features. Is there a way to have Integrator export scripts that could be run in command mode on a Linux server?
Take a look at the Oracle Endeca Integrator Server product. It is a long-running, headless service with a web management interface. You can then execute Integrator projects and ETL graphs on the remote server, or schedule them to run periodically.
Endeca 3.0 has Integrator web Administration console as well but I am not sure to what extent it would be useful to you.
Inside the readme file of the ASP.NET Universal providers NuGet package is this quote
The SqlMembershipProvider, SqlRoleProvider, SqlProfileProvider classes that shipped in ASP.NET through version 4 support only Microsoft SQL Server and Microsoft SQL Server Express. They do not support newer offerings such as Microsoft SQL Azure and Microsoft SQL Server Compact.
However, when I run the custom aspnet_regsql scripts for Sql Azure http://support.microsoft.com/kb/2006191 and then point my web.config to the SQL Azure database, ASP.NET membership seems to work correctly.
Can anyone tell me what exactly doesn't work with SQL Azure, that the universal providers enable?
The main difference, as far as I can tell, is that when you connect to Sql Azure you need to implement retry logic. The original providers won't retry when a connection error occurs and this will happen from time to time with SQL Azure.
Also, the original providers don't raise exceptions containing the full SQL error codes so if your database is being throttled you won't know which throttling rule is being applied. At least this is what the SQL Azure support team tell me - I'm currently in the process of upgrading for this reason.
A note of caution: the Universal Providers seem to use a different database schema to the original providers, so you will need to migrate membership data. If you are starting a new project it will be much easier to change these providers before you go live with real users!
the universal providers enable you to use asp.net membership system on sqlazure and also make the implementation sql agnostic. you can take the same app using universal providers and change the datasource to be sql server/sqlexpress/sqlce/localdb etc
I am starting to learn Reporting Services and I read that it uses SQL Server to store the meta data. I am planning to use it in a WinForm app. Does SSRS require SQL Server to be used in a WinForm app? Can it be self contained like Crystal?
You can use LocalReport in order to make reports but you should give it directly the datasource. More over the ReportViewer Redistribuable Package should be installed !