Visual Studio 2013 SSIS OLE DB Destination (Access via Connection Manager) SQL Command Issue - visual-studio-2013

Afternoon Folks,
We are using Visual Studio 2013 and have an SSIS package that we are creating. We have a simple Data Flow Task that essentially takes some data from SQL and Pushes through to an Access Database. It has three DFD flow items:
I have a OLE DB Source (obtains the SQL via a select statement) --> Data Conversion (Convert SQL Data Types to Access) --> OLE DB Destination (Access Database)
The steps selecting the SQL and converting it works fine.
The issue we have is the SQL command that we are using to update the Access 2010 database.
We have tried to run and create a simple UPDATE statement to update a couple of fields with hardcoded data, but this doesn't update. We have also tried creating a stored procedure and then executing this within the SQL Command line in the OLE Destination Editor.
We can see from posts on the net that we can create a procedure in access 2010 and use this. We are also using Native OLE DB\Microsoft Office 12.0 access Database Engine OLE DB Provider. This connection tests successfully.
We can write a SELECT statement within the SQL Command line and this does pull back data. We just seem to have a problem with the UPDATE and or Create Procedure. In turn we are unable to populate the Mappings. the mappings display the destination box but no fields are displayed within here.
We have had a good look around on the internet but we are struggling to find a solution.
Here is a sample of the code in the form of the update statement we are trying to get working.
UPDATE ReferenceFields
INNER JOIN Addresses
ON ReferenceFields.ID = Addresses.ID
SET ReferenceFields.Reference2 =CustomerName,
ReferenceFields.Reference3 = telephone
WHERE Addresses.UPRN = 12345678910
If I Parse the query it is successful but when I select Mappings a warning is displayed....
Error at Data Flow Task [OLE DB Destination [136]]: No column
information was returned by the SQL command
https://msdn.microsoft.com/EN-US/library/office/ff845861.aspx
https://msdn.microsoft.com/en-us/library/ms141044.aspx
SSIS OLEDB destination with SQL command (Insert if not exists)

After some time I have managed to answer this question myself.
I have added unique parameters to each field I need to update and used the # char to set this.
UPDATE ReferenceFields
INNER JOIN Addresses
ON ReferenceFields.ID = Addresses.ID
SET ReferenceFields.Reference2 = #CustomerName,
ReferenceFields.Reference3 = #telephone
WHERE UPRN = #UPRN
This runs through the code and updates the MS Access database.

Related

SSIS Oracle source conatins no output columns

I am trying to develop an SSIS package that truncates a table in Oracle db. Unfortunately i am getting an error
When i am trying to do select from the truncated table, it works fine - connection manager is setup correctly. I've recreated connection manager just in case but that did help.
Truncated table is in the same schema as the user on which ETL is runned.
Despite an error message, task does it's job. Table get's truncated but error message still appears.
Any ideas what could be the reason?
Regards,
Lukas
2.
Component that i used is Oracle Source task inside of DFT.Oracle source is using a query "TRUNCATE TABLE schema_Name.TableName":
When i use "SELECT * FROM schema_Name.TableName" works fine.
I have the solution.
Instead of using "SQL command" inside of Oracle Source task i used "Execute SQL Task" with the same query. I connected to Oracle db using OLE DB.

SSIS error with Oracle - Exception deserialzing the package "Value does not fall within the expected range"

I am taking an existing SQL Server 2016 SSIS package and trying to add an Execute SQL Task. I'm suspicious that the task itself isn't the problem, but just in case, I'll explain it.
It is a simple task to query the count in a table, using an existing OLE connection. I create a user variable, set the ResultSet to Single Row, and put in my statement: "SELECT COUNT(fld) FROM MY_TABLE" It shows no errors, but when I try to execute it, I get the error in the title. I can change it to a BEGIN SELECT COUNT(fld) FROM MY_TABLE; END; or add a GO on the 2nd line - it makes no difference. I can set the ResultSet to None, and it makes no difference. I can create a new OLE Connection and use that, and it makes no difference.
I'm pretty sure it has to do with the Oracle driver, because nothing in the paragraph above makes any sense. I can test the connection and it works, however, and when I use the existing connection, it is parameterized for the project.
So what next? Reinstall the Oracle drivers? Or is there anything less drastic?
I started a new project, make a new OLE connection, a new Execute SQL task, and call an Oracle procedure. It works just fine. It's just trying to edit an existing project that doesn't work. Sometimes.
Oh, I'm also using Visual Studio 2019.

Issue in copying HEX data from ORACLE db (source) to AZURE SQL (sink) through ADF

Scenario:
I am trying to copy data from source ORACLE database to sink AZURE SQL using ADF.
I have created Oracle 11gR2 database in my local system (Windows 10) and installed Self Hosted Run time. On adding "data set" in ADF, I can "preview" tables from my local Oracle database.
Now target is Azure SQL and copy activity is like-to-like. So I have created table in AZURE SQL, keeping all column attributes same, barring one RAW column in source.
Problem:
In source table, there is a column of type RAW(2000) and it contains zlib compressed data in HEX format.
For this, as per the mapping spec detailed in https://learn.microsoft.com/en-us/sql/relational-databases/replication/non-sql/data-type-mapping-for-oracle-publishers?view=sql-server-ver15,
I have changed the type for the same field in Azure SQL to varbinary(2000) (also tried with binary(2000)
Source column data in Oracle is as below:
COMPRESS_DATA
--------------------------------------------------------------------------------
076CE1315D719C6A86A13B8E863F4ACA982C3D72CA234B8F2F67C7996896AD39866639FDD699A3B1F8A3A272FB6BA3DAC8C08E3B19BBC3BEAD431BCB050665F5F2946BA3CFB58BBC42431C98FD2B2ABB7DE2DAE84F344EBB6F52EC1FBD677A682BB46EFB54F3A2DBEDFAD0FA6A4AFCD556581F5FEB1D68DA64E4E084F5CF18CCD2C49BDDE31D7DF80E460E3D9C080B9CF2EE6839A6B6F90EECBB6CF24004ACFDB92BC52FB6ECB1DEEA5F5096FDF2628E9F68EBB361BF4F3BE2A38A39CAE5194FA9E7100BF51CBAD30677B8DE2CEFE255216779975602DC7BD1661BF99FAAD6175EAC45CB625A7B5A3C51DCFD1375C94C9B6D5A97AF9F15BD583B574A5F2F8BC1FD0ADA91EC917E9C765E252B7AB92BEC5D1A657984D364453F51475D4331681DBD12F779947F613ACED82E3B2788F2C9CE1D99DD209EC876CEBCF537DC5A85EB84B7ADB75A3DE0E60376D754D9BF0ABD35041E32318E97986ABEEADA54A15AD7B62E51E2288920518AC37C1F27417FBE3F960873B713E6CB037E12B347B4C980EE0ACE454B7381AD6967E7678E4BA7AA364D5F726AA21AE2EAB635657E038992A52A9F4AF92E2504BBB5C9EC5449454EF002D972B3BEB29ACD12878FED78E55601594CBA93FAA6406FF51C4AA4BD7DAFBF408414F5386ECA36BD6AF7BC4E813577DD9A6814D6527A6E2895FB0DC1D8D3658A5BE21E76D3A11536CEB17BAD0FB3261FDE4326B5BC5FC67BB585B2EECF78A4B9069EB8B6AD1BC6E7BCA6FB338E4FF69CE3ABD1E43FBDA1636A7DA3D11A9A3F00F9EB9131ECC78C5A5C66E5D5650FA66DE0AAA34DB80DF3CBC7AE1FF891B7FA94865FC368F9354E90EEA3704E5604AAC681D290448E14121C607DBF4DBD02F7DBD6D49D5C4E29A173897386F1474812C208A6B073FC097F747C8868488ED00E4139C179BC1802ED9D38BF5C463EE49DD14ECF7140132B11938088D2233EC3F6764E7AB9D9924923EFF20C677C3423B08608BBC1F24F1238B5CF19ADD35164496384162328BCC8EB1C27AAA2DA1119257F0C1A52C6772AE013F86062BA72AE6CBAAA33378C3BB240B27980EFC82EEDC9AA382B34A980D6AE183B1F8A5CC6C63C761D64369EA7E70D0F0C0A9647FD601E46B10AE7BB8323155F9DCA050D13FE5598648AAA827834833199DC92E1573EBE55D58124AA8C2251D42A699DD48F0EABB4A2B2A83D1C8C8A0E122105155DDD49B9281FAA4BA1D5005A7132A4ADD420B564496519B27EA46942FF1D7B488D120EE525D8213921FB7F5EB4800F3F969516834643A1592EC320A74767E42C24FC14974C9C6CA78743F686641DD1229D0946DEFC9BF775172D6597321B6B459E5015EF5D8071A0534B1F5DE37AD0B2AC99EC906F76E1E0DE61643A667849A2C6B157CCCCE0E167D91803D9A2207B872B4BA72B9129BB056ECB2B19F161D4F492F9DD9159105AB1796A6144B6DBA638DC91C96ABFFED0D6EBE5D720EEC99EF6DCC1F45500100E03C0335C358B57826AE45294170E8A6048720A06474E9933A6439D1F8648941525512E1E9243C6CCEA366753FD027364C1F70C732CA9F9198E74ABA775750B0FF57871AEF29924940FEFFDE0168C15FC24170F0E0A9B630E6955E2D7F6833B2FFA169B8E209EF12A1B5F859FE186D9FE4FC21B61ADD11EEC7488AA4E5216C545D5D2C2B38C600DAF472EABD9D79C5494AF3D688D7A886FAB579A3A313BECE82267127ADE1F8B9CC0206662B8654D94F02B92DFC9BB275349E23DDD167553C4E93869E381192BEE197AAB3D5C476CF06AA64FFFCD362823FA98F4CFB03FA3AB6BE649ED8FD6EB5BB53FFACD01FF36FB4B128C38397E75C323AF7218B9ED3DF8C9258AA2500E6D5369385E12CB929DC824C87657B13F2BC49BE566E5D7764CE59E887C81CC5273D2A4847A36E2DC99D13EED88A32DF92D8E381EE8D1D114491FD2E88E21AFED1874ED135647EDEA9511589BD090381E42ACF684AC71C0375DB2D9A70A0D4E0272D0206E2833AAD2501411F1BB6FAC983DAB221EDB279F88C9C217AE4289B967A92F2AFA6A5E5B9AC119DAD544D334647A1B3F5A7A2BA48F4DCD9920DB31724EC4B6462729A7E4647AC537734E7AF17B6AF032C090E7DF42FEA38F87AED3EBBA48C152C293CCA0A164B4E8FA752A712998030801AEA669A20A7B7C36EBFB969156B3235EE41BAC00A132744FF65802B2A16F212BE11ED23E469E9866C99709CC1EFFB14CB110B914A918E48ADF96F374451C5A7CDD970855C8B4BBD2F2FE8A34C3AA70D60270EFF2A2461F55C9DEF7D3F66F9681BF4055A56ECF4C788F2C201400874BEEB249B356BD4AF6828E448649FE052C00A0715E3539F1BF2FFDBF7079C75364629CC5DD7AD19ABCF3882FBDF3882C9B44F761B83A59C1A87AC6B067AC8A59A1C182AD300D89E7596833D10D8E5341B5024AB6098F2298278E0C2F10C75148257930AFD2D24086CD6C66DBB941D9F7FE7C79F3902EC29F91379565AF9049D64950DDBBA3EADF1195DBFD53A7A7BA01548F4C75B050B8640A1946AA43A6CD768BC8807F5C6C577E762A2096E6ED035219841601516840E43A402EC7F22407A4B4154C06D06B81118F1A8EC12CDBE09486A83658861504351E35A44AA81A8A3AE48AA386A7470D5707C5C0350B50EC6A6EFA0129A60B17DE0A77C8CEC7DFA4CC03E60000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Now when I am creating the copy step in ADF and in Mapping tab, I opt for "Import Schema", it shows:
for Source, COMPRESS DATA field type is BYTE[ ].
Naturally when I run the pipeline, the above column value comes in Azure SQL, in a different format:
data preview in ADF
Data as in ADF preview and in Azure SQL (post copy)
B2x4nO1dW2/buLYG91sL9D/o5QApkHhE3RUcHGxZF9uJJTuS7MR5GWhspTHGkTOy05n8+7MoybrQkuOkzm6SXTVtY5FcXOvjx8VFSqR9rcPChf7FsazKqjyLMCfKqKN7HvJMx/hdc/Vub9xzOr933MFo+LvX7Q1t0/ERXC78dVD7vEM+YLHFom/T1eokiKe38+/z6Bv18V8+VIahsmvjqqvq3oU5MvsaFB2Q4mgQz8KY6QbRbEHKckgPomm4WIQz5o9HRn9YrZd3YYx4knEeBfEj48dBtLpfxmsmKYvABuWElU84nsG4hZUWL7Z4QVAlEU1XMcsKVA6OheSWjDGP+f1zYGR7/YPUhSbE7vZy+ScYzOjL6GYe3wXr+TJC2fV0LcXVw0hMGw4jfZL+kwBr3gXzRUlWUgizKIFvtpwyvej7cj4loCOi0w7FUdLiJJPJC6U2HHlGST7DAhfY9CdhF6gmw7+mj2zNHEH5y3D+7XYNoiYpKUCb4aVKkaKAxg6i4Ft4F0brXYjsB9jTOUAblUXmPwWzClZmWdzwMVwdM/rtfL1kPCSrWER2EMarP5n+PApJ/uUcobHT1RxgC06t5Fika7pbtdL3r53yHcSWDHce7v4AYiMkQr/hsKTyB7Gw2ngJGzDybuf3BOJUV/AEnDC06RbBMsp7YnptijHDIF4/MtpsFs4OomR60XR80pxMfYx4naPUZ5PuwYw0n9E1r8vcBivmjzCMmIAozcAnbbFYTpMeyAz+jsK4dQhLckAly3CqGnHvCdAz8dx5LqDg1NZxMF0/BIvchR8W1L7QH1a14t8TqLg9Gj4XVOId4Lf242GR1AecS/d3QUDDOB8b4CKfQkYPFtOHRdpRjGAdMvMV83A/g9/eBqpWX3FpVBs0n94G0TdA8yZe3n35/L/tRRD9+X9fPn/5/ET+9fLL50I9UsB4COEus/Hd0zgsipH0w/qS8aSnm8RCK28suG0spw+E5+UoogiTZllqwalEye1GU1uYa6kSK0lSg2o1ORoarSZnfaN5ikiZBDHKbcjcLMEr/00gXT9tyemXz/4QUGCi5WnjoPkS83LkDfbqbAt5Hpl/PWQuBgKFMAOjuEnCghVo+z0sdeY3AbvTk892w/6EFYT061vwAKvMyZ4StueFThmB/c1wJ8cMBEAsy4w8pt/2fmXYO8NB2dsz5O4WewWYa8XflmXmFjdWRR+7W35/I6Ttnem1vmKaq11oHYcL4hkITVPX4CxbTO4bDoyvozrnVcW4Hfj2K4q+HZ/gt8fnu31Co/713iDJfsqsw9X6mIF6BRYfM/7yESZR34I7MpfC7HGWDC2TzBzzDnDgJrpW+0M+MW5yaW66AIQHy9nDdJ1B54bTcA5kZ+LlAxn0l7NwuwaYF2cTXMO0tFHf3ydLY9NsZ61vG7c/otRnkZtrecrAZPsgmuZ4TbrXFmtW40JORcVcaRNj3wcQMMXhKoy/p5HPTTBf1DEa6hIUiVWa4p6aHLtgq+asR83stztVI9hGhQnj6D92sJ7eEuKnLnEZQZwRB7OQ6QcRAd3xTZcxR+5gaGoOcwS//QZ/v3757IFwCCRPmYHXHzCe6Y7BcTFHPcuExHG4WoWL38bLx+AbZNGgvKsxR/hy/PU3Fqz68rk0I4U+FS3XoC8MGDFQ88gyra+twyiXJv5maZBP83xGM2XmSFLYbR1N29YYWzNd75w5kiWJKMqqXz47y/LsOfHAN8uHaHYgBS2W31Il08IdePrAspgjlfWINiJ+Qpv0z9Efi6zP62lIb34Hb/X1EGzNu06H1YaXVNehXU32CQbX2fxm/ka6izMxKMXTAWC4IF1mecMkDvI+jb/TKVEygzpNsX0q63oJGfXlfRjdQktGW0WMcAHeN37cR3xd3kT+OFiE0XQe5CUyoIt8s0JoU4ZE0kE5YVxr3rvkhGXoHs2JrH8yNgw9VVA/Mb8xBNjaHIDqJ30COfR91Nkf27bB6tfvEdt2W6YUz/rbizlrnOvDrm8zRS87Zqx4DnO3KIw+wdDxCUaZbMT59IkMNp+gkNl2x1DIgTn2LeOFAeOH8d08ChZMOw7vwphM/aJP9tj7BK6/PBYQEawKIkxP67sgQlt8C6fzGNQ6Yex+LgdqHnwCR005b1JcxEnxsd7zdcYO4vl6fhcwm14cBZ8+HZQrysWZS3Plx5e7fgp7rrhzl2bPXstd1QWs56x4CaUVL9M3YMxekeYi8yx/fpc55vsgXj/E4df6EP4H2k4zTK3fIQaPrP1mscUsBaKAbPFoq/Ve9kCwvvXqcjasexk+ZczWXPZvUB76H5l+Jy2XT2R3zGNfZEyO8Kgz8s8phKXskWxm8ZsAzxWvKD1hNhTehDE4jZDM9E8ZSRp4eZYvnyEcmd/Pk/WWykL7MaN3e/6Acc2J6R0Xi+zHTPIA9ZTRPRf0+HfquFr6wD4w5B47HnqJng6Xk7p+Sdf2+kzdc2PGgDjoqBNGYRwsvmaT9EE8/5a48Pt4OSURdA3tBY4oparNam/laGy5rZz1LTcYi5S5ha2FAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
Expected outcome:
The content should be copied to Azure SQL as is from source (HEX).
Please help/advice on how this can be achieved as we are planning to move 10TB of Oracle data into Azure SQL but this is the base issue blocking it.
Please try create the table with nvarchar(max) data type in Azure SQL database.
I'm glad to hear that "Changing it to NVARCHAR(MAX) while creating the Azure SQL table solved the problem".
It's my pleasure to help you!

SSIS Data Flow Task doesn't load all rows from OLE DB

I'm trying to load a table from SQL Server using the Microsoft OLE DB Provider into an Oracle table (using the Oracle Provider for OLE DB). The package is a straight forward OLE DB Source (SQL Server) -> OLE DB Destination (Oracle).
I'm using SQL Server 2008 R2 and Oracle 11g.
Every time I run the package, I get a different number of rows in the destination table, and BIDS reports fewer rows read than there are in the source table. The number of rows returned is different each time I run it. I get no errors or kickouts, but the boxes for the source and destination remain yellow even after BIDS says "Package completed successfully".
Dumping the source table into a flat file instead of the Oracle destination works fine, and I get all the rows that I expect. I can use this flat file to pull the information into the Oracle destination table without problems as well.
Even though I have a work-around, I want to understand why this is happening, and what I can do to resolve this problem without having to use flat files.
Edit: Looks like even using the flat file to Oracle doesn't bring over all the rows. The first time was just luck?
Edit/Update: Running the package out of Integration Services (not BIDS) seems to have eliminated the problem (tested three times). Still don't understand why this is happening though.

How do I connect to my local SQL instance in Visual Studio?

I've installed VS2010 Express and all associated SQL. I've got an instance running on my local machines of .\sqlexpress and I can connect to it with SSMS. I've created some tables etc.
I want to use Linq to SQL so I have created a new empty .dbml file in my solution. I assume the next step is to connect to the database, but I don't know how as I've never done this before.
I tried Add Connection as it suggests, and the Data Source I can choose Access Database, SQL Server Compact 3.5 or SQL Server Database File.
The Database File one says, "use it to connect to a local instance", so I click that. Then it asks for a database file name, new or existing. Surely I should be typing in the instance name not doing this file thing? If I do type something in, it crease a .mdf in my documents folder and it doesn't really help me much.
Any ideas how I can get this to work?
it's easier than you think.
you create a new database in SQL Management Studio, you call it
whatever, myDatabase...
from Visual Studio you select the SQL Provider and you select the
local instance and the myDatabase as db name.
everything is basically as you said except that you should have created a database and you should then select it in VS.
doing this from a new EF model, VS will store the connection string compatible with EF format in the app.config so you will be able to check it and see how such connection string is written.
I finally figured this out via a blog post or two somewhere online. With the express edition, the option to connect to a local database in the normal way via IP or ./sqlexpress is not present. You have to connect using a File, by finding the .mdf file on your hard disk (within the SQL server folder somewhere) that represents your database, and selecting it.
I have no idea what happens if you update your database schema in SSMS, or why they've restricted it to this bizarre method, but it works and I have been able to query my database.

Resources