We have an SSIS package downloading data from an Oracle database to an SQL Server datawarehouse. For this datawarehouse, several environments are set up; Development, Test and Production. Dev and test share a machine, Prod is stand-alone.
When the SSIS package is run on the PROD machine, it downloads the Varchar2 columns from our Oracle source database to MSSQL in DT_WSTR format and saves this to a NVarchar column. I.E. all steps involved support Unicode.
When this same package is run against the same source database on the DEV/Test box, it somehow sees the external columns as being Varchar, derives this to DT_STR in the data flow and refuses to store this in an NVarchar column.
All OS's are Win2K8r2, MSSQL 2008 64 bits. The package is run in 32bits mode, same behaviour is seen when run from BIDS or from SQL Agent.
Anyone care to guess why? I've already seen the suggestion to disable validating external metadata (https://stackoverflow.com/a/18383598/2903056), but that's not a practical suggestion for our situation.
An old question I know, but seems to still be relevant. And since I could not find a suitable answer in the last 3 months I have been searching, I figure now is as good a time as any to post my findings.
I have had the same curious behaviour and have finally been able to resolve it.
My layout looked like this:
Oracle 10g R2 database on Windows 2003 Server (lets call it ORA)
Dev machine with Windows 8, Visual Studio 2012 + SSDT, Sql Express 2012,
ODT 12.1.0.21 (lets call that DEV)
Sql Server 2012 on Windows 2012 Server, Oracle Client 11.2 (lets call that TEST)
Both DEV and TEST were connecting to ORA. DEV was reporting VARCHAR2 columns as DT_WSTR while TEST would insist that they are DT_STR.
I then installed ODT 12.1.0.21 on TEST and the problem was solved. Notably, I used the "machine wide" option during the install. I am not sure how much of an impact that had.
There seems to be a difference in the datatypes that are returned by the Oracle OleDb providers across the different versions of the client side components.
Check the value of the NLS_LANG in the registry.
reg query HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ORACLE\KEY_<orahome> /f NLS_LANG
If it matches the server's character set, OraOLEDB will use regular (non-Unicode) datatype DBTYPE_STR, otherwise it uses Unicode-mode, datatype DBTYPE_WSTR.
If the NLS_LANG field is missing, it defaults to US7ASCII which almost certainly will not match your database and you will be using Unicode datatypes.
To get the server's characterset, do:
SELECT parameter, value FROM nls_database_parameters WHERE parameter = 'NLS_CHARACTERSET';
Check Metadata validation property value if it true make it false
Related
A coworker made an SSIS package that pulls data from Oracle and transfers it to a SQL Server database that is nearly identical, so it's a lot of data flow tasks simply with an OLE DB Source (Oracle) to an OLE DB Destination (SS). When I open it on my computer, I get the error "Column [column name] cannot convert between unicode and non-unicode string data types" on all the source tasks. If I add a data conversion task to convert the unicode columns to non-unicode, all works fine but I really want this to work like how he has it because it's running on the production server like this. My best guess is it has to do with the installing of the Oracle client or drivers or the NLS_LANG variable but I can't seem to solve it. My environment variable NLS_LANG = AMERICAN_AMERICA.WE8ISO8859P1
I was worried something went wrong with my Oracle client installation because of my registry values. Now I have 3 clients installed since I went through the install process again. These are the third client's reg values and I added NLS_LANG myself and rebooted. I'm more of a SQL Server developer, so I'm possibly saying something wrong here.
The solution was to set the NLS_LANG environment variable value to AMERICAN_AMERICA.WE8MSWIN1252 to match what my coworker had and what my registry has because I somehow didn't notice they were different! However, neither NLS_LANG were set to start with so the real solution was to add this in. I rebooted and when I reopened the package, got zero errors.
enter image description hereWe are migrating Oracle DB from 12c Windows Oracle to 18c Linux Oracle installation. Some of the columns (derived columns) in views has become unicode and not usable by downstream SSIS. Any reason why these views are giving error in SSIS? SSIS is complaining that in 18c columns have become unicode
12c View definition is
enter image description here
18 c View defintion
enter image description here
I think your problem matches this :
Oraoledb: Cannot Convert Between Unicode And Non-Unicode String Data Types (Doc ID 960508.1)
This is due to a difference in metadata reported by the OleDb provider depending upon whether NLS_LANG environment variable is set. Typically this behaviour would be observed when the SSIS package is developed in an environment that has NLS_LANG set, and then deployed to an environment that does not have it set. The difference in metadata results in the error, and is being investigated via Oracle bug number bug 7836009.
To resolve this issue, set NLS_LANG to the same setting as the box the package was developed on, which result in the same metadata being reported in both cases.The NLS_LANG environment variable should be set in the registry under
HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\KEY_<homename>\NLS_LANG
Export the NLS_LANG variable in Linux with the same value as the original database
The behavior can also be worked around by refreshing the SSIS package after deployment, which will refresh the metadata.
Hope it helps.
Regards.
My package runs fine from both my desktop and my ETL server when I RDP into it. However, when running as part a job, I get the following error on all my string columns: "Error: Column "***STATUS" cannot convert between unicode and non-Unicode string data types."
The error occurs on an OLE DB Command component that updates a table in an Oracle database. None of my columns on either the SQL/SSIS side nor the Oracle side are Unicode. Here's the metadata directly leading into my OLD DB Command component.
I verified that the External Columns on the OLE DB Command component in question exactly match that metadata. I've also tried explicitly converting the columns to Unicode before inserting in case they were Unicode (I know they're not) on the Oracle side, but that leads to a hard error (red X) and the same message.
Here's the Oracle schema:
Command:
Anyone have any idea on how to get this to run from the agent?
Based on the following oracle support case:
Oraoledb: Cannot Convert Between Unicode And Non-Unicode String Data Types
The following error message cause is:
Developing an SSIS package that uses the Oracle OLEDB Provider on a 32 bit operating system and then deploying to a 64 bit SQL Server installation
Possible Workarounds
Note that i didn't tested these workarounds before.
(1) Try running the package in 32-bit mode:
From Visual Studio
GoTo Project properties >> Debugging >> Run64BitRuntime = False
From SQL Agent
Check the following link:
SSIS Package Not Running as 32bit in SQL Server 2012
(2) Install Oracle x64 oledb provider
64-bit Oracle Data Access Components (ODAC) Downloads
On the OLE DB Source ensure you have delay validation set to true.
I set the default code page to false and had the code page as 1252.
I have tried this both with 32 bit and 64 bit.
Also need to make the flag validateexternalmetadata = false
We have a SQL Server production box that has a 32-bit Oracle 11g driver installed. Our current test SQL Server which has the same driver installed has been replaced by a new server that has the 64-bit 12c driver. When we test a SSIS 2012 package that pulls from Oracle and loads SQL, we will run on the new Test SQL Server using the 64-bit DTEXEC.exe utility to run the package.
Q1: Is it by virtue of using the 64 bit DTEXEC utility that the package will automatically seek out the 64 bit version of the driver?
I assume that in the connection string below, Provider=MSDAORA.1 refers to the MS OLE Provider for Oracle, so, I assume that I am not using the Oracle driver. I am not using a DSN.
Q2: Is it looking up the value of the "Data Source" property of the connectionstring, MyOracleResource.MyCompany.COM, and finding the corresponding entry in my tnsname.ora file in one of these folders based on the DTEXEC version that I use to run the package?
C:\Oracle\product\11203_32bit\CLIENT_1\NETWORK\ADMIN
C:\Oracle\product\11203_64bit\CLIENT_1\NETWORK\ADMIN
If so, I am surprised that the MS driver would be dependent on the tnsnames file that is created as a result of installing the Oracle client.
In the Project properties, under Property Pages->Config Properties->Debugging, there is a property, "Run64BitRunTime."
Q3: Am I correct to presume that when running within the IDE that this is the equivalent of selecting either a 32 or 64 bit versiobn of the DTEXE utility to run the package from the cmd line?
When I look at the properties on my Oracle connection object, I see an "ID" property with a value of {0cbfe196-1a88-4b62-8522-b34dbb37ba71}.
Q4: Is this GUID used to identify a particular driver version of the driver and therefore something that could cause of problem when testing in an environment that does not match Production?
<Configuration ConfiguredType="Property" Path="\Package.Connections[MyOracleDb].Properties[ConnectionString]" ValueType="String">
<ConfiguredValue>Data Source=MyOracleResource.MyCompany.COM;User ID=MyUser;Password=MyPassword;Provider=MSDAORA.1;Persist Security Info=True;</ConfiguredValue>
</Configuration>
Q1 - Yes - when you execute a package under the 64 bit version of DTEXEC, it will user the 64 bit drivers. You can test this locally by only installing one or the other and try to run the package. Running under the wrong version will cause the package to fail because it cannot find the drivers.
Q2 - how the server gets resolved depends on how your organization has configured your environment. For example, where I work, there is an environment variable pushed onto each machine which points to network directories where the TNSNAMES files are stored. You must consult your Oracle guru to understand how this works in your environment.
Q3 - The Run64BitRunTime property will cause the SSIS designer to use different versions of DtsDebugHost.exe. Working in a similar way to DTEXEC, it will either run the 32 or 64 bit version depending on how that is set. This is also why that property has no affect once the package is running in production - because you are using a different program to run the package entirely.
Q4 - the id property is assigned and used internally by SSIS. If you click around, you will see that every component has an id. So this has nothing to do with any external elements.
I have installed Oracle Database 11.2.0.1 Win64 and also Client 11.2.0.1 Win64 in my windows 7 Ultimate and I was able to access Oracle database using Sql*plus
using sqlplus / as sysdba. After that I installed ODP.NET ODTwithODAC1120320_32bit to work with Oracle using C# in Visual Studio 2010. When i tried to add connection to Oracle database using ODP it pops out ora-12560 :TNS: protocol adapter error and even during logging to Oracle using SqlPlus / as sysdba.
I have read several articles that say stopped service will cause but mine is running.
What could be the problem???
The main problem that I faced Connecting to Oracle 11g using Server explorer in VS 2010 was not version compatibility of ODP with Oracle database but it's architecture. After such errors I uninstalled Oracle Database 11g using deinstall.bat file then cleaned Registry Key ../Software/Oracle and cleaned up temp files, did a system restart. Then installed Oracle 11g Database x32 Database server, ODP.NET x32, checked Env vars.
First success was that I was able to connect to DB using SQL*Plus. After that started VS 2010 and tried to connect ..... Connected! wow has worked in VS IDE. I was so happy :)
I remember how many times I reinstalled Oracle(cleaning,configuring checking listeners, restarting services). Then my hard work really paid off.
here is a screenshot
Generally, I advise to install Oracle server under different credentials than what you usually use. The Oracle server (under Windows) makes use of a few environment variables; and the same is the case with ODP.NET as well. And while you can't easily switch between different sets of env variables in your user profile, you can assign different env var values to a (different) user under which the Oracle server is installed and/or being run.
If you already have your Oracle server installed (which, I suspect, is your case), then try:
Create a new user in your Windows, make him sufficiently privileged. ("Admin" rights will do fine. :-))
Log in as the new user.
Set up ORACLE_HOME, ORACLE_SID, NLS_LANGUAGE and PATH environment variables for that particular user (not globally for the whole Windows!) to point to your Oracle server.
Log in as the original user.
Start the Services management console ("services.msc").
Change "Log On" credentials for the OracleServiceSOMETHING and Oracle SOMETHING VSS Writer Service and OracleJobSchedulerSOMETHING and OracleOraDb11g_home1TNSListener to that new user+password you just created. (... where SOMETHING is usually the name of your instance)
Stop all Oracle services.
Set up ORACLE_HOME and PATH env var to point to your ODP.NET root and root\bin folders respectively; set up the ORACLE_SID and NLS_LANGUAGE vars to whatever values you need.
Start all Oracle services. (After this moment they should be running under their own credentials.)
Let's pray that it works.
I myself would have to improvise, too, if this scenario didn't work. But so far I have successfully run two Oracle servers and one Oracle client on the same machine this way with no problems, so I hope it works for you too. If it does not, then there's still the option of reinstalling your Oracle server completely.
Don't forget about backing up your database ... just in case something horrible happens.