I am using the Attunity Oracle connector in SSIS to connect to a remote Oracle Server.
In my SSIS package, I need to connect to an Oracle database to fetch the data based on datetime parameters.
I followed the suggestions here to write a SELECT query with a parameter:
Created a package variable
Set the variable to evaluate as expression true
Put the query in expression along with parameter as a different package variable
Set the expression for [Oracle Source].[SqlCommand] at Data Flow to package variable (containing the query as expression)
I am good up to here, but if you are setting an expression for [Oracle Source].[SqlCommand] at the Data Flow, then what Query do I set in the "Oracle Source" inside the Data Flow task? How do I get the output columns and perform transformations?
I am not able to execute the package until I set a valid Oracle data source.
Every recommendation says to set the [Oracle Source].[SqlCommand] property at the Data Flow, but nobody mentions how to configure the Oracle source.
Am I missing something here?
Update (2014/02/18) -
Based on comments by #billinkc, I created the data source with non-parameter query and added the expression at the data flow. When I execute the package, the query inside the data source changed to whatever is there in my package variable expression but it throws an error:
OCI error encountered. ORA-00936: missing expression
Here is my WHERE clause of the query, with the variable timestamp -
Where SL.RECEIVED_DATE = TO_DATE( #[User::Last_Run_Timestamp] , 'dd/mon/yyyy HH24:MI:SS')
To parameterize with Attunity Oracle data source, you need to get your metadata set first. This is usually done by just using the unparameterized query as the source. Then, in the Control Flow, on the Data Flow's Expressions you will sub in the SSIS Variable as a source.
It is important that your SSIS Variable be set with Evaluate as Expression set to true and then your formula must be created correctly. Unlike a PowerShell, the tokens are not replaced within in a string. Instead, you'll need to use classic string concatenation techniques. The following demonstrates casting the Variable #[User::Last_Run_Timestamp] to a string which allows me to concatenate, via +, with the rest of my filter.
"SELECT * FROM Table SL Where SL.RECEIVED_DATE = TO_DATE( "
+ (DT_WSTR, 24)#[User::Last_Run_Timestamp]
+ " , 'dd/mon/yyyy HH24:MI:SS')"
I just had to deal with this one. This is not very intuitive, but follow along...
On the Control Flow designer, right click on the Data Flow itself and open Properties.
Find 'Expressions' and click the ellipse to open the Expression Editor.
Under property, select [Oracle Source].[SqlCommand] and then you can build an expression.
More details: http://sornanara.blogspot.com/2013/11/ssis-use-dynamic-sql-in-oracle-source.html
Related
We have a requirement to fetch only change data from Source(Oracle) using SSIS Data Tool. We don't want to fetch entire data in Mapping and then do Insert/Update.
We are looking for something like e.g. Using variable and something below
Select * from where Last_update_Date >= Variable
and Variable should get its data as
Variable = Select MAX(Last_update_Date) from
You can use the free Change Data Capture Service for Oracle by Attunity. It will use a log reader to capture changes from Oracle and stage them in a SQL Server database for consumption by the SSIS CDC pipeline.
I am trying to create a BI Publisher data model which runs the Oracle query below -
SELECT *
FROM audit_YYYYMM
(this should be the YYYYMM of the current date)
How do I setup a parameter default value within the datamodel to grab the YYYYMM from the SYSDATE?
How do I append this parameter within the data set SQL Query?
I tried SELECT * FROM audit_:Month_YYYYMM
(where I had a string parameter called Month_YYYMM)
This did not work.
You are going to have to use something like EXECUTE_IMMEDIATE. And you may have to make a separate PL/SQL package to launch rather than use the built in data definition stuff.
https://docs.oracle.com/cd/B19306_01/appdev.102/b14261/executeimmediate_statement.htm
I have deployed an SSIS package which has a query that pulls data from oracle. The query has some variables in it and I wanted to see what variable were being inputted at run time. Is there a way maybe via sql profiler that I can capture the query that sql server is sending out to Oracle when the sql job runs?
How are these variables being 'evaluated' when the SSIS package executed? I imagine you get this from a table or generating on the 'fly'.
You have the following option :
Capture them and assign them in SSIS variable(s)
In case you are doing something complex - create a variable something like - User::SQLCommand. In this case Use expression to generate the SQL String that needs to be executed in Oracle. Use this in your OLEDB Source Editor as "Sql command from variable" to execute the statement in Oracle
Use a SQL statement to insert the variable details captured in previous step in your intended audit table
--
Please mark if this answers your question!
I am storing dates though SSIS to an oracle table.
I am using a Execute SQL Task,
when i view the dates in SSIS it looks like, example '01-APR-2008'
but when it is stored in oracle database, it stores it as '08-APR-01', as you can see the dates are inverted and i have no way controlling the behavior here.
Anyone has an idea, i can post the whole configuration if required,
The Execute SQL task has a query like the one below:
"INSERT INTO Test.TimeTable (TIME_DIM_ID)
select '"+(DT_STR, 50,1252) #[User::ReadingDate] +"'
from dual"
where #User:: ReadingDate is a variable of Data Type DateTime,
and its value is (01-APR-08 9:38:27 PM) this is just the default value, the variable is populated by another task, and the dates in the variable are then picked up by the EXECUTE SQL Task, but they are inverted at the point of storage.
Can you show the query that is used in the sql task?
If you know the format of the date that is coming into the database, you should always typecast the date appropriately. ( I am assuming the input is coming in as a string).
so instead of '01-APR-2008' , use to_date('01-APR-2008', 'DD-MON-YYYY') (don't rely on the session's NLS Parameters)
When trying to enter a SQL query with parameters using the Oracle OLE DB provider I get the following error:
Parameters cannot be extracted from the SQL command. The provider might not help to parse parameter information from the command. In that case, use the "SQL command from variable" access mode, in which the entire SQL command is stored in a variable.
ADDITIONAL INFORMATION:
Provider cannot derive parameter information and SetParameterInfo has not been called. (Microsoft OLE DB Provider for Oracle)
I have tried following the suggestion here but don't quite understand what is required:Parameterized queries against Oracle
Any ideas?
To expand on the link given in the question:
Create a package variable
Double click on the package variable name. (This allows you to access the properties of the variable)
Set the property 'EvaluateAsExpression' to true
Enter the query in the expression builder.
Set the OLE DB source query to SQL Command from Variable
The expression builder can dynamically create expressions using variable to create 'parametised queries'.
So the following 'normal' query:
select * from book where book.BOOK_ID = ?
Can be written in the expression builder as:
"select * from book where book.BOOK_ID = " + #[User::BookID]
You can then do null handling and data conversion using the expression builder.
If You use Data Flow Task and use OLE DB Source, and you need parameterize your Query :
Create Variable to save "Full" of Query statement : Right Click on blank area outside the package - and Click Variables :
Click Add Variables on Variables Window :
Make the name is SQL_DTFLOW_FULL or something that can you understand easily. The variable data type is STRING
Create Variable(s) to save your parameter(s).
i.e, the full of Query stamements is :
SELECT * FROM BOOK WHERE BOOK_ID = #BookID --#BookID is SQL Parameter
at the sample above, I have just one parameter : #BookID, so I need to create one variable to save my parameter. Add more variables depends on your Queries.
Give it name SQL_DTFLOW_BOOKID
The variable data type is STRING
So, you need make your SSIS neat, and the variables is sorted in understandable parts.
Try to make the variable name is SQL_{TASK NAME}_{VariableName}
Make Expression for SQL_DTFLOW_FULL variable, click on number 1, and start fill number 2. Make Your SQL Statements to be a correct SQL Statement using string block. String block usually using "Double Quote" at the beginning and the end. Concat the variables with the string block.
Click evaluate Expression, to showing result, to make sure your query is correct, copy-paste the Query result at SSMS.
Make sure by yourself that the variables is free from SQL Injection using your own logic. (Use your developer instinct)
Open the Data Flow Task, open the OLE DB Source Editor by double click the item.
Select the Data Access Mode : SQL Command From Variable
Select the Variable Name : SQL_DTFLOW_FULL
Click Preview to make sure it works.
That is all, my way to prevent this SSIS failure case. Since I use this way, I never got that problem, you know, SSIS something is weird.
To change the variable value, set it before Data Flow Task, the SQL Result of SQL_DTFLOW_FULL variable will changed every you change your variable value.
In my case the issue was that i had comments within the sql in the normal form of /* */ and i also had column aliases as "Column name" instead of [Column Name].
Once i removed them it works.
Also try to have your parameter ? statement within the WHERE clause and not within the JOINS, that was part of the issue too.