We have a requirement to fetch only change data from Source(Oracle) using SSIS Data Tool. We don't want to fetch entire data in Mapping and then do Insert/Update.
We are looking for something like e.g. Using variable and something below
Select * from where Last_update_Date >= Variable
and Variable should get its data as
Variable = Select MAX(Last_update_Date) from
You can use the free Change Data Capture Service for Oracle by Attunity. It will use a log reader to capture changes from Oracle and stage them in a SQL Server database for consumption by the SSIS CDC pipeline.
Related
I am trying to create a BI Publisher data model which runs the Oracle query below -
SELECT *
FROM audit_YYYYMM
(this should be the YYYYMM of the current date)
How do I setup a parameter default value within the datamodel to grab the YYYYMM from the SYSDATE?
How do I append this parameter within the data set SQL Query?
I tried SELECT * FROM audit_:Month_YYYYMM
(where I had a string parameter called Month_YYYMM)
This did not work.
You are going to have to use something like EXECUTE_IMMEDIATE. And you may have to make a separate PL/SQL package to launch rather than use the built in data definition stuff.
https://docs.oracle.com/cd/B19306_01/appdev.102/b14261/executeimmediate_statement.htm
I have created a temporary table in oracle sql developer but I forgot to save it and now I want to reuse the query but I don't remember the code used then. Is there a process to get query used creation of temp table?
You can use dbms_metadata.get_ddl()
select dbms_metadata.get_ddl('TABLE', 'YOUR_TABLE_NAME_HERE')
from dual;
The result is a CLOB with the complete DDL. You might need to adjust the display in SQL Developer to make the content of that value fully visible (I don't use SQL Developer, so I don't know if that is necessary and if so, what you would need to do)
Edit:
It seems SQL Developer can't display the result of this query properly unless you use the "Run Script" option. And with that you need to use a SET LONG 60000 (or some other big number) before you run it, to see the complete source code:
I have deployed an SSIS package which has a query that pulls data from oracle. The query has some variables in it and I wanted to see what variable were being inputted at run time. Is there a way maybe via sql profiler that I can capture the query that sql server is sending out to Oracle when the sql job runs?
How are these variables being 'evaluated' when the SSIS package executed? I imagine you get this from a table or generating on the 'fly'.
You have the following option :
Capture them and assign them in SSIS variable(s)
In case you are doing something complex - create a variable something like - User::SQLCommand. In this case Use expression to generate the SQL String that needs to be executed in Oracle. Use this in your OLEDB Source Editor as "Sql command from variable" to execute the statement in Oracle
Use a SQL statement to insert the variable details captured in previous step in your intended audit table
--
Please mark if this answers your question!
I am using the Attunity Oracle connector in SSIS to connect to a remote Oracle Server.
In my SSIS package, I need to connect to an Oracle database to fetch the data based on datetime parameters.
I followed the suggestions here to write a SELECT query with a parameter:
Created a package variable
Set the variable to evaluate as expression true
Put the query in expression along with parameter as a different package variable
Set the expression for [Oracle Source].[SqlCommand] at Data Flow to package variable (containing the query as expression)
I am good up to here, but if you are setting an expression for [Oracle Source].[SqlCommand] at the Data Flow, then what Query do I set in the "Oracle Source" inside the Data Flow task? How do I get the output columns and perform transformations?
I am not able to execute the package until I set a valid Oracle data source.
Every recommendation says to set the [Oracle Source].[SqlCommand] property at the Data Flow, but nobody mentions how to configure the Oracle source.
Am I missing something here?
Update (2014/02/18) -
Based on comments by #billinkc, I created the data source with non-parameter query and added the expression at the data flow. When I execute the package, the query inside the data source changed to whatever is there in my package variable expression but it throws an error:
OCI error encountered. ORA-00936: missing expression
Here is my WHERE clause of the query, with the variable timestamp -
Where SL.RECEIVED_DATE = TO_DATE( #[User::Last_Run_Timestamp] , 'dd/mon/yyyy HH24:MI:SS')
To parameterize with Attunity Oracle data source, you need to get your metadata set first. This is usually done by just using the unparameterized query as the source. Then, in the Control Flow, on the Data Flow's Expressions you will sub in the SSIS Variable as a source.
It is important that your SSIS Variable be set with Evaluate as Expression set to true and then your formula must be created correctly. Unlike a PowerShell, the tokens are not replaced within in a string. Instead, you'll need to use classic string concatenation techniques. The following demonstrates casting the Variable #[User::Last_Run_Timestamp] to a string which allows me to concatenate, via +, with the rest of my filter.
"SELECT * FROM Table SL Where SL.RECEIVED_DATE = TO_DATE( "
+ (DT_WSTR, 24)#[User::Last_Run_Timestamp]
+ " , 'dd/mon/yyyy HH24:MI:SS')"
I just had to deal with this one. This is not very intuitive, but follow along...
On the Control Flow designer, right click on the Data Flow itself and open Properties.
Find 'Expressions' and click the ellipse to open the Expression Editor.
Under property, select [Oracle Source].[SqlCommand] and then you can build an expression.
More details: http://sornanara.blogspot.com/2013/11/ssis-use-dynamic-sql-in-oracle-source.html
I am storing dates though SSIS to an oracle table.
I am using a Execute SQL Task,
when i view the dates in SSIS it looks like, example '01-APR-2008'
but when it is stored in oracle database, it stores it as '08-APR-01', as you can see the dates are inverted and i have no way controlling the behavior here.
Anyone has an idea, i can post the whole configuration if required,
The Execute SQL task has a query like the one below:
"INSERT INTO Test.TimeTable (TIME_DIM_ID)
select '"+(DT_STR, 50,1252) #[User::ReadingDate] +"'
from dual"
where #User:: ReadingDate is a variable of Data Type DateTime,
and its value is (01-APR-08 9:38:27 PM) this is just the default value, the variable is populated by another task, and the dates in the variable are then picked up by the EXECUTE SQL Task, but they are inverted at the point of storage.
Can you show the query that is used in the sql task?
If you know the format of the date that is coming into the database, you should always typecast the date appropriately. ( I am assuming the input is coming in as a string).
so instead of '01-APR-2008' , use to_date('01-APR-2008', 'DD-MON-YYYY') (don't rely on the session's NLS Parameters)