I have an SSIS package set up to pull data from an Oracle database into SQL Server. I recently ran into an issue that was preventing this data pull.
The following code works fine in Oracle SQL Developer (it returns rows, as it should):
SELECT a.MyField ,
a.MyOtherField,
a.FromDate
FROM MyTable a
WHERE a.FromDate BETWEEN CONCAT('01-', TO_CHAR(ADD_MONTHS(SYSDATE, -13), 'MON-YY')) AND TO_CHAR(LAST_DAY(SYSDATE), 'DD-MON-YY')
However, when using this as the SQL command text of an OLE DB Source component in SSIS, it returns no records.
I'm not sure if this is an SSIS issue or a difference in language syntax (I believe this is due to the different systems' date syntax, but I do not have a strong enough grasp of PL/SQL to know how to correct this.).
Any ideas?
Most likely, the problem is that you are comparing a date to a string. This forces Oracle to do an implicit cast using the session's NLS_DATE_FORMAT. Since this is session-specific, it is fragile as different clients will end up doing the conversion differently.
You are almost certainly better off rewriting the query in order to compare a date to other dates, i.e.
SELECT a.MyField ,
a.MyOtherField,
a.FromDate
FROM MyTable a
WHERE a.FromDate BETWEEN trunc( add_months( sysdate, -13 ), 'MM' ) AND
trunc( last_day( sysdate ) )
Related
I'm running a query across a database link to a Sybase server from Oracle.
In it's where clause is a restriction on date, and I want it tied to sysdate, so something like this:
select * from some_remote_view where some_numeric_key = 1 and
some_date > sysdate+2
The problem is, when I do explain plan, only the condition some_numeric_key = 1 shows up in the actual sql that is getting remoted to the sybase server. Oracle is expecting to perform the date filter on its side.
This is causing a performance nightmare - I need that date filter remoted across to have this query working quickly
Even if I try something like casting the sysdate to a charcater string like this:
to_char(sysdate-2,'YYYY-MM-DD')
It still does not remote it.
Is there anything I can do to get Oracle to remote this date filter across the db link to Sybase?
Doing integration between Oracle and other platforms I often run into this problem, not just with SYSDATE but with other non-standard functions as well.
There are two methods to work around the issue, the first being the most reliable in my experience.
First, you can create a view on the remote db with the filters you need, then on the Oracle side you just select from the new view without additional filters.
Second, if you are not allowed to create objects on the remote side, try using bind variables (of the correct data type!) in your Oracle SELECT statement, e.g.:
declare
v_some_date constant date := sysdate + 2;
begin
insert into oracle_table (...)
select ...
from remote_table#db_link t
where t.some_numeric_key = 1
and t.some_date > v_some_date;
commit;
end;
/
I have created a procedure in oracle as follows:
create or replace PROCEDURE SP_X_AVERAGE
(
profile out SYS_REFCURSOR,
rx out SYS_REFCURSOR,
)
as
BEGIN
open profile for
select
avg(to_number(profile_netassets)) AS netassets
from
fgp;
open rx for
select
avg(to_number(a_price)) as twr
from
r_x
where
gq_date <= add_months(to_date(sysdate, 'mm/dd/yyyy'), -12);
END SP_X_AVERAGE;
It doesn't run, giving the following error:
ORA-01843: not a valid month
If I remove the where condition in the second sql then it runs successfully.
Altering a session using an sql in the same procedure doesnot work too.
Please help.
I am running this procedure in sql-developer (ubuntu Oneiric 11)
SYSDATE is already a DATE so you don't need to apply TO_DATE() to it. However, more recent versions of Oracle are tolerant of such things and handle them gracefully.
So that leaves the matter of r_x.gq_date: what data type is that? If it is a string then the chances are you have values in there which will not cast to a date, or at last don't match your default NLS_FORMAT.
"we have to keep it as "VARCHAR2(40 BYTE)" it is having date in it like this : '1/2/2003'"
Bingo. Is that the same as your NLS_DATE_FORMAT? If not you will need to cast the column:
to_date(gq_date, 'mm/dd/yyyy') <= add_months(sysdate, -12);
This may not solve your problem if the column contains strings which aren't in that format. This is a common side-effect of using strings to hold things which aren't strings.
In PLSQL I run:
truncate table MyOracleTableName;
commit work;
insert into MyOracleTablename
select a,b,c,trunc(sysdate) as datadate
from AnotherOracleTableName
where there is a ton of nasty criteria
union
select a,b,c,trunc(sysdate) as datadate from AnotherOracleTableName
where there is a ton of different nasty criteria;
commit work;
In PLSQL Developer this inserts one row.
When I run the SQL (without the semi colons and the commit work statements) in SSIS, I get a primary key violation from MyOracleTableName.
I have validated that the truncate from SSIS is committed in Oracle.
When I run the SQL above in PLSQL Developer and replace the union with union all, I see a second row and the insert fails for a PK violation. As it should with a union all allowing the duplicate.
This is currently part of an SSIS 2005 package using MSDAORA where it works just fine. I am now re-writing in SSIS 2008 using Native OLE DB providor for Oracle.
I cannot use MSDAORA in my new environment. Is this a driver issue and is there a work around other than breaking these into multiple statements where the second inserts only what is not already in MyOracleTableName?
Regards.
I figured out the problem after dinner.
The Primary key constraint is a composite key on columns A and B. The Union de-dups on columns a,b,c and the date. In Oracle the trunc(sysdate) returns mm/dd/yyyy. In SSIS the trunc(sysdate) is being parsed out to the second or milisecond. This results in two unique rows (to SQL Server and Microsoft) due to the timestamp, and then attempts to insert duplicate rows where columns a,b, and c are duplicated.
The solution is this:
truncate table MyOracleTableName;
commit work;
insert into MyOracleTablename
select a.*,
trunc(sysdate) as datadate
from(
select a,b,c
from AnotherOracleTableName
where there is a ton of nasty criteria
union
select a,b,c from AnotherOracleTableName
where there is a ton of different nasty criteria) a
commit work;
This allows the union to kill the duplicate and runs the trunc(sysdate) once thereby presenting the single row to my primary key constraint.
Thank you.
I'm migrating some data from one oracle schema/table to a new schema/table on the same database.
The migration script does the following:
create table newtable as select
...
cast(ACTIVITYDATE as date) as ACTIVITY_DATE,
...
FROM oldtable where ACTIVITYDATE > sysdate - 1000;
If I look at the original data, it looks fine - here's one record:
select
activitydate,
to_char(activitydate, 'MON DD,YYYY'),
to_char(activitydate, 'DD-MON-YYYY HH24:MI:SS'),
dump(activitydate),
length(activitydate)
from orginaltable where oldpk = 1067514
Result:
18-NOV-10 NOV 18,2010 18-NOV-2010 12:59:15 Typ=12 Len=7: 120,110,11,18,13,60,16
The migrated data, showing that the data is corrupt:
select
activity_date,
to_char(activity_date, 'MON DD,YYYY'),
to_char(activity_date, 'DD-MON-YYYY HH24:MI:SS'),
dump(activity_date),
length(activity_date)
from newtable
where id = 1067514
Result:
18-NOV-10 000 00,0000 00-000-0000 00:00:00 Typ=12 Len=7: 120,110,11,18,13,0,16
Around 5000 out of 350k records show this problem.
Can anyone explain how this happened?
UPDATE:
I don't find any published reference to this specific type of DATE corruption on the Oracle support site. (It may be there, my quick searches just didn't turn it up.)
Baddate Script To Check Database For Corrupt dates [ID 95402.1]
Bug 2790435 - Serial INSERT with parallel SELECT and type conversion can insert corrupt data [ID 2790435.8]
The output from the DUMP() function is showing the date value is indeed invalid:
Typ=12 Len=7: 120,110,11,18,13,0,16
We expect that the minutes byte should be a value between one and sixty, not zero.
The 7 bytes of a DATE value represent, in order, century(+100), year(+100), month, day, hour(+1), minutes(+1), seconds(+1).
The only time I have seen invalid DATE values like this when a DATE value was being supplied as a bind variable, from a Pro*C program (where the bind value is supplied in the internal 7 byte representation, entirely bypassing the normal validation routines that catch invalid dates e.g. Feb 30)
There is no reason to expect the behavior you're seeing, given the Oracle syntax you posted.
This is either a spurious anomaly (memory corruption?) or if this is repeatable, then it's a flaw (bug) in the Oracle code. If it's a flaw in the Oracle code, the most likely suspects would be "newish" features in an un-patched release.
(I know CAST is a standard SQL function that's been around for ages in other databases. I guess I'm old school, and have never introduced it into my Oracle-syntax repertoire. I don't know what version of Oracle it was that introduced the CAST, but I would have stayed away from it in the first release it appeared in.)
The big 'red flag' (that another commenter noted) is that CAST( datecol AS DATE).
You would expect the optimizer to treat that as equivalent to date_col ... but past experience shows us that TO_NUMBER( number_col ) is actually interpreted by the optimizer as TO_NUMBER( TO_CHAR ( number_col ) ).
I suspect something similar might be going on with that unneeded CAST.
Based on that one record you showed, I suspect the issue is with values with a "59" value for minutes or seconds, and possibly a "23" value for hours, would be the ones that show the error.
I would try checking for places where the minutes, hour or seconds are stored as 0:
SELECT id, DUMP(activitydate)
FROM newtable
WHERE DUMP(activitydate) LIKE '%,0,%'
OR DUMP(activitydate) LIKE '%,0'
I've seen similar things to spence7593, again with Pro*C.
It is possible to create invalid dates programmatically using a DBMS_STATS package.
Not sure if there is a similar mechanism to reverse that.
create or replace function stats_raw_to_date (p_in raw) return date is
v_date date;
v_char varchar2(25);
begin
dbms_stats.CONVERT_RAW_VALUE(p_in, v_date);
return v_date;
exception
when others then return null;
end;
/
select stats_raw_to_date(utl_raw.cast_to_raw(
chr(120)||chr(110)||chr(11)||chr(18)||chr(13)||chr(0)||chr(16)))
from dual;
I'm reading a pipe delimited file with SQL Loader and want to populate a LAST_UPDATED field in the table I am populating. My Control File looks like this:
LOAD DATA
INFILE SampleFile.dat
REPLACE
INTO TABLE contact
FIELDS TERMINATED BY '|'
OPTIONALLY ENCLOSED BY '"'
(
ID,
FIRST_NAME,
LAST_NAME,
EMAIL,
DEPARTMENT_ID,
LAST_UPDATED SYSTIMESTAMP
)
For the LAST_UPDATED field I've tried SYSTIMESTAMP and CURRENT_TIMESTAMP and neither work. SYSDATE however works fine but doesn't give me the time of day.
I am brand new to SQL Loader so I really know very little about what it is or isn't capable of. Thanks.
Have you tried the following:
CURRENT_TIMESTAMP [ (precision) ]
select current_timestamp(3) from dual;
CURRENT_TIMESTAMP(3)
-----------------------------
10-JUL-04 19.11.12.686 +01:00
To do this in SQLLDR, you will need to use EXPRESSION in the CTL file so that SQLLDR knows to treat the call as SQL.
Replace:
LAST_UPDATED SYSTIMESTAMP
with:
LAST_UPDATED EXPRESSION "current_timestamp(3)"
I accepted RC's answer because ultimately he answered what I was asking but my unfamiliarity with some of Oracle's tools led me to make this more difficult than it needed to be.
I was trying to get SQL*Loader to record a timestamp instead of just a date. When I used SYSDATE, and then did a select on the table it was only listing the the date (05-AUG-09).
Then, I tried RC's method (in the comments) and it worked. However, still, when I did a select on the table I got the same date format. Then it occurred to me it could just be truncating the remainder for display purposes. So then I did a:
select TO_CHAR(LAST_UPDATED,'MMDDYYYY:HH24:MI:SS') from contact;
And it then displayed everything. Then I went back to the control file and changed it back to SYSDATE and ran the same query and sure enough, the HH:MI:SS was there and accurate.
This is all being done in SqlDeveloper. I don't know why it defaults to this behavior. Also what threw me off are the following two statements in sqldeveloper.
SELECT CURRENT_TIMESTAMP FROM DUAL; //returns a full date and time
SELECT SYSDATE FROM DUAL; // returns only a date
If you want to use the table defined default you can use:
ROWDATE EXPRESSION "DEFAULT"
In Sql Developer run:
ALTER SESSION SET NLS_DATE_FORMAT='YYYY-MM-DD HH24:MI:SS'
and then check it with
SELECT SYSDATE FROM DUAL