I am trying to query the past week's additions to an Oracle database overnight and need to use macros to populate the dates. I am able to run the query below if I hard-code the actual dates. I've tried double and single quotes on the macro vars &sd and &ed. Please advise.
data _null_;
sd = dhms(today()-7,00,00,00);
ed = dhms(today()-1,23,59,59);
call symput("sd", put(sd, datetime20.));
call symput("ed", put(ed, datetime20.));
run;
%put &sd &ed;
proc sql;
connect to oracle (user=x password=x path=x);
create table weekly_test as
select * from connection to oracle
(select * from x.Estimates
where state_fips_code = '41'
and altered_date between
to_date('&sd','DDMONYYYY:HH24:MI:SS')
and to_date('&ed','DDMONYYYY:HH24:MI:SS'));
disconnect from oracle;
quit;
error
ORACLE execute error: ORA-01858: a non-numeric character was found where a numeric was expected.
and with double quotes
and altered_date between
to_date("&sd",'DDMONYYYY:HH24:MI:SS')
and to_date("&ed",'DDMONYYYY:HH24:MI:SS'));
this error
ERROR: ORACLE prepare error: ORA-00904: " 21MAR2012:23:59:59": invalid identifier. SQL
statement: select * from X.Estimates where state_fips_code = '41' and altered_date
between to_date(" 15MAR2012:00:00:00",'DDMONYYYY:HH24:MI:SS') and to_date("
21MAR2012:23:59:59",'DDMONYYYY:HH24:MI:SS').
The best bet is to define your macro variable with single quotes around the values. In fact, I don't think it's necessary to format it as a datetime literal; just construct a normal ANSI date string (YYYY-MM-DD) and you can also get rid of the TO_DATE function call.
For example, try these two statements:
%let SD=%str(%')%sysfunc( putn( %sysfunc(intnx(day,%sysfunc(today()) ,-7)),yymmdd10.))%str(%');
%let ED=%str(%')%sysfunc( putn( %sysfunc(intnx(day,%sysfunc(today()) ,-1)),yymmdd10.))%str(%');
Those define SD as today()-7 and ED as today()-1 (using pure macro code rather than a data step). Then, in your query, reference these macro variables unquoted:
proc sql;
connect to oracle (user=x password=x path=x);
create table weekly_test as
select * from connection to oracle
(select * from x.Estimates
where state_fips_code = '41'
and altered_date between &sd and &ed
);
disconnect from oracle;
quit;
Many thanks Bob. I tried the code you posted and got ORA-01861: literal does not match format string. Anyway you got me thinking on the right path. I just added code to put single quotes around my dates in the data step and it worked. For anyone with similar problems code is below.
data _null_;
sd = dhms(today()-7,00,00,00);
ed = dhms(today()-1,23,59,59);
call symput('sd',"'"|| trim(left(put(sd, datetime20.)))||"'");
call symput('ed', "'"||trim(left(put(ed, datetime20.)))||"'");
run;
%put &sd &ed;
proc sql;
connect to oracle (user=x password=x path=x);
create table weekly_test as
select * from connection to oracle
(select * from x.Estimates
where state_fips_code = '41'
and altered_date between
to_date(&sd,'DDMONYYYY:HH24:MI:SS')
and to_date(&ed,'DDMONYYYY:HH24:MI:SS'));
disconnect from oracle;
quit;
This works....
%LET SD = %SYSFUNC(intnx(day,"&SYSDATE9"d,-7,b),date9.) ;
%LET ED = %SYSFUNC(intnx(day,"&SYSDATE9"d,-1,b),date9.) ;
%PUT &SD &ED ;
proc sql ;
connect to oracle (user=x password=x path=x);
create table weekly_test as
select * from connection to oracle
(select * from x.Estimates
where state_fips_code = '41'
and altered_date between %BQUOTE('&SD') and %BQUOTE('&ED')
);
disconnect from oracle ;
quit ;
Related
I have a program that has been working for years. Today, we upgraded from SAS 9.4M3 to 9.4M7.
proc setinit
Current version: 9.04.01M7P080520
Since then, I am not able to get the same results as before the upgrade.
Please note that I am querying on Oracle databases directly.
Trying to replicate the issue with a minimal, reproducible SAS table example, I found that the issue disappear when querying on a SAS table instead of on Oracle databases.
Let's say I have the following dataset:
data have;
infile datalines delimiter="|";
input name :$8. id $1. value :$8. t1 :$10.;
datalines;
Joe|A|TLO
Joe|B|IKSK
Joe|C|Yes
;
Using the temporary table:
proc sql;
create table want as
select name,
min(case when id = "A" then value else "" end) as A length 8
from have
group by name;
quit;
Results:
name A
Joe TLO
However, when running the very same query on the oracle database directly I get a missing value instead:
proc sql;
create table want as
select name,
min(case when id = "A" then value else "" end) as A length 8
from have_oracle
group by name;
quit;
name A
Joe
As per documentation, the min() function is behaving properly when used on the SAS table
The MIN function returns a missing value (.) only if all arguments are missing.
I believe this happens when Oracle don't understand the function that SAS is passing it - the min functions in SAS and Oracle are very different and the equivalent in SAS would be LEAST().
So my guess is that the upgrade messed up how is translates the SAS min function to Oracle, but it remains a guess. Does anyone ran into this type of behavior?
EDIT: #Richard's comment
options sastrace=',,,d' sastraceloc=saslog nostsuffix;
proc sql;
create table want as
select t1.name,
min(case when id = 'A' then value else "" end) as A length 8
from oracle_db.names t1 inner join oracle_db.ids t2 on (t1.tid = t2.tid)
group by t1.name;
ORACLE_26: Prepared: on connection 0
SELECT * FROM NAMES
ORACLE_27: Prepared: on connection 1
SELECT UI.INDEX_NAME, UIC.COLUMN_NAME FROM USER_INDEXES UI,USER_IND_COLUMNS UIC WHERE UI.TABLE_NAME='NAMES' AND
UIC.TABLE_NAME='NAMES' AND UI.INDEX_NAME=UIC.INDEX_NAME
ORACLE_28: Executed: on connection 1
SELECT statement ORACLE_27
ORACLE_29: Prepared: on connection 0
SELECT * FROM IDS
ORACLE_30: Prepared: on connection 1
SELECT UI.INDEX_NAME, UIC.COLUMN_NAME FROM USER_INDEXES UI,USER_IND_COLUMNS UIC WHERE UI.TABLE_NAME='IDS' AND
UIC.TABLE_NAME='IDS' AND UI.INDEX_NAME=UIC.INDEX_NAME
ORACLE_31: Executed: on connection 1
SELECT statement ORACLE_30
ORACLE_32: Prepared: on connection 0
select t1."NAME", MIN(case when t2."ID" = 'A' then t1."VALUE" else ' ' end) as A from
NAMES t1 inner join IDS t2 on t1."TID" = t2."TID" group by t1."NAME"
ORACLE_33: Executed: on connection 0
SELECT statement ORACLE_32
ACCESS ENGINE: SQL statement was passed to the DBMS for fetching data.
NOTE: Table WORK.SELECTED_ATTR created, with 1 row and 2 columns.
! quit;
NOTE: PROCEDURE SQL used (Total process time):
real time 0.34 seconds
cpu time 0.09 seconds
Use the SASTRACE= system option to log SQL statements sent to the DBMS.
options SASTRACE=',,,d';
will provide the most detailed logging.
From the prepared statement you can see why you are getting a blank from the Oracle query.
select
t1."NAME"
, MIN ( case
when t2."ID" = 'A' then t1."VALUE"
else ' '
end
) as A
from
NAMES t1 inner join IDS t2 on t1."TID" = t2."TID"
group by
t1."NAME"
The SQL MIN () aggregate function will exclude null values from consideration.
In SAS SQL, a blank value is also interpreted as null.
In SAS your SQL query returns the min non-null value TLO
In Oracle transformed query, the SAS blank '' is transformed to ' ' a single blank character, which is not-null, and thus ' ' < 'TLO' and you get the blank result.
The actual MIN you want to force in Oracle is min(case when id = "A" then value else null end) which #Tom has shown is possible by omitting the else clause.
The only way to see the actual difference is to run the query with trace in the prior SAS version, or if lucky, see the explanation in the (ignored by many) "What's New" documents.
Why are you using ' ' or '' as the ELSE value? Perhaps Oracle is treating a string with blanks in it differently than a null string.
Why not use null in the ELSE clause?
or just leave off the ELSE clause and let it default to null?
libname mylib oracle .... ;
proc sql;
create table want as
select name
, min(case when id = "A" then value else null end) as A length 8
from mylib.have_oracle
group by name
;
quit;
Also try running the Oracle code yourself, instead of using implicit pass thru.
proc sql;
connect to oracle ..... ;
create table want as
select * from connection to oracle
(
select name,
min(case when id = "A" then value else null end) as A length 8
from have_oracle
group by name
)
;
quit;
When I try to reproduce this in Oracle I get the result you are looking for so I suspect it has something to do with SAS (which I'm not familiar with).
with t as (
select 'Joe' name, 'A' id, 'TLO' value from dual union all
select 'Joe' name, 'B' id, 'IKSK' value from dual union all
select 'Joe' name, 'C' id, 'Yes' value from dual
)
select name
, min(case when id = 'A' then value else '' end) as a
from t
group by name;
NAME A
---- ----
Joe TLO
Unrelated, if you are only interested in id = 'A' then a better query would be:
select name
, min(value) as a
from t
where id = 'A'
group by name;
I am trying to pass PL/SQL query from file via VBA to the Oracle DB.
Simple test query like SELECT * FROM DUAL
works flawlessly.
However just a bit more complicated (even not yet PL/SQL) like
SELECT x, y, z FROM table WHERE DATE = '5may2017'
already fails (actually no explicit error, just empty result).
I'm connecting to Conn.Provider = "OraOLEDB.Oracle.1", apply user schema by ALTER SESSION SET CURRENT_SCHEMA=MyUserSchema. Finally my last code-snippet is:
With Cmd
.Properties("PLSQLRSet") = True '(actually this line doesnt affect)'
.ActiveConnection = Conn
.CommandType = adCmdText
.CommandText = SqlStatement
Set rs = .Execute
End With
Sheets(1).Range("A1").CopyFromRecordset rs
So, excel sheet is populated with the 1st test simpliest query, however remains empty for the more complex part (I have no semicolon in the end :) )
However final sql-script will be 400+ lines long...
What am I missing at this step?
You might try instead ANSI Date 'YYYY-MM-DD' or ANSI Time 'HH:MM:SS' SQL formatting below for an implicit conversion with SQL ANSI DATE or SQL ANSI TIME operator.
SELECT x, y, z FROM table WHERE COLUMN_DATE = DATE '2017-05-05'
SELECT x, y, z FROM table WHERE COLUMN_TIME = TIME '23:59:59'
- according such "COL_" exists
A must-keep cheatsheet for quick reference PSOUG Datatypes
NB: For Toad users, check your NLS SELECT * FROM NLS_SESSION_PARAMETERS;
; your NLS_DATE_FORMAT is not default one DD-MON-RR.
I'm using Eclipse with Spring, Oracle SQL and JdbcTemplate. I'm trying to delete certain data based on how old the data are, the SQL command works in Oracle Sql Developer and does what I want.
DELETE from TABLE where DATE_COLUMN < SYSDATE - INTERVAL '30' DAY;
However, in my DAOImpl.java,
int thirty = 30;
String sql = "DELETE from TABLE where DATE_COLUMN < SYSDATE - INTERVAL ? DAY";
jdbcTemplate.update(sql, thirty);
I'm getting ORA-00933: SQL command not properly ended whenever I do ? and
Caused by: java.sql.SQLException: Invalid column index when I do '?' '.
I'm not sure it's ? or "?"
How do I use jdbcTemplate.update() correctly? I've tried adding the Integer.class argument and params objects but the sql command still doesn't work.
Originally I'm just trying to pass in as an int then I tried passing in a String number 30 but it still doesn't work.
I'm really not sure what I'm doing wrong.
I'm connecting to the db and everything works fine, I've tried some easier sql command like
String sql = "select count(*) from TABLE";
return jdbcTemplate.queryForObject(sql, Integer.class);
and it works and returns.
EDIT:
I can do '30' in place of my ? in my String sql = "DELETE from TABLE where DATE_COLUMN < SYSDATE - INTERVAL ? DAY"; and it runs correctly, however I do not want to hard code it.
I have a small problem with loading data from an Oracle database into QlikView 11 using the following script:
SET ThousandSep='.';
SET DecimalSep=',';
SET MoneyThousandSep='.';
SET MoneyDecimalSep=',';
SET MoneyFormat='#.##0,00 €;-#.##0,00 €';
SET TimeFormat='hh:mm:ss';
SET DateFormat='DD.MM.YYYY';
SET TimestampFormat='DD.MM.YYYY hh:mm:ss[.fff]';
SET MonthNames='Jan;Feb;Mrz;Apr;Mai;Jun;Jul;Aug;Sep;Okt;Nov;Dez';
SET DayNames='Mo;Di;Mi;Do;Fr;Sa;So';
ODBC CONNECT TO [Oracle X;DBQ=db1.dc.man.lan] (XUserId is X, XPassword is Y);
SQL SELECT *
FROM UC140017."TABLE_1";
SQL SELECT *
FROM UC140017."TABLE_2";
SQL SELECT *
FROM UC140017."TABLE_3";
SQL SELECT *
FROM UC140017."TABLE_4";
SQL SELECT *
FROM UC140017."TABLE_5";
This results in the following output:
Connecting to Oracle X;DBQ=db1.dc.man.lan
Connected
TABLE_1 2.421 lines fetched
TABLE_2 1 lines fetched
TABLE_2 << TABLE_3 2 lines fetched
TABLE_2 << TABLE_4 22 lines fetched
TABLE_2 << TABLE_5 22 lines fetched
There is no reason why TABLE_3, TABLE_4 & TABLE_5 are joined to TABLE_2. This relationship doesn't exist in the database and I don't see the option to change this in QlikView. Does anyone of you know where this is coming from and has suggestions how to fix this? Thanks!
Best,
Christoph
If the columns in Table_2,Table_3,Table_4 and Table_5 are the same number and same names then QV will auto concatenate them in one table. To avoid this you can use "NoConcatenate" prefix:
SQL SELECT *
FROM UC140017."TABLE_1";
NoConcatenate
SQL SELECT *
FROM UC140017."TABLE_2";
NoConcatenate
SQL SELECT *
FROM UC140017."TABLE_3";
NoConcatenate
SQL SELECT *
FROM UC140017."TABLE_4";
NoConcatenate
SQL SELECT *
FROM UC140017."TABLE_5";
This will force QV to treat all tables as different tables. Be aware that, if this is the case, then after the reload you will have massive synthetic key.
I'm using SAS to access an Oracle database. The problem is that the function / stored procedure lives on one server in Oracle - which is fine when my data lives there too - but when the data is on a different server I still want to use that function. So I loaded some macros with the personal id's to pass them to the function in a loop. It works, but it's painfully slow. I don't need 'optimal', just 'reasonable'...my datasets will max around 100,000 rows. I've read that creating a dataset is one of the most resource intensive jobs in SAS, so I'm experimenting with creating an empty table and insert into, but I haven't noticed much gain yet.
So the question is - can I use the Oracle stored procedures for data on a different server in a reasonable amount of time within SAS? (Either by improving my existing approach or something completely different)
My first attempt (around 25 minutes for 13,000 personal id's):
%MACRO STATE() ;
options nosource nonotes;
%* 2. get macro max loop n;
proc sql noprint;
select left(put(count(distinct pidm),10.)) into :loopn from examp
;quit;
%* 3. load macros with the pidms of interest;
proc sql noprint;
select distinct pidm into :pidm1 - :pidm&loopn from examp order by pidm;
quit;
%Do i = 1 %TO &loopn ; /*build em */
%* %put **************LOOP &i OF &loopn *********************;
proc sql noprint;
connect to oracle as mycon(user=xxxxxx password=xxxxxxx path='PROD') ;
create table subsetdat&i as
select * from connection to mycon
(select %quote(&&pidm&i) as pidm ,UILIB.ADDR.STATE(&&pidm&i, 'MA') as state
from dual);
disconnect from mycon ;
; quit;
%END;
data state; set subsetdat1-subsetdat&loopn ; /*stack 'em */
%Do j = 1 %TO &loopn ; /*drop 'em */
proc sql ;
drop table subsetdat&j
;
%END;
options source notes;
%MEND STATE ;
options nomprint;
%STATE() ;
Move to loop inside the proc sql, thereby removing the overhead of creating multiple datasets from multiple pass-through queries, and use a union all to 'stack' the individual query results together.
%MACRO STATE() ;
options nosource nonotes;
/* 2. get macro max loop n; */
proc sql noprint;
select left(put(count(distinct pidm),10.)) into :loopn from examp
;quit;
/* 3. load macros with the pidms of interest; */
proc sql noprint;
select distinct pidm into :pidm1 - :pidm&loopn from examp order by pidm;
quit;
/* Build single pass-thru query with multiple select ... union all select ... etc */
proc sql noprint;
connect to oracle as mycon(user=xxxxxx password=xxxxxxx path='PROD') ;
create table state as
select * from connection to mycon
(%DO I = 1 %TO &loopn ; /*build em */
select %quote(&&pidm&i) as pidm ,UILIB.ADDR.STATE(&&pidm&i, 'MA') as state from dual
%IF &I lt &LOOPN %THEN %DO ; /* if not last iteration do a `union all` */
union all
%END ;
%END ;
) ;
disconnect from mycon ;
quit;
options source notes;
%MEND STATE ;
options nomprint;
%STATE() ;