What is the difference between creating a 'normal' UDF in Oracle and one that is a macro? For example, for the macro, they give an example of:
CREATE FUNCTION date_string(dat DATE)
RETURN VARCHAR2 SQL_MACRO(SCALAR) IS
BEGIN
RETURN q'{
TO_CHAR(dat, 'YYYY-MM-DD')
}';
END;
/
And for a function.
What would be a use-case where a macro would be more useful than a normal function? And when a function would be better used that a macro? Does one ever have either performance benefits or limitations over the other?
The SQL & PL/SQL languages have separate runtime engines. This means every time a SQL statement calls a PL/SQL UDF, there's a context switch (and vice-versa).
While each switch is fast, calling a PL/SQL function thousands or millions of times in a SQL statement can make it significantly slower.
SQL macros work differently. At parse time the database resolves the expression to become part of the statement. It searches for the parameter names in the return string. Then effectively does a find/replace of these with the text of whatever you've passed for these parameters.
For example, if you run:
select date_string ( date_col )
from some_table;
The final SQL statement is effectively:
select to_char ( date_col, 'yyyy-mm-dd' )
from some_table;
This means there's no runtime context switch. This can lead to good performance gains, for example:
create function date_string_macro ( dat date )
return varchar2 sql_macro(scalar) is
begin
return q'{ to_char(dat, 'yyyy-mm-dd') }';
end;
/
create function date_string_plsql ( dat date )
return varchar2 is
begin
return to_char(dat, 'yyyy-mm-dd' );
end;
/
declare
start_time pls_integer;
begin
start_time := dbms_utility.get_time ();
for rws in (
select *
from dual
where date_string_plsql ( sysdate + level ) > '2021'
connect by level <= 1000000
) loop
null;
end loop;
dbms_output.put_line (
'PL/SQL runtime = ' || ( dbms_utility.get_time () - start_time )
);
start_time := dbms_utility.get_time ();
for rws in (
select *
from dual
where date_string_plsql ( sysdate + level ) > '2021'
connect by level <= 100000
) loop
null;
end loop;
dbms_output.put_line (
'Macro runtime = ' || ( dbms_utility.get_time () - start_time )
);
end;
/
PL/SQL runtime = 570
Macro runtime = 54
Around 10x faster in this case!
Because the expression becomes part of the SQL statement, as a side benefit the optimizer has full visibility of the underlying expression. This may lead to better execution plans.
This enables you to get the code-reuse benefits of PL/SQL functions (e.g. common formulas, string formatting, etc.) with the performance of pure SQL.
So why not make all existing PL/SQL function macros?
There are a couple of other important differences between them.
First up, order of argument evaluation. PL/SQL uses application-order, macros use normal-order. This can lead to behaviour differences in some cases:
create or replace function first_not_null (
v1 int, v2 int
)
return int as
begin
return coalesce ( v1, v2 );
end first_not_null;
/
select first_not_null ( 1, 1/0 ) from dual;
ORA-01476: divisor is equal to zero
create or replace function first_not_null (
v1 int, v2 int
)
return varchar2 sql_macro ( scalar ) as
begin
return ' coalesce ( v1, v2 ) ';
end first_not_null;
/
select first_not_null ( 1, 1/0 ) from dual;
FIRST_NOT_NULL(1,1/0)
1
Secondly - and more importantly - resolving the expression only happens in SQL. If you call a SQL macro in PL/SQL, it returns the string as-is:
exec dbms_output.put_line ( first_not_null ( 1, 0 ) );
coalesce ( v1, v2 )
Finally SQL macros also have a table variant. This allows you to create template queries you can pass tables and columns to.
I've discussed table macros in several blog posts, showing how you can use table macros to write generic top-N per group functions and reusable CSV-to-rows functions
Related
I have a function inside my package that is meant to split up a comma-separated varchar2 input into rows, ie. 'one, two, three' into:
one
two
three
I have declared the function as:
function unpack_list(
string_in in varchar2
) return result_str
is
result_rows result_str;
begin
with temp_table as (
SELECT distinct trim(regexp_substr(string_in, '[^,]+', 1, level)) str
FROM (SELECT string_in FROM dual) t
CONNECT BY instr(string_in, ',', 1, level - 1) > 0)
select str bulk collect into result_rows from temp_table;
RETURN result_rows;
end;
and the return type as:
type result_str is table of varchar2(100);
However, calling the function like:
select * from unpack_list('one1, two2')
gives the following error:
ORA-00902: Invalid datatype
any ideas what causes this?
You are calling a PL/SQL function that returns a PL/SQL collection type (both defined in your package) from a SQL context. You can't do that directly. You can call the function from a PL/SQL context, assigning the result to a variable of the same type, but that isn't how you're trying to use it. db<>fiddle showing your set-up, your error, and it working in a PL/SQL block.
You could declare the type at schema level instead, as #Littlefoot showed:
create type result_str is table of varchar2(100);
and remove the package definition, which would clash; that works for both SQL and PL/SQL (db<>fiddle).
Or if you can't create a schema-level type, you could use a built-in one:
function unpack_list(
string_in in varchar2
) return sys.odcivarchar2list
is
result_rows sys.odcivarchar2list;
begin
with temp_table as (
SELECT distinct trim(regexp_substr(string_in, '[^,]+', 1, level)) str
FROM (SELECT string_in FROM dual) t
CONNECT BY instr(string_in, ',', 1, level - 1) > 0)
select str bulk collect into result_rows from temp_table;
RETURN result_rows;
end;
which also works for both SQL and PL/SQL (db<>fiddle).
Or you could use a pipelined function, with your PL/SQL collection type:
function unpack_list(
string_in in varchar2
) return result_str pipelined
is
begin
for r in (
SELECT distinct trim(regexp_substr(string_in, '[^,]+', 1, level)) str
FROM (SELECT string_in FROM dual) t
CONNECT BY instr(string_in, ',', 1, level - 1) > 0)
loop
pipe row (r.str);
end loop;
RETURN;
end;
which works in SQL, or in SQL running within PL/SQL, but not with direct assignment to a collection variable (db<>fiddle).
Which approach you take depends on how you need to call the function really. there may be some performance differences, but you might not notice unless they are called repeatedly and intensively.
The reason of the error was described earlier, so I will post another possible solution. For Oracle 19c (version 19.7) and above you may skip creation of table type and use SQL_MACRO addition. Returned query will be integrated into the main query.
create function unpack_list (
string_in varchar2
)
return clob
sql_macro(table)
is
begin
return q'[
select distinct
trim(regexp_substr(
unpack_list.string_in,
'[^,]+', 1, level
)) as str
from dual
connect by
instr(
unpack_list.string_in,
',', 1, level - 1
) > 0
]';
end;
/
select *
from unpack_list(
string_in => 'one,two'
)
| STR |
| :-- |
| one |
| two |
db<>fiddle here
I have a query like this
SELECT ID,REF_ID,BATCHNO FROM reporttbl
where POSTING_DT >= '06/01/2020' and POSTING_DT <= '06/30/2020'
and I need it every month, so I would like to put it in a view, but as the date changes every month, it would be great to have a date parameter that I can pass to the view when calling it. Is there a way on how can i achieved this?
I'm new to oracle, appreciate every help. Thank youu.
From 19.6 you can create parameterized views using SQL macros.
create or replace function get_month (
tab dbms_tf.table_t, start_date date, end_date date
) return varchar2 sql_macro as
retval int;
begin
return 'select * from tab
where dt >= start_date and dt < end_date + 1';
end get_month;
/
create table t (
c1 int, dt date
);
insert into t
with rws as (
select level c1, add_months ( date'2019-12-25', level ) dt
from dual
connect by level <= 10
)
select * from rws;
select * from get_month (
t, date'2020-06-01', date'2020-07-01'
);
C1 DT
6 25-JUN-2020 00:00:00
select * from get_month (
t, date'2020-08-01', date'2020-09-01'
);
C1 DT
8 25-AUG-2020 00:00:00
This query return the data for the previous month, i.e. the month befort the current month at the time of the query (= sysdate).
You use the trunc with 'MM' to get the months first and the arithmetic with add_months
SELECT ID,REF_ID,BATCHNO FROM reporttbl
where POSTING_DT >= add_months(trunc(sysdate,'MM'),-1) and POSTING_DT < trunc(sysdate,'MM')
Another way to do this is using a function that retrieves the parameters from a table, thereby you don't need to manipulate any DDL. The idea here is
Using a table to store the parameters, basically you need parameter value and a parameter description.
Using a function to retrieve the value of that parameter when the input is the parameter name
Using the function call inside the view.
You can then manipulate the view automatically by modifying the values of the parameter table.
Table
create table my_param_table
( param_description varchar2(100) ,
param_value varchar2(100),
enabled varchar2(1)
) ;
Function
create or replace function f_retr_param ( p_value in varchar2 )
return varchar2
is
declare
v_value my_param_table.value_param%type;
begin
select value into v_value from my_table_of_parameters
where upper(value_param) = upper(p_value) ;
return v_value;
exception when others then raise;
end;
/
View
create or replace force view my_view as
SELECT ID,REF_ID,BATCHNO FROM reporttbl
where POSTING_DT >= f_retr_param ( p_value => 'p_start_date' );
and POSTING_DT <= f_retr_param ( p_value => 'p_end_date' );
There are ways to "parameterize" a view e.g. using Oracle contexts, but they aren't often useful and certainly not for your case.
If your query really just selects from one table with just the dates as predicates then a view doesn't add much value either. You could create a SQL script (in a file e.g. myquery.sql) using bind variables:
SELECT ID,REF_ID,BATCHNO FROM reporttbl
where POSTING_DT >= to_date(:from_date) and POSTING_DT <= to_date(:to_date);
Then every month you can just open the file and run it, and it will prompt you for the 2 dates. Or you can run as a script like this and it will also prompt you:
#myquery.sql
Or if you use substitution strings '&1.' and '&2.' instead:
SELECT ID,REF_ID,BATCHNO FROM reporttbl
where POSTING_DT >= to_date('&1.') and POSTING_DT <= to_date('&2.');
Then you can pass the dates in on the command line like this:
#myquery '06/01/2020' '06/30/2020'
(Because &1. means first parameter on command line, etc.)
CREATE FUNCTION dbo.Alphaorder (#str VARCHAR(50))
returns VARCHAR(50)
BEGIN
DECLARE #len INT,
#cnt INT =1,
#str1 VARCHAR(50)='',
#output VARCHAR(50)=''
SELECT #len = Len(#str)
WHILE #cnt <= #len
BEGIN
SELECT #str1 += Substring(#str, #cnt, 1) + ','
SET #cnt+=1
END
SELECT #str1 = LEFT(#str1, Len(#str1) - 1)
SELECT #output += Sp_data
FROM (SELECT Split.a.value('.', 'VARCHAR(100)') Sp_data
FROM (SELECT Cast ('<M>' + Replace(#str1, ',', '</M><M>') +
'</M>' AS XML) AS Data) AS A
CROSS APPLY Data.nodes ('/M') AS Split(a)) A
ORDER BY Sp_data
RETURN #output
END
SELECT dbo.Alphaorder ('juan') --> ajnu
That looks like a badly written function to begin with.
If you need to do something like that function does in Oracle SQL, you can do it directly with Oracle SQL features, you don't need to write your own function.
Even if for some reason you do need to write a function, it is perhaps easiest to let SQL do the work for you (the same as you would do if your context was straight SQL).
Something like this:
create or replace function alphaorder(str varchar2) return varchar2
as
output varchar2(4000);
begin
select listagg(ch) within group (order by ch)
into output
from ( select substr(str, level, 1) as ch
from dual
connect by level <= length(str)
)
;
return output;
end;
/
Note that in Oracle you can't limit the length of the input or of the output string; you can only declare the data type. Then in the function itself you can check length and throw an error if the input is longer than 50 characters, but why bother? Let the function work in full generality.
Here's how you would call the function (and check that it works as required):
select alphaorder('juan') as alpha_ordered from dual;
ALPHA_ORDERED
-------------
ajnu
I am trying to split a comma delimited string using regexp_substr, but get "into clause is expected" error:
select regexp_substr(improv,'[^,]+', 1, level) from dual
connect by regexp_substr(improv, '[^,]+', 1, level) is not null;
"improv" is a varchar(100) variable, and I am running the above statement inside a PLSQL block.
In PL/SQL you need to output the SQl query INTO a variable.
However, since this query is going to generate multiple rows you probably want to use BULK COLLECT INTO rather than just INTO and to put the output into a user-defined collection or a VARRAY (which SYS.ODCIVARCHAR2LIST is an example of. Note: you cannot use the MEMBER OF operator with a VARRAY):
DECLARE
list_of_improvs SYS.ODCIVARCHAR2LIST;
BEGIN
SELECT regexp_substr(improv,'[^,]+', 1, level)
BULK COLLECT INTO list_of_improvs
FROM DUAL
CONNECT BY regexp_substr(improv, '[^,]+', 1, level) is not null;
END;
/
Update:
In response to your comment - you can use it like this (although it is unclear what you are trying to achieve so I have just put your code into a snippet without trying to work out what you intend it to do):
DECLARE
list_of_improvs SYS.ODCIVARCHAR2LIST;
BEGIN
SELECT regexp_substr(improv,'[^,]+', 1, level)
BULK COLLECT INTO list_of_improvs
FROM DUAL
CONNECT BY regexp_substr(improv, '[^,]+', 1, level) is not null;
FOR i IN 1 .. list_of_improvs.COUNT LOOP
DBMS_OUTPUT.PUT_LINE( improvs(i) );
END LOOP;
update line_cap
set cap_up = list_of_improvs(1)
where id IN ( SELECT Column_Value
FROM TABLE( list_of_improvs ) );
END;
/
You can't use IN directly with a collection or VARRAY and need to use a TABLE() collection expression in a nested query to get the values out.
If you are using a user-defined SQL collection - i.e. defined with a statement like this:
CREATE TYPE StringList IS TABLE OF VARCHAR2(4000);
Then you can use the MEMBER OF operator:
DECLARE
list_of_improvs StringList;
BEGIN
-- as above
update line_cap
set cap_up = list_of_improvs(1)
where id MEMBER OF list_of_improvs;
END;
/
But you cannot use the MEMBER OF operator with VARRAYs (like SYS.ODCIVARCHAR2LIST).
However, you don't need PL/SQL for that (and eliminate costly context switches between the PL/SQL and SQL execution scopes) and could just use a MERGE statement something like:
MERGE INTO line_cap dst
USING (
SELECT MIN( value ) KEEP ( DENSE_RANK FIRST ORDER BY ROWNUM ) OVER () AS first_value,
value
FROM (
SELECT regexp_substr(improv,'[^,]+', 1, level) AS value
FROM DUAL
CONNECT BY regexp_substr(improv, '[^,]+', 1, level) is not null
)
) src
ON ( src.value = dst.id )
WHEN MATCHED THEN
UPDATE
SET cap_up = first_value;
I put this here instead of a comment to MT0's answer, as comments don't work for formatting. Anyway here's a complete example using MT0's answer to load an array and then looping through it to display the contents. This will show you how to access the contents of the list. Give the credit to MT0 for the answer to your original question.
DECLARE
list_of_improvs SYS.ODCIVARCHAR2LIST;
i number;
BEGIN
SELECT regexp_substr('1,2,3,4,5','(.*?)(,|$)', 1, level, NULL, 1)
BULK COLLECT INTO list_of_improvs
FROM dual
CONNECT BY level <= regexp_count('1,2,3,4,5', ',') + 1;
i := list_of_improvs.FIRST; -- Get first element of array
while i is not null LOOP
DBMS_OUTPUT.PUT_LINE(list_of_improvs(i));
i := list_of_improvs.NEXT(i); -- Get next element of array
END LOOP;
END;
/
I have a big query with nesting and left join and Ineed to create a view out of it so as not to run it from the application. The issue is I need the date range and some other fields as input parameters since it will vary from the front end for each request.
I just looked up and saw some posts referring to using SYS_CONTEXT for parameterized views and need to know exactly how do I create the view for example with 2 parameters - fromdate, todate and how I invoke the view from the application.
Just for info I am using grails/groovy for developing the application.
and here is the query I want to create view out of..
select
d.dateInRange as dateval,
eventdesc,
nvl(td.dist_ucnt, 0) as dist_ucnt
from (
select
to_date(fromdate,'dd-mon-yyyy') + rownum - 1 as dateInRange
from all_objects
where rownum <= to_date(fromdate,'dd-mon-yyyy') - to_date(todate,'dd-mon-yyyy') + 1
) d
left join (
select
to_char(user_transaction.transdate,'dd-mon-yyyy') as currentdate,
count(distinct(grauser_id)) as dist_ucnt,
eventdesc
from
gratransaction, user_transaction
where gratransaction.id = user_transaction.trans_id and
user_transaction.transdate between to_date(fromdate,'dd-mon-yyyy') and to_date(todate,'dd-mon-yyyy')
group by to_char(user_transaction.transdate, 'dd-mon-yyyy'), eventdesc
) td on td.currentdate = d.dateInRange order by d.dateInRange asc
The context method is described here: http://docs.oracle.com/cd/B28359_01/network.111/b28531/app_context.htm
e.g. (example adapted from the above link)
CREATE CONTEXT dates_ctx USING set_dates_ctx_pkg;
CREATE OR REPLACE PACKAGE set_dates_ctx_pkg IS
PROCEDURE set(d1 in date, d2 in date);
END;
/
CREATE OR REPLACE PACKAGE BODY set_dates_ctx_pkg IS
PROCEDURE set(d1 in date, d2 in date) IS
BEGIN
DBMS_SESSION.SET_CONTEXT('dates_ctx', 'd1', TO_CHAR(d1,'DD-MON-YYYY'));
DBMS_SESSION.SET_CONTEXT('dates_ctx', 'd2', TO_CHAR(d2,'DD-MON-YYYY'));
END;
END;
/
Then, set the dates in your application with:
BEGIN set_dates_ctx_pkg.set(mydate1, mydate2); END;
/
Then, query the parameters with:
SELECT bla FROM mytable
WHERE mydate
BETWEEN TO_DATE(
SYS_CONTEXT('dates_ctx', 'd1')
,'DD-MON-YYYY')
AND TO_DATE(
SYS_CONTEXT('dates_ctx', 'd2')
,'DD-MON-YYYY');
The advantage of this approach is that it is very query-friendly; it involves no DDL or DML at runtime, and therefore there are no transactions to worry about; and it is very fast because it involves no SQL - PL/SQL context switch.
Alternatively:
If the context method and John's package variables method are not possible for you, another one is to insert the parameters into a table (e.g. a global temporary table, if you're running the query in the same session), then join to that table from the view. The downside is that you now have to make sure you run some DML to insert the parameters whenever you want to run the query.
I have just made a workaround for this annoying Oracle disadvantage. Like this
create or replace package pkg_utl
as
type test_record is record (field1 number, field2 number, ret_prm_date date);
type test_table is table of test_record;
function get_test_table(prm_date date) return test_table pipelined;
end;
/
create or replace package body pkg_utl
as
function get_test_table(prm_date date) return test_table pipelined
is
begin
for item in (
select 1, 2, prm_date
from dual
) loop
pipe row (item);
end loop;
return;
end get_test_table;
end;
/
it still requires a package, but at least i can use it in more convinient way:
select *
from table(pkg_utl.get_test_table(sysdate))
i am not sure about performance...
To use parameters in a view one way is to create a package which will set the values of your parameters and have functions that can be called to get those values. For example:
create or replace package MYVIEW_PKG as
procedure SET_VALUES(FROMDATE date, TODATE date);
function GET_FROMDATE
return date;
function GET_TODATE
return date;
end MYVIEW_PKG;
create or replace package body MYVIEW_PKG as
G_FROM_DATE date;
G_TO_DATE date;
procedure SET_VALUES(P_FROMDATE date, P_TODATE date) as
begin
G_FROM_DATE := P_FROMDATE;
G_TO_DATE := P_TODATE;
end;
function GET_FROMDATE
return date is
begin
return G_FROM_DATE;
end;
function GET_TODATE
return date is
begin
return G_TO_DATE;
end;
end MYVIEW_PKG;
Then your view can be created thus:
create or replace view myview as
select
d.dateInRange as dateval,
eventdesc,
nvl(td.dist_ucnt, 0) as dist_ucnt
from (
select
MYVIEW_PKG.GET_FROMDATE + rownum - 1 as dateInRange
from all_objects
where rownum <= MYVIEW_PKG.GET_FROMDATE - MYVIEW_PKG.GET_TODATE + 1
) d
left join (
select
to_char(user_transaction.transdate,'dd-mon-yyyy') as currentdate,
count(distinct(grauser_id)) as dist_ucnt,
eventdesc
from
gratransaction, user_transaction
where gratransaction.id = user_transaction.trans_id and
user_transaction.transdate between MYVIEW_PKG.GET_FROMDATE and MYVIEW_PKG.GET_TODATE
group by to_char(user_transaction.transdate, 'dd-mon-yyyy'), eventdesc
) td on td.currentdate = d.dateInRange order by d.dateInRange asc;
And to run it you must set the values first:
exec MYVIEW_PKG.SET_VALUES(trunc(sysdate)-1,trunc(sysdate));
And then calls to it will use these values:
select * from myview;