I have two type of select statement with the same complicated where clause.
One - Returns transaction detailed data
select field_1, field_2, field_3, ... , field_30 from my_table where my_where_clause
Second - Returns transaction data grouped by(distinct) merchants
select distinct field_1, field_2, field_8 from my_table where my_where_clause
Statements are called separately.
I want to simplify my code and not to repeat this complicated where clause in both statements without loosing performance
In dynamic SQL it's possible but I don't want to use dynamic SQL.
Any suggestions?
Suggestion: you can try GROUPING SETS expression.
It allows you to selectively specify the set of groups that you want to create within a GROUP BY clause. In
In your case, you can specify 2 sets, one group by set for all fields from 1 to 30 and another set for fields 1,2&8.
Link- https://docs.oracle.com/cd/E40518_01/server.761/es_eql/src/reql_aggregation_grouping_sets.html
However, it will return the output of both the groups in a single resultset, not sure if this fits in your design.
So you could encapsulate this statement, in a view or function, e,g,:
create or replace view view_1 as
select field_1, field_2, field_3, ... , field_30
from my_table
where my_where_clause
Then your second query could be
select distinct * from view_1;
You said that you are using this query from java. Try this.
create or replace function get_cursor(p_type varchar2 default null/* other paramethers*/ ) return sys_refcursor
is
result_curosr sys_refcursor;
begin
open result_curosr for 'select '||p_type||' object_type,status from user_objects' /* where clausele */ ;
return result_curosr;
end;
And usage of this from java.
Connection con = ...
CallableStatement callableStatement = con.prepareCall("declare c sys_refcursor; begin ? := get_cursor(?); end ; ");
callableStatement.registerOutParameter(1, OracleTypes.CURSOR);
callableStatement.setString(2, "Distinct"); // for distinct
or
callableStatement.setNull(2, OracleTypes.VARCHAR); // for full results
callableStatement.executeUpdate();
ResultSet rs = (ResultSet) callableStatement.getObject(1);
while(rs.next()) {
System.err.println(rs.getString(1));
}
rs.close();
con.close();
Other solution.
Add one more parameter and do simple deduplication using all columns from query. But i don't see any advantages.
select object_type,status from
(select object_type,status, row_number() over( partition by object_type,status order by 1) rn from user_objects /* your_where_clusue */
) where rn = case when 'DISTIINCT'/* <- paramete here :isDistinct */ = 'DISTIINCT' then 1 else rn end;
You can make dynamic SQL more readable by using multi-line strings, alternative quotes, and templates.
declare
v_select varchar2(32767);
v_where varchar2(32767);
v_code varchar2(32767) := '
##SELECT##
##WHERE##
';
begin
--Populate the clauses.
if ... then
v_select := 'select field_1, field_2, field_3, ... , field_30 from my_table';
else
v_select := 'select distinct field_1, field_2, field_8 from my_table';
end if;
if ... then
v_where :=
q'[
where field_1 = 'foo'
and field_2 = :bind1
...
]';
else
v_where :=
q'[
where field_2 = 'bar'
and field_2 = :bind2
...
]';
end if;
--Fill in the code.
v_code := replace(v_code, '##SELECT##', v_select);
v_code := replace(v_code, '##WHERE##', v_where);
--Print the code to check the formatting. Remove after testing.
dbms_output.put_line(v_code);
--Run it.
execute immediate v_code using ...;
end;
/
It's not perfect but it prevents ugly concatenation. And it's much better than the anti-patterns needed to avoid dynamic SQL at all costs. In most languages features like polymorphism and reflection are better than dynamic code. PL/SQL does not have good support for those advanced features so it's usually better to build the code as a string.
Related
I need to execute a query where the where-clause is generated based on user input. The input consists of 0 or more pairs of varchar2s.
For example:
[('firstname','John')
,('lastname','Smith')
,('street','somestreetname')]
This would translate into:
where (attrib = 'firstname' and value = 'John')
and (attrib = 'lastname' and value = 'Smith')
and (attrib = 'street' and value = 'somestreetname')
This is not the actual data structure as there are several tables but for this example lets keep it simple and say the values are in 1 table. Also I know the parentheses are not necessary in this case but I put them there to make things clear.
What I do now is loop over them and concatinate them to the SQL string. I made a stored proc to generate this where-clause which might also not be very secure since I just concat to the original query.
Something like the following, where I try to get the ID's of the nodes that correspond with the requested parameters:
l_query := select DISTINCT n.id from node n, attribute_values av
where av.node_id = n.id ' || getWhereClause(p_params)
open l_rc
for l_query;
fetch l_rc bulk collect into l_Ids;
close l_rc;
But this is not secure so I'm looking for a way that can guaranty security and prevent SQL-Injection attacks from happening.
Does anyone have any idea on how this is done in a secure way? I would like to use bindings but I don't see how I can do this when you dont know the number of parameters.
DB: v12.1.0.2 (i think)
It's still a bit unclear and generalised, but assuming you have a schema-level collection type, something like:
create type t_attr_value_pair as object (attrib varchar2(30), value varchar2(30))
/
create type t_attr_value_pairs as table of t_attr_value_pair
/
then you can use the attribute/value pairs in the collection for the bind:
declare
l_query varchar2(4000);
l_rc sys_refcursor;
type t_ids is table of number;
l_ids t_ids;
l_attr_value_pairs t_attr_value_pairs;
-- this is as shown in the question; sounds like it isn't exactly how you have it
p_params varchar2(4000) := q'^[('firstname','John')
,('lastname','Smith')
,('street','somestreetname')]^';
begin
-- whatever mechanism you want to get the value pairs into a collection;
-- this is just a quick hack to translate your example string
select t_attr_value_pair(rtrim(ltrim(
regexp_substr(replace(p_params, chr(10)), '(.*?)(,|$)', 1, (2 * level) - 1, null, 1),
'[('''), ''''),
rtrim(ltrim(
regexp_substr(replace(p_params, chr(10)), '(.*?)(,|$)', 1, 2 * level, null, 1),
''''), ''')]'))
bulk collect into l_attr_value_pairs
from dual
connect by level <= regexp_count(p_params, ',') / 2 + 1;
l_query := 'select DISTINCT id from attribute_values
where (attrib, value) in ((select attrib, value from table(:a)))';
open l_rc for l_query using l_attr_value_pairs;
fetch l_rc bulk collect into l_ids;
close l_rc;
for i in 1..l_ids.count loop
dbms_output.put_line('id ' || l_ids(i));
end loop;
end;
/
although it doesn't need to be dynamic with this approach:
...
begin
-- whatever mechamism you want to get the value pairs into a collection
...
select DISTINCT id
bulk collect into l_ids
from attribute_values
where (attrib, value) in ((select attrib, value from table(l_attr_value_pairs)));
for i in 1..l_ids.count loop
dbms_output.put_line('id ' || l_ids(i));
end loop;
end;
/
or with a join to the table collection expression:
select DISTINCT av.id
bulk collect into l_ids
from table(l_attr_value_pairs) t
join attribute_values av on av.attrib = t.attrib and av.value = t.value;
Other collection types will need different approaches.
Alternatively, you could still build up your where clause with one condition per attribute/value pair, while still making them bind variables - but you would need two levels of dynamic SQL, similar to this.
I am looking for a way to search columns with a LONG datatype.
I know those are deprecated (and I've always hated working with them...), but for some reason Oracle themselves continue to use them in their own tables and views...
Basically I want to build a query on SYS.USER_TAB_SUBPARTITIONS with the WHERE-clause filtering a specific HIGH_VALUE.
HIGH_VALUE is of the LONG datatype and the only way I know to filter those things, is by using the undocumented function dbms_metadata_util.long2varchar
When executing a query with this function however, the returned value is NULL.
select sys.dbms_metadata_util.long2varchar(2000,'SYS.USER_TAB_SUBPARTITIONS','HIGH_VALUE', rowid) from USER_TAB_SUBPARTITIONS;
This is most likely because USER_TAB_SUBPARTITIONS is not actually a table, but a view. And views don't have rowids...
However, it seems to be a strange kind of view, as its definition does not show any underlying base table. Instead it just creates a synonym on itself.
So, to my actual question(s): Is there any other way to query LONG? Does anybody know the "base table" of USER_TAB_SUBPARTITIONS?
Yes, data type LONG in Oracle System-Views is a pain. When I have to use such values I use this one:
DECLARE
high_value INTEGER;
BEGIN
FOR aPart IN (SELECT * FROM USER_TAB_SUBPARTITIONS) LOOP
EXECUTE IMMEDIATE 'BEGIN :ret := '||aPart.HIGH_VALUE||'; END;' USING OUT high_value;
SELECT ...
WHERE ... = high_value;
end loop;
END;
Note, in this example HIGH_VALUE is an integer value. However, it can be anything else (e.g. a TIMESTAMP), consider this in your procedure. For example like this:
FUNCTION IntervalType(tableName IN VARCHAR2) RETURN VARCHAR2 IS
EXPRESSION_IS_OF_WRONG_TYPE EXCEPTION;
PRAGMA EXCEPTION_INIT(EXPRESSION_IS_OF_WRONG_TYPE, -6550);
ds INTERVAL DAY TO SECOND;
ym INTERVAL YEAR TO MONTH;
str VARCHAR2(1000);
BEGIN
SELECT INTERVAL
INTO str
FROM USER_PART_TABLES
WHERE TABLE_NAME = tableName;
EXECUTE IMMEDIATE 'BEGIN :ret := '||str||'; END;' USING OUT ym;
RETURN 'YEAR TO MONTH Interval of '||ym;
EXCEPTION
WHEN EXPRESSION_IS_OF_WRONG_TYPE THEN
EXECUTE IMMEDIATE 'BEGIN :ret := '||str||'; END;' USING OUT ds;
RETURN 'DAY TO SECOND Interval of '||ds;
END IntervalType;
If you query ALL_VIEWS or DBA_VIEWS, you will find the definition of the view USER_TAB_SUBPARTITIONS
SELECT TEXT
FROM all_views
WHERE view_name = 'USER_TAB_SUBPARTITIONS';
You will see that the HIGH_VALUE comes from hiboundval column of sys.tabsubpart$.
There is one other way we use to extract the HIGH_VALUE . You may use SUBSTR() to extract the exact values from the extracted HIGH_VALUE.
DECLARE
v_high_value VARCHAR2(100);
BEGIN
SELECT EXTRACTVALUE (
DBMS_XMLGEN.GETXMLTYPE (
'SELECT high_value
FROM all_tab_partitions
WHERE partition_name='''
|| YOUR_PARTITION_NAME
|| '''
AND table_owner='''
|| YOUR_TABLE_OWNER
|| '''
AND table_name='''
|| YOUR_TABLE
|| ''''),
'ROWSET/ROW/HIGH_VALUE') INTO v_high_value
FROM DUAL;
END;
/
You may refer Ask TOM article here
I am trying to create a PL/SQL script that extracts a root "object" together with all children and other relevant information from an oracle production database. The purpose is to create a set of test-data to recreate issues that are encountered in production. Due to data protection laws the data needs to be anonymized when extracted - object names, certain types of id's, and monetary amounts need to be replaced.
I was trying to create one or more temporary translation tables, which would contain both the original values and anonymized versions. Then I would join the real data with the translation tables and output the anonymized values wherever required.
DECLARE
rootId integer := 123456;
TYPE anonTableRow IS RECORD
(
id NUMBER,
fieldC NUMBER,
anonymizedFieldC NUMBER
);
TYPE anonTable IS TABLE OF anonTableRow;
anonObject anonTable;
BEGIN
FOR cursor_row IN
(
select
id,
fieldC,
1234 -- Here I would create anonymized values based on rowNum or something similar
from
prodTable
where id = rootId
)
LOOP
i := i + 1;
anonObject(i) := cursor_row;
END LOOP;
FOR cursor_row IN
(
select
prod_table.id,
prod_table.fieldB,
temp_table.anonymizedFieldC fieldC,
prod_table.fieldD
from
prod_table
inner join table(temp_table) on prod_table.id = temp_table.id
where prod_table.id = 123456789
)
LOOP
dbms_output.put_line('INSERT INTO prod_table VALUES (' || cursor_row.id || ', ' || cursor_row.fieldB || ', ' || cursor_row.fieldC || ', , ' || cursor_row.fieldD);
END LOOP;
END;
/
However I ran into several problems with this approach - it seems to be near impossible to join oracle PL/SQL tables with real database tables. My access to the production database is severely restricted, so I cannot create global temporary tables, declare types outside PL/SQL or anything of that sort.
My attempt to declare my own PL/SQL types failed with the problems mentioned in this question - the solution does not work for me because of the limited permissions.
Is there a pure PL/SQL way that does not require fancy permissions to achieve something like the above?
Please Note: The above code example is simplified a lot and would not really require a separate translation table - in reality I need access to the original and translated values in several different queries, so I would prefer not having to "recalculate" translations everywhere.
If your data is properly normalized, then I guess this should only be necessary for internal IDs (not sure why you need to translate them though).
The following code should work for you, keeping the mappings in Associative Arrays:
DECLARE
TYPE t_number_mapping IS TABLE OF PLS_INTEGER INDEX BY PLS_INTEGER;
mapping_field_c t_number_mapping;
BEGIN
-- Prepare mapping
FOR cur IN (
SELECT 101 AS field_c FROM dual UNION ALL SELECT 102 FROM dual -- test-data
) LOOP
mapping_field_c(cur.field_c) := mapping_field_c.COUNT; -- first entry mapped to 1
END LOOP;
-- Use mapping
FOR cur IN (
SELECT 101 AS field_c FROM dual UNION ALL SELECT 102 FROM dual -- test-data
) LOOP
-- You can use the mapping when generating the `INSERT` statement
dbms_output.put_line( cur.field_c || ' mapped to ' || mapping_field_c(cur.field_c) );
END LOOP;
END;
Output:
101 mapped to 1
102 mapped to 2
If this isn't a permanent piece of production code, how about "borrowing" an existing collection type - e.g. one define in SYS that you can access.
Using this script from your schema you can generate a SQL Plus script to describe all SYS-owned types:
select 'desc ' || type_name from all_types
where typecode = 'COLLECTION'
and owner = 'SYS';
Running the resulting script will show you the structure of all the ones you can access. This one looks potentially suitable for example:
SQL> desc KU$_PARAMVALUES1010
KU$_PARAMVALUES1010 TABLE OF SYS.KU$_PARAMVALUE1010
Name Null? Type
----------------------------------------- -------- ----------------------------
PARAM_NAME VARCHAR2(30)
PARAM_OP VARCHAR2(30)
PARAM_TYPE VARCHAR2(30)
PARAM_LENGTH NUMBER
PARAM_VALUE_N NUMBER
PARAM_VALUE_T VARCHAR2(4000)
Of course, you can't guarantee that type will still exist or be the same or be accessible to you after a database upgrade, hence my caveat at the start.
More generic way to achieve this goal.
In my example i'm using xquery flwor expressions and dbms_xmlstore. Knowledge about xquery is mandatory.
create table mask_user_objects as select * from user_objects where rownum <0;
declare
v_s_table varchar2(30) := 'USER_OBJECTS'; --uppercase!!!
v_d_table varchar2(30) := 'MASK_USER_OBJECTS'; --uppercase!!!
v_mask_columns xmltype := xmltype('<COLS><OBJECT_NAME>XXXX</OBJECT_NAME>
<DATA_OBJECT_ID>-1</DATA_OBJECT_ID>
<OBJECT_TYPE/>
</COLS>'); --uppercase!!!
insCtx DBMS_XMLSTORE.ctxType;
r NUMBER;
v_source_table xmltype;
v_cursor sys_refcursor;
begin
open v_cursor for 'select * from '||v_s_table||' where rownum <100 ';
v_source_table := xmltype(v_cursor);
close v_cursor;
-- Load source table into xmltype.
insCtx := DBMS_XMLSTORE.newContext(v_d_table); -- Get saved context
for rec in (
select tt.column_value from xmltable('
let $col := $anomyze/COLS
for $i in $doc/ROWSET/ROW
let $row := $i
return <ROWSET>
<ROW>
{
for $x in $row/*
return if(
exists($col/*[name() = $x/name()] )
) then element{$x/name()}{ $col/*[name() = $x/name()]/text() }
else element{$x/name()}{$x/text()}
}
</ROW>
</ROWSET>
'
passing v_source_table as "doc"
, v_mask_columns as "anomyze"
) tt) loop
null;
r := DBMS_XMLSTORE.insertXML(insCtx, rec.column_value);
end loop;
DBMS_XMLSTORE.closeContext(insCtx);
end;
I have been trying to create PL/SQL statement that creates a dynamic query in order to get dynamic columns. Since I don't have much idea about these Oracle PL/SQL statements; I am confused about few things.
Is it mandatory to have stored procedure for creating dynamic queries ?
The following query does not throw any error and even results nothing. What I am trying to do in the following query is to get sum of FKOD_AMOUNT according to FKBAB_SOURCE_ID which is foreign key for PFS_SOURCE_ID.
declare
sql_query varchar2(3000) := 'select FKOM_OFFICE_ID,FKBAM_BUDGET_ID ';
begin
for x in (select distinct PFS_SOURCE_ID,PFS_SOURCE_ENG from PBS_FC_SOURCE WHERE PFS_UPPER_SOURCE_ID!=0 )
loop
sql_query := sql_query ||
' , sum(case when FKBAB_SOURCE_ID = '||x.PFS_SOURCE_ID||' then FKOD_AMOUNT ELSE 0 end) as '||x.PFS_SOURCE_ENG;
dbms_output.put_line(sql_query);
end loop;
sql_query := sql_query || ' FROM FMS_K_OFFICEWISE_DTL
JOIN FMS_K_OFFICEWISE_MST ON FMS_K_OFFICEWISE_MST.FKOM_OFFICE_MST_ID=FMS_K_OFFICEWISE_DTL.FKOD_OFFICE_MST_ID
JOIN FMS_K_BUDGET_ALLOCATION_DTL ON FMS_K_BUDGET_ALLOCATION_DTL.FKBAD_BUDGET_ALLOC_DTL_ID=FMS_K_OFFICEWISE_DTL.FKOD_BUDGET_ALLOC_AD_ID
JOIN FMS_K_BUDGET_ALLOCATION_MST ON FMS_K_BUDGET_ALLOCATION_MST.FKBAM_BUDGET_ALLOC_ID=FMS_K_BUDGET_ALLOCATION_DTL.FKBAB_BUDGET_ALLOC_ID
JOIN PBS_FC_BUDGET ON PBS_FC_BUDGET.PFB_BUDGET_ID=FMS_K_BUDGET_ALLOCATION_MST.FKBAM_BUDGET_ID
GROUP BY FKOM_OFFICE_ID,FKBAM_BUDGET_ID ';
dbms_output.put_line(sql_query);
end;
How can I execute 'sql_query' ?
To have an answer(removed my comments), this is pl/sql, not a select statement, so you can't just run it.
With execute immediate you can use INTO clause to store the results into some variables or arrays.
But you may make the query a cursor, put the cursor into a pipelined function and then
select * from table(your_pipelined_function)
As of now I am using IF ELSE to handle this condition
IF INPUT_PARAM IS NOT NULL
SELECT ... FROM SOMETABLE WHERE COLUMN = INPUT_PARAM
ELSE
SELECT ... FROM SOMETABLE
Is there any better way to do this in a single query without IF ELSE loops. As the query gets complex there will be more input parameters like this and the amount of IF ELSE required would be too much.
One method would be to use a variant of
WHERE column = nvl(var, column)
There are two pitfalls here however:
if the column is nullable, this clause will filter null values whereas in your question you would not filter the null values in the second case. You could modify this clause to take nulls into account but it turns ugly:
WHERE nvl(column, impossible_value) = nvl(var, impossible_value)
Of course if somehow the impossible_value is ever inserted you will run into some other kind of (fun) problems.
The optimizer doesn't understand correctly this type of clause. It will sometimes produce a plan with a UNION ALL but if there are more than a couple of nvl, you will get full scan even if perfectly valid indexes are present.
This is why when there are lots of parameters (several search fields in a big form for example), I like to use dynamic SQL:
DECLARE
l_query VARCHAR2(32767) := 'SELECT ... JOIN ... WHERE 1 = 1';
BEGIN
IF param1 IS NOT NULL THEN
l_query := l_query || ' AND column1 = :p1';
ELSE
l_query := l_query || ' AND :p1 IS NULL';
END IF;
/* repeat for each parameter */
...
/* open the cursor dynamically */
OPEN your_ref_cursor FOR l_query USING param1 /*,param2...*/;
END;
You can also use EXECUTE IMMEDIATE l_query INTO l_result USING param1;
This should work
SELECT ... FROM SOMETABLE WHERE COLUMN = NVL( INPUT_PARAM, COLUMN )