declare #sql varchar(max)
declare #pimkeys varchar(max)
set #pimkeys = '69966,69965' -- input comma delimited values inside single quotation marks
SET #SQL =
'with basedata as(
SELECT
ec.Pim_key,pm.Parent_PIM_Key,pm.Item_Type,pm.Company_name,pm.Part_number,pm.Product_Desc,pm.Novation_Description
,ec.Name ,ec.Value
FROM edw..Products_PPS_Attributes AS ec
join edw..DM_Product_Master as pm on ec.Pim_Key = pm.PIM_Key
where ec.pim_key in ('+#pimkeys+'))
Select * from basedata
PIVOT ( MAX(Value) FOR Name IN( Sync_Code,Product_Line_Brand ) ) AS p'
execute (#SQL)
SQL Developer is the platform Oracle provides for doing SQL Server migrations. One of the features is our scratch editor.
You can paste in T-SQL, and it will spit out PL/SQL.
Tools > Migration > Scratch Editor
It's not guaranteed to have a 100% successful translation, just like any other translator, but should give you a nice start.
Here's what we end up with, without any testing.
DECLARE
v_pimkeys VARCHAR2(4000);
v_SQL varchar2(4000); -- i had to manually add this after the translation
BEGIN
v_pimkeys := '69966,69965' ;-- input comma delimited values inside single quotation marks
v_SQL := 'with basedata as(
SELECT
ec.Pim_key,pm.Parent_PIM_Key,pm.Item_Type,pm.Company_name,pm.Part_number,pm.Product_Desc,pm.Novation_Description
,ec.Name ,ec.Value
FROM edw..Products_PPS_Attributes AS ec
join edw..DM_Product_Master as pm on ec.Pim_Key = pm.PIM_Key
where ec.pim_key in (' || v_pimkeys || '))
Select * from basedata
PIVOT ( MAX(Value) FOR Name IN( Sync_Code,Product_Line_Brand ) ) AS p' ;
EXECUTE IMMEDIATE v_SQL;
END;
Related
i have a scenario where i need to create a dynamic query with some if/else statements, i have prepared the query but unable to use this inside loop below is the snippet of what i am trying to acheive.
Query :
select user_name into username from table1 where user ='user1';
for(query)
Loop
Dbms_output.put_line('user name ' || user_name);
END Loop;
is this possible to use the vaiable in the for loop ?
Not really. Notice in the 19c PL/SQL Reference it says:
cursor
Name of an explicit cursor (not a cursor variable) that is not open when the cursor FOR LOOP is entered.
You have to code this the long way, by explicitly fetching the cursor into a record until you hit cursorname%notfound, e.g.
create or replace procedure cheese_report
( cheese_cur in sys_refcursor )
as
type cheese_detail is record
( name varchar2(15)
, region varchar2(30) );
cheese cheese_detail;
begin
loop
fetch cheese_cur into cheese;
dbms_output.put_line(cheese.name || ', ' || cheese.region);
exit when cheese_cur%notfound;
end loop;
close cheese_cur;
end;
Test:
declare
cheese sys_refcursor;
begin
open cheese for
select 'Cheddar', 'UK' from dual union all
select 'Gruyere', 'France' from dual union all
select 'Ossau Iraty', 'Spain' from dual union all
select 'Yarg', 'UK' from dual;
cheese_report(cheese);
end;
/
Cheddar, UK
Gruyere, France
Ossau Iraty, Spain
Yarg, UK
In Oracle 21c you can simplify this somewhat, though you still have to know the structure of the result set:
create or replace procedure cheese_report
( cheese_cur in sys_refcursor )
as
type cheese_detail is record
( cheese varchar2(15)
, region varchar2(30) );
begin
for r cheese_detail in values of cheese_cur
loop
dbms_output.put_line(r.cheese || ', ' || r.region);
end loop;
end;
You could parse an unknown ref cursor using dbms_sql to find the column names and types, but it's not straightforward as you have to do every processing step yourself. For an example of something similar, see www.williamrobertson.net/documents/refcursor-to-csv.shtml.
The database-schema (Source and target) are very large (each has over 350 tables). I have got the task to somehow merge these two tables into one. The data itself (whats in the tables) has to be migrated. I have to be careful that there are no double entries for primary keys before or while merging the schemata. Has anybody ever done that already and would be able to provide me his solution or could anyone help me get a approach to the task? My approaches all failed and my advisor just tells me to get help online :/
To my approach:
I have tried using the "all_constraints" table to get all pks from my db.
SELECT cols.table_name, cols.column_name, cols.position, cons.status, cons.owner
FROM all_constraints cons, all_cons_columns cols
WHERE cols.owner = 'DB'
AND cons.constraint_type = 'P'
AND cons.constraint_name = cols.constraint_name
AND cons.owner = cols.owner
ORDER BY cols.table_name, cols.position;
I also "know" that there has to be a sequence for the primary keys to add values to it:
CREATE SEQUENCE seq_pk_addition
MINVALUE 1
MAXVALUE 99999999999999999999
START WITH 1
INCREMENT BY 1
CACHE 20;
Because I am a noob if it comes to pl/sql (or sql in general)
So how/what I should do next? :/
Here is a link for an ERD of the database: https://ufile.io/9tdoj
virus scan: https://www.virustotal.com/#/file/dbe5f418115e50313a2268fb33a924cc8cb57a43bc85b3bbf5f6a571b184627e/detection
As promised to help in my comment, i had prepared a dynamic code which you can try to get the data merged with the source and target tables. The logic is as below:
Step1: Get all the table names from the SOURCE schema. In the query below you can you need to replace the schema(owner) name respectively. For testing purpose i had taken only 1 table so when you run it,remove the table name filtering clause.
Step2: Get the constrained columns names for the table. This is used to prepared the ON clause which would be later used for MERGE statement.
Step3: Get the non-constrainted column names for the table. This would be used in UPDATE clause while using MERGE.
Step4: Prepare the insert list when the data doesnot match ON conditon of MERGE statement.
Read my inline comments to understand each step.
CREATE OR REPLACE PROCEDURE COPY_TABLE
AS
Type OBJ_NME is table of varchar2(100) index by pls_integer;
--To hold Table name
v_obj_nm OBJ_NME ;
--To hold Columns of table
v_col_nm OBJ_NME;
v_othr_col_nm OBJ_NME;
on_clause VARCHAR2(2000);
upd_clause VARCHAR2(4000);
cntr number:=0;
v_sql VARCHAR2(4000);
col_list1 VARCHAR2(4000);
col_list2 VARCHAR2(4000);
col_list3 VARCHAR2(4000);
col_list4 varchar2(4000);
col_list5 VARCHAR2(4000);
col_list6 VARCHAR2(4000);
col_list7 VARCHAR2(4000);
col_list8 varchar2(4000);
BEGIN
--Get Source table names
SELECT OBJECT_NAME
BULK COLLECT INTO v_obj_nm
FROM all_objects
WHERE owner LIKE 'RU%' -- Replace `RU%` with your Source schema name here
AND object_type = 'TABLE'
and object_name ='TEST'; --remove this condition if you want this to run for all tables
FOR I IN 1..v_obj_nm.count
loop
--Columns with Constraints
SELECT column_name
bulk collect into v_col_nm
FROM user_cons_columns
WHERE table_name = v_obj_nm(i);
--Columns without Constraints remain columns of table
SELECT *
BULK COLLECT INTO v_othr_col_nm
from (
SELECT column_name
FROM user_tab_cols
WHERE table_name = v_obj_nm(i)
MINUS
SELECT column_name
FROM user_cons_columns
WHERE table_name = v_obj_nm(i));
--Prepare Update Clause
FOR l IN 1..v_othr_col_nm.count
loop
cntr:=cntr+1;
upd_clause := 't1.'||v_othr_col_nm(l)||' = t2.' ||v_othr_col_nm(l);
upd_clause:=upd_clause ||' and ' ;
col_list1:= 't1.'||v_othr_col_nm(l) ||',';
col_list2:= col_list2||col_list1;
col_list5:= 't2.'||v_othr_col_nm(l) ||',';
col_list6:= col_list6||col_list5;
IF (cntr = v_othr_col_nm.count)
THEN
--dbms_output.put_line('YES');
upd_clause:=rtrim(upd_clause,' and');
col_list2:=rtrim( col_list2,',');
col_list6:=rtrim( col_list6,',');
END IF;
dbms_output.put_line(col_list2||col_list6);
--dbms_output.put_line(upd_clause);
End loop;
--Update caluse ends
cntr:=0; --Counter reset
--Prepare ON clause
FOR k IN 1..v_col_nm.count
loop
cntr:=cntr+1;
--dbms_output.put_line(v_col_nm.count || cntr);
on_clause := 't1.'||v_col_nm(k)||' = t2.' ||v_col_nm(k);
on_clause:=on_clause ||' and ' ;
col_list3:= 't1.'||v_col_nm(k) ||',';
col_list4:= col_list4||col_list3;
col_list7:= 't2.'||v_col_nm(k) ||',';
col_list8:= col_list8||col_list7;
IF (cntr = v_col_nm.count)
THEN
--dbms_output.put_line('YES');
on_clause:=rtrim(on_clause,' and');
col_list4:=rtrim( col_list4,',');
col_list8:=rtrim( col_list8,',');
end if;
dbms_output.put_line(col_list4||col_list8);
-- ON clause ends
--Prepare merge Statement
v_sql:= 'MERGE INTO '|| v_obj_nm(i)||' t1--put target schema name before v_obj_nm
USING (SELECT * FROM '|| v_obj_nm(i)||') t2-- put source schema name befire v_obj_nm here
ON ('||on_clause||')
WHEN MATCHED THEN
UPDATE
SET '||upd_clause ||
' WHEN NOT MATCHED THEN
INSERT
('||col_list2||','
||col_list4||
')
VALUES
('||col_list6||','
||col_list8||
')';
dbms_output.put_line(v_sql);
execute immediate v_sql;
end loop;
End loop;
END;
/
Execution:
exec COPY_TABLE
Output:
anonymous block completed
PS: i have tested this with a table with 2 columns out of which i was having unique key constraint .The DDL of table is as below:
At the end i wish you could understand my code(you being a noob) and implement something similar if the above fails for your requirement.
CREATE TABLE TEST
( COL2 NUMBER,
COLUMN1 VARCHAR2(20 BYTE),
CONSTRAINT TEST_UK1 UNIQUE (COLUMN1)
) ;
Oh dear! Normally, such a question would be quickly closed as "too broad", but we need to support victims of evil advisors!
As for the effort, I would need a week full time for an experienced expert plus two days quality checking for an experierenced QA engineer.
First of all, there is no way that such a complex data merge will work on the first try. That means that you'll need test copies of both schemas that can be easily rebuild. And you'll need a place to try it out. Normally this is done with an export of both schemas and an empty dev database.
Next, you need both schemas close enough to be able to compare the data. I'd do it with an import of the export files mentione above. If the schema names are identical than rename one during import.
Next, I'd doublecheck if the structure is really identical, with queries like
SELECT a.owner, a.table_name, b.owner, b.table_name
FROM all_tables a
FULL JOIN all_tables b
ON a.table_name = b.table_name
AND a.owner = 'SCHEMAA'
AND b.owner = 'SCHEMAB'
WHERE a.owner IS NULL or b.owner IS NULL;
Next, I'd check if the primary and unique keys have overlaps:
SELECT id FROM schemaa.table1
INTERSECT
SELECT id FROM schemab.table1;
As there are 300+ tables, I'd generate those queries:
DECLARE
stmt VARCHAR2(30000);
n NUMBER;
schema_a CONSTANT VARCHAR2(128 BYTE) := 'SCHEMAA';
schema_b CONSTANT VARCHAR2(128 BYTE) := 'SCHEMAB';
BEGIN
FOR c IN (SELECT owner, constraint_name, table_name,
(SELECT LISTAGG(column_name,',') WITHIN GROUP (ORDER BY position)
FROM all_cons_columns c
WHERE s.owner = c.owner
AND s.constraint_name = c.constraint_name) AS cols
FROM all_constraints s
WHERE s.constraint_type IN ('P')
AND s.owner = schema_a)
LOOP
dbms_output.put_line('Checking pk '||c.constraint_name||' on table '||c.table_name);
stmt := 'SELECT count(*) FROM '||schema_a||'.'||c.table_name
||' JOIN '||schema_b||'.'||c.table_name
|| ' USING ('||c.cols||')';
--dbms_output.put_line('Query '||stmt);
EXECUTE IMMEDIATE stmt INTO n;
dbms_output.put_line('Found '||n||' overlapping primary keys in table '||c.table_name);
END LOOP;
END;
/
1st of all, for 350 tables, most probably, would need an dynamic SQL.
Declare a CURSOR or a COLLECTION - table of VARCHAR2 with all table names.
Declare a string variable to build the dynamic SQL.
loop through the entire list of the tables name and, for each table generates a string which will be executed as SQL with EXECUTE IMMEDIATE command.
The dynamic SQL which will be built, should insert the values from source table into the target table. In case the PK already exists, in target table, should be checked the field which represents the last updated date and if in source table it is bigger than in target table, then perform an update in target table, else, do nothing.
Is the below form safe against SQL injection? I have to do this because the p_select_statement would be a simple select * from table where lastModifiedTime > :p_asof and <conditions>, but the code that calls getByFilter expects the columns to be in the order col1, col2, col3 while select * may return col2, col3, col1
OPEN CURSOR <dynamic_select_statement> USING <bind variable>
Example...
PROCEDURE getByFilter (
p_select_statement IN VARCHAR2,
p_asof IN TIMESTAMP,
p_cur OUT SYS_REFCURSOR
) AS
full_select_statement VARCHAR2(32767);
BEGIN
full_select_statement := 'SELECT
col1,
col2,
col3
FROM (' || p_select_statement || ')';
OPEN p_cur FOR full_select_statement USING p_asof;
END getByFilter;
No, not if the conditions can be set dynamically:
VARIABLE cur REFCURSOR;
DECLARE
conditions VARCHAR2(4000)
:= '1 = 0 UNION ALL '
|| 'select username AS col1, password AS col2, other_column AS col3 '
|| 'FROM your_secret_password_table';
sql VARCHAR2(4000)
:= 'select * from table where lastModifiedTime > :p_asof and ' || conditions;
BEGIN
PROCEDURE getByFilter (
p_select_statement => sql,
p_asof => SYSTIMESTAMP,
p_cur => :curr
);
END;
/
PRINT cur;
Yes, your example is clearly an example of SQL injection. It counts as SQL injection if your procedure executes some input verbatim as a query (or part of a query, as in this case).
To answer your question about whether unsafe input could do damage, I can think of two possibilities:
The SELECT passed as an argument to the procedure could be designed to be very resource-intensive. For example a query with a large cartesian join:
SELECT * FROM AnyTable
CROSS JOIN AnyTable
CROSS JOIN AnyTable
CROSS JOIN AnyTable
CROSS JOIN AnyTable
ORDER BY 1;
Generates a result set of a few trillion rows, and attempts to sort it. That could be a denial-of-service attack.
The SELECT includes calling a stored function that modifies data.
SELECT SomeFunctionThatDeletes() FROM AnyTable;
Whether your procedure is unsafe depends on whether the calling code allows untrusted input to become the argument to your procedure.
Suppose you had some safeguard like this (written in PHP, but the same applies to any language):
$user_option = $_GET['option'];
$queries = [
1 => 'select * from table where lastModifiedTime > :p_asof and <conditions>',
2 => 'select * from table where lastModifiedTime < :p_asof and <conditions>'
];
$stmt = $db->prepare('call getByFilter(?)');
$stmt->execute([$queries[$user_option]);
Now the user can pick with an input of '1' or '2' which query to run, but they have to pick one or the other of these hard-coded queries. They can't do anything mischievous like change the SQL query that is run.
That's a type of whitelisting. In other words, your application code must limit the choices so that the user's input can pick only safe choices. The user input should never be allowed to be executed as SQL directly without some kind of whitelisting like this.
You could also implement the whitelisting inside your procedure.
I have two type of select statement with the same complicated where clause.
One - Returns transaction detailed data
select field_1, field_2, field_3, ... , field_30 from my_table where my_where_clause
Second - Returns transaction data grouped by(distinct) merchants
select distinct field_1, field_2, field_8 from my_table where my_where_clause
Statements are called separately.
I want to simplify my code and not to repeat this complicated where clause in both statements without loosing performance
In dynamic SQL it's possible but I don't want to use dynamic SQL.
Any suggestions?
Suggestion: you can try GROUPING SETS expression.
It allows you to selectively specify the set of groups that you want to create within a GROUP BY clause. In
In your case, you can specify 2 sets, one group by set for all fields from 1 to 30 and another set for fields 1,2&8.
Link- https://docs.oracle.com/cd/E40518_01/server.761/es_eql/src/reql_aggregation_grouping_sets.html
However, it will return the output of both the groups in a single resultset, not sure if this fits in your design.
So you could encapsulate this statement, in a view or function, e,g,:
create or replace view view_1 as
select field_1, field_2, field_3, ... , field_30
from my_table
where my_where_clause
Then your second query could be
select distinct * from view_1;
You said that you are using this query from java. Try this.
create or replace function get_cursor(p_type varchar2 default null/* other paramethers*/ ) return sys_refcursor
is
result_curosr sys_refcursor;
begin
open result_curosr for 'select '||p_type||' object_type,status from user_objects' /* where clausele */ ;
return result_curosr;
end;
And usage of this from java.
Connection con = ...
CallableStatement callableStatement = con.prepareCall("declare c sys_refcursor; begin ? := get_cursor(?); end ; ");
callableStatement.registerOutParameter(1, OracleTypes.CURSOR);
callableStatement.setString(2, "Distinct"); // for distinct
or
callableStatement.setNull(2, OracleTypes.VARCHAR); // for full results
callableStatement.executeUpdate();
ResultSet rs = (ResultSet) callableStatement.getObject(1);
while(rs.next()) {
System.err.println(rs.getString(1));
}
rs.close();
con.close();
Other solution.
Add one more parameter and do simple deduplication using all columns from query. But i don't see any advantages.
select object_type,status from
(select object_type,status, row_number() over( partition by object_type,status order by 1) rn from user_objects /* your_where_clusue */
) where rn = case when 'DISTIINCT'/* <- paramete here :isDistinct */ = 'DISTIINCT' then 1 else rn end;
You can make dynamic SQL more readable by using multi-line strings, alternative quotes, and templates.
declare
v_select varchar2(32767);
v_where varchar2(32767);
v_code varchar2(32767) := '
##SELECT##
##WHERE##
';
begin
--Populate the clauses.
if ... then
v_select := 'select field_1, field_2, field_3, ... , field_30 from my_table';
else
v_select := 'select distinct field_1, field_2, field_8 from my_table';
end if;
if ... then
v_where :=
q'[
where field_1 = 'foo'
and field_2 = :bind1
...
]';
else
v_where :=
q'[
where field_2 = 'bar'
and field_2 = :bind2
...
]';
end if;
--Fill in the code.
v_code := replace(v_code, '##SELECT##', v_select);
v_code := replace(v_code, '##WHERE##', v_where);
--Print the code to check the formatting. Remove after testing.
dbms_output.put_line(v_code);
--Run it.
execute immediate v_code using ...;
end;
/
It's not perfect but it prevents ugly concatenation. And it's much better than the anti-patterns needed to avoid dynamic SQL at all costs. In most languages features like polymorphism and reflection are better than dynamic code. PL/SQL does not have good support for those advanced features so it's usually better to build the code as a string.
Here is the package created by passing 3 input parameters to function
CREATE OR replace PACKAGE "PKG_CAMPAIGN_EMAIL_QTY"
AS
FUNCTION Getcampaignoutgoingemailqty(
tablename IN VARCHAR2,
ActivatedDate IN DATE,
CompletedDate IN DATE)
RETURN NUMBER;
END pkg_campaign_email_qty;
/
Here is the query to get the count
SELECT
(
pkg_campaign_email_qty.Getcampaignoutgoingemailqty(
9142632263013677974,
To_date('20/10/2015', 'DD/MM/YYYY'),
To_date('30/11/2015', 'DD/MM/YYYY')
)
) AS
email
FROM dual;
Getting ORA-00933: SQL command not properly ended in oracle
here is the package body
CREATE OR REPLACE PACKAGE BODY "PKG_CAMPAIGN_EMAIL_QTY" as
FUNCTION getCampaignOutgoingEmailQty(tableName IN VARCHAR2 ,ActivatedDate DATE,CompletedDate DATE) RETURN NUMBER IS
OutgoingEmailQuantity NUMBER;
begin
EXECUTE IMMEDIATE 'select NVL(COUNT(1),0) from campaign_'||tableName||'
join flat_interactions out_email on campaign_'||tableName||'.fullname=out_email.o_parent_id and out_email.N9135700037713613964=9135706250013621563 and out_email.D9135699928113613119 between TO_DATE(''ActivatedDate'',''MM/dd/YYYY'') and TO_DATE(''CompletedDate'',''MM/dd/YYYY'')' INTO OutgoingEmailQuantity;
RETURN OutgoingEmailQuantity ;
EXCEPTION
WHEN OTHERS THEN
RETURN 0;
end getCampaignOutgoingEmailQty;
end PKG_CAMPAIGN_EMAIL_QTY;
/
I don't know if this is the cause of the error you're seeing, but there's an issue with the function code - if you look inside the execute immediate, you've got
'<snip> and out_email.D9135699928113613119 between TO_DATE(''ActivatedDate'',''MM/dd/YYYY'') and TO_DATE(''CompletedDate'',''MM/dd/YYYY'')'
This means that, passing those parameters in, you'll end up trying to execute a sql statement of:
select NVL(COUNT(1),0)
from campaign_9142632263013677974
join flat_interactions out_email on campaign_9142632263013677974.fullname = out_email.o_parent_id
and out_email.N9135700037713613964 = 9135706250013621563
and out_email.D9135699928113613119 between TO_DATE('ActivatedDate','MM/dd/YYYY')
and TO_DATE('CompletedDate','MM/dd/YYYY');
So, you're trying to convert the strings "ActivatedDate" and "CompletedDate" into dates when clearly they're not dates.
Rather than do that, I would use bind variables, something like:
create or replace package body pkg_campaign_email_qty
as
function getcampaignoutgoingemailqty(tablename in varchar2 ,activateddate date,completeddate date)
return number
is
outgoingemailquantity number;
begin
execute immediate 'select NVL(COUNT(1),0)'||chr(10)||
'from campaign_'||tablename||chr(10)||
' join flat_interactions out_email on campaign_'||tablename||'.fullname=out_email.o_parent_id'||chr(10)||
' and out_email.N9135700037713613964=9135706250013621563'||chr(10)||
' and out_email.D9135699928113613119 between :ActivatedDate and :CompletedDate' into outgoingemailquantity using activateddate, completeddate;
return outgoingemailquantity;
exception
when others then
return 0;
end getcampaignoutgoingemailqty;
end pkg_campaign_email_qty;
/
N.B. untested, since you didn't provide any table definitions.
As an aside, with your calling query, the outer brackets are unnecessary and I would remove them, so your query would become:
SELECT pkg_campaign_email_qty.Getcampaignoutgoingemailqty(9142632263013677974,
To_date('20/10/2015', 'DD/MM/YYYY'),
To_date('30/11/2015', 'DD/MM/YYYY')) email
FROM dual;
You are using wrong the dates in the dynamic sql.
I will use binds for that:
EXECUTE IMMEDIATE
'select NVL(COUNT(1),0)
from campaign_'||tableName||'
join flat_interactions out_email on
campaign_'||tableName||'.fullname=out_email.o_parent_id and
out_email.N9135700037713613964=9135706250013621563 and
out_email.D9135699928113613119 between :ActivatedDate and :CompletedDate'
INTO OutgoingEmailQuantity
USING ActivatedDate, CompletedDate;
What you want to do is harder to follow:
EXECUTE IMMEDIATE
'select NVL(COUNT(1),0)
from campaign_'||tableName||'
join flat_interactions out_email on
campaign_'||tableName||'.fullname=out_email.o_parent_id and
out_email.N9135700037713613964=9135706250013621563 and
out_email.D9135699928113613119 between
TO_DATE('||to_char(ActivatedDate,'MM/dd/YYYY')||',''MM/dd/YYYY'')
and
TO_DATE('||to_char(CompletedDate,'MM/dd/YYYY')||',''MM/dd/YYYY'')'
INTO OutgoingEmailQuantity;
SELECT
(
pkg_campaign_email_qty.Getcampaignoutgoingemailqty(9142632263013677974, To_date('20/10/2015', 'DD/MM/YYYY'), To_date('30/11/2015', 'DD/MM/YYYY')) ) AS
email
FROM dual
); <- missing