Everything below I am able to do in separate operations, but as I am relatively new to XQuery I am struggling to work out how to perform multiple operations in one go which would be neat.
I am trying to update an XML column, with some xml data from another column (this XML has come from a spreadsheet with two columns, promotion numbers and the department numbers within each promotion). Which I load into a table and then run the below against.
INSERT INTO proms
select promid, '<Promotion><MultibuyGroup><MMGroupID>'||depts||'</MMGroupID>
</MultibuyGroup></Promotion>' DEPTS
from (
SELECT promid, listagg (id,'</MMGroupID><MMGroupID>') within GROUP
(ORDER BY id) as depts FROM mmgroups
GROUP BY promid
);
Creating a table with a column for PROMID and an XML Column like the below (just one MMGroup in example for ease of read.
<Promotion><MultibuyGroup><MMGroupID>1</MMGroupID></Promotion></MultibuyGroup>
When I run the below, I can successfully update any XML in the PROMOTIONS where the value of ID column matches the value of PROMID in the table I have created above.
merge into PROMOTIONS tgt
using (
select PROMID
, xmlquery('/Promotion/MultibuyGroup/MMGroupID'
passing xmlparse(document DEPTS)
returning content
) as new_mmg
from PROMS WHERE PROMID = 'EMP35Level1'
) src
on (tgt.ID = src.PROMID)
when matched then update
set tgt.xml =
xmlserialize(document
xmlquery(
'copy $d := .
modify
insert node $new_mmg as last into $d/Promotion/MultibuyGroup
return $d'
passing xmlparse(document tgt.xml)
, src.new_mmg as "new_mmg"
returning content
)
no indent
) ;
However what I would like my query to do is to delete any existing MMGroupID nodes from the target xml (if they exist), then replace them with all of the nodes from source xml.
Also within the target xml is a LastUpdated Node which I would like to update with the SYSDATE at time of update
Also two separate columns LAST_UPDATED DATE and ROW_UPDATED NUMBER (20,0) which has the epoch time of update would be nice to update at the same time.
<Promotion>
<LastUpdated>2018-08-23T14:56:35+01:00</LastUpdated>
<MajorVersion>1</MajorVersion>
<MinorVersion>52</MinorVersion>
<PromotionID>EMP35Level1</PromotionID>
<Description enabled="1">Staff Discount 15%</Description>
<MultibuyGroup>
<AlertThresholdValue>0.0</AlertThresholdValue>
<AlertThresholdValue currency="EUR">0.0</AlertThresholdValue>
<UseFixedValueInBestDeal>0</UseFixedValueInBestDeal>
<UpperThresholdValue>0.0</UpperThresholdValue>
<UpperThresholdValue currency="EUR">0.0</UpperThresholdValue>
<GroupDescription>Employee Discount 15%</GroupDescription>
<Rolling>0</Rolling>
<DisableOnItemDiscount>1</DisableOnItemDiscount>
<UniqueItems>0</UniqueItems>
<AllItems>0</AllItems>
<RoundingRule>3</RoundingRule>
<UseLowestNetValue>0</UseLowestNetValue>
<TriggerOnLostSales>0</TriggerOnLostSales>
<MMGroupID>2</MMGroupID>
<MMGroupID>8</MMGroupID>
<MMGroupID>994</MMGroupID>
</MultibuyGroup>
<Timetable>
<XMLSchemaVersion>1</XMLSchemaVersion>
<CriterionID/>
<StartDate>1970-01-01T00:00:00+00:00</StartDate>
<FinishDate>2069-12-31T00:00:00+00:00</FinishDate>
</Timetable>
<AllowedForEmployeeSale>1</AllowedForEmployeeSale>
<Notes enabled="1"/>
<AlertMessage enabled="1"/>
</Promotion>
Since posting, have edited the query to be:
merge INTO PROMOTIONS3 tgt
using (
SELECT PROMID
,xmlquery('/Promotion/MultibuyGroup/MMGroupID'
passing xmlparse(document DEPTS)
returning content
) as new_mmg
FROM PROMS WHERE PROMID = 'EMP35Level1'
) src
ON (tgt.ID = src.PROMID)
when matched then update
SET tgt.xml =
xmlserialize(document
xmlquery(
'copy $d := .
modify(
delete nodes $d/Promotion/MultibuyGroup/MMGroupID,
insert node $new_mmg as last into $d/Promotion/MultibuyGroup,
replace value of node $d/Promotion/LastUpdated with current-dateTime())
return $d'
passing xmlparse(document tgt.xml)
,src.new_mmg as "new_mmg"
returning content
)
no indent
)
,last_updated = (SELECT SYSDATE FROM dual)
,row_updated = (SELECT ( SYSDATE - To_date('01-01-1970 00:00:00','DD-MM-YYYY HH24:MI:SS') ) * 24 * 60 * 60 * 1000 FROM dual) ;
So almost correct, except I need
<LastUpdated>2018-08-23T14:56:35+01:00</LastUpdated>
Not
<LastUpdated>2018-11-09T11:53:10.591000+00:00</LastUpdated>
So I need to figure that out.
Cheers.
For syntax you should google for XQuery Update Facility.
An example
xmlquery(
'copy $d := .
modify(
delete nodes $d/Promotion/MMGroupID,
replace value of node $d/Promotion/LastUpdated with current-date(),
insert node <node1>x</node1> as last into $d/Promotion/MultibuyGroup,
insert node <node2>x</node2> as last into $d/Promotion/MultibuyGroup)
return $d
'
Related
e.g, below query:
create table t1 ( age int, name varchar(10) )
insert into t1 values(1, 'name'),(2, 'surname')
copy select * from t1 into 't1.dat' DELIMITERS '|','
','"' null as '';
The copy select cmd returns -1 as the affected row count, although it should return 2 as the value. Not sure why this is so. At many other times, I have seen the same query returning correctly the affected row count.
If I run the same query in the Dbeaver tool Iam using, I see this:
Updated Rows: -1
Query: copy select * from t1 into 't1.dat' DELIMITERS '|','
','"' null as ''
Finish time: Sat Apr 30 16:53:28 IST 2022
I think you have to use an absolute path to the export file, e.g.
copy select * from t1 into '/home/user/t1.dat' DELIMITERS '|','
','"' null as '';
When you see a message like 2 affected rows, it might actually be due to the result summary of the successfully completed INSERT INTO statement. But that doesn't mean the COPY INTO <file> statement is successful though.
I've tried to find an answer on several forums with no luck, so perhaps you can help me out.
I've got an INSERT ALL request that inserts thousands of rows at once.
INSERT ALL
INTO my_table (field_x, field_y, field_z) VALUES ('value_x1', 'value_y1', 'value_z1')
INTO my_table (field_x, field_y, field_z) VALUES ('value_x2', 'value_y2', 'value_z2')
...
INTO my_table (field_x, field_y, field_z) VALUES ('value_xn', 'value_yn', 'value_zn')
SELECT * FROM DUAL;
Now I'd like to amend it to update rows when some criteria are met. For each row, I could have something like:
MERGE INTO my_table m
USING (SELECT 'value_xi' x, 'value_yi' y, 'value_zi' z FROM DUAL) s
ON (m.field_x = s.x and m.field_y = s.y)
WHEN MATCHED THEN UPDATE SET
field_z = s.z,
WHEN NOT MATCHED THE INSERT (field_x, field_y, field_z)
VALUE(s.x, s.y, s.z);
Is there a way for me to do a kind of "MERGE ALL" that would allow to have all those merge requests in one?
Or maybe I'm missing the point and there's a better way to do this?
Thanks,
Edit: One possible solution is to use "UNION ALL" for a set of selects from dual, as follows:
MERGE INTO my_table m
USING (
select '' as x, '' as y, '' as z from dual
union all select 'value_x1', 'value_y1', 'value_z1' from dual
union all select 'value_x2', 'value_y2', 'value_z2' from dual
[...]
union all select 'value_xn', 'value_yn', 'value_zn' from dual
) s
ON (m.field_x = s.x and m.field_y = s.y)
WHEN MATCHED THEN UPDATE SET
field_z = s.z,
WHEN NOT MATCHED THEN INSERT (field_x, field_y, field_z)
VALUES (s.x, s.y, s.z);
NB: I've used a first empty row to be able generate all rows in the same format when I write the request. I also specify the columns names there.
Another solution would be to create a temporary table, INSERT ALL data into it, then merge with the target table and delete the temporary table.
If you're passing in tens of thousands of rows from your python script, I would do:
Create a global temporary table (GTT - this is a permanent table that holds data at session level)
Get your python script to insert the rows into the GTT
Use the GTT in the Merge statement, e.g.:
merge into your_main_table tgt
using your_gtt src
on (<join conditions>)
when matched then
update ...
when not matched then
insert ...;
Usually the best solution to capture history would be to create a trigger which takes snapshot of the record while updating into a history table however :
1.) My table contains 60 columns but I want to capture history for only 10 of them.
2.) The data from source come's with a Submit date which is not captured in the target table. The history need's to be captured based on that Submit date sent by the source and not based on current sysdate.
3.) We have control over the Select, Update process.
Proposed Solution:
We created a function as below
In(Primary Key, Submit_Date, Column_Name, Old Value, New Value)
PRAGMA AUTONOMOUS_TRANSACTION;
Fetch max(Submit_date) From History using PK;
If max_Submit_date is Null:
Insert in History using PK, 01-01-1700 as Submit date, Column Name, Old_Value;
Insert into History using PK & Submit_date & new Value;
Elsif max_Submit_date = Submit_date
Update History using PK & Submit_date with new Value;
Elsif max_Submit_date < Submit_date
Insert into History using PK & Submit_date & new Value;
End if;
commit;
While selecting data for update we added
select .... ,
DECODE(T.Column_VALUE_1,S.Column_VALUE_1,NULL,Function(PK,'COLUMN_NAME_1', S.Submit_Date, T.Column_VALUE_1, S.Column_VALUE_1)) XYZ,
DECODE(T.Column_VALUE_2,S.Column_VALUE_2,NULL,Function(PK,'COLUMN_NAME_2', S.Submit_Date, T.Column_VALUE_2, S.Column_VALUE_2)) XYZ,
From Source_Table S Join Target Table T Where ...
I can see that the solution is not ideal or efficient. Please advise if the requirement can be met in any other way.
One solution could be to perform it for all records, at once, utilizing a MERGE INTO statement. Since your submit_date is part of insert and also comparison, ideal solution to avoid conflict would be to have a version column in your History table, containing 1 for initial version and 2,3,4.. and so on for subsequent versions.
And instead of comparing with decode and calling function, you could exclude all such records in where clause condition.
MERGE INTO HISTORY h USING
(
with m(pk,max_Submit_date) AS
( select pk,submit_date,version
FROM HISTORY o where
version IN ( select MAX(version) max_version
FROM HISTORY i where i.pk = o.pk ) ,
select .... PK,
CASE WHEN m.max_Submit_date is Null
THEN 01-01-1700
WHEN m.max_Submit_date <= s.Submit_date THEN
s.Submit_date
END as Submit_date,
decode( m.max_Submit_date,NULL,T.Column_VALUE_1, S.Column_VALUE_1)
decode( m.max_Submit_date,NULL,T.Column_VALUE_2, S.Column_VALUE_2)
..
CASE WHEN m.max_Submit_date is Null
THEN 1
WHEN m.max_Submit_date <= s.Submit_date
THEN m.max_version + 1 new_version
ELSE
m.max_version
END version
From Source_Table S Join Target Table T
JOIN m ON m.pk = s.pk
Where ..
( T.Column_VALUE_1 != S.Column_VALUE_1 ) OR
( T.Column_VALUE_2 != S.Column_VALUE_2 ) OR
..
..
) cur
ON ( cur.pk = h.pk and h.version = cur.version )
WHEN MATCHED THEN
Update SET h.Submit_date = cur.Submit_date,
h.Column_VALUE_1 = s.Column_VALUE_1,
h.Column_VALUE_2 = s.Column_VALUE_2
..
WHEN NOT MATCHED THEN INSERT
INSERT (PK,submit_date,column_1,..) VALUES (..);
A weird request maybe but. My boss wants me to create an admin version of a page we have that displays data from an oracle query in a table.
The admin page, instead of displaying the data (query returns 1 row), needs to return the table name and column name
Ex: Instead of:
Name Initial
==================
Bob A
I want:
Name Initial
============================
Users.FirstName Users.MiddleInitial
I realize I can do this in code but would rather just modify the query to return the data I want so I can leave the report generation code mostly alone.
I don't want to do it in a stored procedure.
So when I spit out the data in the report using something like:
blah blah = MyDataRow("FirstName")
I can leave that as is but instead of it displaying "BOB" it would display "Users.FirstName"
And I want to do the query using select * if possible instead of listing all the columns
So for each of the columns I am querying in the * , I want to get (instead of the column value) the tablename.ColumnName or tablename|columnName
hope you are following- I am confusing myself...
pseudo:
select tablename + '.' + Columnname as WhateverTheColumnNameIs
from Table1
left join Table2 on whatever...
Join Table_Names on blah blah
Whew- after writing all this I think I will just do it on the code side.
But if you are up for it maybe a fun challenge
Oracle does not provide an authentic way(there is no pseudocolumn) to get the column name of a table as a result of a query against that table. But you might consider these two approaches:
Extract column name from an xmltype, formed by passing cursor expression(your query) in the xmltable() function:
-- your table
with t1(first_name, middle_name) as(
select 1,2 from dual
), -- your query
t2 as(
select * -- col1 as "t1.col1"
--, col2 as "t1.col2"
--, col3 as "t1.col3"
from hr.t1
)
select *
from ( select q.object_value.getrootelement() as col_name
, rownum as rn
from xmltable('//*'
passing xmltype(cursor(select * from t2 where rownum = 1))
) q
where q.object_value.getrootelement() not in ('ROWSET', 'ROW')
)
pivot(
max(col_name) for rn in (1 as "name", 2 as "initial")
)
Result:
name initial
--------------- ---------------
FIRST_NAME MIDDLE_NAME
Note: In order for column names to be prefixed with table name, you need to list them
explicitly in the select list of a query and supply an alias, manually.
PL/SQL approach. Starting from Oracle 11g you could use dbms_sql() package and describe_columns() procedure specifically to get the name of columns in the cursor(your select).
This might be what you are looking for, try selecting from system views USER_TAB_COLS or ALL_TAB_COLS.
This question already has answers here:
SQL IN Clause 1000 item limit
(5 answers)
Closed 8 years ago.
Is there any way to get around the Oracle 10g limitation of 1000 items in a static IN clause? I have a comma delimited list of many of IDs that I want to use in an IN clause, Sometimes this list can exceed 1000 items, at which point Oracle throws an error. The query is similar to this...
select * from table1 where ID in (1,2,3,4,...,1001,1002,...)
Put the values in a temporary table and then do a select where id in (select id from temptable)
select column_X, ... from my_table
where ('magic', column_X ) in (
('magic', 1),
('magic', 2),
('magic', 3),
('magic', 4),
...
('magic', 99999)
) ...
I am almost sure you can split values across multiple INs using OR:
select * from table1 where ID in (1,2,3,4,...,1000) or
ID in (1001,1002,...,2000)
You may try to use the following form:
select * from table1 where ID in (1,2,3,4,...,1000)
union all
select * from table1 where ID in (1001,1002,...)
Where do you get the list of ids from in the first place? Since they are IDs in your database, did they come from some previous query?
When I have seen this in the past it has been because:-
a reference table is missing and the correct way would be to add the new table, put an attribute on that table and join to it
a list of ids is extracted from the database, and then used in a subsequent SQL statement (perhaps later or on another server or whatever). In this case, the answer is to never extract it from the database. Either store in a temporary table or just write one query.
I think there may be better ways to rework this code that just getting this SQL statement to work. If you provide more details you might get some ideas.
Use ...from table(... :
create or replace type numbertype
as object
(nr number(20,10) )
/
create or replace type number_table
as table of numbertype
/
create or replace procedure tableselect
( p_numbers in number_table
, p_ref_result out sys_refcursor)
is
begin
open p_ref_result for
select *
from employees , (select /*+ cardinality(tab 10) */ tab.nr from table(p_numbers) tab) tbnrs
where id = tbnrs.nr;
end;
/
This is one of the rare cases where you need a hint, else Oracle will not use the index on column id. One of the advantages of this approach is that Oracle doesn't need to hard parse the query again and again. Using a temporary table is most of the times slower.
edit 1 simplified the procedure (thanks to jimmyorr) + example
create or replace procedure tableselect
( p_numbers in number_table
, p_ref_result out sys_refcursor)
is
begin
open p_ref_result for
select /*+ cardinality(tab 10) */ emp.*
from employees emp
, table(p_numbers) tab
where tab.nr = id;
end;
/
Example:
set serveroutput on
create table employees ( id number(10),name varchar2(100));
insert into employees values (3,'Raymond');
insert into employees values (4,'Hans');
commit;
declare
l_number number_table := number_table();
l_sys_refcursor sys_refcursor;
l_employee employees%rowtype;
begin
l_number.extend;
l_number(1) := numbertype(3);
l_number.extend;
l_number(2) := numbertype(4);
tableselect(l_number, l_sys_refcursor);
loop
fetch l_sys_refcursor into l_employee;
exit when l_sys_refcursor%notfound;
dbms_output.put_line(l_employee.name);
end loop;
close l_sys_refcursor;
end;
/
This will output:
Raymond
Hans
I wound up here looking for a solution as well.
Depending on the high-end number of items you need to query against, and assuming your items are unique, you could split your query into batches queries of 1000 items, and combine the results on your end instead (pseudocode here):
//remove dupes
items = items.RemoveDuplicates();
//how to break the items into 1000 item batches
batches = new batch list;
batch = new batch;
for (int i = 0; i < items.Count; i++)
{
if (batch.Count == 1000)
{
batches.Add(batch);
batch.Clear()
}
batch.Add(items[i]);
if (i == items.Count - 1)
{
//add the final batch (it has < 1000 items).
batches.Add(batch);
}
}
// now go query the db for each batch
results = new results;
foreach(batch in batches)
{
results.Add(query(batch));
}
This may be a good trade-off in the scenario where you don't typically have over 1000 items - as having over 1000 items would be your "high end" edge-case scenario. For example, in the event that you have 1500 items, two queries of (1000, 500) wouldn't be so bad. This also assumes that each query isn't particularly expensive in of its own right.
This wouldn't be appropriate if your typical number of expected items got to be much larger - say, in the 100000 range - requiring 100 queries. If so, then you should probably look more seriously into using the global temporary tables solution provided above as the most "correct" solution. Furthermore, if your items are not unique, you would need to resolve duplicate results in your batches as well.
Yes, very weird situation for oracle.
if you specify 2000 ids inside the IN clause, it will fail.
this fails:
select ...
where id in (1,2,....2000)
but if you simply put the 2000 ids in another table (temp table for example), it will works
below query:
select ...
where id in (select userId
from temptable_with_2000_ids )
what you can do, actually could split the records into a lot of 1000 records and execute them group by group.
Here is some Perl code that tries to work around the limit by creating an inline view and then selecting from it. The statement text is compressed by using rows of twelve items each instead of selecting each item from DUAL individually, then uncompressed by unioning together all columns. UNION or UNION ALL in decompression should make no difference here as it all goes inside an IN which will impose uniqueness before joining against it anyway, but in the compression, UNION ALL is used to prevent a lot of unnecessary comparing. As the data I'm filtering on are all whole numbers, quoting is not an issue.
#
# generate the innards of an IN expression with more than a thousand items
#
use English '-no_match_vars';
sub big_IN_list{
#_ < 13 and return join ', ',#_;
my $padding_required = (12 - (#_ % 12)) % 12;
# get first dozen and make length of #_ an even multiple of 12
my ($a,$b,$c,$d,$e,$f,$g,$h,$i,$j,$k,$l) = splice #_,0,12, ( ('NULL') x $padding_required );
my #dozens;
local $LIST_SEPARATOR = ', '; # how to join elements within each dozen
while(#_){
push #dozens, "SELECT #{[ splice #_,0,12 ]} FROM DUAL"
};
$LIST_SEPARATOR = "\n union all\n "; # how to join #dozens
return <<"EXP";
WITH t AS (
select $a A, $b B, $c C, $d D, $e E, $f F, $g G, $h H, $i I, $j J, $k K, $l L FROM DUAL
union all
#dozens
)
select A from t union select B from t union select C from t union
select D from t union select E from t union select F from t union
select G from t union select H from t union select I from t union
select J from t union select K from t union select L from t
EXP
}
One would use that like so:
my $bases_list_expr = big_IN_list(list_your_bases());
$dbh->do(<<"UPDATE");
update bases_table set belong_to = 'us'
where id in ($bases_list_expr)
UPDATE
Instead of using IN clause, can you try using JOIN with the other table, which is fetching the id. that way we don't need to worry about limit. just a thought from my side.
Instead of SELECT * FROM table1 WHERE ID IN (1,2,3,4,...,1000);
Use this :
SELECT * FROM table1 WHERE ID IN (SELECT rownum AS ID FROM dual connect BY level <= 1000);
*Note that you need to be sure the ID does not refer any other foreign IDS if this is a dependency. To ensure only existing ids are available then :
SELECT * FROM table1 WHERE ID IN (SELECT distinct(ID) FROM tablewhereidsareavailable);
Cheers