Translate hierarchical Oracle query to DB2 query - oracle

I work primarily with SAS and Oracle and am still new to DB2. Im faced with needing a hierarchical query to separate a clob into chunks that can be pulled into sas. SAS has a limit of 32K for character variables so I cant just pull the dataset in normally.
I found an old stackoverflow question about the best way to pull a clob into a sas data set but it is written in Oracle.
Import blob through SAS from ORACLE DB
Since I am new to DB2 and the syntax for this type of join seems very different I was hoping to find someone that could help convert it and explain the syntax. I find the Oracle syntax to be much easier to understand. I'm not sure in DB2 if you would use a CTE recursion like this https://www.ibm.com/support/knowledgecenter/en/SSEPEK_10.0.0/apsg/src/tpc/db2z_xmprecursivecte.html or if you would use hierarchical queries like this https://www.ibm.com/support/knowledgecenter/en/ssw_ibm_i_71/sqlp/rbafyrecursivequeries.htm
Here is the Oracle query.
SELECT
id
, level as chunk_id
, regexp_substr(clob_value, '.{1,32767}', 1, level, 'n') as clob_chunk
FROM (
SELECT id, clob_value
FROM schema.table
WHERE id = 1
)
CONNECT BY LEVEL <= regexp_count(clob_value, '.{1,32767}',1,'n')
order by id, chunk_id;
The table has two fields the id and the clob_value and would look like this.
ID CLOB_VALUE
1 really large clob
2 medium clob
3 another large clob
The thought is I would want this result. I would only ever be doing this one row at a time where id= which ever row I am processing.
ID CHUNK_ID CLOB
1 1 clob_chunk1of3
1 2 clob_chunk2of3
1 3 clob_chunk3of3
Thanks for any time spent reading and helping.

Here is a solution that should work in DB2 with few changes (but please be advised that I don't know DB2 at all; I am just using Oracle features that are in the SQL Standard, so they should be implemented identically - or almost so - in DB2).
Below I create a table with your sample data; then I show how to chunk it into substrings of length at most 8 characters. Although the strings are short, I defined the column as CLOB and I am using CLOB tools; this should work on much larger CLOBs.
You can make both the chunk size and the id into bind parameters, if needed. In my demo below I hardcoded the chunk size and I show the result for all IDs in the table. In case the CLOB is NULL, I do return one chunk (which is NULL, of course).
Note that touching CLOBs in a query is very expensive; so most of the work is done without touching the CLOBs. I only work on them as little as possible.
PREP WORK
drop table tbl purge; -- If needed
create table tbl (id number, clob_value clob);
insert into tbl (id, clob_value)
select 1, 'really large clob' from dual union all
select 2, 'medium clob' from dual union all
select 3, 'another large clob' from dual union all
select 4, null from dual -- added to check handling
;
commit;
QUERY
with
prep(id, len) as (
select id, dbms_lob.getlength(clob_value)
from tbl
)
, rec(id, len, ord, pos) as (
select id, len, 1, 1
from prep
union all
select id, len, ord + 1, pos + 8
from rec
where len >= pos + 8
)
select id, ord, dbms_lob.substr(clob_value, 8, pos)
from tbl inner join rec using (id)
order by id, ord
;
ID ORD CHUNK
---- ---- --------
1 1 really l
1 2 arge clo
1 3 b
2 1 medium c
2 2 lob
3 1 another
3 2 large cl
3 3 ob
4 1

Another option is to enable the Oracle compatibility in Db2 and just issue the hierarchical query.
This GitHub repository has background information on SQL recursion in DB2, including the Oracle-style syntax and a side by side example (both work against the Db2 sample database):
-- both queries are against the SAMPLE database
-- and should return the same result
SELECT LEVEL, CAST(SPACE((LEVEL - 1) * 4) || '/' || DEPTNAME
AS VARCHAR(40)) AS DEPTNAME
FROM DEPARTMENT
START WITH DEPTNO = 'A00'
CONNECT BY NOCYCLE PRIOR DEPTNO = ADMRDEPT;
WITH tdep(level, deptname, deptno) as (
SELECT 1, CAST( DEPTNAME AS VARCHAR(40)) AS DEPTNAME, deptno
FROM department
WHERE DEPTNO = 'A00'
UNION ALL
SELECT t.LEVEL+1, CAST(SPACE(t.LEVEL * 4) || '/' || d.DEPTNAME
AS VARCHAR(40)) AS DEPTNAME, d.deptno
FROM DEPARTMENT d, tdep t
WHERE d.admrdept=t.deptno and d.deptno<>'A00')
SELECT level, deptname
FROM tdep;

Related

Oracle query to concat a column data having two rows into one

I have an Oracle column with "Long" datatype, as the column has the 4000 chars limit, we tend to insert data into two rows or more than that, now we want to display 2 rows of data into one, I tried but there are some of the other error like char to long and so on, could you please help.
Id, Series, Serialized
1 1 abc
1 1 def
3 2 gdf
Id, Series
1 1
1 1
3 2
Output
Id, Series, Serialized
1 1 abcdef
3 2 gdf
Here's the table scripts
CREATE TABLE "LOGMESSAGE"
( "ID" VARCHAR2(36 BYTE) NOT NULL ENABLE,
"SERIALIZEDMESSAGE" LONG,
"SERIES" NUMBER(10,0)
)
CREATE TABLE "LOGENTRY"
( "ID" VARCHAR2(36 BYTE) NOT NULL ENABLE
)
insert into "LOGMESSAGE" values(
'd8dcd593-af52-425a-8bf2-d93f78a601c6','{"name":"miogetadapter001.xml","type":"miogetadapterresponse","id":"56e125af-4202-44cb-bf90-58df03776793","time":"2021-10-05T06:27:04.8595987Z","source":"miochannelsource","adapter":"miotestadapter","channel":"miotestchannel","original":"<RESTTestResponse TestTimestamp=\"2021/10/05 11:57:05\">\r\n <Name>miotestadapter</Name>\r\n <IsActive>true</IsActive>\r\n <Description>Added during MIO install for confirmation test. Version=3.0.0.0</Description>\r\n <type>ChannelAdapterConfiguration</type>\r\n <AdapterConfigurationTypeName>FileAdapterConfiguration</AdapterConfigurationTypeName>\r\n <AdapterType>FileAdapter</AdapterType>\r\n <AdapterVersion>1</AdapterVersion>\r\n <AdapterKind>default</AdapterKind>\r\n <ContentFormat>Xml</ContentFormat>\r\n <ChannelSourceName>miochannelsource</ChannelSourceName>\r\n <MessageChannelName>miotestchannel</MessageChannelName>\r\n <MomConnectionName></MomConnectionName>\r\n <OutboundNameFormat>{messagename}</OutboundNameFormat>\r\n <InboundNameFormat>{messagename}</InboundNameFormat>\r\n <DoNotSend>false</DoNotSend>\r\n <DoNotSendOutbound>false</DoNotSendOutbound>\r\n <ShutdownMaxTime>15</ShutdownMaxTime>\r\n <IsXml>true</IsXml>\r\n <WakeUpInterval>-1</WakeUpInterval>\r\n <UserName></UserName>\r\n <Password></Password>\r\n <DomainName></DomainName>\r\n <VerifyUnique>true</VerifyUnique>\r\n <UniqueIncludesTimestamp>true</UniqueIncludesTimestamp>\r\n <UniqueCacheExpiration>0</UniqueCacheExpiration>\r\n <ClearOriginalContents>true</ClearOriginalContents>\r\n <BufferSettings>\r\n <Description></Description>\r\n <Kind>Persistent</Kind>\r\n <ConnectionName></ConnectionName>\r\n <ConnectionString>folder name=%AppData%</ConnectionString>\r\n <Interval>2000</Interval>\r\n <Expiration>-1</Expiration>\r\n <MaxCount>-1</MaxCount>\r\n </BufferSettings>\r\n <InProcessExpiration>120</InProcessExpiration>\r\n <InboundFilters />\r\n <OutboundFilters />\r\n <OutboundFailPlugin>LogAndDiscard</OutboundFailPlugin>\r\n <OutboundFailConfiguration>LogAndDiscard</OutboundFailConfiguration>\r\n <OutboundRetryInterval>15000</OutboundRetryInterval>\r\n <OutboundMaxRetries>5</OutboundMaxRetries>\r\n <SendMessageMaxRetries>10</SendMessageMaxRetries>\r\n <SendMessageRetryInterval>10000</SendMessageRetryInterval>\r\n <InboundUri>C:\\Program Files\\Opcenter Connect MOM\\Channel Adapter Host\\inbound</InboundUri>\r\n <OutboundUri>C:\\Program Files\\Opcenter Connect MOM\\Channel Adapter Host\\outbound</OutboundUri>\r\n <ErrorUri></ErrorUri>\r\n <InboundDriveToMap></InboundDriveToMap>\r\n <InboundFilenameFilter>*.*</InboundFilenameFilter>\r\n <OutboundDriveToMap></OutboundDriveToMap>\r\n <MaxReadRetries>10</MaxReadRetries>\r\n <RetryDelay>15000</RetryDelay>\r\n <DeleteInterval>15</DeleteInterval>\r\n <EncodingName>UTF-8</EncodingName>\r\n</RESTTestResponse>","contents":"<RESTTestResponse TestTimestamp=\"2021/10/05 11:57:05\">\r\n <Name>miotestadapter</Name>\r\n <IsActive>true</IsActive>\r\n <Description>Added during MIO install for confirmation test. Version=3.0.0.0</Description>\r\n <type>ChannelAdapterConfiguration</type>\r\n <AdapterConfigurationTypeName>FileAdapterConfiguration</AdapterConfigurationTypeName>\r\n <AdapterType>FileAdapter</AdapterType>\r\n <AdapterVersion>1</AdapterVersion>\r\n <AdapterKind>default</AdapterKind>\r\n <ContentFormat>Xml</ContentFormat>\r\n <ChannelSourceName>miochannelsource</ChannelSourceName>\r\n <MessageChannelName>miotestchannel</MessageChannelName>\r\n <MomConnectionName></MomConnectionName>\r\n <OutboundNameFormat>{messagename}</OutboundNameFormat>\r\n <InboundNameFormat>{messagename}</InboundNameFormat>\r\n <DoNotSend>false</DoNotSend>\r\n <DoNotSendOutbound>false</DoNotSendOutbound>\r\n <ShutdownMaxTime>15</ShutdownMaxTime>\r\n <IsXml>true</IsXml>\r\n <WakeUpInterval>-1</WakeUpInterval>\r\n <UserName></UserName>\r\n <Password></Password>\r\n <DomainName></DomainName>\r\n <VerifyUnique>true</VerifyUniqu',
0)
insert into "LOGMESSAGE" values(
'd8dcd593-af52-425a-8bf2-d93f78a601c6','e>\r\n <UniqueIncludesTimestamp>true</UniqueIncludesTimestamp>\r\n <UniqueCacheExpiration>0</UniqueCacheExpiration>\r\n <ClearOriginalContents>true</ClearOriginalContents>\r\n <BufferSettings>\r\n <Description></Description>\r\n <Kind>Persistent</Kind>\r\n <ConnectionName></ConnectionName>\r\n <ConnectionString>folder name=%AppData%</ConnectionString>\r\n <Interval>2000</Interval>\r\n <Expiration>-1</Expiration>\r\n <MaxCount>-1</MaxCount>\r\n </BufferSettings>\r\n <InProcessExpiration>120</InProcessExpiration>\r\n <InboundFilters />\r\n <OutboundFilters />\r\n <OutboundFailPlugin>LogAndDiscard</OutboundFailPlugin>\r\n <OutboundFailConfiguration>LogAndDiscard</OutboundFailConfiguration>\r\n <OutboundRetryInterval>15000</OutboundRetryInterval>\r\n <OutboundMaxRetries>5</OutboundMaxRetries>\r\n <SendMessageMaxRetries>10</SendMessageMaxRetries>\r\n <SendMessageRetryInterval>10000</SendMessageRetryInterval>\r\n <InboundUri>C:\\Program Files\\Opcenter Connect MOM\\Channel Adapter Host\\inbound</InboundUri>\r\n <OutboundUri>C:\\Program Files\\Opcenter Connect MOM\\Channel Adapter Host\\outbound</OutboundUri>\r\n <ErrorUri></ErrorUri>\r\n <InboundDriveToMap></InboundDriveToMap>\r\n <InboundFilenameFilter>*.*</InboundFilenameFilter>\r\n <OutboundDriveToMap></OutboundDriveToMap>\r\n <MaxReadRetries>10</MaxReadRetries>\r\n <RetryDelay>15000</RetryDelay>\r\n <DeleteInterval>15</DeleteInterval>\r\n <EncodingName>UTF-8</EncodingName>\r\n</RESTTestResponse>","empty":false,"contentsformat":"Xml","contentshash":"64882DB2ED9382F3075C94A0657C3034","hash":"135D1EEE963EC392868AF2AFF7DE5711","priority":1,"outbound":true,"request":false,"response":true,"requestid":"06da6532-bfd0-4a77-9637-b4da6f48b23a","events":[],"attributes":{"filename":"miogetadapter001.xml","restcommand":"/api/channeladapters","httpverb":"GET","querystring":"name=miotestadapter"},"tagdata":"","adaptertagdata":{"properties":"{\r\n \"encoding\": \"UTF-8\",\r\n \"source\": \"miochannelsource\",\r\n \"adapter\": \"miotestadapter\",\r\n \"directory\": \"C:\\\\Program Files\\\\Opcenter Connect MOM\\\\Channel Adapter Host\\\\inbound\"\r\n}"},"status":"","correlationid":"","maxretrycount":10,"retrycount":0,"automapped":false,"express":false,"inhibitEvent":false,"eventonly":false,"eventttl":0,"check":{"timestamp":"2021-10-05T06:27:04.8626024Z","stages":[{"name":"Created","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:04.8626024Z","time":0,"duration":0},{"name":"Adapter Host Added","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:04.8645993Z","time":1,"duration":1},{"name":"Adapter Host Dispatch","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:04.869603Z","time":7,"duration":6,"dispatcher":"BrokerBalanced"},{"name":"Broker Received","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:04.8905926Z","time":27,"duration":20},{"name":"Broker Added","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:04.8925959Z","time":29,"duration":2},{"name":"Broker Dispatch","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:04.9285978Z","time":65,"duration":36,"availThreads":24,"dispatcher":"fifowithpredecessors"},{"name":"Broker Added","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:05.0966056Z","time":234,"duration":169},{"name":"Broker Dispatch","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:05.1066034Z","time":244,"duration":10,"availThreads":24,"dispatcher":"fifowithpredecessors"},{"name":"Adapter Host Received","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:05.1216049Z","time":259,"duration":15},{"name":"Adapter Host Send","machineName":"VM-VDIP49-03","timestamp":"2021-10-05T06:27:05.1486105Z","time":286,"duration":27}]}}',
1)
insert into "LOGENTRY"
values ('d8dcd593-af52-425a-8bf2-d93f78a601c6')
**Tried Query**
SELECT M.Id,RTRIM(XMLAGG(XMLELEMENT(E,M.SerializedMessage,',').EXTRACT('//text()') ORDER BY M.SerializedMessage).GetClobVal(),',') AS LIST1,M.Series
from LOGENTRY E INNER JOIN LOGMESSAGE M ON E.Id = M.Id group by M.Id,M.series;
**Error**
ORA-00932: inconsistent datatypes: expected CHAR got LONG
00932. 00000 - "inconsistent datatypes: expected %s got %s"
The long datatype by itself has lots of restrictions and is very difficult to work with in general. There's a reason why it's deprecated and you're just giving yourself extra work, finding ways around it, instead of using clobs.
But to answer your question, this is, I think, probably what you wanted:
SELECT listagg(a.linetext, '') within group (order by rownum)
FROM
(
SELECT 'a' linetext from dual
UNION ALL
SELECT 'a' linetext from dual
UNION ALL
SELECT '2' linetext from dual
UNION ALL
SELECT 'b' linetext from dual
) a
Replace the subquery with a query of your table and it will concat your results, which you get in separate lines.

How to write correct left Join of two tables?

I want to join two tables, first table primary key data type is number, and second table primary key data type is VARCHAR2(30 BYTE). How to join both tables.
I tried this code but second tables all values are null. why is that?
SELECT a.act_phone_no,a.act_actdevice,a.bi_account_id, a.packag_start_date, c.identification_number,
FROM ACTIVATIONS_POP a
left JOIN customer c
on TO_CHAR(a.act_phone_no) = c.msisdn_voice
first table
act_phone_no bi_account_id
23434 45345
34245 43556
Second table
msisdn_voice identification_number
23434 321113
34245 6547657
It seems that you didn't tell us everything. Query works, if correctly written, on such a sample data:
SQL> with
2 -- Sample data
3 activations_pop (act_phone_no, bi_account_id) as
4 (select 23434, 45345 from dual union all
5 select 34245, 43556 from dual
6 ),
7 customer (msisdn_voice, identification_number) as
8 (select '23434', 321113 from dual union all
9 select '34245', 6547657 from dual
10 )
11 -- query works OK
12 select a.act_phone_no,
13 a.bi_account_id,
14 c.identification_number
15 from activations_pop a join customer c on to_char(a.act_phone_no) = c.msisdn_voice;
ACT_PHONE_NO BI_ACCOUNT_ID IDENTIFICATION_NUMBER
------------ ------------- ---------------------
23434 45345 321113
34245 43556 6547657
SQL>
What could be wrong? Who knows. If you got some result but columns from the CUSTOMER table are empty (NULL?), then they really might be NULL, or you didn't manage to join rows on those columns (left/right padding with spaces?). Does joining on e.g.
on to_char(a.act_phone_no) = trim(c.msisdn_voice)
or
on a.act_phone_no = to_number(c.msisdn_voice)
help?
Consider posting proper test case (CREATE TABLE and INSERT INTO statements).
You are using Oracle ?
Please check the below demo
SELECT a.act_phone_no, a.bi_account_id, c.identification_number
FROM ACTIVATIONS_POP a
left JOIN customer c
on TO_CHAR(a.act_phone_no) = c.msisdn_voice;
SQLFiddle

Fastest way of doing field comparisons in the same table with large amounts of data in oracle

I am recieving information from a csv file from one department to compare with the same inforation in a different department to check for discrepencies (About 3/4 of a million rows of data with 44 columns in each row). After I have the data in a table, I have a program that will take the data and send reports based on a HQ. I feel like the way I am going about this is not the most efficient. I am using oracle for this comparison.
Here is what I have:
I have a vb.net program that parses the data and inserts it into an extract table
I run a procedure to do a full outer join on the two tables into a new table with the fields in one department prefixed with '_c'
I run another procedure to compare the old/new data and update 2 different tables with detail and summary information. Here is code from inside the procedure:
DECLARE
CURSOR Cur_Comp IS SELECT * FROM T.AEC_CIS_COMP;
BEGIN
FOR compRow in Cur_Comp LOOP
--If service pipe exists in CIS but not in FM and the service pipe has status of retired in CIS, ignore the variance
If(compRow.pipe_num = '' AND cis_status_c = 'R')
continue
END IF
--If there is not a summary record for this HQ in the table for this run, create one
INSERT INTO t.AEC_CIS_SUM (HQ, RUN_DATE)
SELECT compRow.HQ, to_date(sysdate, 'DD/MM/YYYY') from dual WHERE NOT EXISTS
(SELECT null FROM t.AEC_CIS_SUM WHERE HQ = compRow.HQ AND RUN_DATE = to_date(sysdate, 'DD/MM/YYYY'))
-- Check fields and update the tables accordingly
If (compRow.cis_loop <> compRow.cis_loop_c) Then
--Insert information into the details table
INSERT INTO T.AEC_CIS_DET( Fac_id, Pipe_Num, Hq, Address, AutoUpdatedFl,
DateTime, Changed_Field, CIS_Value, FM_Value)
VALUES(compRow.Fac_ID, compRow.Pipe_Num, compRow.Hq, compRow.Street_Num || ' ' || compRow.Street_Name,
'Y', sysdate, 'Cis_Loop', compRow.cis_loop, compRow.cis_loop_c);
-- Update information into the summary table
UPDATE AEC_CIS_SUM
SET cis_loop = cis_loop + 1
WHERE Hq = compRow.Hq
AND Run_Date = to_date(sysdate, 'DD/MM/YYYY')
End If;
END LOOP;
END;
Any suggestions of an easier way of doing this rather than an if statement for all 44 columns of the table? (This is run once a week if it matters)
Update: Just to clarify, there are 88 columns of data (44 of duplicates to compare with one suffixed with _c). One table lists each field in a row that is different so one row can mean 30+ records written in that table. The other table keeps tally of the number of discrepencies for each week.
First of all I believe that your task can be implemented (and should be actually) with staight SQL. No fancy cursors, no loops, just selects, inserts and updates. I would start with unpivotting your source data (it is not clear if you have primary key to join two sets, I guess you do):
Col0_PK Col1 Col2 Col3 Col4
----------------------------------------
Row1_val A B C D
Row2_val E F G H
Above is your source data. Using UNPIVOT clause we convert it to:
Col0_PK Col_Name Col_Value
------------------------------
Row1_val Col1 A
Row1_val Col2 B
Row1_val Col3 C
Row1_val Col4 D
Row2_val Col1 E
Row2_val Col2 F
Row2_val Col3 G
Row2_val Col4 H
I think you get the idea. Say we have table1 with one set of data and the same structured table2 with the second set of data. It is good idea to use index-organized tables.
Next step is comparing rows to each other and storing difference details. Something like:
insert into diff_details(some_service_info_columns_here)
select some_service_info_columns_here_along_with_data_difference
from table1 t1 inner join table2 t2
on t1.Col0_PK = t2.Col0_PK
and t1.Col_name = t2.Col_name
and nvl(t1.Col_value, 'Dummy1') <> nvl(t2.Col_value, 'Dummy2');
And on the last step we update difference summary table:
insert into diff_summary(summary_columns_here)
select diff_row_id, count(*) as diff_count
from diff_details
group by diff_row_id;
It's just rough draft to show my approach, I'm sure there is much more details should be taken into account. To summarize I suggest two things:
UNPIVOT data
Use SQL statements instead of cursors
You have several issues in your code:
If(compRow.pipe_num = '' AND cis_status_c = 'R')
continue
END IF
"cis_status_c" is not declared. Is it a variable or a column in AEC_CIS_COMP?
In case it is a column, just put the condition into the cursor, i.e. SELECT * FROM T.AEC_CIS_COMP WHERE not (compRow.pipe_num = '' AND cis_status_c = 'R')
to_date(sysdate, 'DD/MM/YYYY')
That's nonsense, you convert a date into a date, simply use TRUNC(SYSDATE)
Anyway, I think you can use three single statements instead of a cursor:
INSERT INTO t.AEC_CIS_SUM (HQ, RUN_DATE)
SELECT comp.HQ, trunc(sysdate)
from AEC_CIS_COMP comp
WHERE NOT EXISTS
(SELECT null FROM t.AEC_CIS_SUM WHERE HQ = comp.HQ AND RUN_DATE = trunc(sysdate));
INSERT INTO T.AEC_CIS_DET( Fac_id, Pipe_Num, Hq, Address, AutoUpdatedFl, DateTime, Changed_Field, CIS_Value, FM_Value)
select comp.Fac_ID, comp.Pipe_Num, comp.Hq, comp.Street_Num || ' ' || comp.Street_Name, 'Y', sysdate, 'Cis_Loop', comp.cis_loop, comp.cis_loop_c
from T.AEC_CIS_COMP comp
where comp.cis_loop <> comp.cis_loop_c;
UPDATE AEC_CIS_SUM
SET cis_loop = cis_loop + 1
WHERE Hq IN (Select Hq from T.AEC_CIS_COMP)
AND trunc(Run_Date) = trunc(sysdate);
They are not tested but they should give you a hint how to do it.

How do I display a field's hidden characters in the result of a query in Oracle?

I have two rows that have a varchar column that are different according to a Java .equals(). I can't easily change or debug the Java code that's running against this particular database but I do have access to do queries directly against the database using SQLDeveloper. The fields look the same to me (they are street addresses with two lines separated by some new line or carriage feed/new line combo).
Is there a way to see all of the hidden characters as the result of a query?I'd like to avoid having to use the ascii() function with substr() on each of the rows to figure out which hidden character is different.
I'd also accept some query that shows me which character is the first difference between the two fields.
Try
select dump(column_name) from table
More information is in the documentation.
As for finding the position where the character differs, this might give you an idea:
create table tq84_compare (
id number,
col varchar2(20)
);
insert into tq84_compare values (1, 'hello world');
insert into tq84_compare values (2, 'hello' || chr(9) || 'world');
with c as (
select
(select col from tq84_compare where id = 1) col1,
(select col from tq84_compare where id = 2) col2
from
dual
),
l as (
select
level l from dual
start with 1=1
connect by level < (select length(c.col1) from c)
)
select
max(l.l) + 1position
from c,l
where substr(c.col1,1,l.l) = substr(c.col2,1,l.l);
SELECT DUMP('€ÁÑ', 1016)
FROM DUAL
... will print something like:
Typ=96 Len=3 CharacterSet=WE8MSWIN1252: 80,c1,d1

How to put more than 1000 values into an Oracle IN clause [duplicate]

This question already has answers here:
SQL IN Clause 1000 item limit
(5 answers)
Closed 8 years ago.
Is there any way to get around the Oracle 10g limitation of 1000 items in a static IN clause? I have a comma delimited list of many of IDs that I want to use in an IN clause, Sometimes this list can exceed 1000 items, at which point Oracle throws an error. The query is similar to this...
select * from table1 where ID in (1,2,3,4,...,1001,1002,...)
Put the values in a temporary table and then do a select where id in (select id from temptable)
select column_X, ... from my_table
where ('magic', column_X ) in (
('magic', 1),
('magic', 2),
('magic', 3),
('magic', 4),
...
('magic', 99999)
) ...
I am almost sure you can split values across multiple INs using OR:
select * from table1 where ID in (1,2,3,4,...,1000) or
ID in (1001,1002,...,2000)
You may try to use the following form:
select * from table1 where ID in (1,2,3,4,...,1000)
union all
select * from table1 where ID in (1001,1002,...)
Where do you get the list of ids from in the first place? Since they are IDs in your database, did they come from some previous query?
When I have seen this in the past it has been because:-
a reference table is missing and the correct way would be to add the new table, put an attribute on that table and join to it
a list of ids is extracted from the database, and then used in a subsequent SQL statement (perhaps later or on another server or whatever). In this case, the answer is to never extract it from the database. Either store in a temporary table or just write one query.
I think there may be better ways to rework this code that just getting this SQL statement to work. If you provide more details you might get some ideas.
Use ...from table(... :
create or replace type numbertype
as object
(nr number(20,10) )
/
create or replace type number_table
as table of numbertype
/
create or replace procedure tableselect
( p_numbers in number_table
, p_ref_result out sys_refcursor)
is
begin
open p_ref_result for
select *
from employees , (select /*+ cardinality(tab 10) */ tab.nr from table(p_numbers) tab) tbnrs
where id = tbnrs.nr;
end;
/
This is one of the rare cases where you need a hint, else Oracle will not use the index on column id. One of the advantages of this approach is that Oracle doesn't need to hard parse the query again and again. Using a temporary table is most of the times slower.
edit 1 simplified the procedure (thanks to jimmyorr) + example
create or replace procedure tableselect
( p_numbers in number_table
, p_ref_result out sys_refcursor)
is
begin
open p_ref_result for
select /*+ cardinality(tab 10) */ emp.*
from employees emp
, table(p_numbers) tab
where tab.nr = id;
end;
/
Example:
set serveroutput on
create table employees ( id number(10),name varchar2(100));
insert into employees values (3,'Raymond');
insert into employees values (4,'Hans');
commit;
declare
l_number number_table := number_table();
l_sys_refcursor sys_refcursor;
l_employee employees%rowtype;
begin
l_number.extend;
l_number(1) := numbertype(3);
l_number.extend;
l_number(2) := numbertype(4);
tableselect(l_number, l_sys_refcursor);
loop
fetch l_sys_refcursor into l_employee;
exit when l_sys_refcursor%notfound;
dbms_output.put_line(l_employee.name);
end loop;
close l_sys_refcursor;
end;
/
This will output:
Raymond
Hans
I wound up here looking for a solution as well.
Depending on the high-end number of items you need to query against, and assuming your items are unique, you could split your query into batches queries of 1000 items, and combine the results on your end instead (pseudocode here):
//remove dupes
items = items.RemoveDuplicates();
//how to break the items into 1000 item batches
batches = new batch list;
batch = new batch;
for (int i = 0; i < items.Count; i++)
{
if (batch.Count == 1000)
{
batches.Add(batch);
batch.Clear()
}
batch.Add(items[i]);
if (i == items.Count - 1)
{
//add the final batch (it has < 1000 items).
batches.Add(batch);
}
}
// now go query the db for each batch
results = new results;
foreach(batch in batches)
{
results.Add(query(batch));
}
This may be a good trade-off in the scenario where you don't typically have over 1000 items - as having over 1000 items would be your "high end" edge-case scenario. For example, in the event that you have 1500 items, two queries of (1000, 500) wouldn't be so bad. This also assumes that each query isn't particularly expensive in of its own right.
This wouldn't be appropriate if your typical number of expected items got to be much larger - say, in the 100000 range - requiring 100 queries. If so, then you should probably look more seriously into using the global temporary tables solution provided above as the most "correct" solution. Furthermore, if your items are not unique, you would need to resolve duplicate results in your batches as well.
Yes, very weird situation for oracle.
if you specify 2000 ids inside the IN clause, it will fail.
this fails:
select ...
where id in (1,2,....2000)
but if you simply put the 2000 ids in another table (temp table for example), it will works
below query:
select ...
where id in (select userId
from temptable_with_2000_ids )
what you can do, actually could split the records into a lot of 1000 records and execute them group by group.
Here is some Perl code that tries to work around the limit by creating an inline view and then selecting from it. The statement text is compressed by using rows of twelve items each instead of selecting each item from DUAL individually, then uncompressed by unioning together all columns. UNION or UNION ALL in decompression should make no difference here as it all goes inside an IN which will impose uniqueness before joining against it anyway, but in the compression, UNION ALL is used to prevent a lot of unnecessary comparing. As the data I'm filtering on are all whole numbers, quoting is not an issue.
#
# generate the innards of an IN expression with more than a thousand items
#
use English '-no_match_vars';
sub big_IN_list{
#_ < 13 and return join ', ',#_;
my $padding_required = (12 - (#_ % 12)) % 12;
# get first dozen and make length of #_ an even multiple of 12
my ($a,$b,$c,$d,$e,$f,$g,$h,$i,$j,$k,$l) = splice #_,0,12, ( ('NULL') x $padding_required );
my #dozens;
local $LIST_SEPARATOR = ', '; # how to join elements within each dozen
while(#_){
push #dozens, "SELECT #{[ splice #_,0,12 ]} FROM DUAL"
};
$LIST_SEPARATOR = "\n union all\n "; # how to join #dozens
return <<"EXP";
WITH t AS (
select $a A, $b B, $c C, $d D, $e E, $f F, $g G, $h H, $i I, $j J, $k K, $l L FROM DUAL
union all
#dozens
)
select A from t union select B from t union select C from t union
select D from t union select E from t union select F from t union
select G from t union select H from t union select I from t union
select J from t union select K from t union select L from t
EXP
}
One would use that like so:
my $bases_list_expr = big_IN_list(list_your_bases());
$dbh->do(<<"UPDATE");
update bases_table set belong_to = 'us'
where id in ($bases_list_expr)
UPDATE
Instead of using IN clause, can you try using JOIN with the other table, which is fetching the id. that way we don't need to worry about limit. just a thought from my side.
Instead of SELECT * FROM table1 WHERE ID IN (1,2,3,4,...,1000);
Use this :
SELECT * FROM table1 WHERE ID IN (SELECT rownum AS ID FROM dual connect BY level <= 1000);
*Note that you need to be sure the ID does not refer any other foreign IDS if this is a dependency. To ensure only existing ids are available then :
SELECT * FROM table1 WHERE ID IN (SELECT distinct(ID) FROM tablewhereidsareavailable);
Cheers

Resources