Selecting big size docs from oracle db - oracle

So want to get Id's of documents which are bigger than 60 mb:
SELECT
DOCS.ID
FROM DOCS
where LENGTH(DOCS.DOCUMENT) > (60*1024*1024)
and I get this error :
SQL Error: ORA-00932: inconsistent datatypes: expected NUMBER got LONG BINARY
00932. 00000 - "inconsistent datatypes: expected %s got %s"
document is savad as a long raw ...
probably should cast somehow to long???
The only solution I came up with was To alter table from long raw to BLOB, then for some divine reason you have to recreate ALL indexes for that table... it's the only way.

Indeed this is a problem, but you can use PL/SQL as a workaround:
create or replace function get_doc_length(iDocId in number) return number is
aLong long;
begin
select d.document into aLong from docs d where d.id = iDocId;
return length(aLong);
end;
/
This function (which cannot use a LONG as parameter, unfortunately, so must be specific to table DOCS) can then be used like this:
SELECT DOCS.ID
FROM DOCS
where get_doc_length(DOCS.ID) > (60*1024*1024)
EDIT : ok, this does not work for big LONGs. You have to dig even more into SQL's arcanes to make this work:
create or replace function get_doc_length(iDocId in number) return number is
myQuery varchar2(200);
myCursor binary_integer;
myRes pls_integer;
myDoc clob;
long_val long;
long_len integer;
buf_len integer;
cur_pos number;
begin
myQuery := 'select d.document from docs d where d.id = ' || iDocId;
-- Create cursor, parse and bind.
myCursor := DBMS_SQL.OPEN_CURSOR;
DBMS_SQL.PARSE(myCursor, myQuery, DBMS_SQL.NATIVE);
DBMS_SQL.DEFINE_COLUMN_LONG(myCursor, 01);
myRes := DBMS_SQL.EXECUTE(myCursor);
-- Fetch row (only one normally)
if DBMS_SQL.FETCH_ROWS(myCursor) > 0 then
-- Create CLOB.
DBMS_LOB.CREATETEMPORARY(myDoc, false, DBMS_LOB.CALL);
-- Piecewise fetching of the LONG column, appending to the CLOB.
buf_len := 32760;
cur_pos := 0;
loop
DBMS_SQL.COLUMN_VALUE_LONG(myCursor, 01, buf_len, cur_pos, long_val, long_len);
exit when long_len = 0;
DBMS_LOB.APPEND(myDoc, long_val);
cur_pos := cur_pos + long_len;
end loop;
end if;
DBMS_SQL.CLOSE_CURSOR(myCursor);
return length(myDoc);
end;
/
The DBMS_SQL package allows to convert a LONG into a CLOB by looping on it and appending pieces of the LONG gradually.
If you want the binary length of the CLOB instead of its char length (this may differ), you can use dbms_lob.getlength instead of length as this other SO post shows.

SELECT DOCS.ID
FROM DOCS
WHERE dbms_lob.getlength(DOCS.DOCUMENT) > (60*1024*1024)

Related

Oracle 11g maximum statement length possible

I am facing the issue with very long statements in Oracle 11g (~220k-symbol selects with a lot of autogenerated IN (...) clauses). Selects are fired by JDBC causing SQLException 17410 No more data from socket.
On the other hand, selects with the same structure but with shorter lengths (~100k symbols) are executed fine.
The problem is that there is no reference in the docs on how to manage maximum statement length. There is just a note: The limit on how long a SQL statement can be depends on many factors, including database configuration, disk space, and memory which is not informative at all.
Can anyone share an experience on how to estimate this maximum length and what database tweaks (if any) can help to increase it?
Having thousands of expressions in IN (...) seems to be a bad design. Better insert such values into a (temporary) table and limit the result by JOIN or IN (SELECT ...)
Anyway, the statement length is almost unlimited. I have an application where I run large statements like this:
DECLARE
cmd DBMS_SQL.VARCHAR2A;
cur INTEGER := DBMS_SQL.OPEN_CURSOR;
res INTEGER;
BEGIN
cmd(1) := 'INSERT INTO';
cmd(cmd.LAST+1) := 'some columns, ';
cmd(cmd.LAST+1) := 'and some more,'; -- each line is limited to 32767 chars
DBMS_SQL.PARSE(cur, cmd, cmd.FIRST, cmd.LAST, TRUE, DBMS_SQL.NATIVE);
res := DBMS_SQL.EXECUTE(cur);
DBMS_SQL.CLOSE_CURSOR(cur);
END;
Note, DBMS_SQL.EXECUTE does not fetch any data. For SELECT the procedure would be
DECLARE
cmd DBMS_SQL.VARCHAR2A;
cur INTEGER := DBMS_SQL.OPEN_CURSOR;
res INTEGER;
refCur SYS_REFCURSOR;
BEGIN
cmd(1) := 'SELECT ';
cmd(cmd.LAST+1) := 'some columns, ';
cmd(cmd.LAST+1) := 'and some more,'; -- each line is limited to 32767 chars
DBMS_SQL.PARSE(cur, cmd, cmd.FIRST, cmd.LAST, TRUE, DBMS_SQL.NATIVE);
res := DBMS_SQL.EXECUTE(cur);
refCur := DBMS_SQL.TO_REFCURSOR(cur);
FETCH refCur BULK COLLECT INTO ...;
CLOSE refCur;
END;
or use DBMS_SQL.EXECUTE_AND_FETCH if you prefer. Procedure DBMS_SQL.RETURN_RESULT may also help.
So far I never faced any limitation with this method.

VARCHAR2(32767) not able to handle strings in stored procedure

I am concatenating string using cursor (to form query to execute later). Here, the query that will be formed is going to be way bigger that what VARCHAR2(32767) can handle. There fore, I am getting error on proc execution - ORA-06502: PL/SQL: numeric or value error: character string buffer too small.
I used CLOB data type as well bu got error ORA-06502: PL/SQL: numeric or value error.
My code is here below:
CREATE OR REPLACE PROCEDURE sp_Market
IS
Names VARCHAR2(32767);
BEGIN
DECLARE CURSOR cur IS ('Select ID, Order_of, field_name
FROM pld_medicare_config');
BEGIN
FOR i IN cur
LOOP
Names := Names || i.sqql;
END LOOP;
dbms_output.put_line(Names);
END;
END sp_Market;
How can I handle my string of queries and what data type is there to accomplish the task?
CLOB is OK (as far as I can tell); I doubt queries you store in there are that big.
Remove dbms_output.put_line call from the procedure; I suspect it is the one that raises the error.
I'm not sure how you got any runtime error, as your procedure won't compile.
The valid PL/SQL version would look something like this:
create or replace procedure sp_market is
names varchar2(32767);
begin
for r in (
select id, order_of, field_name
from pld_medicare_config
)
loop
names := names || ' ' || r.field_name;
end loop;
names := ltrim(names);
dbms_output.put_line(names);
end sp_market;
If names needs to be longer, change the datatype to clob.
Use the CLOB datatype and append data using the dbms_lob.writeappend procedure. This is the reference (Oracle 18c).
The error probably origins with the dbms_output.put_line call. The procedure is defined for varchar2 arguments only which means that an implicit conversion takes place during the call. It will fail for clob contents longer than 32767 chars/bytes.
Alternatively you may declare a collection over varchar2(4000) and fill the collection elements sequentially:
CREATE OR REPLACE PROCEDURE sp_Market
IS
TYPE tLongString IS TABLE OF VARCHAR2(4000) INDEX BY BINARY_INTEGER;
cNames tLongString;
BEGIN
DECLARE CURSOR cur IS Select ID, Order_of, field_name, sqql FROM pld_medicare_config;
BEGIN
FOR i IN cur
LOOP
cNames(cNames.COUNT+1) := i.sqql;
END LOOP;
END;
END sp_Market;
Note
Rectified code, will compile now.

How to select max value from a temporary table whose field is unnamed

I'm writing a PL/SQL function that needs to get dates from a few different tables and return the most recent one. My approach has been this:
Create a temporary table to hold the dates:
CREATE TYPE t_dates IS TABLE OF DATE;
/
Create a few local variables:
l_date DATE;
l_dates t_dates := t_dates();
l_idx INTEGER := 0;
l_output_date DATE;
Then select into a variable each date I'm interested in, and add it to the temporary table:
SELECT it.date
INTO l_date
FROM interesting_table it
WHERE it.id = 1
;
l_dates.extend;
l_idx := l_idx + 1;
l_dates(l_idx) := l_date;
After the temporary table has been populated, now I just want to select the max value from it. How do I do that? Something like...
SELECT max(*)
INTO l_output_date
FROM l_dates
;
RETURN l_output_date;
But I'm not sure I can reference my temporary table like that, nor how to find the max of a column that is unnamed.
Edit: When I test the above, I get an error: PL/SQL: ORA-00936: missing expression related to the line where I have SELECT max(*). However, I don't believe that this is my only error, because if I change the last block to:
SELECT *
INTO l_output_date
FROM l_dates
WHERE rownum = 1
;
which, just for testing, should select the first date in the temp table, then the error becomes PL/SQL: ORA-00942: table or view does not exist, which indicates to me that I can't refer to my temp table in this manner.
I have since learned from the comments that I should have max(column_value) rather than max(*), but using this, I still get the table or view does not exist error.
It is not possible to select from the nested tables, like you have tried.
You may have to use a logic to find the maximum value
l_idx := l_idx + 1;
l_dates(l_idx) := l_date;
if(l_date > l_dates(l_idx-1) OR l_idx =1) THEN
max_date :=l_date;
End if;
For your case, Rather than fetching a list of dates then getting the max date, you can find the max date in your query itself
SELECT max(it.date)
INTO l_date
FROM interesting_table it
WHERE it.id = 1
;

Getting inconsistent datatypes error while fetching data from ref cursor into a nested table

I'm getting an error while fetching data from ref cursor into a nested table. Here is what I'm doing.
Object and table types:
CREATE OR REPLACE TYPE obj_report_org IS OBJECT
(dummy1 VARCHAR2(50),
dummy2 VARCHAR2(50),
dummy3 VARCHAR2(50),
dummy4 NUMBER(10));
CREATE OR REPLACE TYPE tab_report_org IS TABLE OF obj_report_org;
Procedure:
CREATE OR REPLACE PROCEDURE areaReport_test(v_name1 VARCHAR2,
v_name2 VARCHAR2, tab_report_set OUT tab_report_org)
IS
str VARCHAR2(2000);
v_num NUMBER := 1;
recordset SYS_REFCURSOR;
lv_tab_report_set tab_report_org := tab_report_org();
BEGIN
str := ' SELECT tab2.dummy1 ,tab3.dummy2,tab2.dummy3,tab1.dummy4 '||
' FROM tab1,tab2, tab3, tab4'||
' WHERE <JOIN CONDITIONS>';
OPEN recordset FOR
str||' START WITH tab1.name = :name1'||
' CONNECT BY PRIOR tab1.id = tab1.parent_id'||
' UNION '||
str||' START WITH tab1.name = :name2'||
' CONNECT BY PRIOR tab1.id = tab1.parent_id'
USING v_name1,v_name2;
FETCH recordset BULK COLLECT into lv_tab_report_set;
CLOSE recordset;
tab_report_set := lv_tab_report_set;
END;
Then I have an anonymous block to call the procedure:
DECLARE
l_tab_report_set tab_report_org;
BEGIN
areaReport_test('PRASHANT',null,l_tab_report_set);
FOR i in 1 .. l_tab_report_set.count
LOOP
DBMS_OUTPUT.PUT_LINE(
l_tab_report_set(i).dummy1|| ' | ' ||
l_tab_report_set(i).dummy2|| ' | ' ||
l_tab_report_set(i).dummy3|| ' | ' ||
l_tab_report_set(i).dummy4|| );
END LOOP;
END;
After running the anonymous block, I'm get this error:
ORA-00932: inconsistent datatypes: expected - got -
ORA-06512: at "AREAREPORT_TEST", line 36
ORA-06512: at line 5
00932. 00000 - "inconsistent datatypes: expected %s got %s"
It seems we cannot bulk fetch data into nested table formed from a SQL object. While sequence of fields and datatypes in object match with select query.
Please advise.
I will start from the bottom:
FETCH recordset BULK COLLECT into lv_tab_report_set;
There is a recordset that you are fetching into table of objects. So, recordset should contain list of objects.
SELECT tab2.dummy1 ,tab3.dummy2,tab2.dummy3,tab1.dummy4...
It is a list of columns and not an object. Here is a list of objects:
select obj_report_org(tab2.dummy1 ,tab3.dummy2,tab2.dummy3,tab1.dummy4) ...
few remarks about code in general
When you use UNION instead of UNION ALL, Oracle would sort both results and eliminate duplicates (there is no other way to do that). So just think about if you really need UNION and remember that there is a price for that.
You UNION two datasets that looks like could be combined together
START WITH tab1.name in (:name1, :name2)
Continues after ORA-22950
You are implisitly ordering your elements, but how ORACLE knows how to sort them?
By dummy1 or dummy2 or ...?
You have to options:
do not use sorting
that basically means no DISTINCT, GROUP BY, ORDER BY and no UNION (you are OK with UNION ALL)
define sorting for your objects
Follow up about methods in ORACLE Types
Your oracle type could have additional methods, some of them are special, notice 'ORDER MEMBER FUNCTION' below (copy from the link):
<!-- language: pl/sql -->
CREATE OR REPLACE TYPE location_typ AS OBJECT (
building_no NUMBER,
city VARCHAR2(40),
ORDER MEMBER FUNCTION match (l location_typ) RETURN INTEGER
);
/
CREATE OR REPLACE TYPE BODY location_typ AS
ORDER MEMBER FUNCTION match (l location_typ) RETURN INTEGER IS
BEGIN
IF building_no < l.building_no THEN
RETURN -1; -- any negative number will do
ELSIF building_no > l.building_no THEN
RETURN 1; -- any positive number will do
ELSE
RETURN 0;
END IF;
END;
END;/
Its not a problem of fetch its kind of datatype mismatch. Opening cursors and fetching is correct what you have written.

PL/SQL DBMS_XMLQUERY max size

My oracle version is 11g release 2
Our system is using DBMS_XMLQUERY to transform sql result into xml, but recently the data is becoming large and we get this error: ORA-06502: PL/SQL: numeric or value error
The reason seems to be DBMS_XMLQUERY cannot handle too many records, but oracle's official document doesn't show the limitation. So maybe I have done something wrong. You can reproduce the problem in the following steps:
step1:
create table XMLDATA ( data_id int primary key, data_str
VARCHAR2(100) );
step2:
INSERT INTO XMLDATA values(1, 'test0123456789');
INSERT INTO XMLDATA values(2, 'test0123456789');
INSERT INTO XMLDATA values(3, 'test0123456789');
....
INSERT INTO XMLDATA values(500, 'test0123456789');
step3:
CREATE OR REPLACE
function test(total in int) return clob is
i int;
vn_ctx DBMS_XMLQUERY.ctxHandle;
BEGIN
vn_ctx := DBMS_XMLQUERY.NEWCONTEXT('select data_id, data_str from XMLDATA where rownum <= ' || total);
DBMS_XMLQuery.propagateOriginalException(vn_ctx,true);
DBMS_XMLQUERY.useNullAttributeIndicator(vn_ctx,true);
DBMS_XMLQUERY.SETROWTAG(vn_ctx, 'ITEM');
DBMS_XMLQUERY.SETROWSETTAG(vn_ctx, 'PODATA');
return DBMS_XMLQUERY.GETXML(vn_ctx);
END;
step4:
execute function test with a number greater than 400. Then you'll get 'ORA-06502: PL/SQL: numeric or value error'
thanks in advance
EDIT
Really sorry... I got the old code, someone adds a log statement without noticing me. The log statement can only accept a maximum of 32767 characters in one line, so the error is raised. The above function is executed by a debug tool, which gives the same error, so it's the tool's problem, not oracle.
Thanks for your answering and sorry for my naive mistakes...
I get no errors if the variable to hold the return value is big enough.
declare
rv1 clob;
rv2 varchar2(32000);
begin
rv1 := test(400); -- No error
rv2 := test(200); -- No error - returned XML is < 32000 in length.
rv2 := test(400); -- ORA-06502: PL/SQL: numeric or value error'
end;

Resources