I have a script that extracts documents en masse out of an Oracle BLOB table. This is necessary for a huge rewrite and database conversion from Oracle to SQL where the files are going to be stored in an SQL file table. Since the documents have to sit on the file system, I have to get them out and write them out as files. It works great for MOST of my documents. After lots of banging my head on my desk, I finally figured out it's because there is some logic on the front end system that compresses some of the documents--although I really can't figure out the criteria for it doing that. At any rate, I've searched and searched and can't find any sort of Boolean check to see if they are compressed inside the Oracle BLOB table or not, before I extract them. If I try to decompress them ALL as I'm extracting them, I get an error on the ones that weren't compressed. So now I'm thinking I can run them all with the decompression, and then catch the exception and handle the others by exporting without the decompression. I just can't get my syntax correct in my script. This is a new challenge for me, and I don't have a ton of experience writing scripts of this sort, so please forgive my ignorance. Here's the error I get when I try to decompress all of them, so this is what I'm trying to catch:
ORA-29294: A data error occurred during compression or uncompression.
ORA-06512: at "SYS.UTL_SYS_COMPRESS", line 56
ORA-06512: at "SYS.UTL_SYS_COMPRESS", line 226
ORA-06512: at "SYS.UTL_COMPRESS", line 89
ORA-06512: at line 21
Here is the script:
DECLARE
CURSOR C1 IS Select FILE_ID || '---' || substr(DOCUMENTLOCATION,1,instr (DOCUMENTLOCATION,'.')-1)||'.doc' as FILE_NAME, FILE_BLOB, FILE_ID
From DOCUMENTS d inner join CASEJOURNAL c on d.FILE_ID = c.JOURNALENTRYID where (JOURNAL_ENTRY_TYPE = 117 or JOURNAL_ENTRY_TYPE = 3) AND c.DOCUMENTLOCATION Is Not Null AND d.MIME_TYPE = 'application/msword' AND FILE_ID between 1 and 10000;
v_blob_uncomp BLOB;
v_blob BLOB;
blob_length INTEGER;
out_file UTL_FILE.FILE_TYPE;
v_buffer RAW(32767);
chunk_size BINARY_INTEGER := 32767;
blob_position INTEGER := 1;
filename varchar2(255);
BEGIN
--Select BLOB file into variables
FOR I in C1
LOOP
filename := i.FILE_NAME;
v_blob_uncomp := UTL_COMPRESS.LZ_UNCOMPRESS(i.FILE_BLOB);
v_blob := i.FILE_BLOB;
-- Define the output directory
out_file := UTL_FILE.FOPEN('fileloc',filename,'wb',chunk_size);
--Get length of BLOB file and save to variable.
blob_length := DBMS_LOB.getlength(v_blob);
-- Write the data to the file
WHILE blob_position <= blob_length LOOP
IF blob_position + chunk_size - 1 > blob_length THEN
chunk_size := blob_length - blob_position + 1;
END IF;
DBMS_LOB.read(v_blob_uncomp, chunk_size, blob_position, v_buffer);
UTL_FILE.PUT_RAW(out_file, v_buffer, TRUE);
blob_position := blob_position + chunk_size;
END LOOP;
UTL_FILE.FCLOSE(out_file);
END LOOP;
END;
I know that the script works when I don't decompress any blobs, however the ones that were compressed don't open. It also works when I do the decompress on certain files that I know were compressed. I'm just trying to get this to work within my loop for ALL files somehow. TIA!
When you want to catch a particular Oracle error code in PL/SQL, you basically have two options:
A) catch all exceptions; in the handler, test whether the error message matches the one you are looking for; if so, handle it; if not, re-raise it. This would look something like:
BEGIN
v_blob := := UTL_COMPRESS.LZ_UNCOMPRESS(i.FILE_BLOB);
EXCEPTION
WHEN OTHERS THEN
IF sqlerrm LIKE 'ORA-29294%' THEN
v_blob := i.FILE_BLOB;
ELSE
RAISE;
END IF;
END;
B) declare an exception variable and map it to the specific error code you care about, then catch only that exception. This would look something like this:
DECLARE
compression_error EXCEPTION;
pragma exception_init ( compression_error, -29294 );
BEGIN
v_blob := UTL_COMPRESS.LZ_UNCOMPRESS(i.FILE_BLOB);
EXCEPTION
WHEN compression_error THEN
v_blob := i.FILE_BLOB;
END;
Either way, I'd suggest wrapping this in a function.
I also note that your code shown doesn't reset blob_position to 1 when it starts processing a new BLOB.
Instead of "try and fail" approach you may consider to use magic numbers:
Read few bytes from the beginning of the file, if it starts, lets say with 504B (PK), then there is a fat chance that it would be zip archive.
Related
I have a procedure in which I'm trying to write a source code (1290 lines) to dbms_output like this:
dbms_output.put_line(DBMS_METADATA.GET_DDL('FUNCTION', 'name', 'owner')); --MYPROC, line 6
I'm getting :
ORA-06502: PL/SQL: numeric or value error
ORA-06512: in "MYPROC", line 6
.
This error occures in toad.
I can execute in editor tab of toad:
SELECT DBMS_METADATA.GET_DDL('FUNCTION', 'name', 'owner') FROM DUAL;
I mean I'm getting the source code in 'Data grid'.
Same happens when I try to store the code in a CLOB variable:
src CLOB;
...
src := DBMS_METADATA.GET_DDL('FUNCTION', 'name', 'owner') ; --MYPROC, line 6
Any clue?
From the documentation for dbms_output:
The maximum line size is 32767 bytes.
That means that you can't pass more than that in a single put_line call. You are currently passing your whole CLOB, which at 1290 lines is likely to exceed that limit. And the error you get when you do that is "ORA-06502: PL/SQL: numeric or value error", as you are seeing.
You can split your CLOB into smaller chunks, and as it is already multiple lines it makes sense to make each chunk a single line from the DDL. You can do that by looking for newline characters, extracting all the text up to the next one, and printing that. You need a few variables to keep track of where you are. Something like this should work for you:
declare
src clob;
src_length pls_integer;
pos pls_integer := 1;
buffer varchar2(32767);
amount pls_integer := 32767;
begin
src := dbms_metadata.get_ddl('FUNCTION', 'TEST_FUNCTION_1', user);
src_length := dbms_lob.getlength(src);
while pos < src_length loop
-- read to next newline if there is one, rest of CLOB if not
if dbms_lob.instr(src, chr(10), pos) > 0 then
-- see how many charcaters there are until next newline
amount := dbms_lob.instr(src, chr(10), pos) - pos;
-- if there are any, read them into the buffer; otherwise clear it
if amount > 0 then
dbms_lob.read(src, amount, pos, buffer);
else
buffer := null;
end if;
pos := pos + amount + 1; -- skip newline character
else
-- no newline so read everything that is left
amount := 32767;
dbms_lob.read(src, amount, pos, buffer);
pos := pos + amount;
end if;
dbms_output.put_line(buffer);
end loop;
end;
/
It won't work if you have a single line (without or without a newline at the end) that is more than 32k, which hopefully won't be an issue with DDL. (You could sort of handle it, but doing so would inject additional newlines, which wouldn't be good either.)
What you are saying can't be true. DBMS_OUTPUT.PUT_LINE can't be used at SQL level, it belongs to PL/SQL.
What is MYPROC and what does it contain at line #6?
Which "editor" is "I can execute in editor"?
Numeric or value error is usually related to the fact that you're trying to store "large" values into a "small" variable:
SQL> declare
2 l_var varchar2(2);
3 begin
4 l_var := 'ABC';
5 end;
6 /
declare
*
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error: character string buffer too small
ORA-06512: at line 4
SQL>
which is what - I presume - you did.
Another cause is wrongly declared variable, e.g.
SQL> declare
2 l_var number;
3 begin
4 l_var := 'A';
5 end;
6 /
declare
*
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error: character to number conversion error
ORA-06512: at line 4
SQL>
I'll try to guess what you might be doing:
SQL> set serveroutput on
SQL> DECLARE
2 src CLOB;
3 BEGIN
4 src := DBMS_METADATA.GET_DDL ('PACKAGE', 'MY_PKG', 'SCOTT');
5 DBMS_OUTPUT.put_line ('len = ' || DBMS_LOB.getlength (src));
6 END;
7 /
len = 67239
PL/SQL procedure successfully completed.
SQL>
As you can see, it works OK for me. Package isn't that small (see its length), so - can't really tell what you did wrong. I'd suggest you to do exactly as I did - copy/paste code I posted above (those 7 lines), fix information (function, its name, owner) and post the result by editing the original question, not as a comment.
create or replace directory MYCSV as 'E:\sqlloader\';
grant read, write on directory MYCSV to public;
declare
F UTL_FILE.FILE_TYPE;
V_LINE VARCHAR2 (1000);
V_id NUMBER(4);
V_NAME VARCHAR2(10);
V_risk VARCHAR2(10);
BEGIN
F := UTL_FILE.FOPEN ('MYCSV', 'testfile.csv', 'R');
IF UTL_FILE.IS_OPEN(F) THEN
LOOP
BEGIN
UTL_FILE.GET_LINE(F, V_LINE, 1000);
IF V_LINE IS NULL THEN
EXIT;
END IF;
V_id := REGEXP_SUBSTR(V_LINE, '[^,]+', 1, 1);
V_NAME := REGEXP_SUBSTR(V_LINE, '[^,]+', 1, 2);
V_risk := REGEXP_SUBSTR(V_LINE, '[^,]+', 1, 3);
INSERT INTO loader_tab VALUES(V_id, V_NAME, V_risk);
COMMIT;
EXCEPTION
WHEN NO_DATA_FOUND THEN
EXIT;
END;
END LOOP;
END IF;
UTL_FILE.FCLOSE(F);
END;
/
CSV file content wherein I need to start loading from 1,a,aa and need to skip first 4 lines:
portal,,
ex portal,,
,,
i_id,i_name,risk
1,a,aa
2,b,bb
3,c,cc
4,d,dd
5,e,ee
6,f,ff
7,g,gg
8,h,hh
9,i,ii
10,j,jj
I want to load the data from excel but I am getting an invalid file operation error. Will someone help with this? Not able to load the data from an excel file. I am Getting invalid file operation error though file is present in my local system.
though file is present in my local system
It won't work unless your local system (I presume you mean your own PC) also runs the database into which you're trying to load data. Oracle directory (probably in 99% of all cases) resides on the database server.
I want to load the data from excel
It won't work either, if that's really an Excel file. Code you posted suggests that it is a comma-separated values file (textual, that is), and yes - it should be such a file, not XLSX.
I am having one program in Oracle PL/SQL.The program does some batch processing ie. it sends data to another system through REST API in batches of fixed number of records. the request and response object are clob and hence i am creating temporary lob and freeing it for each iteration.
My question is ,can't i create temp lob once and resuse it for every batch i process and then free it at last only once. Basically i want to bring create and free out of the loop so that it can improve performance and reuse the memory.
When i try to bring it outside loop, i will need to initialize clob variable at the start of each iteration, so i tried it using empty_clob() but did not work.Also assigning null does not work.
I am getting error as "Invalid lob locator specified at ..."
Below is my pseudo code
for i in start_batch to end_batch
loop
dbms_lob.createtemporary(l_clob,TRUE);
...code to generate request object.
dbms_lob.freetemporary(l_clob,TRUE) ;
end loop
Huh. I swear that worked, but you are correct. I shouldn't try to remember these things. I guess assigning '' to a clob does set it to null. You can't use a null clob with dbms_lob.append, since it's expecting basically a pointer. Try using the concatenation operator, ||.
I've confirmed this works:
declare
l_clob clob;
begin
for i in 1..5 loop
l_clob := '';
for j in 1..5 loop
l_clob := l_clob || 'a';
end loop;
dbms_output.put_line(l_clob);
end loop;
end;
Edit:
I'm not sure it's true that a clob concatenated with a varchar is a varchar and therefore limited to 32 kB. But that does contradict what the documentation says. Take this for example:
declare
c clob;
begin
for i in 1..40000 loop
c := c || 'a';
end loop;
dbms_output.put_line('len=' || dbms_lob.getlength(c));
end;
Result:
len=40000
I am trying to find a way to get files from the database server via APEX. I can't find any documentation about this issue.
Can I avoid using the plsql code?
PS: I'm launching APEX on tomcat on different server that db is.
As we talked on the comments, let me show you an example. I write you a lot of thing so take your time...
Upload file
You can have an object in Apex that allows the user to browse for a file to be uploaded. At the end, the user press a button that triggers the action . The button upload submits the page, and the action is after submit for the button itself. Any file upload is stored automatically in apex_application_temp_files ( keep in mind I removed a lot of controls I have regarding format of the file, size, etc ).
First create the directory
create or replace directory yourdirectory as '/your_path' ;
grant read, write on directory yourdirectory to your_user ;
The code in the button:
declare
v_error VARCHAR2(400);
v_filename VARCHAR2(400);
v_name VARCHAR2(400);
v_blob blob;
vodate number(8);
begin
SELECT filename,blob_content,name, to_number(regexp_replace(filename,'[0-9]{4}[0-9]{2}[0-9]{2}'))
INTO v_filename,v_blob,v_name,vodate
FROM apex_application_temp_files
WHERE name = :P2_FILE;
apex_debug.enable ( p_level => 5 );
apex_debug.message(p_message => 'v_filename is '||v_filename||' ', p_level => 5) ;
apex_debug.message(p_message => 'v_name is '||v_name||' ', p_level => 5) ;
apex_debug.message(p_message => 'vodate is '||to_number(substr(v_filename,14,8)) ||' ', p_level => 5) ;
-- insert into filesystem
p_write_blob_to_file(p_name=>v_name);
EXCEPTION
WHEN OTHERS THEN
raise;
end;
The important part here is the code p_write_blob_to_file. This is the code of that procedure, keeping in consideration that in my case p_dir takes a default value.
CREATE OR REPLACE procedure p_write_blob_to_file (p_name IN VARCHAR2, p_dir IN VARCHAR2 default 'your_directory' )
IS
l_blob BLOB;
l_blob_length INTEGER;
l_out_file UTL_FILE.file_type;
l_buffer RAW (32767);
l_chunk_size BINARY_INTEGER := 32767;
l_blob_position INTEGER := 1;
l_file_name varchar2(2000);
v_mime_type varchar2(2000);
BEGIN
-- Retrieve the BLOB for reading
SELECT blob_content, filename, mime_type
INTO l_blob, l_file_name, v_mime_type
FROM apex_application_temp_files
WHERE name = p_name;
-- Retrieve the SIZE of the BLOB
l_blob_length := DBMS_LOB.getlength (l_blob);
-- Open a handle to the location where you are going to write the BLOB
-- to file.
l_out_file :=
UTL_FILE.fopen (p_dir,
l_file_name,
'wb',
l_chunk_size);
-- Write the BLOB to file in chunks
WHILE l_blob_position <= l_blob_length
LOOP
IF l_blob_position + l_chunk_size - 1 > l_blob_length
THEN
l_chunk_size := l_blob_length - l_blob_position + 1;
END IF;
DBMS_LOB.read (l_blob,
l_chunk_size,
l_blob_position,
l_buffer);
UTL_FILE.put_raw (l_out_file, l_buffer, TRUE);
l_blob_position := l_blob_position + l_chunk_size;
END LOOP;
-- Close the file handle
UTL_FILE.fclose (l_out_file);
END p_write_blob_to_file;
/
Download File
In order to download the file, you need the opposite path.
The button or link download must be associated to a component PL/SQL
The button loads the file first from the directory in the server into a column
The button then download the file
I was going to write all the commands here , but you have a find very good example here of both actions:
Load file from directory into column
https://renaps.com/en/blog/how-to/how-to-load-file-content-to-a-blob-field-and-unload-blob-content-to-a-file-on-the-os
Download file
https://oracle-base.com/articles/misc/apex-tips-file-download-from-a-button-or-link#apex-button
Try to experiment with this and let me know any issues you might find. The only tricky thing here is that in Apex you need to pass the name of the file you want to download. So the user must know the name, exactly as it is in the server. What you can't do is provide a graphical interface to the server in order to select the file.
I've got a question how to import a .csv file into a Oracle Forms application.
We are using Oracle Forms 11g on a Oracle 12c Database.
Now we want to import a .csv file with the Forms applicationso our customers can import this file and write the data into the database.
My plan is to create an application where the user can import a .csv file with a filechooser. The data from the .csv will be read and an output shows the user the data in this application. Then the user should be able to save it into the database through a button.
I've tried several searches but haven't found the right solution for this kind of problem. The only solutions I've found were a direct import of a .csv file into a database but not through Oracle Forms
Is it even possible to load .csv files in Oracle Forms?
If anyone has a good solution or anything else that might be helpfull i would be thankfull for that.
I've found a suitable solution for me now. Maybe I could do it better but this helps me so far.
I've found the following Blog: http://tfathy.blogspot.de/2009/03/reading-from-file.html
The Code works 100% by copy&paste and helps me from now on to complete the task.
Maybe it might help anyone else too.
Here is the solutioncode:
Reading From File
March 19, 2009
--------------------------------------------------
Declare
vfilename varchar2(500);
in_file Client_Text_IO.File_Type;
linebuf VARCHAR2(1800);
BEGIN
vfilename := client_get_file_name('c:/temp/', File_Filter=>'Comma Dialimeted Files (*.csv)|*.csv|');
in_file := client_Text_IO.Fopen(vfilename, 'r');
GO_BLOCK('Emp');
FIRST_RECORD;
LOOP
Client_Text_IO.Get_Line(in_file, linebuf);
p_output_line(linebuf);
Client_Text_IO.New_Line;
Next_record;
END LOOP;
FIRST_RECORD;
EXCEPTION
WHEN no_data_found THEN
Client_Text_IO.Put_Line('Closing the file...');
Client_Text_IO.Fclose(in_file);
END;
-------------------------------------------------------
PROCEDURE p_output_line(p_line varchar2) IS
vLINE VARCHAR2(4000);
vVALUE VARCHAR2(1000);
vCOMMA_COUNT NUMBER;
vREPORT_DATE DATE;
BEGIN
vLINE := p_line;
vCOMMA_COUNT := LENGTH(vLINE)- LENGTH(REPLACE(vLINE,',','')); -- COUNT THE NUMBER OF COMMAS
FOR I IN 1.. vCOMMA_COUNT+1 LOOP
vVALUE := SUBSTR(vLINE,1,INSTR(vLINE,',')-1); -- IF vLINE = 123,ABC,9877 THEN VVALUE WILL BE 123
IF vVALUE IS NULL THEN
vVALUE := vLINE;
END IF;
vLINE := SUBSTR(vLINE,INSTR(vLINE,',')+1) ; -- CHANGE 123,ABC,9877 TO BE ABC,9877
IF I = 1 THEN
:DATA.BMK_NAME := vVALUE;
ELSIF I = 2 THEN
vREPORT_DATE := last_day(to_date(vVALUE,'dd-mm-yyyy'));
:DATA.REPORT_DATE := vREPORT_DATE;
ELSIF I = 3 THEN
:DATA.BMK_RETURN := to_number(vVALUE);
END IF;
END LOOP;
EXCEPTION
WHEN NO_DATA_FOUND THEN
MESSAGE('Please Check the data type is appropriate on you excel file');
MESSAGE('Please Check the data type is appropriate on you excel file');
END;
-----------------------------------------------------------------------
-- notes
1- you must install webutil version 106 or later
2- make sure that you attached and compiled the webutill.pll scucessfuly
You can just use webutil to show a filechooser to select the file and upload it to the application server. If you use a shared directory between the application server and the db server you can use an external table to show the input of the file to the user. And then after the button you just insert the data from the external table in another table.