I would like to perform a BULK copy with this code, please? - oracle

I think this one will do the job of writing all the files to a directory all at once if I can get past the "invalid operation error"
Your help, as always, is greatly appreciated.
create or replace
PROCEDURE GetbFile
IS
l_output utl_file.file_type;
vstart NUMBER := 1;
bytelen NUMBER := 32000;
x NUMBER;
my_vr RAW(32000);
v_name VARCHAR2(32760);
BEGIN
FOR recFiles IN (SELECT dbms_lob.getlength(BLOB_VALUE) as len,
FILE_NAME,
BLOB_VALUE from Gfile)
LOOP
l_output := utl_file.fopen('THE_DIR', 'file_name'||'.dot', 'w', 32760);
IF recFiles.len < 32760 THEN
utl_file.put_raw(l_output, recFiles.BLOB_VALUE);
utl_file.fflush(l_output);
ELSE -- write in pieces
vstart := 1;
WHILE vstart < recFiles.len
LOOP
dbms_lob.read(recFiles.BLOB_VALUE, bytelen, vstart, my_vr);
utl_file.put_raw(l_output, my_vr);
utl_file.fflush(l_output);
-- set the start position for the next cut
vstart := vstart + bytelen;
-- set the end position if less than 32000 bytes
x := x - bytelen;
IF x < 32000 THEN
bytelen := x;
END IF;
END LOOP;
END IF;
End Loop;
dbms_output.put_line('End');
utl_file.fclose(l_output);
END GetFile;

Why did you replace the prior version of the code with this version? If you already have a version of the code that is working to write a single BLOB to the file system, it's very easy to just call that code in a loop. It's also a better way of designing modular code.
When you get an error, please post the error stack. That will include the Oracle error number, the error message, and the line number of the error. Tell us what line that corresponds to in your code (particularly if there are formatting differences between what you post here and what code you actually run).
You cannot in a single thread copy every LOB to a file at the same time. A single thread can do one thing at a time so it can copy one file at a time. You can loop so that you copy each file sequentially. I'm still not clear whether that is what you want to do or if you really want to spawn 800 threads each of which writes a single LOB to the file system.
You need to close the file inside the loop since you open the file in the loop (note that keeping the old code would make it much easier to avoid this sort of error). And assuming that you want to use the file name from the table, you'd want to use recFiles.file_name in your call to fopen, not a hard-coded string 'file_name' which would try to write every LOB to the same physical file.
Given that, my guess is that you want something like this (note that it would still be better form to modularize this code but since you're trying to avoid that, I'll assume you have a good reason for that)
create or replace
PROCEDURE GetbFile
IS
l_output utl_file.file_type;
vstart NUMBER := 1;
bytelen NUMBER := 32000;
x NUMBER;
my_vr RAW(32000);
v_name VARCHAR2(32760);
BEGIN
FOR recFiles IN (SELECT dbms_lob.getlength(BLOB_VALUE) as len,
FILE_NAME,
BLOB_VALUE from Gfile)
LOOP
l_output := utl_file.fopen('THE_DIR', recFiles.file_name||'.dot', 'w', 32760);
IF recFiles.len < 32760 THEN
utl_file.put_raw(l_output, recFiles.BLOB_VALUE);
utl_file.fflush(l_output);
ELSE -- write in pieces
vstart := 1;
WHILE vstart < recFiles.len
LOOP
dbms_lob.read(recFiles.BLOB_VALUE, bytelen, vstart, my_vr);
utl_file.put_raw(l_output, my_vr);
utl_file.fflush(l_output);
-- set the start position for the next cut
vstart := vstart + bytelen;
-- set the end position if less than 32000 bytes
x := x - bytelen;
IF x < 32000 THEN
bytelen := x;
END IF;
END LOOP;
END IF;
utl_file.fclose(l_output);
End Loop;
dbms_output.put_line('End');
END GetFile;

Related

PL SQL output in hackerrank

set serveroutput on;
DECLARE
I NUMBER;
J NUMBER;
BEGIN
FOR I IN REVERSE 1..20
LOOP
FOR J IN 1..I
LOOP
DBMS_OUTPUT.PUT('* ') ; -- printing *
END LOOP;
DBMS_OUTPUT.NEW_LINE; -- for new line
END LOOP;
END;
'
Can anyone tell me why this code is not showing any output in the Hackerank question (Draw The Triangle) even after selecting Oracle?
https://www.hackerrank.com/challenges/draw-the-triangle-1/problem
That site doesn't seem to ever show your the output from your submission, unhelpfully, but does with just 'run code'. Surprisingly it does seem to understand PL/SQL; and even more surprisingly it handles the set serveroutput on, which is a SQL\Plus/SQL Developer client command.
But you need to add a terminating / after your code, on a line on its own - again, just like SQL*Plus (though SQL Developer is sometimes doesn't complain):
END LOOP;
END;
/
Your code doesn't produce the expected output because it has a trailing space on each line. Instead of:
DBMS_OUTPUT.PUT('* ') ; -- printing *
skip the space on the last iteration:
DBMS_OUTPUT.PUT('*') ; -- printing *
IF J < I THEN
DBMS_OUTPUT.PUT(' ');
END IF;
So this produces the expected output, and passes the test:
set serveroutput on;
DECLARE
I NUMBER; -- redundant
J NUMBER; -- redundant
BEGIN
FOR I IN REVERSE 1..20
LOOP
FOR J IN 1..I
LOOP
DBMS_OUTPUT.PUT('*') ; -- printing *
IF J < I THEN
DBMS_OUTPUT.PUT(' ');
END IF;
END LOOP;
DBMS_OUTPUT.NEW_LINE; -- for new line
END LOOP;
END;
/
However, your original code also passes the test, despite the trailing spaces, if you just add the terminating /:

Oracle PL/SQL speed of NVL/LENGTH/TRIM calls versus IS NOT NULL AND != ' '

I try to find the best way to check if a CHAR/VARCHAR2 variable contains characters (NULL or spaces should be considered the same, as "no-value"):
I know there are several solutions, but it appears that (NVL(LENGTH(TRIM(v)),0) > 0) is faster than (v IS NOT NULL AND v != ' ')
Any idea why? Or did I do something wrong in my test code?
Tested with Oracle 18c on Linux, UTF-8 db charset ...
I get the following results:
time:+000000000 00:00:03.582731000
time:+000000000 00:00:02.494980000
set serveroutput on;
create or replace procedure test1
is
ts timestamp(3);
x integer;
y integer;
v char(500);
--v varchar2(500);
begin
ts := systimestamp;
--v := null;
v := 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';
for x in 1..50000000
loop
if v is not null and v != ' ' then
y := x;
end if;
end loop;
dbms_output.put_line('time:' || (systimestamp - ts) ) ;
end;
/
create or replace procedure test2
is
ts timestamp(3);
x integer;
y integer;
v char(500);
--v varchar2(500);
begin
ts := systimestamp;
--v := null;
v := 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa';
for x in 1..50000000
loop
if nvl(length(trim(v)),0) > 0 then
y := x;
end if;
end loop;
dbms_output.put_line('time:' || (systimestamp - ts) ) ;
end;
/
begin
test1();
test2();
end;
/
drop procedure test1;
drop procedure test2;
quit;
The best practice is to ignore the speed difference between small functions and use whatever is easiest.
In realistic database programming, the time to run functions like NVL or IS NOT NULL is completely irrelevant compared to the time needed to read data from disk or the time needed to join data. If one function saves 1 seconds per 50 million rows, nobody will notice. Whereas if a SQL statement reads 50 million rows with a full table scan instead of using an index, or vice-versa, that could completely break an application.
It's unusual to care about these kinds of problems in a database. (But not impossible - if you have a specific use case, then please add it to the question.) If you really need optimal procedural code you may want to look into writing an external procedure in Java or C.

File corruption with UTL_FILE script

I have definitely searched FAR and WIDE for an answer to this, but I can't find anything! I am using a UTL_FILE script to pull down some file BLOBS from an Oracle table and save them to a file directory. It's working for a lot of the files, but I have narrowed it down by process of elimination that it's having issues for files that have an "unconventional" file name, albeit still a valid one, the files are getting corrupted in the transfer. They may have only been 30kb originally, but export as 5kb and cannot be opened. So I know it's not a large file size issue. The files open just fine through the application, have a valid MIME type encoding, and would otherwise open fine on a file system, but UTL_FILE doesn't seem to like them. They are files that have an extra "." in them ie: john.smith.doc, or a pound sign ie: Smith #12345.doc or parentheses, etc. I cannot change the source file names in the Oracle table, but I have been concatenating an ID number on to them when saving them out so I can reference it as a key for an ETL load into SQL file table later. Maybe I also need to write a complicated REGEXP to rename the files on the fly and strip out the bad characters, but I'm not sure that will work because I don't know at what point UTL_FILE is choking on them. If it's at the source, then that won't help. Has anyone else encountered this problem? Here is my script:
DECLARE
CURSOR C1 IS Select FILE_ID || '---' || substr(DOCUMENTLOCATION,1,instr
(DOCUMENTLOCATION,'.')-1)||'.doc' as FILE_NAME, FILE_BLOB, FILE_ID
From DOCUMENTS d inner join CASEJOURNAL c on d.FILE_ID = c.JOURNALENTRYID
where (JOURNAL_ENTRY_TYPE = 117 or JOURNAL_ENTRY_TYPE = 3) AND
c.DOCUMENTLOCATION Is Not Null AND d.MIME_TYPE = 'application/msword'
AND FILE_ID BETWEEN 785 AND 3380;
l_file UTL_FILE.FILE_TYPE;
l_buffer RAW(32000);
l_amount INTEGER := 32000;
l_pos INTEGER := 1;
l_blob BLOB;
l_blob_len INTEGER;
l_filename varchar2(255);
BEGIN
--Select BLOB file into variables
FOR I in C1
LOOP
Select FILE_ID || '---' || substr(DOCUMENTLOCATION,1,instr
(DOCUMENTLOCATION,'.')-1) ||'.doc' as FILE_NAME, FILE_BLOB INTO l_filename,
l_blob From DOCUMENTS d inner join CASEJOURNAL c on d.FILE_ID =
c.JOURNALENTRYID where (JOURNAL_ENTRY_TYPE = 117 or JOURNAL_ENTRY_TYPE =
3) AND c.DOCUMENTLOCATION Is Not Null AND d.MIME_TYPE
= 'application/msword' and d.FILE_ID = I.FILE_ID;
-- Define the output directory
l_file := UTL_FILE.FOPEN('\\myfiledirectory',l_filename,'wb',32000);
l_pos := 1;
l_amount := 32000;
--Get length of BLOB file and save to variable.
l_blob_len := DBMS_LOB.getlength(l_blob);
-- Write the data to the file
--If small enough for single write:
IF l_blob_len < 32000 THEN
UTL_FILE.PUT_RAW (l_file, l_blob);
UTL_FILE.FFLUSH(l_file);
--Write in pieces if larger than 32k
ELSE
l_pos := 1;
WHILE l_pos < l_blob_len AND l_amount > 0
LOOP
DBMS_LOB.read(l_blob, l_amount, l_pos, l_buffer);
UTL_FILE.PUT_RAW(l_file, l_buffer);
UTL_FILE.FFLUSH(l_file);
--Set start position for next write
l_pos := l_pos + l_amount;
--Set end position if less than 32k.
l_blob_len := l_blob_len - l_amount;
IF l_blob_len < 32000 THEN
l_amount := l_blob_len;
END IF;
END LOOP;
END IF;
UTL_FILE.FCLOSE(l_file);
END LOOP;
END;
The file name isn't going to affect how the bytes are written out once the file has been opened. You seem to be truncating the file if it's more than 32k. Your loop does this:
WHILE l_pos < l_blob_len AND l_amount > 0
LOOP
... but then you change both l_pos and l_blob_len within the loop; once the adjusted l_pos falls below the remaining l_blob_len you exit the loop, too early. You don't need to adjust l_blob_len, or even adjust l_amount - that is the maximum number of bytes to read, it doesn't matter if it's higher than what's left.
So change the loop to:
WHILE l_pos < l_blob_len AND l_amount > 0
LOOP
DBMS_LOB.read(l_blob, l_amount, l_pos, l_buffer);
UTL_FILE.PUT_RAW(l_file, l_buffer);
UTL_FILE.FFLUSH(l_file);
--Set start position for next write
l_pos := l_pos + l_amount;
END LOOP;
Not really related to your problem, but you don't need to reselect the data inside your cursor loop. You've already got the values you need in your i cursor variable, so you can do:
FOR I in C1
LOOP
l_filename := i.file_name;
l_blob := i.file_blob;
-- Define the output directory
...
Or don't bother with the l_filename and l_blob local variables at all; since you only refer to them inside the cursor loop anyway, use i.file_name and i.file_blob directly everywhere, e.g.
l_file := UTL_FILE.FOPEN('\\myfiledirectory',i.file_name,'wb',32000);
l_blob_len := DBMS_LOB.getlength(i.file_blob);
etc.

How do I fix "numeric or value error" message?

I have been stuck with this issue now all morning. I actually saw this code here and decide to use it for our purpose here.
The issue that I am running into is that when we execute the code, sometimes it writes a file from db to the folder.
Other times, we get "numeric or value error"
Can any expert please help me fix it?
Here is the code I am using:
create or replace
PROCEDURE getfile(pfname VARCHAR2, display_name IN VARCHAR2)
IS
vblob BLOB;
vstart NUMBER := 1;
bytelen NUMBER := 32000;
len NUMBER;
my_vr RAW(32000);
x NUMBER;
v_name VARCHAR2(32760);
lv_str_len NUMBER;
l_output utl_file.file_type;
BEGIN
-- define output directory
--lv_str_len := Length(pfname);
--v_name := display_name||upper(substr(pfname,lv_str_len-3,lv_str_len));
v_name := display_name;
l_output := utl_file.Fopen('My_DIR', v_name, 'w', 32760);
-- get length of blob
SELECT dbms_lob.Getlength(FILENAME)
INTO len
FROM GENERAL.GUBFILE
WHERE gubfile_name = pfname;
-- dbms_output.put_line('Length: '||len);
-- save blob length
x := len;
-- select blob into variable
SELECT BLOBVALUE
INTO vblob
FROM FILES
WHERE filename = pfname;
-- if small enough for a single write
IF len < 32760 THEN
-- dbms_output.put_line('Single write ');
utl_file.Put_raw(l_output, vblob);
utl_file.Fflush(l_output);
ELSE -- write in pieces
-- dbms_output.put_line('multi write '||vstart);
vstart := 1;
WHILE vstart < len LOOP
dbms_lob.READ(vblob, bytelen, vstart, my_vr);
utl_file.Put_raw(l_output, my_vr);
utl_file.Fflush(l_output);
-- set the start position for the next cut
vstart := vstart + bytelen;
-- set the end position if less than 32000 bytes
x := x - bytelen;
IF x < 32000 THEN
bytelen := x;
END IF;
END LOOP;
END IF;
dbms_output.Put_line('End');
utl_file.Fclose(l_output);
END getfile;
The exact error is:
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at "USER.GETFILE", line 40
ORA-06512: at line 8
The error comes from utl_file.put_raw.
The maximum size of the buffer parameter is 32767 bytes.
You check for IF len < 32760 THEN, however, I see no guarantee in your code, that the len variable actually holds the length of the vblob variable that is the buffer in the put_raw call.
So I suppose the vblob variable's actual length is longer then 32767 and that is the reason for the error.
Hence I suggest to delete this piece of code:
IF len < 32760 THEN
-- dbms_output.put_line('Single write ');
utl_file.Put_raw(l_output, vblob);
utl_file.Fflush(l_output);
ELSE
also the END IF; of course, and always go for the 'write in pieces' branch.
I see now, you've done this based on the Burleson example which is good http://www.dba-oracle.com/t_writing_blob_clob_os_file.htm
but you see, unlike you, Burleson gets the len variable and the vblob variable from the same table and the same field.
-- get length of blob
SELECT dbms_lob.getlength(productblob)
INTO len
FROM products
WHERE id = product_id;
-- save blob length
x := len;
-- select blob into variable
SELECT product_blob
INTO vblob
FROM products
WHERE id = product_id;
EDIT
So an other option would be to fix the select for getting length. This means you'll have to replace this select:
-- get length of blob
SELECT dbms_lob.Getlength(FILENAME)
INTO len
FROM GENERAL.GUBFILE
WHERE gubfile_name = pfname;
with this:
-- get length of blob
SELECT dbms_lob.Getlength(BLOBVALUE)
INTO len
FROM FILES
WHERE filename = pfname;

Do I really need the parameters in this stored procedure?

The following stored procedure is intended to grab all the BLOB values from an Oracle database and save them into a folder called OraFolder.
It compiles fine but I have 2 questions.
1, there are 2 parameters, pname and display_name. I must admit that I don't know what they are there for because I just googled the code which seems to fit into our need.
My question is do I really need the 2 params given that we are trying to extract ALL BLOB values into a folder?
2, If your answer is yes, I do need them, how I do I use them?
Finally, there is an Entry_Id, I just kept getting an error that it is not declared. I had to remove it. What is it used for?
Sorr, I am not an Oracle guy, just trying to figure out a wa to fix a problem that is dropped on my laps.
Thanks in advance
Here is the complete stored proc.
create or replace PROCEDURE blob2file(pfname VARCHAR2, display_name in varchar2) IS
vblob BLOB;
vstart NUMBER := 1;
bytelen NUMBER := 32000;
len NUMBER;
my_vr RAW(32000);
x NUMBER;
v_name varchar2(100);
lv_str_len NUMBER;
l_output utl_file.file_type;
BEGIN
-- define output directory
lv_str_len := length(pfname);
--v_name := display_name||upper(substr(pfname,lv_str_len-3,lv_str_len));
v_name := display_name;
l_output := utl_file.fopen('MY_FOLDER', v_name, 'w', 32760);
-- get length of blob
SELECT dbms_lob.getlength(blob_content)
INTO len
FROM portal.WWDOC_DOCUMENT$
WHERE FILENAME = pfname;
-- dbms_output.put_line('Length: '||len);
-- save blob length
x := len;
-- select blob into variable
SELECT blob_content
INTO vblob
FROM portal.WWDOC_DOCUMENT$
WHERE FILENAME = pfname;
-- if small enough for a single write
IF len < 32760 THEN
-- dbms_output.put_line('Single write ');
utl_file.put_raw(l_output,vblob);
utl_file.fflush(l_output);
ELSE -- write in pieces
-- dbms_output.put_line('multi write '||vstart);
vstart := 1;
WHILE vstart < len
LOOP
dbms_lob.read(vblob,bytelen,vstart,my_vr);
utl_file.put_raw(l_output,my_vr);
utl_file.fflush(l_output);
-- set the start position for the next cut
vstart := vstart + bytelen;
-- set the end position if less than 32000 bytes
x := x - bytelen;
IF x < 32000 THEN
bytelen := x;
END IF;
END LOOP;
END IF;
dbms_output.put_line('End');
utl_file.fclose(l_output);
EXCEPTION
when others then dbms_output.put_line('ERROR:'||entry_id);
END blob2file;
The pfname param defines which file to grab, so you don't need that since you want them all.
The display_name param defines the output directory, so if you want to hard code the directory, you could.
Since you want all files, you'll need to loop through all records in the table and output them one at a time:
CREATE OR REPLACE PROCEDURE blob2file
IS
l_output utl_file.file_type;
vstart NUMBER := 1;
bytelen NUMBER := 32000;
x NUMBER;
my_vr RAW(32000);
BEGIN
FOR recFiles IN (SELECT dbms_lob.getlength(BLOB_CONTENT) as len,
FILENAME,
BLOB_CONTENT
FROM PORTAL.WWDOC_DOCUMENT$)
LOOP
l_output := utl_file.fopen('MY_FOLDER', '/hard code the path here/', 'w', 32760);
IF recFiles.len < 32760 THEN
utl_file.put_raw(l_output, recFiles.BLOB_CONTENT);
utl_file.fflush(l_output);
ELSE -- write in pieces
vstart := 1;
WHILE vstart < refFiles.len
LOOP
dbms_lob.read(recFiles.BLOB_CONTENT, bytelen, vstart, my_vr);
utl_file.put_raw(l_output, my_vr);
utl_file.fflush(l_output);
-- set the start position for the next cut
vstart := vstart + bytelen;
-- set the end position if less than 32000 bytes
x := x - bytelen;
IF x < 32000 THEN
bytelen := x;
END IF;
END LOOP;
END IF;
utl_file.fclose(l_output);
dbms_output.put_line('End');
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line('ERROR: ' || SQLERRM);
END blob2file;
Looks like vname is the name of the file which the blob is written to. And pname is the key of the blob in the table. So if you're dumping all blobs in the table then you don't need either of these, but you will need to come up with a unique filename for each blob.

Resources