Hung up on For In Loop and variable for external table loader - oracle

Question

I have to load a number of files every day into our database system. My solution was to use a java procedure to generate a table of all the files in the directory folder and loop through each of them through the external table loader. I'm running into two hangups with this
Declare
what_to_load VARCHAR2(255);
CURSOR folder_contents
IS
select filename
from database.DIR_LIST
where filename like 'DCOpenOrders_%'
and filename like '%.csv';
BEGIN
DELETE FROM database.DIR_LIST;
database.GET_DIR_LIST( 'directory_path_files_are_in' );
FOR each_record IN folder_contents
LOOP
what_to_load := each_record.filename;
EXECUTE IMMEDIATE 'DROP table database.my_table';
execute immediate 'CREATE table database.my_table
(Region VARCHAR2(10),
District VARCHAR2(10),
Originating_Store VARCHAR2(80),
Order_Date VARCHAR2(30),
Ship_Location VARCHAR2(10),
Orig_Ord_No VARCHAR2(30),
Field_G VARCHAR2(30),
Line_No VARCHAR2(10),
POS_UPC VARCHAR2(30),
Item_Descr VARCHAR2(80),
Ord_Qty VARCHAR2(10),
Line_Status VARCHAR2(30),
Report_Date VARCHAR2(30),
Ship_Type VARCHAR2(30),
ERR_FLAG VARCHAR2(10),
ERR_LOG VARCHAR2(800)
)
ORGANIZATION EXTERNAL
( type oracle_loader
default directory WORK_DIR
access parameters
( records delimited by NEWLINE
skip 1
fields terminated by '',''
optionally enclosed by ''"''
missing FIELD VALUES are NULL)
location ('''||each_record.filename||''')
)
reject limit unlimited';
Execute Immediate 'Grant All on database.my_table to USER';
* merge statement goes here*
End Loop;
commit;
end;
Again, the idea is that every time this runs it will get the new list of csv files in the dir_list table with the java procedure get_dir_list, then for every file name I set as equal to the variable and use the variable in the external table loader to load up the file.
I'm running into [s]two[/s] problems
EDIT: Ok, making the corrections below to cursor row identification, now I hit the point where when I go to the second pass through my cursor appears to be wrong or missing - it will go through a loop just fine if the only action is to do a put_line. But with an execute immediate statement in there such as the "Grant All" then as soon as it completes one pass it throws ORA-08103 at the top of the loop and refuses to go on
3) I'm aware of an ask tom on this (https://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:37593123416931) that says to use the alter table command. However when I try that it doesn't accept my attempt at that
execute immediate 'alter table database.my_table location('''||filename||''')';
throws out an error (plus I'd still need to get it to do another loop there to put the name of the current file into the external loader)
Any suggestions or help? I should note that we are on windows, not unix (since most solutions people offer on these places assume the latter) and I can't grab another program or module to do the job due to approval restrictions (since that seems to be another common solution)
Thanks!

Answer 1

For your first problem, your cursor loop variable is confusingly called filename. Your are referring to that record directly, instead of the column from the cursor. Changing the name slightly to make it a little clearer:
FOR filenames IN folder_contents
LOOP
what_to_load := filesnames.filename;
The rest is less obvious, but it isn't going to be happy that you're dropping and recreating the table in the middle of a block that refers to it statically. You need to make all references dynamic:
execute immediate 'Grant All on database.my_table ...';
-- grant to who/what? and why?
And your merge will have to be dynamic too. At least unless you can get the alter table to work, but you haven't said what the problem is with that. Actually, from what you posted, that's the same cursor variable reference problem:
execute immediate 'alter table database.my_table location('''||filenames.filename||''')';
If you aren't dropping/creating the table in the block, and create it once statically and just alter it, then you can use a static merge - just the alter needs to be dynamic.
A simpler approach might be to create the external table once, with a specific fixed name; loop through the list of real files; and for each of those in turn, rename or copy that to the fixed file name and perform the merge. Each time you query the external table it rereads the file anyway, so changing its contents in the background is OK. Dropping/recreating or even altering the table then wouldn't be necessary.
You could also, as that Ask Tom pst mentions, supply all the file names to the external table at once, as they have the same structure, either with the drop/create or with the alter approach.