Bulk data file output PL/SQL - performance

Question

In my PL/SQL function, I m going to output a csv file for 14M million rows. There will be 3 fields in each row.
Im afraid the I/O process running time. I can create and output more than one csv file for this batch. I don want to use line by line output.
What is the best and quick way to writing file. For example, fetching data then filling into a record. And then write the file.
Thanks.

Answer 1

From Oracle 10g onwards, it is possible to write a CLOB to a file with a single call, using the DBMS_XSLPROCESSOR.CLOB2FILE procedure. In the following example, we will prepare a temporary CLOB with our data instead of writing it with UTL_FILE. When all source data has been added to the CLOB, we will write it to a flat-file in a single call.
SQL> DECLARE
2
3 v_file CLOB;
4 v_buffer VARCHAR2(32767);
5 v_name VARCHAR2(128) := 'clob2file_buffered.txt';
6 v_lines PLS_INTEGER := 0;
7 v_eol VARCHAR2(2);
8 v_eollen PLS_INTEGER;
9 c_maxline CONSTANT PLS_INTEGER := 32767;
10
11 BEGIN
12
13 v_eol := CASE
14 WHEN DBMS_UTILITY.PORT_STRING LIKE 'IBMPC%'
15 THEN CHR(13)||CHR(10)
16 ELSE CHR(10)
17 END;
18 v_eollen := LENGTH(v_eol);
19
20 DBMS_LOB.CREATETEMPORARY(v_file, TRUE);
21
22 FOR r IN (SELECT x || ',' || y || ',' || z AS csv
23 FROM source_data)
24 LOOP
25
26 IF LENGTH(v_buffer) + v_eollen + LENGTH(r.csv) <= c_maxline THEN
27 v_buffer := v_buffer || v_eol || r.csv;
28 ELSE
29 IF v_buffer IS NOT NULL THEN
30 DBMS_LOB.WRITEAPPEND(
31 v_file, LENGTH(v_buffer) + v_eollen, v_buffer || v_eol
32 );
33 END IF;
34 v_buffer := r.csv;
35 END IF;
36
37 v_lines := v_lines + 1;
38
39 END LOOP;
40
41 IF LENGTH(v_buffer) > 0 THEN
42 DBMS_LOB.WRITEAPPEND(
43 v_file, LENGTH(v_buffer) + v_eollen, v_buffer || v_eol
44 );
45 END IF;
46
47 DBMS_XSLPROCESSOR.CLOB2FILE(v_file, 'DUMP_DIR', v_name);
48 DBMS_LOB.FREETEMPORARY(v_file);
49
50 DBMS_OUTPUT.PUT_LINE('File='||v_name||'; Lines='||v_lines);
51
52 END;
53 /
File=clob2file_buffered.txt; Lines=1000000
PL/SQL procedure successfully completed.
Elapsed: 00:00:28.65
The CLOB-specific code is highlighted above and is self-explanatory (perhaps with the exception of the end-of-line character assignment on lines 13-17 which is different for Windows. UTL_FILE manages the port-specific end-of-line conversions for us, but with CLOBs we must manage this ourselves).
Of particular interest is the DBMS_XSLPROCESSOR call on line 47, which is our only write operation to the destination flat-file. We can see overall that this technique is similar in performance to our buffered UTL_FILE mechanism (the CLOB method was slightly quicker). We therefore have an alternative method for writing data, but there will be additional costs associated with using CLOBs (for example, temporary tablespace and buffer cache). If the volume of data to be dumped is high, then this method might put too much stress on our temporary tablespace and cause problems for other users (large sort operations, hash joins, global temporary tables etc).
Care should therefore be taken when using this method.