I have seen many examples where you can convert a SQL Query output to a CSV file by calling it from a stored procedure as shown in PL/SQL code below. However, in my case, I have a stored procedure called Management with 3 input parameters. Once these inputs are confirmed, the stored procedure extracts the result with 10 columns based on the code in the PL/SQL.
My aim is to convert this extract from the Management stored procedure to a CSV. The examples I found so far are all to do with converting SQL query only to CSV by calling it from a stored procedure.
I'm trying to avoid writing the code to convert the output to CSV in the Management stored procedure by calling it from another stored procedure. Is this possible?
The Management stored procedure is called by the SQL developer and the values are returned by applying a select query from two tables and applying multiple cases and if statements.
An example will be much appreciated.
CREATE OR REPLACE PROCEDURE export_to_csv
IS
v_file UTL_FILE.file_type;
v_string VARCHAR2 (4000);
CURSOR c_emp
IS
SELECT empno,
ename,
deptno,
sal,
comm
FROM emp;
BEGIN
v_file :=
UTL_FILE.fopen ('CSVDIR',
'empdata.csv',
'w',
1000);
-- if you do not want heading then remove below two lines
v_string := 'Emp Code, Emp Name, Dept, Salary, Commission';
UTL_FILE.put_line (v_file, v_string);
FOR cur IN c_emp
LOOP
v_string :=
cur.empno
|| ','
|| cur.ename
|| ','
|| cur.deptno
|| ','
|| cur.sal
|| ','
|| cur.comm;
UTL_FILE.put_line (v_file, v_string);
END LOOP;
UTL_FILE.fclose (v_file);
EXCEPTION
WHEN OTHERS
THEN
IF UTL_FILE.is_open (v_file)
THEN
UTL_FILE.fclose (v_file);
END IF;
END;
The management SP will return values and you will use those values to create the CSV file with this new SP (export_to_csv)? If this is the case then
Create one Package Spec and define 2 SPs (Management and export_to_csv).
Then in the Package body, copy the logic of Management sp and at the end of Management SP, call your new SP (export_to_csv).
If you already have Package then directly use that package instead of creating the new one.
Related
I would like to create record from refcursor. My code:
set serveroutput on
DECLARE
c_curs SYS_REFCURSOR;
v_id NUMBER;
BEGIN
pck_prov.get_value_type_list (1, c_curs); --> procedure called here
-- I guess this is the place where record can be created from cursor.
LOOP
FETCH c_curs
INTO v_id;--instead of fetching into variable I would like to fetch into row
EXIT WHEN c_curs%NOTFOUND;
DBMS_OUTPUT.PUT_LINE(v_id);--if fetching is done into row, only selected columns can be printed, like myrow.id
END LOOP;
CLOSE c_curs;
END;
Please note: I know how to create record from cursor which is defined with select statement as it is described here. What I don't know is how to use same technique for refcursors.
EDIT:
Code from here is just what I need, but it throws error:
set serveroutput on
VAR c_curs refcursor;
EXECUTE pck_prov.get_value_type_list(1, :c_curs);
BEGIN
FOR record_test IN :c_curs LOOP
dbms_output.put_line(record_test.id);
END LOOP;
END;
Error: error PLS-00456: item 'SQLDEVBIND1Z_1' is not a cursor.
Just to clarify question:
In my database there is around 200 packages.
Every package has several stored procedures inside - and usually each procedure is combined with columns from different tables. That is why it would be the best to have some dynamically created cursor, so I can make simple select just like in the example I've posted.
From Oracle 12.1 onward, you could use the DBMS_SQL.return_result procedure. SQL Plus displays the contents of implicit statement results automatically. So, rather than defining explicit ref cursor out parameters, the RETURN_RESULT procedure in the DBMS_SQL package allows you to pass them out implicitly.
DECLARE
c_curs SYS_REFCURSOR;
v_id NUMBER;
BEGIN
pck_prov.get_value_type_list (1, c_curs);
DBMS_SQL.return_result(c_curs); --> Passes ref cursor output implicitly
END;
/
In fact, no need of this separate PL/SQL block, you could add the DBMS_SQL.return_result(c_curs) statement in your original pck_prov.get_value_type_list procedure itself.
Just define a PL/SQL RECORD type that matches the cursor and FETCH into it.
DECLARE
c_curs SYS_REFCURSOR;
TYPE rec_t IS RECORD ( object_name VARCHAR2(30), object_type VARCHAR2(30) );
v_rec rec_t;
BEGIN
OPEN c_curs FOR
SELECT object_name,
object_type
FROM dba_objects
WHERE object_name like 'DBA%TAB%';
LOOP
FETCH c_curs
INTO v_rec;
EXIT WHEN c_curs%NOTFOUND;
DBMS_OUTPUT.PUT_LINE(v_rec.object_name || ' - ' || v_rec.object_type);
END LOOP;
CLOSE c_curs;
END;
DBA_ADVISOR_SQLA_TABLES - VIEW
DBA_ADVISOR_SQLA_TABVOL - VIEW
DBA_ADVISOR_SQLW_TABLES - VIEW
DBA_ADVISOR_SQLW_TABVOL - VIEW
DBA_ALL_TABLES - VIEW
etc...
Is there any possibility to export data from around 300 tables within single schema with millions of records to CSV or TXT using any PL/SQL procedure?
What do you propose, which is fastest way to do it? For the moment I do not need to import these exported files to any other schema...
I tried with Toad manually exporting table by table...
you can try following steps.
write a loop to get the table names
use cursors to fetch the data from each table
use SYS.UTL_FILE utilities to write the data to files in any required format.
This is a very high level solution. But i am sure it will work.
I have created a utility by which you can generate PL/SQL procedures to export data from a table. It will take the following parameters, table name, column names, directory name and the delimiter. You can generate 50 procedures for 50 tables in no time to export data from Oracle. Check this link Generate PL/SQL Procedure to export data into CSV
I managed to dynamically go through all tables and get column names and write to a file. I am struggling into part how to fetch data rows from tables dynamically when execute immediate query? how should I save data rows and than fetch it and write to files?
Here is the code:
DECLARE p_table VARCHAR2 (100);
l_file UTL_FILE.FILE_TYPE;
l_string VARCHAR2 (10000);
query_string VARCHAR2 (4000);
BEGIN
FOR tab IN (SELECT *
FROM dba_tables
WHERE owner = 'XYZ' AND table_name LIKE 'XYZ%')
LOOP
p_table := tab.table_name;
l_file :=
UTL_FILE.FOPEN ('my_path',
tab.table_name || '.txt',
'w',
10000);
l_string := NULL;
FOR col_he IN (SELECT *
FROM dba_tab_columns
WHERE owner = 'DWHCO' AND table_name = p_table)
LOOP
CASE
WHEN l_string IS NULL
THEN
l_string := col_he.column_name;
ELSE
l_string := l_string || ',' || col_he.column_name;
END CASE;
END LOOP;
UTL_FILE.PUT_LINE (l_file, l_string); --Printng table columns
query_string := 'select ' || l_string || ' from DWHCO.' || p_table
--Execute immediate query_string into ??????????;
--??????
UTL_FILE.FCLOSE (l_file); END LOOP;END;
The Data Dump procedure is helpful for programmatically exporting many tables to simple formats like CSV.
First, install the package using the above link. The below code creates a directory, cycles through tables, and exports each table as CSV.
create or replace directory temp_dir as 'C:\temp';
begin
for tables in
(
select
owner||'_'||table_name||'.csv' file_name,
'select * from "'||owner||'"."'||table_name||'"' v_sql
from dba_tables
where owner = 'XYZ'
and table_name like 'XYZ%'
order by 1
) loop
data_dump
(
query_in => tables.v_sql,
file_in => tables.file_name,
directory_in => 'TEMP_DIR',
delimiter_in => ',',
header_row_in => true
);
end loop;
end;
/
I want to create procedure, that will use cursor, which is the same for arbitrary tables. My current one looks like this:
create or replace
PROCEDURE
some_name(
p_talbe_name IN VARCHAR2,
p_chunk_size IN NUMBER,
p_row_limit IN NUMBER
) AS
CURSOR v_cur IS
SELECT common_column,
ora_hash(substr(common_column, 1, 15), p_chunk_size) as chunk_number
-- Here it can find the table!
FROM p_table_name;
TYPE t_sample IS TALBE OF v_cur%rowtype;
v_sample t_sample;
BEGIN
OPEN v_cur;
LOOP FETCH v_cur BULK COLLECT INTO v_sample LIMIT p_row_limit;
FORALL i IN v_sample.first .. v_sample.last
INSERT INTO chunks VALUES v_sample(i);
COMMIT;
EXIT WHEN v_cur%notfound;
END LOOP;
CLOSE v_cur;
END;
The problem is that it cannot find the table named p_table_name which I want to parametrize. The thing is that I need to create chunks based on hashes for common_column which exists in all intended tables. How to deal with that problem? Maybe there is the equivalent oracle code that will do the same thing? Then I need the same efficiency for the query. Thanks!
I would do this as a single insert-as-select statement, complicated only by the fact you're passing in the table_name, so we need to use dynamic sql.
I would do it something like:
CREATE OR REPLACE PROCEDURE some_name(p_table_name IN VARCHAR2,
p_chunk_size IN NUMBER,
p_row_limit IN NUMBER) AS
v_table_name VARCHAR2(32); -- 30 characters for the tablename, 2 for doublequotes in case of case sensitive names, e.g. "table_name"
v_insert_sql CLOB;
BEGIN
-- Sanitise the passed in table_name, to ensure it meets the rules for being an identifier name. This is to avoid SQL injection in the dynamic SQL
-- statement we'll be using later.
v_table_name := DBMS_ASSERT.ENQUOTE_LITERAL(p_table_name);
v_insert_sql := 'insert into chunks (common_column_name, chunk_number)'||CHR(10)|| -- replace the column names with the actual names of your chunks table columns.
'select common_column,'||CHR(10)||
' ora_hash(substr(common_column, 1, 15), :p_chunk_size) AS chunk_number'||CHR(10)||
'from '||v_table_name||CHR(10)||
'where rownum <= :p_row_limit';
-- Used for debug purposes, so you can see the definition of the statement that's going to be run.
-- Remove before putting the code in production / convert to proper logging code:
dbms_output.put_line(v_insert_sql);
-- Now run the statement:
EXECUTE IMMEDIATE v_insert_sql USING p_chunk_size, p_row_limit;
-- I've included the p_row_limit in the above statement, since I'm not sure if your original code loops through all the rows once it processes the
-- first p_row_limit rows. If you need to insert all rows from the p_table_name into the chunks table, remove the predicate from the insert sql and the extra bind variable passed into the execute immediate.
END some_name;
/
By using a single insert-as-select statement, you are using the most efficient way of doing the work. Doing the bulk collect (which you were using) would use up memory (storing the data in the array) and cause extra context switches between the PL/SQL and SQL engines that the insert-as-select statement avoids.
I need to debug in pl/sql to figure times of procedures, I want to use:
SELECT systimestamp FROM dual INTO time_db;
DBMS_OUTPUT.PUT_LINE('time before procedure ' || time_db);
but I don't understand where the output goes to and how can I redirect it to a log file that will contain all the data I want to collect?
DBMS_OUTPUT is not the best tool to debug, since most environments don't use it natively. If you want to capture the output of DBMS_OUTPUT however, you would simply use the DBMS_OUTPUT.get_line procedure.
Here is a small example:
SQL> create directory tmp as '/tmp/';
Directory created
SQL> CREATE OR REPLACE PROCEDURE write_log AS
2 l_line VARCHAR2(255);
3 l_done NUMBER;
4 l_file utl_file.file_type;
5 BEGIN
6 l_file := utl_file.fopen('TMP', 'foo.log', 'A');
7 LOOP
8 EXIT WHEN l_done = 1;
9 dbms_output.get_line(l_line, l_done);
10 utl_file.put_line(l_file, l_line);
11 END LOOP;
12 utl_file.fflush(l_file);
13 utl_file.fclose(l_file);
14 END write_log;
15 /
Procedure created
SQL> BEGIN
2 dbms_output.enable(100000);
3 -- write something to DBMS_OUTPUT
4 dbms_output.put_line('this is a test');
5 -- write the content of the buffer to a file
6 write_log;
7 END;
8 /
PL/SQL procedure successfully completed
SQL> host cat /tmp/foo.log
this is a test
As an alternative to writing to a file, how about writing to a table? Instead of calling DBMS_OUTPUT.PUT_LINE you could call your own DEBUG.OUTPUT procedure something like:
procedure output (p_text varchar2) is
pragma autonomous_transaction;
begin
if g_debugging then
insert into debug_messages (username, datetime, text)
values (user, sysdate, p_text);
commit;
end if;
end;
The use of an autonomous transaction allows you to retain debug messages produced from transactions that get rolled back (e.g. after an exception is raised), as would happen if you were using a file.
The g_debugging boolean variable is a package variable that can be defaulted to false and set to true when debug output is required.
Of course, you need to manage that table so that it doesn't grow forever! One way would be a job that runs nightly/weekly and deletes any debug messages that are "old".
use
set serveroutput on;
for example:
set serveroutput on;
DECLARE
x NUMBER;
BEGIN
x := 72600;
dbms_output.put_line('The variable X = '); dbms_output.put_line(x);
END;
If you are just testing your PL/SQL in SQL Plus you can direct it to a file like this:
spool output.txt
set serveroutput on
begin
SELECT systimestamp FROM dual INTO time_db;
DBMS_OUTPUT.PUT_LINE('time before procedure ' || time_db);
end;
/
spool off
IDEs like Toad and SQL Developer can capture the output in other ways, but I'm not familiar with how.
In addition to Tony's answer, if you are looking to find out where your PL/SQL program is spending it's time, it is also worth checking out this part of the Oracle PL/SQL documentation.
Using UTL_FILE instead of DBMS_OUTPUT will redirect output to a file:
http://oreilly.com/catalog/oraclebip/chapter/ch06.html
As a side note, remember that all this output is generated in the server side.
Using DBMS_OUTPUT, the text is generated in the server while it executes your query and stored in a buffer. It is then redirected to your client app when the server finishes the query data retrieval. That is, you only get this info when the query ends.
With UTL_FILE all the information logged will be stored in a file in the server. When the execution finishes you will have to navigate to this file to get the information.
Hope this helps.
Its possible write a file directly to the DB server that hosts your database, and that will change all along with the execution of your PL/SQL program.
This uses the Oracle directory TMP_DIR; you have to declare it, and create the below procedure:
CREATE OR REPLACE PROCEDURE write_log(p_log varchar2)
-- file mode; thisrequires
--- CREATE OR REPLACE DIRECTORY TMP_DIR as '/directory/where/oracle/can/write/on/DB_server/';
AS
l_file utl_file.file_type;
BEGIN
l_file := utl_file.fopen('TMP_DIR', 'my_output.log', 'A');
utl_file.put_line(l_file, p_log);
utl_file.fflush(l_file);
utl_file.fclose(l_file);
END write_log;
/
Here is how to use it:
1) Launch this from your SQL*PLUS client:
BEGIN
write_log('this is a test');
for i in 1..100 loop
DBMS_LOCK.sleep(1);
write_log('iter=' || i);
end loop;
write_log('test complete');
END;
/
2) on the database server, open a shell and
tail -f -n500 /directory/where/oracle/can/write/on/DB_server/my_output.log
An old thread, but there is another alternative.
Since 9i you can use pipelined table function.
First, create a type as a table of varchar:
CREATE TYPE t_string_max IS TABLE OF VARCHAR2(32767);
Second, wrap your code in a pipelined function declaration:
CREATE FUNCTION fn_foo (bar VARCHAR2) -- your params
RETURN t_string_max PIPELINED IS
-- your vars
BEGIN
-- your code
END;
/
Replace all DBMS_OUTPUT.PUT_LINE for PIPE ROW.
Finally, call it like this:
SELECT * FROM TABLE(fn_foo('param'));
Hope it helps.
Try This:
SELECT systimestamp INTO time_db FROM dual ;
DBMS_OUTPUT.PUT_LINE('time before procedure ' || time_db);
I was wondering how to go about exporting a query from PL/SQL to an text file or csv file. The query I have in mind exports a huge amount of data (about 1 gig). So I'd also like the data split across multiple files;
out1.csv
out2.csv
out3.csv
I'd like to be able to decide how many files to split it across.
Anyone have any idea how to do this?
Use UTL_FILE.
A well known ( probably the most complete discussion on this topic ) discussion on this can be found at Ask Tom, Here , note that many of the examples there date back to oracle 8, so there may be better ways to do it in your version of Oracle.
Try this
For creating MYDIR
create or replace directory MYDIR as 'F:/DATA/';
Grant all permission to MYDIR via SYS user
execute this procedure
CREATE OR REPLACE PROCEDURE export_to_csv(refcur out sys_refcursor) IS
v_file UTL_FILE.file_type;
v_string VARCHAR2(4000);
CURSOR c_emp IS
SELECT ROLE_ID, ROLE_DESC FROM role_mst;
BEGIN
open refcur for
SELECT ROLE_ID, ROLE_DESC FROM role_mst;
v_file := UTL_FILE.fopen('MYDIR', 'empdata.csv', 'w', 1000);
-- if you do not want heading then remove below two lines
v_string := 'Emp Code, Emp Name';
UTL_FILE.put_line(v_file, v_string);
FOR cur IN c_emp LOOP
v_string := cur.ROLE_ID || ',' || cur.ROLE_DESC;
UTL_FILE.put_line(v_file, v_string);
END LOOP;
UTL_FILE.fclose(v_file);
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line(sqlerrm);
IF UTL_FILE.is_open(v_file) THEN
UTL_FILE.fclose(v_file);
END IF;
END;