I have a requirement, where there are about 2 million rows of data and 1000 columns in 1 table in oracle database. I want this data to be dumped out from this table into a .txt file with PIPE as separator. This file should be created on UNIX application server and not on database server. After that, the file will be FTPed to other server, where it will be loaded by other application. This process of generating file should be fast(in like 1 hour max)
Any suggestions, as to what will be the best way to do this?
Assuming: The application server has Oracle software installed, including SQL*Plus, and can access the database server.
To expand on #Popeye's comment, create a file dump_stuff.sql and put in it something like:
set pagesize 0
set linesize 50000
set trimspool on
set trimout on
set feedback off
set markup csv on delimiter | quote off
set feedback off
col owner format a30
col object_name format a50
col subobject_name format a50
col object_type format a30
spool stuff_output.txt
select owner, object_name, subobject_name, object_type from dba_objects;
spool off
Invoke SQL*Plus on the application server, connect to the target database, and do the following:
set termout off
#dump_stuff
On my database, dumping 78,000 rows from dba_objects took just over 2 seconds. The termoout off command prevents output going to the screen, but still goes to the spool file.
Sample output:
OWNER|OBJECT_NAME|SUBOBJECT_NAME|OBJECT_TYPE
SYS|UNDO$||TABLE
SYS|C_COBJ#||CLUSTER
SYS|PROXY_ROLE_DATA$||TABLE
SYS|I_CDEF2||INDEX
SYS|I_PROXY_ROLE_DATA$_1||INDEX
SYS|FILE$||TABLE
SYS|UET$||TABLE
SYS|I_FILE#_BLOCK#||INDEX
...
You will need to modify the value for the set linesize 50000 to be long enough to avoid your data from the 1,000 columns from wrapping around.
Documentation of the SQL*Plus set command: https://docs.oracle.com/en/database/oracle/oracle-database/12.2/sqpug/SET-system-variable-summary.html#GUID-A6A5ADFF-4119-4BA4-A13E-BC8D29166FAE
I have to study a table on monetdb that probably has many columns.
When I do
SELECT * from cat.data limit 1;
I get
1 tuple !5600 columns dropped!
Which I interpret as not getting all the columns from the console.
I am using mclient to connect to the database.
I tried withe DESC, DESCRIBE - didnt work. Any help?
Indeed. See mclient --help
It is possible to extend the width of your output to see more columns.
Alternatively, within the mclient console use the \d tablename command
You need to use \w-1 command before executing your query.
sql>\w-1
sql>SELECT * from cat.data limit 1 ;
This will show all the columns in the terminal. The text will be wrapped.
I have two broad questions:
Q1. Are executing the lines of code in Oracle SQL Developer and executing the same code in sqlplus command prompt same things?
(reason I ask this that I heard that not all the sqlplus commands are executable in SQL Developer. If answer to above question is yes then please then few useful links will help).
Q2. I am spooling the results of a sql query to a .csv file, but the thing is that columns names are truncated according the maximum length of the values in that column.
My code is:
set colsep ","
spool C:\Oracle\sample.csv
select * from acct_clas_rule;
spool off;
Output of above code is (middle column is having null values)
ABCD_,ABCD_,ABC
-----,-----,---
AB , ,WSD
ABCD , ,WSD
ABCD , ,WSD
SG , ,WSD
KD , ,WSD
WD , ,LKJ
KLHGF, ,LKO
WSDFG, ,LOK
WSDF , ,LKO
WS , ,GH
In above output, columns names have been truncated. I want full names of the columns to be displayed. Can anyone help?
I have seen the question in this link, but I didn't understand how to apply the answers provided there as there was no particular example cited. I am new to these things so I couldn't understand.
Original names of my columns (from left to right in above table) are :
ABCD_DFGT_SDF, ABCD_EDF_GH, ABCD_DFRE
PS -
1. I am using Oracle SQL developer to run sqlplus commands. I think because of which few of my commands are not working (like set underline, set linesize etc.).Please let me know if this is the case. I actually want remove those underlines beneath the columns names.
2. Also let me know that whether you answer is applicable to Oracle SQL Developer or sqlplus.
Thank You
There are a couple of things you can do, in addition to #JChomel's approach - that will work in either SQL Develoepr or SQL*Plus, while these suggestions are specific to SQL Developer.
Let's start with a dummy query based on a CTE to get something like your situation:
set colsep ","
with acct_clas_rule (abdc_1, abcd_2, abcd_3) as (
select cast('AB' as varchar2(5)), cast(null as varchar2(5)), cast('WSD' as varchar2(4)) from dual
union all select 'ABCD', null, 'WSD' from dual
-- ...
union all select 'WS', null, 'GH' from dual
)
select * from acct_clas_rule;
When run as a script in SQL Developer (from the document+arrow icon, or F5) the output is:
ABDC_,ABCD_,ABCD
-----,-----,----
AB , ,WSD
ABCD , ,WSD
WS , ,GH
If you change the query to include the SQL Developer-specific formatting hint /*csv*/ then you get the output you want:
select /*csv*/ * from acct_clas_rule;
"ABDC_1","ABCD_2","ABCD_3"
"AB","","WSD"
"ABCD","","WSD"
"WS","","GH"
except that the strings are all enclosed in double-quotes, which might not be what you really want. (It depends what you're doing with the spooled file and whether any of your string values contain commas, which would confuse Excel for instance).
With more recent versions of SQL Developer you can get exactly the same result without modifying the query using the sqlformat option:
set sqlformat csv
select * from acct_clas_rule;
but again you get the double-quotes. You could change the double-quotes to different enclosure characters, but that probably doesn't help.
A different approach is to use the built-in export tools instead of spool. If you run the query as a statement (green 'play button' icon, or control-enter) then rather than appearing in the Script Output panel, a new Query Result panel will open next to that, showing the results as a grid.
If you right-click on the results you'll get a contextual menu, and choosing Export... from that will give you a wizard to export the results in a format of your choice, including CSV:
You can leave the left and right enclosures as double-quotes to get the same results as the options above, except that null values use the word 'null' instead of an empty string:
"ABDC_1","ABCD_2","ABCD_3"
"AB",null,"WSD"
"ABCD",null,"WSD"
"WS",null,"GH
or you can change them, or remove them by choosing 'none', which gives you:
ABDC_1,ABCD_2,ABCD_3
AB,null,WSD
ABCD,null,WSD
WS,null,G
#thatjeffsmith has commented on how to change the null-replacement text for the /*csv*/ approach. For export it looks like having the word 'null' might have been a bug in 17.2.0; in 17.3.1 that does not appear, and instead you see:
ABDC_1,ABCD_2,ABCD_3
AB,,WSD
ABCD,,WSD
WS,,GH
or enclosed empty strings ("") if you leave the enclosures set.
Q1: TI'm not an expert of SQL Developer. There might be a mode like "command line" or something where you could get similar result, but unsure.
Q2: You have to set the right option to sqlplus: here is a trick I know of (it will also remove the --- --- --- that will cause other issue):
SET HEADING OFF`, to avoid column name to be printed
Union all the column names at the beginning of your script:
set colsep ","
spool C:\Oracle\sample.csv
select 'ABCD_DFGT_SDF', 'ABCD_EDF_GH', 'ABCD_DFRE' from dual
UNION ALL
select * from acct_clas_rule;
spool off;
use the sql plus set command -----> set underline off
In an attempt to more easily view the output in SQLPlus, I fiddled with window-size properties and font(tiniest font).
But this is what I get :
It is exactly the same, just much less readable.
I just want to run the query :
select * from user_tables where rownum < 10 ;
Without the messy unreadable lines (currently I can't tell apart the data from table headings)
With the help of one useful SQL*Plus option and one additional helper option it is possible. Put
set linesize 32767
set trimout on
before your select.
Enjoy.
I have some issues with how sqlplus output is formatted for the terminal and I am just thinking of writing a script around sqlplus and fixing these.
On the other hand, wow that seems really lame. Because Oracle has several tons of tools written. Yet it seems difficult to get what I want. Does anyone have another suggestion?
First, I want smarter column widths. If I create a table with a column whose max size is 200 characters but then I put "abc", "xyz" and "123" in it, do I need a 200-space wide column on the terminal to display the contents? I do not think so. I think I need 3 characters plus a couple for padding. Yet Oracle insists on giving me a 200-character wide column. Unless there is somewhere to fix this.
Second, I want easy access to a sideways display of the columns, like using \G at the end of the command in MySQL. I know there is a way to do this in Oracle but it seemed complicated. Why could there not just be a simple switch? Like a \G at the end of the command? There can be if I wrap the output to sqlplus and do this myself.
So, the question seems to be this. Should I write a simple script around sqlplus to give me what I want, or is there a way to get Oracle to give me this behavior In sqlplus? And if there is, how much extra information will I have to stuff into my head to make this work? Because it does not seem as though it should be very complicated. But Oracle is certainly not making it easy.
First of all I suggest you look over the SQL*plus reference - you might find some useful tips there like adjusting a column width
COL column_name for a20
you can set up your own settings in the GLOGIN file. over time, like any other CMD, you'll get your preferences just right.
To describe a table you can use DESC. if you want more data write your own script and reuse it with #.
If all this doesn't work for you, you can always switch to a GUI like Toad or SQL developer.
EDIT:
I'm adding one of my own scripts to show you some tricks on how to make SQL*Plus more friendly on the command line. This one is for getting segment sizes.
/* This is the trick - clears &1 and &2 if received an empty string */
set ver off feed off
col 1 new_v 1
col 2 new_v 2
select 1,2 from dual where 1=0;
variable p_owner varchar2(30)
variable p_segment varchar2(30)
/* set bind variables */
begin
:p_owner := '&1';
:p_segment := '&2';
end;
/
set feed 1
break on segment_type skip 1
column MB for a25
select
segment_type,
decode(gi_segment_name + gi_segment_type + gi_tablespace_name , 3 ,'...Grand Total', segment_name) SEGMENT_NAME,
to_char(round(MB,3),'99,999,999.99') MB ,
nvl(tablespace_name,'-*-') tablespace_name
from (
select tablespace_name , segment_type , segment_name , sum(bytes/1024/1024) MB ,
grouping_id(segment_name) gi_segment_name ,
grouping_id(segment_type) gi_segment_type ,
grouping_id(segment_type) gi_tablespace_name
from dba_segments
where ((:p_owner is null and owner = user) or owner like upper(:p_owner))
and (:p_segment is null or segment_name like upper('%'||:p_segment||'%'))
group by rollup(tablespace_name, segment_type , segment_name)
)
where not (gi_segment_name = 1 and gi_segment_type = 0 and gi_tablespace_name = 0)
order by decode(segment_type,'TABLE','0','TABLE PARTITION','1','INDEX','2','INDEX PARTITION','3',segment_type) ,
(case when segment_name like '.%' then 'z' else 'a' end) ,
gi_segment_name ,
MB desc ,
segment_name;
clear break
/* clear definition for &1 and &2 after being used.
allows the variable to be null the next run. */
undefine 1
undefine 2
I'll walk you through some of the things iv'e done here
The script accepts two parameters. The first 4 lines clears the
parameter if none received. if you don't do this SQL*Plus will prompt
you for them. And we dont want that.
Setting the binds was more of a big deal in past version. It's
intended to save Hard / Soft parse. latest version solve this
problem. It's still a best practice though.
The break is a nice touch. You'll see it.
The grouping Id show me the sub totals on several levels.
I've added two parameter, owner and segment name. both can contain
wild card. both can be null. If non provided the query will fetch the
current user segments.
Order by decode enabled me to set a custom sort order for different
segment types. You can change it as you wish.
I'm executing the script like this
my segments :
#seg
Scott's segments
#seg scott
Scott's Emp related segments
#seg scott emp
I have similar scripts for session, longops, wait events, tables, constraints, locks, kill session etc .... during my daily routine i rarely write SQL for querying this stuff any more.