I'm spooling out delimited text files from SQL*Plus, but every column is printed as the full size per its definition, rather than the data actually in that row.
For instance, a column defined as 10 characters, with a row value of "test", is printing out as "test " instead of "test". I can confirm this by selecting the column along with the value of its LENGTH function. It prints "test |4".
It kind of defeats the purpose of a delimiter if it forces me into fixed-width. Is there a SET option that will fix this, or some other way to make it print only the actual column data.
I don't want to add TRIM to every column, because if a value is actually stored with spaces I want to be able to keep them.
Thanks
I have seen many SQL*plus script, that create text files like this:
select A || ';' || B || ';' || C || ';' || D
from T
where ...
It's a strong indication to me that you can't just switch to variable length output with a SET command.
Instead of ';' you can of course use any other delimiter. And it's up to your query to properly escape any characters that could be confused with a delimiter or a line feed.
Generally, I'd forget SQL Plus as a method for getting CSV out of Oracle.
Tom Kyte has written a nice little Pro-C unloader
Personally I've written a utility which does similar but in perl
Related
I am having a csv file as source with ; as Column Delimiter,LF as Row Delimiter and my data is enclosed within "". If i get LF(Row delimiter) in data we should not consider it as Row Delimiter.My target is Oracle database.
How to get the required output below while using informatica.
Input:
"Ram";"Hyderabad"LF
"Sita";"Hyderabad,LF
INDIA-500084."LF
Required Output should be 2 rows only:
Name Address
Ram Hyderabad
Sita Hyderabad, INDIA-500084.
Wrong Output i am getting is 3 rows:
Name Address
Ram Hyderabad
Sita Hyderabad,
INDIA-500084.
Looks to me that you need to do a find & replace on your source before processing to get rid of those LF strings within double quotes.
Unfortunatelly, Informatica probably does a split on LF to rows first, so you need to reprocess the source before Informatica reads it. Try using Command as Source and use sed.
In the 'Config Object' tab of the session put the override for 'Custom Properties' as MatchQuotesPastEndOfLine=Yes;
This will read the lines even after the LF till it sees the end of quotes.
I am trying to load data into Oracle database using sqlloader,
My data looks like following.
1|2|3|4|5|6|7|8|9|10
I do not want to load first and last column into table,
I want to load 2|3|4|5|6|7|8|9 into one field.
The table I am trying to load into has only one filed named 'field1'.
If anyone has this kind of experience, could you give some advice?
I tried BOUNDFILLER, FILLER and so on, I could not make it.
Help me. :)
Load the entire row from the file into a BOUNDFILLER, then extract the part you need into the column. You have to tell sqlldr that the field is terminated by the carriage return/linefeed (assuming a Windows OS) so it will read the entire line from the file as one field. here the whole line from the file is read into "dummy" as BOUNDFILLER. "dummy" does not match a column name, and it's defined as BOUNDFILLER anyway, so the whole row is "remembered". The next line in the control file starts with a column that DOES match a column name, so sqlldr attempts to execute the expression. It extracts a substring from the saved "dummy" and puts it into the "col_a" column.
The regular expression in a nutshell returns the part of the string after but not including the first pipe, and before but not including the last pipe. Note the double backslashes. In my environment anyway, when using a backslash to take away the special meaning of the pipe (not needed when between the square brackets) it gets stripped when passing from sqlldr to the regex engine so two backslashes are required in the control file
(normally a pipe symbol is a logical OR) so one gets through in the end. If you have trouble here, try one backslash and see what happens. Guess how long THAT took me to figure out!
load data
infile 'x_test.dat'
TRUNCATE
into table x_test
FIELDS TERMINATED BY x'0D0A'
(
dummy BOUNDFILLER,
col_a expression "regexp_substr(:dummy, '[^|]*\\|(.+)\\|.*', 1, 1, NULL, 1)"
)
EDIT: Use this to test the regular expression. For example, if there is an additional pipe at the end:
select regexp_substr('1|2|3|4|5|6|7|8|9|10|', '[^|]*\|(.+)\|.*\|', 1, 1, NULL, 1)
from dual;
2nd edit: For those uncomfortable with regular expressions, this method uses nested SUBSTR and INSTR functions:
SQL> with tbl(str) as (
select '1|2|3|4|5|6|7|8|9|10|' from dual
)
select substr(str, instr(str, '|')+1, (instr(str, '|', -1, 2)-1 - instr(str
, '|')) ) after
from tbl;
AFTER
---------------
2|3|4|5|6|7|8|9
Deciding which is easier to maintain is up to you. Think of the developer after you and comment at any rate! :-)
I am trying to code a sqlldr.ctl file WHEN Clause to limit the records imported to those matching a portion of the current Schema's name.
The code I have (which does NOT work) is:
LOAD DATA
TRUNCATE INTO TABLE TMP_PRIM_ACCTS
when REGION_NUM = substr(user,-3,3)
Fields terminated by "|" Optionally enclosed by '"'
Trailing NULLCOLS
( PORTFOLIO_ACCT,
PRIMARY_ACCT_ID NULLIF (PRIMARY_ASSET_ID="NULL"),
REGION_NUM NULLIF (PARTITION_NUM="NULL")
)
sqlldr returns:
SQL*Loader-350: Syntax error at line 3.
Expecting quoted string or hex identifier, found "substr".
when PARTITION_NUM = substr(user,-3,3)
I cannot put single quotes around "user", because that turns it into the literal string "user". Can anyone explain how I can reference the "active" User in this WHEN Clause?
Thank you!
Can you try something like this? (now I can't make test with SQLLDR, but this is syntax I used for changing values):
when REGION_NUM = "substr(:user,-3,3)"
It doesn't look like you can. The documentation only shows fixed values:
Trying to use an expression in when that clause (or in nullif; thought I'd try to see if you could cause a rejection based on null PK value) you just see the literal value in the log:
Table TMP_PRIM_ACCTS, loaded when REGION_NUM = 0X73756273747228757365722c2d332c3329(character 'substr(user,-3,3)')
which is sort of what you referred when you said you couldn't quote user, but you'd have to quite the whole thing anyway. Using :user doesn't work either, the colon is seen as just another character, it doesn't try to find a column called user instead.
The simplest approach may be to pre-process the data file and remove any rows which don't match the pattern (e.g. via a regex). That would actually be slightly easier if you used an external table instead of SQL*Loader.
Alternatively, generate your control file and embed the correct literal value based on the user you'll connect as.
I need to export a select query output to text file using spool. The output by default adds double quotes around the column. How to Prevent it?
The query returns only one column.
Ideally i need to remove headers as well as the double quotes from the output. below is the script file i use in oracle Developer. but heading is also not removed.
set echo off
SET HEADING OFF
SET PAGESIZE 0
SET COLSEP ''
spool 'D:\public\cvs_txPCG.txt'
select /*csv*/
pcg from temptx;
spool off;
output
"PCG"
"76259737020150320000504281565213310052440093515652109.2909.290101"
"19519905620160502000504283153419040044861008644759203.3903.390101"
"49424051620160220000504284594314590009220713032964404.3804.380202"
"88761197020151025000504284594315180036700812132964401.9901.990101"
You can't get rid of the double-quotes with the /*csv*/ directive. The slightly more convenient way of doing this now is with the sqlformat command, but even with sqlformat delimited you have to have the string enclosed.
Using CSV format for a single column seems rather pointless anyway though, unless you have delimiters in your string you want to escape. If you are really exporting that one column and don't have delimiters then just leave out the /*csv*/ directive. If you have multiple columns or strings that may contain delimiters then the enclosures are useful to stop those delimiters being misinterpreted when opening in Excel or some other external tool.
If you really want CSV format without enclosures you can either build up the output manually by concatenating the column(s) with comma literals, which still allows you to spool to a file:
select pcg ||','|| some_other_col ||','|| etc from ...
Or you can run the query using Run Statement (Ctrl-Enter) and then export the results from the data grid, from the context menu that appears when you right-click on the output. When you export and choose CSV or delimited format you can choose whether to include the header and which delimiter and enclosures to use - and you're allowed to choose 'none'. If you don't want to set that on each export you can set it as a default, from Tools->Preferences:
You can't do it with the /*csv*/ directive or sqlformat csv though.
I know it is too old but got the answer with my search today, simply use
set markup csv on quote off
select 1 id, 'Venkat' name from dual;
Output will be
ID,NAME
1,Venkat
Didn't even think of any other SET commands
If the double quotes are part of the data you can use the trim() function to remove them.
select /*csv*/
trim(both '"' from pcg) as pcg from temptx;
Oracle documentation for the trim function:
https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions199.htm
I have a large input file, values are pipe delimited. And there are 20 values in a row. after 19th pipe, if new line character comes, that is a record.
But my input file is having \n not only after 19 pipes but also in the other values. sample line looks like this...
101101|this\nis my sample|12547|sample\nxyz|......(19th pipe)|end of record\n
I am new to Hadoop and I don't know how to divide lines to create key value pairs based on this condition.
Another related question I have is, input split happens at the client side and if I have to split the input file conditionally on the client side(one machine), will it not be very slow considering the large file? Please help.
In Hive NULL column values are represented as "\N" that's the default behaviour of Hive. This is done to differentiate NULL and "NULL" (string NULL).
If you don't want \N to appears to appear in your export you can use COALESCE UDF.
Roughly your query may look like this
SELECT
COALESCE (my_column, '') AS my_column
FROM
my_table