From tests done, it seems that loading does not fail if the file contains extra delimiters.
Let say I have a control file with 3 columns a,b,c as follows
LOAD DATA
TRUNCATE INTO TABLE test
FIELDS TERMINATED BY "|" trailing nullcols
(
a,
b,
c
)
But the file contains extra delimiters in some of the rows. As example:
1|2|3
4|5|6
7|8|9|10
11|12|13
In the above example, row 3 contains an extra delimiter but loading is successful and the last value (10) is not loaded.
Can you specify some option in the control file to treat row 3 as bad or discard?
Validations of the input file can be done prior to loading but wanted to check if the loader itself supports such feature.
Related
I am trying to load a txt file with | (pipe) delimiter to an Oracle table via SQL loader utility. All the fields are enclosed with double quotes. But there are some text fields in the files that have additional double quotes in addition to the enclosed ones that needs to be retained. All the table columns are defined as VARCHAR. Here's the control parameters am using
OPTIONS (DIRECT=TRUE,SKIP=1)
LOAD DATA
CHARACTERSET UTF8
INFILE aaa.txt
APPEND INTO TABLE info_table
FIELDS TERMINATED BY "|"
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
This is my sample file
"1"|"High "Gold Tip" Tea, 600"
"2"|""10000 Beers, Wines & Spirits""
Table should be loaded with the below details
Record 1:
Column 1 - 1
Column 2 - High "Gold Tip" Tea, 600
Record 2:
Column 1 - 2
Column 2 - 10000 Beers, Wines & Spirits
Unfortunately, there's nothing much to be said.
File format is bad. You can't enclose values into characters that are used in those fields themselves. As data contain double quotes, you'll have to optionally enclose values into something else, not double quotes.
However, as you already split values with pipe characters, what do you need double quotes to optionally enclose those field values? Omit them from the file and you won't have any problem (of such kind, of course; who knows what might come next, but that's another story).
I am having a csv file as source with ; as Column Delimiter,LF as Row Delimiter and my data is enclosed within "". If i get LF(Row delimiter) in data we should not consider it as Row Delimiter.My target is Oracle database.
How to get the required output below while using informatica.
Input:
"Ram";"Hyderabad"LF
"Sita";"Hyderabad,LF
INDIA-500084."LF
Required Output should be 2 rows only:
Name Address
Ram Hyderabad
Sita Hyderabad, INDIA-500084.
Wrong Output i am getting is 3 rows:
Name Address
Ram Hyderabad
Sita Hyderabad,
INDIA-500084.
Looks to me that you need to do a find & replace on your source before processing to get rid of those LF strings within double quotes.
Unfortunatelly, Informatica probably does a split on LF to rows first, so you need to reprocess the source before Informatica reads it. Try using Command as Source and use sed.
In the 'Config Object' tab of the session put the override for 'Custom Properties' as MatchQuotesPastEndOfLine=Yes;
This will read the lines even after the LF till it sees the end of quotes.
We are using flat files.but how to skip the last records in flat files.
use the Following in your ctl File if your tail record always starts with T! Haven't tested it in a while but worked for me the last time i worked with loading a staging table.
LOAD DATA
INFILE 'file.dat' BADFILE 'file.bad' DISCARDFILE 'file.dis'
APPEND
INTO TABLE tablename
-- This will skip the tail record
WHEN (01) <> 'T'
(
column names);
You can skip the header rows using the SKIP clause but to skip the last records you will have to use the WHEN clause. Typically, your trailing records (last records) will not be identical to the other records in the file and there should be an indicator to specify that this is a trailer record. You need to construct such a condition in your control file that this condition does not get satisfied.
Here is the Oracle documentation on the WHEN clause.
http://docs.oracle.com/cd/B14117_01/server.101/b10825/ldr_control_file.htm#i1005657
Here are some examples on conditional loading.
http://www.orafaq.com/wiki/SQL*Loader#Conditional_Load
Your original post needs more detail but for completeness the control file options clause has a LOAD = n option that tells sqlldr how many rows to load. If you have 100 rows and don't want to load the last 5, specify LOAD = 95.
I need to create a SQL*Loader control file that can read source files with missing columns. For example, file a:
a;b;c
And file b:
a;b
My control file looks like this:
load data
APPEND
into table table_of_parameter
fields terminated by ";" optionally enclosed by '"'
TRAILING NULLCOLS
(
parameters1,
parameters2,
parameters3
)
And I want this to be the result of loading both files:
select * from table_of_parameter
parameters1 parameters2 parameters3
a b c
a b null (or something else)
Is there a way to do this in a control file?
i solved this issue..
the problem was that the last paramether is "not Null"
so, if i put a different amount of value, it populare previous parameters that have "default value null" and can't populate this paramether.
Solved moved it before this paramether
I do have a flat file with not a fixed structure like
name,phone_num,Address
bob,8888,2nd main,5th floor,avenue road
Here the last column Address has the value 2nd main,5th floor,avenue road but since the same delimeter , is used for seperating columns also i am not getting any clue how to handle the same.
the structure of flat file may change from file to file.
How to handle such kind of flat files while importing using Informatica or SQL * Loader or UTL Files
I will not have any access to flat file just i should read the data from it but i can't edit the data in flat file.
Using SQLLoader
load data
append
into table schema.table
fields terminated by '~'
trailing nullcols
(
line BOUNDFILLER,
name "regexp_substr(:line, '^[^,]+')",
phone_num "regexp_substr(:line, '[^,]+', 1, 2)",
Address "regexp_replace(:line, '^.*?,.*?,')"
)
you need to change your source file to enclose the fields in an escape character eg:
name,phone_num,Address
bob,8888,^2nd main,5th floor,avenue road^
then in sql-loader you'd put:
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '^'
just pick a delimiter that doesn't normally appear in your data.
If you could get the source data enclosed within double quotes ( or any quotes for that matter) you can make use of 'Optional Quotes' option in Informatica while reading from Flat file