I am using below CTL file to load data into table
Load data
Append
Into table abc
Fields terminated by ',' optionally enclosed by '"'
Trailing nullcols
(
R_date date 'mm/dd/yyyy hh:mm:ss'
)
Csv file value is as
R_date
09/12/2023 12:30:34
08/11/2023 22;30:45
In table abc r_date column datatype is date.
Ora-01840 input value not long enough for date format.
Noting we have written in above file
I think you want:
R_date date "mm/dd/yyyy hh24:mi:ss"
Related
The data from the infile is in the format MM/DD/YYYY how do I tell the control file to load it into the database as YYYYMM?
When you specify the columns in the INFILE declare just identify the format the data is held in. Like this
load data
infile 'whatever.csv'
into table t23
fields terminated by ','
trailing nullcols
(
col_1 integer
, col_2 char
, col_3 date "MM/DD/YYYY"
, col_4 date "MM/DD/YYYY"
, col_5 char
)
Don't worry about the "to" date format. That is only for display purposes. Oracle stores dates in its own internal representation.
Are you trying to load the MM/DD/YYYY data into a char/varchar2 field, or into a date field?
If you're trying to load it into a date field and you want to preserve the day of the month, APC's answer is correct. You can always just present the YYYYMM if that's what you want to do.
If you're trying to load it into a date field and you want to truncate it to the first day of the month, I think something like this would work:
date_column date "MM/DD/YYYY" "trunc(:date_column, 'mm')"
If inserting into a CHAR/VARCHAR2 column, you'd could to transform it a little differently:
vc2_column char "substr(:vc2_column, 7, 4) || substr(:vc2_column, 1, 2)"
I am trying to store the date and timestamp values in timestamp column using hive. The source file contain the values of date or sometimes timestamps.
Is there a way to read both date and timestamp by using the timestamp data type in hive.
Input:
2015-01-01
2015-10-10 12:00:00.232
2016-02-01
Output which I am getting:
null
2015-10-10 12:00:00.232
null
Is it possible to read both values by using timestamp data type.
DDL:
create external table mytime(id string ,t timestamp) ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 'hdfs://xxx/data/dev/ind/'
I was able think of a workaround. tried this with a small set of data:
Load the data with inconsistent date data into a hive table say table1 by making the column as string datatype .
Now create another table table2 with the datatype as timestamp for the required column and load the data from table1 to table2 using the transformation INSERT OVERWRITE TABLE table2 select id,if(length(tsstr) > 10, tsstr, concat(tsstr,' 00:00:00')) from table1;
This should load the data in required format.
Code as below:
`
create table table1
(
id int,
tsstr string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
LOCATION '/user/cloudera/hive/table1.tb';
Data:
1,2015-04-15 00:00:00
2,2015-04-16 00:00:00
3,2015-04-17
LOAD DATA LOCAL INPATH '/home/cloudera/data/tsstr' INTO TABLE table1;
create table table2
(
id int,
mytimestamp timestamp
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
LOCATION '/user/cloudera/hive/table2.tb';
INSERT INTO TABLE table2 select id,if(length(tsstr) > 10, tsstr, concat(tsstr,' 00:00:00')) from table1;
Result shows up as expected:
Hive is similar to any other database in terms of datatype mapping and hence requires a uniform values for a specific column to be stored under a conformed datatype. The data in your file for second column has non-uniform data i.e, some are in date format while others in timestamp format.
In order to not to lose the date, as suggested by #Kishore , make sure you have a uniform datatype in the file and get the file with timestamp values as 2016-01-01 00:00:000 where there are only dates.
I have text comma separated file in format below.
"RAZY","","","","","","","","N","Y","27-JUL-14","1","98727076437","27-JUL-14"
"TEST","","","","","","","","N","Y","27-JUN-14","1","98727076437","27-JUN-79"
I am trying to load this using a SQL loader controlfile.
OPTIONS (SKIP=0)
LOAD DATA
INFILE '/home/test.txt'
BADFILE '/home/test.bad'
DISCARDFILE '/home/test.dis'
APPEND
INTO TABLE "TEST"
fields terminated by ","
optionally enclosed by '"'
TRAILING NULLCOLS
(
NAME ,
TITLE ,
INITIAL ,
SURNAME ,
STREET ,
TOWN ,
COUNTY ,
POSTCODE ,
TELEPHONE ,
FLAG ,
Update_DATE DATE "TO_DATE(:Update_DATE,'DD-MON-RR')",
Update_STATUS ,
VISIT_REFERENCE ,
LOADED_DATE DATE "TO_DATE(:LOADED_DATE,'DD-MON-RR')"
)
But returned an error "ORA-01821: date format not recognized"
Thanks
Try this.
Update_DATE DATE "DD-MON-RR",
LOADED_DATE DATE "DD-MON-RR"
I have csv file that contains shown below :
year,mon,day
2014,12,01
2013,10,30
and abc's table
create table abc (
year varchar2(4),
mon varchar2(2),
day varchar2(2),
date1 date
)
how to tell loader.ctl to merger year,mon,day become date?
like that ?
OPTIONS (SKIP=11, errors=12000)
LOAD DATA
APPEND INTO TABLE abc (
year "trim (:year)",
mon "trim (:mon)",
day "trim (:day)",
date1 "to_date (year||mon||day,'yyyymmdd')"
)
Since your fields are comma separated, you should include fields terminated by "," in your control file.
date1 is not present in the data file, but is computed. So use EXPRESSION parameter to specify that.
in to_date function, use colons to reference the previous fields.
So, your control file should look like this,
OPTIONS (SKIP=11, errors=12000)
LOAD DATA
APPEND INTO TABLE abc
fields terminated by ","
(
year "trim (:year)",
mon "trim (:mon)",
day "trim (:day)",
date1 EXPRESSION "to_date (:year||:mon||:day,'yyyymmdd')"
)
I have csv file that has "17 September, 2009 11:06:06 AM" as COMPLETED_ON variable
I am using sql loader to load data to oracle with folowing:
LOAD DATA
INFILE 'c:/load/file_name.csv'
APPEND
INTO TABLE tbl_name
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(
COMPLETED_ON DATE "not sure what to put here",
)
in oracle I have this column created as follows:
"COMPLETED_ON" TIMESTAMP (0) WITH LOCAL TIME ZONE
How do I change COMPLETED_ON date in control file?
I'm not 100% with the LOAD DATA INFILE syntax, but I know the format string 'DD Month, YYYY HH:MI:SS AM' matches your date format. I was able to use it in a TO_DATE to convert your sample date. Try this:
LOAD DATA
INFILE 'c:/load/CW_COMPLIANCE.csv'
APPEND
INTO TABLE tbl_name
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(
COMPLETED_ON DATE 'DD Month, YYYY HH:MI:SS AM',
)