Oracle:inner join between file and table - oracle

I'm new to oracle and plsql, so just bear with me.
I have a file TYPES.txt,
id,name,values
1,aaa,32
2,bbb,23
3,cvv,12
4,fff,54
I also have a table in my db, PARTS.ATTRIBUTES
id,name,props,crops
1,aaa,100,zzzz
2,bbb,200,yyyy
3,cvv,300,xxxx
4,fff,400,wwww
5,sasa,343,gfgg
6,uyuy,897,hhdf
I'd like to do an INNER JOIN on the file TYPES and ATTRIBUTES based on the column name. Now, i have done this by initially loading file TYPES into a temp table and then doing INNER JOIN between the temp table and ATTRIBUTES table.
But i'd like to know whether it is possible to do INNER JOIN between TYPES file and ATTRIBUTES table without making use of a temp table.
I understand that i can load the file using and get respective rows using following script:
declare
file utl_file.file_type;
line varchar2(500);
begin
file :=utl_file.fopen('USER_DIR','TYPES.txt','r');
loop
utl_file.get_line(file ,line);
dbms_output.put_line(line);
end loop;
exception
when others then
utl_file.fclose(file);
end;
Could someone be kind enough to explain to me how i can do the join between file contents and the db table?
P.S. The file TYPES.txt is dynamically generated and can have different number of columns at different times.

One cleaner approach is to use an EXTERNAL TABLE.
Use a create statement like this to create TYPES_external table.
CREATE TABLE TYPES_external (
id NUMBER(5),
name VARCHAR2(50),
Values VARCHAR2(50)
)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY USER_DIR
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY ','
MISSING FIELD VALUES ARE NULL
(
id NUMBER(5),
name VARCHAR2(50),
Values VARCHAR2(50)
)
)
LOCATION ('TYPES.txt','TYPES.txt')
)
PARALLEL 5
REJECT LIMIT UNLIMITED;
Once created , you can use this external table(TYPES_external) just as you
use any database table for select operation.

Related

Oracle External Table - No Rows Selected

I've created an external table using the following definition
CREATE TABLE EXT_TABLE (CID NUMBER, CNAME VARCHAR2(20), FEES NUMBER)
ORGANIZATION EXTERNAL
(
TYPE ORACLE_LOADER
DEFAULT DIRECTORY TEST_DIR
ACCESS PARAMETERS
(RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY ','
(
CID INTEGER,
CNAME CHAR(20),
FEES INTEGER
)
)
LOCATION ('DATA.TXT'))
REJECT LIMIT UNLIMITED;
table has been created. But, when i try to select data from table, I don't find any records
SQL> select * from ext_table;
no rows selected
I've made sure directory and table have sufficient privileges for the user.
The data in text file;
1,JAVA,300
2,LINUX,400
3,ORACLE,400
4,EXCEL,500
RECORD is not the keyword you should use here.
it must be RECORDS.
Use this:
RECORDS DELIMITED BY NEWLINE
Cheers!!

How do you insert data into complex data type "Struct" in Hive

I'm completely new to Hive and Stack Overflow. I'm trying to create a table with complex data type "STRUCT" and then populate it using INSERT INTO TABLE in Hive.
I'm using the following code:
CREATE TABLE struct_test
(
address STRUCT<
houseno: STRING
,streetname: STRING
,town: STRING
,postcode: STRING
>
);
INSERT INTO TABLE struct_test
SELECT NAMED_STRUCT('123', 'GoldStreet', London', W1a9JF') AS address
FROM dummy_table
LIMIT 1;
I get the following error:
Error while compiling statement: FAILED: semanticException [Error
10044]: Cannot insert into target because column number type are
different 'struct_test': Cannot convert column 0 from struct to
array>.
I was able to use similar code with success to create and populate a data type Array but am having difficulty with Struct. I've tried lots of code examples I've found online but none of them seem to work for me... I would really appreciate some help on this as I've been stuck on it for quite a while now! Thanks.
your sql error. you should use sql:
INSERT INTO TABLE struct_test
SELECT NAMED_STRUCT('houseno','123','streetname','GoldStreet', 'town','London', 'postcode','W1a9JF') AS address
FROM dummy_table LIMIT 1;
You can not insert complex data type directly in Hive.For inserting structs you have function named_struct. You need to create a dummy table with data that you want to be inserted in Structs column of desired table.
Like in your case create a dummy table
CREATE TABLE DUMMY ( houseno: STRING
,streetname: STRING
,town: STRING
,postcode: STRING);
Then to insert in desired table do
INSERT INTO struct_test SELECT named_struct('houseno',houseno,'streetname'
,streetname,'town',town,'postcode',postcode) from dummy;
No need to create any dummy table : just use command :
insert into struct_test
select named_struct("houseno","house_number","streetname","xxxy","town","town_name","postcode","postcode_name");
is Possible:
you must give the columns names in sentence from dummy or other table.
INSERT INTO TABLE struct_test
SELECT NAMED_STRUCT('houseno','123','streetname','GoldStreet', 'town','London', 'postcode','W1a9JF') AS address
FROM dummy
Or
INSERT INTO TABLE struct_test
SELECT NAMED_STRUCT('houseno',tb.col1,'streetname',tb.col2, 'town',tb.col3, 'postcode',tb.col4) AS address
FROM table1 as tb
CREATE TABLE IF NOT EXISTS sunil_table(
id INT,
name STRING,
address STRUCT<state:STRING,city:STRING,pincode:INT>)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '.';
INSERT INTO sunil_table 1,"name" SELECT named_struct(
"state","haryana","city","fbd","pincode",4500);???
how to insert both (normal and complex)data into table

How to compare two Oracle schemas to get delta changes by Altering the table not to drop and recreate it

I've already tried out a tool named TOYS. I found it free but unfortunately it didn't work.
Then, I tried "RED-Gate Schema Compare for Oracle" but it uses the technique to drop and recreate the table mean while I need to just alter the table with the newly added/dropped columns.
Any help is highly appreciated
Thanks
Starting from Oracle 11g you could use dbms_metadata_diff package and specifically compare_alter() function to compare metadata of two schema objects:
Schema #1 HR
create table tb_test(
col number
)
Schema #2 HR2
create table tb_test(
col_1 number
)
select dbms_metadata_diff.compare_alter( 'TABLE' -- schema object type
, 'TB_TEST' -- object name
, 'TB_TEST' -- object name
, 'HR' -- by default current schema
, 'HR2'
) as res
from dual;
Result:
RES
-------------------------------------------------
ALTER TABLE "HR"."TB_TEST" ADD ("COL_1" NUMBER);
ALTER TABLE "HR"."TB_TEST" DROP ("COL");

PL/SQL XML parsing into relational tables

I am using a PL/SQL procedure for inserting values from XML to relational tables. The XML file resides in an XMLTYPE column.
Columns of table (OFFLINE_XML) containing XML are
ID, XML_FILE, STATUS
There are two table in which i want to insert the values i.e. DEPARTMENT and SECTIONS
Structure of DEPARTMENT is as under:-
ID, NAME
Structure of SECTIONS table is:-
ID, NAME, DEPARTMENT_ID
Now there is a third table (LIST_1) in which i want to insert the values which already exists in both the above mentioned tables.
Structure of LIST_1 is :-
ID, DEPARTMENT_ID,DEPARTMENT_NAME,SECTIONS_ID, SECTIONS_NAME
XML format is as under:-
<ROWSET>
<DEPARTMENT>
<DEPARTMENT_ID>DEP22681352268280797</DEPARTMENT_ID>
<DEPARTMENT_NAME>myDEPARTMENT</DEPARTMENT_NAME>
<SECTIONS_ID>6390135666643567</SECTIONS_ID>
<SECTIONS_NAME>mySection</SECTIONS_NAME>
</DEPARTMENT>
<DEPARTMENT>
<DEPARTMENT_ID>DEP255555555550797</DEPARTMENT_ID>
<DEPARTMENT_NAME>myDEPARTMENT2</DEPARTMENT_NAME>
<SECTIONS_ID>63901667779243567</SECTIONS_ID>
<SECTIONS_NAME>mySection2</SECTIONS_NAME>
</DEPARTMENT>
</ROWSET>
DECLARE
BEGIN
insert all
into department (id, name)
values (unit_id, unit_name)
into sections (id, name, department _id)
values ( sect_id, sect_name, department _id)
select department .id as department _id
, department.name as department_name
, sect.id as sect_id
, sect.name as sect_name
from OFFLINE_XML
, xmltable('/ROWSET/DEPARTMENT'
passing OFFLINE_XML.xml_file
columns
"ID" varchar2(20) path 'UNIT_ID'
, "NAME" varchar2(20) path 'UNIT_NAME'
) department
, xmltable('/ROWSET/DEPARTMENT'
passing OFFLINE_XML.xml_file
columns
"ID" varchar2(20) path 'SECTIONS_ID'
, "NAME" varchar2(20) path 'SECTIONS_NAME'
) sect
where status = 3;
EXCEPTION
WHEN DUP_VAL_ON_INDEX THEN
dbms_output.put_line('Duplicate='|| department.id );
--insert into LIST_1 values(ID,DEPARTMENT_ID, SECTIONS_ID, DEPARTMENT_NAME,SECTIONS_NAME);
END;
Now the problem is that how can i insert or identify the values on the basis of primary key which already exists in table DEPARTMENT and SECTIONS and thereafter insert the existing values in LIST_1 table.
------An updated effort --------------
I came up with another solution but this again is giving me problem. In the under mentioned procedure cursor tends to repeat for every xquery. I don't know how am i going to handle this issue..
DECLARE
department_id varchar2(20);
department_name varchar2(20);
sect_id varchar2(20);
sect_name varchar2(20);
sections_unit_id varchar2(20);
var number;
CURSOR C1 IS
select
sect.id as sect_id
, sect.name as sect_name
, sect.unit_id as sections_unit_id
from OFFLINE_XML
, xmltable('/ROWSET/DEPARTMENT'
passing OFFLINE_XML.xml_file
columns
"ID" varchar2(20) path 'UNIT_ID'
, "NAME" varchar2(20) path 'UNIT_NAME'
) DEPARTMENT
, xmltable('/ROWSET/DEPARTMENT'
passing OFFLINE_XML.xml_file
columns
"ID" varchar2(20) path 'SECTIONS_ID'
, "NAME" varchar2(20) path 'SECTIONS_NAME'
, "DEPARTMENT_ID" varchar2(20) path 'DEPARTMENT_ID'
) sect
where status = 3;
BEGIN
FOR R_C1 IN C1 LOOP
BEGIN
var :=1;
--insert into sections_temp_1 (id, name)values ( R_C1.sect_id, R_C1.sect_name);
-- commit;
dbms_output.put_line('Duplicate='||var);
EXCEPTION
WHEN DUP_VAL_ON_INDEX THEN
dbms_output.put_line('Duplicate='||R_C1.sect_id);
END;
var:=var+1;
END LOOP;
END;
Seems that first of all you need a little bit more complicated XQuery to extract rows from XMLType field.
There are no need to extract sections and departments separately and after that try to match it back.
Try this variant:
select
department_id,
department_name,
sections_id,
sections_name
from
OFFLINE_XML xml_list,
xmltable(
'
for $dept in $param/ROWSET/DEPARTMENT
return $dept
'
passing xml_list.xml_file as "param"
columns
"DEPARTMENT_ID" varchar2(100) path '//DEPARTMENT/DEPARTMENT_ID',
"DEPARTMENT_NAME" varchar2(4000) path '//DEPARTMENT/DEPARTMENT_NAME',
"SECTIONS_ID" varchar2(100) path '//DEPARTMENT/SECTIONS_ID',
"SECTIONS_NAME" varchar2(4000) path '//DEPARTMENT/SECTIONS_NAME'
) section_list
where
xml_list.Status = 3
SQL fiddle - 1
After that you got a dataset which can be outer joined to existing tables on it's primary keys (or something other - depends on required logic) if you want to find if any values already exists:
select
offline_set.offline_xml_id,
offline_set.department_id,
offline_set.department_name,
offline_set.sections_id,
offline_set.sections_name,
nvl2(dept.id,'Y', 'N') is_dept_exists,
nvl2(sect.id,'Y', 'N') is_sect_exists
from
(
[... skipped text of previous query ...]
) offline_set,
department dept,
sections sect
where
dept.id (+) = offline_set.department_id
and
sect.id (+) = offline_set.sections_id
SQL fiddle - 2
Because I actually unaware about logic behind this requirements, I can't suggest any future processing instructions. But it seems that you missed reference to OFFLINE_XML table in LIST_1 which needed to identify source of errors/duplicates.
The best way to do this would be with Oracle's built in error logging. Use DBMS_ERRLOG.CREATE_ERROR_LOG() to generate a logging table for each target table (i.e. SECTION and DEPARTMENT in your case). Find out more.
The syntax for using these tables with INSERT ALL is not intuitive but this is what to do:
insert all
into department (id, name)
values (unit_id, unit_name)
log errors into err$_department ('XML Load failure')
into sections (id, name, department_id)
values ( sect_id, sect_name, department_id)
log errors into err$_section ('XML Load failure')
select department.id as department_id
....
You can put any (short-ish) string into the error log label, but make sure it's something which will help you local the relevant records. You may wish to set the REJECT LIMIT to some value depending on whether you wish to fail on one (or a couple of) error, or process the whole XML and sort it out afterwards. Find out more.
I suggest you use separate logs for each target tables rather one log for both for two reasons:
In my expereince solutions which leverage Oracle's built-in feartures tend to scale better and be more robust than hand-rolled code.
It's a better fit for what might happen. You have three circumstances which might cause loading to hurl DUP_VAL_ON_INDEX:
Record has duplicate Department ID
Record has duplicate Section ID
Record has duplicate Department ID and duplicate Section ID
Separate tables make it easier to understand what's gone awry. This is a major boon when loading large amounts of data.
"i need to inform my user that this much of duplicate entries were
found in xml"
You can still do that with two error logs. Heck, you can even join the error logs into a view called LIST_1 is that is so very important to you.

How to duplicate all data in a table except for a single column that should be changed

I have a question regarding a unified insert query against tables with different data
structures (Oracle). Let me elaborate with an example:
tb_customers (
id NUMBER(3), name VARCHAR2(40), archive_id NUMBER(3)
)
tb_suppliers (
id NUMBER(3), name VARCHAR2(40), contact VARCHAR2(40), xxx, xxx,
archive_id NUMBER(3)
)
The only column that is present in all tables is [archive_id]. The plan is to create a new archive of the dataset by copying (duplicating) all records to a different database partition and incrementing the archive_id for those records accordingly. [archive_id] is always part of the primary key.
My problem is with select statements to do the actual duplication of the data. Because the columns are variable, I am struggling to come up with a unified select statement that will copy the data and update the archive_id.
One solution (that works), is to iterate over all the tables in a stored procedure and do a:
CREATE TABLE temp as (SELECT * from ORIGINAL_TABLE);
UPDATE temp SET archive_id=something;
INSERT INTO ORIGINAL_TABLE (select * from temp);
DROP TABLE temp;
I do not like this solution very much as the DDL commands muck up all restore points.
Does anyone else have any solution?
How about creating a global temporary table for each base table?
create global temporary table tb_customers$ as select * from tb_customers;
create global temporary table tb_suppliers$ as select * from tb_suppliers;
You don't need to create and drop these each time, just leave them as-is.
You're archive process is then a single transaction...
insert into tb_customers$ as select * from tb_customers;
update tb_customers$ set archive_id = :v_new_archive_id;
insert into tb_customers select * from tb_customers$;
insert into tb_suppliers$ as select * from tb_suppliers;
update tb_suppliers$ set archive_id = :v_new_archive_id;
insert into tb_suppliers select * from tb_suppliers$;
commit; -- this will clear the global temporary tables
Hope this helps.
I would suggest not having a single sql statement for all tables and just use and insert.
insert into tb_customers_2
select id, name, 'new_archive_id' from tb_customers;
insert into tb_suppliers_2
select id, name, contact, xxx, xxx, 'new_archive_id' from tb_suppliers;
Or if you really need a single sql statement for all of them at least precreate all the temp tables (as temp tables) and leave them in place for next time. Then just use dynamic sql to refer to the temp table.
insert into ORIGINAL_TABLE_TEMP (SELECT * from ORIGINAL_TABLE);
UPDATE ORIGINAL_TABLE_TEMP SET archive_id=something;
INSERT INTO NEW_TABLE (select * from ORIGINAL_TABLE_TEMP);

Resources