I have the following table
Create table my_source(
ID number(15) not null,
Col_1 Varchar 2(3000),
Col_2 Varchar 2(3000),
Col_3 Varchar 2(3000),
Col_4 Varchar 2(3000),
Col_5 Varchar 2(3000),
...
Col_90 Varchar 2(3000)
);
This table have 6,926,220 rows.
Now I am going to create two table based on this table.
Target1
Create table el_temp as
select
id,
Col_1,
Col_2,
Col_3,
Col_4,
Col_5,
Col_6,
Col_7,
Col_8,
Col_9,
Col_10,
Col_11,
Col_12
from
my_source;
Target2:
Create table el_temp2 as
select DISTINCT
id,
Col_1,
Col_2,
Col_3,
Col_4,
Col_5,
Col_6,
Col_7,
Col_8,
Col_9,
Col_10,
Col_11,
Col_12
from
my_source;
select count(*) from el_temp; -- 6926220
select count(*) from el_temp2; --6880832
The only difference between el_temp and el_temp2 is the "distinct" operator.
Now I got the following result from SQL Developer
It is a surprise result to me that EL_TEMP, the one with more rows have a smaller size, while the el_temp2 have less row but a bigger size.
Could anyone share me any reason and how to avoid this happen?
Thanks in advance!
The most likely cause is that the table has undergone some updates to existing rows over its lifetime.
By default, when you create a table, we reserve 10% of the space in each block for rows to grow (due to updates). As updates occur, that space is used up, so your blocks might be (on average) around 95% full.
When you do "create table as select" from that table to another, we will take those blocks and pad them out again to 10% free space, thus making it slightly larger.
If PCTFREE etc is unfamiliar to you, I've also got a tutorial video to get you started here
https://youtu.be/aOZMp5mncqA
Related
I am trying to partition an existing table in Oracle 12C R1 using below SQL statement.
ALTER TABLE TABLE_NAME MODIFY
PARTITION BY RANGE (DATE_COLUMN_NAME)
INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
(
PARTITION part_01 VALUES LESS THAN (TO_DATE('01-SEP-2017', 'DD-MON-RRRR'))
) ONLINE;
Getting error:
Error report -
SQL Error: ORA-14006: invalid partition name
14006. 00000 - "invalid partition name"
*Cause: a partition name of the form <identifier> is
expected but not present.
*Action: enter an appropriate partition name.
Partition needs to be done on the basis of data datatype column with the interval of one month.
Min value of Date time column in the Table is 01-SEP-2017.
You can't partition an existing table like that. That statement is modifying the partition that hasn't been created yet. I don't know the automatic way to do this operation and I am not sure that you can do it.
Although I have done this thing many times but with manual steps. Do the following if you can't find an automated solution:
Create a partitioned table named table_name_part with your clauses and all your preferences.
Insert into this partitioned table all rows from original table. Pay attention to compression. If you have some compression on table (Basic or HCC) you have to use + APPEND hint.
Create on partitioned table your constrains and indexes from the original table.
Rename the tables and drop the original table. Do not drop it until you make some counts on them.
I saw that your table has the option to auto-create partition if it does not exists. (NUMTOYMINTERVAL(1,'MONTH')) So you have to create your table with first partition only. I assume that you have here a lot of read-only data, so you won't have any problem with consistency instead of last month. Probably there is some read-write data so there you have to be more careful with the moment when you want to insert data in new table and switch tables.
Hope to help you. As far as I know there might be a package named DBMS_REDEFINITION that can help you with an automated version of my steps. If you need more details or need some help on my method, please don't hesitate.
UPDATE:
From Oracle 12c R2 you can convert a table from an unpartitioned to a partitioned one with your method. Find a link below. Now this is a challenge for me and I am trying to convert, but I think there is no way to make this conversion online in 12c R1.
In previous releases you could partition a non-partitioned table using
EXCHANGE PARTITION or DBMS_REDEFINITION in an "almost online" manner,
but both methods require multiple steps. Oracle Database 12c Release 2
makes it easier than ever to convert a non-partitioned table to a
partitioned table, requiring only a single command and no downtime.
https://oracle-base.com/articles/12c/online-conversion-of-a-non-partitioned-table-to-a-partitioned-table-12cr2
Solution
I found a solution for you. Here you will have all of my steps that I run to convert table online. :)
1. Create regular table and populate it.
CREATE TABLE SCOTT.tab_unpartitioned
(
id NUMBER,
description VARCHAR2 ( 50 ),
created_date DATE
);
INSERT INTO tab_unpartitioned
SELECT LEVEL,
'Description for ' || LEVEL,
ADD_MONTHS ( TO_DATE ( '01-JAN-2017', 'DD-MON-YYYY' ),
-TRUNC ( DBMS_RANDOM.VALUE ( 1, 4 ) - 1 ) * 12 )
FROM DUAL
CONNECT BY LEVEL <= 10000;
COMMIT;
2. Create partitioned table with same structure.
--If you are on 11g create table with CREATE TABLE command but with different name. ex: tab_partitioned
CREATE TABLE SCOTT.tab_partitioned
(
id NUMBER,
description VARCHAR2 ( 50 ),
created_date DATE
)
PARTITION BY RANGE (created_date)
INTERVAL( NUMTOYMINTERVAL(1,'YEAR'))
(PARTITION part_2015 VALUES LESS THAN (TO_DATE ( '01-JAN-2016', 'DD-MON-YYYY' )),
PARTITION part_2016 VALUES LESS THAN (TO_DATE ( '01-JAN-2017', 'DD-MON-YYYY' )),
PARTITION part_2017 VALUES LESS THAN (TO_DATE ( '01-JAN-2018', 'DD-MON-YYYY' )));
--this is an alter command that works only in 12c.
ALTER TABLE tab_partitioned
MODIFY
PARTITION BY RANGE (created_date)
(PARTITION part_2015 VALUES LESS THAN (TO_DATE ( '01-JAN-2016', 'DD-MON-YYYY' )),
PARTITION part_2016 VALUES LESS THAN (TO_DATE ( '01-JAN-2017', 'DD-MON-YYYY' )),
PARTITION part_2017 VALUES LESS THAN (TO_DATE ( '01-JAN-2018', 'DD-MON-YYYY' )));
3. Check if the table can be converted. This procedure should run without any error.
Prerequisites: table should have an UNIQUE INDEX and a Primary Key constraint.
EXEC DBMS_REDEFINITION.CAN_REDEF_TABLE('SCOTT','TAB_UNPARTITIONED');
4. Run the following steps like I have done.
EXEC DBMS_REDEFINITION.START_REDEF_TABLE('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED');
var num_errors varchar2(2000);
EXEC DBMS_REDEFINITION.COPY_TABLE_DEPENDENTS('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED', 1,TRUE,TRUE,TRUE,FALSE,:NUM_ERRORS,FALSE);
SQL> PRINT NUM_ERRORS -- Should return 0
EXEC DBMS_REDEFINITION.SYNC_INTERIM_TABLE('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED');
EXEC DBMS_REDEFINITION.FINISH_REDEF_TABLE('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED');
At the end of the script you will see that the original table is partitioned.
Try Oracle Live SQL I used to use Oracle 11g EE and got the same error message. So I tried Oracle live SQL and it perfectly worked. It has very simple and easy to understand interface,
For example, I'm creating a sales table and inserting some dummy data and partition it using range partitioning method,
CREATE TABLE sales
(product VARCHAR(300),
country VARCHAR(100),
sales_year DATE);
INSERT INTO sales (product, country, sales_year )
VALUES ('Computer','Kazakhstan',TO_DATE('01/02/2018','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Mobile Phone','China',TO_DATE('23/12/2019','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Camara','USA',TO_DATE('20/11/2020','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Watch','Bangladesh',TO_DATE('19/03/2020','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Cake','Sri Lanka',TO_DATE('13/04/2021','DD/MM/YYYY'));
ALTER TABLE sales MODIFY
PARTITION BY RANGE(sales_year)
INTERVAL(INTERVAL '1' YEAR)
(
PARTITION sales_2018 VALUES LESS THAN(TO_DATE('01/01/2019','DD/MM/YYYY')),
PARTITION sales_2019 VALUES LESS THAN(TO_DATE('01/01/2020','DD/MM/YYYY')),
PARTITION sales_2020 VALUES LESS THAN(TO_DATE('01/01/2021','DD/MM/YYYY')),
PARTITION sales_2021 VALUES LESS THAN(TO_DATE('01/01/2022','DD/MM/YYYY'))
)ONLINE;
Finally, I can write SELECT query for partitions to confirm that the partitions are created successfully.
SELECT *
FROM sales PARTITION (sales_2020);
And it gives the expected output,
Table myfirst3 have 4 columns and 1.2 million records.
Table mtl_object_genealogy has over 10 million records.
Running the below code takes very long time. How to tune this code using with options?
WITH level1 as (
SELECT mln_parent.lot_number,
mln_parent.inventory_item_id,
gen.lot_num ,--fg_lot,
gen.segment1,
gen.rcv_date.
FROM mtl_lot_numbers mln_parent,
(SELECT MOG1.parent_object_id,
p.segment1,
p.lot_num,
p.rcv_date
FROM mtl_object_genealogy MOG1 ,
myfirst3 p
START WITH MOG1.object_id = p.gen_object_id
AND (MOG1.end_date_active IS NULL OR MOG1.end_date_active > SYSDATE)
CONNECT BY nocycle PRIOR MOG1.parent_object_id = MOG1.object_id
AND (MOG1.end_date_active IS NULL OR MOG1.end_date_active > SYSDATE)
UNION all
SELECT p1.gen_object_id,
p1.segment1,
p1.lot_num,
p1.rcv_date
FROM myfirst3 p1 ) gen
WHERE mln_parent.gen_object_id = gen.parent_object_id )
select /*+ NO_CPU_COSTING */ *
from level1;
execution plan
CREATE TABLE APPS.MYFIRST3
(
TO_ORGANIZATION_ID NUMBER,
LOT_NUM VARCHAR2(80 BYTE),
ITEM_ID NUMBER,
FROM_ORGANIZATION_ID NUMBER,
GEN_OBJECT_ID NUMBER,
SEGMENT1 VARCHAR2(40 BYTE),
RCV_DATE DATE
);
CREATE TABLE INV.MTL_OBJECT_GENEALOGY
(
OBJECT_ID NUMBER NOT NULL,
OBJECT_TYPE NUMBER NOT NULL,
PARENT_OBJECT_ID NUMBER NOT NULL,
START_DATE_ACTIVE DATE NOT NULL,
END_DATE_ACTIVE DATE,
GENEALOGY_ORIGIN NUMBER,
ORIGIN_TXN_ID NUMBER,
GENEALOGY_TYPE NUMBER,
);
CREATE INDEX INV.MTL_OBJECT_GENEALOGY_N1 ON INV.MTL_OBJECT_GENEALOGY(OBJECT_ID);
CREATE INDEX INV.MTL_OBJECT_GENEALOGY_N2 ON INV.MTL_OBJECT_GENEALOGY(PARENT_OBJECT_ID);
Your explain plan shows some very big numbers. The optimizer reckons the final result set will be about 3227,000,000,000 rows. Just returning that many rows will take some time.
All table accesses are Full Table Scans. As you have big tables that will eat time too.
As for improvements, it's pretty hard to for us understand the logic of your query. This is your data model, you business rules, your data. You haven't explained anything so all we can do is guess.
Why are you using the WITH clause? You only use the level result set once, so just have a regular FROM clause.
Why are you using UNION ALL? That operation just duplicates the records retrieved from myfirst3 ( all those values are already included as rows where MOG1.object_id = p.gen_object_id.
The MERGE JOIN CARTESIAN operation is interesting. Oracle uses it to implement transitive closure. It is an expensive operation but that's because treewalking a hierarchy is an expensive thing to do. It is unfortunate for you that you are generating all the parent-child relationships for a table with 27 million records. That's bad.
The full table scans aren't the problem. There are no filters on myfirst3 so obviously the database has to get all the records. If there is one parent for each myfirst3 record that's 10% of the contents mtl_object_genealogy so a full table scan would be efficient; but you're rolling up the entire hierarchy so it's like you're looking at a much greater chunk of the table.
Your indexes are irrelevant in the face of such numbers. What might help is a composite index on mtl_object_genealogy(OBJECT_ID, PARENT_OBJECT_ID, END_DATE_ACTIVE).
You want all the levels of PARENT_OBJECT_ID for the records in myfirst3. If you run this query often and mtl_object_genealogy is a slowly changing table you should consider materializing the transitive closure into a table which just has records for all the permutations of leaf records and parents.
To sum up:
Ditch the WITH clause
Drop the UNION ALL
Tune the tree-walk with a composite index (or materializing it)
Need help query performance.
I have a table A joining to a view and it is taking 7 seconds to get the results. But when i do select query on view i get the results in 1 seconds.
I have created the indexes on the table A. But there is no improvements in the query.
SELECT
ITEM_ID, BARCODE, CONTENT_TYPE_CODE, DEPARTMENT, DESCRIPTION, ITEM_NUMBER, FROM_DATE,
TO_DATE, CONTACT_NAME, FILE_LOCATION, FILE_LOCATION_UPPER, SOURCE_LOCATION,
DESTRUCTION_DATE, SOURCE, LABEL_NAME, ARTIST_NAME, TITLE, SELECTION_NUM, REP_IDENTIFIER,
CHECKED_OUT
FROM View B,
table A
where B.item_id=A.itemid
and status='VALID'
AND session_id IN ('naveen13122016095800')
ORDER BY item_id,barcode;
CREATE TABLE A
(
ITEMID NUMBER,
USER_NAME VARCHAR2(25 BYTE),
CREATE_DATE DATE,
SESSION_ID VARCHAR2(240 BYTE),
STATUS VARCHAR2(20 BYTE)
)
CREATE UNIQUE INDEX A_IDX1 ON A(ITEMID);
CREATE INDEX A_IDX2 ON A(SESSION_ID);
CREATE INDEX A_IDX3 ON A(STATUS);'
So querying the view joined to a table is slower than querying the view alone? This is not surprising, is it?
Anyway, it doesn't make much sense to create separate indexes on the fields. The DBMS will pick one index (if any) to access the table. You can try a composed index:
CREATE UNIQUE INDEX A_IDX4 ON A(status, session_id, itemid);
But the DBMS will still only use this index when it sees an advantage in this over simply reading the full table. That means, if the DBMS expects to have to read a big amount of records anyway, it won't indirectly access them via the index.
At last two remarks concerning your query:
Don't use those out-dated comma-separated joins. They are less readable and more prone to errors than explicit ANSI joins (FROM View B JOIN table A ON B.item_id = A.itemid).
Use qualifiers for all columns when working with more than one table or view in your query (and A.status='VALID' ...).
UPDATE: I see now, that you are not selecting any columns from the table, so why join it at all? It seems you are merely looking up whether a record exists in the table, so use EXISTS or IN accordingly. (This may not make it faster, but a lot more readable at least.)
SELECT
ITEM_ID, BARCODE, CONTENT_TYPE_CODE, DEPARTMENT, DESCRIPTION, ITEM_NUMBER, FROM_DATE,
TO_DATE, CONTACT_NAME, FILE_LOCATION, FILE_LOCATION_UPPER, SOURCE_LOCATION,
DESTRUCTION_DATE, SOURCE, LABEL_NAME, ARTIST_NAME, TITLE, SELECTION_NUM, REP_IDENTIFIER,
CHECKED_OUT
FROM View
WHERE itemid IN
(
SELECT itemid
FROM A
WHERE status = 'VALID'
AND session_id IN ('naveen13122016095800')
)
ORDER BY item_id, barcode;
I am not sure if this is duplicated ,I havent find it in the search.
I have a table called mytable that has column STORY the type of this column is CLOB
mytable
The elder tree
Soldiers
Going for a hunt
The blue moon
If i write :
select story from mytable
I will have the result:
Mytable
1-clob
2-clob
3-clob
4-clob
What I want what inside CLOB , can I achieve that ?
dbms_lob.substr( clob, bytes, startbyte );
but in sql you can retrieve only 4000 bytes into varchar
I'm having a weird problem with index organized table. I'm running Oracle 11g standard.
i have a table src_table
SQL> desc src_table;
Name Null? Type
--------------- -------- ----------------------------
ID NOT NULL NUMBER(16)
HASH NOT NULL NUMBER(3)
........
SQL> select count(*) from src_table;
COUNT(*)
----------
21108244
now let's create another table and copy 2 columns from src_table
set timing on
SQL> create table dest_table(id number(16), hash number(20), type number(1));
Table created.
Elapsed: 00:00:00.01
SQL> insert /*+ APPEND */ into dest_table (id,hash,type) select id, hash, 1 from src_table;
21108244 rows created.
Elapsed: 00:00:15.25
SQL> ALTER TABLE dest_table ADD ( CONSTRAINT dest_table_pk PRIMARY KEY (HASH, id, TYPE));
Table altered.
Elapsed: 00:01:17.35
It took Oracle < 2 min.
now same exercise but with IOT table
SQL> CREATE TABLE dest_table_iot (
id NUMBER(16) NOT NULL,
hash NUMBER(20) NOT NULL,
type NUMBER(1) NOT NULL,
CONSTRAINT dest_table_iot_PK PRIMARY KEY (HASH, id, TYPE)
) ORGANIZATION INDEX;
Table created.
Elapsed: 00:00:00.03
SQL> INSERT /*+ APPEND */ INTO dest_table_iot (HASH,id,TYPE)
SELECT HASH, id, 1
FROM src_table;
"insert" into IOT takes 18 hours !!! I have tried it on 2 different instances of Oracle running on win and linux and got same results.
What is going on here ? Why is it taking so long ?
The APPEND hint is only useful for a heap-organized table.
When you insert into an IOT, I suspect that each row has to be inserted into the real index structure separately, causing a lot of re-balancing of the index.
When you build the index on a heap table, a temp segment is used and I'm guessing that this allows it to reduce the re-balancing overhead that would otherwise take place.
I suspect that if you created an empty, heap-organized table with the primary key, and did the same insert without the APPEND hint, it would take more like the 18 hours.
You might try putting an ORDER BY on your SELECT and see how that affects the performance of the insert into the IOT. It's not guaranteed to be an improvement by any means, but it might be.