Hi I'm using the below select query which fetches all the rows from the table based on a single condition. It is like an export that we are geenrating. Here is the query.
Select emp_name, phone_number, emp_id from Employees where
emp_dept = 100;
Suppose if i have millions of records here and i'm not allowed to create any temporary table and index, what is the best way to optimize this query. Please note i need all the records of the table so the filters are actually less.
In the process of learning Oracle. So please let me know in case any mistakes are there.
Thanks.
Since
i'm not allowed to create any temporary table and index
then your only option seems to be to run this in parallel:
select --+ parallel(E,16)
emp_name, phone_number, emp_id
from employees E
where emp_dept = 100;
And your useful options are, depending on many factors ...
to ask your local DBA to add an index for you
to ask your local DBA to reorganize the table for you (partitioning, clusters)
to ask your local DBA to create a on-commit fast-refreshable query-rewritable mview for you
to ask your company to buy Exadata machine for you
etc.
Related
I only have a superficial understanding on partitions in Oracle, but, I know you can create persistent partitions on Oracle, for example within a create table statement, but, when using partition by clauses within a select statement? Will Oracle create a persistent partition, for caching reasons or whatever, or will the partition be "temporary" in some sense (e.g., it will be removed at the end of the session, the query, or after some time...)?
For example, for a query like
SELECT col1, first_value(col2)
over (partition by col3 order by col2 nulls last) as colx
FROM tbl
If I execute that query, will Oracle create a partition to speed up the execution if I execute it again, tomorrow or three months later? I'm worry about that because I don't know if it could cause memory exhaustion if I abuse that feature.
partition by is used in the query(windows function) to fetch the aggregated result using the windows function which is grouped by the columns mentioned in the partition by. It behaves like group by but has ability to provide grouped result for each row without actually grouping the final outcome.
It has nothing to do with table/index partition.
scope of this partition by is just this query and have no impact on table structure.
I am connecting to oracle using an ETL tool.The operation what I am doing is truncating an existing table and inserting records into that table from a different table. This is working fine for 15 to 20 cycles of job run. After that my job got stuck in the portion where its inserting record.Is there anything wrong which I am doing here. Please find the query I am using below.Could some one help on this, from the previous experience.
truncate table TABLE1;
insert into TABLE1 select * from TABLE_SRC where TYPE in('MP','DA')
and ID in(select ID from TABLE_SRC where TYPE in('MP','DA') and FLAG='Y');
commit;
I believe the table is going in lock situation.
Check with dba’s .
Select * from dba_lock ;
I have a fact table with millions of records. The table is range partitioned on a date column.
FACT_AUM (ACCOUNT_ID VARCHAR2(30),MARKET_VALUE NUMBER(20,6), POSTING_DATE DATE);
I have another temp table
ACCOUNT_TMP (ACCOUNT_ID VARCHAR2(30), POSTING_DATE DATE);
When I run this query by hard coding the date I see partition pruning happens and the results come back quickly
SELECT A.ACCOUNT_ID, SUM(A.MARKET_VALUE) FROM
FACT_AUM A JOIN ACCOUNT_TMP B ON A.ACCOUNT_ID = B.ACCOUNT_ID
AND A.POSTING_DATE=TO_DATE('30-DEC-2016',DD-MON-YYYY') GROUP BY
A.ACCOUNT_ID;
when I run the following, I don't see partition pruning and the query keeps spinning
SELECT A.ACCOUNT_ID, SUM(A.MARKET_VALUE) FROM
FACT_AUM A JOIN ACCOUNT_TMP B ON A.ACCOUNT_ID = B.ACCOUNT_ID
AND A.POSTING_DATE = B.POSTING_DATE GROUP BY
A.ACCOUNT_ID;
Any insights on this would be helpful.
Oracle used partition pruning while you hard coded the value, because Oracle felt it would get benefit of doing the partition pruning there.
When you joined the fact table with your temporary ( i would reword it to staging) table, Oracle wouldn't be able to guess which all partitions would it have to hit for computing the answer. Please note Oracle will assess what would be the range of values available in the staging table.
But unless you provide stats of the tables involved, i couldn't dwell into more important topics of the table ordering and tables joins. For quick fix use an Order hint or nested loop hint.
I have one table in oracle where data gets inserted from some third party. I want to populate master tables from that table. So, what will be the best way performance wise using collection.
E.g. Suppose, the table into which data will get populated from third party is 'EMP_TMP'.
Now I want to populate 'EMPLOYEE' master table through procedure which will get populated from EMP_TMP Table.
Here again there is one condition like IF SAME EMPID (this is not primary key) EXISTS then we have to UPDATE FULL TABLE which consists of SAME EMPID ELSE we have INSERT NEW RECORD.
[Note: Here EMPID is VARCHAR2 and EMPNO will be primary key where we will use SEQUENCE]
I think here merge will not perform much better performancewise since we cant use collection in MERGE statement.
Well, if performance is your primary consideration, and you don't like MERGE, then how about this (run as script, single transaction):
delete from EMPLOYEE where emp_id IN (
select emp_id from EMP_TMP);
insert into EMPLOYEE
select * from EMP_TMP;
commit;
Obviously not the "safest" approach (and as written assumes exact same table definitions and you have the rollback), but should be fast (you could also mess with IN vs EXISTS etc). And I couldn't quite understand your post if emp_id or emp_no was the common key in these 2 tables, but use whichever makes sense in your situation.
Create a procedure, you need to be using PL/SQL.
Do an update first then test sql%rowcount.
If it is 0, no updates where done and you have to do an insert instead.
I think that this is fairly efficient.
pseudo code
Update table;
if sql%rowcount = 0 then
//get new sequence number
insert into table;
END IF;
COMMIT;
HTH
Harv
I am facing problem in loading data. I have to copy 800,000 rows from one table to another in Oracle database.
I tried for 10,000 rows first but the time it took is not satisfactory. I tried using the "BULK COLLECT" and "INSERT INTO SELECT" clause but for both the cases response time is around 35 minutes. This is not the desired response I'm looking for.
Does anyone have any suggestions?
Anirban,
Using an "INSERT INTO SELECT" is the fastest way to populate your table. You may want to extend it with one or two of these hints:
APPEND: to use direct path loading, circumventing the buffer cache
PARALLEL: to use parallel processing if your system has multiple cpu's and this is a one-time operation or an operation that takes place at a time when it doesn't matter that one "selfish" process consumes more resources.
Just using the append hint on my laptop copies 800,000 very small rows below 5 seconds:
SQL> create table one_table (id,name)
2 as
3 select level, 'name' || to_char(level)
4 from dual
5 connect by level <= 800000
6 /
Tabel is aangemaakt.
SQL> create table another_table as select * from one_table where 1=0
2 /
Tabel is aangemaakt.
SQL> select count(*) from another_table
2 /
COUNT(*)
----------
0
1 rij is geselecteerd.
SQL> set timing on
SQL> insert /*+ append */ into another_table select * from one_table
2 /
800000 rijen zijn aangemaakt.
Verstreken: 00:00:04.76
You mention that this operation takes 35 minutes in your case. Can you post some more details, so we can see what exactly is taking 35 minutes?
Regards,
Rob.
I would agree with Rob. Insert into () select is the fastest way to do this.
What exactly do you need to do? If you're trying to do a table rename by copying to a new table and then deleting the old, you might be better off doing a table rename:
alter table
table
rename to
someothertable;
INSERT INTO SELECT is the fastest way to do it.
If possible/necessary, disable all indexes on the target table first.
If you have no existing data in the target table, you can also try CREATE AS SELECT.
As with the above, I would recommend the Insert INTO ... AS select .... or CREATE TABLE ... AS SELECT ... as the fastest way to copy a large volume of data between two tables.
You want to look up the direct-load insert in your oracle documentation. This adds two items to your statements: parallel and nologging. Repeat the tests but do the following:
CREATE TABLE Table2 AS SELECT * FROM Table1 where 1=2;
ALTER TABLE Table2 NOLOGGING;
ALTER TABLE TABLE2 PARALLEL (10);
ALTER TABLE TABLE1 PARALLEL (10);
ALTER SESSION ENABLE PARALLEL DML;
INSERT INTO TABLE2 SELECT * FROM Table 1;
COMMIT;
ALTER TABLE 2 LOGGING:
This turns off the rollback logging for inserts into the table. If the system crashes, there's not recovery and you can't do a rollback on the transaction. The PARALLEL uses N worker thread to copy the data in blocks. You'll have to experiment with the number of parallel worker threads to get best results on your system.
Is the table you are copying to the same structure as the other table? Does it have data or are you creating a new one? Can you use exp/imp? Exp can be give a query to limit what it exports and then imported into the db. What is the total size of the table you are copying from? If you are copying most of the data from one table to a second, can you instead copy the full table using exp/imp and then remove the unwanted rows which would be less than copying.
try to drop all indexes/constraints on your destination table and then re-create them after data load.
use /*+NOLOGGING*/ hint in case you use NOARCHIVELOG mode, or consider to do the backup right after the operation.