I have a kind of metadata table in my database that I want to cull of old records.
I have created a "select" statement that selects the rows I want to delete:
select m.tablename, m.OWNERNAME
from MAPINFO.MAPINFO_MAPCATALOG m
left outer join sys.ALL_TABLES t
on TRIM(m.tablename) = t.TABLE_NAME and TRIM(m.OWNERNAME) = t.owner
where t.num_rows is null
This gives me 113 rows.
However, I can't figure out how to convert this to a "delete". I was going to just use:
delete from MAPINFO.MAPINFO_MAPCATALOG where tablename in (...)
But this deletes 115 rows. There are two problems:
I need to compare tablename AND ownername. By only comparing tablename I'd be deleting two tables that shouldn't be deleted.
The table has no unique keys and I'm not in a position to create them.
Given the above, how should I go about performing this delete?
You can do this:
delete from MAPINFO.MAPINFO_MAPCATALOG where (tablename, ownername) in (...)
Related
I am using Toad for oracle 12c. I need to copy a table and data (40M) from one shcema to another (prod to test). However there is an unique key(not the PK for this table) called record_Id col which has something data like this 3.000*******19E15. About 2M rows has same numbers(I believe its because very large number) which are unique in prod. When I try to copy it violets the unique key of that col. I am using toad "export data to another schema" function to copy the data.
when I execute query in prod
select count(*) from table_name
OR
select count(distinct(record_id) from table_name
Both query gives the exact same numbers of data.
I don't have DBA permission. How do I copy all data without violating unique key of the table.
Thanks in advance!
You can use UPSERT for decisional INSERT or UPDATE or you may write small procedure for this.
you may consider to use NOT EXISTS, but your data is big and it might not be resource efficient.
insert into prod_tab
select * from other_tab t1 where NOT exists (
select 1 from prod_tab t2 where t1.id = t2.id
);
In Oracle you can use a MERGE query for that.
The following query proceeds as follows for each data row :
if the source record_id does not yet exist in the target table, a new record is inserted
else, the existing record is updated with source values
For the sake of the example, I assumed that there are two other columns in the table : column1 and column2.
MERGE INTO target_table t1
USING (SELECT * from source_table t2)
ON (t1.record_id = t2.record_id)
WHEN MATCHED THEN UPDATE SET
t1.column1 = t2.column1,
t1.column2 = t2.column2
WHEN NOT MATCHED THEN INSERT
(record_id, column1, column2) VALUES (t2.record_id, t2.column1, t2.column2)
I have a source table, and a history table. They are identical, except the history table has a notes column as the last column.
The table is large, 55 columns, which I do not want to list all the columns. We do not want to use a trigger on this, but just create the history entry in the code itself.
If I simply do an INSERT INTO <history> SELECT * FROM <source> WHERE...... I will get "not enough values".
I'm hoping to do something of this nature: (Note this is just an anonymous block)
DECLARE
v_old_rec company_table%ROWTYPE;
BEGIN
SELECT * INTO v_old_rec
FROM company_table
WHERE company_id = 32789;
INSERT INTO company_table_hist
VALUES v_old_rec || ',MONTHLY UPDATE';
END;
Anything like this possible, so I do not have to list 55 columns?
Thank you.
It's quite simple actually:
INSERT INTO company_table_hist
(SELECT t.*, MONTHLY_UPD
FROM company_table t
WHERE company_id = 32789);
Of course monthly_upd must be bound somehow
Wanted to optimize a query with the minus that it takes too much time ... if they can give thanked help.
I have two tables A and B,
Table A: ID, value
Table B: ID
I want all of Table A records that are not in Table B. Showing the value.
For it was something like:
Select ID, value
FROM A
WHERE value> 70
MINUS
Select ID
FROM B;
Only this query is taking too long ... any tips how best this simple query?
Thank you for attention
Are ID and Value indexed?
The performance of Minus and Not Exists depend:
It really depends on a bunch of factors.
A MINUS will do a full table scan on both tables unless there is some
criteria in the where clause of both queries that allows an index
range scan. A MINUS also requires that both queries have the same
number of columns, and that each column has the same data type as the
corresponding column in the other query (or one convertible to the
same type). A MINUS will return all rows from the first query where
there is not an exact match column for column with the second query. A
MINUS also requires an implicit sort of both queries
NOT EXISTS will read the sub-query once for each row in the outer
query. If the correlation field (you are running a correlated
sub-query?) is an indexed field, then only an index scan is done.
The choice of which construct to use depends on the type of data you
want to return, and also the relative sizes of the two tables/queries.
If the outer table is small relative to the inner one, and the inner
table is indexed (preferrable a unique index but not required) on the
correlation field, then NOT EXISTS will probably be faster since the
index lookup will be pretty fast, and only executed a relatively few
times. If both tables a roughly the same size, then MINUS might be
faster, particularly if you can live with only seeing fields that you
are comparing on.
Minus operator versus 'not exists' for faster SQL query - Oracle Community Forums
You could use NOT EXISTS like so:
SELECT a.ID, a.Value
From a
where a.value > 70
and not exists(
Select b.ID
From B
Where b.ID = a.ID)
EDIT: I've produced some dummy data and two datasets for testing to prove the performance increases of indexing. Note: I did this in MySQL since I don't have Oracle on my Macbook.
Table A has 2600 records with 2 columns: ID, val.
ID is an autoincrement integer
Val varchar(255)
Table b has one column, but more records than Table A. Autoincrement (in gaps of 3)
You can reproduce this if you wish: Pastebin - SQL Dummy Data
Here is the query I will be using:
select a.id, a.val from tablea a
where length(a.val) > 3
and not exists(
select b.id from tableb b where b.id = a.id
);
Without Indexes, the runtime is 986ms with 1685 rows.
Now we add the indexes:
ALTER TABLE `tablea` ADD INDEX `id` (`id`);
ALTER TABLE `tableb` ADD INDEX `id` (`id`);
With Indexes, the runtime is 14ms with 1685 rows. That's 1.42% the time it took without indexes!
I have 2 tables that are the same structure. One is a temp one and the other is a prod one. The entire data set gets loaded each time and sometimes this dataset will have deleted records from the prior datasets. I load the dataset into temp table first and if any records were deleted I want to deleted them from the prod table also.
So how can I find the records that exist in prod but not in temp? I tried outer join but it doesn't seem to be working. It's returning all the records from the table in the left or right depending on doing left or right outer join.
I then also want to delete those records in the prod table.
One way would be to use the MINUS operator
SELECT * FROM table1
MINUS
SELECT * FROM table2
will show all the rows in table1 that do not have an exact match in table2 (you can obviously specify a smaller column list if you are only interested in determining whether a particular key exists in both tables).
Another would be to use a NOT EXISTS
SELECT *
FROM table1 t1
WHERE NOT EXISTS( SELECT 1
FROM table2 t2
WHERE t1.some_key = t2.some_key )
How about something like:
SELECT * FROM ProdTable WHERE ID NOT IN
(select ID from TempTable);
It'd work the same as a DELETE statement as well:
DELETE FROM ProdTable WHERE ID NOT IN
(select ID from TempTable);
MINUS can work here
The following statement combines results with the MINUS operator, which returns only rows returned by the first query but not by the second:
SELECT * FROM prod
MINUS
SELECT * FROM temp;
Minus will only work if the table structure is same
Im looking for the best solution (performance wise) to achieve this.
I have to insert records into a table, avoiding duplicates.
For example, take table A
Insert into A (
Select DISTINCT [FIELDS] from B,C,D..
WHERE (JOIN CONDITIONS ON B,C,D..)
AND
NOT EXISTS
(
SELECT * FROM A ATMP WHERE
ATMP.SOMEKEY = A.SOMEKEY
)
);
I have an index over A.SOMEKEY, just to optimize the NOT EXISTS query, but i realize that inserting on an indexed table will be a performance hit.
So I was thinking of duplicating Table A in a Global Temporary Table, where I would keep the index. Then, removing the index from Table A and executing the query, but modified
Insert into A (
Select DISTINCT [FIELDS] from B,C,D..
WHERE (JOIN CONDITIONS ON B,C,D..)
AND
NOT EXISTS
(
SELECT * FROM GLOBAL_TEMPORARY_TABLE_A ATMP WHERE
ATMP.SOMEKEY = A.SOMEKEY
)
);
This would solve the "inserting on an index table", but I would have to update the Global Temporary A with each insertion I make.
I'm kind of lost here,
Is there a better way to achieve this?
Thanks in advance,
if the column A.SOMEKEY is declared NOT NULL and if you insert a large amound of data, a NOT IN clause might be more efficient than your NOT EXISTS since it will be able to use a HASH ANTI-JOIN.
INSERT INTO A
(SELECT DISTINCT FIELDS
FROM B, C, D ..
WHERE (JOIN CONDITIONS ON B, C, D..)
AND [B].SOMEKEY NOT IN (SELECT SOMEKEY FROM A)
AND [B].SOMEKEY IS NOT NULL;
HASH ANTI-JOINS are brutally efficient with large data sets.
I don't think the temporary table is a good idea in that case because you will be in one of these two cases:
the temporary table is indexed on SOMEKEY, your point about inserting into an indexed table being therefore moot
the temporary table is unindexed and your anti-join will be inefficient
Which method is the most efficient will probably depends upon the volume of data.
How about having the index on the table A.
create table b (same structure as table a) with NOLOGGING
Insert /*+APPEND */ into b (
Select DISTINCT [FIELDS] from B,C,D..
WHERE (JOIN CONDITIONS ON B,C,D..)
AND
NOT EXISTS
(
SELECT * FROM A ATMP WHERE
ATMP.SOMEKEY = A.SOMEKEY
)
);
Then drop the index on A and INSERT INTO A SELECT * FROM B
You could make B a global temporary table, but make sure that the data is persistent for the session as dropping the index will implictly commit.