I have a target table T1 which doesn't have a Date field. The current size is increasing rapidly. Hence I need to add a Date field and also perform table partitioning on this target table.
T1 has PRIMARY KEY (DOCID, LABID)
and has CONSTRAINT FOREIGN KEY (DOCID) REFERENCES T2
Table T2 is also complex table and has many rules in it.
T2 has PRIMARY KEY (DOCID)
My question is, as I need to partition T1. Is it possible NOT TO perform any step for T2 before T1 is partition? DBA told me that I need to partition T2 first before I touch T1??
There is no need to partition T2 before partitioning T1. Foreign key constraints do not care in the slightest bit about partitioning.
Best of luck.
You have – as proposed by others - two options. The first one is to add a redundant column DATE to the table T1 (child) and introduce the range partitioning on this column.
The second option is to use reference partitioning. Below is the simplified DDL for those options.
Range partitioning on child
create table T2_P2 /* parent */
(docid number not null,
trans_date date not null,
pad varchar2(100),
CONSTRAINT t2_p2_pk PRIMARY KEY(docid)
);
create table T1_P2 /* child */
(docid number not null,
labid number not null,
trans_date date not null, /** redundant column **/
pad varchar2(100),
CONSTRAINT t1_p2_pk PRIMARY KEY(docid, labid),
CONSTRAINT t1_p2_fk
FOREIGN KEY(docid) REFERENCES T2_P2(docid)
)
PARTITION BY RANGE(trans_date)
( PARTITION Q1_2016 VALUES LESS THAN (TO_DATE('01-APR-2016','DD-MON-YYYY'))
);
Reference partition
create table T2_RP /* parent */
(docid number not null,
trans_date date not null,
pad varchar2(100),
CONSTRAINT t2_rp_pk PRIMARY KEY(docid)
)
PARTITION BY RANGE(trans_date)
( PARTITION Q1_2016 VALUES LESS THAN (TO_DATE('01-APR-2016','DD-MON-YYYY'))
);
create table T1_RP /* child */
(docid number not null,
labid number not null,
pad varchar2(100),
CONSTRAINT t1_rp_pk PRIMARY KEY(docid, labid),
CONSTRAINT t1_rp_fk
FOREIGN KEY(docid) REFERENCES T2_RP(docid)
)
PARTITION BY REFERENCE(t1_rp_fk);
Your question is basically if the first option is possible, so the answer is YES.
To decide if the first option is preferable I’d suggest checking three criteria:
Migration
The first option requires a new DATE column in the child table that must be initialized during the migration (and of course correct maintained by the application).
Lifecycle
It could be that the lifecycle of both tables is the same (e.g. both parent and child records are kept for 7 year). In this case is preferable if both tables are partitioned (on the same key).
Access
For queries such as below you will profit from the reference partitioning (both tables are pruned - i.e. only the partitions with the accessed date are queried).
select * from T2_RP T2 join T1_RP T1 on t2.docid = t1.docid
where t2.trans_date = to_date('01012016','ddmmyyyy');
In the first option you will (probalbly) end with FTS on T2 and to get pruning on T1 you need to add predicate T2.trans_date = t1.trans_date
Having said that, I do not claim that the reference partition is superior. But I think it's worth to examine both options in your context and see which one is better.
Related
We have realized that we need to use Oracle (12 c) interval partitioning.We have an hierarchical entity model with lots of #OneToMany relationships.We want to use "partitioning by range" (day,month... ) on the "parent/root" entity (A) and "partition by reference" on all child entities (B).
From the Oracle documentation: "Reference partitioning allows the partitioning of two tables related to one another by referential constraints. The partitioning key is resolved through an existing parent-child relationship, enforced by enabled and active primary key and foreign key constraints." The problem is that child entities (B) can refer to other entities ( C) that they don't have any link with the "parent/root" entity (A). I can create partitions on A and on B but when I want to drop the partition on A ( partition on B on cascade), I get an error integrity error and it fails. it works only if I delete all records on C and B and then partition them. I don't want to do that as it's not efficient and slow compared to dropping partitions directly
Please is there a way to create a partition on table C based on A(creation_date) without adding any foreign constraint between A and C?
Small example to illustrate the case
A - parent entity
B - child entity to A
C - child entity to B
create table
A (
id number primary key,
creation_date date
)
partition by range (creation_date)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
partition p1 values less than (to_date('20180501','yyyymmdd'))
);
create table
B (
id number primary key,
value varchar2(5),
a_id number not null,
constraint fk_ba foreign key (a_id) references A
)
partition by reference(fk_ba);
create table
C (
id number primary key,
code varchar2(5),
b_id number not null,
constraint fk_cb foreign key (b_id) references B
);
It was not clear in the documentation that partition by reference is transitive ( A->B->C). I have discovered it by testing. we can create table C "partition by reference (fk_cb)" with the result that when we drop A partition the matching B and C partitions get dropped.
In code, I tried with #Interleaved in 1-many relationship at non-owning side to get child list. Could anyone help with below questions:
How to implement bidirectional relationship e.g. get parent from child for 1-1, 1-many relationship
Regarding many-many relationship, what are best practices to implement it and how to implement bidirectional relationship for it.
Thank you very much.
Cloud Spanner currently doesn't offer a way to enforce foreign-key constraints between non-interleaved tables. You will have to enforce such constraints in your application logic. You could use DML statements in Cloud Spanner(that come with the ability to read-your-writes in a Cloud Spanner transaction) to enforce these constraints at insert time by inserting into your tables as follows:
INSERT INTO Referenced(key1,value1) VALUES ('Referenced','Value1');
INSERT INTO Referencing(key2, value2, key1)
SELECT 'Referencing', 'Value2', key1 FROM Referenced WHERE
key1 = 'Referenced';
Running the two statements in a read-write transaction will ensure that the PK-FK relationship between the Referenced and Referencing table is always maintained at insert time. You may have to similarly modify update requests/SQL update statements in your application logic to enforce the PK-FK constraint for updates.
For a 1-many relationship, when using interleaved tables, then the child row's primary key already contains the primary key of its parent, so it is trivial to get the parent row.
CREATE TABLE parent (
parent_key INT64 NOT NULL,
...
) PRIMARY KEY (parent_key);
CREATE TABLE child (
parent_key INT64 NOT NULL,
child_key INT64 NOT NULL,
...
) PRIMARY KEY (parent_key, child_key),
INTERLEAVE IN PARENT parent ON DELETE CASCADE;
If for some reason you do not have the key of the parent, and only the key of the child, then for efficiency you would need to create an index for the reverse lookup:
CREATE INDEX child_to_parent_index
ON child (
child_key
);
and force use of that index when performing the query for the parent:
SELECT
p.*
FROM
parent as p
JOIN
child#{FORCE_INDEX=child_by_id_index} AS c ON p.parent_key = c.parent_key
WHERE
c.child_key = #CHILD_KEY_VALUE;
Many-many relationships would have to be implemented using a 'mapping' table linking table1-key to table2-key.
You will also need a top-level index to get efficient reverse-lookups, and use the FORCE_INDEX directive as above in your queries.
And as #adi mentioned, foreign key constraints would have to be enforced by the application.
CREATE TABLE table1 (
table1_key INT64 NOT NULL,
...
) PRIMARY KEY (table1_key);
CREATE TABLE table2 (
table2_key INT64 NOT NULL,
...
) PRIMARY KEY (table2_key);
CREATE TABLE table1_table2_map (
table1_key INT64 NOT NULL,
table2_key INT64 NOT NULL,
) PRIMARY KEY (table1_key, table2_key);
CREATE INDEX table2_table1_map_index
ON table1_table2_map (
table2_key
) STORING (
table1_key
);
Your application would be responsible for keeping the referential integrity of the mapping table - deleting the mapping rows when rows in table1 or table2 are deleted
If you want to use interleaved tables, then if your application needs to perform bi-directional lookups, you may have to create 2 mapping tables - as a child of each parent, so that finding the mappings from both directions are equally efficient.
CREATE TABLE table1 (
table1_key INT64 NOT NULL,
...
) PRIMARY KEY (table1_key);
CREATE TABLE table2 (
table2_key INT64 NOT NULL,
...
) PRIMARY KEY (table2_key);
CREATE TABLE table1_table2_map (
table1_key INT64 NOT NULL,
table2_key INT64 NOT NULL,
) PRIMARY KEY (table1_key, table2_key),
INTERLEAVE IN PARENT table1 ON DELETE CASCADE;
CREATE TABLE table2_table1_map (
table2_key INT64 NOT NULL,
table1_key INT64 NOT NULL,
) PRIMARY KEY (table2_key, table1_key),
INTERLEAVE IN PARENT table2 ON DELETE CASCADE;
Note that the application needs to keep both of these mapping tables up to date -- ie when deleting a row from table1, the application has to get the referenced table2_key values and delete the mappings from the table2_table1_map (and vice versa).
I am trying to partition my table as below
CREATE TABLE ABC(
id VARCHAR2(100) primary key,
datecreated DATE)
PARTITION BY RANGE (datecreated) INTERVAL (NUMTODSINTERVAL(1,'DAY'))
SUBPARTITION BY HASH (ID) SUBPARTITIONS 4
(PARTITION lessthan2018 VALUES LESS THAN (TIMESTAMP' 2018-01-01 00:00:00') );
If I use WHERE clause with column "ID" will the performance improve? Will the performance be the same if I just partition by date. Will it perform the same since ID is already a primary key?
If id is the primary key then you will have a unique global index on that column, and partitioning will not make any difference because the index already takes you to the physical address of the specified row.
Also, dropping, truncating or exchanging a partition will invalidate the index unless you specify the update global indexes clause.
If id was not the PK but something non-unique like a product type, or area code, then a query for just that column without any date would need to check every partition.
I have 3 tables,which are not created with the ON DELETE CASCADE option, nor is it an option to create them as such.
I may need to delete from all three tables in succession. Is there a way to do this using only the promotion_id as a key? Because I need to delete in reverse order, the promotion_id is gone by the time I get to the dependent tables.
I am thinking that the only way to do this is to SELECT the keys of the 3 tables using a JOIN, and then use them individually. But it would be nice if there was a pure SQL solution to it.
I am using JDBC, Spring, and Oracle. Thanks.
create table test_rates (
rate_id varchar2(10) primary key,
rate number
);
create table test_offers (
offer_id varchar2(10) primary key ,
rate_id varchar2(10),
foreign key (rate_id) references test_rates (rate_id)
);
create table test_promotions (
promotion_id varchar2(10) primary key ,
offer_id varchar2(10),
foreign key (offer_id) references test_offers (offer_id)
);
insert into test_rates (rate_id,rate) values (1,199);
insert into test_offers (offer_id,rate_id) values (11,1);
insert into test_promotions (promotion_id,offer_id) values (21,11);
commit;
delete from test_promotions where promotion_id = 21;
delete from test_offers where offer_id in (select offer_id from test_promotions where promotion_id = 1); -- key is gone by now
In the common case, when there are N promotions (N>1) for a single offer, it wouldn't make sense to delete the offer if one promotion only is deleted. You would end up with orphaned promotions.
If you want to delete a rate, it would make sense to start with deleting all children promotions then all children offers then delete the rate. In that case though, the rate_id could be used along the way.
If you delete a child record, there's no need to delete the parent record unless of course this is a requirement, in which case start with looking for the parent id and see above.
I am using Postgres 8.4. My system configuration is window 7 32 bit 4 gb ram and 2.5ghz.
I have a database in Postgres with 10 tables t1, t2, t3, t4, t5.....t10.
t1 has a primary key a sequence id which is a foreign key reference to all other tables.
The data is inserted in database (i.e. in all tables) apart from t1 all other tables have nearly 50,000 rows of data but t1 has one 1 row whose primary key is referenced from all other tables. Then I insert the 2nd row of data in t1 and again 50,000 rows with this new reference in other tables.
The issue is when I want to delete all the data entries that are present in other tables:
delete from t1 where column1='1'
This query takes nearly 10 min to execute.
I created indexes also and tried but the performance is not at all improving.
what can be done?
I have mentioned a sample schema below
CREATE TABLE t1
(
c1 numeric(9,0) NOT NULL,
c2 character varying(256) NOT NULL,
c3ver numeric(4,0) NOT NULL,
dmlastupdatedate timestamp with time zone NOT NULL,
CONSTRAINT t1_pkey PRIMARY KEY (c1),
CONSTRAINT t1_c1_c2_key UNIQUE (c2)
);
CREATE TABLE t2
(
c1 character varying(100),
c2 character varying(100),
c3 numeric(9,0) NOT NULL,
c4 numeric(9,0) NOT NULL,
tver numeric(4,0) NOT NULL,
dmlastupdatedate timestamp with time zone NOT NULL,
CONSTRAINT t2_pkey PRIMARY KEY (c3),
CONSTRAINT t2_fk FOREIGN KEY (c4)
REFERENCES t1 (c1) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE CASCADE,
CONSTRAINT t2_c3_c4_key UNIQUE (c3, c4)
);
CREATE INDEX t2_index ON t2 USING btree (c4);
Let me know if there is anything wrong with the schema.
With bigger tables and more than just two or three values, you need an index on the referenced column (t1.c1) as well as the referencing columns (t2.c4, ...).
But if your description is accurate, that can not be the cause of the performance problem in your scenario. Since you have only 2 distinct values in t1, there is just no use for an index. A sequential scan will be faster.
Anyway, I re-enacted what you describe in Postgres 9.1.9
CREATE TABLE t1
( c1 numeric(9,0) PRIMARY KEY,
c2 character varying(256) NOT NULL,
c3ver numeric(4,0) NOT NULL,
dmlastupdatedate timestamptz NOT NULL,
CONSTRAINT t1_uni_key UNIQUE (c2)
);
CREATE temp TABLE t2
( c1 character varying(100),
c2 character varying(100),
c3 numeric(9,0) PRIMARY KEY,
c4 numeric(9,0) NOT NULL,
tver numeric(4,0) NOT NULL,
dmlastupdatedate timestamptz NOT NULL,
CONSTRAINT t2_uni_key UNIQUE (c3, c4),
CONSTRAINT t2_c4_fk FOREIGN KEY (c4)
REFERENCES t1(c1) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE CASCADE
);
INSERT INTO t1 VALUES
(1,'OZGPIGp7tgp97tßp97tß97tgP?)/GP)7gf', 234, now())
,(2,'agdsOZGPIGp7tgp97tßp97tß97tgP?)/GP)7gf', 4564, now());
INSERT INTO t2
SELECT'shOahaZGPIGp7tgp97tßp97tß97tgP?)/GP)7gf'
,'shOahaZGPIGp7tgp97tßp97tß97tgP?)/GP)7gf'
, g, 2, 456, now()
from generate_series (1,50000) g
INSERT INTO t2
SELECT'shOahaZGPIGp7tgp97tßp97tß97tgP?)/GP)7gf'
,'shOahaZGPIGp7tgp97tßp97tß97tgP?)/GP)7gf'
, g, 2, 789, now()
from generate_series (50001, 100000) g
ANALYZE t1;
ANALYZE t2;
EXPLAIN ANALYZE DELETE FROM t1 WHERE c1 = 1;
Total runtime: 53.745 ms
DELETE FROM t1 WHERE c1 = 1;
58 ms execution time.
Ergo, there is nothing fundamentally wrong with your schema layout.
Minor enhancements:
You have a couple of columns defined numeric(9,0) or numeric(4,0). Unless you have a good reason to do that, you are probably a lot better off using just integer. They are smaller and faster overall. You can always add a check constraint if you really need to enforce a maximum.
I also would use text instead of varchar(n)
And reorder columns (at table creation time). As a rule of thumb, place fixed length NOT NULL columns first. Put timestamp and integer first and numeric or text last. More here..