I'm new to Postgres and even newer to understanding how explain works. I have a query below which is typical, I just replace the date:
explain
select account_id,
security_id,
market_value_date,
sum(market_value) market_value
from market_value_history mvh
inner join holding_cust hc on hc.id = mvh.owning_object_id
where
hc.account_id = 24766
and market_value_date = '2015-07-02'
and mvh.created_by = 'HoldingLoad'
group by account_id, security_id, market_value_date
order by security_id, market_value_date;
Attached is a screenshot of explain
The count for holding_cust table is 2 million rows and market_value_history table has 163 million rows
Below are the table definitions and indexes for market_value_history and holding_cust:
I'd appreciate any advice you may be able to give me on tuning this query.
CREATE TABLE public.market_value_history
(
id integer NOT NULL DEFAULT nextval('market_value_id_seq'::regclass),
market_value numeric(18,6) NOT NULL,
market_value_date date,
holding_type character varying(25) NOT NULL,
owning_object_type character varying(25) NOT NULL,
owning_object_id integer NOT NULL,
created_by character varying(50) NOT NULL,
created_dt timestamp without time zone NOT NULL,
last_modified_dt timestamp without time zone NOT NULL,
CONSTRAINT market_value_history_pkey PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
);
ALTER TABLE public.market_value_history
OWNER TO postgres;
-- Index: public.ix_market_value_history_id
-- DROP INDEX public.ix_market_value_history_id;
CREATE INDEX ix_market_value_history_id
ON public.market_value_history
USING btree
(owning_object_type COLLATE pg_catalog."default", owning_object_id);
-- Index: public.ix_market_value_history_object_type_date
-- DROP INDEX public.ix_market_value_history_object_type_date;
CREATE UNIQUE INDEX ix_market_value_history_object_type_date
ON public.market_value_history
USING btree
(owning_object_type COLLATE pg_catalog."default", owning_object_id, holding_type COLLATE pg_catalog."default", market_value_date);
CREATE TABLE public.holding_cust
(
id integer NOT NULL DEFAULT nextval('holding_cust_id_seq'::regclass),
account_id integer NOT NULL,
security_id integer NOT NULL,
subaccount_type integer,
trade_date date,
purchase_date date,
quantity numeric(18,6),
net_cost numeric(18,2),
adjusted_net_cost numeric(18,2),
open_date date,
close_date date,
created_by character varying(50) NOT NULL,
created_dt timestamp without time zone NOT NULL,
last_modified_dt timestamp without time zone NOT NULL,
CONSTRAINT holding_cust_pkey PRIMARY KEY (id)
)
WITH (
OIDS=FALSE
);
ALTER TABLE public.holding_cust
OWNER TO postgres;
-- Index: public.ix_holding_cust_account_id
-- DROP INDEX public.ix_holding_cust_account_id;
CREATE INDEX ix_holding_cust_account_id
ON public.holding_cust
USING btree
(account_id);
-- Index: public.ix_holding_cust_acctid_secid_asofdt
-- DROP INDEX public.ix_holding_cust_acctid_secid_asofdt;
CREATE INDEX ix_holding_cust_acctid_secid_asofdt
ON public.holding_cust
USING btree
(account_id, security_id, trade_date DESC);
-- Index: public.ix_holding_cust_security_id
-- DROP INDEX public.ix_holding_cust_security_id;
CREATE INDEX ix_holding_cust_security_id
ON public.holding_cust
USING btree
(security_id);
-- Index: public.ix_holding_cust_trade_date
-- DROP INDEX public.ix_holding_cust_trade_date;
CREATE INDEX ix_holding_cust_trade_date
ON public.holding_cust
USING btree
(trade_date);
Two things:
As Dmitry pointed out, you should look at creating an Index on market_value_date field. Its possible that post that you have a completely different query plan, which may or may not bring up other bottlenecks, but it should certainly remove this seq-Scan.
Minor (since I doubt if it affects performance), but secondly, if you aren't enforcing field length by design, you may want to change createdby field to TEXT. As can be seen in the query, its trying to cast all createdby fields to TEXT for this query.
Related
For Example ,
I have created a table ,
CREATE DATABASE es_db;
USE es_db;
DROP TABLE IF EXISTS es_table;
CREATE TABLE es_table (
id BIGINT(20) UNSIGNED NOT NULL,
PRIMARY KEY (id),
UNIQUE KEY unique_id (id),
client_name VARCHAR(32) NOT NULL,
modification_time TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
insertion_time TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);
Now assume i have to select those data which are greater than time i give as input .
consider this query for example,
SELECT *, UNIX_TIMESTAMP(modification_time) AS unix_ts_in_secs FROM es_table WHERE (UNIX_TIMESTAMP(modification_time) > :sql_last_modifiedvalue AND modification_time < NOW()) ORDER BY modification_time ASC
Is there away to translate the same to native query ? i can achieve the same with jdbctemplate but would like to know if this is possible with native query?
I am facing an issue with my datastage job. I have to fill a table ttperiodeas in Oracle from a .csv file. The SQL query in Oracle connector is shown in this screenshot:
Oracle connector
And here is the oracle script
CREATE TABLE TTPERIODEAS
(
CDPARTITION VARCHAR2(5 BYTE) NOT NULL ENABLE,
CDCOMPAGNIE NUMBER(4,0) NOT NULL ENABLE,
CDAPPLI NUMBER(4,0) NOT NULL ENABLE,
NUCONTRA CHAR(15 BYTE) NOT NULL ENABLE,
DTDEBAS NUMBER(8,0) NOT NULL ENABLE,
DTFINAS NUMBER(8,0) NOT NULL ENABLE,
TAUXAS NUMBER(8,5) NOT NULL ENABLE,
CONSTRAINT PK_TTPERIODEAS
PRIMARY KEY (CDPARTITION, CDCOMPAGNIE, CDAPPLI, NUCONTRA, DTDEBAS)
)
PARTITION BY LIST(CDPARTITION)
(PARTITION P_PERIODEAS_13Q VALUES ('13Q'));
When running the job, I get the following message error and the table is not filled.:
The index 'USINODSD0.SYS_C00249007' its partition is unusable
Please I need help thanks
The index is global (i.e. not partitioned) because there is no using index local at the end of the definition. This is also true for the PK index shown above. (I'm assuming they are two different things, because by default the DDL above would create an index named PK_TTPERIODEAS, so I'm not sure what SYS_C00249007 is.) If you can drop and rebuild them as local indexes (i.e. partitioned to match the table) then truncating or dropping a partition will no longer invalidate indexes.
For example, you could rebuild the primary key as:
alter table ttperiodeas
drop primary key;
alter table ttperiodeas
add constraint pk_ttperiodeas primary key (cdpartition,cdcompagnie,cdappli,nucontra,dtdebas)
using index local;
I don't know how SYS_C00249007 is defined, but you could use something similar.
The create table command might be something like:
create table ttperiodeas
( cdpartition varchar2(5 byte) not null
, cdcompagnie number(4,0) not null
, cdappli number(4,0) not null
, nucontra varchar2(15 byte) not null
, dtdebas number(8,0) not null
, dtfinas number(8,0) not null
, tauxas number(8,5) not null
, constraint pk_ttperiodeas
primary key (cdpartition,cdcompagnie,cdappli,nucontra,dtdebas)
using index local
)
partition by list(cdpartition)
( partition p_periodeas_13q values ('13Q') );
Alternatively, you could add the update global indexes clause when dropping the partition:
alter table demo_temp drop partition p_periodeas_14q update global indexes;
(By the way, NUCONTRA should probably be a standard VARCHAR2 and not CHAR, which is intended for cross-platform compatibility and ANSI completeness, and in practice just wastes space and creates bugs.)
the message says that the index for the given partition is unusable: so you could try to rebuild the correponding index partition by the use of
create index [index_name] rebuild partition [partition_name]
(with the fitting values for [index_name] and [partition_nme].
Before you do that you should check the status of the index partitions in user_indexes - since your error message looks not like Oracle error messages usually do.
But since the index is global as William Robertson pointed out, this is not applicable for the given situation.
I've got 2 entities:
1) EMPLOYEES (Parent)
CREATE TABLE EMPLOYEES (
employee_id NUMBER (3) NOT NULL,
first_name VARCHAR (20) NOT NULL,
last_name VARCHAR (20) NOT NULL,
job_title VARCHAR (20) NOT NULL,
employee_type VARCHAR (1) NOT NULL,
salary NUMBER (5),
hourly_pay NUMBER (5,2),
bonus_pay NUMBER (5,2),
CONSTRAINT employee_pk PRIMARY KEY(employee_id));
2) EMPLOYEE_HISTORY (Child)
CREATE TABLE EMPLOYEE_HISTORY (
start_date DATE NOT NULL,
employee_id NUMBER (3) NOT NULL,
end_date DATE,
job_title VARCHAR (10) NOT NULL,
hourly_rate NUMBER (5,2) NOT NULL,
CONSTRAINT employee_history_pk PRIMARY KEY(start_date, employee_id));
I'm trying to create:
ALTER TABLE employee_history
ADD CONSTRAINT employee_history_fk
FOREIGN KEY (employee_id)
REFERENCES employee_history(employee_id);
When I do this, I get an error
ORA-02270: no matching unique or primary key for this column-list
My guess is that I cannot create the constraint on just employee_id because I have a composite key in my child table. I understand when an employee gets put into the database, the parent table is filled out and the "start date" should be filled out along with everything else. However, I do not understand how this would work if I had start_date in my parent table as well. I would be able to create my constraint, yes, but how will I be able to keep a record of changes in start_date if my start_date was inputted at the time of when the employee was entered into the database.I thought about using job_title as a primary key instead of start_date because it's present in both tables, but what happens when an employee gets promoted and demoted again? Won't a duplicate value constraint come up when the same employee_id and job_title is getting inserted?
Your references clause needs to reference the parent table. Not the child table
ALTER TABLE employee_history
ADD CONSTRAINT employee_history_fk
FOREIGN KEY (employee_id)
REFERENCES employee(employee_id); -- employee not employee_history
The SQL you posted is trying to create a self-referential foreign key where employee_history is both the parent and the child. That doesn't make sense in this case.
This is my table structure:
CREATE TABLE semantified_content_key_word
(
id bigint NOT NULL,
semantified_content_id bigint,
key_word_text text,
content_date timestamp without time zone NOT NULL,
context_id bigint NOT NULL,
CONSTRAINT pk_sckw_id PRIMARY KEY (id )
)
WITH (
OIDS=FALSE
);
ALTER TABLE semantified_content_key_word
OWNER TO postgres;
-- Index: idx_sckw_kwt
-- DROP INDEX idx_sckw_kwt;
CREATE INDEX idx_sckw_kwt
ON semantified_content_key_word
USING gin
(to_tsvector('english'::regconfig, key_word_text) );
-- Index: idx_sckw_sc_id
-- DROP INDEX idx_sckw_sc_id;
CREATE INDEX idx_sckw_sc_id
ON semantified_content_key_word
USING btree
(semantified_content_id );
Following is the data :
INSERT INTO semantified_content_key_word (id, semantified_content_id, key_word_text, content_date, context_id)
VALUES (7347, 7347, ', agreementnumber customer servicecreditdate the guarantor taken exhausted prior being pursuant avoidance doubt shall remain liable case non incomplete', '2014-11-21 00:00:00', 111);
INSERT INTO semantified_content_key_word (id, semantified_content_id, key_word_text, content_date, context_id)
VALUES (7356, 7356, ', ; agreementnumber agreementperiod aircraftmodel commencementdate customer enginemodel enginequantity enginetype foddeductibleamount llpminimumbuild servicecreditdate steppedpopularrate takeoffderate termdate turnaroundtime ion ls initiated manager otherwise) inform whether proposed qualified view lnltlated confirm satisfies criteria out article instruct programme accordingly determined meet pursuant paragraph a) treated subject only g) below b)', '2014-11-21 00:00:00', 111);
INSERT INTO semantified_content_key_word (id, semantified_content_id, key_word_text, content_date, context_id)
VALUES (7441, 7441, ', activationdate agreementnumber enginemodel enginetype llpminimumbuild servicecreditdate steppedpopularrate turnaroundtime leap-1a as united continental customer 1/ neutral qec configuration engines shop maintenance: each engine ', '2014-11-17 00:00:00', 111);
-------------------------------------------------------------
select sckw.*
FROM semantified_content_key_word sckw
where TO_TSVECTOR(sckw.key_word_text) ## TO_TSQUERY('exhausted');
This is the query which i am running. And the keyword "exhausted" is present only in one of the rows but i am getting all the 3 rows.
How to avoid the rows where the keyword is not present
Thanks
Prasanna
I've executed following query in toad:
CREATE TABLE ACTWEB.usuarios
(
id INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
matricula INTEGER,
nome CHAR(50) NOT NULL,
senha CHAR(50),
nivel INTEGER,
maleta INTEGER,
email CHAR(50),
acessos INTEGER,
datacriacao DATE,
dataalteracao DATE
UNIQUE (id)
)
LOGGING
NOCOMPRESS
NOCACHE
NOPARALLEL;
And got this ORA message:
ORA-00907 missing right parenthesis
Cause: A left parenthesis has been entered without a closing right parenthesis, or extra information was contained in the parentheses. All parentheses must be entered in pairs.
Action: Correct the syntax and retry the statement.
There are a couple of errors in your syntax. The NOT NULL has to occur after the IDENTITY clause. The range has to be specified without a , and the UNIQUE keyword has to appear at the column directly:
CREATE TABLE ACTWEB.usuarios
(
id INTEGER GENERATED ALWAYS AS IDENTITY (START WITH 1 INCREMENT BY 1) NOT NULL UNIQUE,
matricula INTEGER,
nome CHAR(50) NOT NULL,
senha CHAR(50),
nivel INTEGER,
maleta INTEGER,
email CHAR(50),
acessos INTEGER,
datacriacao DATE,
dataalteracao DATE
)
LOGGING NOCOMPRESS NOCACHE NOPARALLEL;
You specified unique not right, to solve it you have several ways:
First, create unique key explicitly:
create table tab1(
id number(10),
CONSTRAINT id_uk UNIQUE (id)
)
Second, create unique key as an option of the column:
create table tab1(
id number(10) UNIQUE
)
Third, add the unique key constraint using alter table:
create table tab1(
id number(10)
);
alter table tab1 add constraint id_uk unique(id);
Forth, create unique key implicitly by creating unique index:
create table tab1(
id number(10)
);
CREATE UNIQUE INDEX id_uk ON tab1 (id);