Merge with subquery in Vertica - vertica

Question

I was trying to insert only distinct records from source table to destination table using Merge command in Vertica. I had tried the same thing in SQL using subquery but in vertica its not allowing.
following is the error:
[Vertica][VJDBC](5665) ERROR: Subquery in MERGE is not supported
please help me with this.
Thanks in advance

Answer 1

It was not possible to use Subquery in vertica merge.
MERGE INTO Target TGT
USING (Select distinct * from Sources) as SRC
ON TGT.CD = SRC.CD
I created a view for :
Select distinct * from Sources
used it like this:
MERGE INTO Target TGT
USING SRC_VW as SRC
ON TGT.CD = SRC.CD
it solved the purpose. Just thought of posting it for future use only.

Answer 2

Do you need to perform a MERGE operation with big data volumes?
Then, you should consider enabling an optimized MERGE plan instead of a standard MERGE, which is considerably faster.
Should you be interested, shout, and I'll explain you the details as yet another answer here.
Happy playing
Marco the Sane

Answer 3

Optimized MERGE possibility.
I'm basing it on a mutilated Slowly Changing Dimension table. It brings the idea across, but don't take it as a design blueprint for anything in reality ....!
Let’s say you have, among other rows, these in your target table. cust_key is the surrogate primary key, and the combination of cust_id and cust_to_dt is the other unique identifier of the table:
cust_key|cust_id|cust_to_dt|cust_first_name |cust_last_name |cust_phoneno |cust_loyalty_lvl|cust_org_id
1| 7|9999-12-31|Lionel |Prosser |[phone]| 1| 0
2| 14|9999-12-31|Hotblack |Desiato |[phone]| 2| 0
3| 21|9999-12-31|Slartibartfast |Slartibartfast |[phone]| 3| 0
4| 28|9999-12-31|Pizpot |Gargravarr |[phone]| 4| 0
5| 35|9999-12-31|Know-Nothing-Bozo|the Non-Wonder Dog|[phone]| 5| 0
6| 42|9999-12-31|Fook |Fook |[phone]| 6| 0
The definition of the target table would be this:
-- create a stand-alone sequence
CREATE SEQUENCE source_seq;
-- the target's surrogate key defaults to its sequence's NEXTVAL ….
CREATE TABLE target (
cust_key BIGINT DEFAULT target_seq.NEXTVAL NOT NULL
, cust_id BIGINT
, cust_to_dt DATE
, cust_first_name NCHAR VARYING(30)
, cust_last_name NCHAR VARYING(30)
, cust_phoneno NCHAR VARYING(30)
, cust_loyalty_lvl BIGINT
, cust_org_id BIGINT
, PRIMARY KEY(cust_key)
)
UNSEGMENTED ALL NODES;
Your source table – without the surrogate key value for starters – looks like this. 6 rows are updates of existing ones, and 1 new row, Thor with cust_id 43.:
cust_id|cust_to_dt|cust_first_name |cust_last_name |cust_phoneno |cust_loyalty_lvl|cust_org_id
7|9999-12-31|Lionel |Prosser |[phone]| 2| 0
14|9999-12-31|Hotblack |Desiato |[phone]| 3| 0
21|9999-12-31|Slartibartfast |Slartibartfast |[phone]| 4| 0
28|9999-12-31|Pizpot |Gargravarr |[phone]| 5| 0
35|9999-12-31|Know-Nothing-Bozo|the Non-Wonder Dog|[phone]| 6| 0
42|9999-12-31|Fook |Fook |[phone]| 7| 0
43|9999-12-31|Thor |Thor |[phone]| 6| 1
Your MERGE source table is built like the target table.:
CREATE TABLE merge_src LIKE target INCLUDING PROJECTIONS;
You fill it from the delta table so that you pick the existing surrogate keys from the target, and where they don’t exist, you use target_seq.NEXTVAL to get a new surrogate key. In order to do that, you left join your source to your target using the the combination of cust_id and cust_to_dt, the other possibility to uniquely identify the target rows:
INSERT /*+ DIRECT */ INTO merge_src
SELECT
IFNULL(tgt.cust_key,target_seq.NEXTVAL) AS cust_key
, src.cust_id
, src.cust_to_dt
, src.cust_first_name
, src.cust_last_name
, src.cust_phoneno
, src.cust_loyalty_lvl
, src.cust_org_id
FROM source src
LEFT
JOIN target tgt
USING(cust_id,cust_to_dt)
;
Your merge source table now looks like this – the new row with cust_id 43, Thor, gets a new surrogate key:
SQL>select * from merge_src;
cust_key|cust_id|cust_to_dt|cust_first_name |cust_last_name |cust_phoneno |cust_loyalty_lvl|cust_org_id
1| 7|9999-12-31|Lionel |Prosser |[phone]| 2| 0
2| 14|9999-12-31|Hotblack |Desiato |[phone]| 3| 0
3| 21|9999-12-31|Slartibartfast |Slartibartfast |[phone]| 4| 0
4| 28|9999-12-31|Pizpot |Gargravarr |[phone]| 5| 0
5| 35|9999-12-31|Know-Nothing-Bozo|the Non-Wonder Dog|[phone]| 6| 0
6| 42|9999-12-31|Fook |Fook |[phone]| 7| 0
250,007| 43|9999-12-31|Thor |Thor |[phone]| 6| 1
The MERGE between merge_src and target uses an optimized MERGE, using the surrogate key in the ON clause (I just replaced the code for the digraph objects for the GraphViz format by [...] placeholders). Look especially at the Access Path for the DELETE – it has a MERGEJOIN (inputs presorted) and uses Side Information Passing (SIP1):
SQL>EXPLAIN
...>MERGE /*+DIRECT*/
...>INTO target t
...>USING merge_src s
...> ON s.cust_key = t.cust_key
...>WHEN MATCHED THEN UPDATE SET
...> cust_key = s.cust_key
...>, cust_id = s.cust_id
...>, cust_to_dt = s.cust_to_dt
...>, cust_first_name = s.cust_first_name
...>, cust_last_name = s.cust_last_name
...>, cust_phoneno = s.cust_phoneno
...>, cust_loyalty_lvl = s.cust_loyalty_lvl
...>, cust_org_id = s.cust_org_id
...>WHEN NOT MATCHED THEN INSERT (
...> cust_key
...>, cust_id
...>, cust_to_dt
...>, cust_first_name
...>, cust_last_name
...>, cust_phoneno
...>, cust_loyalty_lvl
...>, cust_org_id
...>) VALUES (
...> s.cust_key
...>, s.cust_id
...>, s.cust_to_dt
...>, s.cust_first_name
...>, s.cust_last_name
...>, s.cust_phoneno
...>, s.cust_loyalty_lvl
...>, s.cust_org_id
...>);
QUERY PLAN
------------------------------
QUERY PLAN DESCRIPTION:
------------------------------
EXPLAIN MERGE /*+DIRECT*/ INTO target t USING merge_src s ON s.cust_key = t.cust_key WHEN MATCHED THEN UPDATE SET cust_key = s.cust_key , cust_id = s.cust_id , cust_to_dt = s.cust_to_dt , cust_first_name = s.cust_first_name , cust_last_name = s.cust_last_name , cust_phoneno = s.cust_phoneno , cust_loyalty_lvl = s.cust_loyalty_lvl , cust_org_id = s.cust_org_id WHEN NOT MATCHED THEN INSERT ( cust_key , cust_id , cust_to_dt , cust_first_name , cust_last_name , cust_phoneno , cust_loyalty_lvl , cust_org_id ) VALUES ( s.cust_key , s.cust_id , s.cust_to_dt , s.cust_first_name , s.cust_last_name , s.cust_phoneno , s.cust_loyalty_lvl , s.cust_org_id )
Access Path:
+-DML INSERT [Cost: 0, Rows: 0]
| Target Projection: scd.target_super
| Target Prep:
| +---> STORAGE ACCESS for s [Cost: 40, Rows: 7 (NO STATISTICS)] (PATH ID: 3)
| | Projection: scd.merge_src_super
| | Materialize: s.cust_key, s.cust_id, s.cust_to_dt, s.cust_first_name, s.cust_last_name, s.cust_phoneno, s.cust_loyalty_lvl, s.cust_org_id
[...]
Lock Map - Table (Mode)
----------------------------------------------
target(I)
----------------------------------------------
------------------------------
QUERY PLAN DESCRIPTION:
------------------------------
EXPLAIN MERGE /*+DIRECT*/ INTO target t USING merge_src s ON s.cust_key = t.cust_key WHEN MATCHED THEN UPDATE SET cust_key = s.cust_key , cust_id = s.cust_id , cust_to_dt = s.cust_to_dt , cust_first_name = s.cust_first_name , cust_last_name = s.cust_last_name , cust_phoneno = s.cust_phoneno , cust_loyalty_lvl = s.cust_loyalty_lvl , cust_org_id = s.cust_org_id WHEN NOT MATCHED THEN INSERT ( cust_key , cust_id , cust_to_dt , cust_first_name , cust_last_name , cust_phoneno , cust_loyalty_lvl , cust_org_id ) VALUES ( s.cust_key , s.cust_id , s.cust_to_dt , s.cust_first_name , s.cust_last_name , s.cust_phoneno , s.cust_loyalty_lvl , s.cust_org_id )
Access Path:
+-DML DELETE [Cost: 0, Rows: 0]
| Target Projection: scd.target_super (DELETE ON CONTAINER)
| Target Prep:
| +---> JOIN MERGEJOIN(inputs presorted) [Semi] [Cost: 11, Rows: 7 (NO STATISTICS)] (PATH ID: 1)
| | Join Cond: (VAL(2) = t.cust_key)
| | +-- Outer -> STORAGE ACCESS for t [Cost: 5, Rows: 6 (NO STATISTICS)] (PATH ID: 2)
| | | Projection: scd.target_super
| | | Materialize: t.cust_key
| | | Runtime Filter: (SIP1(MergeJoin): t.cust_key)
| | +-- Inner -> SELECT [Cost: 5, Rows: 7 (NO STATISTICS)] (PATH ID: 3)
| | | +---> STORAGE ACCESS for s [Cost: 5, Rows: 7 (NO STATISTICS)] (PATH ID: 4)
| | | | Projection: scd.merge_src_super
| | | | Materialize: s.cust_key
Lock Map - Table (Mode)
----------------------------------------------
target(X)
----------------------------------------------
Hope all this makes some sense…
Happy playing -
Marco the Sane