how can I select not null values from table ...
lets image table as follows:
master_id | date_update | name | dpt | channel | source | active
---------------------------------------------------------------------------
1 | 1/2/2015 15:43:21 | NULL | NULL | NULL | NULL | y
1 | 1/2/2015 15:43:21 | NULL | FIN | NULL | NULL | n
1 | 1/2/2015 15:40:16 | Elvis | NULL | NULL | NULL | n
1 | 1/2/2015 15:26:38 | NULL | NULL | CH1 | NULL | n
1 | 1/2/2015 14:57:02 | NULL | NULL | NULL | S1 | n
5 | 2/2/2015 15:28:02 | NULL | NULL | CH2 | NULL | y
5 | 1/2/2015 10:13:01 | Sarah | NULL | NULL | NULL | n
The result I would like to get is:
master_id | date_update | name | dpt | channel | source | active
---------------------------------------------------------------------------
1 | 1/2/2015 15:43:21 | Elvis | FIN | CH1 | S1 | y
5 | 2/2/2015 15:28:02 | Sarah | NULL | CH2 | NULL | y
You can notice, that DATE_UPDATE and ACTIVE column is the latest one ...
Which method is the most optimized one?
I tried the combination of listagg (to merge rows into one) and then get the last informations via row_number() over() or max() over() but I am not sure if it is the best performance solution ... joins are also not the best solution (because there are 17 columns I need to merge) ..
The Oracle MAX() aggregate function should ignore NULL values, meaning it should pick up on the non NULL value in each master_id group as being the "max" for that column/group.
SELECT master_id, MAX(date_update), MAX(name), MAX(dpt), MAX(channel), MAX(source),
MAX(active)
FROM image
GROUP BY master_id
Related
I'm newbie with proxysql, this my enviroment:
Centos 7 , proxysql-1.4.13 . I define 2 host group ids : host group id 2 for mysql server that can write , host group id 3 for mysql server that can read.
All queries that begin with insert, update, delete, alter should be routed to host group id 2.
All queries that begin with select should be routed to host group id 3.
So here are my rules :
Admin> select rule_id,active,digest,match_digest,destination_hostgroup,flagIN,flagOUT,next_query_flagIN,sticky_conn,apply from mysql_query_rules;
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| rule_id | active | digest | match_digest | destination_hostgroup | flagIN | flagOUT | next_query_flagIN | sticky_conn | apply |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| 101 | 1 | NULL | ^insert | 2 | 0 | NULL | NULL | NULL | 1 |
| 102 | 1 | NULL | ^update | 2 | 0 | NULL | NULL | NULL | 1 |
| 103 | 1 | NULL | ^delete | 2 | 0 | NULL | NULL | NULL | 1 |
| 104 | 1 | NULL | ^alter | 2 | 0 | NULL | NULL | NULL | 1 |
| 105 | 1 | NULL | ^select | 3 | 0 | NULL | NULL | NULL | 1 |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
And they works fine. By the way, are these rules ok ? Should I replace match_digest by match_pattern ?
However , my application has 1 feature that insert data into a table (create booking) then select (almost immediately) new data from that table .
So if select query fails (because new data has not replicated to host group id 3 yet), application feature will run wrong.
I want to route the select query right after insert query to host group id 2 so it will get new data successfully.
I read proxysql document https://github.com/sysown/proxysql/wiki/Main-(runtime)#mysql_query_rules and this discussion https://github.com/sysown/proxysql/pull/825 , I think this is solution for me ,isn't it ? I still don't understand about these flagIN,flagOUT,next_query_flagIN,sticky_conn stuff clearly but I will give it a try.
I know the insert query digest is 0xCDD6DB677604AFA7
The select query digest is 0x0DCD2E8ADF6A66CB
Then I add 2 new rules:
Admin> select rule_id,active,digest,match_digest,destination_hostgroup,flagIN,flagOUT,next_query_flagIN,sticky_conn,apply from mysql_query_rules;
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| rule_id | active | digest | match_digest | destination_hostgroup | flagIN | flagOUT | next_query_flagIN | sticky_conn | apply |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
| 1 | 1 | 0xCDD6DB677604AFA7 | NULL | 2 | 0 | NULL | 1 | 1 | 1 |
| 2 | 1 | 0x0DCD2E8ADF6A66CB | NULL | 2 | 1 | NULL | NULL | NULL | 1 |
| 101 | 1 | NULL | ^insert | 2 | 0 | NULL | NULL | NULL | 1 |
| 102 | 1 | NULL | ^update | 2 | 0 | NULL | NULL | NULL | 1 |
| 103 | 1 | NULL | ^delete | 2 | 0 | NULL | NULL | NULL | 1 |
| 104 | 1 | NULL | ^alter | 2 | 0 | NULL | NULL | NULL | 1 |
| 105 | 1 | NULL | ^select | 3 | 0 | NULL | NULL | NULL | 1 |
+---------+--------+--------------------+--------------+-----------------------+--------+---------+-------------------+-------------+-------+
They work fine, the select query right after insert query is route to host group id 2 so it gets new data successfully and application feature runs ok.
But am I doing right ?
I'm confused because stats_mysql_query_rules result:
Before application feature run:
Admin> select * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 1 | 20 |
| 2 | 20 |
| 101 | 33 |
| 102 | 0 |
| 103 | 2 |
| 104 | 0 |
| 105 | 903 |
+---------+------+
After application feature run (10 bookings):
Admin> select * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 1 | 30 |
| 2 | 30 |
| 101 | 43 |
| 102 | 0 |
| 103 | 2 |
| 104 | 0 |
| 105 | 1313 |
+---------+------+
Why rule_id 101 hits rate increase from 33 --> 43 ? So rule_id 101 (match_digest ^insert) also match insert query in application feature ? Does it mean I'm doing wrong ?
I have a table with data at hour level. I want to find the count of hours and the values for col1 and col2 for all hours in an array. Input Table
+-----+-----+-----+
| hour| col1| col2|
+-----+-----+-----+
| 00 | 0.0 | a |
| 04 | 0.1 | b |
| 08 | 0.2 | c |
| 12 | 0.0 | d |
+-----+-----+-----+
I am using the below query to get the column values in an array
Query:
select count(hr), map_values(str_to_map(concat_ws(',',collect_set(concat_ws(':',reflect('java.util.UUID','randomUUID'),cast(col1 as string)))))) as col1_arr, map_values(str_to_map(concat_ws(',',collect_set(concat_ws(':',reflect('java.util.UUID','randomUUID'),cast(col2 as string)))))) as col2_arr from table;
Output that i am getting, values in col2_arr are not in the same sequence with col1_arr. Please suggest how can i get the values in array/list for different columns in same sequence.
+----------+-----------------+----------+
| count(hr)| col1_arr | col2_arr |
+----------+-----------------+----------+
| 4 | 0.0,0.1,0.2,0.0 | b,a,c,d |
+----------+----------------+-----------+
Required output:
+----------+-----------------+----------+
| count(hr)| col1_arr | col2_arr |
+----------+-----------------+----------+
| 4 | 0.0,0.1,0.2,0.0 | a,b,c,d |
+----------+----------------+-----------+
Thanks
select count(*) as cnt
,concat_ws(',',sort_array(collect_list(hour))) as hour
,regexp_replace(concat_ws(',',sort_array(collect_list(concat_ws(':',hour,cast(col1 as string))))),'..:','') as col1
,regexp_replace(concat_ws(',',sort_array(collect_list(concat_ws(':',hour,col2)))),'..:','') as col2
from mytable
;
+-----+-------------+-------------+---------+
| cnt | hour | col1 | col2 |
+-----+-------------+-------------+---------+
| 4 | 00,04,08,12 | 0,0.1,0.2,0 | a,b,c,d |
+-----+-------------+-------------+---------+
I have a select statement that generate set value thereafter I want insert that set of values into another table, MY concern is I'm using select statement in select I'm using one one more select clause((select max(org_id)+1 from org)) where I'm trying to get max value and increment by one but I'm not able get incremented value instead I'm getting same value you can see column name id_limit
select abc,abc1,abc3,abc4,(select max(org_id)+1 from org) as id_limit from xyz
current output
-----------------------------------------------------------------
| abc | abc1 | abc3 | abc4 | id_limit |
----------------------------------------------------------------|
| BUSINESS_UNIT | 0 | 100 | London | 6 |
| BUSINESS_UNIT | 0 | 200 | Sydney | 6 |
| BUSINESS_UNIT | 0 | 300 | Kiev | 6 |
-----------------------------------------------------------------
I'm trying to get expected out output
-----------------------------------------------------------------
| abc | abc1 | abc3 | abc4 | id_limit |
----------------------------------------------------------------|
| BUSINESS_UNIT | 0 | 100 | London | 6 |
| BUSINESS_UNIT | 0 | 200 | Sydney | 7 |
| BUSINESS_UNIT | 0 | 300 | Kiev | 8 |
-----------------------------------------------------------------
Yes, in Oracle 12.
create table foo (
id number generated by default on null as identity
);
https://oracle-base.com/articles/12c/identity-columns-in-oracle-12cr1
In previous versions you use sequence/trigger as explained here:
How to create id with AUTO_INCREMENT on Oracle?
I am little confused over the execution plan of an Oracle query. This is in Oracle Enterprise Edition 11.2.0.1.0 on platform IBM AIX 6.1. I have a table TEST1 (1 million rows) and another table TEST2 (50,000 rows). Both tables have identical columns. There is a view created as a union of these 2 tables. I am firing a query on this view, with an indexed column in the WHERE clause. What I could find is that the index is not used and a full table scan is resulted. With a slight modification of the query, it started using the index. I am wondering how this particular change can result in the plan change.
Please find the complete DDL + DML below. I have given simplified example. Actual schema and requirements are bit more complex. In fact the query in question is dynamically constructed and executed by an OCI code generator. My intention here is not to get alternatives, but to really understand what could be the logical reasoning behind the plan change (between, I am an application programmer and not a database administrator). Your help is much appreciated.
DROP TABLE TEST1 CASCADE CONSTRAINTS ;
DROP TABLE TEST2 CASCADE CONSTRAINTS ;
CREATE TABLE TEST1
(
ID NUMBER(20) NOT NULL,
NAME VARCHAR2(40),
DAY NUMBER(20)
)
PARTITION BY RANGE (DAY)
(
PARTITION P001 VALUES LESS THAN (2),
PARTITION P002 VALUES LESS THAN (3),
PARTITION P003 VALUES LESS THAN (4),
PARTITION P004 VALUES LESS THAN (5),
PARTITION P005 VALUES LESS THAN (6),
PARTITION P006 VALUES LESS THAN (7),
PARTITION P007 VALUES LESS THAN (8),
PARTITION P008 VALUES LESS THAN (9),
PARTITION P009 VALUES LESS THAN (10),
PARTITION P010 VALUES LESS THAN (11),
PARTITION P011 VALUES LESS THAN (12),
PARTITION P012 VALUES LESS THAN (13),
PARTITION P013 VALUES LESS THAN (14),
PARTITION P014 VALUES LESS THAN (15),
PARTITION P015 VALUES LESS THAN (16),
PARTITION P016 VALUES LESS THAN (17),
PARTITION P017 VALUES LESS THAN (18),
PARTITION P018 VALUES LESS THAN (19),
PARTITION P019 VALUES LESS THAN (20),
PARTITION P020 VALUES LESS THAN (21),
PARTITION P021 VALUES LESS THAN (22),
PARTITION P022 VALUES LESS THAN (23),
PARTITION P023 VALUES LESS THAN (24),
PARTITION P024 VALUES LESS THAN (25),
PARTITION P025 VALUES LESS THAN (26),
PARTITION P026 VALUES LESS THAN (27),
PARTITION P027 VALUES LESS THAN (28),
PARTITION P028 VALUES LESS THAN (29),
PARTITION P029 VALUES LESS THAN (30),
PARTITION P030 VALUES LESS THAN (31)
) ;
CREATE INDEX IX_ID on TEST1 (ID) INITRANS 4 STORAGE(FREELISTS 16) LOCAL
(
PARTITION P001,
PARTITION P002,
PARTITION P003,
PARTITION P004,
PARTITION P005,
PARTITION P006,
PARTITION P007,
PARTITION P008,
PARTITION P009,
PARTITION P010,
PARTITION P011,
PARTITION P012,
PARTITION P013,
PARTITION P014,
PARTITION P015,
PARTITION P016,
PARTITION P017,
PARTITION P018,
PARTITION P019,
PARTITION P020,
PARTITION P021,
PARTITION P022,
PARTITION P023,
PARTITION P024,
PARTITION P025,
PARTITION P026,
PARTITION P027,
PARTITION P028,
PARTITION P029,
PARTITION P030
) ;
CREATE TABLE TEST2
(
ID NUMBER(20) PRIMARY KEY NOT NULL,
NAME VARCHAR2(40),
DAY NUMBER(20)
) ;
CREATE OR REPLACE VIEW TEST_V AS
SELECT
ID, NAME, DAY
FROM
TEST1
UNION
SELECT
ID, NAME, DAY
FROM
TEST2 ;
begin
for count in 1..1000000
loop
insert into test1 values(count, 'John', mod(count, 30) + 1) ;
end loop ;
end ;
/
begin
for count in 1000000..1050000
loop
insert into test2 values(count, 'Mary', mod(count, 30) + 1) ;
end loop ;
end ;
/
commit ;
set lines 300 ;
set pages 1000 ;
-- Actual query
explain plan for
SELECT Key FROM
(
WITH recs AS
(
SELECT * FROM TEST_V WHERE ID = 70000
)
(
SELECT 1 AS Key FROM recs WHERE NAME = 'John'
)
UNION
(
SELECT 2 AS Key FROM recs WHERE NAME = 'Mary'
)
) ;
select * from table(dbms_xplan.display()) ;
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | TempSpc | Cost (%CPU) | Time | Pstart | Pstop |
------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1611K | 4721K | | 13559 (1) | 00:02:43 | | |
| 1 | VIEW | | 1611K | 4721K | | 13559 (1) | 00:02:43 | | |
| 2 | TEMP TABLE TRANSFORMATION | | | | | | | | |
| 3 | LOAD AS SELECT | SYS_TEMP_0FD9D6610_34D3B6C | | | | | | | |
|* 4 | VIEW | TEST_V | 805K | 36M | | 10403 (1) | 00:02:05 | | |
| 5 | SORT UNIQUE | | 805K | 36M | 46M | 10403 (8) | 00:02:05 | | |
| 6 | UNION-ALL | | | | | | | | |
| 7 | PARTITION RANGE ALL | | 752K | 34M | | 721 (1) | 00:00:09 | 1 | 30 |
| 8 | TABLE ACCESS FULL | TEST1 | 752K | 34M | | 721 (1) | 00:00:09 | 1 | 30 |
| 9 | TABLE ACCESS FULL | TEST2 | 53262 | 2496K | | 68 (0) | 00:00:01 | | |
| 10 | SORT UNIQUE | | 1611K | 33M | 43M | 13559 (51) | 00:02:43 | | |
| 11 | UNION-ALL | | | | | | | | |
|* 12 | VIEW | | 805K | 16M | | 1429 (1) | 00:00:18 | | |
| 13 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6610_34D3B6C | 805K | 36M | | 1429 (1) | 00:00:18 | | |
|* 14 | VIEW | | 805K | 16M | | 1429 (1) | 00:00:18 | | |
| 15 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6610_34D3B6C | 805K | 36M | | 1429 (1) | 00:00:18 | | |
------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - filter("ID"=70000)
12 - filter("NAME"='John')
14 - filter("NAME"='Mary')
-- Modified query (only change is absence of outermost SELECT)
explain plan for
WITH recs AS
(
SELECT * FROM TEST_V WHERE ID = 70000
)
(
SELECT 1 AS Key FROM recs WHERE NAME = 'John'
)
UNION
(
SELECT 2 AS Key FROM recs WHERE NAME = 'Mary'
) ;
select * from table(dbms_xplan.display()) ;
PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU) | Time | Pstart | Pstop |
-----------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 4 | 88 | 6 (67) | 00:00:01 | | |
| 1 | TEMP TABLE TRANSFORMATION | | | | | | | |
| 2 | LOAD AS SELECT | SYS_TEMP_0FD9D6611_34D3B6C | | | | | | |
| 3 | VIEW | TEST_V | 2 | 96 | 4 (50) | 00:00:01 | | |
| 4 | SORT UNIQUE | | 2 | 96 | 4 (75) | 00:00:01 | | |
| 5 | UNION-ALL | | | | | | | |
| 6 | PARTITION RANGE ALL | | 1 | 48 | 1 (0) | 00:00:01 | 1 | 30 |
| 7 | TABLE ACCESS BY LOCAL INDEX ROWID | TEST1 | 1 | 48 | 1 (0) | 00:00:01 | 1 | 30 |
|* 8 | INDEX RANGE SCAN | IX_ID | 1 | | 1 (0) | 00:00:01 | 1 | 30 |
| 9 | TABLE ACCESS BY INDEX ROWID | TEST2 | 1 | 48 | 1 (0) | 00:00:01 | | |
|* 10 | INDEX UNIQUE SCAN | SYS_C001242692 | 1 | | 1 (0) | 00:00:01 | | |
| 11 | SORT UNIQUE | | 4 | 88 | 6 (67) | 00:00:01 | | |
| 12 | UNION-ALL | | | | | | | |
|* 13 | VIEW | | 2 | 44 | 2 (0) | 00:00:01 | | |
| 14 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6611_34D3B6C | 2 | 96 | 2 (0) | 00:00:01 | | |
|* 15 | VIEW | | 2 | 44 | 2 (0) | 00:00:01 | | |
| 16 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6611_34D3B6C | 2 | 96 | 2 (0) | 00:00:01 | | |
-----------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
8 - access("ID"=70000)
10 - access("ID"=70000)
13 - filter("NAME"='John')
15 - filter("NAME"='Mary')
quit ;
thanks & regards,
Reji
I can not reproduce this in 11.2.0.3, I don't think there is a logical explanation for this behavior other than: you hit a bug, that apparently is solved in 11.2.0.3.
One thing that jumped immediately in my eye is the lack of object statistics and - if your output was complete - the fact that OPTIMIZER_DYNAMIC_SAMPLING is set to 0. You could try to reproduce with OPTIMIZER_DYNAMIC_SAMPLING=2. In that case the dynamic sampler kicks in if the object statistics are missing. BTW: don't use this feature instead of correct optimizer statistics. More info about dynamic sampling Dynamic sampling and its impact on the Optimizer
In your - nice documented - question and script/test case you try to make use of append and nologging. This only works for bulk inserts, not for row inserts with values. What would happen is for every insert: push-up the highwater mark and dump a full block of data in the free block, in your case that would have only 1 row .... Luckily, the database ignores this instruction.
Before you fire SQL to a table, make sure that you give it optimizer statistics. This will certainly help your case.
Oracle has 2 functions - rank() and dense_rank() - which i've found very useful for some applications. I am doing something in mysql now and was wondering if they have something equivalent to those?
Nothing directly equivalent, but you can fake it with some (not terribly efficient) self-joins. Some sample code from a collection of MySQL query howtos:
SELECT v1.name, v1.votes, COUNT(v2.votes) AS Rank
FROM votes v1
JOIN votes v2 ON v1.votes < v2.votes OR (v1.votes=v2.votes and v1.name = v2.name)
GROUP BY v1.name, v1.votes
ORDER BY v1.votes DESC, v1.name DESC;
+-------+-------+------+
| name | votes | Rank |
+-------+-------+------+
| Green | 50 | 1 |
| Black | 40 | 2 |
| White | 20 | 3 |
| Brown | 20 | 3 |
| Jones | 15 | 5 |
| Smith | 10 | 6 |
+-------+-------+------+
how about this "dense_rank implement" in MySQL
CREATE TABLE `person` (
`id` int(11) DEFAULT NULL,
`first_name` varchar(20) DEFAULT NULL,
`age` int(11) DEFAULT NULL,
`gender` char(1) DEFAULT NULL);
INSERT INTO `person` VALUES
(1,'Bob',25,'M'),
(2,'Jane',20,'F'),
(3,'Jack',30,'M'),
(4,'Bill',32,'M'),
(5,'Nick',22,'M'),
(6,'Kathy',18,'F'),
(7,'Steve',36,'M'),
(8,'Anne',25,'F'),
(9,'Mike',25,'M');
the data before dense_rank() like this
mysql> select * from person;
+------+------------+------+--------+
| id | first_name | age | gender |
+------+------------+------+--------+
| 1 | Bob | 25 | M |
| 2 | Jane | 20 | F |
| 3 | Jack | 30 | M |
| 4 | Bill | 32 | M |
| 5 | Nick | 22 | M |
| 6 | Kathy | 18 | F |
| 7 | Steve | 36 | M |
| 8 | Anne | 25 | F |
| 9 | Mike | 25 | M |
+------+------------+------+--------+
9 rows in set (0.00 sec)
the data after dense_rank() like this,including "partition by" function
+------------+--------+------+------+
| first_name | gender | age | rank |
+------------+--------+------+------+
| Anne | F | 25 | 1 |
| Jane | F | 20 | 2 |
| Kathy | F | 18 | 3 |
| Steve | M | 36 | 1 |
| Bill | M | 32 | 2 |
| Jack | M | 30 | 3 |
| Mike | M | 25 | 4 |
| Bob | M | 25 | 4 |
| Nick | M | 22 | 6 |
+------------+--------+------+------+
9 rows in set (0.00 sec)
the query statement is
select first_name,t1.gender,age,FIND_IN_SET(age,t1.age_set) as rank from person t2,
(select gender,group_concat(age order by age desc) as age_set from person group by gender) t1
where t1.gender=t2.gender
order by t1.gender,rank