MySQL 5.7 wrong results on sort / subquery - sorting

Question

Very strange results in MySQL 5.7 (specifically 5.7.13-0ubuntu0.16.04.2 ).
I suspect this could be a bug in MySQL.
DROP TABLE IF EXISTS `test_grids_1`;
CREATE TABLE `test_grids_1` (
`unq_id` int(11) NOT NULL DEFAULT '0',
`var_fld` int(11) DEFAULT '0'
) ENGINE=InnoDB;
INSERT INTO `test_grids_1` VALUES
(1,4500),
(2,6000);
DROP TABLE IF EXISTS `test_grid_dtl_1`;
CREATE TABLE `test_grid_dtl_1` (
`dtl_id` int(11) NOT NULL DEFAULT '0',
`unq_id` int(11) DEFAULT '0',
`dtl_var` decimal(14,2) DEFAULT '0.00'
) ENGINE=InnoDB;
INSERT INTO `test_grid_dtl_1` VALUES
(1,1,2.00),
(2,1,2.40),
(3,2,2.30);
SELECT
( g.calc_var * d.dtl_var ) new_var,
g.calc_var
FROM
(
SELECT
unq_id,
IF ( var_fld > 5000, ( 1 / var_fld ) , 5000 ) calc_var
FROM
test_grids_1
) g
INNER JOIN
test_grid_dtl_1 d
ON d.unq_id = g.unq_id;
+--------------+----------+
| new_var | calc_var |
+--------------+----------+
| 10000.000000 | 5000 |
| 12000.000000 | 5000 |
| 0.000383 | 0.0002 |
+--------------+----------+
SELECT
( g.calc_var * d.dtl_var ) new_var,
g.calc_var
FROM
(
SELECT
unq_id,
IF ( var_fld > 5000, ( 1 / var_fld ) , 5000 ) calc_var
FROM
test_grids_1
) g
INNER JOIN
test_grid_dtl_1 d
ON d.unq_id = g.unq_id
ORDER BY
1;
+--------------+----------+
| new_var | calc_var |
+--------------+----------+
| 0.000383 | 0.0002 |
| 10000.000000 | 99.9999 |
| 12000.000000 | 99.9999 |
+--------------+----------+
3 rows in set (0.00 sec)
When the sort is included it causes the returned values for certain criteria to be completely incorrect.
Values that are expected to be 5000 are suddenly 99.9999.
If anyone could please check and confirm similar behaviour on other 5.7 installations, it would be great.
Thanks

Answer 1

What's going on?
The query is being bit by implicit casting by MySQL with joining and ordering.
Solution
Let's solve the problem first and then discuss how we got there. Notice the change of 1/var_fld to 1.0/var_fld and 5000 to 5000.0.
SELECT g.calc_var * d.dtl_var as new_var, g.calc_var
FROM (
SELECT unq_id, IF (var_fld > 5000, 1.0/var_fld, 5000.0) calc_var
FROM test_grids_1
) g
INNER JOIN test_grid_dtl_1 d ON d.unq_id = g.unq_id
ORDER BY 1
+---------------+------------+
| new_var | calc_var |
+---------------+------------+
| 0.0003833 | 0.00017 |
| 10000.0000000 | 5000.00000 |
| 12000.0000000 | 5000.00000 |
+---------------+------------+
You could re-write the query slight differently as well by using cast. Notice that I have included the hexadecimal value in the last column. It will be useful as you read on:
SELECT g.calc_var * d.dtl_var as new_var, g.calc_var, hex(g.calc_var) as hcalc_var
FROM (
SELECT unq_id, IF (var_fld > 5000, cast(1/var_fld as decimal(15,5)), 5000.0) calc_var
FROM test_grids_1
) g
INNER JOIN test_grid_dtl_1 d ON d.unq_id = g.unq_id
ORDER BY 1
+---------------+------------+-----------+
| new_var | calc_var | hcalc_var |
+---------------+------------+-----------+
| 0.0003910 | 0.00017 | 0 |
| 10000.0000000 | 5000.00000 | 1388 |
| 12000.0000000 | 5000.00000 | 1388 |
+---------------+------------+-----------+
Other solution
Notice the replacement of if with case statement.
SELECT g.calc_var * d.dtl_var as new_var, g.calc_var
FROM (
SELECT unq_id, case when var_fld > 5000 then 1/var_fld else 5000 end calc_var
FROM test_grids_1
) g
INNER JOIN test_grid_dtl_1 d ON d.unq_id = g.unq_id
ORDER BY 1
+--------------+-----------+
| new_var | calc_var |
+--------------+-----------+
| 0.000383 | 0.0002 |
| 10000.000000 | 5000.0000 |
| 12000.000000 | 5000.0000 |
+--------------+-----------+
Notice how case statement didn't need any kind of casting to achieve almost the same result. However, to get to the exact same result as the first query, you'd have to do something like this -
Yet another one
Notice the 1.0/var_fld and 5000.0 along with cast instead of if
SELECT g.calc_var * d.dtl_var as new_var, g.calc_var
FROM (
SELECT unq_id, case when var_fld > 5000 then 1.0/var_fld else 5000.0 end calc_var
FROM test_grids_1
) g
INNER JOIN test_grid_dtl_1 d ON d.unq_id = g.unq_id
ORDER BY 1
How did this come to highlight?
Let's look at the original query; I've added a new field hex(g.calc_var) that is a hexadecimal representation of g.calc_var.
SELECT g.calc_var * d.dtl_var as new_var, g.calc_var, hex(g.calc_var) as hcalc_var
FROM (
SELECT unq_id, IF (var_fld > 5000, 1/var_fld, 5000) calc_var
FROM test_grids_1
) g
INNER JOIN test_grid_dtl_1 d ON d.unq_id = g.unq_id
ORDER BY 1
+--------------+----------+-----------+
| new_var | calc_var | hcalc_var |
+--------------+----------+-----------+
| 0.000383 | 0.0002 | 0 |
| 10000.000000 | 99.9999 | 1388 |
| 12000.000000 | 99.9999 | 1388 |
+--------------+----------+-----------+
Compare the results with the very first query in the solution part
SELECT g.calc_var * d.dtl_var as new_var, g.calc_var, hex(g.calc_var) as hcalc_var
FROM (
SELECT unq_id, IF (var_fld > 5000, 1.0/var_fld, 5000.0) calc_var
FROM test_grids_1
) g
INNER JOIN test_grid_dtl_1 d ON d.unq_id = g.unq_id
ORDER BY 1
+---------------+------------+-----------+
| new_var | calc_var | hcalc_var |
+---------------+------------+-----------+
| 0.0003833 | 0.00017 | 0 |
| 10000.0000000 | 5000.00000 | 1388 |
| 12000.0000000 | 5000.00000 | 1388 |
+---------------+------------+-----------+
Notice that the hex value is exactly the same in both the queries but the decimal values are different.
How can 5000 end up as 99.9999?
select cast(5000 as decimal(6,4)) as test;
+---------+
| test |
+---------+
| 99.9999 |
+---------+
1 row in set, 1 warning (0.00 sec)
show warnings;
+---------+------+-----------------------------------------------+
| Level | Code | Message |
+---------+------+-----------------------------------------------+
| Warning | 1264 | Out of range value for column 'test' at row 1 |
+---------+------+-----------------------------------------------+
Like that! When 5000 is cast to a decimal with length 6 including 4 decimals, the result is the maximum that will fit decimal(6,4). Ouch.
In this case a warning is thrown, which is good. One could catch it during testing. However, the query in question does not throw any warning. And that's not good.
This leads to multiple questions
Why is casting happening correctly without order by?
What in the order by is causing casting to happen the way we noticed?
Why is case...end showing better results than if(...) even when no casting is done?
Why is warning not thrown when casting like this is done?
You might want to put a bug report into MySQL folks. I don't have latest MariaDB installed so I can't say if this issue exists in MariaDB also. I got curious and installed 10.0.25-MariaDB-0ubuntu0.16.04.1. It appears the same issue is in MariaDB 10.0.25 also.
Is this preventable?
Yes. Whenever dealing with int to float/double/decimal, cast implicitly to get predictable results. I still expect MySQL to look into this corner case. If someone comes across documentation that explains this behavior, please add a comment to this answer so that I can educate myself.