how to avoid DB::Exception: Division by zero in clickhouse when I use divide(Decimal v1,Decimal v2) and v2 comes from a subquery. I tried to use case when v2 = 0 then 0 else divide end but didn't work!
It works in recent CH versions
SELECT if(b = toDecimal64(0, 10), toDecimal64(0, 10), a / b) AS res
FROM
(
SELECT
materialize(toDecimal64(1, 10)) AS a,
materialize(toDecimal64(0, 10)) AS b
)
┌─res─┐
│ 0 │
└─────┘
Related
'Hi, I'm trying to calculate how many months to achieve a million dollars with compounding interest and monthly investments. There are my tries.'
'This first code work, but I want to replace the 92 in the rage with a compare formula like fv >= 1000000.'
'When I place the range like here, it doesn't work.'
Try while-loop may help:
pv = 130000 # present value
i = 4000 # regular monthly investment
r = 0.1375 # annual interest rate
n = 0 # number of months
# for n in range(0, 92):
fv = 0
while fv < 1000000:
fv = pv * (1 + r / 12) ** n + i * (((1 + r / 12) ** n - 1) / (r / 12))
n += 1 #don't forget
print(fv)
print(n)
You need to manually increase the value of n
The goal is to generate fake data.
We generate a set of parameters,
## Simulated data
df_3 = DataFrame(y = [0,1], size = [250,250], x1 =[2.,0.], x2 =[-1.,-2.])
Now, I want to generate the fake data per se,
df_knn =by(df_3, :y) do df
DataFrame(x_1 = rand(Normal(df[1,:x1],1), df[1,:size]),
x_2 = rand(Normal(df[1,:x2],1), df[1,:size]))
end
How I can replace by with groupby, here?
SOURCE: This excerpt is from the book, Data Science with Julia (2019).
I think this is what you mean here:
julia> combine(groupby(df_3, :y)) do df
DataFrame(x_1 = rand(Normal(df[1,:x1],1), df[1,:size]),
x_2 = rand(Normal(df[1,:x2],1), df[1,:size]))
end
500×3 DataFrame
Row │ y x_1 x_2
│ Int64 Float64 Float64
─────┼─────────────────────────────
1 │ 0 1.88483 0.890807
2 │ 0 2.50124 -0.280708
3 │ 0 1.1857 0.823002
⋮ │ ⋮ ⋮ ⋮
498 │ 1 -0.611168 -0.856527
499 │ 1 0.491412 -3.09562
500 │ 1 0.242016 -1.42652
494 rows omitted
I have a table with tuple partitions: (0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1), (3, 0), ...
CREATE TABLE my_table
(
id Int32,
a Int32,
b Float32,
c Int32
)
ENGINE = MergeTree
PARTITION BY
(
intDiv(id, 1000000),
a < 20000 AND b > 0.6 AND c >= 100
)
ORDER BY id;
I need only rows with partition (<any number>, 1) and I'm looking for a way to use partition value in a query like
SELECT *
FROM my_table
WHERE my_table.partition[2] == 1;
Does ClickHouse have such a feature?
In version 21.6 was added virtual columns _partition_id and _partition_value that can help you:
SELECT
*,
_partition_id,
_partition_value
FROM my_table
WHERE (_partition_value.2) = 1
And what is the problem with
where (a < 20000 AND b > 0.6 AND c >= 100) = 1
???
insert into my_table select 1, 3000000, 0, 0 from numbers(100000000);
insert into my_table select 1, 0, 10, 200 from numbers(100);
SET send_logs_level = 'debug';
set force_index_by_date=1;
select sum(id) from my_table where (a < 20000 AND b > 0.6 AND c >= 100) = 1;
...Selected 1/7 parts by partition key...
┌─sum(id)─┐
│ 100 │
└─────────┘
1 rows in set. Elapsed: 0.002 sec.
Though (_partition_value.2) = 1 will be faster because it does not require to read columns a,b,c for filtering.
So I was trying to write out the code to perform the one variable polynomial division algorithm using the open source SymPy. It already has a leading term function, so I thought it would be easy. The book, Ideals, Varieties, and Algorithms by Cox, Little and O'Shea gives the pseudo-code to be:
Input: g, f
Output: q, r
q := 0; r := f
WHILE r <> 0 AND LT (g) divides LT (r ) DO
q := q + LT (r )/ LT (g)
r := r − ( LT (r )/ LT (g))g
So my python code is:
from sympy import *
x = symbols('x')
f= x**4-5*x**2-2*x+7
g = 3*x**2 + 4*x - 2
q = 0
r = f
while (r != 0 & degree(g) < LT(r ) ):
q = q + LT (r )/ LT (g)
r = r - ( LT (r )/ LT (g))*g
print(q,r)
But this gives an error message: TypeError: unsupported operand type(s) for &: 'int' and 'Integer'. So in the console, I query type(r != 0) and it gives bool, but for type(degree(g) < degree(r)), it gives sympy.logic.boolalg.BooleanTrue. I tried to look up the documentation for sympy.logic.boolalg.BooleanTrue, and I couldn't find how to make it an object so that I can combine with a bool using a logical &.
Why does sympy have its own boolean variables and how do I get it to play well with the normal boolean variables?
SymPy has its own Boolean class because needs to be used in the Basic expression construction and needs to support methods like subs etc to be used consistently in the SymPy architecture.
The Boolean type can be used with bool e.g.:
In [7]: S.true & True
Out[7]: True
In [8]: S.true and True
Out[8]: True
The problem in your example is actually to do with operator preference:
In [9]: r != 0 & degree(g) < LT(r )
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-9-f2737cccd83e> in <module>
----> 1 r != 0 & degree(g) < LT(r )
TypeError: unsupported operand type(s) for &: 'int' and 'Integer'
In [10]: (r != 0) & (degree(g) < LT(r))
Out[10]:
4
2 < x
In [11]: r != (0 & degree(g)) < LT(r)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-11-c7a6030e1401> in <module>
----> 1 r != (0 & degree(g)) < LT(r)
TypeError: unsupported operand type(s) for &: 'int' and 'Integer'
It is 0 & degree(g) which is evaluated first and SymPy's Integer type can not be used with & which is used for symbolic "and" in SymPy:
In [14]: (x<1) & (x>-1)
Out[14]: x > -1 ∧ x < 1
You probably meant to use Python's and operator which has a lower precedence than &:
In [15]: r != 0 and degree(g) < LT(r)
Out[15]:
4
2 < x
This will not actually work though in the while loop because the Boolean expression is indeterminate:
---> 10 while (r != 0 and degree(g) < LT(r ) ):
11 q = q + LT (r )/ LT (g)
12 r = r - ( LT (r )/ LT (g))*g
~/current/sympy/sympy/sympy/core/relational.py in __nonzero__(self)
382
383 def __nonzero__(self):
--> 384 raise TypeError("cannot determine truth value of Relational")
385
386 __bool__ = __nonzero__
TypeError: cannot determine truth value of Relational
This is because the truth of the condition is unknowable:
In [18]: degree(g) < LT(r)
Out[18]:
4
2 < x
I think what you meant to check there is rem(LT(g), LT(r)) == 0.
from sympy import *
x = symbols('x')
f= x**4-5*x**2-2*x+7
g = 3*x**2 + 4*x - 2
q = 0
r = f
while (r != 0 and rem(LT(g), LT(r)) == 0):
q = q + LT (r )/ LT (g)
r = r - ( LT (r )/ LT (g))*g
print(q,r)
Output:
0 x**4 - 5*x**2 - 2*x + 7
Thanks so much, Oscar (though I still don't understand the need for the sympy boolean to be different). Your response allowed me to find other issues and the following code works (which is in function form):
from sympy import *
x = symbols('x')
# the division algorithm, returns the quotient and remainder
def DivAlg(f,g):
q = 0 # quotient
r = f # remainder
while (r != 0) & (degree(g) <= degree(r)):
q = q + LT(r)/LT(g)
r = r - expand(( LT(r)/LT(g)) * g) # the expand command is necessary!
return [q,r]
Can try DivAlg(x**4-x**3-4*x**2-5*x-3, x**2+3*x-5).
I also used this to write a function that uses the Euclidean algorithm to find the GCD of two one-variable polynomials:
def GCD(f,g):
h = f
s = g
while (s!= 0):
r = DivAlg(h,s)[1]
h = s
s = r
return h/LC(h) # LC(h) is the coefficient of highest order term.
You can try it out with print(GCD(x**3-x**2-x-2,x**4-x**3-4*x**2-5*x-3)).
This is the code needed to do problem 1.5.8 (p. 46) of the fun book, Ideals, Varieties, and Algorithms by Cox, Little, and O'Shea. I should note that sympy - of course - already has an implementation of this with the following command: gcd(x**3-x**2-x-2,x**4-x**3-4*x**2-5*x-3,domain=QQ)
I'm trying to get the exact equivalent (not functional) of this vb.net code in F#:
Function FastPow(ByVal num As Double, ByVal exp As Integer) As Double
Dim res As Double = 1
If exp < 1 Then
If exp = 0 Then Return res
exp = -exp
num = 1 / num
End If
Do While exp > 1
If exp Mod 2 = 1 Then
res = res * num
num = num * num
exp = exp >> 1
Loop
Return res * num
End Function
I wrote this:
let FastPow num exp =
let mutable ex = exp
let mutable res = 1
let mutable n = num
if ex < 1 then
if ex = 0 then res
ex <- -ex
n <- 1 / n
while ex > 1 do
if (ex % 2 = 1) then
res <- res * n
n <- n * n
exp >>> 1
res * n
but in the line "if ex = 0 then res" at res I got an error:
"This expression has type int but is here used with type unit".
I cannot understand why it gives me that error.
Edit: i actually got a warning as well:
"This expression should have type 'unit', but has type 'int'."
at "if (ex % 2 = 1) then"
In F#, a function's return value is the last expression evaluated in the function. So, lets focus on the following:
if ex < 1 then
if ex = 0 then res (* <--- this is not an early return *)
ex <- -ex (* <--- F# evaluates this code after the *)
n <- 1 / n (* if statement *)
Additionally, if statements have return values, which also happens to be the last value executed in the if statement. If an if statement isn't the return value of a function, it should have the return type unit. Notice that variable assignment has a return type of unit.
We need to rewrite your code to accomodate your early return, so we can do this:
let FastPow2 num exp =
if exp = 0 then 1
else
let mutable ex = exp
let mutable res = 1
let mutable n = num
if ex < 1 then
ex <- -ex
n <- 1 / n
while ex > 1 do
if (ex % 2 = 1) then (* still have a bug here *)
res <- res * n
n <- n * n
exp >>> 1 (* <--- this is not a variable assignment *)
res * n
We still have a bug, although I think F# is reporting the error in the wrong place. The expression exp >>> 1 returns an int, it does not assign any variables, so its not equivalent to your original C# code. I think you meant to use the ex variable instead. We can fix your code as follows:
let FastPow2 num exp =
if exp = 0 then 1
else
let mutable ex = exp
let mutable res = 1
let mutable n = num
if ex < 1 then
ex <- -ex
n <- 1 / n
while ex > 1 do
if (ex % 2 = 1) then
res <- res * n
n <- n * n
ex <- ex >>> 1
res * n
Now your function is fixed, but its really really ugly. Lets convert it to more idiomatic F#. You can replace the if statement with pattern matching, and replace the while loop with recursion:
let FastPow2 num exp =
match exp with
| 0 -> 1
| _ ->
let rec loop ex res n =
if ex > 1 then
let newRes = if ex % 2 = 1 then res * n else res
loop (ex >>> 1) newRes (n * n)
else res * n
let ex, n = if exp < 1 then (-exp, 1 / num) else (exp, num)
loop ex 1 n
Much better! Theres still some more room to beautify this function, but you get the idea :)
The problem is for an if statment to resolve to a value rather than unit, you need both the "then" part and the "else" part, both of which resolve to the same type.
For example:
let a = if true then 1;;
Will generate the same error - expression has type int but used with type unit.
However:
let a = if true then 1 else 0;;
Will evaluate to int without an error.
This is about as close as you can get, as others have already said you can't jump out of the middle of a functional and there's one place were you don't update a variable (at the bottom of the while).
let FastPow num exp =
let mutable exp = exp
let mutable res = 1
let mutable n = num
match exp with
| O -> n <- num
| _ when exp < 1 ->
exp <- -exp
n <- 1 / n
| _ ->
while exp > 1 do
if (exp % 2 = 1) then
res <- res * n
n <- n * n
exp <- exp >>> 1
res * n
I could be more beautiful if it was written more functionally.
It means that after then there should be some expression, but you have integer value. You cannot jump out from the middle of the function.
Edit
"If" didn't work because of
ex >>> 1
should be
ex <- ex >>> 1
Here's code that works:
let FastPow num exp =
let calcExp num exp =
let mutable res = 1.0
let mutable n = num
let mutable ex = exp
while ex > 1 do
if ((ex % 2) = 1) then
res <- res * n
n <- n * n
ex <- ex >>> 1
res * n
match exp with
| ex when ex = 0 -> 1.0
| ex when ex < 0 -> calcExp (1.0/num) -exp
| _ -> calcExp num exp
I just take out calculation as separate function, and at the end there is checking for arguments
Thanks for the answers. This is the current non-functional version.
let FastPow num exp =
let mutable ex = exp
let mutable res = 1.0
let mutable n = num
if ex = 0 then 1.0
else
if ex < 1 then
ex <- -ex
n <- 1.0 / n
while ex > 1 do
if (ex % 2 = 1) then res <- res * n
n <- n * n
ex <- ex >>> 1
res * n
Now that I have a working version I will try to make it more functional but that's outside the scope of this question.
EDIT: I got better results that I expected so I will post the recursive version optimized for speed (slightly faster than the iterative version and about 10% faster than the C# iterative version (!!!) in my computer):
let rec loop res num exp =
if exp = 0 then res
elif (exp % 2) = 1 then loop (res * num) (num * num) (exp / 2)
else loop res (num * num) (exp / 2)
let FP num exp =
let n = if exp < 0 then 1.0 / num else num
loop 1.0 n (Math.Abs(exp))