My modified block of code from here works for XOR'ing python lists via using functions (XOR and AND) of the Sympy library (first block of code below). However, I am stumped on how to iterate via sympy matrices (second block of code below).
The python lists code that works is:
from sympy import And, Xor
from sympy.logic import SOPform, simplify_logic
from sympy import symbols
def LogMatrixMult (A, B):
rows_A = len(A)
cols_A = len(A[0])
rows_B = len(B)
cols_B = len(B[0])
if cols_A != rows_B:
print ("Cannot multiply the two matrices. Incorrect dimensions.")
return
# Create the result matrix
# Dimensions would be rows_A x cols_B
C = [[0 for row in range(cols_B)] for col in range(rows_A)]
for i in range(rows_A):
for j in range(cols_B):
for k in range(cols_A):
# I can add Sympy's in simplify_logic(-)
C[i][j] = Xor(C[i][j], And(A[i][k], B[k][j]))
return C
b, c, d, e, f, w, x, y, z = symbols('b c d e f w x y z')
m1 = [[b,c,d,e]]
m2 = [[w,x],[x,z],[y,z],[z,w]]
result = simplify_logic(LogMatrixMult(m1, m2)[0][0])
print(result)
In the block below using Sympy matrices note that the i,j,k and C, A, B definitions is from me trying to modify to use the iterator, I don't know if this needed or correct.
from sympy import And, Xor
from sympy.matrices import Matrix
from sympy.logic import SOPform, simplify_logic
from sympy import symbols, IndexedBase, Idx
from sympy import symbols
def LogMatrixMultArr (A, B):
rows_A = A.rows
cols_A = A.cols
rows_B = B.rows
cols_B = B.cols
i,j,k = symbols('i j k', cls=Idx)
C = IndexedBase('C')
A = IndexedBase('A')
B = IndexedBase('B')
if cols_A != rows_B:
print ("Cannot multiply the two matrices. Incorrect dimensions.")
return
# Create the result matrix
# Dimensions would be rows_A x cols_B
C = [[0 for row in range(cols_B)] for col in range(rows_A)]
for i in range(rows_A):
for j in range(cols_B):
for k in range(cols_A):
# I can add Sympy's in simplify_logic(-)
C[i,j] = Xor(C[i,j], And(A[i,k], B[k,j]))
# C[i][j] = Xor(C[i][j],And(A[i][k],B[k][j]))
return C
b, c, d, e, f, w, x, y, z = symbols('b c d e f w x y z')
P = Matrix([w,x]).reshape(1,2)
Q = Matrix([y,z])
print(LogMatrixMultArr(P,Q))
The error I get is: TypeError: list indices must be integers or slices, not tuple
C[i,j] = Xor(C[i,j], And(A[i,k], B[k,j]))
Now I believe I have to do something with some special way of sympy's iterating but am stuck on how to get it to work in the code - if I do even need this methodology.
Also, if anyone knows how to do something such as the above using XOR and And (non-bitwise) instead of + and * operators in a faster way, please do share.
Thanks.
I think the problem is with IndexedBase objects. I'm not competent on these but it seems you do not use them right. If you replace
i,j,k = symbols('i j k', cls=Idx)
C = IndexedBase('C')
A = IndexedBase('A')
B = IndexedBase('B')
by
C = zeros(rows_A, cols_B)
and remove C = [[0 for row in range(cols_B)] for col in range(rows_A)], then it works.
Related
Just as a silly example, say that I wish to solve for the following nonlinear equation x^2 - F(c)=0, where c can take different values between zero and one and F is a standard normal CDF. If I wish to solve for one particular value of c, I would use the following code:
c = linspace(0,1,100);
L = length(c);
x0 = c;
function Y = eq(x)
Y = x^2 - cdfnor("PQ",x-c(1),0,1)
endfunction
xres = fsolve(x0(1),eq);
My question is: Is there a way to solve for the equation for each value of c (and not only c(1))? Specifically, if I can use a loop over fsolve? If so, how?
Just modify your script like this:
c = linspace(0,1,100);
L = length(c);
x0 = c;
function Y = eq(x)
Y = x^2 - cdfnor("PQ",x-c,zeros(c),ones(c))
endfunction
xres = fsolve(x0,eq);
To assert all elements of a list equal, is there no "Equal()" in z3py? There is for example Distinct().
Using numpy works:
from z3 import *
import numpy as np
s = Solver()
B = [BitVec(f"b{j}", 7) for j in range(11)]
C_eq = [B[i] == B[j] for i in range(3) for j in range(3) if i != j]
s.add(C_eq)
print(s.check())
print(s.model())
Indeed z3py supports Distinct, but not Equals. The reason is that a straightforward encoding of Distinct requires quadratic number of constraints, and it can bog the solver down. Having a Distinct primitive allows the solver to avoid this bottleneck. Equals, however, only requires a linear number of constraints in the number of variables and typically doesn't need any special help to be efficient.
To simplify your programming, you can define it yourself, however. Something like this:
from z3 import *
def Equals(*xs):
constr = True
if xs:
base = xs[0]
for x in xs[1:]:
constr = And(constr, base == x)
return constr
And then use it like this:
x, y, z = Ints('x y z')
print(Equals())
print(Equals(x))
print(Equals(x, y))
print(Equals(x, y, z))
which prints:
True
True
And(True, x == y)
And(And(True, x == y), x == z)
We want to compare a^b to c^d, and tell if the first is smaller, greater, or equal (where ^ denotes exponentiation).
Obviously, for very large numbers, we cannot explicitely compute these values.
The most common approach in this situation is to apply log on both sides and compare b * log(a) to d * log(c). The issue here is that logs are floating-point operations, and as such we cannot trust our answer with 100% confidence (there might be some values which are incredibly close, and because of floating-point error we get a wrong answer).
Is there an algorithm for solving this problem? I've been scouring the intrernet for this, but I can only find solutions which work for particular cases only (e.g. in which one exponent is a multiple of another), or which use floating point in some way (logarithms, division) etc.
This is sort of two questions in one:
Are they equal?
If not, which one is greater?
As Peter O. observes, it's easiest to build in a language that provides an arbitrary-precision fraction type. I'll use Python 3.
Let's assume without loss of generality that a ≤ c (swap if necessary) and b is relatively prime to d (divide both by the greatest common divisor).
To get at the core of the question, I'm going to assume that a, c > 0 and b, d ≥ 0. Removing this assumption is tedious but not difficult.
Equality test
There are some easy cases where a = 1 or b = 0 or c = 1 or d = 0.
Separately, necessary conditions for a^b = c^d are
i. b ≥ d, since otherwise b < d, which together with a ≤ c implies a^b < c^d;
ii. a is a divisor of c, since we know from (i) that a^b = c^d is a divisor of c^b = c^(b−d) c^d.
When these conditions hold, we can divide through by a^d to reduce the problem to testing whether a^(b−d) = (c/a)^d.
In Python 3:
def equal_powers(a, b, c, d):
while True:
lhs_is_one = a == 1 or b == 0
rhs_is_one = c == 1 or d == 0
if lhs_is_one or rhs_is_one:
return lhs_is_one and rhs_is_one
if a > c:
a, b, c, d = c, d, a, b
if b < d:
return False
q, r = divmod(c, a)
if r != 0:
return False
b -= d
c = q
def test_equal_powers():
for a in range(1, 25):
for b in range(25):
for c in range(1, 25):
for d in range(25):
assert equal_powers(a, b, c, d) == (a ** b == c ** d)
test_equal_powers()
Inequality test
Once we've established that the two quantities are not equal, it's time to figure out which one is greater. (Without the equality test, the code here could run forever.)
If you're doing this for real, you should consult an actual reference on computing elementary functions. I'm just going to try to do the simplest thing that works.
Time for a calculus refresher. We have the Taylor series
−log x = (1−x) + (1−x)^2/2 + (1−x)^3/3 + (1−x)^4/4 + ...
To get a lower bound, truncate the series. To get an upper bound, we can truncate but replace the final term (1−x)^n/n with (1−x)^n/n (1/x), since
(1−x)^n/n (1/x)
= (1−x)^n/n (1 + (1−x) + (1−x)^2 + ...)
= (1−x)^n/n + (1−x)^(n+1)/n + (1−x)^(n+2)/n + ...
> (1−x)^n/n + (1−x)^(n+1)/(n+1) + (1−x)^(n+2)/(n+2) + ...
To get a good convergence rate, we're going to want 0.5 ≤ x < 1, which we can achieve by dividing x by a power of two.
In Python, we'll represent a real number as an infinite generator of shrinking intervals that contain the true value. Once the intervals for b log a and d log c are disjoint, we can determine how they compare.
import fractions
def minus(x, y):
while True:
x_lo, x_hi = next(x)
y_lo, y_hi = next(y)
yield x_lo - y_hi, x_hi - y_lo
def times(b, x):
for lo, hi in x:
yield b * lo, b * hi
def restricted_log(a):
series = 0
n = 0
numerator = 1
while True:
n += 1
numerator *= 1 - a
series += fractions.Fraction(numerator, n)
yield -(series + fractions.Fraction(numerator * (1 - a), (n + 1) * a)), -series
def log(a):
n = 0
while a >= 1:
a = fractions.Fraction(a, 2)
n += 1
return minus(restricted_log(a), times(n, restricted_log(fractions.Fraction(1, 2))))
def less_powers(a, b, c, d):
lhs = times(b, log(a))
rhs = times(d, log(c))
while True:
lhs_lo, lhs_hi = next(lhs)
rhs_lo, rhs_hi = next(rhs)
if lhs_hi < rhs_lo:
return True
if rhs_hi < lhs_lo:
return False
def test_less_powers():
for a in range(1, 10):
for b in range(10):
for c in range(1, 10):
for d in range(10):
if a ** b != c ** d:
assert less_powers(a, b, c, d) == (a ** b < c ** d)
test_less_powers()
Inputs:
1) I = Tensor of dim (N, C, X) (Input)
2) W = Tensor of dim (N, X, Y) (Weight)
Output:
1) O = Tensor of dim (N, C, Y) (Output)
I want to compute:
I = I.view(N, C, X, 1)
W = W.view(N, 1, X, Y)
PROD = I*W
O = PROD.sum(dim=2)
return O
without incurring N * C * X * Y memory overhead.
Basically I want to calculate the weighted sum of a feature map wherein the weights are the same along the channel dimension, without incurring memory overhead per channel.
Maybe I could use
from itertools import product
O = torch.zeros(N, C, Y)
for n, x, y in product(range(N), range(X), range(Y)):
O[n, :, y] += I[n, :, x]*W[n, x, y]
return O
but that would be slower (no broadcasting) and I'm not sure how much memory overhead would be incurred by saving variables for the backward pass.
You can use torch.bmm (https://pytorch.org/docs/stable/torch.html#torch.bmm). Just do torch.bmm(I,W)
To verify the results :
import torch
N, C, X, Y= 100, 10, 9, 8
i = torch.rand(N,C,X)
w = torch.rand(N,X,Y)
o = torch.bmm(i,w)
# desired result code
I = i.view(N, C, X, 1)
W = w.view(N, 1, X, Y)
PROD = I*W
O = PROD.sum(dim=2)
print(torch.allclose(O,o)) # should output True if outputs are same.
EDIT: Ideally, I would assume using pytorch's internal matrix multiplication is efficient. However, you can also measure the memory usage with tracemalloc (at least on CPU). See https://discuss.pytorch.org/t/measuring-peak-memory-usage-tracemalloc-for-pytorch/34067 for GPU.
import torch
import tracemalloc
tracemalloc.start()
N, C, X, Y= 100, 10, 9, 8
i = torch.rand(N,C,X)
w = torch.rand(N,X,Y)
o = torch.bmm(i,w)
# output is a tuple indicating current memory and peak memory
print(tracemalloc.get_traced_memory())
You can do the same with other code and see the bmm implementation is indeed efficient.
import torch
import tracemalloc
tracemalloc.start()
N, C, X, Y= 100, 10, 9, 8
i = torch.rand(N,C,X)
w = torch.rand(N,X,Y)
I = i.view(N, C, X, 1)
W = w.view(N, 1, X, Y)
PROD = I*W
O = PROD.sum(dim=2)
# output is a tuple indicating current memory and peak memory
print(tracemalloc.get_traced_memory())
I have a vector X of 20 real numbers and a vector Y of 20 real numbers.
I want to model them as
y = ax^2+bx + c
How to find the value of 'a' , 'b' and 'c'
and best fit quadratic equation.
Given Values
X = (x1,x2,...,x20)
Y = (y1,y2,...,y20)
i need a formula or procedure to find following values
a = ???
b = ???
c = ???
Thanks in advance.
Everything #Bartoss said is right, +1. I figured I just add a practical implementation here, without QR decomposition. You want to evaluate the values of a,b,c such that the distance between measured and fitted data is minimal. You can pick as measure
sum(ax^2+bx + c -y)^2)
where the sum is over the elements of vectors x,y.
Then, a minimum implies that the derivative of the quantity with respect to each of a,b,c is zero:
d (sum(ax^2+bx + c -y)^2) /da =0
d (sum(ax^2+bx + c -y)^2) /db =0
d (sum(ax^2+bx + c -y)^2) /dc =0
these equations are
2(sum(ax^2+bx + c -y)*x^2)=0
2(sum(ax^2+bx + c -y)*x) =0
2(sum(ax^2+bx + c -y)) =0
Dividing by 2, the above can be rewritten as
a*sum(x^4) +b*sum(x^3) + c*sum(x^2) =sum(y*x^2)
a*sum(x^3) +b*sum(x^2) + c*sum(x) =sum(y*x)
a*sum(x^2) +b*sum(x) + c*N =sum(y)
where N=20 in your case. A simple code in python showing how to do so follows.
from numpy import random, array
from scipy.linalg import solve
import matplotlib.pylab as plt
a, b, c = 6., 3., 4.
N = 20
x = random.rand((N))
y = a * x ** 2 + b * x + c
y += random.rand((20)) #add a bit of noise to make things more realistic
x4 = (x ** 4).sum()
x3 = (x ** 3).sum()
x2 = (x ** 2).sum()
M = array([[x4, x3, x2], [x3, x2, x.sum()], [x2, x.sum(), N]])
K = array([(y * x ** 2).sum(), (y * x).sum(), y.sum()])
A, B, C = solve(M, K)
print 'exact values ', a, b, c
print 'calculated values', A, B, C
fig, ax = plt.subplots()
ax.plot(x, y, 'b.', label='data')
ax.plot(x, A * x ** 2 + B * x + C, 'r.', label='estimate')
ax.legend()
plt.show()
A much faster way to implement solution is to use a nonlinear least squares algorithm. This will be faster to write, but not faster to run. Using the one provided by scipy,
from scipy.optimize import leastsq
def f(arg):
a,b,c=arg
return a*x**2+b*x+c-y
(A,B,C),_=leastsq(f,[1,1,1])#you must provide a first guess to start with in this case.
That is a linear least squares problem. I think the easiest method which gives accurate results is QR decomposition using Householder reflections. It is not something to be explained in a stackoverflow answer, but I hope you will find all that is needed with this links.
If you never heard about these before and don't know how it connects with you problem:
A = [[x1^2, x1, 1]; [x2^2, x2, 1]; ...]
Y = [y1; y2; ...]
Now you want to find v = [a; b; c] such that A*v is as close as possible to Y, which is exactly what least squares problem is all about.