Computing a single element of the adjugate or inverse of a symbolic binary matrix - matrix

I'm trying to get a single element of an adjugate A_adj of a matrix A, both of which need to be symbolic expressions, where the symbols x_i are binary and the matrix A is symmetric and sparse. Python's sympy works great for small problems:
from sympy import zeros, symbols
size = 4
A = zeros(size,size)
x_i = [x for x in symbols(f'x0:{size}')]
for i in range(size-1):
A[i,i] += 0.5*x_i[i]
A[i+1,i+1] += 0.5*x_i[i]
A[i,i+1] = A[i+1,i] = -0.3*(i+1)*x_i[i]
A_adj_0 = A[1:,1:].det()
A_adj_0
This calculates the first element A_adj_0 of the cofactor matrix (which is the corresponding minor) and correctly gives me 0.125x_0x_1x_2 - 0.28x_2x_2^2 - 0.055x_1^2x_2 - 0.28x_1x_2^2, which is the expression I need, but there are two issues:
This is completely unfeasible for larger matrices (I need this for sizes of ~100).
The x_i are binary variables (i.e. either 0 or 1) and there seems to be no way for sympy to simplify expressions of binary variables, i.e. simplifying polynomials x_i^n = x_i.
The first issue can be partly addressed by instead solving a linear equation system Ay = b, where b is set to the first basis vector [1, 0, 0, 0], such that y is the first column of the inverse of A. The first entry of y is the first element of the inverse of A:
b = zeros(size,1)
b[0] = 1
y = A.LUsolve(b)
s = {x_i[i]: 1 for i in range(size)}
print(y[0].subs(s) * A.subs(s).det())
print(A_adj_0.subs(s))
The problem here is that the expression for the first element of y is extremely complicated, even after using simplify() and so on. It would be a very simple expression with simplification of binary expressions as mentioned in point 2 above. It's a faster method, but still unfeasible for larger matrices.
This boils down to my actual question:
Is there an efficient way to compute a single element of the adjugate of a sparse and symmetric symbolic matrix, where the symbols are binary values?
I'm open to using other software as well.
Addendum 1:
It seems simplifying binary expressions in sympy is possible with a simple custom substitution which I wasn't aware of:
A_subs = A_adj_0
for i in range(size):
A_subs = A_subs.subs(x_i[i]*x_i[i], x_i[i])
A_subs

You should make sure to use Rational rather than floats in sympy so S(1)/2 or Rational(1, 2) rather than 0.5.
There is a new (undocumented and for the moment internal) implementation of matrices in sympy called DomainMatrix. It is likely to be a lot faster for a problem like this and always produces polynomial results in a fully expanded form. I expect that it will be much faster for this kind of problem but it still seems to be fairly slow for this because is is not sparse internally (yet - that will probably change in the next release) and it does not take advantage of the simplification from the symbols being binary-valued. It can be made to work over GF(2) but not with symbols that are assumed to be in GF(2) which is something different.
In case it is helpful though this is how you would use it in sympy 1.7.1:
from sympy import zeros, symbols, Rational
from sympy.polys.domainmatrix import DomainMatrix
size = 10
A = zeros(size,size)
x_i = [x for x in symbols(f'x0:{size}')]
for i in range(size-1):
A[i,i] += Rational(1, 2)*x_i[i]
A[i+1,i+1] += Rational(1, 2)*x_i[i]
A[i,i+1] = A[i+1,i] = -Rational(3, 10)*(i+1)*x_i[i]
# Convert to DomainMatrix:
dM = DomainMatrix.from_list_sympy(size-1, size-1, A[1:, 1:].tolist())
# Compute determinant and convert back to normal sympy expression:
# Could also use dM.det().as_expr() although it might be slower
A_adj_0 = dM.charpoly()[-1].as_expr()
# Reduce powers:
A_adj_0 = A_adj_0.replace(lambda e: e.is_Pow, lambda e: e.args[0])
print(A_adj_0)

Related

sympy matrix to explicit sum and back (to matrix notation)

I am working in sympy with symbolic matrices.
Once made explicit I can not return to implicit representations.
I tried to work something out with the pair of .as_explicit() and MatrixExpr.from_index_summation(expr)
But the latter seems to expect an explicit sigma notation sum, not a sum of indexed elements.
As a minimal working example here is my approach on matrix multiplication:
A = MatrixSymbol('A',3,4)
B = MatrixSymbol('B',4,3)
Matrix_Notation = A * B
Expanded = (A * B).as_explicit()
FromSummation = MatrixExpr.from_index_summation(Expanded)
Here we can see, that FromSummation is still the same as Expanded
I suppose that the Expanded expression should be converted to sigma sums such that .from_index_summation can be expected to work. But how can this be done?

Numeric and symbolic gradients don't match although Hessians do

For context, I have a small project in MATLAB where I try to replicate an algorithm involving some optimisation with the Newton algorithm. Although my issue is mainly with MATLAB, maybe it's my lacking profound background knowledge what's keeping me from finding a solution, so feel free to redirect me to the appropriate StackExchange site if needed.
The function I need to calculate the gradient vector and Hessian matrix for the optimization is :
function [zi] = Zi(lambda,j)
zi = m(j)*exp(-(lambda*v_tilde(j,:).'));
end
function [z] = Z(lambda)
res = arrayfun(#(x) Zi(lambda,x),1:length(omega));
z = sum(res);
end
function [f] = F(lambda)
f = log(Z(lambda));
end
where omega and v_tilde are Matrices of n d-Dimensional vectors and lambda is the d-Dimensional argument to the function. (right now, m(j) are just selectors (1 or 0), but the algorithm allows to refine these, so they shouldn't be removed.
I use the Derivest Suite to calculate the gradient and Hessian numerically, and, although logically slow for high dimensions, the algorithm as a whole works.
I implemented the same solution using the sym package, so that I could compute the gradient and Hessian in advance for some fix n and d, so they can then be evaluated quickly when needed. This would be the symbolic version:
V_TILDE = sym('v_tilde',[d,n])
syms n k
lambda = sym('lambda',[d,1]);
F = log(M*exp(-(transpose(V_TILDE)*lambda)));
matlabFunction( grad_F, 'File', sprintf('Grad_%d_dim_%d_n.m',d,n_max), 'vars',{a,lambda,V_TILDE});
matlabFunction( hesse_F, 'File', sprintf('Hesse_%d_dim_%d_n.m',d,n_max), 'vars',{a,lambda,V_TILDE});
As n is fix, there is no need to iterate over omega. The gradient and Hessian of this can be obtained through the corresponding functions of sym and then stored as matlabFunctions.
However, when I test both implementations against some values, they don't match, surprisingly though, the values of the hessian matrix match while the values of the gradient don't (the numerical calculation being correct), and the Newton algorithm iterates until the values are just NaN. These are some example values for d=2 and n=8:
Omega:
12.6987 91.3376
95.7507 96.4889
15.7613 97.0593
95.7167 48.5376
70.6046 3.1833
27.6923 4.6171
9.7132 82.3458
95.0222 3.4446
v:
61.2324
52.2271
gNum = HNum = 1.0e+03 *
8.3624 1.4066 -0.5653
-1.1496 -0.5653 1.6826
gSym = HSym = 1.0e+03 *
-52.8700 1.4066 -0.5653
-53.3768 -0.5653 1.6826
As you can see, the values of HNum and HSym match, but the gradients don't.
I'm happy to give any more context information, code snippets, or anything that helps. Thank you in advance!
Edit: As requested, here is a minimal test. The problem is basically that the values of gNum and gSym don't match (longer explanation above):
omega = [[12.6987, 91.3376];[95.7507, 96.4889];[15.7613, 97.0593];
[95.7167, 48.5376];[70.6046, 3.1833];[27.6923, 4.6171];[9.7132, 82.3458];
[95.0222, 3.4446]];
v = [61.2324;52.2271];
gradStr = sprintf('Grad_%d_dim_%d_n',length(omega(1,:)),length(omega));
hesseStr = sprintf('Hesse_%d_dim_%d_n',length(omega(1,:)),length(omega));
g = str2func(gradStr);
H = str2func(hesseStr);
selector = ones(1,length(omega)); %this will change, when n_max>n
vtilde = zeros(length(omega),length(omega(1,:)));
for i = 1:length(omega)
vtilde(i,:) = omega(i,:)-v;
end
lambda = zeros(1,length(omega(1,:))); % start of the optimization
[gNum,~,~] = gradest(#F,lambda)
[HNum,~] = hessian(#F,lambda)
gSym = g(selector,lambda.',omega.')
HSym = H(selector,lambda.',omega.')
Note: The DerivestSuite is a small library (~6 source files) that can be obtained under https://de.mathworks.com/matlabcentral/fileexchange/13490-adaptive-robust-numerical-differentiation

c++ eigen A.inverse()*B not equal to A.ldlt().solve(B)

I would like to compute the trace of the product of two given matrices, say A and B, Trace(AInv * B) where * is the regular matrix product, AInv is the inverse of A (being symmetric and positive definite) and B is symmetric.
Solution 1: computing the inverse explicitely
Noting that Trace(AInv * B) is equivalent to taking the sum of the componentwise product of AInv and B:
double sol1 = (A.inverse().cwiseProduct(B)).sum();
Solution 2: using ldlt decomposition from the Eigen library
double sol2 = (A.selfadjointView<Lower>().ldlt().solve(B)).trace();
Theoretically, these solutions should be the same, but in my test, they don't. Sounds like I am missing something. As .ldlt().solve() is not made to compute matrix inverse but rather solve a linear system, my question is : does .ldlt() perform any sort of normalization? If not, what I am doing wrong?
Many thanks!
The statement to compute sol1 is wrong: you need to either transpose one of the operands or use a matrix-matrix product: correct versions:
double sol1 = (A.inverse().cwiseProduct(B.transpose())).sum();
double sol1 = (A.inverse().lazyProduct(B)).diagonal().sum();
double sol1 = (A.inverse().lazyProduct(B)).trace();
double sol1 = (A.inverse() * B).diagonal().sum();
double sol1 = (A.inverse() * B).trace();
Note that, in Eigen, when you write (A*B).diagonal() only diagonal elements of A*B are computed;, not the off-diagonal ones.
In general, it is not recommended to explicitly compute the inverse of a matrix, and using either A.lu().solve(B) or A.ldlt().solve(B) will give you more accurate results and will be faster too because, unless A is very small (2, 3, 4), A.inverse() is equivalent to A.lu().solve(I). In the future, Eigen will very likely rewrite expressions like:
A.inverse() * B
as:
A.lu().solve(B)
for you anyway.

Matrix valued undefined functions in SymPy

I'm looking for a possibility to specify matrix quantities that depend on a variables. For scalars that works as follows, using undefined functions:
from sympy import *
x = Function('f')(t)
diff(x,t)
For Matrix Symbols like
x = MatrixSymbol('x',3,3)
i cannot find an equivalent. There is
i,j = Symbols('i j')
x = FunctionMatrix(6,1,Lambda((i,j),f))
but this is not what i need as you need to specify the contents of the matrix. The context is that i have equations
which should be derived in time and contain matrix valued elements.
I cannot deal with the elements of the matrices one by one.
Thanks!
I'm not sure about what you want, but I think you want to make a Matrix with differentiable elements. In that case, see if this works for you.
Create a matrix with function elements:
X = sym.FunctionMatrix(6,1,lambda i,j:sym.Function("x_%d%d" % (i,j))(t))
M = sym.Matrix(X)
M.diff(t)
This results in
Matrix([
[Derivative(x_00(t), t)],
[Derivative(x_10(t), t)],
[Derivative(x_20(t), t)],
[Derivative(x_30(t), t)],
[Derivative(x_40(t), t)],
[Derivative(x_50(t), t)]])
You may then replace stuff as you need.
Also, it may be preferrable if you populate the matrix with the expressions you need before differentiating. Leaving them as undefined functions may make it harder for you to simplify after substitution.

Numpy Hermitian Matrix class

Are you aware of something like a hermitian matrix class in numpy? I'd like to optimize matrix calculations like
B = U * A * U.H
, where A (and thus, B) are hermitian. Without specification, all matrix elements of B are calculated. In fact, it should be able to save a factor of about 2 here. Do I miss something?
The method I need should take take the upper/lower triangle of A, the full matrix of U and return the upper/lower triangle of B.
I don't think there exists a method for your specific problem, but with a little thought you might be able to build an algorithm from the low-level BLAS routines that are wrapped in SciPy. For example, dgemm, dsymm, and dtrmm do general, symmetric, and triangular matrix products respectively. Here's an example of using them:
from scipy.linalg.blas import dgemm, dsymm, dtrmm
A = np.random.rand(10, 10)
B = np.random.rand(10, 10)
S = np.dot(A, A.T) # symmetric matrix
T = np.triu(S) # upper triangular matrix
# normal matrix-matrix product
assert np.allclose(dgemm(1, A, B), np.dot(A, B))
# symmetric mat-mat product using only upper-triangle
assert np.allclose(dsymm(1, T, B), np.dot(S, B))
# upper-triangular mat-mat product
assert np.allclose(dtrmm(1, T, B), np.dot(T, B))
There are many other low-level BLAS routines available; I find the NETLIB page to be a good resource to learn what they do. You may be able to cleverly use some combination of the available routines to efficiently solve the problem you have in mind.
Edit: it looks like there are LAPACK routines that quickly compute exactly what you want: dsytrd or zhetrd, but unfortunately these don't appear to be wrapped directly in scipy.linalg.lapack, though scipy does provide cython wrappers for them. Best of luck!
I needed tridiagonal reduction of a symmetric/Hermitian matrix A,
T = Q^H * A * Q
– presumably OP's underlying problem – and I've just submitted a pull request to SciPy for properly interfacing LAPACK's {s,d}sytrd (for real symmetric matrices) and {c,z}hetrd (for Hermitian matrices). All routines use either only the upper or the lower triangular part of the matrix.
Once this has been merged, it can be used like
import numpy as np
n = 3
A = np.zeros((n, n), dtype=dtype)
A[np.triu_indices_from(A)] = np.arange(1, 2*n+1, dtype=dtype)
# query lwork -- optional
lwork, info = sytrd_lwork(n)
assert info == 0
data, d, e, tau, info = sytrd(A, lwork=lwork)
assert info == 0
The vectors d and e now contain the main diagonal and the upper and lower diagonal, respectively.

Resources