I have a 3x3 matrix; I am interested in finding the eigen values and corresponding eigen vector numerically. I know for smaller system we can take determinant and will solve the determinant equation , which results in eigen values. But I have the matrix size which is pretty big; say 1000x1000. To show the problem, I have chosen 3x3 (which is dummy representation of the actual problem). Now how to compute the eigen values and eigen vectors in python.
import numpy as np
eg_matrix=np.array([[-2-x, -4, 2], [-2, 1-x, 2],[4,2,5-x]])
The system which is similar to my problem is; lambda_3 corresponds to eigen value.
You only give your matrix to the eigenvalue solver
import numpy as np
A=np.array([[-2, -4, 2], [-2, 1, 2],[4,2,5]])
x, V = np.linalg.eig(A)
assert np.allclose(V # np.diag(x), A # V)
For your example it gives x = array([-5., 3., 6.])
Each element of x plugged in the expression np.det(A - x * np.eye(len(A))) will be close to zero.
for instance
np.linalg.det(A - np.eye(len(A))*x[0])
gives 4.849454171562702e-13
V is the eigenvector matrix.
Related
I am using Pytorch. I have a 4d array with shape (C, H, W, K) and I want to multiply it to another 4d-array with shape (N, C, H, W).
I want a final 2-d matrix (N, K).
How should I implement this in Pytorch?
And, more generally, how does higher order matrix multiplication works? Are there any rules to keep in mind in general?
You can use view to reshape the 4D tensors into simple 2D matrices and then multiply them
x1 = x1.view(-1, x1.shape[-1]) # from C-H-W-k -> CHW-K
x2 = x2.view(x2.shape[0], -1) # from N-C-H-W -> N-CHW
out = x2 # x1 # matrix multiplication
Note that the complexity of view, in generalm is minimal as it usually does not incur copying and moving of tensor values in memory - only changes to the header of the tensor representation.
For more advanced and more flexible tensor multiplication, you might want to look at einsum.
I found a Python library for Laplacian Score Feature Selection. But the implementation is seemingly different from the research paper.
I implemented the selection method according to the algorithm from the paper (https://papers.nips.cc/paper/2909-laplacian-score-for-feature-selection.pdf), which is as follows:
However, I found a Python library that implements the Laplacian method (https://github.com/jundongl/scikit-feature/blob/master/skfeature/function/similarity_based/lap_score.py).
So to check that my implementation was correct, I ran both versions on my dataset, and got different answers. In the process of debugging, I saw that the library used different formulas when calculating the affinity matrix (The S matrix from the paper).
The paper uses this formula:
, While the library uses
W_ij = exp(-norm(x_i - x_j)/2t^2)
Further investigation revealed that the library calculates the affinity matrix as follows:
t = kwargs['t']
# compute pairwise euclidean distances
D = pairwise_distances(X)
D **= 2
# sort the distance matrix D in ascending order
dump = np.sort(D, axis=1)
idx = np.argsort(D, axis=1)
idx_new = idx[:, 0:k+1]
dump_new = dump[:, 0:k+1]
# compute the pairwise heat kernel distances
dump_heat_kernel = np.exp(-dump_new/(2*t*t))
G = np.zeros((n_samples*(k+1), 3))
G[:, 0] = np.tile(np.arange(n_samples), (k+1, 1)).reshape(-1)
G[:, 1] = np.ravel(idx_new, order='F')
G[:, 2] = np.ravel(dump_heat_kernel, order='F')
# build the sparse affinity matrix W
W = csc_matrix((G[:, 2], (G[:, 0], G[:, 1])), shape=
n_samples,n_samples))
bigger = np.transpose(W) > W
W = W - W.multiply(bigger) + np.transpose(W).multiply(bigger)
return W
I'm not sure why the library squares each value in the distance matrix. I see that they also do some reordering, and they use a different heat kernel formula.
So I'd just like to know if any of the resources (The paper or the library) are wrong, or if they're somehow equivalent, or if anyone knows why they differ.
I have some data in a 2D plane. I want to calculate their variance but the regression line could be vertical to x-axis, what's the proper way to obtain the variance?
If you are fitting a line to data but are okay with it being vertical, this looks less like linear regression (which assumes y is a function of x) and more like principal component analysis. Which can be done with sklearn as follows, computing the variance along the way.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
data = np.random.multivariate_normal([1, 1], [[0.2, 0], [0, 4]], size=100)
pca = PCA(n_components=2)
pca.fit(data)
print('Residual variance', pca.explained_variance_[1])
I fit two components here, which together explain all the variance since the data is 2D. The first component is the line that takes place of the regression line in this model. The second is where the residuals are, so the residual variance comes from there. Visualization:
line_direction = pca.components_[0]
M = np.abs(data).max()
t = np.linspace(-M, M)
center = data.mean(axis=0)
line = line_direction*t[:, None] + center
plt.plot(line[:, 0], line[:, 1], 'r')
plt.plot(data[:, 0], data[:, 1], '.')
plt.axes().set_aspect('equal', 'datalim')
plt.show()
The simulated data is random, but this is what it can look like:
Residual variance 0.23184791439896069
This is the variance in the direction perpendicular to the chosen line, not in the vertical direction (which wouldn't be appropriate here).
Related: Total least squares
I want to create a N * N lower triangular matrix with a length N*(N+1)/2 vector. I find tensorflow has a function tf.contrib.distributions.fill_triangular that achieves my goal.
However when I give
from tensorflow.contrib.distributions import fill_triangular
it says
cannot import name fill_triangular
My tensorflow version is 1.4.1. Can anyone let me know where is fill_triangular or how to create a N * N lower triangular matrix with a length N*(N+1)/2 vector?
fill_triangular is currently only available in the master version of Tensorflow (docs); it is not included even in the latest stable version 1.5 (docs), let alone 1.4.1.
You can build your matrix using numpy.tril_indices; here is an example with N=3:
import numpy as np
a = np.array([1, 2, 3, 4, 5, 6]) # this is your N*(N+1)/2 vector
tri = np.zeros((3,3)) # initialize an NxN zero matrix
tri[np.tril_indices(3, 0)] = a
tri
the result is
array([[ 1., 0., 0.],
[ 2., 3., 0.],
[ 4., 5., 6.]])
I have a really big (1.5M x 16M) sparse csr scipy matrix A. What i need to compute is the similarity of each pair of rows. I have defined the similarity as this:
Assume a and b are two rows of matrix A
a = (0, 1, 0, 4)
b = (1, 0, 2, 3)
Similarity (a, b) = 0*1 + 1*0 + 0*2 + 4*3 = 12
To compute all pairwise row similarities I use this (or Cosine similarity):
AT = np.transpose(A)
pairs = A.dot(AT)
Now pairs[i, j] is the similarity of row i and row j for all such i and j.
This is quite similar to pairwise Cosine similarity of rows. So If there is an efficient parallel algorithm that computes pairwise Cosine similarity it would work for me as well.
The problem: This dot product is very slow because it uses just one cpu (I have access to 64 of those cpus on my server).
I can also export A and AT to a file and run any other external program that does the multiplication in parallel and get the results back to the Python program.
Is there any more efficient way of doing this dot product? or computing the pairwise similarity in Parallel?
I finally used the 'Cosine' distance metric of scikit-learn and its pairwise_distances functions which support sparse matrices and is highly parallelised.
sklearn.metrics.pairwise.pairwise_distances(X, Y=None, metric='euclidean', n_jobs=1, **kwds)
I could also divide A into n horizontal parts and use the parallel python package to run multiple multiplications and horizontally stack the results later.
I wrote own implementation using sklearn. It is not parallel but it quite fast for large matrices.
from scipy.sparse import spdiags
from sklearn.preprocessing import normalize
def get_similarity_by_x_dot_x_greedy_for_memory(sp_matrix):
sp_matrix = sp_matrix.tocsr()
matrix = sp_matrix.dot(sp_matrix.T)
# zero diagonal
diag = spdiags(-matrix.diagonal(), [0], *matrix.shape, format='csr')
matrix = matrix + diag
return matrix
def get_similarity_by_cosine(sp_matrix):
sp_matrix = normalize(sp_matrix.tocsr())
return get_similarity_by_x_dot_x_greedy_for_memory(sp_matrix)