I would like to get at least one solution for one t for this:
row_sum(Y) = x.t + row_sum(B)
Y and B are matrix(n, m) and x a vector(n).
I know B, I know row_sum(Y) but not x.
I would like to get the Y and x for some t value.
Y variate linearly with t but x is constant vector.
How I can get create an equation with the sum of row on Y and B to get at the same times x and Y?
To get the sum of each row in a matrix, multiply it by a column vector of all ones. In general, if Z is a matrix, then
Z * ones(Z.shape[1], 1)
returns such a sum.
Here is a complete example of such manipulations, using the notation of your example.
from sympy import *
t = symbols('t')
Y = Matrix([[1+4*t, 2-t], [3-5*t, 4+t]])
x = symarray('x', (2,))
B = Matrix([[5, 4], [3, 2]])
solve(Y*ones(2, 1) - x*t - B*ones(2, 1))
The output is [{x_0: -3*x_1 - 9, t: 2/(x_1 + 4)}]. The answer is not unique since there are only 2 equations with three unknowns t, x_0, x_1.
Related
I have the following problem: I have a list of items
[O_0,..., O_n]
, where each item is represented by a binary power (o_0 represented by 2^0, ..., o_n by 2^n). I have constructed a list of combinations of these elements (each combination is represented by the sum of the binary representations of the items). For example I have
combs = [3, 9, 15, ......].
Given a new combination of these items say C_1, I would like to test if any of the elements of combs is included in C_1. An efficient and fast way that I thought of was to compute for each element c_i from combs, test if c_i & C_1 == c_i which means that it is true for this element. It is fast since I am doing a bitiwise and.
My problem is that instead of having 1 element C_1, I have a very big number of them C_1, ..., C_k, and I have to test for each one of them the above condition. SO I was wondering if there are any faster ways than the one I mentioned to test the condition of all of the elements (this is actually the same problem as testing if a set is a subset of another, which is why i chose binary representation from the beginning to transform the problem into a binary one).
My understanding of the problem: given a collection of k sets Y and a collection of m sets X, we would like to find a subset S of Y such that for all y in S, there exists x in X s.t. x is a subset of y.
I will assume that sets are represented by n-vectors of zeros and ones denoting inclusion. Here is the setup:
import pandas as pd # for drop_duplicates & benchmarks
import numpy as np
np.random.seed(0)
n = 100 # 100 "atomic" elements
m = 1000 # small sets
k = 1000 # large sets
X = pd.DataFrame(np.random.randint(0, 2, size=(m, n))).drop_duplicates().values
Y = pd.DataFrame(np.random.randint(0, 2, size=(k, n))).drop_duplicates().values
# For each row y in Y, we would like to check if there exists a row x in X
# s.t. x represents a subset of Y
def naive(Y, X):
# O(k^2 + m^2)
for i, y in enumerate(Y):
for x in X:
found_subset = False
if (x <= y).all():
yield i
found_subset = True
if found_subset:
break
def naive_as_array(Y, X):
return np.array(list(naive(Y, X)))
The naive function iterates over all pairs of sets that may satisfy the inclusion relation and short-circuits whenever appropriate. The runtime is O(m + k) where m = len(combs).
As an alternative, we can consider the following recursive algorithm processing each element (from 1 to n) at a time:
def contains(Y, X):
"""
Y : k x n indicator array specifying sets
X : m x n indicator array specifying sets
output: subset Z of [0..k-1] s.t. i in Z iff there exists x in X s.t.
# x is a subset of Y[i]. Z is represented by a 1D numpy array.
"""
k, n = Y.shape
assert Y.shape[1] == X.shape[1]
detected = np.zeros(k, dtype=np.bool)
inds = np.arange(k)
# utility to account for sets that already have a subset detected
def account_for_detected(Y, inds):
mask = ~detected[inds]
Y = Y[mask]
inds = inds[mask]
return Y, inds
# inductively reduce Y.shape[1] (==X.shape[1])
def f(Y, X, inds):
if Y.shape[0] == 0 or X.shape[0] == 0:
# collection Y is empty; inculsions are impossible
return
# avoid redundant comparisons by dropping sets y in Y
# if it is already known that y contains some element of X
Y, inds = account_for_detected(Y, inds)
if Y.shape[1] == 1:
# Y and X are collections of singletons
Y = np.ravel(Y)
X = np.ravel(X)
X_vals = np.zeros(2, dtype=np.int)
X_vals[X] = 1
if X_vals[0] > 0:
detected[inds] = True
elif X_vals[1] > 0:
detected[inds[Y==1]] = True
return
else:
# make a recursive call
Ymask = Y[:,0] == 0
Xmask = X[:,0] == 0
# if x in X is a subset of y in Y, x[0] <= y[0]
f(Y[Ymask,1:], X[Xmask,1:], inds[Ymask])
# by now, detected is updated in the outer scope
# process the remaining Y's
f(Y[~Ymask,1:], X[:,1:], inds[~Ymask])
# done
# make call at root:
f(Y, X, inds)
# return indices
return np.where(detected)[0]
At step d between 1 and n, we split sets Y into Y0 and Y1 where Y0 contains sets in Y that do not contain element d, and Y1 contains sets in Y that do contain element d. Similarly, we define X0 and X1. A key observation is that sets in X1 cannot occur as subsets of sets in Y0. Therefore, we can reduce the number of comparisons in the recursive call.
Timings:
%timeit contains(Y, X)
%timeit naive_as_array(Y, X)
10 loops, best of 3: 185 ms per loop
1 loop, best of 3: 2.39 s per loop
I have a logical matrix X of n points, where X(i, j) == 1 if points i and j are neighbors and 0 otherwise.
I would like to create a cell array Y with each entry Y{i} (i from 1 to n) containing an array with the indeces of point i's neighbors.
In other words, I would like to vectorize the following:
n = 10;
X = (rand(n, n) < 0.5);
Y = cell(1, 10);
for i = 1:10
[Y{i}] = find(X(i, :));
end
As one approach you can use accumarray -
[R,C] = find(X.') %//'
Y = accumarray(C(:),R(:),[],#(x) {x})
If you need each cell to be a row vector, you need to add one transpose there with x, like so -
Y = accumarray(C(:),R(:),[],#(x) {x.'})
As another approach, you can also use arrayfun, but I don't think this would be a vectorized solution -
Y = arrayfun(#(n) R(C==n),1:max(C),'Uni',0)
If you don't care about the order of elements in each cell, you can avoid the transpose of X to get R and C like so -
[R,C] = find(X)
Then, interchange the positions of R and C with the accumarray and arrayfun based approaches as listed earlier.
Here's some more voodoo:
Y = mat2cell(nonzeros(bsxfun(#times, X, 1:size(X,1)).').', 1, sum(X,2));
The most important function here is bsxfun. To see how the code works, I suggest you observe partial results from innermost outwards: first bsxfun(#times, X, 1:size(X,1)).', then nonzeros(...), etc.
I have a data set of the form:
[9.1 5.6 7.4] => 8.5, [4.1 4.4 5.2] => 4.9, ... , x => y(x)
So x is a real vector of three elements and y is a scalar function.
I'm assuming a weighted average model of this data:
y(x) = (a * x[0] + b * x[1] + c * x[2]) / (a+b+c) + E(x)
where E is an unknown random error term.
I need an algorithm to find a,b,c, that minimizes total sum square error:
error = sum over all x of { E(x)^2 }
for a given data set.
Assume that the weights are normalized to sum to 1 (which happily is without loss of generality), then we can re-cast the problem with c = 1 - a - b, so we are actually solving for a and b.
With this we can write
error(a,b) = sum over all x { a x[0] + b x[1] + (1 - a - b) x[2] - y(x) }^2
Now it's just a question of taking the partial derivatives d_error/da and d_error/db and setting them to zero to find the minimum.
With some fiddling, you get a system of two equations in a and b.
C(X[0],X[0],X[2]) a + C(X[0],X[1],X[2]) b = C(X[0],Y,X[2])
C(X[1],X[0],X[2]) a + C(X[1],X[1],X[2]) b = C(X[1],Y,X[2])
The meaning of X[i] is the vector of all i'th components from the dataset x values.
The meaning of Y is the vector of all y(x) values.
The coefficient function C has the following meaning:
C(p, q, r) = sum over i { p[i] ( q[i] - r[i] ) }
I'll omit how to solve the 2x2 system unless this is a problem.
If we plug in the two-element data set you gave, we should get precise coefficients because you can always approximate two points perfectly with a line. So for example the first equation coefficients are:
C(X[0],X[0],X[2]) = 9.1(9.1 - 7.4) + 4.1(4.1 - 5.2) = 10.96
C(X[0],X[1],X[2]) = -19.66
C(X[0],Y,X[2]) = 8.78
Similarly for the second equation: 4.68 -13.6 4.84
Solving the 2x2 system produces: a = 0.42515, b = -0.20958. Therefore c = 0.78443.
Note that in this problem a negative coefficient results. There is nothing to guarantee they'll be positive, though "real" data sets may produce this result.
Indeed if you compute weighted averages with these coefficients, they are 8.5 and 4.9.
For fun I also tried this data set:
X[0] X[1] X[2] Y
0.018056028 9.70442075 9.368093544 6.360312244
8.138752835 5.181373099 3.824747424 5.423581239
6.296398214 4.74405298 9.837741509 7.714662742
5.177385358 1.241610571 5.028388255 4.491743107
4.251033792 8.261317658 7.415111851 6.430957844
4.720645386 1.0721718 2.187147908 2.815078796
1.941872069 1.108191586 6.24591771 3.994268819
4.220448549 9.931055481 4.435085917 5.233711923
9.398867623 2.799376317 7.982096264 7.612485261
4.971020963 1.578519218 0.462459906 2.248086465
I generated the Y values with 1/3 x[0] + 1/6 x[1] + 1/2 x[2] + E where E is a random number in [-0.1..+0.1]. If the algorithm is working correctly we'd expect to get roughly a = 1/3 and b = 1/6 from this result. Indeed we get a = .3472 and b = .1845.
OP has now said that his actual data are larger than 3-vectors. This method generalizes without much trouble. If the vectors are of length n, then you get an n-1 x n-1 system to solve.
I have number A (build from digits 0,1,2,3). I want to find the smallest x and y, that if I do x^y i got the biggest number smaller than A
x^y <= A x^y is maximal
Plus x and y must not be decimal numbers, only "integers"
For example:
A = 7 => x^y = 2^2
A = 28 => x^y = 3^3
A = 33 => x^y = 2^5
etc
Edit:
As izomorphius suggested in comment, it will have always solution for x = A and y = 1. But that is not desirable result. I want x and y to be as much close numbers, as it can be.
A naive solution could be:
The "closest yet not higher" number to A by doing a^y for some constant a is:
afloor(log_a(A)) [where log_a(A) is the logarithm with base a of A, which can be calculated as log(A)/log(a) in most programming languages]
By iterating all as in range [2,A) you can find this number.
This solution is O(A * f(A)) where f(A) is your pow/log complexity
P.S. If you want your exponent (y) be larger then 1, you can simply iterate in range [2,sqrt(A)] - it will reduce the time complexity to O(sqrt(A) * f(A)) - and will get you only numbers with an exponent larger then 1.
It is not clear what you are asking, but I will try to guess.
We first solve the equation z^z = a for a real number z. Let u and v be z rounded down and up, respectively. Among the three candidates (u,u), (v,u), (u,v) we choose the largest one that does not exceed a.
Example: Consder the case a = 2000. We solve z^z = 2000 by numerical methods (see below) to get an approximate solution z = 4.8278228255818725. We round down an up to obtain u = 4 and v = 5. We now have three candidates, 4^4 = 256, 4^5 = 1023 and 5^4 = 625. They are all smaller than 2000, so we take the one that gives the largest answer, which is x = 4, y = 5.
Here is Python code. The function solve_approx does what you want. It works well for a >= 3. I am sure you can cope with the cases a = 1 and a = 2 by yourself.
import math
def solve(a):
""""Solve the equation x^x = a using Newton's method"""
x = math.log(a) / math.log(math.log(a)) # Initial estimate
while abs (x ** x - a) > 0.1:
x = x - (x ** x - a) / (x ** x * (1 + math.log(x)))
return x
def solve_approx(a):
""""Find two integer numbers x and y such that x^y is smaller than
a but as close to it as possible, and try to make x and y as equal
as possible."""
# First we solve exactly to find z such that z^z = a
z = solve(a)
# We round z up and down
u = math.floor(z)
v = math.ceil(z)
# We now have three possible candidates to choose from:
# u ** zdwon, v ** u, u ** v
candidates = [(u, u), (v, u), (u, v)]
# We filter out those that are too big:
candidates = [(x,y) for (x,y) in candidates if x ** y <= a]
# And we select the one that gives the largest result
candidates.sort(key=(lambda key: key[0] ** key[1]))
return candidates[-1]
Here is a little demo:
>>> solve_approx(5)
solve_approx(5)
(2, 2)
>>> solve_approx(100)
solve_approx(100)
(3, 4)
>>> solve_approx(200)
solve_approx(200)
(3, 4)
>>> solve_approx(1000)
solve_approx(1000)
(5, 4)
>>> solve_approx(1000000)
solve_approx(1000000)
(7, 7)
I have two problems. I have to calculate two equations:
X = A - inv(B) * Y * inv(B)
and
X = Y + A' * inv(B) * A
where, A, B and Y are known p*p matrices (p can be small or large, depends the situation). Matrices are quite dense, without any structure (except B being non-singular of course).
Is it possible to solve X in those equations without inverting the matrix B? I have to calculate these equations n times, n being hundreds or thousands, and all the matrices change over time.
Thank you very much.
If you can express your updates to your matrix B in the following terms:
Bnew = B + u*s*v
then you can express an update to inv(B) explicitly using the Sherman-Morrison-Woodbury formula:
inv(B + u*s*v) = inv(B) - inv(B)*u*inv(s + v*inv(B)*u)*v*inv(B)
If u and v are vectors (column and row, respectively) and s is scalar, then this expression simplifies:
inv(B + u*s*v) = inv(B) - inv(B)*u*v*inv(B)/(s + v*inv(B)*u)
You would only have to calculate inv(B) once and then update it when it changes with no additional inversions.
It may be preferable not to calculate the full inverse, just simple "matrix divisions" on y and (ynew - y) or a and (anew - a) depending on the size of "n" with respect to "p" in your problem.
Memo-ize inv(B), i.e. only invert B when it changes, and keep the inverse around.
If changes to B are small, possibly you could use a delta-approximation.