how to speed up sympy-matrix of matrics calculation runtime - matrix

I've checked my code, there are no syntax error popping in the console, my matrix structures are correct, hence, mathematical operations are feasible.
if i run my code it takes forever, the console never get to print out the result. I broke the matrix to chokes of smaller code, its still taking time to compute.
from sympy import Symbol,Matrix
az = Matrix([[1, 2, 3, 4,5, 6],
[2, 3, 5, 6, 8, 6],
[4, 5, 6, 4, 8, 5],
[5, 6, 5, 8, 9, 9],
[4, 5, 6, 7, 8, 5],
[5, 6, 5, 8, 3, 9]])
fz = Matrix([[1,0,0,0,0,0],
[0,1,0,0,0,0],
[0,0,1,0,0,0],
[0,0,0,1,0,0],
[0,0,0,0,1,0],
[0,0,0,0,0,1]])
aa = Symbol('aa')
ff = Symbol('ff')
gen = Matrix([[aa, ff, 0, 0, 0, 0],
[ff,aa,ff,0,0,0],
[0,ff,aa,ff,0,0],
[0,0,ff,aa,ff,0],
[0,0,0,ff,aa,ff],
[0,0,0,0,ff,aa]])
inverse_genn = gen**-1
bz = Matrix([[2],
[3],
[3],
[4],
[5],
[5]])
bzz = Matrix([[bz],[bz],[bz],[bz],[bz],[bz]])
solution = inverse_genn*bzz
_1st_displacement = solution[0][0]
_1st_solution = _1st_displacement.subs({aa:az,ff:fz})
print('type',solution[0].shape)
print('solution',solution[0][0])
print('##'*100)
print('displacement matrix', _1st_solution)

Try make it easier to compute. Although an expression may be correct, it is not necessarily efficient for computation.
>>> eq=solution[0][0]
>>> count_ops(eq)
426
>>> s=eq.simplify()
>>> count_ops(s)
10
>>> s
2*(aa**2 - 1)/(aa**3 + aa**2 - 2*aa - 1)
>>> numer(s)/horner(denom(s))
(2*aa**2 - 2)/(aa*(aa*(aa + 1) - 2) - 1)
>>> h = _
>>> from time import time
>>> t=time(); e = h.subs(aa, az);print(time()-t)
0.0860941410065
Since horner does not apply itself to the numerator and denominator automatically, you have to use as_numer_denom to break up your simplified expression. Then you have to rebuild the expression. All together it looks like:
>>> be = zip(solution[0][0].simplify().as_numer_denom(), (1, -1))
>>> _1st_displacement = Mul(*[horner(b)**e for b,e in be])
CAUTION: this was written for an older version of SymPy. See comments below for modifications needed for newer SymPy.

Related

I need to convert a 3 column array into a square matrix?

I am looking to convert my data to a square matrix:
Say your input is a list; you can then convert it to a list of lists (i.e., a proxy to a matrix) with list comprehension:
>>> x = [0, 5, 10, 5, 0, 2, 10, 2, 0]
>>> [x[3*k:3*k+3] for k in range(3)]
[[0, 5, 10], [5, 0, 2], [10, 2, 0]]
To help you parse the line: you are building a list by iterating over k from 0 to 2, where each element will be a slice of x that starts from index 3*k and ends at index 3*k+3. Thus, your list is [x[0:3], x[3:6], x[6:9]].
That said, it's much better to use numpy for all such needs. There, you would do:
>>> import numpy as np
>>> x = np.array([0, 5, 10, 5, 0, 2, 10, 2, 0])
>>> x.reshape(3, 3)
array([[ 0, 5, 10],
[ 5, 0, 2],
[10, 2, 0]])
The reshape() function converts your 1D array into the requested 2D matrix.

Data structure for conditional probabilities with updating conditions

I have a list of lists of 4 integers. All integers in the same list are distinct, e.g.,
data = [[8, 9, 3, 0], [3, 8, 4, 9], [7, 9, 6, 4], [3, 6, 4, 8], [0, 5, 3, 7], [0, 9, 4, 2], [9, 0, 1, 5], [3, 2, 8, 6], [3, 5, 4, 0], [1, 2, 5, 9], [1, 3, 6, 5], [2, 4, 5, 7], [7, 8, 6, 3], [6, 2, 9, 8], [8, 7, 5, 4], [8, 5, 1, 3]]
Currently I have a function that receives the list above and a list of distinct integers certain. The function returns a list with the probabilities of each integer (from 0 to 9) being in a list knowing that it contains the integers in certain.
def probability(data, certain):
probs = [0] * 10
counter_total = 0
set_certain = set(certain)
for d in data:
if set_certain.issubset(d):
counter_total += 1
for i in range(10):
if i in d and i not in set_certain:
probs[i] += 1
probs = [x / counter_total for x in probs]
return probs
Initially, the list certain is empty and values are added later. Is there a data structure I can use in the start of the program so that I don't have to go through all the data again every time I append a new value to certain? The list data can be very big.

Algorithm: Factor Combinations

I'm working on the following algorithm from Leetcode:
Numbers can be regarded as product of its factors. For example,
8 = 2 x 2 x 2;
= 2 x 4.
Write a function that takes an integer n and return all possible combinations of its factors.
Note:
You may assume that n is always positive.
Factors should be greater than 1 and less than n.
Examples:
input: 1
output:
[]
input: 37
output:
[]
input: 12
output:
[
[2, 6],
[2, 2, 3],
[3, 4]
]
input: 32
output:
[
[2, 16],
[2, 2, 8],
[2, 2, 2, 4],
[2, 2, 2, 2, 2],
[2, 4, 4],
[4, 8]
]
Here's the code that I have thus far:
def get_factors(n)
factors = []
(2...n).each do |candidate|
if n % candidate == 0
factors << [candidate, (n/candidate)]
get_factors(n/candidate).each do |factor_set|
factors << [candidate] + factor_set
end
end
end
factors
end
This code works really well, but doesn't handle duplicates (e.g [3, 2, 2] will be inserted along with [2, 2, 3]). I tried using a Set with the following code,
def get_factors(n)
seen = Set.new
factors = []
(2...n).each do |candidate|
if n % candidate == 0 && !seen.include?(candidate)
factors << [candidate, (n/candidate)]
get_factors(n/candidate).each do |factor_set|
factors << [candidate] + factor_set
end
end
seen << (n/candidate)
end
factors
end
but that only works to solve some test cases and not others. I'm not sure how to go about ensuring no duplicates in an efficient way? The really inefficient way is to generate some sort of hash value for each array depending on it's elements (and not dependent on order), and while this would work, there definitely should be a better way. Any ideas?
I think always going forward is a good policy (i.e when checking, say, with 5, do not check with 2, 3, 4 etc). That way, searching for duplicates can be eliminated.
Since the algorithm already uses a lot of time, I don't see a problem sorting each answer and removing duplicates. This requires no proof to ensure it works, which the answer provided by mac does.
Code
require 'prime'
def get_factors(n)
primes, nbr = Prime.prime_division(n).transpose
powers = nbr.map { |m| (0..m).to_a }
powers.shift.
product(*powers).
map { |pows| primes.zip(pows).reduce(1) { |t,(pr,po)| t * (pr**po) } }.
sort
end
The array returned includes 1 and n (which are factors of n). If those values should be excluded, replace .sort with .sort - [1, n].
Examples
get_factors(24)
#=> [1, 2, 3, 4, 6, 8, 12, 24]
get_factors(64)
#=> [1, 2, 4, 8, 16, 32, 64]
get_factors(90)
#=> [1, 2, 3, 5, 6, 9, 10, 15, 18, 30, 45, 90]
Explanation
Consider
n = 60
The steps are as follows.
a = Prime.prime_division(30)
#=> [[2, 2], [3, 1], [5, 1]]
Ergo, the primes of 30 are 2, 3 and 5, and
60 = 2**2 * 3**1 * 5**1
See Prime::prime_division. Continuing,
primes, nbr = a.transpose
#=> [[2, 3, 5], [2, 1, 1]]
primes
#=> [2, 3, 5]
nbr
#=> [2, 1, 1]
powers = nbr.map { |m| (0..m).to_a }
#=> [[0, 1, 2], [0, 1], [0, 1]]
This means that each factor will be the product of 0, 1 or 2 2's, 0 or 1 3's and 0 or 1 5's.
b = powers.shift
#=> [0, 1, 2]
powers
#=> [[0, 1], [0, 1]]
c = b.product(*powers)
#=> [[0, 0, 0], [0, 0, 1], [0, 1, 0], [0, 1, 1], [1, 0, 0], [1, 0, 1],
# [1, 1, 0], [1, 1, 1], [2, 0, 0], [2, 0, 1], [2, 1, 0], [2, 1, 1]]
d = c.map { |pows| primes.zip(pows).reduce(1) { |t,(pr,po)| t * (pr**po) } }
#=> [1, 5, 3, 15, 2, 10, 6, 30, 4, 20, 12, 60]
d.sort
#=> [1, 5, 3, 15, 2, 10, 6, 30, 4, 20, 12, 60]
Consider now the calculation of d. The 10th element of c that is passed to the block is [2, 0, 1]. The block calculation for that element is as follows.
pows = [2, 0, 1]
e = primes.zip(pows)
#=> [[2, 2], [3, 0], [5, 1]]
e.reduce(1) { |t,(pr,po)| t * (pr**po) }
#=> 20
The reduce calculation is equivalent to
2**2 * 3**0 * 5**1
#=> 4 * 1 * 5 => 20
The calculations for the other values of c passed to the block are similar.
A simple way is to replace the last line of your method with
factors.map(&:sort).uniq
which sorts all the subarrays and then eliminates duplicates.

Split a 3D numpy array into 3D blocks

I would like to split a 3D numpy array into 3D blocks in a 'pythonic' way. I am working with image sequences that are somewhat large arrays (1000X1200X1600), so I need to split them into pieces to do my processing.
I have written functions to do this, but I am wondering if there is a native numpy way to accomplish this - numpy.split does not seem to do what I want for 3D arrays (but perhaps I don't understand its functionality)
To be clear: the code below accomplishes my task, but I am seeking a faster way to do it.
def make_blocks(x,t):
#x should be a yXmXn matrix, and t should even divides m,n
#returns a list of 3D blocks of size yXtXt
down = range(0,x.shape[1],t)
across = range(0,x.shape[2],t)
reshaped = []
for d in down:
for a in across:
reshaped.append(x[:,d:d+t,a:a+t])
return reshaped
def unmake_blocks(x,d,m,n):
#this takes a list of matrix blocks of size dXd that is m*n/d^2 long
#returns a 2D array of size mXn
rows = []
for i in range(0,int(m/d)):
rows.append(np.hstack(x[i*int(n/d):(i+1)*int(n/d)]))
return np.vstack(rows)
Here are vectorized versions of those loopy implementations using a combination of permuting dims with np.transpose and reshaping -
def make_blocks_vectorized(x,d):
p,m,n = x.shape
return x.reshape(-1,m//d,d,n//d,d).transpose(1,3,0,2,4).reshape(-1,p,d,d)
def unmake_blocks_vectorized(x,d,m,n):
return np.concatenate(x).reshape(m//d,n//d,d,d).transpose(0,2,1,3).reshape(m,n)
Sample run for make_blocks -
In [120]: x = np.random.randint(0,9,(2,4,4))
In [121]: make_blocks(x,2)
Out[121]:
[array([[[4, 7],
[8, 3]],
[[0, 5],
[3, 2]]]), array([[[5, 7],
[4, 0]],
[[7, 3],
[5, 7]]]), ... and so on.
In [122]: make_blocks_vectorized(x,2)
Out[122]:
array([[[[4, 7],
[8, 3]],
[[0, 5],
[3, 2]]],
[[[5, 7],
[4, 0]],
[[7, 3],
[5, 7]]], ... and so on.
Sample run for unmake_blocks -
In [135]: A = [np.random.randint(0,9,(3,3)) for i in range(6)]
In [136]: d = 3
In [137]: m,n = 6,9
In [138]: unmake_blocks(A,d,m,n)
Out[138]:
array([[6, 6, 7, 8, 6, 4, 5, 4, 8],
[8, 8, 3, 2, 7, 6, 8, 5, 1],
[5, 2, 2, 7, 1, 2, 3, 1, 5],
[6, 7, 8, 2, 2, 1, 6, 8, 4],
[8, 3, 0, 4, 4, 8, 8, 6, 3],
[5, 5, 4, 8, 5, 2, 2, 2, 3]])
In [139]: unmake_blocks_vectorized(A,d,m,n)
Out[139]:
array([[6, 6, 7, 8, 6, 4, 5, 4, 8],
[8, 8, 3, 2, 7, 6, 8, 5, 1],
[5, 2, 2, 7, 1, 2, 3, 1, 5],
[6, 7, 8, 2, 2, 1, 6, 8, 4],
[8, 3, 0, 4, 4, 8, 8, 6, 3],
[5, 5, 4, 8, 5, 2, 2, 2, 3]])
Alternative to make_blocks with view_as_blocks -
from skimage.util.shape import view_as_blocks
def make_blocks_vectorized_v2(x,d):
return view_as_blocks(x,(x.shape[0],d,d))
Runtime test
1) make_blocks with original and view_as_blocks based approaches -
In [213]: x = np.random.randint(0,9,(100,160,120)) # scaled down by 10
In [214]: %timeit make_blocks(x,10)
1000 loops, best of 3: 198 µs per loop
In [215]: %timeit view_as_blocks(x,(x.shape[0],10,10))
10000 loops, best of 3: 85.4 µs per loop
2) unmake_blocks with original and transpose+reshape based approaches -
In [237]: A = [np.random.randint(0,9,(10,10)) for i in range(600)]
In [238]: d = 10
In [239]: m,n = 10*20,10*30
In [240]: %timeit unmake_blocks(A,d,m,n)
100 loops, best of 3: 2.03 ms per loop
In [241]: %timeit unmake_blocks_vectorized(A,d,m,n)
1000 loops, best of 3: 511 µs per loop

Random's randint won't work in a for-loop

I'm trying to create a list with random length filled with lists of random lengths by using this code:
import random
solitaire = [None]*(random.randint(1,5))
for pile in solitaire:
number = random.randint(0, 10)
solitaire.append(number)
print(solitaire)
Easy enough I thought but when I ran this code my powershell window froze as it was expecting an input or something, I had to cancel the script with ctr+c and then got the message:
Traceback (most recent call last):
File "sparakod.py", line 254, in <module>
number = random.randint(0, 10)
File "C:\Python34\lib\random.py", line 218, in randint
return self.randrange(a, b+1)
File "C:\Python34\lib\random.py", line 170, in randrange
def randrange(self, start, stop=None, step=1, _int=int):
KeyboardInterrupt
What does this mean? Why won't the code run?
number = random.randint(0, 10)
Seems to work just fine so why won't it inside the for-loop?
you don't say anything about the content of the lists, supposing that they also contain random integers, then a possible solution could be the following:
"""
It creates a list with random length filled with lists of random lengths containing random integers
"""
import random
MIN_LIST_OF_LISTS_LENGTH = 1
MAX_LIST_OF_LISTS_LENGTH = 10
MIN_LIST_LENGTH = 1
MAX_LIST_LENGTH = 5
MIN_LIST_ELEMENT = 1
MAX_LIST_ELEMENT = 10
#This is the list which will containt all the lists
solitaire = list(range(random.randint(MIN_LIST_OF_LISTS_LENGTH,MAX_LIST_OF_LISTS_LENGTH)))
for i, pile in enumerate(solitaire):
solitaire[i] = [
random.randint(MIN_LIST_ELEMENT, MAX_LIST_ELEMENT) for x in
range(0, random.randint(MIN_LIST_LENGTH, MAX_LIST_LENGTH))
]
print(repr(solitaire))
It will generate outputs like these:
[[10, 3], [5, 2, 7, 7, 6], [5], [9, 3, 2, 6], [2, 4, 4], [4, 5, 10, 9, 10]]
or
[[5, 1], [5, 1, 1], [1, 1, 7, 3, 1]]
or
[[9, 1, 6, 7], [10, 7, 1, 7, 4]]

Resources