Related
Restricting BitVec's to the values of a list doesn't work as I expected, at least not by using in.
from z3 import *
s = Solver()
lst = [7, 11, 13, 14, 19, 21, 22, 25, 26, 28, 35, 37, 38, 41, 42, 44, 49, 50]
BV = [BitVec(f"bv1{j + 1}", 8) for j in range(11)]
lst_as_domain = [bv in lst for bv in BV]
s.add(lst_as_domain)
print(lst_as_domain) #[False, False, False, False, False, False, False, False, False, False, False]
print(s.check()) #unsat
If I use list comprehension as follows, it works.
from z3 import *
s = Solver()
lst = [7, 11, 13, 14, 19, 21, 22, 25, 26, 28, 35, 37, 38, 41, 42, 44, 49, 50]
BV = [BitVec(f"bv{j + 1}", 8) for j in range(11)]
lst_as_domain = [Or([B[k] == li for li in lst]) for k in range(11)]
s.add(lst_as_domain)
print(lst_as_domain) #[Or(bv1 == 7, bv1 == 11,... ,bv1 == 50), Or(bv2 == 7,...)..]
print(s.check()) #sat
print(s.model()) #[bv4 = 42, bv7 = 37,..., bv11 = 41]
Why doesn't the first code yield my desired restriction? How can I use in to assert a domain to variables, or is there a short command to achieve this?
Python's built-in in method does not do what you think it should do on symbolic expressions. This is a problem of the very loosely-typed nature of the z3 python bindings: Instead of doing symbolic equality, it checks for object equality, and always get False as an answer which you found when you printed lst_as_domain.
The solution is what you already found. Do not use in. For reuse purposes, I'd define a function like:
def member(x, es):
return Or([x == e for e in es])
And then use it as:
lst_as_domain = [member(bv, lst) for bv in BV]
which will do the right thing and is "close" enough to what you wanted to write in the first place.
This is a common gotcha for the Python bindings, unfortunately. While it tries to make symbolic z3 expressions look and behave like Python expressions themselves, it doesn't always work due to limitations in Python and the z3-Python API itself; which makes it error-prone to use unless you're very careful about what methods are overloaded to work on symbolic expressions and which are not.
Aside: Unfortunately there's no easy way to tell which constructs will work on symbolic values out-of-the-box. You have to study how they're implemented internally. Rule-of-thumb: Anything that Python doesn't allow you to overload, you cannot use on symbolic values. But that's not an easy test, admittedly.
a friend of mine implemented a sparse version of torch.bmm that actually works, but when I try a test, I have a runtime error (that has nothing to do with this implementation), that I don't understand. I have seen a few topics about if but couldn't find a solution. Here is the code, and the error:
if __name__ == "__main__":
tmp = torch.zeros(1).cuda()
batch_csr = BatchCSR()
sparse_bmm = SparseBMM()
i=torch.LongTensor([[0,5,8], [1,5,8], [2,5,8]])
v=torch.FloatTensor([4,3,8])
s=torch.Size([3,500,500])
indices, values, size = i,v,s
a_ = torch.sparse.FloatTensor(indices, values, size).cuda().transpose(2, 1)
batch_size, num_nodes, num_faces = a_.size()
a = a_.to_dense()
for _ in range(10):
b = torch.randn(batch_size, num_faces, 16).cuda()
torch.cuda.synchronize()
time1 = time.time()
result = torch.bmm(a, b)
torch.cuda.synchronize()
time2 = time.time()
print("{} CuBlas dense bmm".format(time2 - time1))
torch.cuda.synchronize()
time1 = time.time()
col_ind, col_ptr = batch_csr(a_.indices(), a_.size())
my_result = sparse_bmm(a_.values(), col_ind, col_ptr, a_.size(), b)
torch.cuda.synchronize()
time2 = time.time()
print("{} My sparse bmm".format(time2 - time1))
print("{} Diff".format((result-my_result).abs().max()))
And the error:
Traceback (most recent call last):
File "sparse_bmm.py", line 72, in <module>
b = torch.randn(3, 500, 16).cuda()
File "/home/bizeul/virtual_env/lib/python2.7/site-packages/torch/_utils.py", line 65, in _cuda
return new_type(self.size()).copy_(self, async)
RuntimeError: cuda runtime error (59) : device-side assert triggered at /b/wheel/pytorch-src/torch/lib/THC/generic/THCTensorCopy.c:18
When running with the command CUDA_LAUNCH_BLOCKING=1, I get the error :
/b/wheel/pytorch-src/torch/lib/THC/THCTensorIndex.cu:121: void indexAddSmallIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 1, SrcDim = 1, IdxDim = -2]: block: [0,0,0], thread: [0,0,0] Assertion `dstIndex < dstAddDimSize` failed.
THCudaCheck FAIL file=/b/wheel/pytorch-src/torch/lib/THCS/generic/THCSTensorMath.cu line=292 error=59 : device-side assert triggered
Traceback (most recent call last):
File "sparse_bmm.py", line 69, in <module>
a = a_.to_dense()
RuntimeError: cuda runtime error (59) : device-side assert triggered at /b/wheel/pytorch-src/torch/lib/THCS/generic/THCSTensorMath.cu:292
The indices that you are passing to create the sparse tensor are incorrect.
here is how it should be:
i = torch.LongTensor([[0, 1, 2], [5, 5, 5], [8, 8, 8]])
How to create a sparse tensor:
Lets take a simpler example. Lets say we want the following tensor:
0 0 0 2 0
0 0 0 0 0
0 0 0 0 20
[torch.cuda.FloatTensor of size 3x5 (GPU 0)]
As you can see, the number (2) needs to be in the (0, 3) location of the sparse tensor. And the number (20) needs to be in the (2, 4) location.
In order to create this, our index tensor should look like this
[[0 , 2],
[3 , 4]]
And, now for the code to create the above sparse tensor:
i=torch.LongTensor([[0, 2], [3, 4]])
v=torch.FloatTensor([2, 20])
s=torch.Size([3, 5])
a_ = torch.sparse.FloatTensor(indices, values, size).cuda()
More comments regarding the assert error by cuda:
Assertion 'dstIndex < dstAddDimSize' failed. tells us that, its highly likely, you've got an index out of bounds. So whenever you notice that, look for places where you might have supplied the wrong indices to any of the tensors.
I'm trying to create a list with random length filled with lists of random lengths by using this code:
import random
solitaire = [None]*(random.randint(1,5))
for pile in solitaire:
number = random.randint(0, 10)
solitaire.append(number)
print(solitaire)
Easy enough I thought but when I ran this code my powershell window froze as it was expecting an input or something, I had to cancel the script with ctr+c and then got the message:
Traceback (most recent call last):
File "sparakod.py", line 254, in <module>
number = random.randint(0, 10)
File "C:\Python34\lib\random.py", line 218, in randint
return self.randrange(a, b+1)
File "C:\Python34\lib\random.py", line 170, in randrange
def randrange(self, start, stop=None, step=1, _int=int):
KeyboardInterrupt
What does this mean? Why won't the code run?
number = random.randint(0, 10)
Seems to work just fine so why won't it inside the for-loop?
you don't say anything about the content of the lists, supposing that they also contain random integers, then a possible solution could be the following:
"""
It creates a list with random length filled with lists of random lengths containing random integers
"""
import random
MIN_LIST_OF_LISTS_LENGTH = 1
MAX_LIST_OF_LISTS_LENGTH = 10
MIN_LIST_LENGTH = 1
MAX_LIST_LENGTH = 5
MIN_LIST_ELEMENT = 1
MAX_LIST_ELEMENT = 10
#This is the list which will containt all the lists
solitaire = list(range(random.randint(MIN_LIST_OF_LISTS_LENGTH,MAX_LIST_OF_LISTS_LENGTH)))
for i, pile in enumerate(solitaire):
solitaire[i] = [
random.randint(MIN_LIST_ELEMENT, MAX_LIST_ELEMENT) for x in
range(0, random.randint(MIN_LIST_LENGTH, MAX_LIST_LENGTH))
]
print(repr(solitaire))
It will generate outputs like these:
[[10, 3], [5, 2, 7, 7, 6], [5], [9, 3, 2, 6], [2, 4, 4], [4, 5, 10, 9, 10]]
or
[[5, 1], [5, 1, 1], [1, 1, 7, 3, 1]]
or
[[9, 1, 6, 7], [10, 7, 1, 7, 4]]
I have code that looks like this:
def Z(m,n):
return CartesianProduct(IntegerRange(m),IntegerRange(n))
for v in Subsets(Z(2,2)):
print v
However, when I try to run it, I get the following error:
Traceback (most recent call last):
File "x.py", line 13, in <module>
for v in Subsets(Z(_sage_const_2 ,_sage_const_2 )):
File "/opt/sage-4.8-linux-64bit-ubuntu_10.04.3_lts-x86_64-Linux/local/lib/python2.6/site-packages/sage/combinat/subset.py", line 234, in __iter__
lset = __builtin__.list(self.s)
File "/opt/sage-4.8-linux-64bit-ubuntu_10.04.3_lts-x86_64-Linux/local/lib/python2.6/site-packages/sage/sets/set.py", line 650, in __iter__
for x in self.set():
File "/opt/sage-4.8-linux-64bit-ubuntu_10.04.3_lts-x86_64-Linux/local/lib/python2.6/site-packages/sage/sets/set.py", line 719, in set
return set(self.object())
TypeError: unhashable type: 'list'
What is the canonical way of getting the set of all subsets of an arbitrary Set?
CartesianProduct returns a list of lists, e.g.:
>>> print list(Z(2,2))
[[0, 0], [0, 1], [1, 0], [1, 1]]
But Subsets can't cope with the elements being lists (it converts its argument to a set internally, and a set in Python is implemented as a hash set, hence the error about "unhashibility"). To fix this, you should convert the internal lists to tuples:
for v in Subsets(tuple(l) for l in Z(2,2)):
print v
Note that this is using a generator expression (rather than a list comprehension) to avoid constructing an intermediate list.
(One could also use map(tuple, Z(2,2)) or import itertools iterools.imap(tuple, Z(2,2)) in place of the generator expression, but the solution given above is the most Pythonic.)
I'm trying to find a neat way of summing a list and a list of lists in the same function, so far I've got:
import operator
"""
Fails late for item = ['a', 'b']
"""
def validate(item):
try:
return sum(item) == sum(range(1, 10))
except TypeError:
return sum(reduce(operator.add, item)) == sum(range(1, 10))
"""
Not valid for item = [1,2,[3,4,5]]
"""
def validate2(item):
if isinstance(item[0], int):
return sum(item) == sum(range(1, 10))
else:
return sum(reduce(operator.add, item)) == sum(range(1, 10))
print validate([1, 2, 3, 4, 5, 6, 7, 8, 9])
print validate([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print validate2([1, 2, 3, 4, 5, 6, 7, 8, 9])
print validate2([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
...but neither of these seem quite right to me (reasons in the doc strings). What I want to know is if there is a better way of summing lists and lists of lists that doesn't require me to catch exceptions or actually analyse the list before the function decides what to do.
Obviously, I'm still expecting ['a', 'b'] to be invalid.
Perhaps you'd find it easier to flatten the list first?
def flatten(xs):
for x in xs:
try:
sub = iter(x)
except TypeError:
yield x
else:
for y in flatten(sub):
yield y
With the above, you can do this:
In [4]: fs = flatten([1,2,[3,4,[5,6],7],8,[9,10]])
In [5]: list(fs)
Out[5]: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Don't forget to describe exactly what you're trying to do. I'm assuming you mean to sum all values to a single value, and not to get eg. [[1,2],[3,4]] -> [3,7]. Here's simple recursion; five lines of code if you skip the tests:
def sums(it):
"""
>>> sums(1)
1
>>> sums([1,2,3])
6
>>> sums([1,2,3,[4,5]])
15
>>> sums(['a','b'])
Traceback (most recent call last):
...
TypeError: unsupported operand type(s) for +: 'int' and 'str'
"""
if getattr(it, "__iter__", None):
return sum(map(sums, it))
else:
return it
if __name__ == "__main__":
import doctest
doctest.testmod()
The external numpy module has many operations (including sum()) which work similarly on scalars, vectors, matrices and even higher-dimensional arrays...
Note however that it doesn't work on mixed lists like [1,2,[3,4,5]], only square matrices! So it doesn't exactly answer your question.