I am trying to randomly generate integer partitions (N into M parts) in pytorch with a minimum partition size of 1.
For example, (3, 1, 1) and (4, 1, 0) are both partitions of 5 into 3 parts but (4, 1, 0)'s minimum partition size is 0 so this should not be allowed
I would like to use this to generate my dataset on demand, so would nice if there was a pytorch (parrallel/gpgpu) solution.
See other questions about generating integer partitions:
Generate restricted weak integer compositions (or partitions) of an integer n into k parts in Python
Elegant Python code for Integer Partitioning
Here's a solution that works in limited cases, M hast to divide N, 2 has to divide M, and the maximium is limited, but this is the behaviour I wanted.
You start of with the equal partition then calculate a delta that sums to zero...
M = 4
N = 16
MINIMUM = 1
assert N % M == 0
assert M % 2 == 0
avg = N // M
equal_partition = torch.full((M,), avg)
half_delta = torch.randint(-(avg-MINIMUM), avg-MINIMUM, (M // 2,))
delta = torch.cat((half_delta, -half_delta), dim=-1)[torch.randperm(M)]
partition = equal_partition + delta
print(partition, partition.sum())
Related
How it would be possible to find/guess the combination of N numbers such as 5 or 7 or whatever that gives a final number R?
For example, determine N = 5 and R = 15
The one possible result/guess that the 5 numbers in which their summation give 15 would be {1,2,3,4,5}
To get n floating point numbers that total a target r:
Fill an array of size n with random numbers in (0,1).
Find the total of the array. Call this T.
Multiply every element in the array by r/T.
-edit (thanks #ruakh) -
To account for floating point error, total the array again, calculate the delta from target, and update the array to correct for it however you like. This will be a very small number. It's probably fine to update the first element of the array, but if you want you could spread it across the full array.
This can be solved by backtracking, this is really slow, because it looks for all combinations, but if you only want one combination use K-1 0' and N, this sum = N
n = 15
k = 2
array = [0] * k
def comb (len, sum):
if len == k - 1:
array[len] = sum
print(array)
return
for i in range(0, n + 1):
if sum - i >= 0:
array[len] = i
comb(len + 1, sum - i)
comb(0, n)
Python code
import random
N = 5
R = 7
result = random.sample(range(0, int(R)), N-1)
result.append(R - sum(result))
this is the actual question
however, it simplifies to
Find all SEMPIPRIMES (numbers which are products of 2 DISTINCT prime factors e.g. 6 (2*3) in range L to R
there will be multiple queries for L and R
we cant precompute semiprimes as N is large
BUT we can store primes as they are only upto 10^6 as per the question
Now, assume i have all primes by sieve of eratostheneses
i need all possible pairs of primes with product between L to R
OR THE QUESTION SIMPLIFIES TO GIVEN A SORTED ARRAY.
FIND ALL POSSIBLE PAIRS WITH PRODUCTS BETWEEN L AND R INCLUSIVE
i am including the part of code in the editorial which does this..
for(int i=0; i<cnt and ar[i]<=r; i++)
{
int lower = L/ar[i];
if(L%ar[i]>0)
lower++;
lower = max(lower, ar[i]+1);
int upper = R/ar[i];
if(upper<lower)
continue;
ans += upper_bound(ar.begin(),ar.end(),upper)-
lower_bound(ar.begin(),ar.end(),lower);
}
Here's one approach, this may not be faster but it seems reasonable.
The number of primes below 10^8 is around 5*10^6.
reference: https://en.wikipedia.org/wiki/Prime-counting_function
But we may not have to keep all the primes, as it would be rather inefficient. We can keep the Semiprimes only.
There's already the generative process for Semprimes. Each Semiprime is a product of 2 distinct prime factors.
So, we can keep an array which will store all the semiprimes, as there will be at most 10^5 semiprimes in the range, we can sort that array. For each query, we will just binary search on the array to find the number of elements in the range.
So, how to save the semiprimes?
We can slightly modify the sieve of Eratosthenes to generate semiprimes.
The idea is we will keep a countDivision array which will store the number of divisor for each integer in range. We only consider a integer semiprime is countDivision index value is 2 for that integer (2 divisors).
def createSemiPrimeSieve(n):
v = [0 for i in range(n + 1)]
# This array will initially store the indexes
# After performing below operations if any
# element of array becomes 1 this means
# that the given index is a semi-prime number
# Storing indices in each element of vector
for i in range(1, n + 1):
v[i] = i
countDivision = [0 for i in range(n + 1)]
for i in range(n + 1):
countDivision[i] = 2
# This array will initially be initialized by 2 and
# will just count the divisions of a number
# As a semiprime number has only 2 prime factors
# which means after dividing by the 2 prime numbers
# if the index countDivision[x] = 0 and v[x] = 1
# this means that x is a semiprime number
# If number a is prime then its
# countDivision[a] = 2 and v[a] = a
for i in range(2, n + 1, 1):
# If v[i] != i this means that it is
# not a prime number as it contains
# a divisor which has already divided it
# same reason if countDivision[i] != 2
if (v[i] == i and countDivision[i] == 2):
# j goes for each factor of i
for j in range(2 * i, n + 1, i):
if (countDivision[j] > 0):
# Dividing the number by i
# and storing the dividend
v[j] = int(v[j] / i)
# Decreasing the countDivision
countDivision[j] -= 1
# A new vector to store all Semi Primes
res = []
for i in range(2, n + 1, 1):
# If a number becomes one and
# its countDivision becomes 0
# it means the number has
# two prime divisors
if (v[i] == 1 and countDivision[i] == 0):
res.append(i)
return res
Credit: https://www.geeksforgeeks.org/print-all-semi-prime-numbers-less-than-or-equal-to-n/
But, generating has same complexity as sieve which is O(nloglogn). If (R-L) is < 10^5 this approach will pass. But as (R-L) can be as big as 10^8 it's not feasible.
Another approach is to count instead of generating.
Let's work on an example.
2 10
Now, let's say, we know all the primes up to 10^6 (as p and q can't be more than 10^6).
primes = [2, 3, 5, 7, 11, ...]
The number of primes below 10^6 is less than 10^5 (so we can store
them in an array) and the time complexity is also manageable.
Now, we can scan our primes array to count the contribution for each prime to generate semiprimes in range (L, R).
First, let's start with 2, how many semiprimes we'll generate with the help of 2.
Let's look at primes = [2, 3, 5, 7, 11, ..]
We can't choose 2, because 2 and 2 are not different (p, q must be different). But, 3 is in, as 2*3 <= 10, so is 5, 2*5 2*5 <= 10.
How to count this?
We will take the lower bound as 2//2 (L//primes[i]) but we have to make sure, we don't consider the current prime again as (p and q must be different) so we take the max of L//primes[i] and primes[i]+1.
For 2, our start number is 3 (because of 2+1 = 3, we can't start at 1 or 2 as if we consider 2 then we'll calculate cases like 2*2 = 4 which is not valid). Our end number is 10//2 = 5, how many numbers are there within range 3, 5 in primes. It's 2 and can be found via a simple binary search.
Rest is easy, we'll have to binary search how many primes are there in range (max(L//primes[i], primes[i]+1), R//primes[i]).
This has complexity pre-processing time complexity of O(10^6*loglog(10^6)) + O(10^6*log(10^6)) and O(log(10^6)) per query.
The question is, given a number D and a sequence of numbers with amount N, find the amount of the combinations of three numbers that have a highest difference value within it that does not exceed the value D. For example:
D = 3, N = 4
Sequence of numbers: 1 2 3 4
Possible combinations: 1 2 3 (3-1 = 2 <= D), 1 2 4 (4 - 1 = 3 <= D), 1 3 4, 2 3 4.
Output: 4
What I've done: link
Well my concept is: iterate through the whole sequence of numbers and find the smallest number that exceeds the D value when subtracted to the current compared number. Then, find the combinations between those two numbers with the currently compared number being a fixed value (which means combination of n [numbers between the two numbers] taken 2). If even the biggest number in the sequence subtracted with the currently compared number does not exceed D, then use a combination of the whole elements taken 3.
N can be as big as 10^5 with the smallest being 1 and D can be as big as 10^9 with the smallest being 1 too.
Problem with my algorithm: overflow occurs when I do a combination of the 1st element and 10^5th element. How can I fix this? Is there a way to calculate that large amount of combination without actually doing the factorials?
EDIT:
Overflow occurs when worst case happens: currently compared number is still in index 0 while all other numbers, when subtracted with the currently compared number, is still smaller than D. For example, the value of number at index 0 is 1, the value of number at index 10^5 is 10^5 + 1 and D is 10^9. Then, my algorithm will attempt to calculate the factorial of 10^5 - 0 which then overflows. The factorial will be used to calculate the combination of 10^5 taken 3.
When you seek for items in value range D in sorted list, and get index difference M, then you should calculate C(M,3).
But for such combination number you don't need to use huge factorials:
C(M,3) = M! / (6 * (M-3)!) = M * (M-1) * (M-2) / 6
To diminish intermediate results even more:
A = (M - 1) * (M - 2) / 2
A = (A * M) / 3
You didn't add the C++ tag to your question, so let me write the answer in Python 3 (it should be easy to translate it to C++):
N = int(input("N = "))
D = int(input("D = "))
v = [int(input("v[{}] = ".format(i))) for i in range (0, N)]
count = 0
i, j = 0, 1
while j + 1 < N:
j += 1
while v[j] - v[i] > D:
i += 1
d = j - i
if d >= 2:
count += (d - 1) * d // 2 # // is the integer division
print(count)
The idea is to move up the upper index of the triples j, while dragging the lower index i at the greatest distance j-i=d where v[j]-v[i]<=D. For each i-j pair, there are 1+2+3+...+d-1 possible triples keeping j fixed, i.e., (d-1)*d/2.
I have suppose two places: -,- .Each of these place has a max limit. Such as first place has max limit of 3 and 2nd place has max limit of 7.
I have other 2 numbers which is totalmaxlimit and other is totalminlimit.
Ex; totalmaxlimit = 6
totalminlimit = 3
I want to write a code where I can fill above two places with all possible permutation and combinations such that the sum of three places is greater than equal to 3 and less than equal to 6.
Example:
3 0
3 1
2 0
2 1
2 4
Also,
2 6 will be wrong result because sum is greater than totalmaxlimit.
4 2 is also wrong as first place has max limit of 3.
Code in any language is fine. Thanks in advance.
Let's assume that:
1) The place is given by A, B coordinates
2) You have a totalMin (m) and a totalMax (M)
3) The rules are that A, B, and A+B should be >= m and <=M
4) The amount of values is given by the equation M-m. (e.g, if M =10 and m=0, we will have 10 valid values).
You can get a permutation by using the formula P = n! / (n-k)! where N is you number of values, and k is the valid numbers you can have.
So, for example, if m = 0, M = 6:
Permutations = (0,6) , (1,5), (2,4), etc...
Basically you have the sum of (X_n, X_M) where, when n grows, M decreases. M = M - n
I hope this helps for now, but I can provide a more 'profissional' formula if you like. The language could be python because I think it would be easier. But first you need to get the algorithm, passing it to code is trivial.
Here's the code:
def permutations(min, max):
# init variables
m = min
M = max
results = []
# main loop
for i in range(min, max):
for j in range(min, max):
if ((i + j) >= min and (i+j) <= max ):
results.append([i, j])
print(results)
Consider a binary sequence b of length N. Initially, all the bits are set to 0. We define a flip operation with 2 arguments, flip(L,R), such that:
All bits with indices between L and R are "flipped", meaning a bit with value 1 becomes a bit with value 0 and vice-versa. More exactly, for all i in range [L,R]: b[i] = !b[i].
Nothing happens to bits outside the specified range.
You are asked to determine the number of possible different sequences that can be obtained using exactly K flip operations modulo an arbitrary given number, let's call it MOD.
More specifically, each test contains on the first line a number T, the number of queries to be given. Then there are T queries, each one being of the form N, K, MOD with the meaning from above.
1 ≤ N, K ≤ 300 000
T ≤ 250
2 ≤ MOD ≤ 1 000 000 007
Sum of all N-s in a test is ≤ 600 000
time limit: 2 seconds
memory limit: 65536 kbytes
Example :
Input :
1
2 1 1000
Output :
3
Explanation :
There is a single query. The initial sequence is 00. We can do the following operations :
flip(1,1) ⇒ 10
flip(2,2) ⇒ 01
flip(1,2) ⇒ 11
So there are 3 possible sequences that can be generated using exactly 1 flip.
Some quick observations that I've made, although I'm not sure they are totally correct :
If K is big enough, that is if we have a big enough number of flips at our disposal, we should be able to obtain 2n sequences.
If K=1, then the result we're looking for is N(N+1)/2. It's also C(n,1)+C(n,2), where C is the binomial coefficient.
Currently trying a brute force approach to see if I can spot a rule of some kind. I think this is a sum of some binomial coefficients, but I'm not sure.
I've also come across a somewhat simpler variant of this problem, where the flip operation only flips a single specified bit. In that case, the result is
C(n,k)+C(n,k-2)+C(n,k-4)+...+C(n,(1 or 0)). Of course, there's the special case where k > n, but it's not a huge difference. Anyway, it's pretty easy to understand why that happens.I guess it's worth noting.
Here are a few ideas:
We may assume that no flip operation occurs twice (otherwise, we can assume that it did not happen). It does affect the number of operations, but I'll talk about it later.
We may assume that no two segments intersect. Indeed, if L1 < L2 < R1 < R2, we can just do the (L1, L2 - 1) and (R1 + 1, R2) flips instead. The case when one segment is inside the other is handled similarly.
We may also assume that no two segments touch each other. Otherwise, we can glue them together and reduce the number of operations.
These observations give the following formula for the number of different sequences one can obtain by flipping exactly k segments without "redundant" flips: C(n + 1, 2 * k) (we choose 2 * k ends of segments. They are always different. The left end is exclusive).
If we had perform no more than K flips, the answer would be sum for k = 0...K of C(n + 1, 2 * k)
Intuitively, it seems that its possible to transform the sequence of no more than K flips into a sequence of exactly K flips (for instance, we can flip the same segment two more times and add 2 operations. We can also split a segment of more than two elements into two segments and add one operation).
By running the brute force search (I know that it's not a real proof, but looks correct combined with the observations mentioned above) that the answer this sum minus 1 if n or k is equal to 1 and exactly the sum otherwise.
That is, the result is C(n + 1, 0) + C(n + 1, 2) + ... + C(n + 1, 2 * K) - d, where d = 1 if n = 1 or k = 1 and 0 otherwise.
Here is code I used to look for patterns running a brute force search and to verify that the formula is correct for small n and k:
reachable = set()
was = set()
def other(c):
"""
returns '1' if c == '0' and '0' otherwise
"""
return '0' if c == '1' else '1'
def flipped(s, l, r):
"""
Flips the [l, r] segment of the string s and returns the result
"""
res = s[:l]
for i in range(l, r + 1):
res += other(s[i])
res += s[r + 1:]
return res
def go(xs, k):
"""
Exhaustive search. was is used to speed up the search to avoid checking the
same string with the same number of remaining operations twice.
"""
p = (xs, k)
if p in was:
return
was.add(p)
if k == 0:
reachable.add(xs)
return
for l in range(len(xs)):
for r in range(l, len(xs)):
go(flipped(xs, l, r), k - 1)
def calc_naive(n, k):
"""
Counts the number of reachable sequences by running an exhaustive search
"""
xs = '0' * n
global reachable
global was
was = set()
reachable = set()
go(xs, k)
return len(reachable)
def fact(n):
return 1 if n == 0 else n * fact(n - 1)
def cnk(n, k):
if k > n:
return 0
return fact(n) // fact(k) // fact(n - k)
def solve(n, k):
"""
Uses the formula shown above to compute the answer
"""
res = 0
for i in range(k + 1):
res += cnk(n + 1, 2 * i)
if k == 1 or n == 1:
res -= 1
return res
if __name__ == '__main__':
# Checks that the formula gives the right answer for small values of n and k
for n in range(1, 11):
for k in range(1, 11):
assert calc_naive(n, k) == solve(n, k)
This solution is much better than the exhaustive search. For instance, it can run in O(N * K) time per test case if we compute the coefficients using Pascal's triangle. Unfortunately, it is not fast enough. I know how to solve it more efficiently for prime MOD (using Lucas' theorem), but O do not have a solution in general case.
Multiplicative modular inverses can't solve this problem immediately as k! or (n - k)! may not have an inverse modulo MOD.
Note: I assumed that C(n, m) is defined for all non-negative n and m and is equal to 0 if n < m.
I think I know how to solve it for an arbitrary MOD now.
Let's factorize the MOD into prime factors p1^a1 * p2^a2 * ... * pn^an. Now can solve this problem for each prime factor independently and combine the result using the Chinese remainder theorem.
Let's fix a prime p. Let's assume that p^a|MOD (that is, we need to get the result modulo p^a). We can precompute all p-free parts of the factorial and the maximum power of p that divides the factorial for all 0 <= n <= N in linear time using something like this:
powers = [0] * (N + 1)
p_free = [i for i in range(N + 1)]
p_free[0] = 1
for cur_p in powers of p <= N:
i = cur_p
while i < N:
powers[i] += 1
p_free[i] /= p
i += cur_p
Now the p-free part of the factorial is the product of p_free[i] for all i <= n and the power of p that divides n! is the prefix sum of the powers.
Now we can divide two factorials: the p-free part is coprime with p^a so it always has an inverse. The powers of p are just subtracted.
We're almost there. One more observation: we can precompute the inverses of p-free parts in linear time. Let's compute the inverse for the p-free part of N! using Euclid's algorithm. Now we can iterate over all i from N to 0. The inverse of the p-free part of i! is the inverse for i + 1 times p_free[i] (it's easy to prove it if we rewrite the inverse of the p-free part as a product using the fact that elements coprime with p^a form an abelian group under multiplication).
This algorithm runs in O(N * number_of_prime_factors + the time to solve the system using the Chinese remainder theorem + sqrt(MOD)) time per test case. Now it looks good enough.
You're on a good path with binomial-coefficients already. There are several factors to consider:
Think of your number as a binary-string of length n. Now we can create another array counting the number of times a bit will be flipped:
[0, 1, 0, 0, 1] number
[a, b, c, d, e] number of flips.
But even numbers of flips all lead to the same result and so do all odd numbers of flips. So basically the relevant part of the distribution can be represented %2
Logical next question: How many different combinations of even and odd values are available. We'll take care of the ordering later on, for now just assume the flipping-array is ordered descending for simplicity. We start of with k as the only flipping-number in the array. Now we want to add a flip. Since the whole flipping-array is used %2, we need to remove two from the value of k to achieve this and insert them into the array separately. E.g.:
[5, 0, 0, 0] mod 2 [1, 0, 0, 0]
[3, 1, 1, 0] [1, 1, 1, 0]
[4, 1, 0, 0] [0, 1, 0, 0]
As the last example shows (remember we're operating modulo 2 in the final result), moving a single 1 doesn't change the number of flips in the final outcome. Thus we always have to flip an even number bits in the flipping-array. If k is even, so will the number of flipped bits be and same applies vice versa, no matter what the value of n is.
So now the question is of course how many different ways of filling the array are available? For simplicity we'll start with mod 2 right away.
Obviously we start with 1 flipped bit, if k is odd, otherwise with 1. And we always add 2 flipped bits. We can continue with this until we either have flipped all n bits (or at least as many as we can flip)
v = (k % 2 == n % 2) ? n : n - 1
or we can't spread k further over the array.
v = k
Putting this together:
noOfAvailableFlips:
if k < n:
return k
else:
return (k % 2 == n % 2) ? n : n - 1
So far so well, there are always v / 2 flipping-arrays (mod 2) that differ by the number of flipped bits. Now we come to the next part permuting these arrays. This is just a simple permutation-function (permutation with repetition to be precise):
flipArrayNo(flippedbits):
return factorial(n) / (factorial(flippedbits) * factorial(n - flippedbits)
Putting it all together:
solutionsByFlipping(n, k):
res = 0
for i in [k % 2, noOfAvailableFlips(), step=2]:
res += flipArrayNo(i)
return res
This also shows that for sufficiently large numbers we can't obtain 2^n sequences for the simply reason that we can not arrange operations as we please. The number of flips that actually affect the outcome will always be either even or odd depending upon k. There's no way around this. The best result one can get is 2^(n-1) sequences.
For completeness, here's a dynamic program. It can deal easily with arbitrary modulo since it is based on sums, but unfortunately I haven't found a way to speed it beyond O(n * k).
Let a[n][k] be the number of binary strings of length n with k non-adjacent blocks of contiguous 1s that end in 1. Let b[n][k] be the number of binary strings of length n with k non-adjacent blocks of contiguous 1s that end in 0.
Then:
# we can append 1 to any arrangement of k non-adjacent blocks of contiguous 1's
# that ends in 1, or to any arrangement of (k-1) non-adjacent blocks of contiguous
# 1's that ends in 0:
a[n][k] = a[n - 1][k] + b[n - 1][k - 1]
# we can append 0 to any arrangement of k non-adjacent blocks of contiguous 1's
# that ends in either 0 or 1:
b[n][k] = b[n - 1][k] + a[n - 1][k]
# complete answer would be sum (a[n][i] + b[n][i]) for i = 0 to k
I wonder if the following observations might be useful: (1) a[n][k] and b[n][k] are zero when n < 2*k - 1, and (2) on the flip side, for values of k greater than ⌊(n + 1) / 2⌋ the overall answer seems to be identical.
Python code (full matrices are defined for simplicity, but I think only one row of each would actually be needed, space-wise, for a bottom-up method):
a = [[0] * 11 for i in range(0,11)]
b = [([1] + [0] * 10) for i in range(0,11)]
def f(n,k):
return fa(n,k) + fb(n,k)
def fa(n,k):
global a
if a[n][k] or n == 0 or k == 0:
return a[n][k]
elif n == 2*k - 1:
a[n][k] = 1
return 1
else:
a[n][k] = fb(n-1,k-1) + fa(n-1,k)
return a[n][k]
def fb(n,k):
global b
if b[n][k] or n == 0 or n == 2*k - 1:
return b[n][k]
else:
b[n][k] = fb(n-1,k) + fa(n-1,k)
return b[n][k]
def g(n,k):
return sum([f(n,i) for i in range(0,k+1)])
# example
print(g(10,10))
for i in range(0,11):
print(a[i])
print()
for i in range(0,11):
print(b[i])