def can_sum(from_: list[int], into: int) -> bool:
divmod_table = [divmod(into, val) for val in from_]
for quotient, modulus in divmod_table: # O(n)
if not modulus:
return True
elif any(not (modulus % val) for val in from_):
return True
for val in from_: # O(n) or O(nlog(n))?
if into - val <= 0:
continue
elif can_sum(from_, into - val):
return True
return False
I tried finding an analogous algorithm implementation using this logic, but there doesn't seem to be any. In my simple analysis I can see that the first for-loop will be O(n) at worst, since the desired number may not be (i) divisible by any selected numbers (ii) have a modulus which is decomposable by any selected numbers, but what confuses me is the second for-loop.
In the second for-loop, the worst case would be an extension on O(n) because of the recursion, but dependent on the primary property of the first loop being unsuccessful, which would then further lead into higher orders of recursion. Maybe I'm misunderstanding time complexity, but the reduction from a desired sum n into a sequence s_k independently relies on k and n.
What is the relation on time complexity between these two variables?
Related
O(n) time is simply one loop, O(n²) is a loop inside a loop where both loops run at kn times (k is an integer). The pattern continues. However, all finite integers k of O(nᵏ) can be constructed by hand that you can simply nest loops inside another, but what about O(nⁿ) where n is an arbitrary value that approaches infinity?
I was thinking a while loop would work here but how do we set up a break condition. Additionally, I believe O(nⁿ) complexity can be implemented using recursion but how would that look pseudocode-wise?
How do you construct a piece of algorithm that runs in O(nⁿ) using only loops or recursion?
A very simple iterative solution would be to calculate nn and then count up to it:
total = 1
for i in range(n):
total *= n
for i in range(total):
# Do something that does O(1) work
This could also be written recursively:
def calc_nn(n, k):
if k == 0:
return 1
return n * calc_nn(n, k - 1)
def count_to(k):
if k != 0:
count_to(k - 1)
def recursive_version(n):
count_to(calc_nn(n, n))
While studying Selection Sort, I came across a variation known as Bingo Sort. According to this dictionary entry here, Bingo Sort is:
A variant of selection sort that orders items by first finding the least value, then repeatedly moving all items with that value to their final location and find the least value for the next pass.
Based on the definition above, I came up with the following implementation in Python:
def bingo_sort(array, ascending=True):
from operator import lt, gt
def comp(x, y, func):
return func(x, y)
i = 0
while i < len(array):
min_value = array[i]
j = i + 1
for k in range(i + 1, len(array), 1):
if comp(array[k], min_value, (lt if ascending else gt)):
min_value = array[k]
array[i], array[k] = array[k], array[i]
elif array[k] == min_value:
array[j], array[k] = array[k], array[j]
j += 1
i = j
return array
I know that this implementation is problematic. When I run the algorithm on an extremely small array, I get a correctly sorted array. However, running the algorithm with a larger array results in an array that is mostly sorted with incorrect placements here and there. To replicate the issue in Python, the algorithm can be ran on the following input:
test_data = [[randint(0, 101) for i in range(0, 101)],
[uniform(0, 101) for i in range(0, 101)],
["a", "aa", "aaaaaa", "aa", "aaa"],
[5, 5.6],
[3, 2, 4, 1, 5, 6, 7, 8, 9]]
for dataset in test_data:
print(dataset)
print(bingo_sort(dataset, ascending=True, mutation=True))
print("\n")
I cannot for the life of me realize where the fault is at since I've been looking at this algorithm too long and I am not really proficient at these things. I could not find an implementation of Bingo Sort online except an undergraduate graduation project written in 2020. Any help that can point me in the right direction would be greatly appreciated.
I think your main problem is that you're trying to set min_value in your first conditional statement and then to swap based on that same min_value you've just set in your second conditional statement. These processes are supposed to be staggered: the way bingo sort should work is you find the min_value in one iteration, and in the next iteration you swap all instances of that min_value to the front while also finding the next min_value for the following iteration. In this way, min_value should only get changed at the end of every iteration, not during it. When you change the value you're swapping to the front over the course of a given iteration, you can end up unintentionally shuffling things a bit.
I have an implementation of this below if you want to refer to something, with a few notes: since you're allowing a custom comparator, I renamed min_value to swap_value as we're not always grabbing the min, and I modified how the comparator is defined/passed into the function to make the algorithm more flexible. Also, you don't really need three indexes (I think there were even a couple bugs here), so I collapsed i and j into swap_idx, and renamed k to cur_idx. Finally, because of how swapping a given swap_val and finding the next_swap_val is to be staggered, you need to find the initial swap_val up front. I'm using a reduce statement for that, but you could just use another loop over the whole array there; they're equivalent. Here's the code:
from operator import lt, gt
from functools import reduce
def bingo_sort(array, comp=lt):
if len(array) <= 1:
return array
# get the initial swap value as determined by comp
swap_val = reduce(lambda val, cur: cur if comp(cur, val) else val, array)
swap_idx = 0 # set the inital swap_idx to 0
while swap_idx < len(array):
cur_idx = swap_idx
next_swap_val = array[cur_idx]
while cur_idx < len(array):
if comp(array[cur_idx], next_swap_val): # find next swap value
next_swap_val = array[cur_idx]
if array[cur_idx] == swap_val: # swap swap_vals to front of the array
array[swap_idx], array[cur_idx] = array[cur_idx], array[swap_idx]
swap_idx += 1
cur_idx += 1
swap_val = next_swap_val
return array
In general, the complexity of this algorithm depends on how many duplicate values get processed, and when they get processed. This is because every time k duplicate values get processed during a given iteration, the length of the inner loop is decreased by k for all subsequent iterations. Performance is therefore optimized when large clusters of duplicate values are processed early on (as when the smallest values of the array contain many duplicates). From this, there are basically two ways you could analyze the complexity of the algorithm: You could analyze it in terms of where the duplicate values tend to appear in the final sorted array (Type 1), or you could assume the clusters of duplicate values are randomly distributed through the sorted array and analyze complexity in terms of the average size of duplicate clusters (that is, in terms of the magnitude of m relative to n: Type 2).
The definition you linked uses the first type of analysis (based on where duplicates tend to appear) to derive best = Theta(n+m^2), average = Theta(nm), worst = Theta(nm). The second type of analysis produces best = Theta(n), average = Theta(nm), worst = Theta(n^2) as you vary m from Theta(1) to Theta(m) to Theta(n).
In the best Type 1 case, all duplicates will be among the smallest elements of the array, such that the run-time of the inner loop quickly decreases to O(m), and the final iterations of the algorithm proceed as an O(m^2) selection sort. However, there is still the up-front O(n) pass to select the initial swap value, so the overall complexity is O(n + m^2).
In the worst Type 1 case, all duplicates will be among the largest elements of the array. The length of the inner loop isn't substantially shortened until the last iterations of the algorithm, such that we achieve a run-time looking something like n + n-1 + n-2 .... + n-m. This is a sum of m O(n) values, giving us O(nm) total run-time.
In the average Type 1 case (and for all Type 2 cases), we don't assume that the clusters of duplicate values are biased towards the front or back of the sorted array. We take it that the m clusters of duplicate values are randomly distributed through the array in terms of their position and their size. Under this analysis, we expect that after the initial O(n) pass to find the first swap value, each of the m iterations of the outer loop reduce the length of the inner loop by approximately n/m. This leads to an expression of the overall run-time for unknown m and randomly distributed data as:
We can use this expression for the average case run-time with randomly distributed data and unknown m, Theta(nm), as the average Type 2 run-time, and it also directly gives us the best and worst case run-times based on how we might vary the magnitude of n.
In the best Type 2 case, m might just be some constant value independent of n. if we have m=Theta(1) randomly distributed duplicate clusters, the best case run time is then Theta(n*Theta(1))) = Theta(n). For example as you would see O(2n) = O(n) performance from bingo-sort with just one unique value (one pass to find the find value, one pass to swap every single value to the front), and this O(n) asymptotic complexity still holds if m is bounded by any constant.
However in the worst Type 2 case we could have m=Theta(n), and bingo sort essentially devolves into O(n^2) selection sort. This is clearly the case for m = n, but if the amount the inner-loop's run-time is expected to decrease by with each iteration, n/m, is any constant value, which is the case for any m value in Theta(n), we still see O(n^2) complexity.
TL;DR - A non-comparison sorting algorithm whose execution time scales based on the number of bits in the datatype; how do you properly evaluate the Big O time complexity?
I have a sorting algorithm "Bitsort" whose time complexity is dependent on the number of bits in the datatype being sorted, and not on the length of the list being sorted; for lists ranging in length from 2 elements to maximum addressable size, the number of passes over the entire list needed to fully sort it is dictated by the size of the datatype used in the list. The simple version of bitsort ( non-optimized but working code example below ) sorts the list 1 bit at a time. In real world terms, this would often mean that a 32-bit system has 32 bits in both the datatype and the maximum length of a list to be sorted; and Log-base2 of 2^32 is 32. Ergo, it could be argued that simple bitsort's time complexity goes from O(NB), where B is the number of bits, to O(NLogN) as N goes to the maximum length array the system can handle. In absolute terms, though, as N goes to infinity, B stays constant, so it could be argued that the time complexity is O(N).
How should the time complexity be evaluated for simple bitsort?
import sys
from random import randrange
def checkOrder(arr):
for i in range(1, len(arr)):
if arr[i - 1] > arr[i]:
return False
return True
def randomArray(length, height):
result = []
for i in range(0, length):
result.append(randrange(0, height, 1))
return result
def swap(arr, left, right):
temp = arr[left]
arr[left] = arr[right]
arr[right] = temp
# pat. pending, but not a troll
def bitsort(arr, begin, end, shiftBit):
if end-begin <= 0 or shiftBit < 0:
return
LP, RP = begin - 1, end + 1
mask = 0b1 << shiftBit
while True:
LP += 1
if LP > end:
bitsort(arr, begin, end, shiftBit - 1)
return
elif LP == RP:
if LP <= begin: # left has 0 or 1 el
pass
else:
bitsort(arr, begin, LP - 1, shiftBit - 1)
if RP >= end: # right has 0 or 1 elements
pass
else:
bitsort(arr, RP, end, shiftBit - 1)
return
if arr[LP] & mask == 0:
continue
else:
while True:
RP -= 1
if RP < begin:
bitsort(arr, begin, end, shiftBit - 1)
return
elif LP == RP:
if LP <= begin:
if RP <= end:
bitsort(arr, begin, end, shiftBit - 1)
return
else:
pass
else:
bitsort(arr, begin, LP - 1, shiftBit - 1)
if RP >= end:
pass
else:
bitsort(arr, RP, end, shiftBit - 1)
return
if (arr[RP] & mask) >> shiftBit == 1:
continue
else:
swap(arr, LP, RP)
break
def main(args):
arr = randomArray(1000000, 2147483648)
bitsort(arr, 0, len(arr) - 1, 31)
print(f'Proper order: {checkOrder(arr)}')
if __name__ == '__main__':
main(sys.argv[1:])
I’m going to trust that your analysis is correct and that the work done, as a function of N and B, is O(NB), and then focus on your question about how to compare this against, say, O(N log N).
The typical convention when analyzing sorting algorithms like this is - at least, from a CS theory perspective - is to classify this as O(NB) rather than O(N log N) or O(N). Here’s a few reasons why:
Imagine that you’re sorting a list of numbers that you know don’t have many bits in them (say, maybe US phone area codes or US zip codes, which are at most ten bits long). In that case, you could tune the algorithm to only work on the first ten bits of each number, reducing the number of passes required to sort everything. A runtime of O(NB) more clearly telegraphs this, since a B decreases the runtime drops accordingly.
While we’re currently using machines with word sizes that are typically 32 or 64, in principle we could crank that word size way up if we wanted to. SIMD instructions, for example, allow for parallel operations on many more bits than this. Or perhaps you’re sorting variable-length integers, like what you might do with RSA keys. In that case, we couldn’t necessarily claim that the runtime is O(N log N), since we aren’t guaranteed that B = O(log N). It also wouldn’t be correct to say that the runtime is O(N) in that latter case, since we can’t pretend that B is a constant.
Explicitly writing out O(NB) allows us to compare this sorting algorithm against other integer sorting algorithms. For example, we have sorting algorithms whose runtimes are O(N log B) and O(N log N / log B). Leaving the B term in makes it easier to see how these other algorithms compare against yours.
A runtime of O(N log N) would be incorrect for large B and much smaller N. In that case, you might do way more than O(log N) rounds of the bitsort routines.
The argument that it's better than O(n Log n) presumes that keys are duplicated in the array. If all the keys are unique, it will take at least log n bits to represent them. This means if at a minimum the sort time is O(n log n), worse if the keys are more sparsely spread out. For example, if we're talking text encoded as ASCII, it's going to much worse.
I have come up with the following algorithm for a task. primarily the for-loop in line 23-24 that is making me unsure.
function TOP-DOWN-N(C, n, p)
let N[1...n, 1...C] be a new array initialized to -1 for all indicies
return N(C, n)
function N(C, n)
if N[C,n] >= 0 then
return N[C,n]
if C < 0 or n < 1 then
return 0
elif C = 0 then
return 1
elif C > 0 and i > 0 then
r = 0
for i = 0 to n do
r += N(C-p[n-i], n-(i+1))
N[C,n] = r
return N
Let's ignore the fact that this algorithm is implemented recursively. In general, if a dynamic programming algorithm is building an array of N results, and computing each result requires using the values of k other results from that array, then its time complexity is in Ω(Nk), where Ω indicates a lower bound. This should be clear: it takes Ω(k) time to use k values to compute a result, and you have to do that N times.
From the other side, if the computation doesn't do anything asymptotically more time-consuming than reading k values from the array, then O(Nk) is also an upper bound, so the time complexity is Θ(Nk).
So, by that logic we should expect that the time complexity of your algorithm is Θ(n2C), because it builds an array of size nC, computing each result uses Θ(n) other results from that array, and that computation is not dominated by something else.
However, your algorithm has an advantage over an iterative implementation because it doesn't necessarily compute every result in the array. For example, if the number 1 isn't in the array p then your algorithm won't compute N(C-1, n') for any n'; and if the numbers in p are all greater than or equal to C, then the loop is only executed once and the running time is dominated by having to initialize the array of size nC.
It follows that Θ(n2C) is the worst-case time complexity, and the best-case time complexity is Θ(nC).
I have an unordered list of n items, and I'm trying to find the most frequent item in that list. I wrote the following code:
def findFrequant(L):
count = int()
half = len(L)//2
for i in L:
for j in L:
if i == j:
count += 1
if count > half:
msg = "The majority vote is {0}".format(i)
return msg
else:
continue
count = 0
return "mixed list!"
Obviously this procedure with the two loops is O(n^2), and I'm trying to accomplish the same task in O(n log n) time. I'm not looking for a fix or someone to write the code for me, I'm simply looking for directions.
I don't recognize the language here so I'm treating it as pseudocode.
This depends on a hashtable with key of the same type of element of L and value type of int. Count each record in hashtable, then iterate the hashtable as a normal collection of key,value pairs applying normal maxlist algorithm.
O(n) is slightly worse than linear. We remember that expense of a good hash is not linear but may be approximated as linear. Linear space used.
def findFrequant(L):
hash = [,]
vc = 0
vv = null
for i in L
if hash.contains(i)
hash[i] = hash[i] + 1
else
hash[i] = 1
for (k,v) in hash
if v > vc
vv = k
vc = v
else if v == vc
vv = null
if (vv == null)
return "mixed list!"
else
return "The majority vote is {0}".format(v)
You could use a Merge Sort, which has a worst case time complexity of O(n log(n)) and a binary search which has a worst case time complexity of O(log(n)).
There are sorting algorithms with faster best case scenarios, like for example bubble sort which has a best case of O(n), but merge sort performs in O(n log(n)) at it's worst, while bubble sort has a worst case of O(n^2).
Pessimistic as we Computer Scientists are, we generally analyze based on the worst-case scenario. Therefore a combination of merge sort and binary search is probably best in your case.
Note that in certain circumstances a Radix sort might perform faster than merge sort, but this is really dependent on your data.