I am doing a ruby problem that wants a method to find all divisors of a number except itself with the output being a sorted array. If the number is prime, list that it is prime.
I am currently trying to teach myself recursion. Simple recursive problems like finding the factorial of a number is pretty basic to understand but I wanted to know if this particular problem could be done recursively. It seems it fits the criteria of one that could but I could not figure it out.
Example n = 15, divisors besides itself are [3,5].
My code that solved the problem.
require 'prime'
def divisors(n)
return "#{n} is prime" if Prime.prime?(n)
x = n/2
arr = []
until x == 1
arr << x if n % x == 0
x -= 1
end
arr.sort
end
Any help doing this recursively would be great or just letting me know it's not a problem that can be done this way would be helpful too.
def divisors(n, x=nil)
return "#{n} is prime" if Prime.prime?(n)
x ||= n/2
arr = []
return arr if x == 1
if n % x == 0
arr << x
end
(arr.concat divisors(n, x - 1)).sort
end
The function is refactored to handle three things:
the initial call (x ||= /2)
base cases (early returns)
iteration logic done through recursion.
An important thing is that the variable which changes during the iteration (x) is placed as a parameter for the method (with a default value, so it can essentially be used as a private parameter)
By the way, I personally found learning Elixir very helpful in understanding recursion. With pattern matching and multiple functional clauses, the initial call, base case, and iteration can be split into their own methods.
I have a two-fold homework problem, Implement Karp-Rabin and run it on a test file and the second part:
For the hash values modulo q, explain why it is a bad idea to use q as a power of 2. Can you construct a terrible example e.g. for q=64
and n=15?
This is my implementation of the algorithm:
def karp_rabin(text, pattern):
# setup
alphabet = 'ACGT'
d = len(alphabet)
n = len(pattern)
d_n = d**n
q = 2**32-1
m = {char:i for i,char in enumerate(alphabet)}
positions = []
def kr_hash(s):
return sum(d**(n-i-1) * m[s[i]] for i in range(n))
def update_hash():
return d*text_hash + m[text[i+n-1]] - d_n * m[text[i-1]]
pattern_hash = kr_hash(pattern)
for i in range(0, len(text) - n + 1):
text_hash = update_hash() if i else kr_hash(text[i:n])
if pattern_hash % q == text_hash % q and pattern == text[i:i+n]:
positions.append(i)
return ' '.join(map(str, positions))
...The second part of the question is referring to this part of the code/algo:
pattern_hash = kr_hash(pattern)
for i in range(0, len(text) - n + 1):
text_hash = update_hash() if i else kr_hash(text[i:n])
# the modulo q used to check if the hashes are congruent
if pattern_hash % q == text_hash % q and pattern == text[i:i+n]:
positions.append(i)
I don't understand why it would be a bad idea to use q as a power of 2. I've tried running the algorithm on the test file provided(which is the genome of ecoli) and there's no discernible difference.
I tried looking at the formula for how the hash is derived (I'm not good at math) trying to find some common factors that would be really bad for powers of two but found nothing. I feel like if q is a power of 2 it should cause a lot of clashes for the hashes so you'd need to compare strings a lot more but I didn't find anything along those lines either.
I'd really appreciate help on this since I'm stumped. If someone wants to point out what I can do better in the first part (code efficiency, readability, correctness etc.) I'd also be thrilled to hear your input on that.
There is a problem if q divides some power of d, because then only a few characters contribute to the hash. For example in your code d=4, if you take q=64 only the last three characters determine the hash (d**3 = 64).
I don't really see a problem if q is a power of 2 but gcd(d,q) = 1.
Your implementation looks a bit strange because instead of
if pattern_hash % q == text_hash % q and pattern == text[i:i+n]:
you could also use
if pattern_hash == text_hash and pattern == text[i:i+n]:
which would be better because you get fewer collisions.
The Thue–Morse sequence has among its properties that its polynomial hash quickly becomes zero when a power of 2 is the hash module, for whatever polynomial base (d). So if you will try to search a short Thue-Morse sequence in a longer one, you will have a great lot of hash collisions.
For example, your code, slightly adapted:
def karp_rabin(text, pattern):
# setup
alphabet = '01'
d = 15
n = len(pattern)
d_n = d**n
q = 32
m = {char:i for i,char in enumerate(alphabet)}
positions = []
def kr_hash(s):
return sum(d**(n-i-1) * m[s[i]] for i in range(n))
def update_hash():
return d*text_hash + m[text[i+n-1]] - d_n * m[text[i-1]]
pattern_hash = kr_hash(pattern)
for i in range(0, len(text) - n + 1):
text_hash = update_hash() if i else kr_hash(text[i:n])
if pattern_hash % q == text_hash % q : #and pattern == text[i:i+n]:
positions.append(i)
return ' '.join(map(str, positions))
print(karp_rabin('0110100110010110100101100110100110010110011010010110100110010110', '0110100110010110'))
outputs a lot of positions, although only three of then are proper matches.
Note that I have dropped the and pattern == text[i:i+n] check. Obviously if you restore it, the result will be correct, but also it is obvious that the algorithm will do much more work checking this additional condition than for other q. In fact, because there are so many collisions, the whole idea of algorithm becomes not working: you could almost as effectively wrote a simple algorithm that checks every position for a match.
Also note that your implementation is quite strange. The whole idea of polynomial hashing is to take the modulo operation each time you compute the hash. Otherwise your pattern_hash and text_hash are very big numbers. In other languages this might mean arithmetic overflow, but in Python this will invoke big integer arithmetic, which is slow and once again loses the whole idea of the algorithm.
Working on project Euler problem (26), and wanting to use an algorithm looking for the prime, p with the largest order of 10 modulo p. Essentially the problem is to look for the denominator which creates the longest repetend in a decimal. After a bunch of wikipedia reading, it looks like the prime described above would fulfill that. But, unfortunately, it looks like taking the very large powers of 10 results in an error. My question then is : is there a way of getting around this error (making the numbers smaller), or should I abandon this strategy and just do long division (with the plan being to focus on the primes).
[of note, in the order_ten method I can get it to run if I limit the powers of 10 to 300 and probably can go a bit long, which goes along with the length of a long]
import math
def prime_seive(limit):
seive_list = [True]*limit
seive_list[0] = seive_list[1] = False
for i in range(2, limit):
if seive_list[i] == True :
n = 2
while i*n < limit :
seive_list[i*n] = False #get rid of multiples
n = n+1
prime_numbers = [i for i,j in enumerate(seive_list) if j == True]
return prime_numbers
def order_ten(n) :
for k in range(1, n) :
if (math.pow(10,k) -1)%n == 0:
return k
primes = prime_seive(1000)
max_order = 0
max_order_d = -1
for x in reversed(primes) :
order = order_ten(x)
if order > max_order :
max_order = order
max_order_d = x
print max_order
print max_order_d
I suspect that the problem is that your numbers get to large when first taking a large power of ten and then computing the value mod n. (For instance If I asked you to compute 10^11 mod 11, you could remark than 10 mod 11 is (-1) and thus 10^11 mod 11 is just (-1)^11 mod 11 ie. -1.)
Maybe you could try programming your own exponentiation routine mod n, something like (in pseudo code)
myPow (int k, int n) {
if (k==0) return 1;
else return ((myPow(k-1,n)*10)%n);
}
This way you never deal with numbers larger than n.
The way it is written you will get a linear complexity in k for computing the power, and thus a quadratic complexity in n for your function order_ten(n). If this is too slow for you could improve the function myPow to use some smart exponentiation.
I'm trying to write a method that returns the nth prime number.
I've worked out a solution but the problem is in my method. I create a large array of numbers that seems to process super slow. (1..104729).to_a to be exact. I chose 104729 because the max n can be is 10000 and the 10000th integer is 104729. I'm looking for a way to optimize my method.
Is 104729 is too large a value? Is there a way to write this so that I'm not creating a large array?
Here's the method:
def PrimeMover(num)
def is_prime(x)
i = 0
nums = (2..x).to_a
while nums[i] < nums.max
if x % nums[i] != 0
i += 1
else
return false
end
end
return true
end
primes_arr = (3..104729).to_a.select {|y| is_prime(y)}
primes_arr[num]
end
require "prime"
def find_prime(nth)
Prime.take(nth).last
end
Combine Ruby's built-in prime library, and a lazy enumerator for performance:
require 'prime'
(1...100_000).lazy.select(&:prime?).take(100).to_a
Or simply, as highlighted by Arturo:
Prime.take(100)
You can use Ruby's built in #prime? method, which seems pretty efficient.
The code:
require 'prime'
primes_arr = (3..104729).to_a.select &:prime?
runs in 2-3 seconds on my machine, which I find somewhat acceptable.
If you need even better performance or if you really need to write your own method, try implementing the Sieve of Erathostenes. Here are some Ruby samples of that: http://rosettacode.org/wiki/Sieve_of_Eratosthenes#Ruby
Here's an optimal a trial division implementation of is_prime without relying on the Prime class:
A prime number is a whole number divisible only by 1 and itself, and 1 is not prime. So we want to know if x divides into anything less than x and greater than 1. So we start the count at 2, and we end at x - 1.
def prime?(x)
return false if x < 2
2.upto(x - 1) do |n|
return false if (x % n).zero?
end
true
end
As soon as x % n has a remainder, we can break the loop and say this number is not prime. This saves you from looping over the entire range. If all the possible numbers were exhausted, we know the number is prime.
This is still not optimal. For that you would need a sieve, or a different detection algorithm to trial division. But it's a big improvement on your code. Taking the nth up to you.
Well, I have this bit of code that is slowing down the program hugely because it is linear complexity but called a lot of times making the program quadratic complexity. If possible I would like to reduce its computational complexity but otherwise I'll just optimize it where I can. So far I have reduced down to:
def table(n):
a = 1
while 2*a <= n:
if (-a*a)%n == 1: return a
a += 1
Anyone see anything I've missed? Thanks!
EDIT: I forgot to mention: n is always a prime number.
EDIT 2: Here is my new improved program (thank's for all the contributions!):
def table(n):
if n == 2: return 1
if n%4 != 1: return
a1 = n-1
for a in range(1, n//2+1):
if (a*a)%n == a1: return a
EDIT 3: And testing it out in its real context it is much faster! Well this question appears solved but there are many useful answers. I should also say that as well as those above optimizations, I have memoized the function using Python dictionaries...
Ignoring the algorithm for a moment (yes, I know, bad idea), the running time of this can be decreased hugely just by switching from while to for.
for a in range(1, n / 2 + 1)
(Hope this doesn't have an off-by-one error. I'm prone to make these.)
Another thing that I would try is to look if the step width can be incremented.
Take a look at http://modular.fas.harvard.edu/ent/ent_py .
The function sqrtmod does the job if you set a = -1 and p = n.
You missed a small point because the running time of your improved algorithm is still in the order of the square root of n. As long you have only small primes n (let's say less than 2^64), that's ok, and you should probably prefer your implementation to a more complex one.
If the prime n becomes bigger, you might have to switch to an algorithm using a little bit of number theory. To my knowledge, your problem can be solved only with a probabilistic algorithm in time log(n)^3. If I remember correctly, assuming the Riemann hypothesis holds (which most people do), one can show that the running time of the following algorithm (in ruby - sorry, I don't know python) is log(log(n))*log(n)^3:
class Integer
# calculate b to the power of e modulo self
def power(b, e)
raise 'power only defined for integer base' unless b.is_a? Integer
raise 'power only defined for integer exponent' unless e.is_a? Integer
raise 'power is implemented only for positive exponent' if e < 0
return 1 if e.zero?
x = power(b, e>>1)
x *= x
(e & 1).zero? ? x % self : (x*b) % self
end
# Fermat test (probabilistic prime number test)
def prime?(b = 2)
raise "base must be at least 2 in prime?" if b < 2
raise "base must be an integer in prime?" unless b.is_a? Integer
power(b, self >> 1) == 1
end
# find square root of -1 modulo prime
def sqrt_of_minus_one
return 1 if self == 2
return false if (self & 3) != 1
raise 'sqrt_of_minus_one works only for primes' unless prime?
# now just try all numbers (each succeeds with probability 1/2)
2.upto(self) do |b|
e = self >> 1
e >>= 1 while (e & 1).zero?
x = power(b, e)
next if [1, self-1].include? x
loop do
y = (x*x) % self
return x if y == self-1
raise 'sqrt_of_minus_one works only for primes' if y == 1
x = y
end
end
end
end
# find a prime
p = loop do
x = rand(1<<512)
next if (x & 3) != 1
break x if x.prime?
end
puts "%x" % p
puts "%x" % p.sqrt_of_minus_one
The slow part is now finding the prime (which takes approx. log(n)^4 integer operation); finding the square root of -1 takes for 512-bit primes still less than a second.
Consider pre-computing the results and storing them in a file. Nowadays many platforms have a huge disk capacity. Then, obtaining the result will be an O(1) operation.
(Building on Adam's answer.)
Look at the Wikipedia page on quadratic reciprocity:
x^2 ≡ −1 (mod p) is solvable if and only if p ≡ 1 (mod 4).
Then you can avoid the search of a root precisely for those odd prime n's that are not congruent with 1 modulo 4:
def table(n):
if n == 2: return 1
if n%4 != 1: return None # or raise exception
...
Based off OP's second edit:
def table(n):
if n == 2: return 1
if n%4 != 1: return
mod = 0
a1 = n - 1
for a in xrange(1, a1, 2):
mod += a
while mod >= n: mod -= n
if mod == a1: return a//2 + 1
It looks like you're trying to find the square root of -1 modulo n. Unfortunately, this is not an easy problem, depending on what values of n are input into your function. Depending on n, there might not even be a solution. See Wikipedia for more information on this problem.
Edit 2: Surprisingly, strength-reducing the squaring reduces the time a lot, at least on my Python2.5 installation. (I'm surprised because I thought interpreter overhead was taking most of the time, and this doesn't reduce the count of operations in the inner loop.) Reduces the time from 0.572s to 0.146s for table(1234577).
def table(n):
n1 = n - 1
square = 0
for delta in xrange(1, n, 2):
square += delta
if n <= square: square -= n
if square == n1: return delta // 2 + 1
strager posted the same idea but I think less tightly coded. Again, jug's answer is best.
Original answer: Another trivial coding tweak on top of Konrad Rudolph's:
def table(n):
n1 = n - 1
for a in xrange(1, n // 2 + 1):
if (a*a) % n == n1: return a
Speeds it up measurably on my laptop. (About 25% for table(1234577).)
Edit: I didn't notice the python3.0 tag; but the main change was hoisting part of the calculation out of the loop, not the use of xrange. (Academic since there's a better algorithm.)
Is it possible for you to cache the results?
When you calculate a large n you are given the results for the lower n's almost for free.
One thing that you are doing is repeating the calculation -a*a over and over again.
Create a table of the values once and then do look up in the main loop.
Also although this probably doesn't apply to you because your function name is table but if you call a function that takes time to calculate you should cache the result in a table and just do a table look up if you call it again with the same value. This save you the time of calculating all of the values when you first run but you don't waste time repeating the calculation more than once.
I went through and fixed the Harvard version to make it work with python 3.
http://modular.fas.harvard.edu/ent/ent_py
I made some slight changes to make the results exactly the same as the OP's function. There are two possible answers and I forced it to return the smaller answer.
import timeit
def table(n):
if n == 2: return 1
if n%4 != 1: return
a1=n-1
def inversemod(a, p):
x, y = xgcd(a, p)
return x%p
def xgcd(a, b):
x_sign = 1
if a < 0: a = -a; x_sign = -1
x = 1; y = 0; r = 0; s = 1
while b != 0:
(c, q) = (a%b, a//b)
(a, b, r, s, x, y) = (b, c, x-q*r, y-q*s, r, s)
return (x*x_sign, y)
def mul(x, y):
return ((x[0]*y[0]+a1*y[1]*x[1])%n,(x[0]*y[1]+x[1]*y[0])%n)
def pow(x, nn):
ans = (1,0)
xpow = x
while nn != 0:
if nn%2 != 0:
ans = mul(ans, xpow)
xpow = mul(xpow, xpow)
nn >>= 1
return ans
for z in range(2,n) :
u, v = pow((1,z), a1//2)
if v != 0:
vinv = inversemod(v, n)
if (vinv*vinv)%n == a1:
vinv %= n
if vinv <= n//2:
return vinv
else:
return n-vinv
tt=0
pri = [ 5,13,17,29,37,41,53,61,73,89,97,1234577,5915587277,3267000013,3628273133,2860486313,5463458053,3367900313 ]
for x in pri:
t=timeit.Timer('q=table('+str(x)+')','from __main__ import table')
tt +=t.timeit(number=100)
print("table(",x,")=",table(x))
print('total time=',tt/100)
This version takes about 3ms to run through the test cases above.
For comparison using the prime number 1234577
OP Edit2 745ms
The accepted answer 522ms
The above function 0.2ms