Related
I want to talk about how to which proceed.
1. Incorrect usage of Scala. I should try to more improve the code.
2. The efficiency of the algorithm is poor. I should think of an efficient algorithm.
Goal:
Can quickly calculate the max number from more than 1,000 Partition Numbers collections.
Partition Number:
e.g.,
5 -> (5), (1, 4), (2, 3), (1, 1, 3), (1, 2, 2), (1, 1, 1, 2), (1, 1, 1, 1, 1)
I ask that "I want to convert from Python to Scala that Partition Function using Vector", and I was taught to use Stream yesterday.
I fixed code, I can use 10, 50, and so on. But using big numbers(e. g., 100, 1,000 or 10,000) weren't calculate max number.
It calculate from Stream.last to Stream.head.
In my understanding that Stream type can add an element at the head only, so the order of the numbers is reversed form the fist code.
code
import scala.math.floor
class PartitionNumbers(startNum: Int, point: Int) {
var maxNum = 0
var tmpNum = 0
private def appendOnes(n: Int, s: Stream[Int] = Stream.empty[Int]): Stream[Int] = {
if (n == 0) s
else appendOnes(n - 1, 1 #:: s)
}
private def partition(n: Int, k: Int, tmpStream: Stream[Int] = Stream.empty): Int = {
if (n == 0) tmpNum = calculate(tmpStream)
else if (n == 1 | k == 1) tmpNum = calculate(appendOnes(n))
else {
if (n >= k) partition(n - k, k, k #:: tmpStream)
partition(n, k - 1, tmpStream)
}
if (maxNum < tmpNum) maxNum = tmpNum
maxNum
}
def searchMax(n: Int = point): Int = {
partition(n, n)
}
def calculate(usePointsStream: Stream[Int], num: Int = startNum): Int = {
if (usePointsStream.isEmpty) {
num
} else {
calculate(usePointsStream.init, floor(num * (100 + usePointsStream.last) / 100).toInt)
}
}
}
output example
val pn_1 = new PartitionNumbers(100, 10)
println(pn_1.searchMax()) // -> 110
val pn_2 = new PartitionNumbers(1000, 50)
println(pn_2.searchMax()) // -> 1630
val pn_3 = new PartitionNumbers(10000, 100)
println(pn_3.searchMax()) // Can't calculate within 3 minutes using Ryzen 7 2700X.
I was reading Spark correlation algorithm source code and while going through the code, I coulddn't understand this particular peace of code.
This is from the file : org/apache/spark/mllib/linalg/BLAS.scala
def spr(alpha: Double, v: Vector, U: Array[Double]): Unit = {
val n = v.size
v match {
case DenseVector(values) =>
NativeBLAS.dspr("U", n, alpha, values, 1, U)
case SparseVector(size, indices, values) =>
val nnz = indices.length
var colStartIdx = 0
var prevCol = 0
var col = 0
var j = 0
var i = 0
var av = 0.0
while (j < nnz) {
col = indices(j)
// Skip empty columns.
colStartIdx += (col - prevCol) * (col + prevCol + 1) / 2
av = alpha * values(j)
i = 0
while (i <= j) {
U(colStartIdx + indices(i)) += av * values(i)
i += 1
}
j += 1
prevCol = col
}
}
}
I do not know Scala and that could be the reason I could not understand it. Can someone explain what is happening here.
It is being called from Rowmatrix.scala
def computeGramianMatrix(): Matrix = {
val n = numCols().toInt
checkNumColumns(n)
// Computes n*(n+1)/2, avoiding overflow in the multiplication.
// This succeeds when n <= 65535, which is checked above
val nt = if (n % 2 == 0) ((n / 2) * (n + 1)) else (n * ((n + 1) / 2))
// Compute the upper triangular part of the gram matrix.
val GU = rows.treeAggregate(new BDV[Double](nt))(
seqOp = (U, v) => {
BLAS.spr(1.0, v, U.data)
U
}, combOp = (U1, U2) => U1 += U2)
RowMatrix.triuToFull(n, GU.data)
}
The correlation is defined here:
https://en.wikipedia.org/wiki/Pearson_correlation_coefficient
The final goal is to understand the Spark correlation algorithm.
Update 1: Relevent paper https://stanford.edu/~rezab/papers/linalg.pdf
I have one value like 24, and I have four textboxes. How can I dynamically generate four values that add up to 24?
All the values must be integers and can't be negative, and the result cannot be 6, 6, 6, 6; they must be different like: 8, 2, 10, 4. (But 5, 6, 6, 7 would be okay.)
For your stated problem, it is possible to generate an array of all possible solutions and then pick one randomly. There are in fact 1,770 possible solutions.
var solutions = [[Int]]()
for i in 1...21 {
for j in 1...21 {
for k in 1...21 {
let l = 24 - (i + j + k)
if l > 0 && !(i == 6 && j == 6 && k == 6) {
solutions.append([i, j, k, l])
}
}
}
}
// Now generate 20 solutions
for _ in 1...20 {
let rval = Int(arc4random_uniform(UInt32(solutions.count)))
println(solutions[rval])
}
This avoids any bias at the cost of initial setup time and storage.
This could be improved by:
Reducing storage space by only storing the first 3 numbers. The 4th one is always 24 - (sum of first 3)
Reducing storage space by storing each solution as a single integer: (i * 10000 + j * 100 + k)
Speeding up the generation of solutions by realizing that each loop doesn't need to go to 21.
Here is the solution that stores each solution as a single integer and optimizes the loops:
var solutions = [Int]()
for i in 1...21 {
for j in 1...22-i {
for k in 1...23-i-j {
if !(i == 6 && j == 6 && k == 6) {
solutions.append(i * 10000 + j * 100 + k)
}
}
}
}
// Now generate 20 solutions
for _ in 1...20 {
let rval = Int(arc4random_uniform(UInt32(solutions.count)))
let solution = solutions[rval]
// unpack the values
let i = solution / 10000
let j = (solution % 10000) / 100
let k = solution % 100
let l = 24 - (i + j + k)
// print the solution
println("\([i, j, k, l])")
}
Here is a Swift implementation of the algorithm given in https://stackoverflow.com/a/8064754/1187415, with a slight
modification because all numbers are required to be positive.
The method to producing N positive random integers with sum M is
Build an array containing the number 0, followed by N-1 different
random numbers in the range 1 .. M-1, and finally the number M.
Compute the differences of subsequent array elements.
In the first step, we need a random subset of N-1 elements out of
the set { 1, ..., M-1 }. This can be achieved by iterating over this
set and choosing each element with probability n/m, where
m is the remaining number of elements we can choose from and
n is the remaining number of elements to choose.
Instead of storing the chosen random numbers in an array, the
difference to the previously chosen number is computed immediately
and stored.
This gives the following function:
func randomNumbers(#count : Int, withSum sum : Int) -> [Int] {
precondition(sum >= count, "`sum` must not be less than `count`")
var diffs : [Int] = []
var last = 0 // last number chosen
var m = UInt32(sum - 1) // remaining # of elements to choose from
var n = UInt32(count - 1) // remaining # of elements to choose
for i in 1 ..< sum {
// Choose this number `i` with probability n/m:
if arc4random_uniform(m) < n {
diffs.append(i - last)
last = i
n--
}
m--
}
diffs.append(sum - last)
return diffs
}
println(randomNumbers(count: 4, withSum: 24))
If a solution with all elements equal (e.g 6+6+6+6=24) is not
allowed, you can repeat the method until a valid solution is found:
func differentRandomNumbers(#count : Int, withSum sum : Int) -> [Int] {
precondition(count >= 2, "`count` must be at least 2")
var v : [Int]
do {
v = randomNumbers(count: count, withSum: sum)
} while (!contains(v, { $0 != v[0]} ))
return v
}
Here is a simple test. It computes 1,000,000 random representations
of 7 as the sum of 3 positive integers, and counts the distribution
of the results.
let set = NSCountedSet()
for i in 1 ... 1_000_000 {
let v = randomNumbers(count: 3, withSum: 7)
set.addObject(v)
}
for (_, v) in enumerate(set) {
let count = set.countForObject(v)
println("\(v as! [Int]) \(count)")
}
Result:
[1, 4, 2] 66786
[1, 5, 1] 67082
[3, 1, 3] 66273
[2, 2, 3] 66808
[2, 3, 2] 66966
[5, 1, 1] 66545
[2, 1, 4] 66381
[1, 3, 3] 67153
[3, 3, 1] 67034
[4, 1, 2] 66423
[3, 2, 2] 66674
[2, 4, 1] 66418
[4, 2, 1] 66292
[1, 1, 5] 66414
[1, 2, 4] 66751
Update for Swift 3:
func randomNumbers(count : Int, withSum sum : Int) -> [Int] {
precondition(sum >= count, "`sum` must not be less than `count`")
var diffs : [Int] = []
var last = 0 // last number chosen
var m = UInt32(sum - 1) // remaining # of elements to choose from
var n = UInt32(count - 1) // remaining # of elements to choose
for i in 1 ..< sum {
// Choose this number `i` with probability n/m:
if arc4random_uniform(m) < n {
diffs.append(i - last)
last = i
n -= 1
}
m -= 1
}
diffs.append(sum - last)
return diffs
}
print(randomNumbers(count: 4, withSum: 24))
Update for Swift 4.2 (and later), using the unified random API:
func randomNumbers(count : Int, withSum sum : Int) -> [Int] {
precondition(sum >= count, "`sum` must not be less than `count`")
var diffs : [Int] = []
var last = 0 // last number chosen
var m = sum - 1 // remaining # of elements to choose from
var n = count - 1 // remaining # of elements to choose
for i in 1 ..< sum {
// Choose this number `i` with probability n/m:
if Int.random(in: 0..<m) < n {
diffs.append(i - last)
last = i
n -= 1
}
m -= 1
}
diffs.append(sum - last)
return diffs
}
func getRandomValues(amountOfValues:Int, totalAmount:Int) -> [Int]?{
if amountOfValues < 1{
return nil
}
if totalAmount < 1{
return nil
}
if totalAmount < amountOfValues{
return nil
}
var values:[Int] = []
var valueLeft = totalAmount
for i in 0..<amountOfValues{
if i == amountOfValues - 1{
values.append(valueLeft)
break
}
var value = Int(arc4random_uniform(UInt32(valueLeft - (amountOfValues - i))) + 1)
valueLeft -= value
values.append(value)
}
var shuffledArray:[Int] = []
for i in 0..<values.count {
var rnd = Int(arc4random_uniform(UInt32(values.count)))
shuffledArray.append(values[rnd])
values.removeAtIndex(rnd)
}
return shuffledArray
}
getRandomValues(4, 24)
This is not a final answer, but it should be a (good) starting point.
How it works: It takes 2 parameters. The amount of random values (4 in your case) and the total amount (24 in your case).
It takes a random value between the total Amount and 0, stores this in an array and it subtracts this from a variable which stores the amount that is left and stores the new value.
Than it takes a new random value between the amount that is left and 0, stores this in an array and it again subtracts this from the amount that is left and stores the new value.
When it is the last number needed, it sees what amount is left and adds that to the array
EDIT:
Adding a +1 to the random value removes the problem of having 0 in your array.
EDIT 2:
Shuffling the array does remove the increased chance of having a high value as the first value.
One solution that is unfortunatly non-deterministic but completely random is as follows:
For a total of 24 in 4 numbers:
pick four random numbers between 1 and 21
repeat until the total of the numbers equals 24 and they are not all 6.
This will, on average, loop about 100 times before finding a solution.
Here's a solution which should have significantly* less bias than some of the other methods. It works by generating the requested number of random floating point numbers, multiplying or dividing all of them until they add up to the target total, and then rounding them into integers. The rounding process changes the total, so we need to correct for that by adding or subtracting from random terms until they add up to the right amount.
func getRandomDoubles(#count: Int, #total: Double) -> [Double] {
var nonNormalized = [Double]()
nonNormalized.reserveCapacity(count)
for i in 0..<count {
nonNormalized.append(Double(arc4random()) / 0xFFFFFFFF)
}
let nonNormalizedSum = reduce(nonNormalized, 0) { $0 + $1 }
let normalized = nonNormalized.map { $0 * total / nonNormalizedSum }
return normalized
}
func getRandomInts(#count: Int, #total: Int) -> [Int] {
let doubles = getRandomDoubles(count: count, total: Double(total))
var ints = [Int]()
ints.reserveCapacity(count)
for double in doubles {
if double < 1 || double % 1 >= 0.5 {
// round up
ints.append(Int(ceil(double)))
} else {
// round down
ints.append(Int(floor(double)))
}
}
let roundingErrors = total - (reduce(ints, 0) { $0 + $1 })
let directionToAdjust: Int = roundingErrors > 0 ? 1 : -1
var corrections = abs(roundingErrors)
while corrections > 0 {
let index = Int(arc4random_uniform(UInt32(count)))
if directionToAdjust == -1 && ints[index] <= 1 { continue }
ints[index] += directionToAdjust
corrections--
}
return ints
}
*EDIT: Martin R has correctly pointed out that this is not nearly as uniform as one might expect, and is in fact highly biased towards numbers in the middle of the 1-24 range. I would not recommend using this solution, but I'm leaving it up so that others can know not to make the same mistake.
As a recursive function the algorithm is very nice:
func getRandomValues(amount: Int, total: Int) -> [Int] {
if amount == 1 { return [total] }
if amount == total { return Array(count: amount, repeatedValue: 1) }
let number = Int(arc4random()) % (total - amount + 1) + 1
return [number] + getRandomValues(amount - 1, total - number)
}
And with safety check:
func getRandomValues(amount: Int, total: Int) -> [Int]? {
if !(1...total ~= amount) { return nil }
if amount == 1 { return [total] }
if amount == total { return Array(count: amount, repeatedValue: 1) }
let number = Int(arc4random()) % (total - amount + 1) + 1
return [number] + getRandomValues(amount - 1, total - number)!
}
As #MartinR pointed out the code above is extremely biased. So in order to have a uniform distribution of the output values you should use this piece of code:
func getRandomValues(amount: Int, total: Int) -> [Int] {
var numberSet = Set<Int>()
// add splitting points to numberSet
for _ in 1...amount - 1 {
var number = Int(arc4random()) % (total - 1) + 1
while numberSet.contains(number) {
number = Int(arc4random()) % (total - 1) + 1
}
numberSet.insert(number)
}
// sort numberSet and return the differences between the splitting points
let sortedArray = (Array(numberSet) + [0, total]).sort()
return sortedArray.enumerate().flatMap{
indexElement in
if indexElement.index == amount { return nil }
return sortedArray[indexElement.index + 1] - indexElement.element
}
}
A javascript implementation for those who may be looking for such case:
const numbersSumTo = (length, value) => {
const fourRandomNumbers = Array.from({ length: length }, () => Math.floor(Math.random() * 6) + 1);
const res = fourRandomNumbers.map(num => (num / fourRandomNumbers.reduce((a, b) => a + b, 0)) * value).map(num => Math.trunc(num));
res[0] += Math.abs(res.reduce((a, b) => a + b, 0) - value);
return res;
}
// Gets an array with 4 items which sum to 100
const res = numbersSumTo(4, 100);
const resSum = res.reduce((a, b) => a + b, 0);
console.log({
res,
resSum
});
Also plenty of different methods of approach can be found here on this question: https://math.stackexchange.com/questions/1276206/method-of-generating-random-numbers-that-sum-to-100-is-this-truly-random
Given a Map of objects and designated proportions (let's say they add up to 100 to make it easy):
val ss : Map[String,Double] = Map("A"->42, "B"->32, "C"->26)
How can I generate a sequence such that for a subset of size n there are ~42% "A"s, ~32% "B"s and ~26% "C"s? (Obviously, small n will have larger errors).
(Work language is Scala, but I'm just asking for the algorithm.)
UPDATE: I resisted a random approach since, for instance, there's ~16% chance that the sequence would start with AA and ~11% chance it would start with BB and there would be very low odds that for n precisely == (sum of proportions) the distribution would be perfect. So, following #MvG's answer, I implemented as follows:
/**
Returns the key whose achieved proportions are most below desired proportions
*/
def next[T](proportions : Map[T, Double], achievedToDate : Map[T,Double]) : T = {
val proportionsSum = proportions.values.sum
val desiredPercentages = proportions.mapValues(v => v / proportionsSum)
//Initially no achieved percentages, so avoid / 0
val toDateTotal = if(achievedToDate.values.sum == 0.0){
1
}else{
achievedToDate.values.sum
}
val achievedPercentages = achievedToDate.mapValues(v => v / toDateTotal)
val gaps = achievedPercentages.map{ case (k, v) =>
val gap = desiredPercentages(k) - v
(k -> gap)
}
val maxUnder = gaps.values.toList.sortWith(_ > _).head
//println("Max gap is " + maxUnder)
val gapsForMaxUnder = gaps.mapValues{v => Math.abs(v - maxUnder) < Double.Epsilon }
val keysByHasMaxUnder = gapsForMaxUnder.map(_.swap)
keysByHasMaxUnder(true)
}
/**
Stream of most-fair next element
*/
def proportionalStream[T](proportions : Map[T, Double], toDate : Map[T, Double]) : Stream[T] = {
val nextS = next(proportions, toDate)
val tailToDate = toDate + (nextS -> (toDate(nextS) + 1.0))
Stream.cons(
nextS,
proportionalStream(proportions, tailToDate)
)
}
That when used, e.g., :
val ss : Map[String,Double] = Map("A"->42, "B"->32, "C"->26)
val none : Map[String,Double] = ss.mapValues(_ => 0.0)
val mySequence = (proportionalStream(ss, none) take 100).toList
println("Desired : " + ss)
println("Achieved : " + mySequence.groupBy(identity).mapValues(_.size))
mySequence.map(s => print(s))
println
produces :
Desired : Map(A -> 42.0, B -> 32.0, C -> 26.0)
Achieved : Map(C -> 26, A -> 42, B -> 32)
ABCABCABACBACABACBABACABCABACBACABABCABACABCABACBA
CABABCABACBACABACBABACABCABACBACABABCABACABCABACBA
For a deterministic approach, the most obvious solution would probably be this:
Keep track of the number of occurrences of each item in the sequence so far.
For the next item, choose that item for which the difference between intended and actual count (or proportion, if you prefer that) is maximal, but only if the intended count (resp. proportion) is greater than the actual one.
If there is a tie, break it in an arbitrary but deterministic way, e.g. choosing the alphabetically lowest item.
This approach would ensure an optimal adherence to the prescribed ratio for every prefix of the infinite sequence generated in this way.
Quick & dirty python proof of concept (don't expect any of the variable “names” to make any sense):
import sys
p = [0.42, 0.32, 0.26]
c = [0, 0, 0]
a = ['A', 'B', 'C']
n = 0
while n < 70*5:
n += 1
x = 0
s = n*p[0] - c[0]
for i in [1, 2]:
si = n*p[i] - c[i]
if si > s:
x = i
s = si
sys.stdout.write(a[x])
if n % 70 == 0:
sys.stdout.write('\n')
c[x] += 1
Generates
ABCABCABACABACBABCAABCABACBACABACBABCABACABACBACBAABCABCABACABACBABCAB
ACABACBACABACBABCABACABACBACBAABCABCABACABACBABCAABCABACBACABACBABCABA
CABACBACBAABCABCABACABACBABCABACABACBACBAACBABCABACABACBACBAABCABCABAC
ABACBABCABACABACBACBAACBABCABACABACBACBAABCABCABACABACBABCABACABACBACB
AACBABCABACABACBACBAABCABCABACABACBABCAABCABACBACBAACBABCABACABACBACBA
For every item of the sequence, compute a (pseudo-)random number r equidistributed between 0 (inclusive) and 100 (exclusive).
If 0 ≤ r < 42, take A
If 42 ≤ r < (42+32), take B
If (42+32) ≤ r < (42+32+26)=100, take C
The number of each entry in your subset is going to be the same as in your map, but with a scaling factor applied.
The scaling factor is n/100.
So if n was 50, you would have { Ax21, Bx16, Cx13 }.
Randomize the order to your liking.
The simplest "deterministic" [in terms of #elements of each category] solution [IMO] will be: add elements in predefined order, and then shuffle the resulting list.
First, add map(x)/100 * n elements from each element x chose how you handle integer arithmetics to avoid off by one element], and then shuffle the resulting list.
Shuffling a list is simple with fisher-yates shuffle, which is implemented in most languages: for example java has Collections.shuffle(), and C++ has random_shuffle()
In java, it will be as simple as:
int N = 107;
List<String> res = new ArrayList<String>();
for (Entry<String,Integer> e : map.entrySet()) { //map is predefined Map<String,Integer> for frequencies
for (int i = 0; i < Math.round(e.getValue()/100.0 * N); i++) {
res.add(e.getKey());
}
}
Collections.shuffle(res);
This is nondeterministic, but gives a distribution of values close to MvG's. It suffers from the problem that it could give AAA right at the start. I post it here for completeness' sake given how it proves my dissent with MvG was misplaced (and I don't expect any upvotes).
Now, if someone has an idea for an expand function that is deterministic and won't just duplicate MvG's method (rendering the calc function useless), I'm all ears!
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>ErikE's answer</title>
</head>
<body>
<div id="output"></div>
<script type="text/javascript">
if (!Array.each) {
Array.prototype.each = function(callback) {
var i, l = this.length;
for (i = 0; i < l; i += 1) {
callback(i, this[i]);
}
};
}
if (!Array.prototype.sum) {
Array.prototype.sum = function() {
var sum = 0;
this.each(function(i, val) {
sum += val;
});
return sum;
};
}
function expand(counts) {
var
result = "",
charlist = [],
l,
index;
counts.each(function(i, val) {
char = String.fromCharCode(i + 65);
for ( ; val > 0; val -= 1) {
charlist.push(char);
}
});
l = charlist.length;
for ( ; l > 0; l -= 1) {
index = Math.floor(Math.random() * l);
result += charlist[index];
charlist.splice(index, 1);
}
return result;
}
function calc(n, proportions) {
var percents = [],
counts = [],
errors = [],
fnmap = [],
errorSum,
worstIndex;
fnmap[1] = "min";
fnmap[-1] = "max";
proportions.each(function(i, val) {
percents[i] = val / proportions.sum() * n;
counts[i] = Math.round(percents[i]);
errors[i] = counts[i] - percents[i];
});
errorSum = counts.sum() - n;
while (errorSum != 0) {
adjust = errorSum < 0 ? 1 : -1;
worstIndex = errors.indexOf(Math[fnmap[adjust]].apply(0, errors));
counts[worstIndex] += adjust;
errors[worstIndex] = counts[worstIndex] - percents[worstIndex];
errorSum += adjust;
}
return expand(counts);
}
document.body.onload = function() {
document.getElementById('output').innerHTML = calc(99, [25.1, 24.9, 25.9, 24.1]);
};
</script>
</body>
</html>
I have created solution for PE P12 in Scala but is very very slow. Can somebody can tell me why? How to optimize this? calculateDevisors() - naive approach and calculateNumberOfDivisors() - divisor function has the same speed :/
import annotation.tailrec
def isPrime(number: Int): Boolean = {
if (number < 2 || (number != 2 && number % 2 == 0) || (number != 3 && number % 3 == 0))
false
else {
val sqrtOfNumber = math.sqrt(number) toInt
#tailrec def isPrimeInternal(divisor: Int, increment: Int): Boolean = {
if (divisor > sqrtOfNumber)
true
else if (number % divisor == 0)
false
else
isPrimeInternal(divisor + increment, 6 - increment)
}
isPrimeInternal(5, 2)
}
}
def generatePrimeNumbers(count: Int): List[Int] = {
#tailrec def generatePrimeNumbersInternal(number: Int = 3, index: Int = 0,
primeNumbers: List[Int] = List(2)): List[Int] = {
if (index == count)
primeNumbers
else if (isPrime(number))
generatePrimeNumbersInternal(number + 2, index + 1, primeNumbers :+ number)
else
generatePrimeNumbersInternal(number + 2, index, primeNumbers)
}
generatePrimeNumbersInternal();
}
val primes = Stream.cons(2, Stream.from(3, 2) filter {isPrime(_)})
def calculateDivisors(number: Int) = {
for {
divisor <- 1 to number
if (number % divisor == 0)
} yield divisor
}
#inline def decomposeToPrimeNumbers(number: Int) = {
val sqrtOfNumber = math.sqrt(number).toInt
#tailrec def decomposeToPrimeNumbersInternal(number: Int, primeNumberIndex: Int = 0,
factors: List[Int] = List.empty[Int]): List[Int] = {
val primeNumber = primes(primeNumberIndex)
if (primeNumberIndex > sqrtOfNumber)
factors
else if (number % primeNumber == 0)
decomposeToPrimeNumbersInternal(number / primeNumber, primeNumberIndex, factors :+ primeNumber)
else
decomposeToPrimeNumbersInternal(number, primeNumberIndex + 1, factors)
}
decomposeToPrimeNumbersInternal(number) groupBy {n => n} map {case (n: Int, l: List[Int]) => (n, l size)}
}
#inline def calculateNumberOfDivisors(number: Int) = {
decomposeToPrimeNumbers(number) map {case (primeNumber, exponent) => exponent + 1} product
}
#tailrec def calculate(number: Int = 12300): Int = {
val triangleNumber = ((number * number) + number) / 2
val startTime = System.currentTimeMillis()
val numberOfDivisors = calculateNumberOfDivisors(triangleNumber)
val elapsedTime = System.currentTimeMillis() - startTime
printf("%d: V: %d D: %d T: %dms\n", number, triangleNumber, numberOfDivisors, elapsedTime)
if (numberOfDivisors > 500)
triangleNumber
else
calculate(number + 1)
}
println(calculate())
You could first check what is slow. Your prime calculation, for instance, is very, very slow. For each number n, you try to divide n by each each number from 5 to sqrt(n), skipping multiples of 2 and 3. Not only you do not skip numbers you already know are not primes, but even if you fix this, the complexity of this algorithm is much worse than the traditional Sieve of Eratosthenes. See one Scala implementation for the Sieve here.
That is not to say that the rest of your code isn't suboptimal as well, but I'll leave that for others.
EDIT
Indeed, indexed access to Stream is terrible. Here's a rewrite that works with Stream, instead of converting everything to Array. Also, note the remark before the first if for a possible bug in your code.
#tailrec def decomposeToPrimeNumbersInternal(number: Int, primes: Stream[Int],
factors: List[Int] = List.empty[Int]): List[Int] = {
val primeNumber = primes.head
// Comparing primeNumberIndex with sqrtOfNumber didn't make any sense
if (primeNumber > sqrtOfNumber)
factors
else if (number % primeNumber == 0)
decomposeToPrimeNumbersInternal(number / primeNumber, primes, factors :+ primeNumber)
else
decomposeToPrimeNumbersInternal(number, primes.tail, factors)
}
Slow compared to....? How do you know it's an issue with Scala, and not with your algorithm?
An admittedly quick read of the code suggests you might be recalculating primes and other values over and over. isPrimeInternal jumps out as a possible case where this might be a problem.
Your code is not compilable, some parts are missing, so I'm guessing here. Some thing that frequently hurts performance is boxing/unboxing taking place in collections. Another thing that I noted is that you cunstruct your primes as a Stream - which is a good thing - but don't take advantage of this in your isPrime function, which uses a primitive 2,3-wheel (1 and 5 mod 6) instead. I might be wrong, but try to replace it by
def isPrime(number: Int): Boolean = {
val sq = math.sqrt(number + 0.5).toInt
! primes.takeWhile(_ <= sq).exists(p => number % p == 0)
}
My scala algorithm that calculates divisors of a given number. It worked fine in the solution of
Project Euler Problem 12.
def countDivisors(numberToFindDivisor: BigInt): Int = {
def countWithAcc(numberToFindDivisor: BigInt, currentCandidate: Int, currentCountOfDivisors: Int,limit: BigInt): Int = {
if (currentCandidate >= limit) currentCountOfDivisors
else {
if (numberToFindDivisor % currentCandidate == 0)
countWithAcc(numberToFindDivisor, currentCandidate + 1, currentCountOfDivisors + 2, numberToFindDivisor / currentCandidate)
else
countWithAcc(numberToFindDivisor, currentCandidate + 1, currentCountOfDivisors, limit)
}
}
countWithAcc(numberToFindDivisor, 1, 0, numberToFindDivisor + 1)
}
calculateDivisors can be greatly improved by only checking for divisors up to the square root of the number. Each time you find a divisor below the sqrt, you also find one above.
def calculateDivisors(n: Int) = {
var res = 1
val intSqrt = Math.sqrt(n).toInt
for (i <- 2 until intSqrt) {
if (n % i == 0) {
res += 2
}
}
if (n == intSqrt * intSqrt) {
res += 1
}
res
}