object BubbleSort {
def main(args : Array[String]) : Unit = {
bubbleSort(Array(50,33,62,21,100)) foreach println
}
def bubbleSort(a:Array[Int]):Array[Int]={
for(i<- 1 to a.length-1){
for(j <- (i-1) to 0 by -1){
if(a(j)>a(j+1)){
val temp=a(j+1)
a(j+1)=a(j)
a(j)=temp
}
}
}
a
}
}
I have the above code supposedly implementing bubble sort in Scala. It is sorting the given numbers in the main but is it a well implemented Bubble Sorting algorithm?
Also what does this line of code mean in pseudocode: for(j <- (i-1) to 0 by -1){
I can't understand it.
Thanks for your help
The best way to figure out what a bit of Scala code does is to run it in the REPL:
scala> 5 to 0 by -1
res0: scala.collection.immutable.Range = Range(5, 4, 3, 2, 1, 0)
So that code counts from (i-1) to 0, backwards.
More generally, x to y creates a Range from integer x to integer y. The by portion modifies this counting. For example, 0 to 6 by 2 means "count from 0 to 6 by 2", or Range(0, 2, 4, 6). In our case, by -1 indicates that we should count backwards by 1.
As for understanding how bubble sort works, you should read the Wikipedia article and use that to help you understand what the code is doing.
This can be the shortest functional implementation of Bubble sort
/**
* Functional implementation of bubble sort
* sort function swaps each element in the given list and create new list and iterate the same operation for length of the list times.
* sort function takes three parameters
* a) iteration list -> this is used to track the iteration. after each iteration element is dropped so that sort function exists the iteration list is empty
* b) source list -> this is source list taken for element wise sorting
* c) result -> stores the element as it get sorted and at end of each iteration, it will be the source for next sort iteration
*/
object Test extends App {
def bubblesort(source: List[Int]) : List[Int] = {
#tailrec
def sort(iteration: List[Int], source: List[Int] , result: List[Int]) : List[Int]= source match {
case h1 :: h2 :: rest => if(h1 > h2) sort(iteration, h1 :: rest, result :+ h2) else sort(iteration, h2 :: rest, result :+ h1)
case l:: Nil => sort(iteration, Nil, result :+ l)
case Nil => if(iteration.isEmpty) return result else sort(iteration.dropRight(1), result, Nil )
}
sort(source,source,Nil)
}
println(bubblesort(List(4,3,2,224,15,17,9,4,225,1,7)))
//List(1, 2, 3, 4, 4, 7, 9, 15, 17, 224, 225)
}
The example you have posted is basically a imperative or Java way of doing bubble sort in Scala which is not bad but defies the purpose of Functional Programming in Scala.. the same code can we written sorter like below (basically combining both the for loops in one line and doing a range on the source length)
def imperativeBubbleSort[T <% Ordered[T]](source: Array[T]): Array[T] = {
for (i <- 0 until source.length - 1; j <- 0 until source.length - 1 - i) {
if (source(j) > source(j + 1)) {
val temp = source(j)
source(j) = source(j + 1)
source(j + 1) = temp
}
}
source
}
Scala Flavor of bubble sort can be different and simple example is below
(basically usage of Pattern matching..)
def bubbleSort[T <% Ordered[T]](inputList: List[T]): List[T] = {
def sort(source: List[T], result: List[T]) = {
if (source.isEmpty) result
else bubble(source, Nil, result)
}
def bubble(source: List[T], tempList: List[T], result: List[T]): List[T] = source match {
case h1 :: h2 :: t =>
if (h1 > h2) bubble(h1 :: t, h2 :: tempList, result)
else bubble(h2 :: t, h1 :: tempList, result)
case h1 :: t => sort(tempList, h1 :: result)
}
sort(inputList, Nil)
}
#tailrec
def bubbleSort(payload: List[Int], newPayload: List[Int], result: List[Int]): List[Int] = {
payload match {
case Nil => result
case s::Nil => bubbleSort(newPayload, List.empty, s::result)
case x::xs => x.compareTo(xs.head) match {
case 0 => bubbleSort(xs, x::newPayload, result)
case 1 => bubbleSort(x::xs.tail, xs.head::newPayload, result)
case -1 => bubbleSort(xs, x::newPayload, result)
}
}
}
val payload = List(7, 2, 5, 10, 4, 9, 12)
bubbleSort(payload, List.empty, List.empty)
Related
I'm really interested in how this algorithm can be implemented. If possible, it would be great to see an implementation with and without recursion. I am new to the language so I would be very grateful for help. All I could come up with was this code and it goes no further:
print(counterOccur("aabcdddeabb"))
def counterOccur(string: String) =
string.toCharArray.toList.map(char => {
if (!char.charValue().equals(char.charValue() + 1)) (char, counter)
else (char, counter + 1)
})
I realize that it's not even close to the truth, I just don't even have a clue what else could be used.
First solution with using recursion. I take Char by Char from string and check if last element in the Vector is the same as current. If elements the same I update last element by increasing count(It is first case). If last element does not the same I just add new element to the Vector(second case). When I took all Chars from the string I just return result.
def counterOccur(string: String): Vector[(Char, Int)] = {
#tailrec
def loop(str: List[Char], result: Vector[(Char, Int)]): Vector[(Char, Int)] = {
str match {
case x :: xs if result.lastOption.exists(_._1.equals(x)) =>
val count = result(result.size - 1)._2
loop(xs, result.updated(result.size - 1, (x, count + 1)))
case x :: xs =>
loop(xs, result :+ (x, 1))
case Nil => result
}
}
loop(string.toList, Vector.empty[(Char, Int)])
}
println(counterOccur("aabcdddeabb"))
Second solution that does not use recursion. It works the same, but instead of the recursion it is using foldLeft.
def counterOccur2(string: String): Vector[(Char, Int)] = {
string.foldLeft(Vector.empty[(Char, Int)])((r, v) => {
val lastElementIndex = r.size - 1
if (r.lastOption.exists(lv => lv._1.equals(v))) {
r.updated(lastElementIndex, (v, r(lastElementIndex)._2 + 1))
} else {
r :+ (v, 1)
}
})
}
println(counterOccur2("aabcdddeabb"))
You can use a very simple foldLeft to accumulate. You also don't need toCharArray and toList because strings are implicitly convertible to Seq[Char]:
"aabcdddeabb".foldLeft(collection.mutable.ListBuffer[(Char,Int)]()){ (acc, elm) =>
acc.lastOption match {
case Some((c, i)) if c == elm =>
acc.dropRightInPlace(1).addOne((elm, i+1))
case _ =>
acc.addOne((elm, 1))
}
}
Here is a solution using foldLeft and a custom State case class:
def countConsecutives[A](data: List[A]): List[(A, Int)] = {
final case class State(currentElem: A, currentCount: Int, acc: List[(A, Int)]) {
def result: List[(A, Int)] =
((currentElem -> currentCount) :: acc).reverse
def nextState(newElem: A): State =
if (newElem == currentElem)
this.copy(currentCount = this.currentCount + 1)
else
State(
currentElem = newElem,
currentCount = 1,
acc = (this.currentElem -> this.currentCount) :: this.acc
)
}
object State {
def initial(a: A): State =
State(
currentElem = a,
currentCount = 1,
acc = List.empty
)
}
data match {
case a :: tail =>
tail.foldLeft(State.initial(a)) {
case (state, newElem) =>
state.nextState(newElem)
}.result
case Nil =>
List.empty
}
}
You can see the code running here.
One possibility is to use the unfold method. This method is defined for several collection types, here I'm using it to produce an Iterator (documented here for version 2.13.8):
def spans[A](as: Seq[A]): Iterator[Seq[A]] =
Iterator.unfold(as) {
case head +: tail =>
val (span, rest) = tail.span(_ == head)
Some((head +: span, rest))
case _ =>
None
}
unfold starts from a state and applies a function that returns, either:
None if we want to signal that the collection ended
Some of a pair that contains the next item of the collection we want to produce and the "remaining" state that will be fed to the next iteration.
In this example in particular, we start from a sequence of A called as (which can be a sequence of characters) and at each iteration:
if there's at least one item
we split head and tail
we further split the tail into the longest prefix that contains items equal to the head and the rest
we return the head and the prefix we got above as the next item
we return the rest of the collection as the state for the following iteration
otherwise, we return None as there's nothing more to be done
The result is a fairly flexible function that can be used to group together spans of equal items. You can then define the function you wanted initially in terms of this:
def spanLengths[A](as: Seq[A]): Iterator[(A, Int)] =
spans(as).map(a => a.head -> a.length)
This can be probably made more generic and its performance improved, but I hope this can be an helpful example about another possible approach. While folding a collection is a recursive approach, unfolding is referred to as a corecursive one (Wikipedia article).
You can play around with this code here on Scastie.
For
str = "aabcdddeabb"
you could extract matches of the regular expression
rgx = /(.)\1*/
to obtain the array
["aa", "b", "c", "ddd", "e", "a", "bb"]
and then map each element of the array to the desired string.1
def counterOccur(str: String): List[(Char, Int)] = {
"""(.)\1*""".r
.findAllIn(str)
.map(m => (m.charAt(0), m.length)).toList
}
counterOccur("aabcdddeabb")
#=> res0: List[(Char, Int)] = List((a,2), (b,1), (c,1), (d,3), (e,1), (a,1), (b,2))
The regular expression reads, "match any character and save it to capture group 1 ((.)), then match the content of capture group 1 zero or more times (\1*).
1. Scala code kindly provided by #Thefourthbird.
Given an array of integers, every element appears three times except for one, which appears exactly once. Find that single one. This is what I have now. But I don't know how to break the for loop once I get the single number "b". Any solution in scala please?
for(Array(a,b) <- nums.sorted.sliding(2))
{
if (a == b){j = j+1}
else
{
if (j < 3) j =1
b
}
}
This will do it.
nums.groupBy(identity).find(_._2.length == 1).get._1
It's a bit unsafe in that it will throw if there is no single-count element. It can be made safer if a default value is returned when no single-count element is found.
nums.groupBy(identity).find(_._2.length == 1).fold(-1)(_._1)
Another way is to sum the array by adding digits of two numbers in base 3 modulo 3 (in other words XOR in base 3). The elements that appear 3 times will become zero, so the result of this sum will be the single number.
def findSingleNumber(numbers: Array[Int]) = {
def add3(a: String, b: String): String = a.zipAll(b, '0', '0').map {
case (i, j) => ((i.toInt + j.toInt) % 3 + '0').toChar
}(collection.breakOut)
val numbersInBase3 = numbers.map(n => Integer.toString(n, 3).reverse)
Integer.parseInt(numbersInBase3.fold("0")(add3).reverse, 3)
}
scala> findSingleNumber(Array(10, 20, 30, 100, 20, 100, 10, 10, 20, 100))
res1: Int = 30
Or representing base 3 numbers as digit arrays:
def findSingleNumber(numbers: Array[Int]) = {
def toBase3(int: Int): Array[Int] =
Iterator.iterate(int)(_ / 3).takeWhile(_ != 0).map(_ % 3).toArray
def toBase10(arr: Array[Int]): Int =
arr.reverseIterator.foldLeft(0)(_ * 3 + _)
def add3(a: Array[Int], b: Array[Int]): Array[Int] = a.zipAll(b, 0, 0).map {
case (i, j) => (i + j) % 3
}
toBase10(numbers.map(toBase3).fold(Array.empty[Int])(add3))
}
I came across another codechef problem which I am attempting to solve in Scala. The problem statement is as follows:
Stepford Street was a dead end street. The houses on Stepford Street
were bought by wealthy millionaires. They had them extensively altered
so that as one progressed along the street, the height of the
buildings increased rapidly. However, not all millionaires were
created equal. Some refused to follow this trend and kept their houses
at their original heights. The resulting progression of heights was
thus disturbed. A contest to locate the most ordered street was
announced by the Beverly Hills Municipal Corporation. The criteria for
the most ordered street was set as follows: If there exists a house
with a lower height later in the street than the house under
consideration, then the pair (current house, later house) counts as 1
point towards the disorderliness index of the street. It is not
necessary that the later house be adjacent to the current house. Note:
No two houses on a street will be of the same height For example, for
the input: 1 2 4 5 3 6 The pairs (4,3), (5,3) form disordered pairs.
Thus the disorderliness index of this array is 2. As the criteria for
determining the disorderliness is complex, the BHMC has requested your
help to automate the process. You need to write an efficient program
that calculates the disorderliness index of a street.
A sample input output provided is as follows:
Input: 1 2 4 5 3 6
Output: 2
The output is 2 because of two pairs (4,3) and (5,3)
To solve this problem I thought I should use a variant of MergeSort,incrementing by 1 when the left element is greater than the right element.
My scala code is as follows:
def dysfunctionCalc(input:List[Int]):Int = {
val leftHalf = input.size/2
println("HalfSize:"+leftHalf)
val isOdd = input.size%2
println("Is odd:"+isOdd)
val leftList = input.take(leftHalf+isOdd)
println("LeftList:"+leftList)
val rightList = input.drop(leftHalf+isOdd)
println("RightList:"+rightList)
if ((leftList.size <= 1) && (rightList.size <= 1)){
println("Entering input where both lists are <= 1")
if(leftList.size == 0 || rightList.size == 0){
println("One of the lists is less than 0")
0
}
else if(leftList.head > rightList.head)1 else 0
}
else{
println("Both lists are greater than 1")
dysfunctionCalc(leftList) + dysfunctionCalc(rightList)
}
}
First off, my logic is wrong,it doesn't have a merge stage and I am not sure what would be the best way to percolate the result of the base-case up the stack and compare it with the other values. Also, using recursion to solve this problem may not be the most optimal way to go since for large lists, I maybe blowing up the stack. Also, there might be stylistic issues with my code as well.
I would be great if somebody could point out other flaws and the right way to solve this problem.
Thanks
Suppose you split your list into three pieces: the item you are considering, those on the left, and those on the right. Suppose further that those on the left are in a sorted set. Now you just need to walk through the list, moving items from "right" to "considered" and from "considered" to "left"; at each point, you look at the size of the subset of the sorted set that is greater than your item. In general, the size lookup can be done in O(log(N)) as can the add-element (with a Red-Black or AVL tree, for instance). So you have O(N log N) performance.
Now the question is how to implement this in Scala efficiently. It turns out that Scala has a Red-Black tree used for its TreeSet sorted set, and the implementation is actually quite simple (here in tail-recursive form):
import collection.immutable.TreeSet
final def calcDisorder(xs: List[Int], left: TreeSet[Int] = TreeSet.empty, n: Int = 0): Int = xs match {
case Nil => n
case x :: rest => calcDisorder(rest, left + x, n + left.from(x).size)
}
Unfortunately, left.from(x).size takes O(N) time (I believe), which yields a quadratic execution time. That's no good--what you need is an IndexedTreeSet which can do indexOf(x) in O(log(n)) (and then iterate with n + left.size - left.indexOf(x) - 1). You can build your own implementation or find one on the web. For instance, I found one here (API here) for Java that does exactly the right thing.
Incidentally, the problem with doing a mergesort is that you cannot easily work cumulatively. With merging a pair, you can keep track of how out-of-order it is. But when you merge in a third list, you must see how out of order it is with respect to both other lists, which spoils your divide-and-conquer strategy. (I am not sure whether there is some invariant one could find that would allow you to calculate directly if you kept track of it.)
Here is my try, I don't use MergeSort but it seems to solve the problem:
def calcDisorderness(myList:List[Int]):Int = myList match{
case Nil => 0
case t::q => q.count(_<t) + calcDisorderness(q)
}
scala> val input = List(1,2,4,5,3,6)
input: List[Int] = List(1, 2, 4, 5, 3, 6)
scala> calcDisorderness(input)
res1: Int = 2
The question is, is there a way to have a lower complexity?
Edit: tail recursive version of the same function and cool usage of default values in function arguments.
def calcDisorderness(myList:List[Int], disorder:Int=0):Int = myList match{
case Nil => disorder
case t::q => calcDisorderness(q, disorder + q.count(_<t))
}
A solution based on Merge Sort. Not super fast, potential slowdown could be in "xs.length".
def countSwaps(a: Array[Int]): Long = {
var disorder: Long = 0
def msort(xs: List[Int]): List[Int] = {
import Stream._
def merge(left: List[Int], right: List[Int], inc: Int): Stream[Int] = {
(left, right) match {
case (x :: xs, y :: ys) if x > y =>
cons(y, merge(left, ys, inc + 1))
case (x :: xs, _) => {
disorder += inc
cons(x, merge(xs, right, inc))
}
case _ => right.toStream
}
}
val n = xs.length / 2
if (n == 0)
xs
else {
val (ys, zs) = xs splitAt n
merge(msort(ys), msort(zs), 0).toList
}
}
msort(a.toList)
disorder
}
Another solution based on Merge Sort. Very fast: no FP or for-loop.
def countSwaps(a: Array[Int]): Count = {
var swaps: Count = 0
def mergeRun(begin: Int, run_len: Int, src: Array[Int], dst: Array[Int]) = {
var li = begin
val lend = math.min(begin + run_len, src.length)
var ri = begin + run_len
val rend = math.min(begin + run_len * 2, src.length)
var ti = begin
while (ti < rend) {
if (ri >= rend) {
dst(ti) = src(li); li += 1
swaps += ri - begin - run_len
} else if (li >= lend) {
dst(ti) = src(ri); ri += 1
} else if (a(li) <= a(ri)) {
dst(ti) = src(li); li += 1
swaps += ri - begin - run_len
} else {
dst(ti) = src(ri); ri += 1
}
ti += 1
}
}
val b = new Array[Int](a.length)
var run = 0
var run_len = 1
while (run_len < a.length) {
var begin = 0
while (begin < a.length) {
val (src, dst) = if (run % 2 == 0) (a, b) else (b, a)
mergeRun(begin, run_len, src, dst)
begin += run_len * 2
}
run += 1
run_len *= 2
}
swaps
}
Convert the above code to Functional style: no mutable variable, no loop.
All recursions are tail calls, thus the performance is good.
def countSwaps(a: Array[Int]): Count = {
def mergeRun(li: Int, lend: Int, rb: Int, ri: Int, rend: Int, di: Int, src: Array[Int], dst: Array[Int], swaps: Count): Count = {
if (ri >= rend && li >= lend) {
swaps
} else if (ri >= rend) {
dst(di) = src(li)
mergeRun(li + 1, lend, rb, ri, rend, di + 1, src, dst, ri - rb + swaps)
} else if (li >= lend) {
dst(di) = src(ri)
mergeRun(li, lend, rb, ri + 1, rend, di + 1, src, dst, swaps)
} else if (src(li) <= src(ri)) {
dst(di) = src(li)
mergeRun(li + 1, lend, rb, ri, rend, di + 1, src, dst, ri - rb + swaps)
} else {
dst(di) = src(ri)
mergeRun(li, lend, rb, ri + 1, rend, di + 1, src, dst, swaps)
}
}
val b = new Array[Int](a.length)
def merge(run: Int, run_len: Int, lb: Int, swaps: Count): Count = {
if (run_len >= a.length) {
swaps
} else if (lb >= a.length) {
merge(run + 1, run_len * 2, 0, swaps)
} else {
val lend = math.min(lb + run_len, a.length)
val rb = lb + run_len
val rend = math.min(rb + run_len, a.length)
val (src, dst) = if (run % 2 == 0) (a, b) else (b, a)
val inc_swaps = mergeRun(lb, lend, rb, rb, rend, lb, src, dst, 0)
merge(run, run_len, lb + run_len * 2, inc_swaps + swaps)
}
}
merge(0, 1, 0, 0)
}
It seems to me that the key is to break the list into a series of ascending sequences. For example, your example would be broken into (1 2 4 5)(3 6). None of the items in the first list can end a pair. Now you do a kind of merge of these two lists, working backwards:
6 > 5, so 6 can't be in any pairs
3 < 5, so its a pair
3 < 4, so its a pair
3 > 2, so we're done
I'm not clear from the definition on how to handle more than 2 such sequences.
I was creating a faster string splitter method. First, I wrote a non-tail recursive version returning List. Next, a tail recursive one using ListBuffer and then calling toList (+= and toList are O(1)). I fully expected the tail recursive version to be faster, but that is not the case.
Can anyone explain why?
Original version:
def split(s: String, c: Char, i: Int = 0): List[String] = if (i < 0) Nil else {
val p = s indexOf (c, i)
if (p < 0) s.substring(i) :: Nil else s.substring(i, p) :: split(s, c, p + 1)
}
Tail recursive one:
import scala.annotation.tailrec
import scala.collection.mutable.ListBuffer
def split(s: String, c: Char): Seq[String] = {
val buffer = ListBuffer.empty[String]
#tailrec def recurse(i: Int): Seq[String] = {
val p = s indexOf (c, i)
if (p < 0) {
buffer += s.substring(i)
buffer.toList
} else {
buffer += s.substring(i, p)
recurse(p + 1)
}
}
recurse(0)
}
This was benchmarked with code here, with results here, by #scala's jyxent.
You're simply doing more work in the second case. In the first case, you might overflow your stack, but every operation is really simple, and :: is as small of a wrapper as you can get (all you have to do is create the wrapper and point it to the head of the other list). In the second case, not only do you create an extra collection initially and have to form a closure around s and buffer for the nested method to use, but you also use the heavierweight ListBuffer which has to check for each += whether it's already been copied out to a list, and uses different code paths depending on whether it's empty or not (in order to get the O(1) append to work).
You expect the tail recursive version to be faster due to the tail call optimization and I think this is right, if you compare apples to apples:
def split3(s: String, c: Char): Seq[String] = {
#tailrec def recurse(i: Int, acc: List[String] = Nil): Seq[String] = {
val p = s indexOf (c, i)
if (p < 0) {
s.substring(i) :: acc
} else {
recurse(p + 1, s.substring(i, p) :: acc)
}
}
recurse(0) // would need to reverse
}
I timed this split3 to be faster, except of course to get the same result it would need to reverse the result.
It does seem ListBuffer introduces inefficiencies that the tail recursion optimization cannot make up for.
Edit: thinking about avoiding the reverse...
def split3(s: String, c: Char): Seq[String] = {
#tailrec def recurse(i: Int, acc: List[String] = Nil): Seq[String] = {
val p = s lastIndexOf (c, i)
if (p < 0) {
s.substring(0, i + 1) :: acc
} else {
recurse(p - 1, s.substring(p + 1, i + 1) :: acc)
}
}
recurse(s.length - 1)
}
This has the tail call optimization and avoids ListBuffer.
Is there a simple and efficient solution to determine the top n elements of a Scala Iterable? I mean something like
iter.toList.sortBy(_.myAttr).take(2)
but without having to sort all elements when only the top 2 are of interest. Ideally I'm looking for something like
iter.top(2, _.myAttr)
see also: Solution for the top element using an Ordering: In Scala, how to use Ordering[T] with List.min or List.max and keep code readable
Update:
Thank you all for your solutions. Finally, I took the original solution of user unknown and adopted it to use Iterable and the pimp-my-library pattern:
implicit def iterExt[A](iter: Iterable[A]) = new {
def top[B](n: Int, f: A => B)(implicit ord: Ordering[B]): List[A] = {
def updateSofar (sofar: List [A], el: A): List [A] = {
//println (el + " - " + sofar)
if (ord.compare(f(el), f(sofar.head)) > 0)
(el :: sofar.tail).sortBy (f)
else sofar
}
val (sofar, rest) = iter.splitAt(n)
(sofar.toList.sortBy (f) /: rest) (updateSofar (_, _)).reverse
}
}
case class A(s: String, i: Int)
val li = List (4, 3, 6, 7, 1, 2, 9, 5).map(i => A(i.toString(), i))
println(li.top(3, _.i))
My solution (bound to Int, but should be easily changed to Ordered (a few minutes please):
def top (n: Int, li: List [Int]) : List[Int] = {
def updateSofar (sofar: List [Int], el: Int) : List [Int] = {
// println (el + " - " + sofar)
if (el < sofar.head)
(el :: sofar.tail).sortWith (_ > _)
else sofar
}
/* better readable:
val sofar = li.take (n).sortWith (_ > _)
val rest = li.drop (n)
(sofar /: rest) (updateSofar (_, _)) */
(li.take (n). sortWith (_ > _) /: li.drop (n)) (updateSofar (_, _))
}
usage:
val li = List (4, 3, 6, 7, 1, 2, 9, 5)
top (2, li)
For above list, take the first 2 (4, 3) as starting TopTen (TopTwo).
Sort them, such that the first element is the bigger one (if any).
repeatedly iterate through the rest of the list (li.drop(n)), and compare the current element with the maximum of the list of minimums; replace, if neccessary, and resort again.
Improvements:
Throw away Int, and use ordered.
Throw away (_ > _) and use a user-Ordering to allow BottomTen. (Harder: pick the middle 10 :) )
Throw away List, and use Iterable instead
update (abstraction):
def extremeN [T](n: Int, li: List [T])
(comp1: ((T, T) => Boolean), comp2: ((T, T) => Boolean)):
List[T] = {
def updateSofar (sofar: List [T], el: T) : List [T] =
if (comp1 (el, sofar.head))
(el :: sofar.tail).sortWith (comp2 (_, _))
else sofar
(li.take (n) .sortWith (comp2 (_, _)) /: li.drop (n)) (updateSofar (_, _))
}
/* still bound to Int:
def top (n: Int, li: List [Int]) : List[Int] = {
extremeN (n, li) ((_ < _), (_ > _))
}
def bottom (n: Int, li: List [Int]) : List[Int] = {
extremeN (n, li) ((_ > _), (_ < _))
}
*/
def top [T] (n: Int, li: List [T])
(implicit ord: Ordering[T]): Iterable[T] = {
extremeN (n, li) (ord.lt (_, _), ord.gt (_, _))
}
def bottom [T] (n: Int, li: List [T])
(implicit ord: Ordering[T]): Iterable[T] = {
extremeN (n, li) (ord.gt (_, _), ord.lt (_, _))
}
top (3, li)
bottom (3, li)
val sl = List ("Haus", "Garten", "Boot", "Sumpf", "X", "y", "xkcd", "x11")
bottom (2, sl)
To replace List with Iterable seems to be a bit harder.
As Daniel C. Sobral pointed out in the comments, a high n in topN can lead to much sorting work, so that it could be useful, to do a manual insertion sort instead of repeatedly sorting the whole list of top-n elements:
def extremeN [T](n: Int, li: List [T])
(comp1: ((T, T) => Boolean), comp2: ((T, T) => Boolean)):
List[T] = {
def sortedIns (el: T, list: List[T]): List[T] =
if (list.isEmpty) List (el) else
if (comp2 (el, list.head)) el :: list else
list.head :: sortedIns (el, list.tail)
def updateSofar (sofar: List [T], el: T) : List [T] =
if (comp1 (el, sofar.head))
sortedIns (el, sofar.tail)
else sofar
(li.take (n) .sortWith (comp2 (_, _)) /: li.drop (n)) (updateSofar (_, _))
}
top/bottom method and usage as above. For small groups of top/bottom Elements, the sorting is rarely called, a few times in the beginning, and then less and less often over time. For example, 70 times with top (10) of 10 000, and 90 times with top (10) of 100 000.
Here's another solution that is simple and has pretty good performance.
def pickTopN[T](k: Int, iterable: Iterable[T])(implicit ord: Ordering[T]): Seq[T] = {
val q = collection.mutable.PriorityQueue[T](iterable.toSeq:_*)
val end = Math.min(k, q.size)
(1 to end).map(_ => q.dequeue())
}
The Big O is O(n + k log n), where k <= n. So the performance is linear for small k and at worst n log n.
The solution can also be optimized to be O(k) for memory but O(n log k) for performance. The idea is to use a MinHeap to track only the top k items at all times. Here's the solution.
def pickTopN[A, B](n: Int, iterable: Iterable[A], f: A => B)(implicit ord: Ordering[B]): Seq[A] = {
val seq = iterable.toSeq
val q = collection.mutable.PriorityQueue[A](seq.take(n):_*)(ord.on(f).reverse) // initialize with first n
// invariant: keep the top k scanned so far
seq.drop(n).foreach(v => {
q += v
q.dequeue()
})
q.dequeueAll.reverse
}
Yet another version:
val big = (1 to 100000)
def maxes[A](n:Int)(l:Traversable[A])(implicit o:Ordering[A]) =
l.foldLeft(collection.immutable.SortedSet.empty[A]) { (xs,y) =>
if (xs.size < n) xs + y
else {
import o._
val first = xs.firstKey
if (first < y) xs - first + y
else xs
}
}
println(maxes(4)(big))
println(maxes(2)(List("a","ab","c","z")))
Using the Set force the list to have unique values:
def maxes2[A](n:Int)(l:Traversable[A])(implicit o:Ordering[A]) =
l.foldLeft(List.empty[A]) { (xs,y) =>
import o._
if (xs.size < n) (y::xs).sort(lt _)
else {
val first = xs.head
if (first < y) (y::(xs - first)).sort(lt _)
else xs
}
}
You don't need to sort the entire collection in order to determine the top N elements. However, I don't believe that this functionality is supplied by the raw library, so you would have to roll you own, possibly using the pimp-my-library pattern.
For example, you can get the nth element of a collection as follows:
class Pimp[A, Repr <% TraversableLike[A, Repr]](self : Repr) {
def nth(n : Int)(implicit ord : Ordering[A]) : A = {
val trav : TraversableLike[A, Repr] = self
var ltp : List[A] = Nil
var etp : List[A] = Nil
var mtp : List[A] = Nil
trav.headOption match {
case None => error("Cannot get " + n + " element of empty collection")
case Some(piv) =>
trav.foreach { a =>
val cf = ord.compare(piv, a)
if (cf == 0) etp ::= a
else if (cf > 0) ltp ::= a
else mtp ::= a
}
if (n < ltp.length)
new Pimp[A, List[A]](ltp.reverse).nth(n)(ord)
else if (n < (ltp.length + etp.length))
piv
else
new Pimp[A, List[A]](mtp.reverse).nth(n - ltp.length - etp.length)(ord)
}
}
}
(This is not very functional; sorry)
It's then trivial to get the top n elements:
def topN(n : Int)(implicit ord : Ordering[A], bf : CanBuildFrom[Repr, A, Repr]) ={
val b = bf()
val elem = new Pimp[A, Repr](self).nth(n)(ord)
import util.control.Breaks._
breakable {
var soFar = 0
self.foreach { tt =>
if (ord.compare(tt, elem) < 0) {
b += tt
soFar += 1
}
}
assert (soFar <= n)
if (soFar < n) {
self.foreach { tt =>
if (ord.compare(tt, elem) == 0) {
b += tt
soFar += 1
}
if (soFar == n) break
}
}
}
b.result()
}
Unfortunately I'm having trouble getting this pimp to be discovered via this implicit:
implicit def t2n[A, Repr <% TraversableLike[A, Repr]](t : Repr) : Pimp[A, Repr]
= new Pimp[A, Repr](t)
I get this:
scala> List(4, 3, 6, 7, 1, 2, 8, 5).topN(4)
<console>:9: error: could not find implicit value for evidence parameter of type (List[Int]) => scala.collection.TraversableLike[A,List[Int]]
List(4, 3, 6, 7, 1, 2, 8, 5).topN(4)
^
However, the code actually works OK:
scala> new Pimp(List(4, 3, 6, 7, 1, 2, 8, 5)).topN(4)
res3: List[Int] = List(3, 1, 2, 4)
And
scala> new Pimp("ioanusdhpisjdmpsdsvfgewqw").topN(6)
res2: java.lang.String = adddfe
If the goal is to not sort the whole list then you could do something like this (of course it could be optimized a tad so that we don't change the list when the number clearly shouldn't be there):
List(1,6,3,7,3,2).foldLeft(List[Int]()){(l, n) => (n :: l).sorted.take(2)}
I implemented such an ranking algorithm recently in the Rank class of Apache Jackrabbit (in Java though). See the take method for the gist of it. The basic idea is to quicksort but terminate prematurely as soon as the top n elements have been found.
Here is asymptotically O(n) solution.
def top[T](data: List[T], n: Int)(implicit ord: Ordering[T]): List[T] = {
require( n < data.size)
def partition_inner(shuffledData: List[T], pivot: T): List[T] =
shuffledData.partition( e => ord.compare(e, pivot) > 0 ) match {
case (left, right) if left.size == n => left
case (left, x :: rest) if left.size < n =>
partition_inner(util.Random.shuffle(data), x)
case (left # y :: rest, right) if left.size > n =>
partition_inner(util.Random.shuffle(data), y)
}
val shuffled = util.Random.shuffle(data)
partition_inner(shuffled, shuffled.head)
}
scala> top(List.range(1,10000000), 5)
Due to recursion, this solution will take longer than some non-linear solutions above and can cause java.lang.OutOfMemoryError: GC overhead limit exceeded.
But slightly more readable IMHO and functional style. Just for job interview ;).
What is more important, that this solution can be easily parallelized.
def top[T](data: List[T], n: Int)(implicit ord: Ordering[T]): List[T] = {
require( n < data.size)
#tailrec
def partition_inner(shuffledData: List[T], pivot: T): List[T] =
shuffledData.par.partition( e => ord.compare(e, pivot) > 0 ) match {
case (left, right) if left.size == n => left.toList
case (left, right) if left.size < n =>
partition_inner(util.Random.shuffle(data), right.head)
case (left, right) if left.size > n =>
partition_inner(util.Random.shuffle(data), left.head)
}
val shuffled = util.Random.shuffle(data)
partition_inner(shuffled, shuffled.head)
}
For small values of n and large lists, getting the top n elements can be implemented by picking out the max element n times:
def top[T](n:Int, iter:Iterable[T])(implicit ord: Ordering[T]): Iterable[T] = {
def partitionMax(acc: Iterable[T], it: Iterable[T]): Iterable[T] = {
val max = it.max(ord)
val (nextElems, rest) = it.partition(ord.gteq(_, max))
val maxElems = acc ++ nextElems
if (maxElems.size >= n || rest.isEmpty) maxElems.take(n)
else partitionMax(maxElems, rest)
}
if (iter.isEmpty) iter.take(0)
else partitionMax(iter.take(0), iter)
}
This does not sort the entire list and takes an Ordering. I believe every method I call in partitionMax is O(list size) and I only expect to call it n times at most, so the overall efficiency for small n will be proportional to the size of the iterator.
scala> top(5, List.range(1,1000000))
res13: Iterable[Int] = List(999999, 999998, 999997, 999996, 999995)
scala> top(5, List.range(1,1000000))(Ordering[Int].on(- _))
res14: Iterable[Int] = List(1, 2, 3, 4, 5)
You could also add a branch for when n gets close to size of the iterable, and switch to iter.toList.sortBy(_.myAttr).take(n).
It does not return the type of collection provided, but you can look at How do I apply the enrich-my-library pattern to Scala collections? if this is a requirement.
An optimised solution using PriorityQueue with Time Complexity of O(nlogk). In the approach given in the update, you are sorting the sofar list every time which is not needed and below it is optimised by using PriorityQueue.
import scala.language.implicitConversions
import scala.language.reflectiveCalls
import collection.mutable.PriorityQueue
implicit def iterExt[A](iter: Iterable[A]) = new {
def top[B](n: Int, f: A => B)(implicit ord: Ordering[B]) : List[A] = {
def updateSofar (sofar: PriorityQueue[A], el: A): PriorityQueue[A] = {
if (ord.compare(f(el), f(sofar.head)) < 0){
sofar.dequeue
sofar.enqueue(el)
}
sofar
}
val (sofar, rest) = iter.splitAt(n)
(PriorityQueue(sofar.toSeq:_*)( Ordering.by( (x :A) => f(x) ) ) /: rest) (updateSofar (_, _)).dequeueAll.toList.reverse
}
}
case class A(s: String, i: Int)
val li = List (4, 3, 6, 7, 1, 2, 9, 5).map(i => A(i.toString(), i))
println(li.top(3, -_.i))