Concatenation of iterators - algorithm

Question

I saw this example in "Programming in Scala" chapter 24 "Collections in depth". This example shows two alternative ways to implement a tree:
by extending Traversable[Int] - here the complexity of def foreach[U](f: Int => U): Unit would be O(N).
by extending Iterable[Int] - here the complexity of def iterator: Iterator[Int] would be O(N log(N)).
This is to demonstrate why it would be helpful to have two separate traits, Traversable and Iterable.
sealed abstract class Tree
case class Branch(left: Tree, right: Tree) extends Tree
case class Node(elem: Int) extends Tree
sealed abstract class Tree extends Traversable[Int] {
def foreach[U](f: Int => U) = this match {
case Node(elem) => f(elem)
case Branch(l, r) => l foreach f; r foreach f
}
}
sealed abstract class Tree extends Iterable[Int] {
def iterator: Iterator[Int] = this match {
case Node(elem) => Iterator.single(elem)
case Branch(l, r) => l.iterator ++ r.iterator
}
}
Regarding the implementation of foreach they say:
traversing a balanced tree takes time proportional to the number of
elements in the tree. To see this, consider that for a balanced tree
with N leaves you will have N - 1 interior nodes of class Branch. So
the total number of steps to traverse the tree is N + N - 1.
That makes sense. :)
However, they mention that the concatenation of the two iterators in the iterator method has time complexity of log(N), so the total complexity of the method would be N log(N):
Every time an element is produced by a concatenated iterator such as
l.iterator ++ r.iterator, the computation needs to follow one
indirection to get at the right iterator (either l.iterator, or
r.iterator). Overall, that makes log(N) indirections to get at a leaf
of a balanced tree with N leaves. So the cost of visiting all elements of a tree went up from about 2N for the foreach traversal method to N log(N) for the traversal with iterator.
????
Why does the computation of the concatenated iterator need to get at a leaf of the left or right iterator?

Answer 1

The pun on "collections in depth" is apt. The depth of the data structure matters.
When you invoke top.iterator.next(), each interior Branch delegates to the iterator of the Branch or Node below it, a call chain which is log(N).
You incur that call chain on every next().
Using foreach, you visit each Branch or Node just once.
Edit: Not sure if this helps, but here is an example of eagerly locating the leaves but lazily producing the values. It would stackoverflow or be slower in older versions of Scala, but the implementation of chained ++ was improved. Now it's a flat chain that gets shorter as it's consumed.
sealed abstract class Tree extends Iterable[Int] {
def iterator: Iterator[Int] = {
def leafIterator(t: Tree): List[Iterator[Int]] = t match {
case Node(_) => t.iterator :: Nil
case Branch(left, right) => leafIterator(left) ::: leafIterator(right)
}
this match {
case n # Node(_) => Iterator.fill(1)(n.value)
case Branch(left # Node(_), right # Node(_)) => left.iterator ++ right.iterator
case b # Branch(_, _) =>
leafIterator(b).foldLeft(Iterator[Int]())((all, it) => all ++ it)
}
}
}
case class Branch(left: Tree, right: Tree) extends Tree {
override def toString = s"Branch($left, $right)"
}
case class Node(elem: Int) extends Tree {
def value = {
Console println "An expensive leaf calculation"
elem
}
override def toString = s"Node($elem)"
}
object Test extends App {
// many leaves
val n = 1024 * 1024
val ns: List[Tree] = (1 to n).map(Node(_)).toList
var b = ns
while (b.size > 1) {
b = b.grouped(2).map { case left :: right :: Nil => Branch(left, right) }.toList
}
Console println s"Head: ${b.head.iterator.take(3).toList}"
}

Answer 2

In this implementation, the topmost branch does NOT know how many elements there are in its left and right sub-branches.
Therefore, the iterator is built recursively with the divide and conquer approach which is clearly represented in the iterator method - you get to each node (case Branch), you produce the iterator of the single node case Node => ... and then you join them.
Without getting into each and every node, it would not know what elements there are and how the tree is structured (odd branches allowed vs not allowed etc.).
EDIT:
Let's have a look inside the ++ method on Iterator.
def ++[B >: A](that: => GenTraversableOnce[B]): Iterator[B] = new Iterator.JoinIterator(self, that)
and then at Iterator.JoinIterator
private[scala] final class JoinIterator[+A](lhs: Iterator[A], that: => GenTraversableOnce[A]) extends Iterator[A] {
private[this] var state = 0 // 0: lhs not checked, 1: lhs has next, 2: switched to rhs
private[this] lazy val rhs: Iterator[A] = that.toIterator
def hasNext = state match {
case 0 =>
if (lhs.hasNext) {
state = 1
true
} else {
state = 2
rhs.hasNext
}
case 1 => true
case _ => rhs.hasNext
}
def next() = state match {
case 0 =>
if (lhs.hasNext) lhs.next()
else {
state = 2
rhs.next()
}
case 1 =>
state = 0
lhs.next()
case _ =>
rhs.next()
}
override def ++[B >: A](that: => GenTraversableOnce[B]) =
new ConcatIterator(this, Vector(() => that.toIterator))
}
From that we can see that joining iterators just creates a recursive structure in the rhs field. Furthermore, let's focus on it a bit more.
Consider an even tree with structure level1 [A]; level2 [B][C]; level 3[D][E][F][F]
When you call JoinIterator on the iterator you preserve the existing lhs iterator. However, you always .toIterator on rhs. Which means that for each subsequent level, the rhs part will be reconstructed. So for B ++ C you get that looks like A.lhs (stands for B) and A.rhs (stands for C.toIterator) where C.toIterator stands for C.lhs and C.rhs etc. Thus, the added complexity.
I hope this answers your question.