Tree Hash: How to verify if a range is tree-hash-aligned? - algorithm

Question

"Tree Hash" is a concept similar to Merkle Tree/Tiger Hash Tree used by Amazon Glacier to verify the data integrity of a subsets of a given datastream.
In order to receive tree hashes from Amazon Glacier when retrieving data, the specified byte range has to be "tree hash aligned".
The concept of "tree hash aligned" is described here.
Quoting from the developer documentation:
A range [A, B] is tree-hash aligned with respect to an archive if and only if when a new tree hash is built over [A, B], the root of the tree hash of that range is equivalent to a node in the tree hash of the whole archive. [...]
Consider [P, Q) as the range query for an archive of N megabytes (MB) and P and Q are multiples of one MB. Note that the actual inclusive range is [P MB, Q MB – 1 byte], but for simplicity, we show it as [P, Q). With these considerations, then
If P is an odd number, there is only one possible tree-hash aligned range—that is [P, P + 1 MB).
If P is an even number and k is the maximum number, where P can be written as 2k * X, then there are at most k tree-hash aligned ranges that start with P. X is an integer greater than 0. The tree-hash aligned ranges fall in the following categories:
For each i, where (0 <= i <= k) and where P + 2i < N, then [P, Q + 2i) is a tree-hash aligned range.
P = 0 is the special case where A = 2[lgN]*0
Now the question: How do I verify programmatically if a given range [startByte, endByte] is tree-hash-aligned? Programming language does not matter.
Test cases:
[0,0) => true
[0,1) => true
[0,2) => false
[0,3) => true
[1,2) => false
[4,5) => true

Answer 1

Here a basic implementation of the is_treehash_aligned function in Python:
import math
def max_k(x):
return 1 + max_k(x/2) if x % 2 == 0 else 0
def is_treehash_aligned(P, Q):
if (Q < P):
return False
elif (P % 2 == 1):
return Q == P
else:
ilen = Q - P + 1 # size(interval)
if not (((ilen & (ilen - 1)) == 0) and ilen != 0):
return False # size(interval) ~ not power of two
if P == 0:
return True
else:
k = max_k(P)
i = int(math.log(ilen, 2))
return i <= k
if (__name__ == "__main__"):
ranges = [(0, 0), (0, 1), (0, 2), (0, 3), (1, 2), \
(4, 5), (6, 7), (2, 4), (6, 8), (5, 6), \
(4, 4), (1, 1), (4194304, 5242879), \
(4194304, 5242880), (4194304, 5242881)]
for r in ranges:
ret = is_treehash_aligned(*r)
print("[" + str(r[0]) + ", " + str(r[1]) + ") => " + str(ret))
The output is:
[0, 0) => True
[0, 1) => True
[0, 2) => False
[0, 3) => True
[1, 2) => False
[4, 5) => True
[6, 7) => True
[2, 4) => False
[6, 8) => False
[5, 6) => False
[4, 4) => True
[1, 1) => True
[4194304, 5242879) => True
[4194304, 5242880) => False
[4194304, 5242881) => False
Note that:
I adopted your notation for intervals rather than the one provided by the instructions. As a consequence, it is possible to assume that each interval is Megabyte aligned.
The result for the test-case [4194304, 5242880) differs from what you put in your original question, though I double-checked it and I am somewhat confident it is correct.
if N is known, which is not the case in your test-cases, then when P == 0 one should also accept any range s.t. Q >= floor(N), and not only those with a size that is a power of two. A similar argument could be made for sub-trees for which there is nothing else on the right. Both of these cases would match the definition of Tree-Hash Alignment given here, but not the instructions for identifying it.
Notes: both the question and the description of the problem appear to be albeit confusing.
The test cases are given with the notation [A, B) where A is the index of the starting block and B is the index of the ending block (included), assuming that the whole archive is comprised by an array --indexed starting from 0-- of N blocks size 1 MB each (except possibly the last one). E.g.:
[0,0) => true
[0,1) => true
[0,2) => false
[0,3) => true
[1,2) => false
[4,5) => true
However, the instructions assume that the ranges are given with the notation [P MB, Q MB – 1 byte].
The instructions are misleading.
For example, here it says:
If P is an even number and k is the maximum number, where P can be written as 2k * X, then there are at most k tree-hash aligned ranges that start with P
The power symbol appears to be omitted, perhaps due to wrong HTML code, as the sentence should be "the largest k s.t. P = (2^k)*X".
Another example is:
For each i, where (0 <= i <= k) and where P + 2i < N, then [P, Q + 2i) is a tree-hash aligned range.
Assume for example that Q = P + 1, i > 0 and k > 0. Then the interval [P, Q + 2^i) has size = Q + 2^i - P = P + 1 + 2^i - P = 2^i + 1 > 1. However, by construction there exists no such tree-hash aligned range with an odd size larger than one. The proposition should be: "[...], then [P, P + 2^i) is a tree-hash aligned range".