How to convert between Hilbert Curve QuadTree and S2 Geometry CellId - hilbert-curve

Problem
Let's say I know the Hilbert Curve Face and Quadtree, such as 4/032212303102122 (face 4, level 15).
Or perhaps I know the S2 Geometry CellId, such as 9749618424903892992.
How can I convert from the one to the other?
Application
(this is the kind of thing you need to do for Pokemon GO and Ingress maps)
Exploration
I'm trying to do this in JavaScript and a library exists for manipulating 64-bit integers (long.js) as well as for S2CellIds (s2-geometry.js).
Also, I'm feeling pretty good about walking the hilbert curve simply by adding or subtracting the base four numbers (except when crossing faces, but that happens rarely enough that I'll be fine... for a while...), just not sure how to go back and forth with the 64-bit id.

It turns out that it's much, much, much easier to do it with strings than with binary - and since this is JavaScript where bitshifting with the long.js would take significantly more time, it's actually faster!
Code Example:
From s2-geometry-javascript:
'use strict';
var Long = require('long');
var S2 = {};
S2.FACE_BITS = 3;
S2.MAX_LEVEL = 30;
S2.POS_BITS = (2 * S2.MAX_LEVEL) + 1;
S2.fromFacePosLevel = function (faceN, posS, levelN) {
var Long = exports.dcodeIO && exports.dcodeIO.Long || require('long');
if (!levelN) {
levelN = posS.length;
}
if (posS.length > levelN) {
posS = posS.substr(0, levelN);
}
var posB = Long.fromString(posS, true, 4).toString(2);
while (posB.length < (2 * levelN)) {
posB = '0' + posB;
}
var bin = Long.fromString(faceN.toString(10), true, 10).toString(2);
while (bin.length < S2.FACE_BITS) {
bin = '0' + bin;
}
bin += posB;
bin += '1';
while (bin.length < (S2.FACE_BITS + S2.POS_BITS)) {
bin += '0';
}
return Long.fromString(bin, true, 2).toString(10);
};
Explanation:
Here's a quick 'n' dirty breakdown of the bits
id encoding
Note that + means concat and NOT add
(padding + face bits) + (padding + position bits) + (lsb marker + padding)
// quadkey 4/032212303102210
// id (base 10) 9749618446378729472
// base 4 10 032212303102210 1000000000000000
// base 2 100 001110100110110011010010100100 1000000000000000000000000000000
face encoding
"human readable" form is base 10
3-bit - i.e. an unfolded 6-sided cube with base 10 face representations of 0,1,2,3,4,5
6 and 7 are unused and invalid
3 binary characters - i.e. 000, 001, 010, 011, 100, 101
110 and 111 are unused and invalid
left-padded to 3-bits with '0's (i.e. 001)
position encoding
"human readable" form is base 4 (quadkey)
61-bit
60 data bits, 1 bit for lsb marker
left-padded to LEVEL with '0's (i.e. 00322130 for level 8)
level encoding
"human readable" form is base 10
the length of hilbert curve quadkey / quadtree string is the level
calculated from the least significant bit in binary form
lsb (least-significant bit) marker is '1', just to right of position
right-padded to MAX_LEVEL*2 (after lsb marker) with a leading '0's
(i.e. '1' for level 30, '1000' for level 27)

Related

How to generate number between a low and a high number using just a bit?

I was asked this question in a interview, so I don't want the solution, just the guidance regarding how to approach it.
You have been given two numbers low and high. And a random generator which generates 0 and 1. I have to generate a number between low and high using that function.
I can get difference between the two numbers and somehow try to generate a number using bit manipulation. But I am not able to figure out how to do that?
You can do:
range = high - low
find n such that 2^n-1 < range <= 2^n
run the random generator n times to generate an int thanks to its binary representation. Something like 010011010 (= 154 in decimal)
add the obtained number to low to get your final number!
Here's a basic bit-by-bit comparison algorithm that gives a random number between low and high, using a random-bit function:
Decrease high by 1 and increase low by 1 (in case the random bits introduced later all equal those in high or low).
Create booleans high_dec and low_inc to store whether at least one 1 in high has been changed into 0, and at least one 0 in low has been changed into 1, and set both of them to false (these will help avoid the result going out of range).
Compare high and low bit-by-bit from MSB to LSB with these cases:
If you find high:1 and low:1 then store a 1 if low_inc=false or store a random bit otherwise (and update high_dec as necessary).
If you find high:1 and low:0 then store a random bit (and update high_dec or low_inc as necessary).
If you find high:0 and low:1 then store a 0 if high_dec=false or store a 1 if low_inc=false or store a random bit otherwise.
If you find high:0 and low:0 then store a 0 if high_dec=false or store a random bit otherwise (and update low_inc as necessary).
Note that the distribution of the random numbers is only uniform if the lowest possible result is a power of 2, and the range is a power of 2. In all cases the whole range is used, but there may be an emphasis on values near the beginning or end of the range.
function between(a, b) {
var lo = (a + 1).toString(2).split(''), // conversion to bit array because
hi = (b - 1).toString(2).split(''), // there is no bit manipulation in JS
lc = false, // low changed
hc = false, // high changed
result = [];
while (lo.length < hi.length) lo.unshift(0); // add leading zeros to low
for (var i = 0; i < hi.length; i++) { // iterate over bits, msb to lsb
var bit = Math.round(Math.random()); // random bit generator
if (hi[i] == 1) {
if (lo[i] == 1) { // case hi:1 lo:1
if (lc == false) bit = 1
else if (bit == 0) hc = true;
} else { // case hi:1 lo:0
if (bit == 0) hc = true
else lc = true;
}
} else {
if (lo[i] == 1) { // case hi:0 lo:1
if (hc == false) bit = 0
else if (lc == false) bit = 1;
} else { // case hi:0 lo:0
if (hc == false) bit = 0
else if (bit == 1) lc = true;
}
}
result.push(bit);
}
return parseInt(result.join(''), 2); // convert bit array to integer
}
document.write(between(999999, 1000100) + "<BR>");

Take exact root of value

I'd like to know how you can take the exact n-th root of a number (in any programming language). When I use a physical calculator, I can type something like sqrt(12) (nicely formatted of course) and get as a result 2 sqrt(3). How can I achieve this not only with square roots but any type of root when representing a number as numerator and denominator. Of course, I would have to use another representation, but I don't have any idea how this works in general.
Thanks in advance.
I doubt this is an efficient way, but it would work. Assuming you want to take the nth root of some number m:
Calculate the prime factorization m = p1a1 * p2a2 * ... * pxax.
For each 1 <= i <= x let ki = ai div n and ri = ai mod n.
The part that gets factored out is then p1k1 * p2k2 * ... * pxkx.
The part that remains "under the root" is p1r1 * p2r2 * ... * pxrx.
The first step is the only tricky one. Once you have found all prime factors of m it is just a matter of looping over those factors and dividing out the multiples of n.
To simplify the n-th root of a number, the algorithm shouldn't do prime factorisation, but rather "n-th power factorisation", i.e. look for the largest n-th power inside the root, which you can then move outside the root. For example: the 3rd root of 250 equals the third root of 2 x 125; since 125 is the third power of 5, you can move it out of the root and get: 5 times the third root of 2.
Algorithm: take the floating-point n-th root of the number, and round it down, then check this and all smaller integers until you find the largest integer whose n-th power divides the number; then divide the number by the n-th power and move the integer out of the root.
This javascript example shows a basic implementation; you could clean it up further by printing 11/root simply as 1; further optimisation is undoubtedly possible.
function integerRoot(number, root) {
var base = number, factor = 1;
var max = Math.floor(Math.pow(base, 1/root));
for (var i = max; i > 1; i--) {
var power = Math.pow(i, root);
if (base % power == 0) {
base /= power;
factor *= i;
break;
}
}
document.write(number + "<SUP>1/" + root + "</SUP> = " +
factor + " × " + base + "<SUP>1/" + root + "</SUP><BR>");
}
integerRoot(25, 3);
integerRoot(27, 3);
integerRoot(81, 3);
integerRoot(135, 3);
integerRoot(375, 3);
integerRoot(8*27*64*17, 3);
UPDATE: This is a more efficient version; I haven't yet taken negative numbers into account, though, so there's definitely room for further improvement.
function simplifyRoot(radicand, degree) {
var factor = 1, base = 1, power;
while ((power = Math.pow(++base, degree)) <= radicand) {
while (radicand % power == 0) {
factor *= base;
radicand /= power;
}
}
return {factor: factor, radicand: radicand, degree: degree};
}
var radicand = 8*27*36*64*125*216, degree = 3;
var simplified = simplifyRoot(radicand, degree);
document.write(radicand + "<SUP>1/" + degree + "</SUP> = " +
simplified.factor + " × " + simplified.radicand + "<SUP>1/" + simplified.degree + "</SUP><BR>");

Efficient way to generate a seemingly random permutation from a very large set without repeating?

I have a very large set (billions or more, it's expected to grow exponentially to some level), and I want to generate seemingly random elements from it without repeating. I know I can pick a random number and repeat and record the elements I have generated, but that takes more and more memory as numbers are generated, and wouldn't be practical after couple millions elements out.
I mean, I could say 1, 2, 3 up to billions and each would be constant time without remembering all the previous, or I can say 1,3,5,7,9 and on then 2,4,6,8,10, but is there a more sophisticated way to do that and eventually get a seemingly random permutation of that set?
Update
1, The set does not change size in the generation process. I meant when the user's input increases linearly, the size of the set increases exponentially.
2, In short, the set is like the set of every integer from 1 to 10 billions or more.
3, In long, it goes up to 10 billion because each element carries the information of many independent choices, for example. Imagine an RPG character that have 10 attributes, each can go from 1 to 100 (for my problem different choices can have different ranges), thus there's 10^20 possible characters, number "10873456879326587345" would correspond to a character that have "11, 88, 35...", and I would like an algorithm to generate them one by one without repeating, but makes it looks random.
Thanks for the interesting question. You can create a "pseudorandom"* (cyclic) permutation with a few bytes using modular exponentiation. Say we have n elements. Search for a prime p that's bigger than n+1. Then find a primitive root g modulo p. Basically by definition of primitive root, the action x --> (g * x) % p is a cyclic permutation of {1, ..., p-1}. And so x --> ((g * (x+1))%p) - 1 is a cyclic permutation of {0, ..., p-2}. We can get a cyclic permutation of {0, ..., n-1} by repeating the previous permutation if it gives a value bigger (or equal) n.
I implemented this idea as a Go package. https://github.com/bwesterb/powercycle
package main
import (
"fmt"
"github.com/bwesterb/powercycle"
)
func main() {
var x uint64
cycle := powercycle.New(10)
for i := 0; i < 10; i++ {
fmt.Println(x)
x = cycle.Apply(x)
}
}
This outputs something like
0
6
4
1
2
9
3
5
8
7
but that might vary off course depending on the generator chosen.
It's fast, but not super-fast: on my five year old i7 it takes less than 210ns to compute one application of a cycle on 1000000000000000 elements. More details:
BenchmarkNew10-8 1000000 1328 ns/op
BenchmarkNew1000-8 500000 2566 ns/op
BenchmarkNew1000000-8 50000 25893 ns/op
BenchmarkNew1000000000-8 200000 7589 ns/op
BenchmarkNew1000000000000-8 2000 648785 ns/op
BenchmarkApply10-8 10000000 170 ns/op
BenchmarkApply1000-8 10000000 173 ns/op
BenchmarkApply1000000-8 10000000 172 ns/op
BenchmarkApply1000000000-8 10000000 169 ns/op
BenchmarkApply1000000000000-8 10000000 201 ns/op
BenchmarkApply1000000000000000-8 10000000 204 ns/op
Why did I say "pseudorandom"? Well, we are always creating a very specific kind of cycle: namely one that uses modular exponentiation. It looks pretty pseudorandom though.
I would use a random number and swap it with an element at the beginning of the set.
Here's some pseudo code
set = [1, 2, 3, 4, 5, 6]
picked = 0
Function PickNext(set, picked)
If picked > Len(set) - 1 Then
Return Nothing
End If
// random number between picked (inclusive) and length (exclusive)
r = RandomInt(picked, Len(set))
// swap the picked element to the beginning of the set
result = set[r]
set[r] = set[picked]
set[picked] = result
// update picked
picked++
// return your next random element
Return temp
End Function
Every time you pick an element there is one swap and the only extra memory being used is the picked variable. The swap can happen if the elements are in a database or in memory.
EDIT Here's a jsfiddle of a working implementation http://jsfiddle.net/sun8rw4d/
JavaScript
var set = [];
set.picked = 0;
function pickNext(set) {
if(set.picked > set.length - 1) { return null; }
var r = set.picked + Math.floor(Math.random() * (set.length - set.picked));
var result = set[r];
set[r] = set[set.picked];
set[set.picked] = result;
set.picked++;
return result;
}
// testing
for(var i=0; i<100; i++) {
set.push(i);
}
while(pickNext(set) !== null) { }
document.body.innerHTML += set.toString();
EDIT 2 Finally, a random binary walk of the set. This can be accomplished with O(Log2(N)) stack space (memory) which for 10billion is only 33. There's no shuffling or swapping involved. Using trinary instead of binary might yield even better pseudo random results.
// on the fly set generator
var count = 0;
var maxValue = 64;
function nextElement() {
// restart the generation
if(count == maxValue) {
count = 0;
}
return count++;
}
// code to pseudo randomly select elements
var current = 0;
var stack = [0, maxValue - 1];
function randomBinaryWalk() {
if(stack.length == 0) { return null; }
var high = stack.pop();
var low = stack.pop();
var mid = ((high + low) / 2) | 0;
// pseudo randomly choose the next path
if(Math.random() > 0.5) {
if(low <= mid - 1) {
stack.push(low);
stack.push(mid - 1);
}
if(mid + 1 <= high) {
stack.push(mid + 1);
stack.push(high);
}
} else {
if(mid + 1 <= high) {
stack.push(mid + 1);
stack.push(high);
}
if(low <= mid - 1) {
stack.push(low);
stack.push(mid - 1);
}
}
// how many elements to skip
var toMid = (current < mid ? mid - current : (maxValue - current) + mid);
// skip elements
for(var i = 0; i < toMid - 1; i++) {
nextElement();
}
current = mid;
// get result
return nextElement();
}
// test
var result;
var list = [];
do {
result = randomBinaryWalk();
list.push(result);
} while(result !== null);
document.body.innerHTML += '<br/>' + list.toString();
Here's the results from a couple of runs with a small set of 64 elements. JSFiddle http://jsfiddle.net/yooLjtgu/
30,46,38,34,36,35,37,32,33,31,42,40,41,39,44,45,43,54,50,52,53,51,48,47,49,58,60,59,61,62,56,57,55,14,22,18,20,19,21,16,15,17,26,28,29,27,24,25,23,6,2,4,5,3,0,1,63,10,8,7,9,12,11,13
30,14,22,18,16,15,17,20,19,21,26,28,29,27,24,23,25,6,10,8,7,9,12,13,11,2,0,63,1,4,5,3,46,38,42,44,45,43,40,41,39,34,36,35,37,32,31,33,54,58,56,55,57,60,59,61,62,50,48,49,47,52,51,53
As I mentioned in my comment, unless you have an efficient way to skip to a specific point in your "on the fly" generation of the set this will not be very efficient.
if it is enumerable then use a pseudo-random integer generator adjusted to the period 0 .. 2^n - 1 where the upper bound is just greater than the size of your set and generate pseudo-random integers discarding those more than the size of your set. Use those integers to index items from your set.
Pre- compute yourself a series of indices (e.g. in a file), which has the properties you need and then randomly choose a start index for your enumeration and use the series in a round-robin manner.
The length of your pre-computed series should be > the maximum size of the set.
If you combine this (depending on your programming language etc.) with file mappings, your final nextIndex(INOUT state) function is (nearly) as simple as return mappedIndices[state++ % PERIOD];, if you have a fixed size of each entry (e.g. 8 bytes -> uint64_t).
Of course, the returned value could be > your current set size. Simply draw indices until you get one which is <= your sets current size.
Update (In response to question-update):
There is another option to achieve your goal if it is about creating 10Billion unique characters in your RPG: Generate a GUID and write yourself a function which computes your number from the GUID. man uuid if you are are on a unix system. Else google it. Some parts of the uuid are not random but contain meta-info, some parts are either systematic (such as your network cards MAC address) or random, depending on generator algorithm. But they are very very most likely unique. So, whenever you need a new unique number, generate a uuid and transform it to your number by means of some algorithm which basically maps the uuid bytes to your number in a non-trivial way (e.g. use hash functions).

Converting data from 8 bits to 12 bits

I am getting signal that is stored as a buffer of char data (8 bits).
I am also getting the same signal plus 24 dB and my boss told me that it should be possible to reconstruct from those two buffers, one (which will be used as output) that will be stored as 12 bits.
I would like to know the mathematical operation that can do that and why choosing +24dB.
Thanks (I am dumb ><).
From the problem statement, I guess you have an analog signal which are sampled at two amlitudes. Both signals has a resolution of 8 bits, but one is shifted and truncated.
You could get a 12 bit signal by combining the upper 4 bits of the first signal, and concatenating them with the second signal.
sOut = ((sIn1 & 0xF0) << 4) | sIn2
If you want to get a little better accuracy, you could try to calculate an average over the common bits of the two signals. Normally, the lower 4 bits of the first signal should be approximately equal to the upper 4 bits of the second signal. Due to rounding-errors or noise, the values could be slightly different. One of the values could even have overflowed, and moved to the other end of the range.
int Combine(byte sIn1, byte sIn2)
{
int a = sIn1 >> 4; // Upper 4 bits
int b1 = sIn1 & 0x0F; // Common middle 4 bits
int b2 = sIn2 >> 4; // Common middle 4 bits
int c = sIn2 & 0x0F; // Lower 4 bits
int b;
if (b1 >= 12 && b2 < 4)
{
// Assume b2 has overflowed, and wrapped around to a smaller value.
// We need to add 16 to it to compensate the average.
b = (b1 + b2 + 16)/2;
}
else if (b1 < 4 && b2 >= 12)
{
// Assume b2 has underflowed, and wrapped around to a larger value.
// We need to subtract 16 from it to compensate the average.
b = (b1 + b2 - 16)/2;
}
else
{
// Neither or both has overflowed. Just take the average.
b = (b1 + b2)/2;
}
// Construct the combined signal.
return a * 256 + b * 16 + c;
}
When I tested this, it reproduced the signal accurately more often than the first formula.

Graph (Chart) Algorithm

Does anyone have a decent algorithm for calculating axis minima and maxima?
When creating a chart for a given set of data items, I'd like to be able to give the algorithm:
the maximum (y) value in the set
the minimum (y) value in the set
the number of tick marks to appear on the axis
an optional value that must appear as a tick (e.g. zero when showing +ve and -ve values)
The algorithm should return
the largest axis value
the smallest axis value (although that could be inferred from the largest, the interval size and the number of ticks)
the interval size
The ticks should be at a regular interval should be of a "reasonable" size (e.g. 1, 3, 5, possibly even 2.5, but not any more sig figs).
The presence of the optional value will skew this, but without that value the largest item should appear between the top two tick marks, the lowest value between the bottom two.
This is a language-agnostic question, but if there's a C#/.NET library around, that would be smashing ;)
OK, here's what I came up with for one of our applications. Note that it doesn't deal with the "optional value" scenario you mention, since our optional value is always 0, but it shouldn't be hard for you to modify.
Data is continually added to the series so we just keep the range of y values up to date by inspecting each data point as its added; this is very inexpensive and easy to keep track of. Equal minimum and maximum values are special cased: a spacing of 0 indicates that no markers should be drawn.
This solution isn't dissimilar to Andrew's suggestion above, except that it deals, in a slightly kludgy way with some arbitrary fractions of the exponent multiplier.
Lastly, this sample is in C#. Hope it helps.
private float GetYMarkerSpacing()
{
YValueRange range = m_ScrollableCanvas.
TimelineCanvas.DataModel.CurrentYRange;
if ( range.RealMinimum == range.RealMaximum )
{
return 0;
}
float absolute = Math.Max(
Math.Abs( range.RealMinimum ),
Math.Abs( range.RealMaximum ) ),
spacing = 0;
for ( int power = 0; power < 39; ++power )
{
float temp = ( float ) Math.Pow( 10, power );
if ( temp <= absolute )
{
spacing = temp;
}
else if ( temp / 2 <= absolute )
{
spacing = temp / 2;
break;
}
else if ( temp / 2.5 <= absolute )
{
spacing = temp / 2.5F;
break;
}
else if ( temp / 4 <= absolute )
{
spacing = temp / 4;
break;
}
else if ( temp / 5 <= absolute )
{
spacing = temp / 5;
break;
}
else
{
break;
}
}
return spacing;
}
I've been using the jQuery flot graph library. It's open source and does axis/tick generation quite well. I'd suggest looking at it's code and pinching some ideas from there.
I can recommend the following:
Set a visually appealing minimum number of major lines. This will depend on the nature of the data that you're presenting and the size of the plot you're doing, but 7 is a pretty good number
Choose the exponent and the multiplier based on a progression of 1, 2, 5, 10, etc. that will give you at least the minimum number of major lines. (ie. (max-min)/(scale x 10^exponent) >= minimum_tick_marks)
Find the minimum integer multiple of your exponent and multiplier that fits within your range. This will be the first major tick. The rest of the ticks are derived from this.
This was used for an application that allowed arbitrary scaling of data an seemed to work well.

Resources