How double-hashing works in leveldb? - leveldb

When I read the implementation of bloom filter in leveldb, I found it use a trick to apply double-hashing.
source
// Use double-hashing to generate a sequence of hash values.
// See analysis in [Kirsch,Mitzenmacher 2006].
uint32_t h = BloomHash(keys[i]);
const uint32_t delta = (h >> 17) | (h << 15); // Rotate right 17 bits
for (size_t j = 0; j < k_; j++) {
const uint32_t bitpos = h % bits;
array[bitpos/8] |= (1 << (bitpos % 8));
h += delta;
}
There is no second hash function, but just rotate 17 bits.
I'm wondering how the 17 chosen?
And, if I want to use the same trick to a uint64_t, how many bits should I rotate?

Related

How to expand the product of a sequence of binomials efficiently?

The product of the sequence of binomials reads
where {a_i} and {b_i} are coefficients in binomials.
I need to expand it to a polynomial
and use all coefficients {c_k} in the polynomial afterwards.
How to expand it efficiently? The speed has priority over the memory occupation because the expansion will be used many times.
What I tried
At present I just come up with an update scheme, which expands the polynomial right after absorbing one binomial.
This scheme needs two arrays — one for results up to i-1 and the other for results up to i.
Here is the C++ code for my naive scheme, but I think this question is irrelevant to what language is used.
#include <iostream>
#include <vector>
int main()
{
using namespace std;
// just an example, the coefficients are actually real numbers in [0,1]
unsigned d = 3;
vector<double> a;
vector<double> b;
a.resize(d, 1); b.resize(d, 1);
// given two arrays, a[] and b[], of length d
vector< vector<double> > coefficients(2);
coefficients[0].resize(d + 1);
coefficients[1].resize(d + 1);
if (d > 0) {
auto &coeff = coefficients[0]; // i = 0
coeff[0] = a[0];
coeff[1] = b[0];
for (unsigned i = 1; i < d; ++i) {// i : [1, d-1]
const auto ai = a[i];
const auto bi = b[i];
const auto &oldCoeff = coefficients[(i-1)%2];
auto &coeff = coefficients[i%2];
coeff[0] = oldCoeff[0] * ai; // j = 0
for (unsigned j = 1; j <= i; ++j) { // j : [1, i]
coeff[j] = oldCoeff[j] * ai + oldCoeff[j-1] * bi;
}
coeff[i+1] = oldCoeff[i] * bi; // j = i
}
}
const auto &coeff = coefficients[(d-1)%2];
for (unsigned i = 0; i < d; ++i) {
cout << coeff[i] << "\t";
}
cout << coeff[d] << '\n';
}

Portable efficient alternative to PDEP without using BMI2?

The documentation for the parallel deposit instruction (PDEP) in Intel's Bit Manipulation Instruction Set 2 (BMI2) describes the following serial implementation for the instruction (C-like pseudocode):
U64 _pdep_u64(U64 val, U64 mask) {
U64 res = 0;
for (U64 bb = 1; mask; bb += bb) {
if (val & bb)
res |= mask & -mask;
mask &= mask - 1;
}
return res;
}
See also Intel's pdep insn ref manual entry.
This algorithm is O(n), where n is the number of set bits in mask, which obviously has a worst case of O(k) where k is the total number of bits in mask.
Is a more efficient worst case algorithm possible?
Is it possible to make a faster version that assumes that val has at most one bit set, ie either equals 0 or equals 1<<r for some value of r from 0 to 63?
The second part of the question, about the special case of a 1-bit deposit, requires two steps. In the first step, we need to determine the bit index r of the single 1-bit in val, with a suitable response in case val is zero. This can easily be accomplished via the POSIX function ffs, or if r is known by other means, as alluded to by the asker in comments. In the second step we need to identify bit index i of the r-th 1-bit in mask, if it exists. We can then deposit the r-th bit of val at bit i.
One way of finding the index of the r-th 1-bit in mask is to tally the 1-bits using a classical population count algorithm based on binary partitioning, and record all of the intermediate group-wise bit counts. We then perform a binary search on the recorded bit-count data to identify the position of the desired bit.
The following C-code demonstrates this using 64-bit data. Whether this is actually faster than the iterative method will very much depend on typical values of mask and val.
#include <stdint.h>
/* Find the index of the n-th 1-bit in mask, n >= 0
The index of the least significant bit is 0
Return -1 if there is no such bit
*/
int find_nth_set_bit (uint64_t mask, int n)
{
int t, i = n, r = 0;
const uint64_t m1 = 0x5555555555555555ULL; // even bits
const uint64_t m2 = 0x3333333333333333ULL; // even 2-bit groups
const uint64_t m4 = 0x0f0f0f0f0f0f0f0fULL; // even nibbles
const uint64_t m8 = 0x00ff00ff00ff00ffULL; // even bytes
uint64_t c1 = mask;
uint64_t c2 = c1 - ((c1 >> 1) & m1);
uint64_t c4 = ((c2 >> 2) & m2) + (c2 & m2);
uint64_t c8 = ((c4 >> 4) + c4) & m4;
uint64_t c16 = ((c8 >> 8) + c8) & m8;
uint64_t c32 = (c16 >> 16) + c16;
int c64 = (int)(((c32 >> 32) + c32) & 0x7f);
t = (c32 ) & 0x3f; if (i >= t) { r += 32; i -= t; }
t = (c16>> r) & 0x1f; if (i >= t) { r += 16; i -= t; }
t = (c8 >> r) & 0x0f; if (i >= t) { r += 8; i -= t; }
t = (c4 >> r) & 0x07; if (i >= t) { r += 4; i -= t; }
t = (c2 >> r) & 0x03; if (i >= t) { r += 2; i -= t; }
t = (c1 >> r) & 0x01; if (i >= t) { r += 1; }
if (n >= c64) r = -1;
return r;
}
/* val is either zero or has a single 1-bit.
Return -1 if val is zero, otherwise the index of the 1-bit
The index of the least significant bit is 0
*/
int find_bit_index (uint64_t val)
{
return ffsll (val) - 1;
}
uint64_t deposit_single_bit (uint64_t val, uint64_t mask)
{
uint64_t res = (uint64_t)0;
int r = find_bit_index (val);
if (r >= 0) {
int i = find_nth_set_bit (mask, r);
if (i >= 0) res = (uint64_t)1 << i;
}
return res;
}

select a group of pairs in order to minimize rms of group

Simplified problem
I have ~40 resistors (all the same value +-5%) and I need to select 12 of them so that they are as similar as possible.
Solution: I list them in order and take the 12 consecutive with the smallest RMS.
The actual problem
I have ~40 resistors (all the same value +-5%) and I have to choose 12 pairs of them so that the resistance of the pairs is as similar as possible.
Notes
The resistance of the pair (R1,R2) is R1+R2.
I do not really care about the programming language, but let's say that I'm looking for a solution in C++ or Python, the two languages I'm most familiar with.
This gives reasonably good results (in MATLAB)
a = ones(40,1) + rand(40,1)*0.1-0.05; % The resistors
vec = zeros(40,2); % Initialize matrix
indices = zeros(40,2); % Initialize matrix
a = sort(a); % Sort vector of resistors
for ii = 1:length(a)
vec(ii,:) = [a(ii) a(ii)]; % Assign resistor values to row ii of vec
indices(ii,:) = [ii,ii]; % Corresponding resistor number (index)
for jj = 1:length(a)
if sum(abs((a(ii)+a(jj))-2*mean(a))) < abs(sum(vec(ii,:))-2*mean(a))
vec(ii,:) = [a(ii) a(jj)]; % Check if the new set is better than the
indices(ii,:) = [ii, jj]; % previous, and update vec and indices if true.
end
end
end
[x, idx] = sort(sum(vec')'); % Sort the sum of the pairs
final_list = indices(idx); % The indices of the sorted pairs
This is the result when I plot it:
This is not optimal but should give somewhat decent results. It's very fast though so if you ever need to choose 1000 pairs out of 10000 resistors...
#include <stdio.h>
#include <math.h>
#include <stdlib.h>
#include <time.h>
#define GROUPS 12
#define N 40
int compare (const void * a, const void * b)
{
return ( *(int*)a - *(int*)b );
}
int main ()
{
// generate random numbers
float *values = (float *)malloc(sizeof(float) * N);
srand(time(0));
for (int i = 0; i < N; i++)
values[i] = 950 + rand()%101;
qsort(values, N, sizeof(float), compare);
// find "best" pairing
float bestrms = -1;
int beststart = -1;
float bestmean = -1;
for (int start = 0; start <= N - 2 * GROUPS; start++)
{
float sum = 0;
for (int i = start; i < start + 2 * GROUPS; i++)
sum += values[i];
float mean = sum / GROUPS;
float square = 0;
for (int i = 0; i < GROUPS; i++)
{
int x = start + 2 * GROUPS - 1 - i;
float first = values[start + i];
// in a sorted sequence of 24 resistors, always pair 1st with 24th, 2nd with 23rd, etc
float second = values[start + 2 * GROUPS - 1 - i];
float err = mean - (first + second);
square += err * err;
}
float rms = sqrt(square/GROUPS);
if (bestrms == -1 || rms < bestrms)
{
bestrms = rms;
beststart = start;
bestmean = mean;
}
}
for (int i = 0; i < GROUPS; i++)
{
float first = values[beststart + i];
float second = values[beststart + 2 * GROUPS - 1 - i];
float err = bestmean - (first + second);
printf("(%f, %f) %f %f\n", first, second, first + second, err);
}
printf("mean %f rms %f\n", bestmean, bestrms);
free(values);
}
Sort them and then pair 1 with 2, 3 with 4, 5 with 6 and so on. Find the difference between each pair and sort again, choosing the 12 with the least difference.
sort them by resistance
pair 1 with 40, 2 with 39 etc, compute R1+R2 for each pair and pick the best set of 12 pairs (needs another sorting step). compute the mean of all select (R1+R2).
try to refine this initial solution successively by trying to plug in one of the remaining 16 resistors for one of the 24 chosen ones. an attempt would be successful if combined resistance of the new pair is closer to the mean than the combined resistance of the old pair. repeat this step until you can't find any further improvement.
this solution will definitely not always compute the optimal solution but it might be good enough. another idea would be simulated annealing but that would be a lot more work and still not guarantee to find the best solution.

Radix Sort for Negative Integers

I am trying to implement radix sort for integers, including negative integers. For non-negative ints, I was planning to create a queue of 10 queues correspondingly for the digits 0-9 and implement the LSD algorithm. But I was kind of confused with negative integers. What I am thinking now, is to go ahead and create another queue of 10 queues for them and separately sort them and then at the end, I will gave 2 lists, one containing negative ints sorted and the other containing non-negative ints. And finally I would merge them.
What do you think about this? Is there more efficient way to handle with negative integers?
You can treat the sign as a special kind of digit. You sort the pile on the units, then the tens, etc. and finally on the sign. This does produce a reversed order for the negatives, you then simply reverse the contents of that bucket. It's how old mechanical card sorters worked.
One more solution is to separate negative integers from the array, make them positive, sort as positive values using radix, then reverse it and append with sorted non-negative array.
Note that the sign bit is the uppermost bit in a signed integer, but all numbers are treated by radix sort as unsigned integers by default. So you need to tell the algorithm that negative numbers are smaller than positive ones. In case of 32-bit signed integers, you can sort three lower bytes first, then sort the fourth (upper) byte with the sign bit inverted so that 0 will be used for negative numbers instead of 1, and consequently they will go first.
I strongly advise to sort numbers byte-by-byte rather than by decimal digits, because it's far easier for the machine to pick up bytes than extract digits.
The accepted answer requires one more pass than necessary.
Just flip the sign bit.
This assumes you are working with a two's-complement representation, which is true for 99% of us.
The following table demonstrates that simply flipping the sign bit will cause two's-complement integers to sort correctly when sorted lexicographically.
The first column gives a 4-bit binary value, the second column gives the interpretation of those bits as signed integers, and the third column gives the interpretation of those bits with the high bit flipped.
Binary | 2s-comp | Flip sign
----------+----------+----------
0000 | 00 | -8
0001 | +1 | -7
0010 | +2 | -6
0011 | +3 | -5
0100 | +4 | -4
0101 | +5 | -3
0110 | +6 | -2
0111 | +7 | -1
1000 | -8 | 00
1001 | -7 | +1
1010 | -6 | +2
1011 | -5 | +3
1100 | -4 | +4
1101 | -3 | +5
1110 | -2 | +6
1111 | -1 | +7
The answer given by punpcklbw recommends only flipping the bit when you are looking at the highest byte, but it would be faster to simply flip the sign bit every time. That's because a single xor to flip the bit will be faster than the branch to decide if you should flip or not.
[An important detail to mention, which some textbooks fail to address properly, is that a real implementation should use radix of 256 instead of radix 10. That allows you to read bytes instead of decimal digits.]
Your radix sort wont be faster than the famous comparison sorts if you dont use "bitshift" and "bitwise AND" for radix calculation.
Computers use 2's complement to represent signed numbers, here the sign-bit lies at the leftmost end of a binary digit, in memory representation
eg
436163157 (as 32 bit number) = 00011001 11111111 01010010 01010101 -436163157 (as 32 bit number) = 11100110 00000000 10101101 10101011
1 (as 32 bit number) = 00000000 00000000 00000000 00000001
-1 (as 32 bit number) = 11111111 1111111 1111111 11111111
0 is represented as = 00000000 00000000 00000000 00000000
Highest negative value as = 10000000 00000000 00000000 00000000
So you see, the more negative a number becomes, it looses that many 1's, a small negative number has many 1's, if you set only the sign-bit to 0, it becomes a very large positive number. Vice versa a small positive number becomes a large negative number.
In radix sort the key to sorting negative numbers is how you handle the last 8 bits, for negative numbers at least the last bit has to be 1, in 32-bit scheme it has to be from 10000000 00000000 00000000 00000000 which is the most negative value farthest from zero to 11111111 11111111 11111111 11111111 which is -1. If you look at the leftmost 8 bits, the magnitude ranges from 10000000 to 11111111, i.e. from 128 to 255.
These values can be obtained by this code piece
V = ( A[i] >> 24 ) & 255
For negative numbers V will always lie from 128 upto 255. For positive numbers it will be from 0 to 127. As said earlier, the value of M will be 255 for -1 and 128 for highest negative number in 32-bit scheme. Build up your histogram as usual. Then from index 128 to 255 do the cumulative sum, then add frequency of 255 to 0, and proceed the cumulative sum from 0 till index 127. Perform the Sort as usual. This technique is both optimal, fast, elegant and neat both in theory and in practice. No need of any kind of separate lists nor order reversal after sorting nor converting all inputs to positive which make the sort slow and messy.
For the code see Radix Sort Optimization A 64-bit version can be built using same concepts
Further read:
http://codercorner.com/RadixSortRevisited.htm
http://stereopsis.com/radix.html
Absolutely! Of course you do have to take care of splitting up the negatives from the positives but luckily this is easy. At the beginning of your sorting algorithm all you have to do is partition your array around the value 0. After that, radix sort below and above the partition.
Here is the algorithm in practice. I derived this from Kevin Wayne and Bob Sedgewick's MSD radix sort: http://algs4.cs.princeton.edu/51radix/MSD.java.html
private static final int CUTOFF = 15;
private static final int BITS_PER_INT = 32;
private static final int BITS_PER_BYTE = 8;
private static final int R = 256;
public void sort(int[] a){
int firstPositiveIndex = partition(0, a, 0, a.length-1);
int[] aux =new int[a.length];
if(firstPositiveIndex>0){
recSort(a, firstPositiveIndex, a.length-1, 0,aux);
recSort(a, 0, firstPositiveIndex-1, 0,aux);
}else{//all positive
recSort(a, 0, a.length-1, 0, aux);
}
}
private void recSort(int[] a, int lo, int hi, int d, int[] aux){
if(d>4)return;
if(hi-lo<CUTOFF){
insertionSort(a,lo, hi);
return;
}
int[] count = new int[R+1];
//compute counts
int bitsToShift = BITS_PER_INT-BITS_PER_BYTE*d-BITS_PER_BYTE;
int mask = 0b1111_1111;
for(int i = lo; i<=hi; i++){
int c = (a[i]>>bitsToShift) & mask;
count[c+1]++;
}
//compute indices
for(int i = 0; i<R; i++){
count[i+1]=count[i]+count[i+1];
}
//distribute
for(int i = lo; i<=hi; i++){
int c = (a[i]>>bitsToShift) & mask;
aux[count[c]+lo] = a[i];
count[c]++;
}
//copy back
for(int i = lo; i<=hi; i++){
a[i]=aux[i];
}
if(count[0]>0)
recSort(a, lo, lo+count[0]-1, d+1, aux);
for(int i = 1; i<R; i++){
if(count[i]>0)
recSort(a, lo+count[i-1], lo+count[i]-1, d+1, aux);
}
}
// insertion sort a[lo..hi], starting at dth character
private void insertionSort(int[] a, int lo, int hi) {
for (int i = lo; i <= hi; i++)
for (int j = i; j > lo && a[j] < a[j-1]; j--)
swap(a, j, j-1);
}
//returns the index of the partition or to the right of where it should be if the pivot is not in the array
public int partition(int pivot, int[] a, int lo, int hi){
int curLo = lo;
int curHi = hi;
while(curLo<curHi){
while(a[curLo]<pivot){
if((curLo+1)>hi)return hi+1;
curLo++;
}
while(a[curHi]>pivot){
if((curHi-1)<lo)return lo-1;
curHi--;
}
if(curLo<curHi){
swap(a, curLo, curHi);
if(a[curLo]!=pivot)curLo++;
if(a[curHi]!=pivot)curHi--;
}
}
return curLo;
}
private void swap(int[] a, int i1, int i2){
int t = a[i1];
a[i1]=a[i2];
a[i2]=t;
}
Probably the easiest way to handle signed values is to offset the starting position for the accumulation (i.e., generation of positional offsets) when operating on the most significant digit. Transforming the input so all digits may be treated as unsigned is also an option, but requires applying an operation over the value array at least twice (once to prepare input and again to restore output).
This uses the first technique as well as byte-sized digits (byte access is generally more efficient):
void lsdradixsort(int* a, size_t n)
{
// isolate integer byte by index.
auto bmask = [](int x, size_t i)
{
return (static_cast<unsigned int>(x) >> i*8) & 0xFF;
};
// allocate temporary buffer.
auto m = std::make_unique<int[]>(n);
int* b = m.get();
// for each byte in integer (assuming 4-byte int).
for ( size_t i, j = 0; j < 4; j++ ) {
// initialize counter to zero;
size_t h[256] = {}, start;
// histogram.
// count each occurrence of indexed-byte value.
for ( i = 0; i < n; i++ )
h[bmask(a[i], j)]++;
// accumulate.
// generate positional offsets. adjust starting point
// if most significant digit.
start = (j != 3) ? 0 : 128;
for ( i = 1+start; i < 256+start; i++ )
h[i % 256] += h[(i-1) % 256];
// distribute.
// stable reordering of elements. backward to avoid shifting
// the counter array.
for ( i = n; i > 0; i-- )
b[--h[bmask(a[i-1], j)]] = a[i-1];
std::swap(a, b);
}
}
This can be done without requiring partitioning or having to practically invert the MSB. Here's a working solution in Java:
public class RadixSortsInterviewQuestions {
private static final int MSB = 64;
static Map.Entry<Integer, Integer> twoSum(long[] a, long sum) {
int n = a.length - 1;
sort(a, MSB, 0, n);
for (int i = 0, j = n; i < j; ) {
long t = a[i] + a[j];
if (t == sum) {
return new SimpleImmutableEntry<>(i, j);
} else if (t < sum) {
i++;
} else {
j--;
}
}
return null;
}
// Binary MSD radix sort: https://en.wikipedia.org/wiki/Radix_sort#In-place_MSD_radix_sort_implementations
private static void sort(long[] a, int d, int lo, int hi) {
if (hi < lo || d < 1) return;
int left = lo - 1;
int right = hi + 1;
for (int i = left + 1; i < right; ) {
if (isBitSet(a[i], d)) {
swap(a, i, --right);
} else {
left++;
i++;
}
}
sort(a, d - 1, lo, left);
sort(a, d - 1, right, hi);
}
private static boolean isBitSet(long x, int k) {
boolean set = (x & 1L << (k - 1)) != 0;
// invert signed bit so that all positive integers come after negative ones
return (k == MSB) != set;
}
private static void swap(long[] a, int i, int j) {
long tmp = a[i];
a[i] = a[j];
a[j] = tmp;
}
}
All proposed solutions here imply performance penalty:
flip highest bit via (a[i] XOR 0x8000000000000000) on grouping stage;
treat sign bit as radix and use extra pass, sorting by it;
separate negative numbers from array;
use special bitmasks, etc.
You don't need them all. Use regular radix sort. On the last iteration you'll have array items splitted into 0..255 groups. Example items:
1 50 200 -500 -300 -2 -1
The only thing to tweak is how we copy those groups back into original array. We should start copy signed 128..255 groups (-128..-1 actually) and then 0..127.
Result:
-500 -300 -2 -1 1 50 200
Tested in PHP 7.4. Regular radix sort implementation is 2-2.5x faster, than QuickSort.
Adding extra xor operation slows down the result to 1.7-1.8x. Using the above mention approach has no performance penalty at all.
The code:
function sortRadix (array &$arr) {
static $groups;
isset($groups) or $groups = [];
$numRadix = 8;
$arrSize = count($arr);
$shift = 0;
for ($i = 0; $i < $numRadix; $i++) {
// Cleaning groups
for ($j = 0; $j < 256; $j++) {
$groups[$j] = [];
}
// Splitting items into radix groups
for ($j = 0; $j < $arrSize; $j++) {
$currItem = $arr[$j];
$groups[(($currItem >> $shift) & 0xFF)][] = $currItem;
}
// Copying sorted by radix items back into original array
$arrPos = 0;
// Treat the last radix with sign bit specially
// Output signed groups (128..256 = -128..-1) first
// Other groups afterwards. No performance penalty, as compared to flipping sign bit
// via (($currItem ^ 0x8000000000000000) >> $shift) & 0xFF)
if ($i === 7) {
for ($j = 128; $j < 256; $j++) {
foreach ($groups[$j] as $item) {
$arr[$arrPos++] = $item;
}
}
for ($j = 0; $j < 128; $j++) {
foreach ($groups[$j] as $item) {
$arr[$arrPos++] = $item;
}
}
} else {
foreach ($groups as $group) {
foreach ($group as $item) {
$arr[$arrPos++] = $item;
}
}
}
// Change shift value for next iterations
$shift += 8;
} // .for
} // .function sortRadix
You can also interpret the histogram (count[]) differently for the most significant byte (which contains the signed bit). Here is a solution in C:
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
static void sortbno(const int32_t* tab, // table of entries
int tabsz, // #entries in tab
int bno, // byte number in T
int* inidx, // current sorted index before this byte
int* outidx) // indices after sorting this byte
{
int count[256];
memset(count, 0, sizeof(count));
// count occurrences of each byte value
for (int i = 0; i < tabsz; i++) {
int32_t x = tab[i];
int v = (x >> (8 * bno)) & 0xff;
count[v]++;
}
// change count[i] so it now reflects the actual
// position of this byte value in outidx
if (bno == sizeof(tab[0]) - 1) {
/* account for signed bit for most-significant-byte */
for (int i = 129; i < 256; i++) {
count[i] += count[i - 1];
}
count[0] += count[255];
for (int i = 1; i < 128; i++) {
count[i] += count[i - 1];
}
} else {
for (int i = 1; i < 256; i++) {
count[i] += count[i - 1];
}
}
// fill outidx[]
for (int i = tabsz - 1; i >= 0; i--) {
int in = inidx[i];
int32_t x = tab[in];
int v = (x >> (8 * bno)) & 0xff;
outidx[--count[v]] = in;
}
}
/**
* Sort tab[].
* Return the indices into tab[] in ascending order.
*/
int* rsort(const int32_t* tab, int tabsz)
{
int* r[2];
r[0] = malloc(tabsz * sizeof(*r[0]));
r[1] = malloc(tabsz * sizeof(*r[1]));
if (! (r[0] && r[1]))
goto bail;
// Artificially assign indices to items
for (int i = 0; i < tabsz; i++) {
r[0][i] = i;
}
// Sort byte by byte. byte #0 is x & 0xff.
int bin = 0;
for (int i = 0; i < (int)sizeof(tab[0]); i++) {
sortbno(tab, tabsz, i, r[bin], r[1-bin]);
bin = !bin;
}
free(r[1-bin]);
return r[bin];
bail:
if (r[0]) free(r[0]);
if (r[1]) free(r[1]);
return 0;
}
You can see below Radix sort implementation for both positive and negative integers in JS.
const getDigit = (num, place) => {
return Math.floor(Math.abs(num) / Math.pow(10, place)) % 10;
}
const maxDigitNumber = arr => {
const digitCount = (num) => {
return Math.abs(num).toString().length;
}
let maxDigit = digitCount(arr[0]);
for(let num of arr) {
const digits = digitCount(num);
if(maxDigit < digits) maxDigit = digits;
}
return maxDigit;
}
const radixSort = arr => {
const maxDigit = maxDigitNumber(arr);
digitIteration:
for(let d = 0; d < maxDigit; d++) {
const bucket = {};
arrIteration:
for(let i = 0; i < arr.length; i++) {
const number = arr[i];
const digitValue = getDigit(number, d);
if(!bucket[digitValue]) bucket[digitValue] = [];
if(number > 0) bucket[digitValue].push(number);
else bucket[digitValue].unshift(number);
};
const newArr = [];
for(let obj in bucket) {
bucket[obj].map(el => {
if(el < 0) newArr.unshift(el);
else newArr.push(el);
});
}
arr = newArr;
}
return arr;
}

binary to decimal base shift

I need an algorithm that converts an arbitrarily sized unsigned integer (which is stored in binary format) to a decimal one. i.e. to make it human-readable ;)
I currently use the maybe (or obviously) somewhat naive way of continuously calculating the modulus and remainder through division through ten.
Unfortunately the speed is somewhat... lame.
e.g.
I calculate 2000^4000 (with my bignum library) which takes roughly 1.5 seconds (no flaming please xD). The print including the necessary base conversion however takes about 15 minutes which is quite annoying.
I have tested bc which does both in a lot less than one second.
How does it do that? (Not the multiplication stuff with ffts and whatever only the base conversion)
I currently use the maybe (or obviously) somewhat naive way of continuously calculating the modulus and remainder through division through ten.
Then you should have O(n^2) complexity, which should fare much better than 15 minutes.
Although, it's worth looking how exactly you do division by 10.
Make sure you apply not generic division of long by long, but simpler algorithm of division long number by standard one.
Make sure that you reuse memory. Allocating 10Kb blocks 10000 times would surely hinder your performance.
edit
How to divide long binary number by 10 in one pass and get both result and reminder. With no additional memory.
Simple pseudo-code (a[0] is the highest order digit)
int r = 0;
for (int i = 0; i < n; ++i) {
r = r * 2 + a[i];
a[i] = r / 10;
r = r % 10;
}
Let's take an example, number 100111011 = 315.
Step 0: r = 1, a[0] = 0
Step 1: r = 2, a[1] = 0
Step 2: r = 4, a[2] = 0
Step 3: r = 9, a[3] = 0
Step 4: r = 9, a[4] = 1
Step 5: r = 9, a[5] = 1
Step 6: r = 8, a[6] = 1
Step 7: r = 7, a[7] = 1
Step 8: r = 5, a[8] = 1
So, reminder is 5 and the result is 000011111 = 31.
I think that bc is using 10^n as base instead of 2. So every internal "digit" is just n decimal digits and at least for decimal input/output the problem becomes trivial.
There's no need to use exponentiation:
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
int main(){
char a[] = "10011";
unsigned long int res= 0;
int i;
for(i = 0; i < strlen(a); i++){
res = (res<<1) + (a[i]-'0');
}
printf("%d",res);
return 0;
}
FIRST UPDATE
Now length shouldn't be a problem...
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
char *doubles(char *);
char *sum(char *,int);
int main(){
char a[] = "10011";
char *res = calloc(2,sizeof(char));
int i;
res[0] = '0';
for(i = 0; i < strlen(a); i++){
res = sum(doubles(res),(a[i]-'0'));
}
printf("%s",res);
return 0;
}
char *doubles(char *s){
int i,len = strlen(s),t = 0;
char *d;
d = calloc(len+1,sizeof(char));
for(i = 0; i < len; i++){
t = ((s[len-i-1]-'0')<<1) + t;
d[len-i] = ('0' + (t%10));
t = t/10;
}
d[0] = t+'0';
return (d[0] == '0')?d+1:d;
}
char *sum(char *s,int n){
int i, len = strlen(s),t = n;
char *d = calloc(len+1,sizeof(char));
for(i = 0; i < len ; i++){
t = (s[len-i-1]-'0') + t;
d[len-i] = ('0' + (t%10));
t = t/10;
}
d[0] = t+'0';
return (d[0] == '0')?d+1:d;
}

Resources