Better Solution to Project Euler #36? - algorithm

Question

Project Euler problem 36 states:
The decimal number, 585 = 1001001001 (binary), is palindromic in both bases.
Find the sum of all numbers, less than one million, which are palindromic in base 10 and base 2.
(Please note that the palindromic number, in either base, may not include leading zeros.)
There is already a solution to this on stack overflow, but I want a more efficient solution.
For example, since the palindrome cannot have leading 0's, no even numbers need to be checked, only odd numbers for which the last bit in binary is a 1. This simple observation already speeds up the brute force "check every number in the range" by a factor of 2.
But I would like to be more clever than that. Ideally, I would like an algorithm with running time proportional to the number of numbers in the sum. I don't think it's possible to do better than that. But maybe that is not possible. Could we for example, generate all palindromic decimal numbers less than one million in time proportional to the number of decimal numbers satisfying that property? (I think the answer is yes).
What is the most efficient algorithm to solve this palindrome sum problem? I would like to consider run-times parameterized by N: the size of the range of numbers (in this case 1 million), D: the set of decimal palindromes in the range, and B: the set of binary palindromes in the range. I hope for a run-time that is o(N) + O( |D intersect B| ), or failing that, O(min(|D|, |B|))
Note: The sequences of binary and decimal palindromes are well known.
e.g. binary palindromes < 100: 0, 1, 3, 5, 7, 9, 15, 17, 21, 27, 31, 33, 45, 51, 63, 65, 73, 85, 93, 99
. . .decimal palindromes < 100:0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 22, 33, 44, 55, 66, 77, 88, 99,
palindromes in both bases: 0, 1, 3, 5, 7, 9, 33, 99
The binary representations of 33 and 99 are 10001 and 1100011 respectively.
The next number which is a palindrome in both is 585 = 1001001001.

Answer 1

The number of palindromes in base b of length 2*k is (b-1)*b^(k-1), as is the number of palindromes of length 2*k-1. So the number of palindromes not exceeding N in any base is O(sqrt(N))¹. So if you generate all palindromes (not exceeding N) in one base and check if they are also palindromes in the other base, you have an O(sqrt(N)*log(N)) algorithm (the log factor comes from the palindrome check). That's o(N), but I don't know yet if it's also O(|D intersect B|).
It's not O(|D intersect B|) :( There are only 32 numbers up to 1010 which are palindromic in both bases. I don't see any pattern that would allow constructing only those.
¹ If N has d digits (in base b), the number of palindromes not exceeding N is between the number of palindromes having at most d-1 digits and the number of palindromes having at most d digits (both limits inclusive). There are (b-1)*b^(k-1) numbers having exactly k digits (in base b), of which (b-1)*b^(floor((k-1)/2))) are palindromes. Summing gives the number of base-b palindromes with at most k digits as either 2*(b^(k/2)-1) (if k is even) or (b-1)*b^((k-1)/2) + 2*(b^((k-1)/2)-1) (if k is odd). Hence, give or take a factor of 2*b, the number of palindromes with at most d digits is b^(d/2). Thus the number of palindromes not exceeding N is roughly N^0.5, with a factor bounded by a multiple of the base considered.

Answer 2

Consider that there are only about 2000 decimal palindromes between 1 and 1000000. From 1 to 999, you can string the number and its reverse together, both with and without duplicating the "middle" digit (the last digit of the left part). For each of those, you check whether it's also a binary palindrome, and if it is, it's part of the sum.

Answer 3

(not an answer to your question but a cute recursive bit-fiddling solution to Project Euler 36)
This may not be the most efficient algorithm but I like how it looks like. I wrote it after reading Daniel Fischer's answer, suggesting to generate all palindromes in one base and then checking in the other base if it's a palindrome too.
In 18 lines of code (including the brackets) it generates all the palindromes in base 2 and then checks if they're also palindrome in base 10.
Takes about 6 ms on my system.
This can probably be optimized (too many bitshifting operation to my taste, there's probably quite some unnecessary junk here) and there may be a better algo, but I "like" (+) the look of my code ; )
#Test
public void testProjectEuler36() {
final int v = rec(1, 1);
assertEquals( 872187, v );
}
public static int rec( final int n, final int z ) {
if ( n > 1000000 )
return 0;
if ( z % 2 == 0 ) {
final int l = n << 1 & -1 << z / 2 + 1;
final int r = n & -1 >>> 32 - z / 2;
return v(n) + rec( l | 1 << z / 2 | r, z + 1 ) + rec( l | r, z + 1 );
} else
return v(n) + rec( n << 1 & -1 << z / 2 + 1 | n & -1 >>> 31 - z / 2, z + 1 );
}
public static int v( final int n ) {
final String s = "" + n;
boolean ok = true;
for ( int j = s.length(), i = j / 2 - 1; i >= 0 && ok; i--)
ok = s.charAt(i) == s.charAt(j-(i+1));
return ok ? n : 0;
}

Answer 4

My first assumptions were entirely wrong, so I've fixed them up. I've provided two algorithms, one iterative and one recursive. They're obviously nowhere near as impressive and efficient as user988052's, but they're definitely easier to read! The first algorithm is iterative and has a runtime of 9ms. The second algorithm is recursive and has a runtime of 16ms. Although the second solution is cleaner, the recursive calls might be slowing it down.
First Algorithm (9ms):
/** Given half a palindrome, construct the rest of the palindrome with
* an optional string inserted in the middle. The returned string is
* only guaranteed to be a palindrome if 'mid' is empty or a palindrome. */
public static String getPalindrome(String bin_part, String mid) {
return bin_part + mid + (new StringBuilder(bin_part)).reverse();
}
/** Check if the specified string is a palindrome. */
public static boolean isPalindrome(String p) {
for (int i=0; i<p.length()/2; i++)
if (p.charAt(i) != p.charAt(p.length()-1-i))
return false;
return true;
}
public static void main(String[] args) {
String[] mids = {"0","1"};
long total = 0;
boolean longDone = false; // have the numbers with extra digits been tested
long start = System.currentTimeMillis();
for (long i=0; i<1000; i++) {
String bin_part = Long.toBinaryString(i);
String bin = getPalindrome(bin_part, "");
long dec = Long.valueOf(bin, 2);
if (dec >= 1000000) break; // totally done
if (isPalindrome(Long.toString(dec)))
total += dec;
if (!longDone) {
for (int m=0; m<mids.length; m++) {
bin = getPalindrome(bin_part, mids[m]);
dec = Long.valueOf(bin, 2);
if (dec >= 1000000) {
longDone = true;
break;
}
if (isPalindrome(Long.toString(dec)))
total += dec;
}
}
}
long end = System.currentTimeMillis();
System.out.println("Total: " + total + " in " + (end-start) + " ms");
}
Second Algorithm (16ms)
public long total = 0;
public long max_value = 1000000;
public long runtime = -1;
public static boolean isPalindrome(String s) {
for (int i=0; i<s.length()/2; i++)
if (s.charAt(i) != s.charAt(s.length()-1-i))
return false;
return true;
}
public void gen(String bin, boolean done) {
if (done) { // generated a valid binary number
// check current value and add to total if possible
long val = Long.valueOf(bin, 2);
if (val >= max_value)
return;
if (isPalindrome(Long.toString(val))) {
total += val;
}
// generate next value
gen('1' + bin + '1', true);
gen('0' + bin + '0', false);
} else { // generated invalid binary number (contains leading and trailing zero)
if (Long.valueOf('1' + bin + '1', 2) < max_value) {
gen('1' + bin + '1', true);
gen('0' + bin + '0', false);
}
}
}
public void start() {
total = 0;
runtime = -1;
long start = System.currentTimeMillis();
gen("",false);
gen("1",true);
gen("0",false);
long end = System.currentTimeMillis();
runtime = end - start;
}
public static void main(String[] args) {
Palindromes2 p = new Palindromes2();
p.start();
System.out.println("Total: " + p.total + " in " + p.runtime + " ms.");
}

Answer 5

Here is the best Python Implementation to solve this problem :
sum = 0
for i in range(1000000):
bina = int(str(bin(i)).replace('0b',''))
if(i==int(str(i)[::-1]))or(bina==int(str(bina)[::-1])):
#print("i : "+str(i))
#print("bina : "+str(bina))
sum+=i
print("Sum of numbers : ",sum)