Iterating through a bit sequence and finding each set bit - algorithm

Question

My question is more or less what's in the title; I'm wondering if there's a fast way to going through a sequence of bits and finding each bit that's set.
More detailed information:
I'm currently working on a data stucture that represents a set of objects. In order to support some operations I need, the structure must be able to perform very fast intersection of subsets internally. The solution I've come up with is to have each subset of the structure's superset represented by a "bit array", where each bit maps to an index in the array that holds the superset's data. Example: if bit #1 is set in a subset, then the element at index 1 in the superset's array is present in the subset.
Each subset consists of an array of ulong big enough that there's enough bits to represent the entire superset (if the superset contains 256 elements, the size of the array must be 256 / 64 = 4). To find the intersection of 2 subsets, S1 and S2, I can simply iterate through S1 and S2's array, and find the bitwise-and between the ulongs at each index.
Now back to what my question is really about:
In order to return the data of a subset, I have to iterate through all the bits in the subset's "bit array" and find the bits that are set. This is how I curently do it:
/// <summary>
/// Gets an enumerator that enables enumeration over the strings in the subset.
/// </summary>
/// <returns> An enumerator. </returns>
public IEnumerator<string> GetEnumerator()
{
int bitArrayChunkIndex = 0;
int bitArrayChunkOffset = 0;
int bitArrayChunkCount = this.bitArray.Length;
while(bitArrayChunkIndex < bitArrayChunkCount)
{
ulong bitChunk = bitArray[bitArrayChunkIndex];
// RELEVANT PART
if (bitChunk != 0)
{
int bit = 0;
while (bit < BIT_ARRAY_CHUNK_SIZE /* 64 */)
{
if(bitChunk.BitIsSet(bit))
yield return supersetData[bitArrayChunkOffset + bit];
bit++;
}
}
bitArrayChunkIndex++;
bitArrayChunkOffset += BIT_ARRAY_CHUNK_SIZE;
// END OF RELEVANT PART
}
}
Is there any obvious ways to optimize this? Any bit hacks to enable it to be done very fast? Thanks!

Answer 1

On INTEL 386+, you can use machine instruction BitSearchFirst.
Following - sample for gcc. This is little tricky for process 64-bit words,
but anyway works quick and efficient.
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
int main(int argc, char **argv) {
uint64_t val;
sscanf(argv[1], "%llx", &val);
printf("val=0x%llx\n", val);
uint32_t result;
if((uint32_t)val) { // first bit is inside lowest 32
asm("bsfl %1,%0" : "=r"(result) : "r"(val));
} else { // first bit is outside lowest 32
asm("bsfl %1,%0" : "=r"(result) : "r"(val >> 32));
result += 32;
}
printf("val=%llu; result=%u\n", val, result);
return 0;
}
Also, in your use x64 architecture, you can try to use bsfq instruction, and remove "if/else"

Answer 2

Take an array of sixteen integers, initialized with the number of bits set for the integers from zero to fifteen (i.e. 0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4). Now take bitchunk % 16, and look up the result in the int array - that's the number of set bits in the first four bits of the chunk. Right shift four times, and repeat the entire operation fifteen more times.
You can do this with an array of 256 integers and 8 bit sub-chunks instead. I wouldn't recommend using an array of 4096 integers with 12 bit sub-chunks, that's getting a bit ridiculous.
int[] lookup = new int[16] {0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4};
int bitCount = 0;
for(int i = 0; i < 16; i++) {
int firstFourBits = bitChunk % 16;
bitCount += lookup[firstFourBits];
bitChunk = butChunk >> 4;
}