Numerical Integration; CUDA development - performance

Question

I need advice on how to proceed and utilize the compute power of CUDA device for numerical integration of a function. Some information about my device is below (irrelevant)
Hardware
Geforce GTX470; Compute Capability 2.0
Problem Description
I have a function like
g(x) = x * f(x, a, b, c)
That I need to integrate as given equation
Now I already have written an integration function, which simply takes g(x), breaks the interval into N sub intervals, computes the result for individual sub interval, and then I sum it up on CPU. For completion purposes I provide below a code example.
__device__ float function(float x, float a, float b, float c) {
// do some complex calculation
return result;
}
__global__ void kernel(float *d_arr, float a, float b, float c, int N) {
int idx = blockIdx.x * blockDim.x + threadIdx.x;
float x = (float)idx / (float)N;
if (idx < N) {
d_arr[idx] = x * function(x, a, b, c);
}
}
The code above is only for demonstration purposes, I actually use Romberg method to integration my g(x) but the idea is the same. My real problem comes because of the fact that I don't have just one set of values (a, b, c), I have multiple values of this set.
I have a 2D array in device memory, precisely (3, 1024) 3 rows, 1024 columns. Each column represent a single set on which an integration function needs to be performed.
The problem arrives when I have to decide whether I shall execute a block of threads such as 1024, keeping in mind that one thread is equivalent to one integration function. In this case the function I wrote above is of no use. Because I want to perform parallel integration for all sets of values, I have to write an integration function, which can do integration sequentially. As an example:
__global__ void kernel(float *d_arr, float a, float b, float c, int N) {
int idx = blockIdx.x * blockDim.x + threadIdx.x;
float sum = 0;
for (int i = 0; i < N; i++) {
float x = (float)i / (float) N;
sum += x * function(x, a, b, c);
}
d_arr[idx] = sum;
}
So you see my point? Option A, seems to be better, but I cannot use it because I don't know how can I do multiple integrals and then distribute each integral to N threads.
How would you do it? Can you suggest me, How can I achieve, both multiple integrals and while each integral can be distributed to N threads? Is there any better way to do it.
Looking forward for your advice.

Answer 1

If I understand your problem correctly, you want to do numerical integration with multiple (1024) sets of inputs (a,b,c), and for each integral you need N sub-intervals. Let's call the number of sets of inputs M.
If N is large enough (let's say > 10000) the first kernel sample you pasted could be good enough (invoking it M times for different set of inputs). Whether or not it utilizes all available device throughput depends on how complex your function is.
I didn't get what exactly you do with the d_arr[] array? Normally for numerical integration you would want to sum it. Right? Are you summing up the results on CPU? Consider using atomicAdd (esp. if you are going to run your kernel on compute cap 3.0 and above gpus) or a parallel scan if you find atomicAdd not fast enough.
If N is small, it's better to launch N*M threads in a single kernel.
In your case as M=1024, you can have every block process one set of inputs (i.e, set blockSize = 1024), and pass (a,b,c) inputs as arrays to the kernel - something like this:
__global__ void kernel(float *d_arr, float *a_array, float *b_array, float *c_array, int totalThreads, int N) {
int idx = blockIdx.x * blockDim.x + threadIdx.x;
float x = (float) blockIdx.x / (float) N;
float a = a_array[threadIdx.x];
float b = b_array[threadIdx.x];
float c = c_array[threadIdx.x];
if (idx < totalThreads) {
// what happen to this array?
d_arr[idx] = x * function(x, a, b, c);
}
}
Again, you would later need to extract elements from d_arr from appropriate positions and sum them up (for each integral).
If your function is not very complex and the above kernel becomes memory bound, you can try the other way round, i.e, having every thread block to process every sub-interval - with different thread block working on different set of inputs. Kernel would look something like this:
(this example assumes that N <= 1024, but it's possible to break up your kernel to take advantage of this approach even if it's not)
__global__ void kernel(float *d_arr, float *a_array, float *b_array, float *c_array, int totalThreads) {
int idx = blockIdx.x * blockDim.x + threadIdx.x;
float x = (float)threadIdx.x / (float) blockDim.x; // N = blockDim.x
float a = a_array[blockIdx.x]; // every thread in block accesses same memory location
float b = b_array[blockIdx.x];
float c = c_array[blockIdx.x];
// d_arr has 'M' elements containing the integral for each input set.
if (idx < totalThreads)
{
atomicAdd(&d_arr[blockIdx.x], x * function(x, a, b, c));
}
}
In the above kernel have a_array, b_array and c_array allocated in constant memory. this will be faster as every thread in block will accesses same location.
As an example, I have also replaced your d_arr writes with an atomicAdd.