accelerate pairwise force calculation in cuda C++ - performance

Question

recently I write a Molecular Dynamics code calculating the ion-electron force using CUDA parallel computing.
the kernel is list as below:
__global__ void (*x,*y,*z,N){
int i = (blockIdx.x * blockDim.x) + threadIdx.x;
while(i<N) {
double dx;
double dy;
double dz;
double dr;
double Fx;
double Fy;
double Fz;
for (int j = 0; j < N; j++){
dx=x[i]-x[j];
dy=y[i]-y[j];
dz=z[i]-z[j];
dr=sqrt(dx*dx+dy*dy+dz*dz)
dr=dr*dr*dr
Fx+=k*q*q*dx/dr
Fy+=k*q*q*dy/dr
Fz+=k*q*q*dz/dr //force=kq^2r/r^3 written in Cartesian coordinate
}
//rest of the code manipulate force is irrelevant to my question and I want to keep my code short
i += blockDim.x * gridDim.x;
}
}
x,y,z are position of the particels, and dx,dy,dz is the xyz distance, Fx, Fy, Fz in the for loop is the sum of force exerting on ith particle, more specifically you need to calculate x[i]-x[j] and run through all js to find the total force, and let the kernel do all i in parallel.
I found this to be slow as I know the GPU is reading the arrays from global memory. When I change x[i] to a number it becomes 10 times faster because it is reading from the register(L1 cache). my array is too big (more than 20000 element with double float) it is impossible to put into the register. But can it still be a little faster using other memories? I know there's constant memory and shared memory but I don't know how to implement. I think the x[i] is sitting at the globe memory causing it to be slow, and all thread is trying to read x[i] at the same time. any way to improve the speed?

Answer 1

Here is a basic version using shared memory to optimize the access pattern a bit.
#define KERNEL_BLOCKSIZE 256
__global__ void __launch_bounds__(KERNEL_BLOCKSIZE)
kernel(const double* x, const double* y, const double* z, int N,
double k, double q, double* fake_out)
{
const int i = blockIdx.x * blockDim.x + threadIdx.x;
/*
* threads beyond the bound still participate in value fetching, so we cannot
* return early
*/
const bool active = i < N;
double xi, yi, zi;
if(active)
xi = x[i], yi = y[i], zi = z[i];
const double kqq = k * q * q;
double Fx = 0., Fy = 0., Fz = 0.;
__shared__ double xt[KERNEL_BLOCKSIZE];
__shared__ double yt[KERNEL_BLOCKSIZE];
__shared__ double zt[KERNEL_BLOCKSIZE];
for(int j = 0; j < N; j += blockDim.x) {
__syncthreads();
const int thread_j = j + threadIdx.x;
if(thread_j < N) {
xt[threadIdx.x] = x[thread_j];
yt[threadIdx.x] = y[thread_j];
zt[threadIdx.x] = z[thread_j];
}
__syncthreads();
for(int l = 0, M = min(KERNEL_BLOCKSIZE, N - j); l < M; ++l) {
const double dx = xi - xt[l], dy = yi - yt[l], dz = zi - zt[l];
// 1 / sqrt(dx*dx + dy+dy + dz*dz)
const double rnorm = rnorm3d(dx, dy, dz);
const double dr = rnorm * rnorm * rnorm;
const double scale = kqq * dr;
Fx += scale * dx;
Fy += scale * dy;
Fz += scale * dz;
}
}
if(active)
fake_out[i] = norm3d(Fx, Fy, Fz);
}
It's nothing fancy and it doesn't solve the inherent issues with the O(N²) runtime. I made the following changes
Get rid of the while loop. The loop counter was declared as int i. The maximum grid dimension in all CUDA devices is 2^31-1. Meaning we can always launch the entire grid with only one loop per thread.
Given the quadratic runtime, we have no chance of ever running such a huge grid, anyway. But if we did have one that is larger, just launch multiple kernels operating on subsets
Use shared memory to buffer blocks. I picked 256 as a fixed blocksize. That tends to work well. 512 may be another size that is worth experimenting with
The whole dr calculation can be folded into a single predefined math function
To get something that at least compiles into reasonable code, I added an output
Double buffering
We can reduce the number of __syncthreads() that are required by using double buffering. However, that doubles the shared memory usage. Platforms that have only 64 kiB of shared memory will suffer limited occupancy. It requires benchmarking to see which version works better.
__global__ void __launch_bounds__(KERNEL_BLOCKSIZE)
kernel_dbuf(const double* x, const double* y, const double* z, int N,
double k, double q, double* fake_out)
{
const int i = blockIdx.x * blockDim.x + threadIdx.x;
const bool active = i < N;
double xi, yi, zi;
if(active)
xi = x[i], yi = y[i], zi = z[i];
const double kqq = k * q * q;
double Fx = 0., Fy = 0., Fz = 0.;
__shared__ double xt[2][KERNEL_BLOCKSIZE];
__shared__ double yt[2][KERNEL_BLOCKSIZE];
__shared__ double zt[2][KERNEL_BLOCKSIZE];
int dbuf = 0;
for(int j = 0; j < N; dbuf ^= 1, j += blockDim.x) {
const int thread_j = j + threadIdx.x;
if(thread_j < N) {
xt[dbuf][threadIdx.x] = x[thread_j];
yt[dbuf][threadIdx.x] = y[thread_j];
zt[dbuf][threadIdx.x] = z[thread_j];
}
__syncthreads();
for(int l = 0, M = min(KERNEL_BLOCKSIZE, N - j); l < M; ++l) {
const double dx = xi - xt[dbuf][l];
const double dy = yi - yt[dbuf][l];
const double dz = zi - zt[dbuf][l];
// 1 / sqrt(dx*dx + dy+dy + dz*dz)
const double rnorm = rnorm3d(dx, dy, dz);
const double dr = rnorm * rnorm * rnorm;
const double scale = kqq * dr;
Fx += scale * dx;
Fy += scale * dy;
Fz += scale * dz;
}
}
if(active)
fake_out[i] = norm3d(Fx, Fy, Fz);
}
Launch the kernel like this:
__host__ void
launch(const double* x, const double* y, const double* z, int N,
double k, double q, double* fake_out, cudaStream_t stream)
{
const int numBlocks = (N + KERNEL_BLOCKSIZE - 1) / KERNEL_BLOCKSIZE;
kernel<<<numBlocks, KERNEL_BLOCKSIZE, 0, stream>>>(x, y, z, N, k, q, fake_out);
}
Other thoughts
People have already commented on the inherent inefficiency of the algorithm
I guess there is a good reason why k and q are separate variables and you don't just pass a precomputed k * q * q to the kernel
Using doubles should always be a last resort when computing on a GPU, in my opinion. Possible avenues to reduce the precision, at least for parts of the algorithm:
Replace the dr computation with one that is less prone to overflows. Like this:
float scale = 1.f / max(max(abs(dx), abs(dy)), abs(dz));
float rnorm = rnorm3df(dx * scale, dy * scale, dz * scale) * scale;
float dr = rnorm * rnorm * rnorm;
Use Kahan summation for Fx, Fy, Fz
Use double only for Fx, Fy, Fz but not x, y, z positions or other computations