compiling ffmpeg with nvidia/cuda failed loading nvcuvid - ffmpeg

Question

I am trying to compile an ffmpeg binary with support for using nvidia GPU's, I am running:
os: ubuntu 18.04
nvidia driver: nvidia-driver-390
cat /usr/local/cuda-9.0/version.txt => CUDA Version 9.0.176
gpu: 2x 1080ti
With the instructions shown on https://developer.nvidia.com/ffmpeg, being:
git clone https://git.ffmpeg.org/ffmpeg.git
cd ffmpeg
./configure --enable-cuda --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
I first found the error ERROR: cuda requested, but not all dependencies are satisfied: ffnvcodec, which was solved with help from https://superuser.com/questions/1299064/error-cuvid-requested-but-not-all-dependencies-are-satisfied-cuda-ffnvcodec. Below is the summary (make and make install just gave a list of commands to run):
git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
cd nv-codec-headers
sed 's###PREFIX###/usr/local#' ffnvcodec.pc.in > ffnvcodec.pc
sudo install -m 0755 -d '/usr/local/include/ffnvcodec'
sudo install -m 0644 include/ffnvcodec/*.h '/usr/local/include/ffnvcodec'
sudo install -m 0755 -d '/usr/local/lib/pkgconfig'
sudo install -m 0644 ffnvcodec.pc '/usr/local/lib/pkgconfig'
With that done, I removed the ffmpeg folder, recloned it and ran within the folder:
./configure --enable-cuda --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp --extra-cflags=-I/usr/local/cuda-9.0/include --extra-ldflags=-L/usr/local/cuda-9.0/lib64
make -j 10
Note that the path /usr/local/cuda-9.0 does contain both the include and lib64 folders, and that this resulted in no errors, just deprecation warnings.
Now when running this particular ffmpeg binary, with absolute paths, using the flag -hwaccel cuvid results in:
~/Documents/ffmpeg-cuda/ffmpeg/ffmpeg -y -hwaccel cuvid -c:v h264_cuvid -vsync 0 -i ~/test.mp4 -vf "scale_npp=1920:1072" -vcodec h264_nvenc ~/out1.mp4 -vf scale_npp=1280:720 -vcodec h264_nvenc out2.mp4
Which, is exactly what they have on their website to test it, results in:
[h264_cuvid # 0x556348eb94c0] Cannot load cuvidGetDecodeStatus
[h264_cuvid # 0x556348eb94c0] Failed loading nvcuvid.
I have also prepended: PKG_CONFIG_PATH="/usr/local/lib/pkgconfig" before ./configure, but also did not make a difference.
What am I doing wrong? Is there something important I missed?

Answer 1

With the ffmpeg repo https://github.com/FFmpeg/nv-codec-headers, which is "FFmpeg version of headers required to interface with Nvidias codec APIs", I glanced over the fact that Linux needed a "396.24 or newer driver". And only CUDA toolkit 9.2 at least, supports this driver version. So, the order of commands I used I have forgotten, but was roughly like below in order to remove nvidia drivers and cuda software;
sudo apt remove cuda
sudo aot-get autoremove --purge cuda
cd /var
# Here I removed files like cuda-repo-9-0-local-xxx
cd /etc/apt/sources.list.d
# Here I removed files like cuda-9-0-xxx.list/save
Now using the GUI software & Updates, in the additional drivers tab, sometimes wouldnt let me change from 390 to 396, so I'm pretty sure I first changed to Nouveau display driver, restarted the computer, and then chose 396 NVIDIA driver (open source), restarted the computer again and then now using 396, a check with nvidia-smi showed 396.51.
I then went to https://developer.nvidia.com/cuda-downloads and downloaded the 9.2 Linux Ubuntu 17.04 .deb file and then:
wget https://developer.nvidia.com/compute/cuda/9.2/Prod2/local_installers/cuda-repo-ubuntu1710-9-2-local_9.2.148-1_amd64
mv cuda-repo-ubuntu1710-9-2-local_9.2.148-1_amd64 cuda-repo-ubuntu1710-9-2-local_9.2.148-1_amd64.deb
sudo dpkg -i --force-overwrite cuda-repo-ubuntu1710-9-2-local_9.2.148-1_amd64.deb
sudo apt-key add /var/cuda-repo-9-2-local/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda
Now with CudaToolKit 9.2 installed with nvidia-drvier version 396.51, re-doing the steps posted in my question, resulted in no errors when running ffmpeg with -hwaccel cuvid. Which by the way makes a MASSIVE difference!