Using OpenCV 2.4.4 with FFmpeg in Windows - windows

Question

I know there are other questions dealing with FFmpeg usage in OpenCV, but most of them appear to be outdated.
By opening up the makefiles in CMake, I can verify that I've got the WITH_FFMPEG flag on. My output folder for the OpenCV build contains a bin folder, within which are Debug and Release folders, each containing a copy of a .dll file entitled opencv_ffmpeg244.dll. I can step into the source code of OpenCV when I create a VideoWriter and verify that the function pointers to the .dll get filled correctly. That much appears to be working.
If I use the FOURCC code of CV_FOURCC_PROMPT, the following codecs work properly:
Microsoft Video 1
Intel IYUV codec
Logitech Video (I420)
Cinepak Codec by Radius
Full Frames (Uncompressed)
The following codecs do not work properly (ie. produce a 0kb video file):
Microsoft RLE
If my understanding is correct, using FFMPEG should allow for encoding video using a whole bunch of new codecs (x264, DIVX, XVID, and so on). However, none of these appear in the prompt. Manually setting them by their FOURCC codes using the macro CV_FOURCC(...) also doesn't work. For instance, using this: CV_FOURCC('X','2','6','4') produces the message:
Could not find encoder for codec id 28: Encoder not found
and makes a video file of size 0kb.
Using this: CV_FOURCC('X','V','I','D') produces no error message, and makes a video file of 6kb that will not play in Windows Media Player or VLC.
I tried manually downloaded the Xvid codec from Xvid.org. Once that was installed, it appeared under the VFW selection in the prompt, and the encoding worked properly. So it's close to a solution, but if I try to set the FOURCC code directly, it still fails as above! I have to pick it from the prompt every time. Isn't FFmpeg supposed to include a whole bunch of codecs? If so, why am I manually downloading the codec instead of using the one built into FFmpeg?
What am I missing here? Is there a way to check that FFMPEG is "enabled"? It seems like the only codecs available in the prompt are VFW codecs, not the FFMPEG ones. The .dll has been built and is sitting in the same folder as the executable, but it appears it's not being used in any way.
Lots of related questions here. Hoping to find somebody knowledgeable about the FFmpeg implementation in OpenCV and with some knowledge of how all of these pieces fit together.

Answer 1

how about running ffmpeg and your application separately and transfer images using piped data?
to get video into opencv program,
ffmpeg -i input.mp4 -vcodec mjpeg -f image2pipe -pix_fmt yuvj420p -r 10 -|program.exe
and for recording etc
program.exe|ffmpeg -r 10 -vcodec mjpeg -f image2pipe -i - -vcodec h264 output.mp4
program.exe should be capable of reading concatenated jpeg images from stdin and writing the same to stdout and the above workflow will work. here's some code to read from stdin and display the video.
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
using namespace cv;
#if defined(_MSC_VER) || defined(WIN32) || defined(_WIN32) || defined(__WIN32__) \
|| defined(WIN64) || defined(_WIN64) || defined(__WIN64__)
# include <io.h>
# include <fcntl.h>
# define SET_BINARY_MODE(handle) setmode(handle, O_BINARY)
#else
# define SET_BINARY_MODE(handle) ((void)0)
#endif
#define BUFSIZE 10240
int main ( int argc, char **argv )
{
SET_BINARY_MODE(fileno(stdin));
std::vector<char> data;
bool skip=true;
bool imgready=false;
bool ff=false;
int readbytes=-1;
while (1)
{
char ca[BUFSIZE];
uchar c;
if (readbytes!=0)
{
readbytes=read(fileno(stdin),ca,BUFSIZE);
for(int i=0;i<readbytes;i++)
{
c=ca[i];
if(ff && c==(uchar)0xd8)
{
skip=false;
data.push_back((uchar)0xff);
}
if(ff && c==0xd9)
{
imgready=true;
data.push_back((uchar)0xd9);
skip=true;
}
ff=c==0xff;
if(!skip)
{
data.push_back(c);
}
if(imgready)
{
if(data.size()!=0)
{
cv::Mat data_mat(data);
cv::Mat frame(imdecode(data_mat,1));
imshow("frame",frame);
waitKey(1);
}else
{
printf("warning");
}
imgready=false;
skip=true;
data.clear();
}
}
}
else
{
throw std::string("zero byte read");
}
}
}
to write to output something like this should work.
void saveFramestdout(cv::Mat& frame,int compression)
{
SET_BINARY_MODE(fileno(stdout));
cv::Mat towrite;
if(frame.type()==CV_8UC1)
{
cvtColor(frame,towrite,CV_GRAY2BGR);
}else if(frame.type()==CV_32FC3)
{
double minVal, maxVal;
minMaxLoc(frame, &minVal, &maxVal);
frame.convertTo(towrite, CV_8U, 255.0/(maxVal - minVal), -minVal * 255.0/(maxVal - minVal));
}
else{
towrite=frame;
}
std::vector<uchar> buffer;
std::vector<int> param(2);
param[0]=CV_IMWRITE_JPEG_QUALITY;
param[1]=compression;//default(95) 0-100
imencode(".jpg",towrite,buffer,param);
uchar* a = &buffer[0];
::write(fileno(stdout),a,buffer.size());
}
the problem with the above is the multiple encode/decode of jpegs, which can be partially solved by linking with libjpeg-turbo. or one could go and figure out how to directly pass raw data from ffmpeg and opencv. for my case, this is quite acceptable as most of the overhead is in encoding or the video processing.

Answer 2

Part Way There
I used CV_FOURCC('x','2','6','4') and although it selected the correct codec, my defaults were wrong and I wasn't sure how to set them properly.
[libx264 # 0x7f9f0b022600] broken ffmpeg default settings detected
relevant c++ source
VideoCapture cap(argv[1]); // open file
if(!cap.isOpened()) { // check if we succeeded
cout << "failed to open file " << argv[1] << endl;
return -1;
}
Mat raw;
VideoWriter out(argv[2], CV_FOURCC('x','2','6','4'), fpsOut, size, true );
for(;;) {
bool bSuccess = cap.read(raw); // get a new frame from camera
if (!bSuccess) {
cout << "finished reading file" << endl;
break;
}
out << raw;
}

Answer 3

Your setup fails because ffmpeg creates a XVID codec for you but fails to operate it. Hence you get invalid files.
Luckily XVID codec is registered in VfW so you can simply remove opencv_ffmpeg*.dll library to disable it. Once ffmpeg is not used, your fourcc "xvid" will trigger the correct codec through VfW mechanism.