In Qt5 /qml it was possible to playback udp stream as rtp payload on Windows MSVC kit , below qml code works on Qt5, now problem is migrating to qt6 (6.3) this stopped working anymore, No error reported .. only if I check Video element error, I get Failed to load source ; in Qt6, QMultimedia back end is Windows media foundation , as of Qt supported-media-formats Under Widows section , It seems that whatever Windows media player can decode and play ..so should also Qt do, strangely, I can play the stream with same url using Windows media player , which also uses media foundation. I also have installed the K-Lite codec pack from codec guide in same reference .. , notably, Qt can detect erroneous url and reports Unsupported url scheme! if the url was erratic , this is not the case! Is this a bug in Qt6 .. or ?
code
GStreamer streaming pipeline ..
gst-launch-1.0 filesrc location=./local_video.mp4 ! decodebin ! queue ! videoconvert ! jpegenc ! queue ! rtpjpegpay ! udpsink host=192.168.0.174 port=5005
QML code:
import QtQuick
import QtQuick.Controls
import QtMultimedia
Window {
id: root
width: 640
height: 480
visible: true
title: qsTr("Hello World")
Component.onCompleted: {
mediaplayer.source = "rtp://192.168.0.174:5005/? udpsrc.caps=application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG, payload=(int)26"
//mediaplayer.source = "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"
mediaplayer.play()
}
Video {
id: mediaplayer
width: 640
height: 480
onErrorChanged: console.log("play error", mediaplayer.errorString , error) // Failed to load source
}
}
Related
I am using xcode 7 , swift 2.0
I am getting voice text to Speech working in Simulator but not in real iphone6 plus device, iOS 9. I have properly imported AVFOUNDATION and its Framework.
I tried...
#IBAction func SpeakTheList(sender: AnyObject) {
let mySpeechUtterance = AVSpeechUtterance(string: speakString)
//let voice = AVSpeechSynthesisVoice(language: "en-US")
// mySpeechUtterance.voice = voice
let voices = AVSpeechSynthesisVoice.speechVoices()
for voice in voices {
if "en-US" == voice.language {
mySpeechUtterance.voice = voice
print(voice.language)
break;
}
}
mySpeechSynthesizer.speakUtterance(mySpeechUtterance)
}
I get the following error :
Building MacinTalk voice for asset: (null)
Is there anything I ned to do settings in my iphone6plus iOS 9 , or I have to download something.
I have found a suggestion here Why I'm getting "Building MacinTalk voice for asset: (null)" in iOS device test
saying that..
" since iOS9, possibly a log event turned on during development that they forgot to turn off"
Just want to add to this (and by extension, the linked discussion in the original post):
I have two devices: an iPad2 and an iPad Air. They are running exactly the same version of iOS (9.2, 13C75). I have the following objective-C++ function for generating speech from Qt using Xcode 7.2 (7C68) on Yosemite:
void iOSTTSClient::speakSpeedGender(const QString &msg, const float speechRateModifier, const QString &gender, const bool cutOff) {
QString noHTML(msg);
noHTML.remove(QRegularExpression("<[^<]*?>"));
AVSpeechUtterance *utterance = [[AVSpeechUtterance alloc] initWithString:noHTML.toNSString()];
/* See https://forums.developer.apple.com/thread/18178 */
const float baseSpeechRate = (m_iOSVersion < 9.0) ? 0.15 : AVSpeechUtteranceDefaultSpeechRate;
utterance.rate = baseSpeechRate * speechRateModifier;
NSString *locale;
if (gender.compare("male", Qt::CaseInsensitive) == 0)
locale = #"en-GB"; // "Daniel" by default
else if (gender.compare("female", Qt::CaseInsensitive) == 0)
locale = #"en-US"; // "Samantha" by default
else
locale = [AVSpeechSynthesisVoice currentLanguageCode];
AVSpeechSynthesisVoice *voice = [AVSpeechSynthesisVoice voiceWithLanguage:locale];
const QString errMsg = QString("Null pointer to AVSpeechSynthesisVoice (could not fetch voice for locale '%1')!").arg(QString::fromNSString(locale));
Q_ASSERT_X(voice, "speakSpeedGender", errMsg.toLatin1().data());
utterance.voice = voice;
static const AVSpeechSynthesizer *synthesizer = [[AVSpeechSynthesizer alloc] init];
if (synthesizer.speaking && cutOff) {
const bool stopped = [synthesizer stopSpeakingAtBoundary:AVSpeechBoundaryImmediate];
Q_ASSERT_X(stopped, "speakSpeedGender", "Could not stop previous utterance!");
}
[synthesizer speakUtterance:utterance];
}
On the iPad Air, everything works beautifully:
Building MacinTalk voice for asset:
file:///private/var/mobile/Library/Assets/com_apple_MobileAsset_MacinTalkVoiceAssets/db2bf75d6d3dbf8d4825a3ea16b1a879ac31466b.asset/AssetData/
But on the iPad2, I hear nothing and get the following:
Building MacinTalk voice for asset: (null)
Out of curiosity, I fired up the iPad2 simulator and ran my app there. I got yet another console message:
AXSpeechAssetDownloader|error| ASAssetQuery error fetching results
(for com.apple.MobileAsset.MacinTalkVoiceAssets) Error Domain=ASError
Code=21 "Unable to copy asset information"
UserInfo={NSDescription=Unable to copy asset information}
However, I heard speech! And I realized I was wearing headphones. Sure enough, when I plugged ear buds into the iPad2, I heard speech there too. So now I'm searching for information about that. The following link is recent and has the usual assortment of this-worked-for-me voodoo (none of it helped me, but maybe will help others with this problem):
https://forums.developer.apple.com/thread/18444
In summary: TTS "works" but is not necessarily audible without headphones/ear buds. It appears to be a hardware settings issue with iOS 9.2. The console messages may or may not be relevant.
Final update: in the interests of full, if sheepish, disclosure, I figured I'd share how I finally solved the issue. The iPad2 in question had the "Use side switch to:" option set to "Mute". I left that alone but went ahead and toggled the switch itself. Wham! Everything worked without ear buds. So if you are unable to hear text-to-speech, try ear-buds. If that works, check whether your device is set to mute!
Do not use pauseSpeakingAtBoundary(). Instead, use stopSpeakingAtBoundary and continueSpeaking. This works for me.
Finally Found that there was a bug in iOS9, soon after XCODE new release 7.2 update, and iOS 9.2 Update release,
I tested same above code, text to speech started working.
I have a video background for the welcome screen of the app but when I run only a blank screen appears.
The string for the bundle path for resource part is perfectly correct so a typo isn't the problem or anything. When I put a breakpoint on the "if path" block though it isn't getting called. This is in view did load and "AVKit" and "AVFoundation" are both imported and there are no errors.
Any insights? Thanks!
let moviePath = NSBundle.mainBundle().pathForResource("Ilmatic_Wearables", ofType: "mp4")
if let path = moviePath {
let url = NSURL.fileURLWithPath(path)
let player = AVPlayer(URL: url)
let playerViewController = AVPlayerViewController()
playerViewController.player = player
playerViewController.view.frame = self.view.bounds
self.view.addSubview(playerViewController.view)
self.addChildViewController(playerViewController)
player.play()
}
Weird, your code should work if moviePath isn't nil as you are saying. Check this way:
if let moviePath != nil {
...
}
Update
Check if your video file's Target Membership is set
EDIT (11/13/2015) - Try this:
Right Click, delete the video file from your project navigator, select move to trash.
Then drag the video from finder back into your project navigator.
When choosing options for adding these files:
check Copy items if needed
check add to targets
Rebuild your project, see if it works now!
I have included sample code that works for me:
import AVFoundation
import AVKit
class ExampleViewController: UIViewController {
func loadAndPlayFile() {
guard let path = NSBundle.mainBundle().pathForResource("filename", ofType: "mp4") else {
print("Error: Could not locate video resource")
return
}
let url = NSURL(fileURLWithPath: path)
let moviePlayer = AVPlayer(URL: url)
let moviePlayerController = AVPlayerViewController()
moviePlayerController.player = moviePlayer
presentViewController(moviePlayerController, animated: true) {
moviePlayerController.player?.play()
}
}
override func viewDidAppear(animated: Bool) {
super.viewDidAppear(animated)
loadAndPlayFile()
}
}
/edit
Check to see if the video file plays in iTunes. If it plays in iTunes, it should also play on the iOS simulator and on a device.
If it doesn't play in iTunes, but does work with VLC Media Player, Quicktime, etc. It need to be re-encoded.
iOS supports many industry-standard video formats and compression standards, including the following:
H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second, Low-Complexity version of the H.264 Baseline Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
H.264 video, up to 768 Kbps, 320 by 240 pixels, 30 frames per second, Baseline Profile up to Level 1.3 with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
Numerous audio formats, including the ones listed in Audio Technologies
Using AVPlayer in a Mac app here to play random videos in fullscreen from a folder, but when I try to play .vob files or .mpg files I just get a black screen that lasts as long as the video lasts.
Does AVFoundation not support playback from these containers? I figured that since they were playable with stock QuickTime Player they would also work with AVPlayer.
The AVURLAsset class has a static methods that you can query for supported video UTIs:
+ (NSArray *)audiovisualTypes
On 10.9.1 it returns these system defined UTIs:
public.mpeg
public.mpeg-2-video
public.avi
public.aifc-audio
public.aac-audio
public.mpeg-4
public.au-audio
public.aiff-audio
public.mp2
public.3gpp2
public.ac3-audio
public.mp3
public.mpeg-2-transport-stream
public.3gpp
public.mpeg-4-audio
Here is an explanation of system UTIs. So it seems that at least the .mpg container should be supported.
According to wiki, .mpg files can contain MPEG-1 or MPEG-2 video, but only MPEG-2 video is supported. So maybe that's why the file loads but nothing is displayed.
QuickTime internally uses QTMovieModernizer in order to play videos in legacy formats (as mentioned in this WWDC session), so maybe you can look into that. It even has a method for determining if a file needs to be modernized:
+ requiresModernization:error:
To get a list of supported extensions, you can use the following function:
import AVKit
import MobileCoreServices
func getAllowedAVPlayerFileExtensions() -> [String] {
let avTypes = AVURLAsset.audiovisualTypes()
var avExtensions = avTypes.map({ UTTypeCopyPreferredTagWithClass($0 as CFString, kUTTagClassFilenameExtension)?.takeRetainedValue() as String? ?? "" })
avExtensions = avExtensions.filter { !$0.isEmpty }
return avExtensions
}
This will return a list like so:
["caf", "ttml", "au", "ts", "mqv", "pls", "flac", "dv", "amr", "mp1", "mp3", "ac3", "loas", "3gp", "aifc", "m2v", "m2t", "m4b", "m2a", "m4r", "aa", "webvtt", "aiff", "m4a", "scc", "mp4", "m4p", "mp2", "eac3", "mpa", "vob", "scc", "aax", "mpg", "wav", "mov", "itt", "xhe", "m3u", "mts", "mod", "vtt", "m4v", "3g2", "sc2", "aac", "mp4", "vtt", "m1a", "mp2", "avi"]
I know there are other questions dealing with FFmpeg usage in OpenCV, but most of them appear to be outdated.
By opening up the makefiles in CMake, I can verify that I've got the WITH_FFMPEG flag on. My output folder for the OpenCV build contains a bin folder, within which are Debug and Release folders, each containing a copy of a .dll file entitled opencv_ffmpeg244.dll. I can step into the source code of OpenCV when I create a VideoWriter and verify that the function pointers to the .dll get filled correctly. That much appears to be working.
If I use the FOURCC code of CV_FOURCC_PROMPT, the following codecs work properly:
Microsoft Video 1
Intel IYUV codec
Logitech Video (I420)
Cinepak Codec by Radius
Full Frames (Uncompressed)
The following codecs do not work properly (ie. produce a 0kb video file):
Microsoft RLE
If my understanding is correct, using FFMPEG should allow for encoding video using a whole bunch of new codecs (x264, DIVX, XVID, and so on). However, none of these appear in the prompt. Manually setting them by their FOURCC codes using the macro CV_FOURCC(...) also doesn't work. For instance, using this: CV_FOURCC('X','2','6','4') produces the message:
Could not find encoder for codec id 28: Encoder not found
and makes a video file of size 0kb.
Using this: CV_FOURCC('X','V','I','D') produces no error message, and makes a video file of 6kb that will not play in Windows Media Player or VLC.
I tried manually downloaded the Xvid codec from Xvid.org. Once that was installed, it appeared under the VFW selection in the prompt, and the encoding worked properly. So it's close to a solution, but if I try to set the FOURCC code directly, it still fails as above! I have to pick it from the prompt every time. Isn't FFmpeg supposed to include a whole bunch of codecs? If so, why am I manually downloading the codec instead of using the one built into FFmpeg?
What am I missing here? Is there a way to check that FFMPEG is "enabled"? It seems like the only codecs available in the prompt are VFW codecs, not the FFMPEG ones. The .dll has been built and is sitting in the same folder as the executable, but it appears it's not being used in any way.
Lots of related questions here. Hoping to find somebody knowledgeable about the FFmpeg implementation in OpenCV and with some knowledge of how all of these pieces fit together.
how about running ffmpeg and your application separately and transfer images using piped data?
to get video into opencv program,
ffmpeg -i input.mp4 -vcodec mjpeg -f image2pipe -pix_fmt yuvj420p -r 10 -|program.exe
and for recording etc
program.exe|ffmpeg -r 10 -vcodec mjpeg -f image2pipe -i - -vcodec h264 output.mp4
program.exe should be capable of reading concatenated jpeg images from stdin and writing the same to stdout and the above workflow will work. here's some code to read from stdin and display the video.
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
using namespace cv;
#if defined(_MSC_VER) || defined(WIN32) || defined(_WIN32) || defined(__WIN32__) \
|| defined(WIN64) || defined(_WIN64) || defined(__WIN64__)
# include <io.h>
# include <fcntl.h>
# define SET_BINARY_MODE(handle) setmode(handle, O_BINARY)
#else
# define SET_BINARY_MODE(handle) ((void)0)
#endif
#define BUFSIZE 10240
int main ( int argc, char **argv )
{
SET_BINARY_MODE(fileno(stdin));
std::vector<char> data;
bool skip=true;
bool imgready=false;
bool ff=false;
int readbytes=-1;
while (1)
{
char ca[BUFSIZE];
uchar c;
if (readbytes!=0)
{
readbytes=read(fileno(stdin),ca,BUFSIZE);
for(int i=0;i<readbytes;i++)
{
c=ca[i];
if(ff && c==(uchar)0xd8)
{
skip=false;
data.push_back((uchar)0xff);
}
if(ff && c==0xd9)
{
imgready=true;
data.push_back((uchar)0xd9);
skip=true;
}
ff=c==0xff;
if(!skip)
{
data.push_back(c);
}
if(imgready)
{
if(data.size()!=0)
{
cv::Mat data_mat(data);
cv::Mat frame(imdecode(data_mat,1));
imshow("frame",frame);
waitKey(1);
}else
{
printf("warning");
}
imgready=false;
skip=true;
data.clear();
}
}
}
else
{
throw std::string("zero byte read");
}
}
}
to write to output something like this should work.
void saveFramestdout(cv::Mat& frame,int compression)
{
SET_BINARY_MODE(fileno(stdout));
cv::Mat towrite;
if(frame.type()==CV_8UC1)
{
cvtColor(frame,towrite,CV_GRAY2BGR);
}else if(frame.type()==CV_32FC3)
{
double minVal, maxVal;
minMaxLoc(frame, &minVal, &maxVal);
frame.convertTo(towrite, CV_8U, 255.0/(maxVal - minVal), -minVal * 255.0/(maxVal - minVal));
}
else{
towrite=frame;
}
std::vector<uchar> buffer;
std::vector<int> param(2);
param[0]=CV_IMWRITE_JPEG_QUALITY;
param[1]=compression;//default(95) 0-100
imencode(".jpg",towrite,buffer,param);
uchar* a = &buffer[0];
::write(fileno(stdout),a,buffer.size());
}
the problem with the above is the multiple encode/decode of jpegs, which can be partially solved by linking with libjpeg-turbo. or one could go and figure out how to directly pass raw data from ffmpeg and opencv. for my case, this is quite acceptable as most of the overhead is in encoding or the video processing.
Part Way There
I used CV_FOURCC('x','2','6','4') and although it selected the correct codec, my defaults were wrong and I wasn't sure how to set them properly.
[libx264 # 0x7f9f0b022600] broken ffmpeg default settings detected
relevant c++ source
VideoCapture cap(argv[1]); // open file
if(!cap.isOpened()) { // check if we succeeded
cout << "failed to open file " << argv[1] << endl;
return -1;
}
Mat raw;
VideoWriter out(argv[2], CV_FOURCC('x','2','6','4'), fpsOut, size, true );
for(;;) {
bool bSuccess = cap.read(raw); // get a new frame from camera
if (!bSuccess) {
cout << "finished reading file" << endl;
break;
}
out << raw;
}
Your setup fails because ffmpeg creates a XVID codec for you but fails to operate it. Hence you get invalid files.
Luckily XVID codec is registered in VfW so you can simply remove opencv_ffmpeg*.dll library to disable it. Once ffmpeg is not used, your fourcc "xvid" will trigger the correct codec through VfW mechanism.
I'm doing video processing application using C++ with OpenCV. This is how I have written coding to initialize web cam.
storage = cvCreateMemStorage( 0 );
capture = cvCaptureFromCAM(1);
cvNamedWindow( "video", 1 );
while( key != 'q' ) {
frame = cvQueryFrame( capture );
if( !frame ) {
fprintf( stderr, "Cannot query frame!\n" );
break;
}
cvFlip( frame, frame, 1 );
frame->origin = 0;
key = cvWaitKey( 1 );
}
Can anyone suggest me a solution in order to increase the speed of capturing frames from web cam. There is like 3 seconds delay when compare to actual web cam video stream with OpenCV application web cam video stream.
Thank you.
What version of opencv are you using? Are you using the build that uses Intel Threading Building Blocks (tbb.dll)? If not, then use it, that is your speed-up right there.
You can also try a bare-bones code just to see what type of speed up you get:
storage = cvCreateMemStorage( 0 );
capture = cvCaptureFromCAM(1);
while (1) {
frame = cvQueryFrame(capture);
cvWaitKey(1);
}
Other than that, I suggest using the c++ interface to opencv, the c interface is quite ugly and might be slower.