I'm investigating an issue with an in-house developed app using WMF to capture UVC data from a Cypress FX3 device. The stream is generated from a test pattern generator fed from an FPGA to the FX3. For a frame size of 1920x1080 (#30FPS), the frame capture works fine. For a frame size of (say) 3264x2448 (at <8FPS to meet throughput restrictions) the app is getting stuck during the ReadSample(). It does appear data is being received as the data image pattern can be seen in memory. Device enumeration looks ok i.e. the reported descriptors look to be correct and SelectFormat() is set accordingly.
Are there any restrictions on frame size?
I see that problem with your hardware is very specific and can be related with hardware part. The frame size - 3264x2448 is very huge. I have experience with Logitecn HD Pro Webcam C920 and I can get max frame size 2304x1536 at 2 FPS and RGB24. If you have connected via USB then it can be problem to transmit it via USB bus (especially for USB2). You must know that Windows Media Foundation is engine of the media part of Windows - for example WindowsStore can work only with Media Foundation. More over, Windows 10 Includes encoder and decoder for HEVC (H265) video and supports 4K and 8K in native, but playing video and working with live video via USB has some difference.
Related
I developed an application in C that can display 4 videos, and 1 sound file in the background.
The video uses the WMP object in the C++ class provided by Microsoft in the WMP SDK. The audio uses Windows' MCI (Media Control Interface) which is sent command strings.
To be able to play lots of different formats, I installed windows.7.codec.pack.
I experience a problem that when playing more than 3 media files (video or audio), the media stall. A video to be started constantly cycles from state 3 (Playing) to state 9 (Preparing new media) to state 10 (Ready to begin playing - without anything happening), and so on and so on. This is seen as a flicker of the video (state 3) followed by a few seconds of nothing (black, or desktop background, state 9). Once a video has started, it plays fine until the end.
Decreasing the number of media files to play to 2 videos and 1 audio lets it play fine; increasing to 3 video or more and 1 audio and this happens. Task Manager shows a CPU load of less than 25%, so CPU cannot be the problem.
It seems that MCI and WMP share stuff in the background because not only do videos stall, also audio stops without reporting errors (querying MCI returns that it is playing, but there is no sound).
I upgraded to windows.7.codec.pack.v4.2.6. This had a terrible performance
I reverted to windows.7.codec.pack.v4.1.6. This has a much better performance but still not perfect.
My question(s):
Is there any way I can configure Windows or the codec package to seamlessly handle 5 streams?
How can I report this problem to the developer?
Are there other codec packages that do not have this problem?
Any other suggestion?
System info: Intel i7-3520M X64 dual core at 2.9Ghz with 8GB physical memory and NVIDIA Quatro K1000M display adapter.
I think I found the solution.
After playing a video, I called the Player's Close() method. The documentation says:
The close method releases Windows Media Player resources.
Remarks
This method closes the current digital media file, not the Player itself.
Indeed I wanted to release the media file so it would no longer be busy in the file system. However, it seems that more resources were released than just the media file. As a consequence, for a next media file to be played, the player had to allocate resources again. It seems that turned out to be a bottle neck.
No longer calling Close() but just giving it the URL (filename) of the next media file to play now solved the problem. (I still have to give some retries sometimes but the general performance is now very well acceptable.)
The media file is released when the next media file starts playing.
I've been working on some WinAPI code to try to get audio latency as low as possible in Windows 10 natively. I eventually found this documentation which indicates the WASAPI in Windows is the lowest level you can get to, and there are even some improvements in Windows 10 to make the latency even lower. Most notably, support for <10ms audio buffer sizes with newer drivers.
I used the WASAPI but the latency still isn't quite low enough. I'm measuring about 80ms round trip using USB microphone and audio devices. I tested the new drivers mentioned and successfully got lower latency and smaller buffer sizes, but the only drivers mentioned seem to be the "High Definition Audio Device" driver, which I traced to hdaudio.sys. However, typical USB audio devices seem to use "USB Audio Device" drivers, implemented in usbaudio.sys instead I believe, which isn't allowing smaller buffer sizes.
It seems like the changes to allow smaller buffer sizes are relatively minor, but for some reason the stock HD Audio drivers contain these updates but the stock USB Audio drivers do not. What is happening here? Do these updated USB Audio drivers even exist in Windows?
For those looking to operate generic USB audio hardware on Windows at low latency, the KoordASIO driver provides a convenient ASIO-WASAPI bridge, enables either WASAPI Shared or Exclusive mode, and configurable buffer size.
https://github.com/koord-live/KoordASIO
Is video data that comes from Webcam to low level USB driver already encoded?
What does the low level webcam driver actually do? What is its main and only minimum mission?
Where can I read the principles or the protocol between the webcam hardware and standard Windows driver?
Ok, based on 5 webcams, that have bought to understand the difference between them, I can make some conclusions.
> Is video data that comes from Webcam to low level USB driver already encoded?
I'm using ManagedMEdiaHelpers project as a base to a background audio streaming project.
The audio output is fine but sometimes is takes 1 to 6 seconds to start playing. During that time the device sends some strange noises similar to hiccups or scratching.
The mp3 I'm trying to stream have the following properies:
Bitrate: 320000
Sampling Rate: 44100
What are the possible causes to receive that kind of noises on the start of the stream when the rest of the mp3 plays just fine?
More info
I noticed that during the hiccups the fps count was below 20 so I tried to profile the application.
I got the following message during the hiccups period:
Waring :Very high CPU usage by system threads: System and other applications
are using 65,02% of the CPU. This CPU usage may be caused by other
tasks that are running on the system or they may be caused by system
operations that are triggered by a user application. Ensure that no
other tasks are running on the system during profiling.
It was an hardware limitation of HTC Radar.
I Just tried the same code on an Samsung OMNIA 7 and the stream is just perfect. Also there's no penalty on the fps count so I think that on this device there's no CPU hogging.
Strangely gsmarena says that both devices have the same CPU.
I am streaming video and audio from my web cam/microphone over UDP. When I view the stream (even on the same machine) there is a delay of about 4 seconds. I have tried setting the UDP Cache setting to 0, or 1 but it doesn't seem to help. I have tried reducing the video and audio bit-rates, using mono sound and reducing the sample-rate all to no avail.
Does anyone have any ideas how I could reduce the delay, to something better suited to for a video conference, i.e < 1 second?
Is there a setting I can apply to the viewer/streamer that can help?
Thanks,
Marc
If you are using rtsp protocol to stream to video/audio, you can adjust the delay at
tools->preferences->all->input/codecs->demuxers->RTP/RTSP->caching value
tools->preferences->all->input/codecs->demuxers->RTP->RTP de-jitter buffer length
Try this.
#!/bin/sh
ETH=eth0
cvlc --miface=$ETH v4l2:///dev/video0 :input-slave=alsa://hw:0,0 :sout=#transcode{vcodec=h264,venc=x264{preset=ultrafast,tune=zerolatency,intra-refresh,lookahead=10,keyint=15},scale=auto,acodec=mpga,ab=128}:rtp{dst=224.10.0.1,port=5004,mux=ts} :sout-keep >/dev/null 2>/dev/null &
vlc1=$!
vlc --miface=$ETH rtp://224.10.0.1 >/dev/null 2>/dev/null &
vlc2=$!
wait $vlc2
kill -9 $vlc1
I've 2 seconds delay with 720p webcam, it produce about 2.5Mbit/s trafic and load for one core ~30%.
In my study of VLC streaming with webcam, I got 2-3 seconds delay for UDP multicast stream transcoded with WMV/ASF container + WMV2 codec from Dell's Creative Integrated Webcam with cif video size.
If using MP4/MOV container + H.264 codec, I got twice the delay of the former with the same settings in bitrate, fps and scale.
I disabled audio in both streaming settings since I wasn't interested in it.
I did the study with two VLC versions:
VLC 1.1.11 (latest Windows stable release)
VLC 2.1.0 (latest nightly build version)
With the first version, I could transcode and stream from the webcam, but it could not playback the stream properly (it just gave a blackened video stream)
With the second version, it worked well for transcoding, streaming and playback.
This study was done on:
Intel Core 2 Duo T7250
4GB DDR2-667 SDRAM
SATA 7200 RPM HDD
GeForce 8400M GS 128MB GDDR3 (+ 128MB shared memory = 256MB video memory)
Windows XP Pro SP3