I'm trying to make a kind of sound application repurposing an old ADSL Router. At this point I already have compiled a kernel with Sound Supporting and Alsa drivers and also compiled alsa-lib and portaudio libraries.
Now I'm making some tests with c++ to know what kind of things I can do in the system (I dream with basic DSP, but the principal wish is to make something to record and play the recorded files). The thing is that I have troubles with something very basic like playing a sine wave. I'm using the paex_sine.c example from http://portaudio.com/docs/v19-doxydocs/group__examples__src.html but when played it has some drops like jitter.
Of course, the first thing that I made was trying to tune the device settings (Buffer size, Bits depth, Sample rate and Device Latency). I'm doing the tests with 2 different sound cards. One is a cheap usb card (the cheapest card of the market), but also did the tests with a Zoom H1 Recorderd, that has USB Sound card functionality, Class Compliant. In both cases I have the same results.
I also tested playing a .wav file with this example https://github.com/hosackm/wavplayer/blob/master/src/wavplay.c but same result. I attach the original and the jittered one.
The hardware is an old Huawei ADSL router. It use an RT63365e 500MHz (surprising with 4 cores) but is working at 420MHz. It also has a 32MB RAM.
I'm pretty sure that there is not a resources issue because the CPU is used at max 11% by the other processes/system. Also have 5MB of RAM free.
So I don't know what could be the source of the issue. It's my first time compiling the Linux Kernel and modding an Embedded System. Do you think in any point that I could research for trying to fix the problem? Maybe I'm missing some kernel configuration at building stage. I read al menuconfig options and I didn't find anything like audio priority or something like that.
Original audio: https://vocaroo.com/1i0Rhyrmyhzf
Audio with jitter: https://vocaroo.com/15l3p2wL5AhH
An image with jitter evidence in the file waveform
Related
I compiled ffmpeg and h264 libraries for android using NDK.
I am recording videos using the muxing.c example from the ffmpeg library. Everything works correct (still haven't worked on the audio) but the camera is dropping frames and it takes around 100ms to save each frame, which is unacceptable.
I have also tried making a queue and saving them into another thread (let's call it B) but at the end I need to wait for around 120 seconds because the background thread (B)is still recording the frames.
Is there a workaround for this issue, besides reducing the video size? Ideally I would like to save the frames in real time, at least reduce the saving time. Is it just that Android is incapable of doing this? .
First of all, check if you can be better served by the hardware encoder (via MediaRecorder or MediaCodec in Java, or using OpenMax from native code).
If for some reason you must encode in software, and your device is multicore, you can gain a lot by compiling x264 to use sliced multithreading. Let me cite my post of 2 years ago:
We are using x264 directly (no ffmpeg code involved), and with ultafast/zerolatency preset we get 30 FPS for VGA on Samsung Note10 (http://www.gsmarena.com/samsung_galaxy_note_10_1_n8000-4573.php) with Quad-core 1.4 GHz Cortex-A9 Exynos 4412 CPU, which is on the paper weaker than Droid's Quad-core 1.5 GHz Krait Qualcomm MDM615m/APQ8064 (http://www.gsmarena.com/htc_droid_dna-5113.php).
Note that x264 build scripts do not enable pthreads for Android (because NDK does not include libpthread.a), bit you can build the library with multithread support (very nice for a Quad-core CPU) if you simply create a dummy libpthread.a, see https://mailman.videolan.org/pipermail/x264-devel/2013-March/009941.html.
Note that encoder setup is only one part of the problem. If you work with the deprecated camera API, you should use preallocated buffers and a background thread for camera callbacks, as I explained elsewhere.
For a project due to perform soon enough I happen to have a problem. Task is to play 4k PRORES 422 files according to a sequence that is written on a file (XML) while listening on a OSC port for CUE signals and some feedback to the operator. Player also can smooth the speed +- 15% and has a general fader, and sends back to OSC controller few data to update the performer.
The playback is now unpredictably un-smooth And I don't know why.
I also tend to think is not a problem of hardware: Machine used is a mac pro 2014 with 10.9 (https://www.apple.com/mac-pro/specs/) with 64mb Ram, all data on SDD and a hell of a graphic card. The un-smoothness in playback is rather unpredictable, random frame drops in different places. I tried to use external time on the player and is a bit better but still not satisfactory. I am going to package it without the editor in a app, but on preliminary test is not that faster.
I wander also what is the best way to examine the code for leaks...
Playback of files in quicktime player uses 20% of cpu and in quartz composer +90%
I am stuck on this issue, having done all the obvious things I think, and would like at least to understand how to profile the performance of the patch to find what is wrong and were.
Suggestions are welcome and thanks for help!
If its not interactive you could try rendering it in quicktime player.
I am using a 32-bit AVR microcontroller (AT32UC3A3256) with High speed USB support. I want to stream data regularly from my PC to the device (without acknowledge of data), so exactly like a USB audio interface, except the data I want to send isn't audio. Such an interface is described here: http://www.edn.com/design/consumer/4376143/Fundamentals-of-USB-Audio.
I am a bit confused about USB isochronous transfers. I understand how a single transfer works, but how and when is the next subsequent transfer planned? I want a continuous stream of data that is calculated a little ahead of time, but streamed with minimum latency and without interruptions (except some occasional data loss). From my understanding, Windows is not a realtime OS so I think the transfers should not be planned with a timer every x milliseconds, but rather using interrupts/events? Or maybe a buffer needs to be filled continuously with as much data as there is available?
I think my question is still about the concepts of USB and not code-related, but if anyone wants to see my code, I am testing and modifying the "USB Vendor Class" example in the ASF framework of Atmel Studio, which contains the firmware source for the AVR and the source for the Windows EXE as well. The Windows example program uses libusb with a supplied driver.
Stephen -
You say "exactly like USB Audio"; but beware! The USB Audio class is very, very complicated because it implements a closed-loop servo system to establish long-term synchronisation between the PC and the audio device. You probably don't need all of that in your application.
To explain a bit more about long-term synchronisation: The audio codec at one end (e.g. the USB headphones) may run at a nominal 48KHz sampling rate, and the audio file at the other end (e.g. the PC) may be designed to offer 48 thousand samples per second, but the PC and the headphones are never going to run at exactly the same speed. Sooner or later there is going to be a buffer overrun or under-run. So the USB audio class implements a control pipe as well as the audio pipe(s). The control pipe is used to negotiate a slight speed-up or slow-down at one end, usually the Device end (e.g. headphones), to avoid data loss. That's why the USB descriptors for audio device class products are so incredibly complex.
If your application can tolerate a slight error in the speed at which data is delivered to the AVR from the PC, you can dispense with the closed-loop servo. That makes things much, much simpler.
You are absolutely right in assuming the need for long-term buffering when streaming data using isochronous pipes. A single isochronous transfer is pointless - you may as well use a bulk pipe for that. The whole reason for isochronous pipes is to handle data streaming. So a lot of look-ahead buffering has to be set up, just as you say.
I use LibUsbK for my iso transfers in product-specific applications which do not fit any preconceived USB classes. There is reasonably good documentation at libusbk for iso transfers. In short - you decide how many bytes per packet and how many packets per transfer. You decide how many buffers to pre-fill (I use five), and offer the libusbk driver the whole lot to start things going. Then you get callbacks as each of those buffers gets emptied by the driver, so you can fill them with new data. It works well for me, even though I have awkward sampling rates to deal with. In my case I set up a bunch of twenty-one packets where twenty of them carry 40 bytes and the twenty-first carries 44 bytes!
Hope that helps
- Tony
I have a C++ application that receives a timestamped audio stream and attempts to play the audio samples as close as possible to the specified timestamp. To do so I need to know the delay (with reasonable accuracy) from when I place the audio samples in the output buffer until the audio is actually heard.
There are many discussions about audio output latency but everything I have found is about minimizing latency. This is irrelevant to me, all I need is an (at run-time) known latency.
On Linux I solve this with snd_pcm_delay() with very good results, but I'm looking for a decent solution for Windows.
I have looked at the following:
With OpenAL I have measured delays at 80ms that are unaccounted for. I assume this isn't a hardcoded value and I haven't found any API to read the latency. There are some extensions to OpenAL that claims to support this but from what I can tell it's only implemented on Linux.
Wasapi has GetStreamLatency() which sounds like the real deal but this is apparently only some thread polling interval or something so it's also useless. I still have 30ms unaccounted delay on my machine.
DirectSound has no API for getting latency? But can I get close enough by just keeping track of my output buffers?
Edit in response to Brad's comment:
My impression of ASIO is that it is primarily targeted for professional audio applications and audio connoiseurs, and that the user might have to install special sound card drivers and I will have to deal with licensing. Feature-wise it seems like a good option though.
I have a machine with 2x3 3ghz dual-core xeon and 4x10krpm scsi 320 disks in raid0.
The capture card is an osprey 560 64 bit pci card.
Operating system is currently Windows Server 2003.
The video-stream that I can open with VLC using direct show is rather nice quality.
However, trying to save this video-stream without loss of quality has proven quite difficult,
using the h264 codec I am able to achieve a satisfying quality, however, all 4 cores jump to 100% load after a few second and then it start dropping frames, the machine is not powerful enough for realtime encoding. I've not been able to achieve satisfying mpeg1 or 4 quality, no matter which bitrate I set..
Thing is, the disks in this machine are pretty fast even by todays standard, and they are bored.. I don't care about disk-usage, I want quality.
I have searched in vain for a way to pump that beautiful videostream that I see in VLC onto the disk for later encoding, I reckon the disks would be fast enough, or maybe something which would apply a light compression, enough that the disks can keep up, but not so much as to loose visible quality.
I have tried FFMPEG as it seems capable of streaming a yuv4 stream down to the disk, but ofcause FFMPEG is unable to open the dshow device ( same error as this guy Ffmpeg streaming from capturing device Osprey 450e fails )
Please recommend a capable and (preferably) software which can do this.
I finally found it out, it is deceptively simple:
Just uncheck the "transcode" option in the window where you can select an encoding preset.
2 gigabytes per minutes is a low price to pay for finally getting those precious memories off of old videotapes in the quality they deserve.