I'm looking for an OSX (or Linux?) application that can recieve data from a webcam/video-input and let you do some image processing on the pixels in something similar to c or python or perl, not that bothered about the processing language.
I was considering throwing one together but figured I'd try and find one that exists already first before I start re-inventing the wheel.
Wanting to do some experiments with object detection and reading of dials and numbers.
If you're willing to do a little coding, you want to take a look at QTKit, the QuickTime framework for Cocoa. QTKit will let you easity set up an input source from the webcam (intro here). You can also apply Core Image filters to the stream (demo code here). If you want to use OpenGL to render or apply filters to the movie, check out Core Video (examples here).
Using theMyMovieFilter demo should get you up and running very quickly.
Found a cross platform tool called 'Processing', actually ran the windows version to avoid further complications getting the webcams to work.
Had to install quick time, and something called gVid to get it to work but after the initial hurdle coding seems like C; (I think it gets "compiled" into Java), and it runs quite fast; even scanning pixels from the webcam in real time.
Still to get it working on OSX.
Depending on what processing you want to do (i.e. if it's a filter that's available in Apple's Core Image filter library), the built-in Photo Booth app may be all you need. There's a comercial set of add-on filters available from the Apple store as well (http://www.apple.com/downloads/macosx/imaging_3d/composerfxeffectsforphotobooth.html).
Related
I'm looking for a way to extract a set of specific pixels from a video (and possibly store them in JSON)
So far i have found FFmpeg which looks like it should do the heavy lifting if i can find the correct commands.
Alternatively I could try using the source and building my own project that just leverages FFmpeg to extract frame data, but i think/hope that's unnecessary.
So if its possible, What commands could accomplish this?
Or perhapse there is a whole different approach i could take, any help would be great!
After a solid day of research into various tools, i stumbled across Accord.NET which is a machine learning SDK, and one of the big parts of machine learning is image processing.
Simply adding the 'Accord.Video.FFMPEG' NuGET package allows me to access their 'VideoFileReader.ReadVideoFrame' and combined with the standard .Net 'Bitmap.GetPixel' i now have a super simple little CLI that can output a pixel colour for my position of choice.
http://accord-framework.net/
I'd like to write some code that generates some pretty simple musical tones (notes) and has them output through the speaker (whatever sound device).
I suspect I'll likely need to generate as MIDI data, which I can go figure out independently, but I'm new to audio programming generally and I'm not sure what the best entry point into the system frameworks is. AudioToolbox has these MusicSequence objects. There's also Core MIDI and Core Audio. None has an obvious interface for "here's a data structure for a bunch of notes, now call this method to play them", so I'll presumably need some combination of these to cobble it together.
I'm confident that OS X supports this. If anyone has context with this kind of work, I'd appreciate a couple basic pointers on where in the docs (or other resources) to start looking for building whatever structures represent music data and where you'd turn around and trigger playback.
OS X does support this, but it's a lot more inherently complex than it might seem at first. There are essentially three pieces:
MusicSequence is the "data structure for a bunch of notes" (along with timing information in the form of a tempo/meter map.
MusicPlayer is the object that controls playback of the MusicSequence.
AUGraph is what you'd use to create an instrument object and hook it up to your physical outputs, to turn the note data into sound.
There's a lot of potential variety in how you set up the AUGraph. For example, the default General MIDI synthesizer is the built-in DLSMusicDevice, but you could also load an FM synth, a sampler, or any number of other instrument units. From there, you could be processing the audio in various ways and routing it to various devices. All that stuff that falls in the general category of "audio processing" happens within the AUGraph.
Apple's PlaySequence sample code does mostly what you're looking for. It's a C++ project—but MusicSequence, MusicPlayer, and AUGraph are plain C APIs, so it should be a decent starting point. https://developer.apple.com/library/mac/samplecode/PlaySequence/Introduction/Intro.html
my aim is to perform analysis (like DFT) on an audio file (mp3).
Then :
my input is a file
And my output is a treatment
I would like to use QTKit framework to perform this, but I am a bit disappointed:
QTMovie is able to open a file but I don't see own to access to decompressed audio buffer
QTSampleBuffer can be treat with QTCaptureDecompressedAudioOutput but I don't find how to open a file (the only input seems to be QTCaptureDeviceInput)
Is there a way to do what I want with QTKit or should I use Core Audio (or other) which will be more difficult (and I prefer Objective-C than C or C++) ?
(Actually I have no code, I am just trying to find the good way and it the first time I use sound...)
QTKit won't let you do that. You'll have to use Core Audio. You could always take a look at this code (which is written for the iPhone but most of the code works on Mac OS X) to understand everything a bit more. It detects frequency using FFT.
I also was afraid of using Core Audio, but in the end it all worked out pretty well.
I have a very specific problem: I want to write my own DMX-Software to control our DMX-fixtures. Does anyone know a interface to use? It would be great if there would be any Framework for using it, so that I only have to sent the channel and the value to the interface.
I noticed your question was for Mac, but I wrote a Windows specific C++ program, which could probably be easily modified. It's adapted from the C# example on Enttec's OpenUSB website. See:
https://github.com/chloelle/DMX_CPP
There's some really good information & code samples (including a working class that I wrote) here: Lighting USB OpenDMX FTD2XX DMXking
Ultimately, you end up setting byte values (between 0 and 255[FF] (brightest) in a byte array.
It's fairly trivial to implement simple effects such as fades or chases.
You would need to use a USB controller to convert your program's instructions to the actual hardware.
I suggest using a simple iphone application talking to a webservice which then interacts with the hardware.
Code samples above are in c# though will show you how to interact with a DMX controller
Is it possible to use the NSSpeechRecognizer with an pre-recorded audio file instead of direct microphone input?
Or is there any other speech-to-text framework for Objective-C/Cocoa available?
Added:
Rather than using voice at the machine that is running the application external devices (e.g. iPhone) could be used for sending just an recorded audio stream to that desktop application. The desktop Cocoa app then would process and do whatever it's supposed to do using the assigned commands.
Thanks.
I don't see any obvious way to switch the input programmatically, though the "Speech" companion guide's first paragraph in the "Recognizing Speech" section seems to imply other inputs can be used. I think this is meant to be set via System Preferences, though. I'm guessing it uses the primary audio input device selected there.
I suspect, though, you're looking for open-ended speech recognition, which NSSpeechRecognizer is not. If you're looking to transform any pre-recorded audio into text (ie, make a transcript of a recording), you're completely out of luck with NSSpeechRecognizer, as you must give it an array of "commands" to listen for.
Theoretically, you could feed it the whole dictionary, but I don't think that would work since you usually have to give it clear, distinct commands. Its performance would suffer, I would guess, if you gave it a bunch of stuff to analyze for (in real time).
Your best bet is to look at third-party open source solutions. There are a few generalized packages out there (none specifically for Cocoa/Objective-C), but this poses another question: What kind of recognition are you looking for? The two main forms of speech recognition ('trained' is more accurate but less flexible for different voices and the recording environment, whereas 'open' is generally much less accurate).
It'd probably be best if you stated exactly what you're trying to accomplish.