I was told this belongs in programming, not in the Signal Processing sub-exchange.
Is there a way to implement spectrum analysis (specifically FFT) on live audio in Windows?
I want to be able to read in audio from a microphone, and have a display on the screen that shows the resultant Fourier Transform.
It would also be useful if I could execute a program if a certain set of FFT characteristics occurred.
Thanks guys!
To do this you have a couple of options depending on your most preferred language/framework basically. I'm not sure how new you are to signal processing so I'll suggest a few options.
Visual Programming
These are all visual programming environments which don't actually require any writing of code, however Simulink and Pure Data both require a runtime for the user to run the program.
Simulink (Paid)
MathWorks/Matlab's visual programming tool that works really well in real-time (in my opinion). Using the Audio System Toolbox, you can easily capture microphone input from your system in realtime and carry out the FFT processing, plot the spectrum and, like you said, if certain FFT conditions are met then to carry out some further processing.
This isn't free software and requires having the Matlab/Simulink run-time installed to be used. You can also script your processing in Matlab's .m language as desired (a cross between Java, JS and C).
Max MSP (Paid)
A similar version to Simulink but developed as a standalone visual programming tool. This will allow you similar freedom to Simulink but I think it will be easier for re-distribution.
You can compile MAX MSP into executables to give to someone straight away. Here is a reference to get you started on using the FFT in MAX. Again, this isn't free but if you wanted to learn more about it then I think it's worth the money (if I recall it's not too expensive).
If you need some more custom processing than the built in modules, I believe you can design custom MAX modules using C or JavaScript. Max is designed to easily get system audio input / outputs and here's a link to get you started.
Bonus: You can design your own Ableton Live plugins with the Max4Live addon which just lets your MAX MSP projects get compiled into .VST format. So you can build custom FX if you are into music production.
Pure Data (PD) (Free)
A very bland open source version of MAX MSP but completely free. It may look dull at first but a lot of researches I know use it to build fairly complex systems that can do some serious data processing. There are also lots of community built extras for PD if you ever needed a custom module. Here is a link to get you started on the FFT in PD. You cannot compile applications with PD, but since it's completely free to install anyone can run your program after installing PD. Another link for troubleshooting audio I/O in PD (if it isn't working right out of the box).
Programming Languages
Now the visual stuff is a really good way to get started if you aren't already introduced to DSP or audio programming. Otherwise here a just a few options and links to get started and where I would recommend.
Matlab & Octave
Like before, the Audio Systems Toolbox supports realtime audio I/O within a Matlab script. This combined with Matlab's built in FFT function can have you setup programming realtime FFTs and plotting the response in no time at all (less than 10 lines of code or something).
Octave has it's own version of the FFT function and different backends for rendering plot responses, but no Audio Systems Toolbox. However, Playrec is also an opensource alternative for audio I/O in Matlab/Octave that supports realtime audio input and output.
(Octave is an open source equivalent to Matlab (Matlab needs a paid license to develop a program), but does not support all Matlab supported features).
Python
Due to the PyAudio module, realtime audio I/O and DSP is more possible using Python! I would recommend Python if you are just starting out for sure since it's a nice introduction into any programming language and can help with teaching the fundamentals of DSP before attempting lower level languages.
Here's where you can get started with real-time non-blocking audio I/O in Python with PyAudio. To plot your data you can use a library such as matplotlib (designed similar to Matlab's easy plotting functionality).
For your FFT there are multiple libraries out there but I'd start with the Scipy / Numpy one.
C
One of the classic (and sometimes) most daunting programming language. With no objects (unless you want to make them yourself) or other high level abstractions, C is one of the few languages that still feels like you're building a lot from the ground up (which personally I like).
To get started with audio, I'd look at, in my opinion, the most widely used cross-platform audio I/O library; portaudio. This will let you access the soundcard data inputs and outputs in realtime on Mac, Linux & Windows.
Once you get this up and running an FFT I would use to get started would be the KissFFT just because of it's pure simplicity to use. If you want to plot the data, I would maybe look at gnuplot, but this is'nt a very pretty route in terms of development.
If you are very new to programming I would not recommend this unless you really want to get stuck in.
C++
Both KissFFT and portaudio will also compile with C++ code, but here are a couple of higher level alternatives.
One of my favourites is the JUCE framework / development environment. It has built in cross-platform audio I/O and already has a custom FFT function as part of the framework. You can build custom VSTs for your music DAW if you want to as well. It also comes with 'easy' (if you know C++) access to graphics windows with higher level access to openGL, so you could get fancy when plotting your data in real-time. If i remember correctly, one of the Demo projects on first installation is a real-time FFT plot you can compile and see the input from your laptop mic. JUCE is free for personal use, but comes with a small license fee as an indie developer.
Otherwise another one that comes to mind is the QT C++ library/framework for UI design (mainly). This is a cross platform easy to use GUI designer, that also has high level classes for obtaining audio input from a Mac/Win/Linux mic. Here is just one example I came across using QT's multimedia classes and FFTReal to plot a realtime FFT spectrum.
Summary
I've suggested a lot of options, but also missed out a few other people may recommend like languages such as R, C#, Java, Rust etc... and there are so many suggestions it's impossible to cover them all but I think this should be enough to get started. If it were me in terms of experience:
Complete Beginner to Programming: Max MSP
Novice / Knows their way around a little bit: Python (with PyAudio)
Programmed in other languages maybe looking to gain more programming skills : C++ with JUCE
Any of these languages you pick will be good in future reference for software positions and many companies / researchers use them to prototype / develop realtime audio processing software.
This is just my opinion but hopefully this gets you well along your way!
Related
I'm now a bit experienced with using OpenGL, which I started using because it's said that it is the only way to invoke video card functions. (besides DirectX - which I like less than OpenGL)
For programming (e.g. in C/C++) the OS gives many APIs, like functions for printing. But these can also be bypassed, by coding in Assembly-language - and call much lower level APIs (which gain speed) or direct CPU calls.
So I started wondering why this wouldn't be possible on the video card. Why should an API like OpenGL or DirectX be needed? The process going on with those is:
API-call >
OS calls video card (with complex opcodes, I think) >
video card responses (in complex binary format) >
OS decodes this format and responses to user (in expected API format)
I believe this should decrease the speed of the rendering process.
So my question is:
Is there any possibility to bypass any graphical API (under Windows) and make direct calls to the video card?
Thanks,
Dennis
Using assembly or bypassing an api doesnt automatically make something faster, often slower as you dont know what the folks that wrote the library know.
it is absolutely possible yes, those libraries are just processor instructions that poke and peek at registers and ram, and you could just as easily poke and peek at registers and ram. The first problem is can you get that information, sure, you can look at the linux drivers or other open source resources. Second, much of the heavy lifting today is done in the graphics chip by logic or graphics processors, so the host is just a go between and not necessarily the bottleneck if there is a bottleneck. And yes you can program the gpus depending on your video card/chip, etc.
You need to determine where the bottleneck really is, if there really is one, maybe the bus is your problem, maybe the operating system is your problem, or the compiler, or the hard disk or the system memory, the processor and architecture itself, caches, etc. At the same time how will you ever learn how to find these things unless you try.
I recommend getting rid of windows completely, no operating system, go bare metal. Take the linux and other open source resources plus anything you can get from the vendor and get closer to the metal. You will also need a lot of info about the pci/pcie bus and bridges, dma controllers, everything in the path. If you dont want to go that low then use linux or bsd or some other command line environment where it is well known how to take over the video system, and take over the video system while retaining an operating system and a development environment (vi/emacs, gcc).
if that is all way too advanced, then I recommend, dabbling in simple gpu routines to get a feel for how the video card works at least at some level and tackle this learning exercise one step at a time.
I am looking for tools and techniques for prototyping (virtual prototyping), simulation, and testing of deeply embedded C code on desktop Windows, including building realistic embedded front panels consisting of buttons, LEDs, and LCD displays (both segmented and graphic).
I'm specifically interested in a possibly low-level approach, using pure C code and raw Win32 API rather than MFC, .NET/C#, vxWidgets or Qt. I'd also like to use free development tools, such as Visual C++ Express with Platform SDK and ResEdit for editing resources.
I'm looking for code examples to render graphic LCDs (from monochrome to 24-bit color) with efficient pixel-level interface, multi-segment LCDs, and owner-drawn buttons that respond both to "depressed" and "released" events.
I am surprised that my original question triggered so many misunderstandings and adverse comments. The strategy of developing deeply embedded C code on one machine (e.g., a PC) and running it on another (the embedded microcontroller) is called "dual targeting" and is really quite common. For example, developing and testing deeply embedded code on the PC is the cornerstone of the recent book "Test-Driven Development for Embedded C" by James Grenning.
Avoiding Target Hardware Bottleneck with Dual Targeting
Please note that dual targeting does not mean that the embedded device has anything to do with the PC. Neither it means that the simulation must be cycle-exact with the embedded target CPU.
Dual targeting simply means that from day one, your embedded code (typically in C) is designed to run on at least two platforms: the final target hardware and your PC. All you really need for this is two C compilers: one for the PC and another for the embedded device.
However, the dual targeting strategy does require a specific way of designing the embedded software such that any target hardware dependencies are handled through a well-defined interface often called the Board Support Package (BSP). This interface has at least two implementations: one for the actual target and one for the PC, for example running Windows. With such interface in place, the bulk of the embedded code can remain completely unaware which BSP implementation it is linked to and so it can be developed quickly on the PC, but can also run on the target hardware without any changes.
While some embedded programmers can view dual targeting as a self-inflicted burden, the more experienced developers generally agree that paying attention to the boundaries between software and hardware is actually beneficial, because it results in more modular, more portable, and more maintainable software with much longer useful lifetime. The investment in dual targeting has also an immediate payback in the vastly accelerated compile-run-debug cycle, which is much faster and more productive on the powerful PC compared to much slower, recourse-constrained deeply embedded target with limited visibility into the running code.
Front Panel Win32 GUI Toolkit
When developing embedded code for devices with non-trivial user interfaces, one often runs into the problem of representing the embedded front panels as GUI elements on the PC. The problem is so common, that I'm really surprised that nobody here could recommend an existing library or an open source project, which would provide a simple C-only interface to the basic elements, such as LCDs, buttons, and LEDs. This is really not that complicated, yet it seems that every embedded developer has to re-invent this wheel over and over again.
So, to help embedded developers interested in prototyping embedded devices on Windows, I have created a "Front Panel Win32 GUI Toolkit" and have posted it online under the GPL open source license (see http://www.state-machine.com/win32). This toolkit relies only on the raw Win32 API in C and currently provides the following elements:
Dot-matrix display for an efficient, pixel-addressable displays such as graphical LCDs, OLEDs, etc. with up to 24-bit color
Segment display for segmented display such as segment LCDs, and segment LEDs with generic, custom bitmaps for the segments.
Owner-drawn buttons with custom “depressed” and “released” bitmaps and capable of generating separate events when depressed and when released.
The toolkit comes with an example and an App Note (see http://www.state-machine.com/win32/AN_Win32-GUI.pdf), showing how to handle input from the owner-drawn buttons, regular buttons, keyboard, and the mouse. You can also view an animated demo at http://www.state-machine.com/win32/front_panel.html.
Regarding the size and complexity of the "Front Panel Win32 GUI Toolkit", the implementation of the aforementioned GUI elements takes only about 250 lines of C. The example with all sources of input and a lot of comments amounts to some 300 lines of C. The toolkit has been tested with the free Visual C++ Express 2010 (with the Express Edition Platform SDK) and the free ResEdit resource editor.
Enjoy!
The appliances you mention in your comment clarification to the question will never be using a windows PC, so low level windows programming is not a requirement in that case. In fact, I'd say its undesirable. Prototyping is about speed. It's about how fast you can put something together to show potential investors or upper management or some other decision maker.
You wouldn't want to spend the extra time with low level C and Win32 api until the project requirements were flushed out enough that you knew that was an absolute requirement for the final project deliverables (perhaps a server/PC monitoring tool?). Until then you want speed of development. Lucky for you the industry has tools for rapid prototyping and development of hardware like you describe.
My Preference for Prototyping with Embedded Development
As for my opinion as a developer, I like the .net microframework (.netmf) simply because I'm already a Microsoft .Net developer and can transfer a lot of my existing skills. Therefor I prototype with a FEZ microcontroller using C# under Visual C# Express 2010 (free as you required). Its fast, easy and you are working on the core of your project in minutes.
If your experience as a developer is different, you may look for a micro controller which is programmed using BASIC, Java or some other language to help with the speed of development by reusing your core skill set.
Addressing your Question Bounty Comments
Astonishingly large portions of the embedded software can be developed
on the desktop computer as opposed on the deeply embedded target. This
avoidance of the "target system bottleneck" can potentially improve
productivity by an order of magnitude, if done right. However, to
develop embedded software on the desktop, one needs to simulate the UI
components, such as displays (both segmented and increasingly
graphical), LEDs, knobs, and buttons. I'm looking for such UI
components written in plain Win32 API in C for easy integration with
embedded code to be developed and tested on the desktop Windows.
I did embedded development full time professionally for well over 4 years as well as many years surrounding that part time. While what you said above is somewhat true, it will not save you time or money which is why everyone is confused about the motivation for this strategy. We spent years trying to put out a windows emulator for this company's hardware devices that would theoretically save time for prototyping. It was always a pain and we spent many more hours of work trying to emulate the experience than if we just went straight from sketched UI drawing specs to real development. The emulator lagged behind hardware development and often wouldn't support the latest features until 6 months or more after the hardware was released. It was a lot of extra work for very little value.
You will spend more of your time developing non-reusable win32 platform code and hardware emulation components than actually writing the code for the core project itself. This only ever makes sense for hardware vendors who provide this emulator as a 'value add' tool to potential 3rd party developers, but it does not make sense for prototyping new hardware designs.
Modern development environments like Visual C# Express 2010 with a FEZ microcontroller can compile, push the project output to the microcontroller, and then begin debugging just as fast or faster than you could compile and run a low level windows app in C emulating LCDs or LEDs or switches, etc... So your comment, "improve productivity by an order of magnitude", is simply no longer true with modern tools. (It may have been prior to the last 10 years or so.)
If you really, truly just want to simulate the embedded hardware visually on a PC use something like adobe flash to mock up a UI. But don't duplicate code by coding for windows when the final device you are prototyping won't be running windows (maybe it will be, but you didn't say that). Use the fastest most reliable prototyping tools available today, which is unequivocally not low level C and win32 api!
Maybe use StackExchange for Electronics?
Because this is a development oriented site, discussion about the merits of specific embedded hardware isn't really relevant. If you decide to refocus on using microcontroller electronics for prototyping (Arduino, FEZ, Propeller, Basic Stamp, Pololu, etc) you might ask for electronics hardware advice on stackexchange for electronics. I will say that most of those platforms are designed to facilitate the prototyping of LCDs, LEDs, buttons and interfaces as you outlined. You can usually assemble a few pre-built modules in a matter of minutes and be ready to start coding your project. Huge time savings can be had here.
You are asking for too much you need to take a look # proteus.
http://www.labcenter.com/products/vsm_overview.cfm
As Mahmoud said, you may find your code solution with prototyping example in proteus professional. It is one of popular software for prototyping, simulation and coding, you can download proteus professional for free and check their manual.
Best of luck
I am new to opengl and using C#,opentk for development. My Application is very light weight (just 2d graphics) and i am planning to use software rendering when hardware rendering is not available.
How do i make sure software rendering works on all computers ? (when hardware rendering is not available.)
Should i distribute Software rendering libraries like Mesa, myself. or it will already available on all (Windows) OS ?
in other words, opengl32.dll is always available on all modern windows OS ( > XP SP2 ) or should i distribute that also ?
( My Application is very simple (simple 2d graphics) as of now. I selected opengl instead of GDI+/WPF because, i may extend it to 3D in future. )
OpenGL is a system library. You should not distribute it with your application. Especially on Unix/Linux systems, where it should be installed using the distribution's package manager.
Since opengl32.dll is included in Windows, it falls back to Software Rendering automatically if the pixel format you chose in your application isn't hardware accelerated by the graphics driver.
I tried leveraging OpenTk as well, but in itself creates a dependency and - particularly as a newbie - doesn't really do anything but confuse learning OpenGL versus learning someone else's interpretation of the framework.
OpenGL is - as the other answerer suggested - a system library. With this, it's a functions contained in a C DLL which you import through the API.
OpenTK imports these functions for you, that's the only real benefit it adds, but in doing so, many of the types are reinterpreted as are the function calls as per the author of OpenTK.
This creates an additional learning curve - as most of the internet references you're going to find are going to be OpenGL - so not only will you be struggling with understanding OpenGL - which isn't easy - but you're also going to be dealing with OpenTK interpretations of the OpenGL standards.
Now keep in mind that MANY open source projects such as OpenTK start as open source until they get sufficient enough user base, when they convert over to a for profit model. So let's say you learn and become dependent on OpenTK, well if/when they switch to a for profit model and you're tapped on cash, you are SOL (Shit outta luck). Or you have to pay their price.
What I did was - I took the source for API mapping for OpenTK's OpenGL mapping and renamed everything as per my tastes. it's a bit of work, but it's worth the labor and it helped me get to understand OpenGL.
As for distribution. I have absolutely no external dependencies I rely on other than the OpenGL DLL which should already be on the system.
ALL the DLLs you need for OpenGl will ALREADY be preinstalled on any windows OS you're dealing with. I can't speak for other OS flavors, but I suspect this may be the case.
On a final note: OpenGL handles 'toggling between software and hardware rendering' innately. So libraries like MESA and OpenTK add VERY little value at HIGH potential costs.
What are those costs?
1) Redistributable packaging and licensing. They still come with a license and most license are subject to change at any time.
2) Conversion from open source or free distribution to a for profit model.
Invest in yourself. OpenGL documentation is vast and at times confusing, and my advice is to avoid the knee jerk temptation to 'take the easy' path versus the leverage other's models - for one simple reason.
I know you're using this for a 2d application. And even if you're using Orthogonal view, the fact of the matter is you're learning OpenGL with is a patterned with 3d in mind. So give yourself the gift up front of teaching yourself because there really is no 'easy path' to understanding 3d - and thus no real value add to the external dependencies you're leaning towards using.
One thing to keep in mind: Modeling. IF/when you switch to 3d modeling, doing vertex creation through hand coding in opengl is a bitch. I use Blender to create my obj models in and read those into my own c# application which reads in the 3d models and lets me manipulate them from there.
I HAD been using C++, which sure is faster, but once I converted the APIs to c# code and started managing my own memory leveraging the garbage collection model, it became SO much easier than having to learn someone else's library.
Dont use redistributeables. And leverage the code from OpenTK, with modification, but don't include OpenTK as a redistributeable.
That's my advice.
Ok, this is an interesting question to ask because everyone has a say in it.
What is your favorite library to program in for GUI's and the language that you program it in. Give a short reason why. (ex. Gtk, Qt, Windows, etc..) Just an FYI, this includes any scripting language that you program a GUI in Python, Perl etc...
Frankly I've always done Gtk in C, but I'm starting to warm up to Qt in C++ with the new KDE. I've never been a big fan of Windows programming.
ChrisW. stated that I did not give a reason for Gtk/Qt so here goes. I started with Gtk because when I started programming GUI's I was working in Linux and there was more Gtk information available. Started utilizing Qt when I started working more in KDE but really the move to Qt was based on trying to move to C++ and learn more languages. I've never been a fan of basic Windows programming, but I do enjoy a little DirectX now and then :P
Recently I had the opportunity to work with both wxWindows and QT, while some time ago I wrote some small programs using FLTK and Gtk. My conclusion is that widget libraries tend to be very similar; each one has its strenghts and its quirks.
Instead of advocating a specific library, then, I would like to advocate the use of high level languages in GUI programming: the development cycle is way faster and GUI programs are rarely CPU bound, so the performance hit is rarely a problem.
If a GUI program has to perform some intense computations, just develop a core library in C or C++, but keep the interface in Python or whatever other interpreted language.
People like to bash Swing for being old, slow and ugly, but it's just not true. Swing is mature, is faster than ever on Java SE 6/10, looks decent enough, and is tolerable to program. Above all, I've found Java + Swing to be the most trouble-free cross-platform combination. It also works remarkably seamlessly with Jython (Python on JVM).
SWT could also be an option, but so far I've been happy with Swing.
I realise you're focusing on application GUIs but if you want a quick, powerful and fun way to visualize anything on your computer, you can't go past Processing
From the site:
Processing is an open source
programming language and environment
for people who want to program images,
animation, and interactions. It is
used by students, artists, designers,
researchers, and hobbyists for
learning, prototyping, and production.
It is created to teach fundamentals of
computer programming within a visual
context and to serve as a software
sketchbook and professional production
tool. Processing is an alternative to
proprietary software tools in the same
domain.
WPF in particular, and XAML in all its reincarnations (WPF, Silverlight, Moonlight).
C# on top of .Net 3.5/Mono: $0
Visual Studio Express/MonoDevelop: $0
Being able to tell the designer "make my program pretty" and continue coding features: priceless.
I liked writing to video memory under DOS: for an animated game (i.e. an Asteroids clone), that was as fast (performant) as I knew how to do it (certainly faster that using the BIOS API).
This is really a somewhat subjective question, so there is no best or correct answer to it. The following is based on my (limited) experience:
I personally like wxWidgets with PLT Scheme for writing simple but flexible GUIs. There are much more advanced toolkits, but I usually do not need their features. wxWidgets is flexible and the Scheme interface follows Scheme traditions of being powerful with a relatively simple structure. I like the fact that wxWidgets is portable, and yet tries not to actually draw its own widgets, but can use native or common toolkits of the environment it is used under. It is written in C++ but I never used its C++ interface.
That is not to say that in my opinion Scheme will generally be the optimal language to write your application in. In fact there are many kinds of applications I would not write in Scheme, even though I like the language. But regarding the GUI programming part, that is my favourite because of its straight-forwardness, and the way that a functional language like Scheme goes well with declarative-style GUI programming.
Of course you will not have the same level of control when using that as when having your program involved in every stage of the window construction and input reaction, by using an event loop (such as with Win32API or Xt/Intrinsics). But that is not always convenient and often unnecessary, and seems to become decreasingly common.
Note: The wxWindows toolkit was renamed wxWidgets, but my installation of a rather recent version of PLT Scheme still comes with the older wxWindows. I am not sure whether there is an updated package of wxWidgets available or if it is going to be included in a future version of PLT Scheme.
Qt4 without question for me. Now that it has an LGPL license it makes sense for all kinds of applications that previously weren't possible. Additionally, it changes C++ in ways that dramatically improve the experience of using the language. (Things like a foreach and forever loop, atomic operations on integers, and memory management)
Gtk and is the primary window-drawing graphical subsystem I have experience working with (and is therefore my favorite XD).
As far as general graphics subsystems go, however, OpenGL (typically in combination with GLUT) has been an easy and productive ride for me. Regrettably I have little DirectX experience to compare to, though :S
For writing souped-up versions of standard Windows components, I loved Borland's VCL, and am very pleased with .NET.
I am using GNUplot to plot large volumes of data that I want to visualize in 3D - e.g., using the pm3d option. This is very slow rendering in software - is there free software that will do this using OpenGL or something?
ROOT will do this.
This beast is one of the major analysis and visualization tools used in the nuclear and particle physics communities these days. It can move a lot of data fast, and has fairly sophisticated set of visualization tools, and supports OpenGL. You can do the work in compiled c++ code, in the cint interpreter, or using the python bindings.
The big downside is that you may have to read a lot of documentation before you're ready to begin.