I'm about to start a project that will record and edit audio files, and I'm looking for a good library (preferably Ruby, but will consider anything other than Java or .NET) for on-the-fly visualization of waveforms.
Does anybody know where I should start my search?
That's a lot of data to be streaming into a browser. Flash or Flex charts is probably the only solution that will be memory efficient. Javascript charting tends to break-down for large data sets.
When displaying an audio waveform, you will want to do some sort of data reduction on the original data, because there is usually more data available in an audio file than pixels on the screen. Most audio editors build a separate file (called a peak file or overview file) which stores a subset of the audio data (usually the peaks and valleys of a waveform) for use at different zoom levels. Then as you zoom in past a certain point you start referencing the raw audio data itself.
Here are some good articles on this:
Waveform Display
Build an Audio Waveform Display
As far as source code goes, I would recommend looking through the Audacity source code. Audacity's waveform display is pretty good and mostly likely does a similar sort of data reduction when rendering the waveforms.
i wrote one:
http://github.com/pangdudu/rude/tree/master/lib/waveform_narray_testing.rb
,nick
The other option is generating the waveforms on the server-side with GD or RMagick. But good luck getting RubyGD to compile.
Processing is often used for visualization, and it has a Ruby port:
https://github.com/jashkenas/ruby-processing/wiki
Related
Is it possible to modify the Vuforia video stream for better tracking performance?
Step 1: Get the raw pixel data from the VuforiaBehaviour.Instance.CameraDevice.GetCameraImage();
Step 2: Modify the pixels with post processing via custom shaders in Unity. For example apply a threshold or edge detection.
Step 3: Vuforia Engine uses the modified video input to track images.
That´s the idea but I´m not sure if Vuforia is gonna pass the modified video into the Vuforia Engine then or still uses the unmodified video input for tracking?
If anybody has experience with that I would be thankful for your help! :)
Vuforia Engine assumes that the input images look like "natural" images. Passing an image belonging to a different domain (e.g., the result of an edge detector) is unlikely to improve tracking performance.
That said, tracking performance is affected by image quality. For example, if images are blurry, tracking robustness is going to suffer. If this is the case you might want to look at trying to adjust system camera parameters via the platform API (iOS, Android, etc.). However, please note that this might or might not be possible depending on the platform. Also, on some platforms when a device tracker like ARKit or ARCore is used, the platform tracker itself adjusts the camera parameters for good tracking performance. For example it might keep the exposure time low to reduce blur.
Assuming a simple product demo e.g. the one found on http://www.sublimetext.com/
i.e. something this isn't traditional high res video and could be reasonable accomplished with:
animated gif
video (can be embedded youtube, custom html5 player, whatever is most competitive)
canvas
The question is, which performs better for the user? Both in terms of:
The size of the files the user must be downloaded to view the 'product demo'
The requirements in terms of processing power to display the 'product demo'
If you feel that there's a superior technology to accomplish this or another metric to judge its usefulness, let me know and I'll adjust accordingly.
I know it's already answered, but as you specifically referred to the Sublime Text animation I assume you're wanting to create something similar?
If that's the case then here is a post explaining how it was created by the Sublime Text author, himself:
http://www.sublimetext.com/~jps/animated_gifs_the_hard_way.html
The interesting part of the article is how he reduces the file size - which I believe is your question.
With a simple animation such as the one at the link you're referring to, with a very low frame rate, a simple animated-PNG of animated GIF will probably be the best solution.
However, you need to consider band-width factor in this. If the final size of the GIF or the PNG is large then probably a buffered video is probably better.
This is because the whole gif/png file needs to be downloaded before it shows (I am not sure how interleaved PNGs works when they contain animation though).
A video may be larger in file size, but as it is typically buffered you will be able to show the animation almost right away.
Using external hosts such as YouTube or others can be beneficial to your site as well as the band-width is drawn from those site and not from your server (in case you use a provider that limits or charge for this in various ways).
For more information on animated PNGs or APNG (as this is not so well-known):
https://en.wikipedia.org/wiki/APNG
The canvas in this is only a displaying device and not really necessary (an image container does the same job and can also animate the GIF/PNG whereas a canvas cannot).
If you use a lot of vectors then canvas can be considered.
CSS3 animation is also an option for things such as presentation slides.
I'm a structural engineering master student work on a seismic evaluation of a temple structure in Portugal. For the evaluation, I have created a 3D block model of the structure and will use a discrete element code to analyze the behaviour of the structure under a variety of seismic (earthquake) records. The software that I will use for the analysis has the ability to produce snapshots of the structure at regular intervals which can then be put together to make a movie of the response. However, producing the images slows down the analysis. Furthermore, since the pictures are 2D images from a specified angle, there is no possibility to rotate and view the response from other angles without re-running the model (a process that currently takes 3 days of computer time).
I am looking for an alternative method for creating a movie of the response of the structure. What I want is a very lightweight solution, where I can just bring in the block model which I have and then produce the animation by feeding in the location and the three principal axis of each block at regular intervals to produce the animation on the fly. The blocks are described as prisms with the top and bottom planes defining all of the vertices. Since the model is produced as text files, I can modify the output so that it can be read and understood by the animation code. The model is composed of about 180 blocks with 24 vertices per block (so 4320 vertices). The location and three unit vectors describing the block axis are produced by the program and I can write them out in a way that I want.
The main issue is that the quality of the animation should be decent. If the system is vector based and allows for scaling, that would be great. I would like to be able to rotate the model in real time with simple mouse dragging without too much lag or other issues.
I have very limited time (in fact I am already very behind). That is why I wanted to ask the experts here so that I don't waste my time on something that will not work in the end. I have been using Rhino and Grasshopper to generate my model but I don't think it is the right tool for this purpose. I was thinking that Processing might be able to handle this but I don't have any experience with it. Another thing that I would like to be able to do is to maybe have a 3D PDF file for distribution. But I'm not sure if this can be done with 3D PDF.
Any insight or guidance is greatly appreciated.
Don't let the name fool you, but BluffTitler DX9, a commercial software, may be what your looking for.
It's simple interface provides a fast learning curve, may quick tutorials to either watch or dissect. Depending on how fast your GPU is, real-time previews are scalable.
Reference:
Model Layer Page
User Submitted Gallery (3D models)
Jim Merry from tetra4D here. We make the 3D CAD conversion tools for Acrobat X to generate 3D PDFs. Acrobat has a 3D javascript API that enables you to manipulate objects, i.e, you could drive translations, rotations, etc of objects from your animation information after translating your model to 3D PDF. Not sure I would recommend this approach if you are in a hurry however. Also - I don't think there are any commercial 3D PDF generation tools for the formats you are using (Rhino, Grasshopper, Processing).
If you are trying to animate geometric deformations, 3D PDF won't really help you at all. You could capture the animation and encode it as flash video and embed in a PDF, but this a function of the multimedia tool in Acrobat Pro, i.e, is not specific to 3D.
I'm thinking of stitching images from 2 or more(currently maybe 3 or 4) cameras in real-time using OpenCV 2.3.1 on Visual Studio 2008.
However, I'm curious about how it is done.
Recently I've studied some techniques of feature-based image stitching method.
Most of them requires at least the following step:
1.Feature detection
2.Feature matching
3.Finding Homography
4.Transformation of target images to reference images
...etc
Now most of the techniques I've read only deal with images "ONCE", while I would like it to deal with a series of images captured from a few cameras and I want it to be "REAL-TIME".
So far it may still sound confusing. I'm describing the detail:
Put 3 cameras at different angles and positions, while each of them must have overlapping areas with its adjacent one so as to build a REAL-TIME video stitching.
What I would like to do is similiar to the content in the following link, where ASIFT is used.
http://www.youtube.com/watch?v=a5OK6bwke3I
I tried to consult the owner of that video but I got no reply from him:(.
Can I use image-stitching methods to deal with video stitching?
Video itself is composed of a series of images so I wonder if this is possible.
However, detecting feature points seems to be very time-consuming whatever feature detector(SURF, SIFT, ASIFT...etc) you use. This makes me doubt the possibility of doing Real-time video stitching.
I have worked on a real-time video stitching system and it is a difficult problem. I can't disclose the full solution we used due to an NDA, but I implemented something similar to the one described in this paper. The biggest problem is coping with objects at different depths (simple homographies are not sufficient); depth disparities must be determined and the video frames appropriately warped so that common features are aligned. This essentially is a stereo vision problem. The images must first be rectified so that common features appear on the same scan line.
You might also be interested in my project from a few years back. It's a program which lets you experiment with different stitching parameters and watch the results in real-time.
Project page - https://github.com/lukeyeager/StitcHD
Demo video - https://youtu.be/mMcrOpVx9aY?t=3m38s
I'm thinking to build a library to manipulate images(my own image type that I will develop), but first I need to understand the structure of a image
How it is mounted?
About the layer technology...
Where I can find some good resources to understand these things?
Thanks.
That all depends on the image format in question.
Most image formats, however, consist of the following:
A header that contains general file information (how long, what format, dimensions, color space, compression algorithm, etc.)
The pixel data (potentially compressed, in which case some other structure may apply)
Other metadata (EXIF, ...)
Many popular image formats such as JPEG or PNG have freely available specifications of the file format.
If you actually want to work with more complex images, containing layers and such (possibly Photoshop or similar) then things get more difficult. They additionally contain layers, so multiple chunks of pixel data, maybe metadata for the layers, in the case of Photoshop even vector data (for layer masks and other paths), etc.
What's more, most primary file formats used by major proprietary image editing software tend to be not fully specified, at least not publicly. There are resources out there but expect them to be incomplete at best.
Still, starting a project like this without much prior knowledge of image file formats in general might not be a feasible idea.
A good start to everyone that needs to know the basics about digital images is the chapter 2 of the classic book by Gonzalez and Woods, Digital Image Processing.
A short answer, roughly speaking: for manipulation in memory, images are 2D arrays. There are lots of variants, but the 2D array is the classic way.
For C, C++ and Python, take a look on OpenCV. For Python, see PIL. For Java, see JAI. Finally, to a overview about an "image structure", take a close look inside IplImage structure in OpenCV documentation.
Image file formats vary wildly. However, depending on which language/platform you're coding against, you may have generalized means of working with images and translating them into the format you chose. Each platform will have its own means of building and accessing images, however, so there's little I can tell you of substance without a declaration of your programming platform of choice.
Personally, I prefer C#/.NET. So here are some links on image manipulation in that platform:
http://www.aspfree.com/c/a/C-Sharp/Basic-Image-Manipulation-using-GDI-and-C/
http://www.aspfree.com/c/a/Code-Examples/Handling-Animation-and-Bitmaps-Using-GDI-for-Image-Manipulation/
Each image format, has a differant structure and comprresion.
Maybe you should explain in more detail your goals.
A quick amazon search yields a couple of books that could be very useful on the subject. Both are based around openGL, one of the most common graphic libraries. The first is a general introduction to computer graphics sort of text book and the second is a manual for openGL (commonly known as the red book).
Computer Graphics with OpenGL (3rd Edition)
OpenGL Programming Guide (The Red Book)
I can personally attest to the usefulness of both books.
If you're interested in the innards of various image file formats wotsit is a pretty good start. If you prefer hardcopy then go to the Encyclopedia of Graphics file formats. And if you want to look at sample sourcecode check out imagemagick. It can open-, convert to-, and save- most popular image file formats written in C++ with interfaces to most other languages.
Unless you're doing something very unique, I would encourage you to use an existing file format. Look at PNG or TIFF. They are incredibly flexible.
As a veteran in the field, I would say that the last thing the world needs is a new image file format. ;-)