Clicking sound when doing FFT.forwardTransform() - tarsosdsp

I'm trying to visualize the FFT of a sound while listening to it using the TarsosDSP library. However, when listening to it, there is a loud clicking noise that's being played and I can't seem to understand where this is coming from. This is happening on a PC.
With all things staying the same in my code, I've narrowed it down to the FFT.forwardTransform() function.
Here's the essential wiring of my application:
Application.java:
InputStream audioStreasm = getClass().getResourceAsStream(selectedSound);
InputStream bufferedIn = new BufferedInputStream(audioStreasm);
AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(bufferedIn);
JVMAudioInputStream audioStream = new JVMAudioInputStream(audioInputStream);
// create a new dispatcher
dispatcher = new AudioDispatcher(audioStream, bufferSize, overlap);
dispatcher.addAudioProcessor(new FFTProcessor(fftSize, sampleRate));
dispatcher.addAudioProcessor(new AudioPlayer(audioStream.getFormat()));
// run the dispatcher (on a new thread).
new Thread(dispatcher, "Audio dispatching").start();
The file is a 1.85Mb wav file.
bufferSize is 1536
sampleRate is 44100
And FFTProcessor.java essentially looks like this:
public class FFTProcessor implements AudioProcessor {
private int sampleRate;
private FFT fft;
private float[] amplitudes;
public FFTProcessor(int fftSize, int sampleRate) {
this.sampleRate = sampleRate;
fft = new FFT(fftSize);
amplitudes = new float[fftSize];
}
#Override
public boolean process(AudioEvent audioEvent) {
float[] audioBuffer = audioEvent.getFloatBuffer();
fft.forwardTransform(audioBuffer);
fft.modulus(audioBuffer, amplitudes);
return true;
}
}
I've tried limiting executions by introducing a form of throttling, thinking it might help.
I also tried not using buffered input, in case there's a buffer underrun.
But the problem is still present and I'm starting to run out of ideas on what to try next.
Appreciate any pointers, thanks!

It appears to be a known issue, as per: https://github.com/JorenSix/TarsosDSP/issues/90
The following modification to the above code fixes the issue of the sound clicking.
dispatcher = new AudioDispatcher(audioStream, (int) audioStream.getFrameLength() - 2, overlap);
However now, the FFT processor seems to be having problems. I'll have to investigate further.
Cheers!

Related

JavaFX Export Scene as PDF/SVG

I have currently developed an ultimately open source application to analyse some data in a table view and visualise the resulting data in some additional plots. A problematic thing with this is, that the generated plots could potentially be useful for end users for e.g. a presentation, further downstream informative discussion and so on. This is why I started working on an export function using ImageWriter
//adding a context menu item to the chart
final MenuItem saveAsPng = new MenuItem("Save as png");
saveAsPng.setOnAction(new EventHandler<ActionEvent>() {
#Override public void handle(ActionEvent event) {
int scale = 6; //6x resolution should be enough, users should downscale if required
final Bounds bounds = bc.getLayoutBounds();
final SnapshotParameters spa = new SnapshotParameters();
spa.setTransform(javafx.scene.transform.Transform.scale(scale, scale));
ImageWriter imageWriter = new ImageWriter();
try {
imageWriter.saveImage(stage, bc.snapshot(spa, null));
} catch (ImageException e) {
e.printStackTrace();
}
}
});
This successfully creates a PNG file with sufficient size, but my ideal solution would be to export in vector-based format, e.g. PDF/SVG. For Swing applications, I knew how to achieve this, but for JFX I couldn't really find a proper solution for that matter. I already investigated several potential ideas, e.g. using a printer dialogue and then exporting as a PDF via virtual printer, but that does result in a bitmap inside the PDF, too.
Any ideas on this?

Custom HttpContent won't build - Could not AOT the assembly (mtouch)

In my efforts to create a progress indicator for uploading videos using HttpClient (SendAsync) in Xamarin Forms, I now have to ask for assistance.
The upload itself works fine, and all other API calls, but when I try to create a custom HttpContent to track the progress of the upload the project won't even build any more.
Error MT3001: Could not AOT the assembly
'[...].iOS/obj/iPhone/Debug/build-iphone7.2-10.1.1/mtouch-cache/Build/theproject.dll'
(MT3001) (theproject.iOS)
Using StreamContent or ByteArrayContent instead the project builds, but I can't get it working to track the progress.
A snippet of code (this is minimal example):
public class ProgressableContent : HttpContent
{
private const int defaultBufferSize = 4096;
private Stream content;
private int progress;
public ProgressableContent(Stream content)
{
this.content = content;
}
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
return Task.Run(async () =>
{
var buffer = new byte[defaultBufferSize];
var size = content.Length;
var uploaded = 0;
using (content) while (true)
{
var length = content.Read(buffer, 0, buffer.Length);
if (length <= 0) break;
uploaded += length;
progress = (int)((float)uploaded / size * 100);
await stream.WriteAsync(buffer, 0, length);
}
});
}
protected override bool TryComputeLength(out long length)
{
length = content.Length;
return true;
}
}
I use this by transforming my byte's to a stream, hopefully correctly:
//... building httpMessage.
httpMessage.Content = new ProgressableContent(await byteArrayContent.ReadAsStreamAsync());
//...
var response = await _httpClient.SendAsync(httpMessage, Cancellation.Token);
//...
The question(s):
Am I somehow causing the error? Is there a "better" way to do this?
Tagged this with Xamarin.iOS also since monotouch is complaining.
Double-click on the error from XS and it should bring you to a web page that provide more description about the issue. E.g.
MT3001 Could not AOT the assembly '*'
This generally indicates a bug in the AOT compiler. Please file a bug
http://bugzilla.xamarin.com with a project that can be used to
reproduce the error.
Sometimes it's possible to work around this by disabling incremental
builds in the project's iOS Build option (but it's still a bug, so
please report it anyways).
The main thing about 3001 is that the AOT compiler did not produce an output binary. There can be several reasons for this. Generally the process crashed and the build logs will give a bit more details why.
Even more important is to attach a self-contained test case to the bug report. Something else, beside the code you pasted, can be playing an important part that led to the crash (and it could be impossible to duplicate or guess what that piece could be). That also gives us a better chance to suggest a workaround to the issue.

Android Viewpager to load images from SD Card

Guys Im using the following custom code to load 20 images from resources and present in a viewpager
public class CustomPagerAdapter extends PagerAdapter {
int[] mResources = {
R.drawable.slide1,
R.drawable.slide2,
R.drawable.slide3,
R.drawable.slide4,
R.drawable.slide5,
R.drawable.slide6,
R.drawable.slide7,
R.drawable.slide8,
R.drawable.slide9,
R.drawable.slide10,
R.drawable.slide11,
R.drawable.slide12,
R.drawable.slide13,
R.drawable.slide14,
R.drawable.slide15,
R.drawable.slide16,
R.drawable.slide17,
R.drawable.slide18,
R.drawable.slide19,
R.drawable.slide20,
};
Context mContext;
LayoutInflater mLayoutInflater;
public CustomPagerAdapter(Context context) {
mContext = context;
mLayoutInflater = (LayoutInflater) mContext.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
}
#Override
public int getCount() {
return mResources.length;
}
#Override
public boolean isViewFromObject(View view, Object object) {
return view == ((LinearLayout) object);
}
#Override
public Object instantiateItem(ViewGroup container, int position) {
View itemView = mLayoutInflater.inflate(R.layout.pager_item, container, false);
ImageView imageView = (ImageView) itemView.findViewById(R.id.imageView);
imageView.setImageResource(mResources[position]);
container.addView(itemView);
return itemView;
}
#Override
public void destroyItem(ViewGroup container, int position, Object object) {
container.removeView((LinearLayout) object);
}
}
This works fine but I want to put the jpgs in a directory on the device so that they can be changed without recompiling the app
I think I need to get the images into the mResource array. I can get the path but not sure what format the code should be instead of using the draw-able lines
i have read articles on here but none make sense to me I am really new to this and the code looks nothing like the code I am using
can anyone point me in the right direction?
Any help is greatly appreciated
Mark
Yes, you can certainly do so. I will try to explain you the process step-by-step,
Step 1
Have a File object pointing to the path, like,
File directory = new File("path-to-directory");
Ensure that the path is to the directory with the images,
Step 2
List all the files inside the directory using listFiles() method, like
File[] allImages = directory.listFiles();
Now you have an array of all the files just like int[] mResources. The only difference being, now you have actual file references, while previously you had resource ids.
Step 3
You can just display the images in the ViewPager just like you did previously. But this is a bit tricky and can take you a considerable amount of time and code to get an image properly displayed from File.
You also need to take care of caching, so that when you load a previously loaded image again, it gets it from the cache.
To do all this, I recommend you to use this library (recommended by Google), Glide.
Setting an image is one line of code,
Glide.with(context).from(file).into(imageView);
That's it. Now you have your images displayed in a ViewPager from a directory in the device.

GWT: Cyclically loading an Image

I try to implement kind of a Videostream that relays on simple JPEG Files.
On my server, a JPEG is being created cyclically by an external Camera.
And I just want to include this Picture in my GWT Application.
My first idea to reload the Picture by a Timer was very simple but not so good: The client opens a connection for each reload-cycle, and the Picture flickers (at least in Firefox).
How could I solve these problems? I was thinking about something like "Web-Sockets", but I don't really know how to do.
I want to avoid a single connection for each reload. My idea was to have something like an open connection that just provides a new Picture as often as the Client asks for.
And how could I avoid the flickering when swapping the Picture?
Any ideas are welcome!
Regards, VanDahlen
A solution to avoid flickering is to have two images absolutely positioned in the same location. A timer would load one or other alternatively in each frame. Set a load handler to each image, so that it changes the z-index when the image is loaded and it restarts the timer.
Adding an extra parameter to the image url, makes the browser ask the server each time to bypass its cache.
If the time between frames is small, normally the browser will re-use the same connection if keep-alive is correctly configured in your server. It normally is enabled with a typical value of 5-15 seconds which you could increase, so if your .jpg images are updated with this periodicity, you don't have to worry and look for a better solution.
I propose a UI solution based on these ideas. But it will work as well if you use a websocket/comet mechanism giving you the last .jpg file in base64 format (just change the url by the value returned).
GWT code:
public void onModuleLoad() {
final Image i1 = new Image();
i1.setWidth("400px");
final Image i2 = new Image();
i2.setWidth("400px");
AbsolutePanel panel = new AbsolutePanel();
panel.add(i1, 0, 0);
panel.add(i2, 0, 0);
panel.setSize("600px", "400px");
RootPanel.get().add(panel);
// You could change this by base64 data if you use comet/websockets
String url = "my_image_url.jpg?";
final Timer loadNext = new Timer() {
boolean b;
int c;
public void run() {
// the counter parameter forces to load the next frame instead of using cache
if (b = !b) {
i1.setUrl(url + c++);
} else {
i2.setUrl(url + c++);
}
}
};
i1.addLoadHandler(new LoadHandler() {
public void onLoad(LoadEvent event) {
i1.getElement().getStyle().setZIndex(1);
i2.getElement().getStyle().setZIndex(0);
loadNext.schedule(1000);
}
});
i2.addLoadHandler(new LoadHandler() {
public void onLoad(LoadEvent event) {
i1.getElement().getStyle().setZIndex(0);
i2.getElement().getStyle().setZIndex(1);
loadNext.schedule(1000);
}
});
loadNext.schedule(1000);
}
If you want to use gwtquery, the code is obviously smaller:
// You could change this by base64 data if you use comet/websockets
final String url = "my_image_url.jpg?";
final GQuery images = $("<img/><img/>").appendTo(document);
images.css($$("position: fixed, top: 10px, left: 600px, width: 400px"));
final Timer timer = new Timer() {
int c;
public void run() {
images.eq(c%2).attr("src", url + c++);
}
};
images.bind("load", new Function(){
public void f() {
$(this).css($$("z-index: 1")).siblings("img").css($$("z-index: 0"));
timer.schedule(1000);
}
});
timer.schedule(1000);

Tools/Guide to Creating an Image Looper in GWT

I'm setting out to create a weather model display tool (web application), and from what I've seen, I'm really liking the idea of using Google Web Tools, and especially the SmartGWT toolkit. My one biggest sticking point at this point is finding some way to create a sort of image "looper" (displaying all images in a particular "set" one after another, not unlike a slide show). For reference, I need functionality (at least on a basic level) similar to this: http://rapidrefresh.noaa.gov/hrrrconus/jsloop.cgi?dsKeys=hrrr:&runTime=2012053007&plotName=cref_sfc&fcstInc=60&numFcsts=16&model=hrrr&ptitle=HRRR%20Model%20Fields%20-%20Experimental&maxFcstLen=15&fcstStrLen=-1&resizePlot=1&domain=full&wjet=1 (though it certainly need not be exactly like that).
Does anyone know of (ideally) some sort of GWT module that can do image looping? Or if not, does it sound like something an intermediate programmer could figure out without too much trouble (I'm willing to accept a challenge), even if I've never explicitly used GWT before? I'm sure I could whip together something that pulls each image in as it goes through a loop, but prefetching them would be even more ideal.
Please comment if you need clarification on anything!
As far as I'm aware there's not a pre-fab solution to do this, although maybe SmartGWT has something I don't know about. In any case, it won't be too hard to roll your own. Here's some code to get you started:
public class ImageLooper extends Composite {
// List of images that we will loop through
private final String[] imageUrls;
// Index of the image currently being displayed
private int currentImage = 0;
// The image element that will be displayed to the user
private final Image image = new Image();
// The Timer provides a means to execute arbitrary
// code after a delay or at regular intervals
private final Timer imageUpdateTimer = new Timer() {
public void run() {
currentImage = (currentImage + 1) % images.length;
image.setUrl(imageUrls[currentImage]);
}
}
// Constructor. I'll leave it to you how you're going to
// build your list of image urls.
public ImageLooper(String[] imageUrls) {
this.imageUrls = imageUrls;
// Prefetching the list of images.
for (String url : imageUrls)
Image.prefetch(url);
// Start by displaying the first image.
image.setUrl(imageUrls[0]);
// Initialize this Composite on the image. That means
// you can attach this ImageLooper to the page like you
// would any other Widget and it will show up as the
// image.
initWidget(image);
}
// Call this method to start the animation
public void playAnimation() {
// Update the image every two seconds
imageUpdateTimer.scheduleRepeating(2000);
}
// Call this method to stop the animation
public void stopAnimation() {
imageUpdateTimer.cancel();
}
}
One annoying thing with this implementation is that you have no way of knowing when your list of images has finished loading; Image.prefetch doesn't have a callback to help you here.

Resources