I was wondering, what are the advantages and disadvantages of Kalman Filter and Exponential Filter? I have a multi-sensor fusion problem and I'm trying to decide which method to choose.
I think, Kalman filter is more computationally complicated but it has a more detailed model of the system so it is more accurate(?) in multi-sensor fusion.
Whereas the Exponential filter is a simple equation but it is limited by the choice of alpha (Higher alpha => less "memory" of the filter and thus lesser smoothing, but more weightage on measurements whereas lower alpha has higher degree of smoothing but sudden changes are not reflected properly.
The Exponential filter is more useful in noise cancellation, when there is jitter etc. whereas the Kalman filter is useful for the actual multi-sensor fusion. Is this correct?
Also, how useful is the Genetic Algorithm for sensor fusion? I am trying to combine a magnetic compass and gyroscope for estimating true orientation.
"what are the advantages and disadvantages of Kalman Filter and Exponential Filter?
I think, Kalman filter is more computationally complicated but it has a more detailed model of the system so it is more accurate(?) in multi-sensor fusion."
That's basically it, in general the better your model the system is, the better your filter will be, regardless of whether you're using a Kalman filter.
"The Exponential filter is more useful in noise cancellation, when there is jitter etc. whereas the Kalman filter is useful for the actual multi-sensor fusion. Is this correct?"
I would disagree with this statement. The Kalman filter is being smart about the noise cancellation. It's a lot smarter than a low pass filter can be because it takes full advantage of all the information stored in the covariance matrix. If the performance measure you're looking at is "How closely does the filtered value match the true value?" I think the best a simple low pass filter can hope to do is match it's performance, and that is only in the simplest case of a random walk . As soon as you have an interesting state-transition matrix I think the low pass filter has no chance, because it can't see how velocity uncertainty leaks into position uncertainty, for example.
"I am trying to combine a magnetic compass and gyroscope for estimating true orientation."
This is exactly the sort of thing a Kalman filter is designed for.
But if you're worried about the complexity of implementing a kalman filter, start by implementing the low pass filter version:
1) Start with a simple simulation
predictedAngle = oldAngle+rotationRate*dt
2) Update the simulation state based on your measurements
rotationRate = alpha1*rotationRate +(1-alpha1)*gyro.rotationRate
filteredAngle = alpha2*predictedAngle+(1-alpha2)*compass.angle
That is basically the framework for the kalman (simplest) filter for this system.
All that's missing is to:
Write everything in matrix format
Add "process noise" during the simulation step
Add a step to Calculate the "optimal kalman gains" instead of using fixed values for the alpha's
Add a step to update the filter's covariance.
"Also, how useful is the Genetic Algorithm for sensor fusion?"
I don't see where they would fit in. Can you elaborate?
The exponential filter is a special case of the Kalman filter that restricts consideration to
Systems with trivial (constant) dynamics, and
a fixed ratio of plant to (plant+measurement) noise (this is what determines the alpha parameter).
Thus, in cases where these assumptions apply, they are equivalent. In other cases, you can achieve better results using the Kalman filter (if you appropriately model the system).
The other main decision is whether you include the velocity in the state space; if you do, then the Kalman filter is the way to go.
Related
I have a stream of data measurements with an initial increasing phase that is followed by a plateau. The measurements are noisy without clear bound. I would like to stop ingesting the stream when the plateau is detected:
while (not_const)
{
add_measurement( stream.get() );
not_const = !is_const();
}
Is there a well-known algorithm for dealing with such problem? I know about Kalman-Filters, but not so sure if they are specifically made for this.
The Kalman filter will cover your noise, so long as the variance is calculable. Yes, it can help in this situation. Depending on your application, you may find that the first derivative of a moving average will do as well for you. Kalman merely optimizes some linear parameters to give a "best" prediction of actual (vs observed-through-noise) values.
You still need to handle your interpretation of that prediction series. You need to define what constitutes a "plateau". How close to 0 do you need the computable slope? Does that figure depend on the preceding input? How abrupt is the transition between the increase and the plateau? The latter considerations would suggest looking at the second derivative as well: a quick-change detector of some ilk.
I am having a problem statement which requires that if a particular error/event happens on
1-Jan-2017 and then on
22-Feb-2017,
3-April-2017,
9-July-2017
so i have to predict that when the next event is gonna occur , I am planning to try it with Kalman Filter theorm but it has very statistical terms and on the internet I didn't found any easy explanation or easy programming example for Kalman filter algorithm which estimates the next event dates . Can someone explain in simple terms or any parellel algorithm which can be used for the same purpose ?
Let Ei be the ith event, and let IETi = Ei+1 - Ei be the ith Inter-Event Time, i.e., the time between one event and the next. Then Ei+1 = Ei + IETi — the next event can be forecast from the most recent event based on IET.
Since the past is already determined the only thing random when you're projecting the next event is IET, so E[Ei+1] = Ei + E[IETi] (where E[] denotes a common notation for expected value). You don't need to know the distribution of IET to estimate its expected value, you only need to assume that the IETs are identically distributed. (They don't even need to be independent.) In other words, if IETs are identically distributed then the average of historical IETs is an unbiased estimator of their expected value.
There is a simple Kalman filter estimator to update estimates of an average as you obtain new data. See equations (2) & (3) from this post on math.stackexchange.
Note that this approach just gives a point predictor for the expected value. It won't allow you to make any probability statements about how likely it is the next event happens before or after some specified date. To do that you would need distributional information about the IETs.
Edit: Thanks to #pjs for his remarks. I will update my answer accordingly as soon as I can. However, many authors in the robotics/computer vision communities (e.g. Thrun et al) seem to directly define Kalman filters as Gaussian filters (and for those that are familiar with the computer vision/SLAM litterature, some computer vision works seem to discard standard EKF-based SLAM Since the Gaussian assumption doesn't hold for 3d points). In #pmj's answer, the Gaussian filter is actually nothing more than a running average and doesn't provide covariance (which can in some applications, be considered as the only justification for using a Kalman filter instead of non-linear minimizations on an equivalent cost-function) , so it seems pretty useless without an assumption on the distribution... So I wonder if this is what motivates said authors choices, or if it is just to simplify the discussion.
I think that Kalman filtering has very little to do with what you want to achieve. I will detail why after briefly explaining, in simple terms, what a Kalman filter does.
A Kalman filter estimates the current state x_t of a dynamic system based on all the previous observations, or in more mathematical terms, it models the probability distribution
p(x_t|z_1,...,z_t)
where the z_i are you observations (i.e measurements). Moreover, it is designed with a Gaussian assumption in mind. That is, it assumes that the distribution of your state/errors, including the one above, are Gaussian. Furthermore, it requires a model that links the measurements to the states, something like
z_t=f(x_t)+some_gaussian_noise
and you alos need a transition model, that links the previous state with the current one, e.g.
x_t=g(x_{t-1})+some_gaussian_noise
This comes with the assumption of having a "complete state": the knowledge of the current state is taken to be enough to predict the next one.
So, this is why I think it won't work with your model:
Given the information you've given, I see no sign that you can assume the distribution of the events is Gaussian. Its is probably not.
You don't have any transition equation, I don't even think it is possible to define one for your problem. Moreover, state completion doesn't hold.
Your state, as well as its observations, seem to be discrete, while Kalman filters are designed with continuous parameter spaces in mind.
Unfortunately, you haven't provided much information, so I can only suggest that you model your problem as a Markov chain, which I think you already had thought about.
Hope this helps a bit.
I'm new in DSP and I have a question in filtering a signal. As I have seen on the internet, IIR and FIR filters are commonly used for filtering a signal. In addition I have also seen another way to filter the signal namely: frequency domain filter (as in Aquilca C++) which apply a rectangular or brickwall window and perform a multiplication to the signal.
I know that we can perfrom the filtering on either time domain (using convolution) or frequency domain (using multiplication), however my question is:
Why can't we just simply use the frequency domain filter method instead of IIR or FIR ? because I think generate a window function is much simpler than generate an IIR filter.
Thanks
Applying a window by multiplication in the frequency domain results in circular convolution, which will contaminate the result. (e.g. the end of the convolution will wrap around to the beginning of your filtered result.)
But you can zero pad your signal by the length of the filter's impulse response, and use a longer FFT to get a linear filter convolution. But a rectangular brick wall has a very long (theoretically infinite) impulse response, so the amount of zero-padding required approaches infinite.
A brick wall filter also has a nasty frequency response (huge ripples) between FFT bins or frequency sample points. Not something to do if you want a near flat frequency response filter.
Developing a window with a flat enough frequency response (between bins or frequency points), and also a short enough impulse response is non trivial.
An IIR filter most often requires far less computation than a frequency domain filter. But overlap add or overlap save FFT fast convolution can be faster than a long FIR filter convolution.
But first, you need to generate a frequency domain filter that does not have an impulse response that is way too long for your FFT size (causing circular convolution wrap-around issues).
You can't really do filtering of real time signals in the frequency domain. (OK, that's not entirely correct, but it's correct enough -- see note!) What you can do is use the overlap-add method to implement a filter with the FFT.
See: https://en.wikipedia.org/wiki/Overlap%E2%80%93add_method
Make no mistake, though -- when you do this, you are implementing a FIR filter. The FFT is used to accelerate convolution of the signal with the filter impulse response, but it is still a convolution.
There are a two big reasons why this isn't done all the time:
It only provides significant performance advantages for very long impulse responses. For most FIR filters used in practice, the normal method is fine; also
FFT filtering introduces a significant delay, since you have to collect a whole block of samples (which has to be as long as the impulse response), and THEN do FFT convolution, before you can produce your first output. Since FFT convolution is only useful for long impulse responses, blocks are always big, so the delays are always significant.
It can also be tricky to implement.
There are some super-clever algorithms that can do FFT convolution without introducing delays. They use a normal FIR for the initial portion of the impulse response, and then increasing length FFT convolutions for the remainder of the impulse response. These are quite tricky to implement and I seem to recall that the technique is patented.
NOTES:
The reason why overlap-add convolution is not "filtering in the frequency domain", is because you still design the filter as a time-domain impulse response.
You can actually "filter in the frequency domain" using better (non-rectangular) overlapping windows, but then your filter isn't LTI, and it doesn't really provide any advantages over the overlap-add method.
The afford of the implementation and runtime overhead of FIR and IIR filter are minimal. It can be done in C in less than 20 lines.
If you want to do the same in the frequency domain, you can design the filter as you want. But you have to
Convert the signal into the frequency domain
Apply the filter
Convert it back into the time domain
Also, there are multiple design choices:
Which window size to use for conversion? (512, 1024,...)
FFT or one of the four DCTs?
windowing function to use?
...
I haven't implemented such a filter (yet). Maybe you will hit more problems on the way.
Baseline: filtering in the frequency domain is much more complicated but it still have its uses
I know that the Complementary Filter has the functions of both LPF and HPF. But I think my understanding on the principal behind it is still unclear.
I am quite new on digital signal processing, and maybe some very fundamental explanations will help a lot.
Say I have a Complementary Filter as follows:
y = a * y + (1 - a) * x
Then my parameter a may be calculated by
a = time_constant / (time_constant + sample_period),
where the sample_period is simply the reciprocal of the sampling_frequency.
The time_constant seems to be at my own choice.
My Questions:
What is the theory behind this calculation?
How do we choose the time_constant properly?
Note: I also posted this question on robotics, as the answers there are likely to be slightly different in emphasis.
What is the theory behind this calculation?
For a human readable introduction I would recommend:
The Balance Filter: A Simple Solution for Integrating Accelerometer and Gyroscope Measurements for a Balancing Platform.
How do we choose the time_constant properly?
Intuitively, the time_constant is the boundary between trusting the high-pass and the low-pass filter part. For shorter times than the time_constant you trust the high-pass filter part more, and for longer times you trust the low-pass part more.
Usually, you have some experience with the physical process you are dealing with and you can guess at least the order of magnitude of the time_constant. For example, if you are fusing accelerometer and gyro data (which I assume you do based on your other question), something between 0.5-1 second is a reasonable first guess. Then, you start tuning your filter by analyzing its performance on actual data and adjusting a accordingly.
I'm interested in image scaling algorithms and have implemented the bilinear and bicubic methods. However, I have heard of the Lanczos and other more sophisticated methods for even higher quality image scaling, and I am very curious how they work.
Could someone here explain the basic idea behind scaling an image using Lanczos (both upscaling and downscaling) and why it results in higher quality?
I do have a background in Fourier analysis and have done some signal processing stuff in the past, but not with relation to image processing, so don't be afraid to use terms like "frequency response" and such in your answer :)
EDIT: I guess what I really want to know is the concept and theory behind using a convolution filter for interpolation.
(Note: I have already read the Wikipedia article on Lanczos resampling but it didn't have nearly enough detail for me)
The selection of a particular filter for image processing is something of a black art, because the main criterion for judging the result is subjective: in computer graphics, the ultimate question is almost always: "does it look good?". There are a lot of good filters out there, and the choice between the best frequently comes down to a judgement call.
That said, I will go ahead with some theory...
Since you are familiar with Fourier analysis for signal processing, you don't really need to know much more to apply it to image processing -- all the filters of immediate interest are "separable", which basically means you can apply them independently in the x and y directions. This reduces the problem of resampling a (2-D) image to the problem of resampling a (1-D) signal. Instead of a function of time (t), your signal is a function of one of the coordinate axes (say, x); everything else is exactly the same.
Ultimately, the reason you need to use a filter at all is to avoid aliasing: if you are reducing the resolution, you need to filter out high-frequency original data that the new, lower resolution doesn't support, or it will be added to unrelated frequencies instead.
So. While you're filtering out unwanted frequencies from the original, you want to preserve as much of the original signal as you can. Also, you don't want to distort the signal you do preserve. Finally, you want to extinguish the unwanted frequencies as completely as possible. This means -- in theory -- that a good filter should be a "box" function in frequency space: with zero response for frequencies above the cutoff, unity response for frequencies below the cutoff, and a step function in between. And, in theory, this response is achievable: as you may know, a straight sinc filter will give you exactly that.
There are two problems with this. First, a straight sinc filter is unbounded, and doesn't drop off very fast; this means that doing a straightforward convolution will be very slow. Rather than direct convolution, it is faster to use an FFT and do the filtering in frequency space...
However, if you actually do use a straight sinc filter, the problem is that it doesn't actually look very good! As the related question says, perceptually there are ringing artifacts, and practically there is no completely satisfactory way to deal with the negative values that result from "undershoot".
Finally, then: one way to deal with the problem is to start out with a sinc filter (for its good theoretical properties), and tweak it until you have something that also solves your other problems. Specifically, this will get you something like the Lanczos filter:
Lanczos filter: L(x) = sinc(pi x) sinc(pi x/a) box(|x|<a)
frequency response: F[L(x)](f) = box(|f|<1/2) * box(|f|<1/2a) * sinc(2 pi f a)
[note that "*" here is convolution, not multiplication]
[also, I am ignoring normalization completely...]
the sinc(pi x) determines the overall shape of the frequency response (for larger a, the frequency response looks more and more like a box function)
the box(|x|<a) gives it finite support, so you can use direct convolution
the sinc(pi x/a) smooths out the edges of the box and (consequently? equivalently?) greatly improves the rejection of undesirable high frequencies
the last two factors ("the window") also tone down the ringing; they make a vast improvement in both the perceptual artifact and the practical incidence of "undershoot" -- though without completely eliminating them
Please note that there is no magic about any of this. There are a wide variety of windows available, which work just about as well. Also, for a=1 and 2, the frequency response does not look much like a step function. However, I hope this answers your question "why sinc", and gives you some idea about frequency responses and so forth.