I am part of an undergrad pilot study - sampling wetlands for carbon stock. Field samples have been collected. Now it is winter, the ground is frozen, and we realize we should have collected bulk density samples. Dried, ground, + sieved samples have been sent to our lab partners for carbon analysis - per the lab's instructions.
Can we still measure bulk density, from our samples?
We still have composite cores with marble-golf ball sized hardened chunks of clay-heavy soil. Also, we have we have remaining ground samples.
Could water displacement, with samples in a plastic bag, removing as much air as possible, work - to determine this soil's volume?
One prof suggested that measuring the chunks may work this way.
Or, chunks could be tied to a 'sinker', sans plastic bag - assuming such short-term water immersion would have minimal effect on clay-heavy soil volume.
??
Is there any way to determine bulk density from our samples?
please and thank you.
Related
in many forums and documents on the internet we hear about "short" and "long" learning and prediction computation time for machine learning algorithms. For example the Decision Tree Algorithm has a short computation time as compared to Neural Networks. But what it is never mentioned is what is "short" and what is "long".
Could you please clarify which unit you would use to measure computation time? Maybe "seconds per sample"? And what are reference values, so that I can predict if it takes 1h, 1day or 1Week?
Thanks
Kind regards
G
First of all, any benchmark figures depend critically on information you've neglected to provide: the domain you're in, the topology in use, input sizes and output complexity. The amount of training time depends on all of those factors, the framework for the implementation, and the platform you're on.
For instance, I can fully train
model class: 4-node linear neural network
model: AND gate
framework: none; Python / SciKit
platform: 32-node Intel Broadwell(tm) box
in three iterations, far less than a second.
However, given
model class: CNN
model: ResNet-50
framework: TensorFlow
platform: Single-card, 4-core Intel Haswell(tm) box
The training will take days.
The units in which we express the speed are those convenient to the audience for the domain. For processing still images, we usually specify the image size and report in images/sec (sometimes called "Hertz"). For audio input, it's usually an average clip length and clips/sec -- but if the processing is O(n) on the input time, it's often given as a scalar. Machine translation is expressed in words or sentences (of a typical length) per second.
For data center managers, the training is also given in electrical power: how many kilowatt-hours does it take to train a particular model?
You asked for reference values: again, read the posting guidelines. Reference values for standard (i.e. popular and stable) models are published on line; you should research these before you post, to help focus your question. They are useful once you have know your application well enough to adjust those figures (optimized for customers) to your own environment.
For instance, let's say you're trying to train ResNet-20 on the 100-class ImageNet data base. However, all you can find are figures for ResNet-50 on the hardware you have. Fortunately, ResNet is roughly linear in scaling by depth: you can take a comparable ResNet-50 training time and multiply by 20/50 to get an estimate for yours. My experience suggests that you should expect a training time 40-50% of the ResNet-50 time.
Similarly, if the figures you find are only for the 1000-class ImageNet, multiply by 100/1000 to get the training time to expect.
Overall, however, your general question has no real answer, other than to start doing your research, run your individual model, and learn how it operates, how it scales, how it responds to changes in hardware architecture, input size, model topology, focused training, etc.
Hello All with my first post here,
I work on tracking objects through images without prior training. I use two features, the color of the region (the ab channels of the Lab space) and the HOG. In my initial experiments, I found that using min. distance classifier with the HOG feature alone has the advantage of low false positives FP but with high FN. On the other hand, using the min. distance classifier with the color alone increases the TP and decreases the FN results but with the price of increasing FP.
My question, is how to combine the two classifiers? I like to know the standard algorithm to do that in an unsupervised way.
I tried to combine the two features into one feature (after normalization) but the HOG dominates the results. Even if I weighted the combined feature, results are worse than either of the two.
The good results I reach till now is to (cascade) the two classifiers, by running the color first to increase the possibilities then run the HOG (with a threshold a little bit higher than that used with HOG alone). I googled the topic but I don't have enough knowledge about classification to find the standard methods.
Thanks for help
I am thinking of trying to make program that will automatically generate drum tabs using an audio file containing only the drumming.
I have thought of using FFT to get an average spectrum peaks during a xxxx ms interval and then compare that to a table containing all the drum parts(snare, tombs, base drum and so no) of that specific drum kit and sound gear.
But i have a feeling that it won't be that easy. Have you guys any suggestions on which methods i could use to solve my issue?
// Eric
It isn't easy for anything except a trivial signal. Almost all western 'classical' and commercial music features coincident drum sounds.
1: Superposition: The original sources add together in a similar manner in the frequency domain as they do in the time domain. Each FFT bin contains contributions from all instruments currently being played (and those which are undamped and still decaying, or resonating sympathetically). Unpicking the various sources is hard - and certainly not a comparison with a library of spectra.
2: The FFT by its definition windows audio in the time domain and yields magnitude and phase of the basis function in each bin over that window period. The best you could say is that content appeared in the bin corresponding to a drum sound within the window period. If you were to compute a 1024 point FFT, the window duration would be 23ms at 44.1kHz. To put this into a musical perspective, 16th notes at 120bpm are 31.3ms apart. You might get away with smaller FFTs.
3: Percussion instrument signals tend to look a lot like noise - at least at the point where the instrument is hit. That is to say, there will be energy spread across the spectrum and no obviously dominant frequencies. After impact, tuned percussion starts to look more 'tonal'.
You probably need to look at a time-domain approach to accurately detect the onset point (onset detection). From there you could look at time or frequency domain characteristics of the signal to try and deduce the instrument in question. There's probably also a lot you could do with a priori knowledge of the genre of music being played, allowing you to predict the patterns that are likely to be present.
This is a particular case of the more generalised audio source separation problem. There's been lots of academic activity in this area, and consequently a lot of published papers describing approaches. Look for source separation, music information retrieval, audio feature detection
Given two byte arrays of data captured from a microphone, how can I determine which one has more spikes in noise? I would assume there is an algorithm I can apply to the data, but I have no idea where to start.
Getting down to it, I need to be able to determine when a baby is crying vs ambient noise in the room.
If it helps, I am using the Microsoft.Xna.Framework.Audio.Microphone class to capture the sound.
you can convert each sample (normalised to a range 1.0 to -1.0) into a decibel rating by applying the formula
dB = 20 * log-base-10 (sample-value)
To be honest, so long as you don't mind the occasional false positive, and your microphone is set up OK, you should have no problem telling the difference between a baby crying and ambient background noise, without going through the hassle of doing an FFT.
I'd recommend you having a look at the source code for a noise gate, which does pretty much what you are after, with configurable attack times & thresholds.
First use a Fast Fourier Transform to transform the signal into the frequency domain.
Then check if the signal in the typical "cry-frequencies" is significantly higher than the other amplitudes.
The preprocessor of the speex codec supports noise vs signal detection, but I don't know if you can get it to work with XNA.
Or if you really want some kind of loudness calculate the sum of squares of the amplitudes from the frequencies you're interested in (for example 50-20000Hz) and if the average of that over the last 30 seconds is significantly higher than the average over the last 10 minutes or exceeds a certain absolute threshold sound the alarm.
Louder at what point? The signal's average amplitude will tell you which one is louder on average, but that is kind of a dumb, brute force way to go about it. It may work for you in practice though.
Getting down to it, I need to be able to determine when a baby is crying vs ambient noise in the room.
Ok, so, I'm just throwing out ideas here; I am by no means an expert on audio processing.
If you know your input, i.e., a baby crying (relatively loud with a high pitch) versus ambient noise (relatively quiet), you should be able to analyze the signal in terms of pitch (frequency) and amplitude (loudness). Of course, if during he recording someone drops some pots and pans onto the kitchen floor, that will be tough to discern.
As a first pass I would simply traverse the signal, maintaining a standard deviation of pitch and amplitude throughout, and then set a flag when those deviations jump beyond some threshold that you will have to define. When they come back down you may be able to safely assume that you captured the baby's cry.
Again, just throwing you an idea here. You will have to see how it works in practice with actual data.
I agree with #Ed Swangren, it will take a lot of playing with samples of data for a lot of sources. To me, it sounds like the trick will be to limit or hopefully eliminate false positives. My experience with babies is they are much louder crying than the environment. so, keeping track of the average measurements (freq/amp/??) of the normal environment and then classifying how well the changes match the characteristics of a crying baby which changes from kid to kid, so you'll probably want a system that 'learns'. Best of luck.
update: you might find this library useful http://naudio.codeplex.com/
For example you measure the data coming from some device, it can be a mass of the object moving on the bridge. Because it is moving the mass will give data which will vibrate in some amplitude depending on the mass of the object. Bigger the mass - bigger the vibrations.
Are there any methods for filtering such kind of noise from that data?
May be using some formulas of vibrations? Have no idea what kind of formulas or algorithms (filters) can be used here. Please suggest anything.
EDIT 2:
Better picture, I just draw it for better understanding:
Not very good picture. From that graph you can see that the frequency is the same every
time, but the amplitude chanbges periodically. Something like that I have when there are no objects on the moving road. (conveyer belt). vibrating near zero value.
When the object moves, I there are the same waves with changing amplitude.
The graph can tell that there may be some force applying to the system and which produces forced occilations. So I am interested in removing such kind of noise. I do not know what force causes such occilations. Soon I hope I will get some data on the non moving road with and without object on it for comparison with moving road case.
What you have in your last plot is basically an amplitude modulated oscillation coming from a function like:
f[x] := 10 * (4 + Sin[x]) * Sin[80 * x]
The constants have been chosen to match your plot (using just a rule of thumb)
The Plot of this function is
That isn't "noise" (although may be some noise is there too), but can be filtered easily.
Let's see your data for the static and moving payloads ....
Edit
Based on your response to several comments, and based in my previous experience with weighting devices:
You are interfacing the physical world, not just getting input from a mouse and keyboard. It is very important for you understand the device, how it works and how it is designed.
You need a calibration procedure. You have to use several master weights to be sure that the device is working properly and linearly in the whole scale, and that the static case is measured much better than your dynamic needs.
You'll not be able to predict if you can measure with several loads in the conveyor until you do some experiments and look very carefully at the resulting plots
You need to be sure that a load placed anywhere in the conveyor shows the same reading. Or at least you should be able to correlate reading and position.
As I said before, you need a lot of info, and it seems that is not available. I always worked as a team with the engineers designing the device.
Don't hesitate to add more info ...
Have you tried filters with lowpass characteristics? There are different approaches for smoothing data (i.e. Savitzky-Golay, Gauss, moving average) but often, a simple N-point median filter is already sufficient.
It really depends on what you're after.
Take a look at this book:
The Scientist and Engineer's Guide to Digital Signal Processing
You can download it for free. In particular, check chapters 14 and 15.
If the frequency changes with mass and you're trying to measure mass, why not measure the frequency of the oscillations and use that as your primary measure?
Otherwise you need a notch filter which is tunable - figure out the frequency of the "noise" and tune the notch filter to that.
Another book to try is Lyons Understanding Digital Signal Processing
In order to smooth the signal, I'd average the previous 2 * n samples where n is the maximum expected wavelength of the vibrations.
This should cause most of the noise to be eliminated.
If you have some idea of the range of frequencies, you could do a simple average as long as the measurement period were sufficiently long to give you the level of accuracy you want to achieve. The more wavelengths worth of data you average against, the smaller the ratio of contributed error from a partial wavelength.
I'd suggest first simulating/modeling this in software like Matlab.
Data you'll need to consider:
The expected range of vibration frequencies
The measurement accuracy you want to achieve
The expected range of mass you'll want to measure
The function of mass to vibration amplitude
You should be able to apply the same principles as noise-cancelling microphones: put two sensors out, then subtract the secondary sensor's (farther away from the good signal source) signal from the primary sensor's (closer to the good signal source) signal.
Obviously, this works best if the "noise" will reach both sensors fairly equally while the "signal" reaches the primary sensor much more strongly.
For things like sound, this is pretty easy to do in the sensor itself, which makes your software a lot easier and more performant. Depending on what you're measuring, this might be easier to do with multiple sets of hardware and doing the cancellation in software.
If you can characterize the frequency spectra of the unwanted vibration noise, you might be able to synthesize a set of (near) minimum phase notch or band reject filter(s) to allow you to acquire your desired signal at your desired S/N ratio with minimized latency or data set size.
Filtering noisy digital signals is straight forward, as previous posters have noted. There are lots of references. You have not however stated what your objectives are clearly, so we cannot point you into a good direction. Are you looking for a single measurement of a single object on a bridge? [Then see other answers].
Are you monitoring traffic on this bridge and weighing each entity as it passes by? Then you need to determine when entities are on the sensor and when they are not. Typically, as long as the sensor's noise floor is significantly lower than the signal you're measuring this can be accomplished by simple thresholding.
Are you trying to measure the vibrations of the bridge caused by other vehicles? In which case you need either a more expensive sensor if you're having problems doing this, or a clearer measuring objective.