I am trying to create a simple GE Historian calculation that calculates the time between two values of a tag. For example, I have a temperature tag called "TAG1" that I want to calculate the time it took to go from 100 degrees to 200 degrees in seconds.
In the past I have calculated the time since the last recorded value using:
Result = DateDiff("s",PreviousGoodTime("TAG1","Now-Second"),CurrentTime())
I believe what is giving me a hard time is I do not know how I can return the timestamp of when a certain tag was a certain value last. Has anybody made a Calculation tag like this before?
Note: GE Historian calculations use VB Script, and PreviousGoodTime is a Built-In function that uses a tagname and a time to start calculation (in this case, "Now-Second" is one second prior to the trigger of the calc)
Update:
Accomplished the end result using two intermediary Boolean calculation tags. One that is True when the temperature is higher than 100 degrees (TAG2) and another that is True when higher than 200 degrees (TAG3). The final calculation tag (TAG1 from original post) is triggered when TAG3 steps from False to True and calculates the time since the last good value of TAG2.
Still curious if this can be accomplished in one single Historian Calculation tag.
Related
I am trying to segment a time series data into different zones.
In each time period, the pressure is running under an allowed max stress level (was not told before hand). Please see the pictures below.
edit each time period is more than a week.
How to detect the start / end of different time period? Would anyone point me some direction?
Once the time different time zones are divided, I guess I could average several max readings in each zone to have the max allowed stress.
I would take let's say enough values for 1h. Then you calculate the average value.
After that, you set the the average value in relation with the one before.
Some Pseudocode, to make it visual.
class Chunk:
private double[] values;//For one hour, for example.
double average();
enum Relation:
FALLING,RISING,EQUAL
func algorithm(Chunk[] chunks){
double averages=new double[chunks.length];
for(int i=0;i<chunks.length;i++)
averages[i]=chunks[i].average();
//Got averages, now make it rising or falling or stay same.
Relation[] relations=new Relation[chunks.length];
for(int i=1;i<chunks.length;i++){
double diff=averages[i]-averages[i-1];
if(diff==0) //TODO, a bit of difference is allowed (Like deviations of +-3)
relations[i]=EQUALS;
else
relations[i]=diff>0?RISING:FALLING;
}
// After that, you have to find sequences of many FALLING or RISING, followed by many EQUALS
}
To proceed with this array of Relations, you could divide it into smaller arrays, calculate the average (Like FALLING=0,RISING=1,EQUAL=2). After that you simply "merge" them like this:
F=FALLING
R=RISING
E=EQUALS
//Before merging
[RREEEEFFEEEEERRREEEE]
//After merging
[REFERE]
And there you can see the mountains and valleys.
Now, to get the exact values, when a mountain or valley starts, you have to extend Chunk a bit.
class Chunk:
//The value on x-Axis + the value of y-Axis
private Tuple<Time,Double>[] values;
//Tuple of Range, this chunk uses and the average value of this range
Tuple<Tuple<Time,Time>,double> average();
Furthermore, you can't use raw Relation anymore, you have to wrap it with the Range, from where it starts to the end.
In a Graphana dashboard with several datapoints, how can I get the difference between the last value and the previouse one for the same metric?
Perhaps the tricky part is that the tiem between 2 datapoins for the same metric is not know.
so the desired result is the <metric>.$current_value - <metric>.$previouse_value for each point in the metricstring.
Edit:
The metrics are stored in graphite/Carbon DB.
thanks
You need to use the derivative function
This is the opposite of the integral function. This is useful for taking a running total metric and calculating the delta between subsequent data points.
This function does not normalize for periods of time, as a true derivative would. Instead see the perSecond() function to calculate a rate of change over time.
Together with the keepLastValue
Takes one metric or a wildcard seriesList, and optionally a limit to the number of ‘None’ values to skip over.
Continues the line with the last received value when gaps (‘None’ values) appear in your data, rather than breaking your line.
Like this
derivative(keepLastValue(your_mteric))
A good example can be found here http://www.perehospital.cat/blog/graphite-getting-derivative-to-work-with-empty-data-points
I'm creating an app to monitor water quality. The temperature data is updated every 2 min to firebase real-time database. App has two requirements
1) It should alert the user when temperature exceed 33 degree or drop below 23 degree - This part is done
2) It should alert user when it has big temperature fluctuation after analysing data every 30min - This part i'm confused.
I don't know what algorithm to use to detect big temperature fluctuation over a period of time and alert the user. Can someone help me on this?
For a period of 30 minutes, your app would give you 15 values.
If you want to figure out a big change in this data, then there is one way to do so.
You can use implement the following method:
Calculate the mean and the standard deviation of the values.
Subtract the data you have from the mean and then take the absolute value of the result.
Compare if the absolute value is greater than one standard deviation, if it is greater then you have a big data.
See this example for better understanding:
Lets suppose you have these values for 10 minutes:
25,27,24,35,28
First Step:
Mean = 27 (apprx)
One standard deviation = 3.8
Second Step: Absolute(Data - Mean)
abs(25-27) = 2
abs(27-27) = 0
abs(24-27) = 3
abs(35-27) = 8
abs(28-27) = 1
Third Step
Check if any of the subtraction is greater than standard deviation
abs(35-27) gives 8 which is greater than 3.8
So, there is a big fluctuation. If all the subtracted results are less than standard deviation, then there is no fluctuation.
You can still improvise the result by selecting two or three standard deviation instead of one standard deviation.
Start by defining what you mean by fluctuation.
You don't say what temperature scale you're using. Fahrenheit, Celsius, Rankine, or Kelvin?
Your sampling rate is a new data value every two minutes. Do you define fluctuation as the absolute value of the difference between the last point and current value? That's defensible.
If the max allowable absolute value is some multiple of your 33-23 = 10 degrees you're in business.
A hardware sensor is sampled precisely (precise period of sampling) using a real-time unit. However, the time value is not sent to the database together with the sampled value. Instead, time of insertion of the record to the database is stored for the sample in the database. The DATETIME type is used, and the GETDATE() function is used to get current time (Microsoft SQL Server).
How can I reconstruct the precise sampling times?
As the sampling interval is (should be) 60 seconds exactly, there was no need earlier for more precise solution. (This is an old solution, third party, with a lot of historical samples. This way it is not possible to fix the design.)
For processing of the samples, I need to reconstruct the correct time instances for the samples. There is no problem with shifting the time of the whole sequence (that is, it does not matter whether the start time is rather off, not absolute). On the other hand, the sampling interval should be detected as precisely as possible. I also cannot be sure, that the sampling interval was exactly 60 seconds (as mentioned above). I also cannot be sure, that the sampling interval was really constant (say, slight differences based on temperature of the device).
When processing the samples, I want to get:
start time
the sampling interval
the sequence o the sample values
When reconstructing the samples, I need to convert it back to tuples:
time of the sample
value of the sample
Because of that, for the sequence with n samples, the time of the last sample should be equal to start_time + sampling_interval * (n - 1), and it should be reasonably close to the original end time stored in the database.
Think in terms of the stored sample times slightly oscillate with respect to the real sample-times (the constant delay between the sampling and the insertion into the database is not a problem here).
I was thinking about calculating the mean value and the corrected standard deviation for the interval calculated from the previous and current sample times.
Discontinuity detection: If the calculated interval is greater than 3 sigma off the mean value, I would consider it a discontinuity of the sampled curve (say, the machine is switched off, or any outer event lead to missing samples. In the case, I want to start with processing a new sequence. (The sampling frequency could also be changed.)
Is there any well known approach to the problem. If yes, can you point me to the article(s)? Or can you give me the name or acronym of the algorithm?
+1 to looking at the difference sequence. We can model the difference sequence as the sum of a low frequency truth (the true rate of the samples, slowly varying over time) and high frequency noise (the random delay to get the sample into the database). You want a low-pass filter to remove the latter.
I have a requirement that goes as follows (trust me, I'm way too old for homework grin)
I have a bunch of tasks that run with various frequencies. They also have a start "seed" date/time . The start seed is sometime in the past, could be one minute ago, could be 5 years ago.
I need to calculate the next run time for the task, using the start seed date/time and the frequency - it cannot simply be "now" + the task frequency (for those of you who have scheduled jobs on MS SQL Server this is a familiar concept)
Now the silly way to do it is to take the start seed and keep adding the frequency until it becomes greater than "now". That's hardly optimal. The naive way to do it would be to take the start seed date, change it to today's date and leave the time as is, then add the frequency until it's greater than now, but that assumes the frequency is a multiple of 24 hours.
So what's the best/quickest way to do this? Bonus points for a C# solution, but this is generic enough to make an interesting puzzle for any language :)
A better method would be to take the difference between the start timestamp and the current timestamp, divide that by the frequency, round the resulting multiplier up to the nearest integer, multiply by the frequency again, and add that to the start timestamp once more.
The act of rounding up will provide the proper offset.
Your answer would essentially be this:
next_time = ceiling((now - seed)/frequency) * frequency + seed
Using the ceiling function ensures that next_time will be >= now.
You would have to do the necessary conversions to be able to perform this arithmetic on the dates (e.g., translate to UNIX time, which is number of seconds since Jan 1, 1970.)
I am not familiar with C# so I can't offer the code, but I assume that C# has date/time utility classes for dealing with date/time arithmetic operations.
Interesting puzzle, thanks for the challenge :)
This should do it in c#. Could almost certainly be slimmed down, but its verbose enough to explain whats going on.
// Initialise with date the event started, and frequency
DateTime startDate = new DateTime(2009, 8,1,9,0,0);
TimeSpan frequency = new TimeSpan(0, 15, 0);
// Store datetime now (so that it doesnt alter during following calculations)
DateTime now = DateTime.Now;
// Calculate the number of ticks that have occured since the event started
TimeSpan pastTimeSpan = now.Subtract(startDate);
// Divide the period that the event has been running by the frequency
// Take the remaining time span
TimeSpan remainingTimeSpan = new TimeSpan(pastTimeSpan.Ticks % frequency.Ticks);
// Calculate last occurence the event ran
DateTime lastOccurence = now.Subtract(remainingTimeSpan);
// Calculate next occurence the event will run
DateTime nextOccurence = lastOccurence.Add(frequency);