I am trying to search figure out how to search for a pattern within a range of timeframes. Obviously, it is likely that the pattern would occur several times based on the timeframes, that’s why I’m particularly interested in the largest number of times it repeats.
To explain what I’m trying to achieve further, say I am searching for a pattern from 2 hour to 15 minute chart and I find it on the 2 hour chart, then I drill into the next timeframe 1 hour, and I end up with two of the patterns on the 1 hour chart, I’ll continue to the 30 minute (in both 1 hour patterns) and to 15 minutes till I get the largest time it occurs.
I believe that a method that returns the next lower timeframe would be needed. I’ve been able to write that, see code below. I would really appreciate some help.
ENUM_TIMEFRAMES findLowerTimeframe(ENUM_TIMEFRAMES timePeriod)
{
int timeFrames[5] = {15, 20, 30, 60, 120};
int TFIndex=ArrayBsearch(timeFrames, (int)timePeriod);
return((ENUM_TIMEFRAMES) timeFrames[TFIndex - 1]);
}
EDIT
I didn't add the specific candlestick pattern because I believe it isn't the most important part of my problem. The crux of the question is how to search for a pattern on several consecutive timeframes to find the largest number of times it occurs within the range of times.
const ENUM_TIMEFRAMES DEFAULT_TIMEFRAMES[5] = {PERIOD_M15, PERIOD_M20, PERIOD_M30, PERIOD_H1, PERIOD_H2};
ENUM_TIMEFRAMES findLowerTimeframe(ENUM_TIMEFRAMES timePeriod)
{
int TFIndex=ArrayBsearch(DEFAULT_TIMEFRAMES,timePeriod);
return(TFIndex>0 ? timeFrames[TFIndex - 1] : PERIOD_CURRENT);
}
Related
I have a time tree which is granulated to 5 min intervals. I have no problem while trying to match the exact minute . But in few cases, i might be searching for a minute which is not exactly matching the node. For ex. if i want to match 10:05 i can. But if the input is 10.03 i get no result. I have epoch times added to the minute node. I wanted to return the closest minute node available for the given input (if its 10.03 then return 10.05). how do i achieve this?
MATCH (startMinute:Minute {epoch:apoc.date.parse('2018-04-12T16:33', 'ms',"yyyy-MM-dd'T'HH:mm")}) return startMinute
My Model is
here
If you want to always round up the time:
MATCH (startMinute:Minute {epoch:
(apoc.date.parse('2018-04-12T16:32', 'ms',"yyyy-MM-dd'T'HH:mm") + 29999)/300000*300000})
RETURN startMinute;
Or, if you want to round to nearest:
MATCH (startMinute:Minute {epoch:
(apoc.date.parse('2018-04-12T16:32', 'ms',"yyyy-MM-dd'T'HH:mm") + 150000)/300000*300000})
RETURN startMinute;
I am a beginner in prolog and was wondering if there was an easy way to convert numbers to time, for comparison.
For example:
The below two lists show bus name, capacity, time it arrives at city, time it departs city.
bus_info(bus1,150, 12:30, 14:30).
bus_info(bus2, 200, 16:00, 18:00).
passenger_info(mike, 21, 17:30). -shows name, age, and time available
I want to check which bus Mike can catch. The answer is bus 2, but how do I calculate this in prolog?
You're just comparing times for a given day so you don't need to convert the numbers to any kind of system time encoding. You only need, say "minutes past midnight" or something like that. For example, 12:30 would be (12*60)+30 minutes past midnight. And you can use that as your comparison units for a daily schedule.
To capture your hours and minutes to do this calculation, if you were to "ask" in Prolog:
bus_info(Bus, Num, StartHH:StartMM, EndHH:EndMM).
You would get two results:
Bus = bus1
Num = 150
StartHH = 12
StartMM = 30
EndHH = 14
EndHH = 30
And
Bus = bus2
Num = 200
StartHH = 16
StartMM = 0
EndHH = 18
EndMM = 0
To assign a numeric value of an expression in Prolog, you need the is predicate. For example:
StartTime is (StartHH * 60) + StartMM.
That basic information should get you started if you've learned how Prolog predicates basically work.
Update: I've provided a brief analysis of the three answers at the bottom of the question text and explained my choices.
My Question: What is the most efficient method of building a fixed interval dataset from a random interval dataset using stale data?
Some background: The above is a common problem in statistics. Frequently, one has a sequence of observations occurring at random times. Call it Input. But one wants a sequence of observations occurring say, every 5 minutes. Call it Output. One of the most common methods to build this dataset is using stale data, i.e. set each observation in Output equal to the most recently occurring observation in Input.
So, here is some code to build example datasets:
TInput = 100;
TOutput = 50;
InputTimeStamp = 730486 + cumsum(0.001 * rand(TInput, 1));
Input = [InputTimeStamp, randn(TInput, 1)];
OutputTimeStamp = 730486.002 + (0:0.001:TOutput * 0.001 - 0.001)';
Output = [OutputTimeStamp, NaN(TOutput, 1)];
Both datasets start at close to midnight at the turn of the millennium. However, the timestamps in Input occur at random intervals while the timestamps in Output occur at fixed intervals. For simplicity, I have ensured that the first observation in Input always occurs before the first observation in Output. Feel free to make this assumption in any answers.
Currently, I solve the problem like this:
sMax = size(Output, 1);
tMax = size(Input, 1);
s = 1;
t = 2;
%#Loop over input data
while t <= tMax
if Input(t, 1) > Output(s, 1)
%#If current obs in Input occurs after current obs in output then set current obs in output equal to previous obs in input
Output(s, 2:end) = Input(t-1, 2:end);
s = s + 1;
%#Check if we've filled out all observations in output
if s > sMax
break
end
%#This step is necessary in case we need to use the same input observation twice in a row
t = t - 1;
end
t = t + 1;
if t > tMax
%#If all remaining observations in output occur after last observation in input, then use last obs in input for all remaining obs in output
Output(s:end, 2:end) = Input(end, 2:end);
break
end
end
Surely there is a more efficient, or at least, more elegant way to solve this problem? As I mentioned, this is a common problem in statistics. Perhaps Matlab has some in-built function I'm not aware of? Any help would be much appreciated as I use this routine a LOT for some large datasets.
THE ANSWERS: Hi all, I've analyzed the three answers, and as they stand, Angainor's is the best.
ChthonicDaemon's answer, while clearly the easiest to implement, is really slow. This is true even when the conversion to a timeseries object is done outside of the speed test. I'm guessing the resample function has a lot of overhead at the moment. I am running 2011b, so it is possible Mathworks have improved it in the intervening time. Also, this method needs an additional line for the case where Output ends more than one observation after Input.
Rody's answer runs only slightly slower than Angainor's (unsurprising given they both employ the histc approach), however, it seems to have some problems. First, the method of assigning the last observation in Output is not robust to the last observation in Input occurring after the last observation in Output. This is an easy fix. But there is a second problem which I think stems from having InputTimeStamp as the first input to histc instead of the OutputTimeStamp adopted by Angainor. The problem emerges if you change OutputTimeStamp = 730486.002 + (0:0.001:TOutput * 0.001 - 0.001)'; to OutputTimeStamp = 730486.002 + (0:0.0001:TOutput * 0.0001 - 0.0001)'; when setting up the example inputs.
Angainor's appears robust to everything I threw at it, plus it was the fastest.
I did a lot of speed tests for different input specifications - the following numbers are fairly representative:
My naive loop: Elapsed time is 8.579535 seconds.
Angainor: Elapsed time is 0.661756 seconds.
Rody: Elapsed time is 0.913304 seconds.
ChthonicDaemon: Elapsed time is 22.916844 seconds.
I'm +1-ing Angainor's solution and marking the question solved.
This "stale data" approach is known as a zero order hold in signal and timeseries fields. Searching for this quickly brings up many solutions. If you have Matlab 2012b, this is all built in to the timeseries class by using the resample function, so you would simply do
TInput = 100;
TOutput = 50;
InputTimeStamp = 730486 + cumsum(0.001 * rand(TInput, 1));
InputData = randn(TInput, 1);
InputTimeSeries = timeseries(InputData, InputTimeStamp);
OutputTimeStamp = 730486.002 + (0:0.001:TOutput * 0.001 - 0.001);
OutputTimeSeries = resample(InputTimeSeries, OutputTimeStamp, 'zoh'); % zoh stands for zero order hold
Here is my take on the problem. histc is the way to go:
% find Output timestamps in Input bins
N = histc(Output(:,1), Input(:,1));
% find counts in the non-empty bins
counts = N(find(N));
% find Input signal value associated with every bin
val = Input(find(N),2);
% now, replicate every entry entry in val
% as many times as specified in counts
index = zeros(1,sum(counts));
index(cumsum([1 counts(1:end-1)'])) = 1;
index = cumsum(index);
val_rep = val(index)
% finish the signal with last entry from Input, as needed
val_rep(end+1:size(Output,1)) = Input(end,2);
% done
Output(:,2) = val_rep;
I checked against your procedure for a few different input models (I changed the number of Output timestamps) and the results are the same. However, I am still not sure I understood your problem, so if something is wrong here let me know.
I'm thinking about an algorithm that will create X most unique concatenations of Y parts, where each part can be one of several items. For example 3 parts:
part #1: 0,1,2
part #2: a,b,c
part #3: x,y,z
And the (random, one case of some possibilities) result of 5 concatenations:
0ax
1by
2cz
0bz (note that '0by' would be "less unique " than '0bz' because 'by' already was)
2ay (note that 'a' didn't after '2' jet, and 'y' didn't after 'a' jet)
Simple BAD results for next concatenation:
1cy ('c' wasn't after 1, 'y' wasn't after 'c', BUT '1'-'y' already was as first-last
Simple GOOD next result would be:
0cy ('c' wasn't after '0', 'y' wasn't after 'c', and '0'-'y' wasn't as first-last part)
1az
1cx
I know that this solution limit possible results, but when all full unique possibilities will gone, algorithm should continue and try to keep most avaible uniqueness (repeating as few as possible).
Consider real example:
Boy/Girl/Martin
bought/stole/get
bottle/milk/water
And I want results like:
Boy get milk
Martin stole bottle
Girl bought water
Boy bought bottle (not water, because of 'bought+water' and not milk, because of 'Boy+milk')
Maybe start with a tree of all combinations, but how to select most unique trees first?
Edit: According to this sample data, we can see, that creation of fully unique results for 4 words * 3 possibilities, provide us only 3 results:
Martin stole a bootle
Boy bought an milk
He get hard water
But, there can be more results requested. So, 4. result should be most-available-uniqueness like Martin bought hard milk, not Martin stole a water
Edit: Some start for a solution ?
Imagine each part as a barrel, wich can be rotated, and last item goes as first when rotates down, first goes as last when rotating up. Now, set barells like this:
Martin|stole |a |bootle
Boy |bought|an |milk
He |get |hard|water
Now, write sentences as We see, and rotate first barell UP once, second twice, third three and so on. We get sentences (note that third barell did one full rotation):
Boy |get |a |milk
He |stole |an |water
Martin|bought|hard|bootle
And we get next solutions. We can do process one more time to get more solutions:
He |bought|a |water
Martin|get |an |bootle
Boy |stole |hard|milk
The problem is that first barrel will be connected with last, because rotating parallel.
I'm wondering if that will be more uniqe if i rotate last barrel one more time in last solution (but the i provide other connections like an-water - but this will be repeated only 2 times, not 3 times like now). Don't know that "barrels" are good way ofthinking here.
I think that we should first found a definition for uniqueness
For example, what is changing uniqueness to drop ? If we use word that was already used ? Do repeating 2 words close to each other is less uniqe that repeating a word in some gap of other words ? So, this problem can be subjective.
But I think that in lot of sequences, each word should be used similar times (like selecting word randomly and removing from a set, and after getting all words refresh all options that they can be obtained next time) - this is easy to do.
But, even if we get each words similar number od times, we should do something to do-not-repeat-connections between words. I think, that more uniqe is repeating words far from each other, not next to each other.
Anytime you need a new concatenation, just generate a completely random one, calculate it's fitness, and then either accept that concatenation or reject it (probabilistically, that is).
const C = 1.0
function CreateGoodConcatenation()
{
for (rejectionCount = 0; ; rejectionCount++)
{
candidate = CreateRandomConcatination()
fitness = CalculateFitness(candidate) // returns 0 < fitness <= 1
r = GetRand(zero to one)
adjusted_r = Math.pow(r, C * rejectionCount + 1) // bias toward acceptability as rejectionCount increases
if (adjusted_r < fitness)
{
return candidate
}
}
}
CalculateFitness should never return zero. If it does, you might find yourself in an infinite loop.
As you increase C, less ideal concatenations are accepted more readily.
As you decrease C, you face increased iterations for each call to CreateGoodConcatenation (plus less entropy in the result)
I have 2 independent but contiguous date ranges. The first range is the start and end date for a project. Lets say start = 3/21/10 and end = 5/16/10. The second range is a month boundary (say 3/1/10 to 3/31/10, 4/1/10 to 4/30/10, etc.) I need to figure out how many days in each month fall into the first range.
The answer to my example above is March = 10, April = 30, May = 16.
I am trying to figure out an excel formula or VBA function that will give me this value.
Any thoughts on an algorithm for this? I feel it should be rather easy but I can't seem to figure it out.
I have a formula which will return TRUE/FALSE if ANY part of the month range is within the project start/end but not the number of days. That function is below.
return month_start <= project_end And month_end >= project_start
Think it figured it out.
=MAX( MIN(project_end, month_end) - MAX(project_start,month_start) + 1 , 0 )