Getting an intersection of two Redis sets in Laravel - laravel

I have a set looking like this (except that it has about 8000 items):
{"id":563244,"stock_id":693,"value":"9.17","span_minutes":1440,"symbol":"BCS"}
{"id":565527,"stock_id":10093,"value":"21.09","span_minutes":2880,"symbol":"GDXS"}
{"id":564311,"stock_id":4241,"value":"91.52","span_minutes":7200,"symbol":"NDAQ"}
{"id":565269,"stock_id":8883,"value":"127.60","span_minutes":5760,"symbol":"SAA"}
Now, what I want is to get entries with:
value between 20 and 100
span_minutes between 5000 and 8000
I can get these by first creating 2 new sorted sets and then using zrangebyscore to extract the entries I want:
ZADD values 9.17 '{"id":563244,"stock_id":693,"value":"9.17","symbol":"BCS"}'
ZADD values 21.09 {"id":565527,"stock_id":10093,"value":"21.09","span_minutes":2880,"symbol":"GDXS"}
ZADD values 91.52 {"id":564311,"stock_id":4241,"value":"91.52","span_minutes":7200,"symbol":"NDAQ"}
ZADD values 127.60 {"id":565269,"stock_id":8883,"value":"127.60","span_minutes":5760,"symbol":"SAA"}
ZADD spans 1440 '{"id":563244,"stock_id":693,"value":"9.17","symbol":"BCS"}'
ZADD spans 2880 {"id":565527,"stock_id":10093,"value":"21.09","span_minutes":2880,"symbol":"GDXS"}
ZADD spans 7200 {"id":564311,"stock_id":4241,"value":"91.52","span_minutes":7200,"symbol":"NDAQ"}
ZADD spans 5760 {"id":565269,"stock_id":8883,"value":"127.60","span_minutes":5760,"symbol":"SAA"}
Now, if I want to filter these sets to get the values I want, I can do this:
$value_range = Redis::zrangebyscore('values',20,100);
$span_range = Redis::zrangebyscore('spans',5000,8000);
These return, respectively:
1) {"id":565527,"stock_id":10093,"value":"21.09","span_minutes":2880,"symbol":"GDXS"}
2) {"id":564311,"stock_id":4241,"value":"91.52","span_minutes":7200,"symbol":"NDAQ"}
and
1) {"id":565269,"stock_id":8883,"value":"127.60","span_minutes":5760,"symbol":"SAA"}
2) {"id":564311,"stock_id":4241,"value":"91.52","span_minutes":7200,"symbol":"NDAQ"}
Now what I need is a way to combine these two sets. From what I can see in the Redis docs, I should be able to use zinterstore for this purpose, but I don't understand the syntax and whatever I try keeps either throwing an error or returning the integer 0. For example:
Redis::zinterstore('intersection', 2, $value_range, $span_range); returns an error.
$intereseciton = Redis::zinterstore(2, $value_range, $span_range); returns 0.
What I should be getting is:
1) {"id":564311,"stock_id":4241,"value":"91.52","span_minutes":7200,"symbol":"NDAQ"}
because it's the only element of the original set that matches both of my conditions.
Maybe I have the wrong syntax, or the wrong approach altogether. How do I effectively get an intersection of the two sorted sets?

As I mentioned in the comments, you need to do the intersection on the client side. Otherwise, you have to write a Lua script to do the job:
--inter.lua
local value_key = KEYS[1]
local span_key = KEYS[2]
local value_min = ARGV[1]
local value_max = ARGV[2]
local span_min = ARGV[3]
local span_max = ARGV[4]
local value_range = redis.call("zrangebyscore", value_key, value_min, value_max)
local span_range = redis.call("zrangebyscore", span_key, span_min, span_max)
-- do intersection
local value_set = {}
for _, item in ipairs(value_range) do
value_set[item] = true
end
local result = {}
for _, item in ipairs(span_range) do
if value_set[item] ~= nil then table.insert(result, item) end
end
return result
Run it like this: redis-cli --eval inter.lua values spans , 20 100 5000 8000

Related

How can I add minutes and seconds to a datetime in lua?

I have a lua function to attempt to convert the time duration of the currently playing song e.g. hh:mm:ss to seconds.
function toSeconds (inputstr)
local mytable = string.gmatch(inputstr, "([^"..":".."]+)");
local conversion = { 60, 60, 24}
local seconds = 0;
--iterate backwards
local count = 0;
for i=1, v in mytable do
count = i+1
end
for i=1, v in mytable do
mytable[count-i]
seconds = seconds + v*conversion[i]
end
return seconds
end
in order to add it to os.time to get the estimated end time of a song.
but the hours may be missing, or the minutes may be missing on a short track.
When running against https://www.lua.org/cgi-bin/demo All I get is input:10: 'do' expected near 'in'
for the test script
function toSeconds (inputstr)
local mytable = string.gmatch(inputstr, "([^"..":".."]+)");
local conversion = { 60, 60, 24}
local seconds = 0;
--iterate backwards
local count = 0;
for i=1, v in mytable do
count = i+1
end
for i=1, v in mytable do
mytable[count-i]
seconds = seconds + v*conversion[i]
end
return seconds
end
print(toSeconds("1:1:1")
You're mixing up the two possible ways of writing a for loop:
a)
for i=1,10 do
print(i, "This loop is for counting up (or down) a number")
end
b)
for key, value in ipairs({"hello", "world"}) do
print(key, value, "This loop is for using an iterator function")
end
The first one, as you can see, simply counts up a number, i in this case. The second one is very generic and can be used to iterate over almost anything (for example using io.lines), but is most often used with pairs and ipairs to iterate over tables.
You also don't write for ... in tab, where tab is a table; you have to use ipairs for that, which then returns an iterator for the table (which is a function)
You're also using string.gmatch incorrectly; it doesn't return a table, but an iterator function over the matches of the pattern in the string, so you can use it like this:
local matches = {}
for word in some_string:gmatch("[^ ]") do
table.insert(matches, word)
end
which gives you an actual table containing the matches, but if you're only going to iterate over that table, you might as well use the gmatch loop directly.
for i=1, v in mytable do
count = i+1
end
I think you're just trying to count the elements in the table here? You can easily get the length of a table with the # operator, so #mytable
If you have a string like hh:mm:ss, but the hours and the minutes can be missing, the easiest thing might be to just fill them with 0. A somewhat hacky but short way to achieve this is to just append "00:00:" to your string, and look for the last 3 numbers in it:
local hours, minutes, seconds = ("00:00:"..inputstr):match("(%d%d):(%d%d):(%d%d)$")
If nothing is missing, you'll end up with something like 00:00:hh:mm:ss, which you only take the last 3 values of to end up with the correct time.

How to compare alternating rows in CSV using RUBY

I have a data set that consists thousand of rows. I would like to count how many times an alarm toggle between ALARM_OPENED and ALARM_NORMALIZED
Here is a data sample. The Alarm toggle twice and hence ideally the count = 2
The issue now is I cannot figure how to
1) compare ALARM _OPENED and ALARM_NORMALIZED for the event type
2) To compare the difference in time between the change in event (the toggling should happen within a time frame of two seconds.)
count = 0
#loop this
if event_type[0] = 'ALARM_OPENED'
if event_type[1] = 'ALARM_NORMALIZED'
#time[0] - time[1] = 2 seconds
count = count + 1
end
end
p count
If you can assume that you always have a bunch of OPENED/NORMALIZED pairs, you can slice the array into pairs:
event_type.each_slice(2) do |opened, normalized|
break unless normalized # unpaired event at the end
# whatever you want to do with the two events here
end

Input to different attributes values from a random.sample list

so this is what I'm trying to do, and I'm not sure how cause I'm new to python. I've searched for a few options and I'm not sure why this doesn't work.
So I have 6 different nodes, in maya, called aiSwitch. I need to generate random different numbers from 0 to 6 and input that value in the aiSiwtch*.index.
In short the result should be
aiSwitch1.index = (random number from 0 to 5)
aiSwitch2.index = (another random number from 0 to 5 different than the one before)
And so on unil aiSwitch6.index
I tried the following:
import maya.cmds as mc
import random
allswtich = mc.ls('aiSwitch*')
for i in allswitch:
print i
S = range(0,6)
print S
shuffle = random.sample(S, len(S))
print shuffle
for w in shuffle:
print w
mc.setAttr(i + '.index', w)
This is the result I get from the prints:
aiSwitch1 <-- from print i
[0,1,2,3,4,5] <--- from print S
[2,3,5,4,0,1] <--- from print Shuffle (random.sample results)
2
3
5
4
0
1 <--- from print w, every separated item in the random.sample list.
Now, this happens for every aiSwitch, cause it's in a loop of course. And the random numbers are always a different list cause it happens every time the loop runs.
So where is the problem then?
aiSwitch1.index = 1
And all the other aiSwitch*.index always take only the last item in the list but the time I get to do the setAttr. It seems to be that w is retaining the last value of the for loop. I don't quite understand how to
Get a random value from 0 to 5
Input that value in aiSwitch1.index
Get another random value from 0 to 6 different to the one before
Input that value in aiSwitch2.index
Repeat until aiSwitch5.index.
I did get it to work with the following form:
allSwitch = mc.ls('aiSwitch')
for i in allSwitch:
mc.setAttr(i + '.index', random.uniform(0,5))
This gave a random number from 0 to 5 to all aiSwitch*.index, but some of them repeat. I think this works cause the value is being generated every time the loop runs, hence setting the attribute with a random number. But the numbers repeat and I was trying to avoid that. I also tried a shuffle but failed to get any values from it.
My main mistake seems to be that I'm generating a list and sampling it, but I'm failing to assign every different item from that list to different aiSwitch*.index nodes. And I'm running out of ideas for this.
Any clues would be greatly appreciated.
Thanks.
Jonathan.
Here is a somewhat Pythonic way: shuffle the list of indices, then iterate over it using zip (which is useful for iterating over structures in parallel, which is what you need to do here):
import random
index = list(range(6))
random.shuffle(index)
allSwitch = mc.ls('aiSwitch*')
for i,j in zip(allSwitch,index):
mc.setAttr(i + '.index', j)

How to get running maximum in Stata?

I would like to get the running maximum by writing Stata code.
I think I am quite close:
gen ctrhigh`iv' = max(ctr, L1.ctr, L2.ctr, L3.ctr, ..., L`iv'.ctr)
As you can see, my data are time series and `iv' represents the window (e.g. 5, 10 or 200 days)
The only problem is that you cannot pass a varlist or string containing numbers to max. E.g. the following is not possible:
local ivs 5 10 50 100 200
foreach iv in `ivs' {
local vals
local i = 1
while (`i' <= `iv') {
vals "`vals' `i'"
local ++i
}
gen ctrhigh`iv' = max(varlist vals) //not possible
}
How would I achieve this instead?
Example of quickly computing a running standard deviation
* standard deviation of ctr, see http://en.wikipedia.org/wiki/Standard_deviation#Rapid_calculation_methods *
gen ctr_sq = ctr^2
by tid: gen ctr_cum = sum(ctr) if !missing(ctr)
by tid: gen ctr_sq_cum = sum(ctr_sq) if !missing(ctr_sq)
foreach iv in $ivs {
if `iv' == 1 continue
by tid: gen ctr_sum = ctr_cum - L`iv'.ctr_cum if !missing(ctr_cum) & !missing(L`iv'.ctr_cum)
by tid: gen ctr_sq_sum = ctr_sq_cum - L`iv'.ctr_sq_cum if !missing(ctr_sq_cum) & !missing(L`iv'.ctr_sq_cum)
by tid: gen ctrsd`iv' = sqrt((`iv' * ctr_sq_sum - ctr_sum^2) / (`iv'*(`iv'-1))) if !missing(ctr_sq_sum) & !missing(ctr_sum)
label variable ctrsd`iv' "Rolling std dev of close ticker rank by `iv' days."
drop ctr_sum ctr_sq_sum
}
drop ctr_sq ctr_cum ctr_sq_cum
Note: this is not an exact sd, it's an approximation. I realize that this is very different from a maximum, but this may serve as an illustration on how to deal with large data computations.
Your example is time series data and implies that you have tsset the data. You don't say whether you also have panel or longitudinal structure. I will assume the worst and assume the latter as it doesn't make the code much worse. So, suppose tsset id date. In fact, that's irrelevant to the code here except to make explicit my assumption that id is an identifier and date a time variable.
An unattractive way to do this is to loop over observations. Suppose window is set to 42.
local window = 42
gen max = .
tsset id date
quietly forval i = 1/`=_N' {
su ctr if inrange(date, date[`i'] - `window', date[`i']) & id == id[`i'], meanonly
replace max = r(max) in `i'
}
So, in words as well: summarize values of ctr if date within window and it's in the same panel (same id), and put the maximum in the current observation.
The meanonly option is not well named. It calculates some other quantities besides the mean, and the maximum is one. But you do want the meanonly option to make summarize go as fast as possible.
See my 2007 paper on events in intervals, freely available at http://www.stata-journal.com/sjpdf.html?articlenum=pr0033
I say unattractive, but this approach does have the advantage that it is easy to work with once you understand it.
I am not setting up an expression with lots of arguments to max(). You said 200 as an example and nothing stated that you might not ask for more, so far as I can see there may be no upper limit on window length, but there will be a limit on how complicated that expression can be.
If I think of a better way to do it, I'll post it. Or someone else will....
It seems like I can pass a string of arguments to max, like so:
* OPTION 1: compute running max by days *
foreach iv in $ivs {
* does not make sense for less than two days *
if `iv' < 2 continue
di "computing running max for ctr interval `iv'"
* set high for this amount of days *
local vars "ctr"
forval i = 1 / `iv' {
local vars "`vars', L`i'.ctr"
}
by tid: gen ctrh`iv' = max(`vars')
}
* OPTION 2: compute running max by days, ensuring that entire range is nonmissing *
foreach iv in $ivs {
* does not make sense for less than two days *
if `iv' < 2 continue
di "computing running max for ctr interval `iv'"
* set high for this amount of days *
local vars "ctr"
local condition "!missing(ctr)"
forval i = 1 / `iv' {
local vars "`vars', L`i'.ctr"
local condition "`condition' & !missing(L`i'.ctr)"
}
by tid: gen ctrh`iv' = max(`vars') if `condition'
}
This computes very quickly and does exactly what I need.
However, if you need an arbitrarily large window I think you should resort to Nick's answer.

Formula for calculating Exotic wagers such as Trifecta and Superfecta

I am trying to create an application that will calculate the cost of exotic parimutuel wager costs. I have found several for certain types of bets but never one that solves all the scenarios for a single bet type. If I could find an algorithm that could calculate all the possible combinations I could use that formula to solve my other problems.
Additional information:
I need to calculate the permutations of groups of numbers. For instance;
Group 1 = 1,2,3
Group 2 = 2,3,4
Group 3 = 3,4,5
What are all the possible permutation for these 3 groups of numbers taking 1 number from each group per permutation. No repeats per permutation, meaning a number can not appear in more that 1 position. So 2,4,3 is valid but 2,4,4 is not valid.
Thanks for all the help.
Like most interesting problems, your question has several solutions. The algorithm that I wrote (below) is the simplest thing that came to mind.
I found it easiest to think of the problem like a tree-search: The first group, the root, has a child for each number it contains, where each child is the second group. The second group has a third-group child for each number it contains, the third group has a fourth-group child for each number it contains, etc. All you have to do is find all valid paths from the root to leaves.
However, for many groups with lots of numbers this approach will prove to be slow without any heuristics. One thing you could do is sort the list of groups by group-size, smallest group first. That would be a fail-fast approach that would, in general, discover that a permutation isn't valid sooner than later. Look-ahead, arc-consistency, and backtracking are other things you might want to think about. [Sorry, I can only include one link because it's my first post, but you can find these things on Wikipedia.]
## Algorithm written in Python ##
## CodePad.org has a Python interpreter
Group1 = [1,2,3] ## Within itself, each group must be composed of unique numbers
Group2 = [2,3,4]
Group3 = [3,4,5]
Groups = [Group1,Group2,Group3] ## Must contain at least one Group
Permutations = [] ## List of valid permutations
def getPermutations(group, permSoFar, nextGroupIndex):
for num in group:
nextPermSoFar = list(permSoFar) ## Make a copy of the permSoFar list
## Only proceed if num isn't a repeat in nextPermSoFar
if nextPermSoFar.count(num) == 0:
nextPermSoFar.append(num) ## Add num to this copy of nextPermSoFar
if nextGroupIndex != len(Groups): ## Call next group if there is one...
getPermutations(Groups[nextGroupIndex], nextPermSoFar, nextGroupIndex + 1)
else: ## ...or add the valid permutation to the list of permutations
Permutations.append(nextPermSoFar)
## Call getPermutations with:
## * the first group from the list of Groups
## * an empty list
## * the index of the second group
getPermutations(Groups[0], [], 1)
## print results of getPermutations
print 'There are', len(Permutations), 'valid permutations:'
print Permutations
This is the simplest general formula I know for trifectas.
A=the number of selections you have for first; B=number of selections for second; C=number of selections for third; AB=number of selections you have in both first and second; AC=no. for both first and third; BC=no. for both 2nd and 3rd; and ABC=the no. of selections for all of 1st,2nd, and third.
the formula is
(AxBxC)-(ABxC)-(ACxB)-(BCxA)+(2xABC)
So, for your example ::
Group 1 = 1,2,3
Group 2 = 2,3,4
Group 3 = 3,4,5
the solution is:: (3x3x3)-(2x3)-(1x3)-(2x3)+(2x1)=14. Hope that helps
There might be an easier method that I am not aware of. Now does anyone know a general formula for First4?
Revised after a few years:-
I re logged into my SE account after a while and noticed this question, and realised what I'd written didn't even answer you:-
Here is some python code
import itertools
def explode(value, unique):
legs = [ leg.split(',') for leg in value.split('/') ]
if unique:
return [ tuple(ea) for ea in itertools.product(*legs) if len(ea) == len(set(ea)) ]
else:
return [ tuple(ea) for ea in itertools.product(*legs) ]
calling explode works on the basis that each leg is separated by a /, and each position by a ,
for your trifecta calculation you can work it out by the following:-
result = explode('1,2,3/2,3,4/3,4,5', True)
stake = 2.0
cost = stake * len(result)
print cost
for a superfecta
result = explode('1,2,3/2,4,5/1,3,6,9/2,3,7,9', True)
stake = 2.0
cost = stake * len(result)
print cost
for a pick4 (Set Unique to False)
result = explode('1,2,3/2,4,5/3,9/2,3,4', False)
stake = 2.0
cost = stake * len(result)
print cost
Hope that helps
AS a punter I can tell you there is a much simpler way:
For a trifecta, you need 3 combinations. Say there are 8 runners, the total number of possible permutations is 8 (total runners)* 7 (remaining runners after the winner omitted)* 6 (remaining runners after the winner and 2nd omitted) = 336
For an exacta (with 8 runners) 8 * 7 = 56
Quinellas are an exception, as you only need to take each bet once as 1/2 pays as well as 2/1 so the answer is 8*7/2 = 28
Simple
The answer supplied by luskin is correct for trifectas. He posed another question I needed to solve regarding First4. I looked everywhere but could not find a formula. I did however find a simple way to determine the number of unique permutations, using nested loops to exclude repeated sequences.
Public Function fnFirst4PermCount(arFirst, arSecond, arThird, arFourth) As Integer
Dim intCountFirst As Integer
Dim intCountSecond As Integer
Dim intCountThird As Integer
Dim intCountFourth As Integer
Dim intBetCount As Integer
'Dim arFirst(3) As Integer
'Dim arSecond(3) As Integer
'Dim arThird(3) As Integer
'Dim arFourth(3) As Integer
'arFirst(0) = 1
'arFirst(1) = 2
'arFirst(2) = 3
'arFirst(3) = 4
'
'arSecond(0) = 1
'arSecond(1) = 2
'arSecond(2) = 3
'arSecond(3) = 4
'
'arThird(0) = 1
'arThird(1) = 2
'arThird(2) = 3
'arThird(3) = 4
'
'arFourth(0) = 1
'arFourth(1) = 2
'arFourth(2) = 3
'arFourth(3) = 4
intBetCount = 0
For intCountFirst = 0 To UBound(arFirst)
For intCountSecond = 0 To UBound(arSecond)
For intCountThird = 0 To UBound(arThird)
For intCountFourth = 0 To UBound(arFourth)
If (arFirst(intCountFirst) <> arSecond(intCountSecond)) And (arFirst(intCountFirst) <> arThird(intCountThird)) And (arFirst(intCountFirst) <> arFourth(intCountFourth)) Then
If (arSecond(intCountSecond) <> arThird(intCountThird)) And (arSecond(intCountSecond) <> arFourth(intCountFourth)) Then
If (arThird(intCountThird) <> arFourth(intCountFourth)) Then
' Debug.Print "First " & arFirst(intCountFirst), " Second " & arSecond(intCountSecond), "Third " & arThird(intCountThird), " Fourth " & arFourth(intCountFourth)
intBetCount = intBetCount + 1
End If
End If
End If
Next intCountFourth
Next intCountThird
Next intCountSecond
Next intCountFirst
fnFirst4PermCount = intBetCount
End Function
this function takes four string arrays for each position. I left in test code (commented out) so you can see how it works for 1/2/3/4 for each of the four positions

Resources