parallel iteration in lua - for-loop

I would like to for-loop through multiple tables in parallel in Lua. I could just do:
for i in range(#table1)
pprint(table1[i])
pprint(table2[i])
end
But I'd rather something like python's zip:
for elem1, elem2 in zip(table1, table2):
pprint(elem1)
pprint(elem2)
end
Is there such a thing in standard Lua (or at least in whatever comes packaged with torch?).

If you want something in Lua that's similar to some Python function, you should look at Penlight first. For this specific case there is the seq.zip function. It seems that Penlight is installed together with Torch, but you can also get it via LuaRocks (which again is bundled with at least one Torch distribution).
Anyway, the seq.zip function in Penlight only supports zipping two sequences. Here is a version that should behave more like Python's zip, i.e. allowing more (or less) than two sequences:
local zip
do
local unpack = table.unpack or unpack
local function zip_select( i, var1, ... )
if var1 then
return var1, select( i, var1, ... )
end
end
function zip( ... )
local iterators = { n=select( '#', ... ), ... }
for i = 1, iterators.n do
assert( type( iterators[i] ) == "table",
"you have to wrap the iterators in a table" )
if type( iterators[i][1] ) ~= "number" then
table.insert( iterators[i], 1, -1 )
end
end
return function()
local results = {}
for i = 1, iterators.n do
local it = iterators[i]
it[4], results[i] = zip_select( it[1], it[2]( it[3], it[4] ) )
if it[4] == nil then return nil end
end
return unpack( results, 1, iterators.n )
end
end
end
-- example code (assumes that this file is called "zip.lua"):
local t1 = { 2, 4, 6, 8, 10, 12, 14 }
local t2 = { "a", "b", "c", "d", "e", "f" }
for a, b, c in zip( {ipairs( t1 )}, {ipairs( t2 )}, {io.lines"zip.lua"} ) do
print( a, b, c )
end

--------------------------------------------------------------------------------
-- Python-like zip() iterator
--------------------------------------------------------------------------------
function zip(...)
local arrays, ans = {...}, {}
local index = 0
return
function()
index = index + 1
for i,t in ipairs(arrays) do
if type(t) == 'function' then ans[i] = t() else ans[i] = t[index] end
if ans[i] == nil then return end
end
return unpack(ans)
end
end
--------------------------------------------------------------------------------
-- Example use:
--------------------------------------------------------------------------------
a = {'a','b','c','d'}
b = {3,2,1}
c = {7,8,9,10,11}
for a,b,c,line in zip(a,b,c,io.lines(arg[0])) do
print(a,b,c,line)
end
print '\n--- Done! ---'

Made a version with variable number of lists based on the other responses and added coroutines, it's not the fastest but I think it's very readable. I'm gonna use it but maybe someone else also finds it usefull.
Also this was made for use with neovim's LuaJit.
local zipgen = function(args)
-- find smallest iterator's size
local min = #args[1]
for i = 2, #args, 1 do
min = #args[i] < min and #args[i] or min
end
-- create list with 'i'th element from all iterators
for i=1, min do
local ans = {}
for j=1, #args do
-- get 'j'th iterator's 'i'th element
ans[j] = args[j][i]
end
-- return list of 'i'th elements
coroutine.yield(unpack(ans))
end
end
-- python like zip iterator
zip = function(...)
local args = {...}
-- return a function that resumes the coroutine when called
return coroutine.wrap(function() zipgen(args) end)
end

Related

Efficiently storing and iterating over Lua multi-level queue

This is the add format:
AddToQueue( String identifier, function callfunc, int priority )
priority is guaranteed to be 0 to 4, with 0 being the highest priority. My current setup is this:
local tQueue = {
[0] = {},
[1] = {},
[2] = {},
[3] = {},
[4] = {}
}
function AddToQueue( sName, funcCallback, iPriority )
queue[iPriority or 0][sName] = funcCallback
end
function CallQueue()
for i = 0, 4 do
for _, func in pairs( queue[i] ) do
local retval = func()
if ( retval ~= nil ) then
return retval
end
end
end
end
This works, but I want to know if there's a better way to store and iterate the functions to prevent doing 5 pair loops every call. Thanks!
If you iterate your queue frequently, and new callbacks additions is infrequent, then you can just store everything in single table, sorting it each time you've added new callback.

Redis interval query

I have data in the form of a tuple (S, T), where S is string and T is integer. Neither S nor T is unique, while their combination is unique. I need to get all tuples where S1 == S2 and |T1 - T2| <= C. Is is possible to do efficiently with Redis?
One way would be to store the data in a list and do the retrieval with a Lua script. First, for tuples of the form (Sn,Tn), insert it like this:
LPUSH myKey S1:T1
LPUSH myKey S2:T2
... and so on
Then, use the Lua script below:
local function split(div,str)
if (div=='') then return false end
local pos,arr = 0,{}
for st,sp in function() return string.find(str,div,pos,true) end do
table.insert(arr,string.sub(str,pos,st-1))
pos = sp + 1
end
table.insert(arr,string.sub(str,pos))
return arr
end
local key = KEYS[1]
local sVal = ARGV[1]
local tVal = tonumber(ARGV[2])
local cVal = tonumber(ARGV[3])
local length = redis.call("LLEN", key)
if (tonumber(length) == 0) then
return nil
end
local data = redis.call("LRANGE", key, 0, tonumber(length))
local retval = {}
for index,val in pairs(data) do
local parts = split(":", val)
if (parts[1] == sVal and math.abs(tonumber(parts[2]) - tVal) <= cVal) then
table.insert(retval, val)
end
end
return retval
Save it as script.lua and execute it with:
$ redis-cli "$(cat script.lua)" 1 myKey sValue tValue cValue
This will return all tuples (in Sn:Tn form) that matches S1 == S2 and |T1 - T2| <= C.

How can I deep-compare 2 Lua tables, which may or may not have tables as keys?

(Also posted on the Lua mailing list)
So I've been writing deep-copy algorithms, and I wanna test them to see if they work the way I want them to. While I do have access to the original->copy map, I want a general-purpose deep-compare algorithm that must be able to compare table keys (tables as keys?).
My deep-copy algorithm(s) are avaliable here: https://gist.github.com/SoniEx2/fc5d3614614e4e3fe131 (it's not very organized, but there are 3 of them, one uses recursive calls, the other uses a todo table, and the other simulates a call stack (in a very ugly but 5.1-compatible way))
Recursive version:
local function deep(inp,copies)
if type(inp) ~= "table" then
return inp
end
local out = {}
copies = (type(copies) == "table") and copies or {}
copies[inp] = out -- use normal assignment so we use copies' metatable (if any)
for key,value in next,inp do -- skip metatables by using next directly
-- we want a copy of the key and the value
-- if one is not available on the copies table, we have to make one
-- we can't do normal assignment here because metatabled copies tables might set metatables
-- out[copies[key] or deep(key,copies)]=copies[value] or deep(value,copies)
rawset(out,copies[key] or deep(key,copies),copies[value] or deep(value,copies))
end
return out
end
Edit: I found things like this which don't really handle tables as keys: http://snippets.luacode.org/snippets/Deep_Comparison_of_Two_Values_3 (Copy of snippet below)
function deepcompare(t1,t2,ignore_mt)
local ty1 = type(t1)
local ty2 = type(t2)
if ty1 ~= ty2 then return false end
-- non-table types can be directly compared
if ty1 ~= 'table' and ty2 ~= 'table' then return t1 == t2 end
-- as well as tables which have the metamethod __eq
local mt = getmetatable(t1)
if not ignore_mt and mt and mt.__eq then return t1 == t2 end
for k1,v1 in pairs(t1) do
local v2 = t2[k1]
if v2 == nil or not deepcompare(v1,v2) then return false end
end
for k2,v2 in pairs(t2) do
local v1 = t1[k2]
if v1 == nil or not deepcompare(v1,v2) then return false end
end
return true
end
Serializing is also not an option, as order of serialization is "random".
As others said, that depends a lot on your definition of equivalence.
If you want this to be true:
local t1 = {[{}] = {1}, [{}] = {2}}
local t2 = {[{}] = {1}, [{}] = {2}}
assert( table_eq(t1, t2) )
If you do, then each time the key in t1 is a table, you'll have to
check its equivalence with every table key in t2 and try them one by
one. This is a way to do it (metatable stuff left out for readability).
function table_eq(table1, table2)
local avoid_loops = {}
local function recurse(t1, t2)
-- compare value types
if type(t1) ~= type(t2) then return false end
-- Base case: compare simple values
if type(t1) ~= "table" then return t1 == t2 end
-- Now, on to tables.
-- First, let's avoid looping forever.
if avoid_loops[t1] then return avoid_loops[t1] == t2 end
avoid_loops[t1] = t2
-- Copy keys from t2
local t2keys = {}
local t2tablekeys = {}
for k, _ in pairs(t2) do
if type(k) == "table" then table.insert(t2tablekeys, k) end
t2keys[k] = true
end
-- Let's iterate keys from t1
for k1, v1 in pairs(t1) do
local v2 = t2[k1]
if type(k1) == "table" then
-- if key is a table, we need to find an equivalent one.
local ok = false
for i, tk in ipairs(t2tablekeys) do
if table_eq(k1, tk) and recurse(v1, t2[tk]) then
table.remove(t2tablekeys, i)
t2keys[tk] = nil
ok = true
break
end
end
if not ok then return false end
else
-- t1 has a key which t2 doesn't have, fail.
if v2 == nil then return false end
t2keys[k1] = nil
if not recurse(v1, v2) then return false end
end
end
-- if t2 has a key which t1 doesn't have, fail.
if next(t2keys) then return false end
return true
end
return recurse(table1, table2)
end
assert( table_eq({}, {}) )
assert( table_eq({1,2,3}, {1,2,3}) )
assert( table_eq({1,2,3, foo = "fighters"}, {["foo"] = "fighters", 1,2,3}) )
assert( table_eq({{{}}}, {{{}}}) )
assert( table_eq({[{}] = {1}, [{}] = {2}}, {[{}] = {1}, [{}] = {2}}) )
assert( table_eq({a = 1, [{}] = {}}, {[{}] = {}, a = 1}) )
assert( table_eq({a = 1, [{}] = {1}, [{}] = {2}}, {[{}] = {2}, a = 1, [{}] = {1}}) )
assert( not table_eq({1,2,3,4}, {1,2,3}) )
assert( not table_eq({1,2,3, foo = "fighters"}, {["foo"] = "bar", 1,2,3}) )
assert( not table_eq({{{}}}, {{{{}}}}) )
assert( not table_eq({[{}] = {1}, [{}] = {2}}, {[{}] = {1}, [{}] = {2}, [{}] = {3}}) )
assert( not table_eq({[{}] = {1}, [{}] = {2}}, {[{}] = {1}, [{}] = {3}}) )
Instead of direct comparison you may try to serialize each of the tables and compare the serialized results. There are several serializers that handle table as keys and can serialize deep and recursive structures. For example, you may try Serpent (available as a standalone module and also included in Mobdebug):
local dump = pcall(require, 'serpent') and require('serpent').dump
or pcall(require, 'mobdebug') and require('mobdebug').dump
or error("can't find serpent or mobdebug")
local a = dump({a = 1, [{}] = {}})
local b = dump({[{}] = {}, a = 1})
print(a==b)
This returns true for me (both Lua 5.1 and Lua 5.2). One of the caveats is that the serialization result depends on the order in which keys are serialized. The tables as key by default are sorted based on their tostring value, which means that the order depends on their allocation address and it's not difficult to come up with an example that fails under Lua 5.2:
local dump = pcall(require, 'serpent') and require('serpent').dump
or pcall(require, 'mobdebug') and require('mobdebug').dump
or error("can't find serpent or mobdebug")
local a = dump({a = 1, [{}] = {1}, [{}] = {2}})
local b = dump({[{}] = {2}, a = 1, [{}] = {1}})
print(a==b) --<-- `false` under Lua 5.2
One way to protect against this is to consistently represent tables in keys comparison; for example, instead of default tostring, you may want to serialize tables and their values and sort the keys based on that (serpent allows a custom handler for sortkeys), which would make the sorting more robust and the serialized results more stable.

generating TTT game tree in lua

I am attempting to write a tic-tac-toe game in lua, and plan on using the minimax algorithm to decide non-human moves. The first step in this involves generating a tree of all possible board states from a single input state. I am trying to recursively do this, but cannot seem to figure out how. (I think) I understand conceptually how this should be done, but am having trouble implementing it in lua.
I am trying to structure my tree in the following manner. Each node is a list with two fields.
{ config = {}, children = {} }
Config is a list of integers (0,1,2) that represent empty, X, and O and defines a TTT board state. Children is a list nodes which are all possible board states one move away from the current node.
Here is my function that I currently have to build the game tree
function tree_builder(board, player)
supertemp = {}
for i in ipairs(board.config) do
--iterate through the current board state.
--for each empty location create a new node
--representing a possible board state
if board.config[i] == 0 then
temp = {config = {}, children = {}}
for j in ipairs(board.config) do
temp.config[j] = board.config[j]
end
temp.config[i] = player
temp.children = tree_builder(temp, opposite(player))
supertemp[i] = temp
end
end
return supertemp
end
The function is called in the following manner:
start_board = {config = {1,0,0,0}, children = {} }
start_board.children = tree_builder(start_board, 1)
When I comment out the recursive element of the function (the line "temp.children = builder(temp, opposite(player))") and only generate the first level of children. the output is correct. When called via code that is conceptually identical to (I am using love2D so formatting is different):
for i in pairs(start_board.children) do
print (start_board.children[i].config)
The three children are:
1,1,0,0
1,0,1,0
1,0,0,1
However, once I add the recursive element, the following is output for the same three children
1,1,2,1
1,1,2,1
1,1,2,1
I have been searching online for help and most of what I have found is conceptual in nature or involves implementation in different languages. I believe I have implemented the recursive element wrongly, but cannot wrap my head around the reasons why.
Don't understand what opposite(player) means in temp.children = tree_builder(temp, opposite(player)).
Notice that a recursion need an end condition.
This is my solution under your structure:
local COL = 3
local ROW = 3
local function printBoard( b )
local output = ""
local i = 1
for _,v in ipairs(b.config) do
output = output .. v .. ( (i % COL == 0) and '\n' or ',' )
i = i + 1
end
print( output )
end
local function shallowCopy( t )
local t2 = {}
for k,v in pairs(t) do
t2[k] = v
end
return t2
end
local MAX_STEP = COL * ROW
local ING = 0
local P1 = 1
local P2 = 2
local TIE = 3
local STATUS = { [P1] = "P1 Win", [P2] = "P2 Win", [TIE] = "Tied" }
local start_board = { config = {P1,0,0,0,0,0,0,0,0}, children = {} }
local function checkIfOver( board, step )
local config = board.config
local over = false
--check rows
for i=0,ROW-1 do
over = true
for j=1,COL do
if 0 == config[i*COL+1] or config[i*COL+j] ~= config[i*COL+1] then
over = false
end
end
if over then
return config[i*COL+1]
end
end
--check cols
for i=1,COL do
over = true
for j=0,ROW-1 do
if 0 == config[i] or config[i] ~= config[i+COL*j] then
over = false
end
end
if over then
return config[i]
end
end
--check diagonals
if config[1] ~= 0 and config[1] == config[5] and config[5] == config[9] then
return config[1]
end
if config[3] ~=0 and config[3] == config[5] and config[5] == config[7] then
return config[3]
end
if step >= MAX_STEP then
return TIE
else
return ING
end
end
local function treeBuilder( board, step )
--check the game is over
local over = checkIfOver( board, step )
if over ~= ING then
printBoard( board )
print( STATUS[over], '\n---------\n' )
return
end
local child
local childCfg
for i,v in ipairs(board.config) do
if 0 == v then
child = { config = {}, children = {} }
childCfg = shallowCopy( board.config )
childCfg[i] = (step % 2 == 0) and P1 or P2
child.config = childCfg
table.insert( board.children, child )
treeBuilder( child, step + 1 )
end
end
end
treeBuilder( start_board, 1 )

How to sort this lua table?

I have next structure
self.modules = {
["Announcements"] = {
priority = 0,
-- Tons of other attributes
},
["Healthbar"] = {
priority = 40,
-- Tons of other attributes
},
["Powerbar"] = {
priority = 35,
-- Tons of other attributes
},
}
I need to sort this table by priorty DESC, other values does not matter.
E.g. Healthbar first, then Powerbar, and then going all others.
// edit.
Keys must be preserved.
// edit #2
Found a solution, thanks you all.
local function pairsByPriority(t)
local registry = {}
for k, v in pairs(t) do
tinsert(registry, {k, v.priority})
end
tsort(registry, function(a, b) return a[2] > b[2] end)
local i = 0
local iter = function()
i = i + 1
if (registry[i] ~= nil) then
return registry[i][1], t[registry[i][1]]
end
return nil
end
return iter
end
You can't sort a records table because entries are ordered internally by Lua and you can't change the order.
An alternative is to create an array where each entry is a table containing two fields (name and priority) and sort that table instead something like this:
self.modulesArray = {}
for k,v in pairs(self.modules) do
v.name = k --Store the key in an entry called "name"
table.insert(self.modulesArray, v)
end
table.sort(self.modulesArray, function(a,b) return a.priority > b.priority end)
for k,v in ipairs(self.modulesArray) do
print (k,v.name)
end
Output:
1 Healthbar 40
2 Powerbar 35
3 Announcements 0

Resources