how to sort a table in lua? - sorting

I have a lua table that contains 2 key pieces of data. I would like to sort the table in ascending order by the "num1" column, or if thats not possible, they by the key value in ascending order
Here's what I have so far:
local widgets = {}
widgets[1568] = {}
widgets[1568]["num1"] = 99999
widgets[1568]["val2"] = "NA"
widgets[246] = {}
widgets[246]["num1"] = 90885
widgets[246]["val2"] = "NA"
widgets[250] = {}
widgets[250]["num1"] = 95689
widgets[250]["val2"] = "NA"
widgets[251] = {}
widgets[251]["num1"] = 95326
widgets[251]["val2"] = "NA"
widgets[252] = {}
widgets[252]["num1"] = 95301
widgets[252]["val2"] = "NA"
widgets[256] = {}
widgets[256]["num1"] = 95303
widgets[256]["val2"] = "NA"
-- ATTEMPT TO SORT
--table.sort(widgets, function(a,b) return tonumber(a.num1.value) < tonumber(b.num1.value) end)
--table.sort(widgets, function(a,b) return tonumber(a.num1) < tonumber(b.num1) end)
--TRY SORTING BY ID:
table.sort(widgets, function(a,b) return tonumber(a) < tonumber(b) end)
for i, v in pairs(widgets) do
print(v.num1)
end
Any suggestions would be appreciated. Right now, I'm reviewing Sort a Table[] in Lua to try to understand the "spairs" function. But that example is slightly different because I have a table within a table...
Thanks.
SOLUTION
In line with the answer below, I created a new table and added the records from the old table, one by one, using table insert like so:
local new_widgets = {}
for i, v in pairs(widgets) do
table.insert(new_widgets, id=v.id, num1= v.num1, num2 = v.num2)
end
then I sorted new_wigets.

Lua tables are hashtables. Their entries have no specific order.
You fake it by using consecutive numerical indices then iterating by incrementing a number (note: internally Lua actually will implement this as an array, but that's an implementation detail; conceptually, table entries have no specific order).
t[2] = "two"
t[3] = "three"
t[1] = "one"
for i=1,#t do print(t[i]) end
ipairs creates an iterator that does the same thing as this for loop.
So, if you want your data sorted, you need to put it in a table with consecutive numeric indices.
In your case, there's a lot of different ways you can do it. Here's one way to skin that cat:
Instead of this:
local widgets = {
[246] = { num1 = 90885, val2 = "NA" }
[250] = { num1 = 95689, val2 = "NA" }
[251] = { num1 = 95326, val2 = "NA" }
[252] = { num1 = 95301, val2 = "NA" }
[256] = { num1 = 95303, val2 = "NA" }
}
You want this:
local widgets = {
{ id = 246, num1 = 90885, val2 = "NA" },
{ id = 250, num1 = 95689, val2 = "NA" },
{ id = 251, num1 = 95326, val2 = "NA" },
{ id = 252, num1 = 95301, val2 = "NA" },
{ id = 256, num1 = 95303, val2 = "NA" },
}
-- sort ascending by num1
table.sort(widgets, function(a,b) return a.num1 < b.num1 end)
for i, widget in ipairs(widgets) do
print(widget.num1)
end
If you need the ability to then lookup a widget quickly by id, you can create a lookup table for that:
local widgetById = {}
for i,widget in pairs(widgets) do
widgetById[widget.id] = widget
end

Related

Table keys not sorting correctly

I have a table that looks like this
{
["slot1"] = {}
["slot2"] = {}
["slot3"] = {}
["slot4"] = {}
["slot5"] = {}
["slot6"] = {}
}
When I do a for k, v in pairs loop I want the keys to go from slot1- the last slot. At the moment when I do a loop the order is inconsistent, slot 5 comes first etc. What is the best way to do this?
also I did not design this table, and I cannot change how the keys look
You can write a simple custom iterator:
local tbl = {
["slot1"] = {},
["slot2"] = {},
["slot3"] = {},
["slot4"] = {},
["slot5"] = {},
["slot6"] = {}
}
function slots(tbl)
local i = 0
return function()
i = i + 1
if tbl["slot" .. i] ~= nil then
return i, tbl["slot" .. i]
end
end
end
for i, element in slots(tbl) do
print(i, element)
end
Output:
1 table: 0xd575f0
2 table: 0xd57720
3 table: 0xd57760
4 table: 0xd5aa40
5 table: 0xd5aa80
6 table: 0xd5aac0
Create a new table:
slot = {}
for k,v in pairs(original_table) do
local i=tonumber(k:match("%d+$"))
slot[i]=v
end
Lua table orders are undeterministic. see here what makes lua tables key order be undeterministic
For your table you can try this
local t = {
["slot1"] = {},
["slot2"] = {},
["slot3"] = {},
["slot4"] = {},
["slot5"] = {},
["slot6"] = {}
}
local slotNumber = 1
while(t['slot' .. slotNumber]) do
slot = t['slot' .. slotNumber]
-- do stuff with slot
slotNumber = slotNumber + 1
end
This method does not handle if the table skips a slot number.

Fastest way to search for a row in a large Google Sheet using/in Google Apps Script

GAS is quite powerful and you could write a full fledged web-app using a Google Sheet as the DB back-end. There are many reasons not to do this but I figure in some cases it is okay.
I think the biggest issue will be performance issues when looking for rows based on some criteria in a sheet with a lot of rows. I know there are many ways to "query" a sheet but I can't find reliable information on which is the fastest.
One of the complexities is that many people can edit a sheet which means there are a variable number of situations you'd have to account for. For the sake of simplicity, I want to assume the sheet:
Is locked down so only one person can see it
The first column has the row number (=row())
The most basic query is finding a row where a specific column equals some value.
Which method would be the fastest?
I have a sheet with ~19k rows and ~38 columns, filled with all sorts of unsorted real-world data. That is almost 700k rows so I figured it would be a good sheet to time a few methods and see which is the fastest.
method 1: get sheet as a 2D array then go through each row
method 2: get sheet as a 2D array, sort it, then using a binary search algorithm to find the row
method 3: make a UrlFetch call to Google visualization query and don't provide last row
method 4: make a UrlFetch call to Google visualization query and provide last row
Here are the my query functions.
function method1(spreadsheetID, sheetName, columnIndex, query)
{
// get the sheet values excluding header,
var rowValues = SpreadsheetApp.openById(spreadsheetID).getSheetByName(sheetName).getSheetValues(2, 1, -1, -1);
// loop through each row
for(var i = 0, numRows = rowValues.length; i < numRows; ++i)
{
// return it if found
if(rowValues[i][columnIndex] == query) return rowValues[i]
}
return false;
}
function method2(spreadsheetID, sheetName, columnIndex, query)
{
// get the sheet values excluding header
var rowValues = SpreadsheetApp.openById(spreadsheetID).getSheetByName(sheetName).getSheetValues(2, 1, -1, -1);
// sort it
rowValues.sort(function(a, b){
if(a[columnIndex] < b[columnIndex]) return -1;
if(a[columnIndex] > b[columnIndex]) return 1;
return 0;
});
// search using binary search
var foundRow = matrixBinarySearch(rowValues, columnIndex, query, 0, rowValues.length - 1);
// return if found
if(foundRow != -1)
{
return rowValues[foundRow];
}
return false;
}
function method3(spreadsheetID, sheetName, queryColumnLetterStart, queryColumnLetterEnd, queryColumnLetterSearch, query)
{
// SQL like query
myQuery = "SELECT * WHERE " + queryColumnLetterSearch + " = '" + query + "'";
// the query URL
// don't provide last row in range selection
var qvizURL = 'https://docs.google.com/spreadsheets/d/' + spreadsheetID + '/gviz/tq?tqx=out:json&headers=1&sheet=' + sheetName + '&range=' + queryColumnLetterStart + ":" + queryColumnLetterEnd + '&tq=' + encodeURIComponent(myQuery);
// fetch the data
var ret = UrlFetchApp.fetch(qvizURL, {headers: {Authorization: 'Bearer ' + ScriptApp.getOAuthToken()}}).getContentText();
// remove some crap from the return string
return JSON.parse(ret.replace("/*O_o*/", "").replace("google.visualization.Query.setResponse(", "").slice(0, -2));
}
function method4(spreadsheetID, sheetName, queryColumnLetterStart, queryColumnLetterEnd, queryColumnLetterSearch, query)
{
// find the last row in the sheet
var lastRow = SpreadsheetApp.openById(spreadsheetID).getSheetByName(sheetName).getLastRow();
// SQL like query
myQuery = "SELECT * WHERE " + queryColumnLetterSearch + " = '" + query + "'";
// the query URL
var qvizURL = 'https://docs.google.com/spreadsheets/d/' + spreadsheetID + '/gviz/tq?tqx=out:json&headers=1&sheet=' + sheetName + '&range=' + queryColumnLetterStart + "1:" + queryColumnLetterEnd + lastRow + '&tq=' + encodeURIComponent(myQuery);
// fetch the data
var ret = UrlFetchApp.fetch(qvizURL, {headers: {Authorization: 'Bearer ' + ScriptApp.getOAuthToken()}}).getContentText();
// remove some crap from the return string
return JSON.parse(ret.replace("/*O_o*/", "").replace("google.visualization.Query.setResponse(", "").slice(0, -2));
}
My binary search algorithm:
function matrixBinarySearch(matrix, columnIndex, query, firstIndex, lastIndex)
{
// find the value using binary search
// https://www.w3resource.com/javascript-exercises/javascript-array-exercise-18.php
// first make sure the query string is valid
// if it is less than the smallest value
// or larger than the largest value
// it is not valid
if(query < matrix[firstIndex][columnIndex] || query > matrix[lastIndex][columnIndex]) return -1;
// if its the first row
if(query == matrix[firstIndex][columnIndex]) return firstIndex;
// if its the last row
if(query == matrix[lastIndex][columnIndex]) return lastIndex;
// now start doing binary search
var middleIndex = Math.floor((lastIndex + firstIndex)/2);
while(matrix[middleIndex][columnIndex] != query && firstIndex < lastIndex)
{
if(query < matrix[middleIndex][columnIndex])
{
lastIndex = middleIndex - 1;
}
else if(query > matrix[middleIndex][columnIndex])
{
firstIndex = middleIndex + 1;
}
middleIndex = Math.floor((lastIndex + firstIndex)/2);
}
return matrix[middleIndex][columnIndex] == query ? middleIndex : -1;
}
This is the function I used to test them all:
// each time this function is called it will try one method
// the first time it is called it will try method1
// then method2, then method3, then method4
// after it does method4 it will start back at method1
// we will use script properties to save which method is next
// we also want to use the same query string for each batch so we'll save that in script properties too
function testIt()
{
// get the sheet where we're staving run times
var runTimesSheet = SpreadsheetApp.openById("...").getSheetByName("times");
// we want to see true speed tests and don't want server side caching so we a copy of our data sheet
// make a copy of our data sheet and get its ID
var tempSheetID = SpreadsheetApp.openById("...").copy("temp sheet").getId();
// get script properties
var scriptProperties = PropertiesService.getScriptProperties();
// the counter
var searchCounter = Number(scriptProperties.getProperty("searchCounter"));
// index of search list we want to query for
var searchListIndex = Number(scriptProperties.getProperty("searchListIndex"));
// if we're at 0 then we need to get the index of the query string
if(searchCounter == 0)
{
searchListIndex = Math.floor(Math.random() * searchList.length);
scriptProperties.setProperty("searchListIndex", searchListIndex);
}
// query string
var query = searchList[searchListIndex];
// save relevant data
var timerRow = ["method" + (searchCounter + 1), searchListIndex, query, 0, "", "", "", ""];
// run the appropriate method
switch(searchCounter)
{
case 0:
// start time
var start = (new Date()).getTime();
// run the query
var ret = method1(tempSheetID, "Extract", 1, query);
// end time
timerRow[3] = ((new Date()).getTime() - start) / 1000;
// if we found the row save its values in the timer output so we can confirm it was found
if(ret)
{
timerRow[4] = ret[0];
timerRow[5] = ret[1];
timerRow[6] = ret[2];
timerRow[7] = ret[3];
}
break;
case 1:
var start = (new Date()).getTime();
var ret = method2(tempSheetID, "Extract", 1, query);
timerRow[3] = ((new Date()).getTime() - start) / 1000;
if(ret)
{
timerRow[4] = ret[0];
timerRow[5] = ret[1];
timerRow[6] = ret[2];
timerRow[7] = ret[3];
}
break;
case 2:
var start = (new Date()).getTime();
var ret = method3(tempSheetID, "Extract", "A", "AL", "B", query);
timerRow[3] = ((new Date()).getTime() - start) / 1000;
if(ret.table.rows.length)
{
timerRow[4] = ret.table.rows[0].c[0].v;
timerRow[5] = ret.table.rows[0].c[1].v;
timerRow[6] = ret.table.rows[0].c[2].v;
timerRow[7] = ret.table.rows[0].c[3].v;
}
break;
case 3:
var start = (new Date()).getTime();
var ret = method3(tempSheetID, "Extract", "A", "AL", "B", query);
timerRow[3] = ((new Date()).getTime() - start) / 1000;
if(ret.table.rows.length)
{
timerRow[4] = ret.table.rows[0].c[0].v;
timerRow[5] = ret.table.rows[0].c[1].v;
timerRow[6] = ret.table.rows[0].c[2].v;
timerRow[7] = ret.table.rows[0].c[3].v;
}
break;
}
// delete the temp file
DriveApp.getFileById(tempSheetID).setTrashed(true);
// save run times
runTimesSheet.appendRow(timerRow);
// start back at 0 if we're the end
if(++searchCounter == 4) searchCounter = 0;
// save the search counter
scriptProperties.setProperty("searchCounter", searchCounter);
}
I have a global variable searchList that is an array of various query strings -- some are in the sheet, some are not.
I ran testit on a trigger to run every minute. After 152 iterations I had 38 batches. Looking at the result, this is what I see for each method:
| Method | Minimum Seconds | Maximum Seconds | Average Seconds |
|---------|-----------------|-----------------|-----------------|
| method1 | 8.24 | 36.94 | 11.86 |
| method2 | 9.93 | 23.38 | 14.09 |
| method3 | 1.92 | 5.48 | 3.06 |
| method4 | 2.20 | 11.14 | 3.36 |
So it appears that, at least for my data-set, is using Google visualization query is the fastest.

Finding highest number in text file

I have a text file that contains 50 student names and scores for each student in the format.
foreName.Surname:Mark
I have figured out how to split up each line into a forename, surname and mark using this code.
string[] Lines = File.ReadAllLines(#"StudentExamMarks.txt");
int i = 0;
var items = from line in Lines
where i++ != 0
let words = line.Split(' ', '.', ':')
select new
{
foreName = words[0],
Surname = words[1],
Mark = words[2]
};
I am unsure of how i would incorporate a findMax algorithm into to find the highest mark and display the pupil with the highest mark. this as i have not used text files that often.
You can use any sorting algorithm there is a Pseudo Code available to find maximum number in any list or array..
Try this code, required just parse all files.
string[] lines = File.ReadAllLines(#"StudentExamMarks.txt");
string maxForeName = null;
string maxSurName = null;
var maxMark = 0;
for (int i = 0; i < lines.Length; i++)
{
var tmp = lines[i].Split(new char[] { ' ', '.', ':' }, StringSplitOptions.RemoveEmptyEntries);
if (tmp.Length == 3)
{
int value = int.Parse(tmp[2]);
if (i == 0 || value > maxMark)
{
maxMark = value;
maxForeName = tmp[0];
maxSurName = tmp[1];
}
}
}

Lua - how to sort a table by the value chain

I'm looking for a method of sorting a Lua table by its values chain. Say, the table:
local vals = {
{ id = "checkpoint4" },
{ id = "checkpoint1", nextid = "checkpoint2" },
{ id = "checkpoint3", nextid = "checkpoint4" },
{ id = "checkpoint2", nextid = "checkpoint3" },
}
Should transform into this after sorting:
local vals = {
{ id = "checkpoint1", nextid = "checkpoint2" },
{ id = "checkpoint2", nextid = "checkpoint3" },
{ id = "checkpoint3", nextid = "checkpoint4" },
{ id = "checkpoint4" },
}
It's not essentially with the exact same names, they might vary. I wanted to make the comparison of numbers after "checkpoint", but it turned out that I have to work with dynamic things like this (already sorted the way I want it to be):
local vals = {
{ id = "checkpoint1", nextid = "cp" },
{ id = "cp", nextid = "chp" },
{ id = "chp", nextid = "mynextcheckpoint" },
{ id = "mynextcheckpoint"},
}
Thanks.
What you are describing is called topological sorting. However, since this is a restricted case, we do not have to implement a complete topological sorting algorithm:
function sort_list(tbl)
local preceding = {}
local ending
local sorted = {}
for i, e in ipairs(tbl) do
if e.nextid == nil then
ending = e
else
preceding[e.nextid] = i
end
end
if ending == nil then
return nil, "no ending"
end
local j = #tbl
while ending ~= nil do
sorted[j] = ending
ending = tbl[preceding[ending.id]]
j = j - 1
end
if sorted[1] == nil then
return nil, "incomplete list"
end
return sorted
end
Usage:
local sorted = sort_list(vals)
local id2val, tailsizes = {}, {}
for _, val in ipairs(vals) do id2val[val.id] = val end
local function tailsize(val) -- memoized calculation of tails sizes
if not tailsizes[val] then
tailsizes[val] = 0 -- cycle-proof
if val.nextid and id2val[val.nextid] then -- dangling nextid proof
tailsizes[val] = tailsize(id2val[val.nextid]) + 1
end
end
return tailsizes[val]
end
-- sorting according to tails sizes
table.sort(vals, function(a,b) return tailsize(a) > tailsize(b) end)

How to sort this lua table?

I have next structure
self.modules = {
["Announcements"] = {
priority = 0,
-- Tons of other attributes
},
["Healthbar"] = {
priority = 40,
-- Tons of other attributes
},
["Powerbar"] = {
priority = 35,
-- Tons of other attributes
},
}
I need to sort this table by priorty DESC, other values does not matter.
E.g. Healthbar first, then Powerbar, and then going all others.
// edit.
Keys must be preserved.
// edit #2
Found a solution, thanks you all.
local function pairsByPriority(t)
local registry = {}
for k, v in pairs(t) do
tinsert(registry, {k, v.priority})
end
tsort(registry, function(a, b) return a[2] > b[2] end)
local i = 0
local iter = function()
i = i + 1
if (registry[i] ~= nil) then
return registry[i][1], t[registry[i][1]]
end
return nil
end
return iter
end
You can't sort a records table because entries are ordered internally by Lua and you can't change the order.
An alternative is to create an array where each entry is a table containing two fields (name and priority) and sort that table instead something like this:
self.modulesArray = {}
for k,v in pairs(self.modules) do
v.name = k --Store the key in an entry called "name"
table.insert(self.modulesArray, v)
end
table.sort(self.modulesArray, function(a,b) return a.priority > b.priority end)
for k,v in ipairs(self.modulesArray) do
print (k,v.name)
end
Output:
1 Healthbar 40
2 Powerbar 35
3 Announcements 0

Resources