Lua pathfinding code needs optimization - performance

After working on my code for a while, optimizing the most obvious things, I've resulted in this:
function FindPath(start, finish, path)
--Define a table to hold the paths
local paths = {}
--Make a default argument
path = path or {start}
--Loop through connected nodes
for i,v in ipairs(start:GetConnectedParts()) do
--Determine if backtracking
local loop = false
for i,vv in ipairs(path) do
if v == vv then
loop = true
end
end
if not loop then
--Make a path clone
local npath = {unpack(path)}
npath[#npath+1] = v
if v == finish then
--If we reach the end add the path
return npath
else
--Otherwise add the shortest part extending from this node
paths[#paths+1] = FindPath(v, finish, npath) --NOTED HERE
end
end
end
--Find and return the shortest path
if #paths > 0 then
local lengths = {}
for i,v in ipairs(paths) do
lengths[#lengths+1] = #v
end
local least = math.min(unpack(lengths))
for i,v in ipairs(paths) do
if #v == least then
return v
end
end
end
end
The problem being, the line noted gets some sort of game script timeout error (which I believe is a because of mass recursion with no yielding). I also feel like once that problem is fixed, it'll probably be rather slow even on the scale of a pacman board. Is there a way I can further optimize it, or perhaps a better method I can look into similar to this?
UPDATE: I finally decided to trash my algorithm due to inefficiency, and implemented a Dijkstra algorithm for pathfinding. For anybody interested in the source code it can be found here: http://pastebin.com/Xivf9mwv

You know that Roblox provides you with the PathfindingService? It uses C-side A* pathing to calculate quite quickly. I'd recommend using it
http://wiki.roblox.com/index.php?title=API:Class/PathfindingService

Try to remodel your algorithm to make use of tail calls. This is a great mechanism available in Lua.
A tail call is a type of recursion where your function returns a function call as the last thing it does. Lua has proper tail calls implementation and it will dress this recursion as a 'goto' under the scenes, so your stack will never blow.
Passing 'paths' as one of the arguments of FindPath might help with that.

I saw your edit about ditching the code, but just to help others stumbling on this question:
ipairs is slower than pairs, which is slower than a numeric for-loop.
If performance matters, never use ipairs, but use a for i=1,#tab loop
If you want to clone a table, use a for-loop. Sometimes, you have to use unpack (returning dynamic amount of trailing nils), but this is not such a case. Unpack is also a slow function.
Replacing ipairs with pairs or numeric for-loops and using loops instead of unpack will increase the performance a lot.
If you want to get the lowest value in a table, use this code snippet:
local lowestValue = values[1]
for k,v in pairs(values) do
if v < lowestValue then
lowestValue = k,v
end
end
This could be rewritten for your path example as so:
local least = #path[1]
for k,v in pairs(path) do
if #v < least then
least = v
end
end
I have to admit, you're very inventive. Not a lot of people would use math.min(unpack(tab)) (not counting the fact it's bad)

Related

Is preallocation possible in this code snippet?

The following is a snippet of code of Ant Colony Optimization. I've removed whatever I feel would absolutely not be necessary to understand the code. The rest I'm not sure as I'm unfamiliar with coding on matlab. However, I'm running this algorithm on 500 or so cities with 500 ants and 1000 iterations, and the code runs extremely slow as compared to other algorithm implementations on matlab. For the purposes of my project, I simply need the datasets, not demonstrate coding capability on matlab, and I had time constraints that simply did not allow me to learn matlab from scratch, as that was not taken into consideration nor expected when the deadline was given, so I got the algorithm from an online source.
Matlab recommends preallocating two variables inside a loop as they are arrays that change size I believe. However I do not fully understand the purpose of those two parts of the code, so I haven't been able to do so. I believe both arrays increment a new item every iteration of the loop, so technically they should both be zero-able and could preallocate the size of both expected final sizes based on the for loop condition, but I'm not sure. I've tried preallocating zeroes to the two arrays, but it does not seem to fix anything as Matlab still shows preallocate for speed recommendation.
I've added two comments on the two variables recommended by MATLAB to preallocate below. If someone would be kind as to skim over it and let me know if it is possible, it'd be much appreciated.
x = 10*rand(50,1);
y = 10*rand(50,1);
n=numel(x);
D=zeros(n,n);
for i=1:n-1
for j=i+1:n
D(i,j)=sqrt((x(i)-x(j))^2+(y(i)-y(j))^2);
D(j,i)=D(i,j);
end
end
model.n=n;
model.x=x;
model.y=y;
model.D=D;
nVar=model.n;
MaxIt=100;
nAnt=50;
Q=1;
tau0=10*Q/(nVar*mean(model.D(:)));
alpha=1;
beta=5;
rho=0.6;
eta=1./model.D;
tau=tau0*ones(nVar,nVar);
BestCost=zeros(MaxIt,1);
empty_ant.Tour=[];
empty_ant.Cost=[];
ant=repmat(empty_ant,nAnt,1);
BestSol.Cost=inf;
for it=1:MaxIt
for k=1:nAnt
ant(k).Tour=randi([1 nVar]);
for l=2:nVar
i=ant(k).Tour(end);
P=tau(i,:).^alpha.*eta(i,:).^beta;
P(ant(k).Tour)=0;
P=P/sum(P);
r=rand;
C=cumsum(P);
j=find(r<=C,1,'first');
ant(k).Tour=[ant(k).Tour j];
end
tour = ant(k).Tour;
n=numel(tour);
tour=[tour tour(1)]; %MatLab recommends preallocation here
ant(k).Cost=0;
for i=1:n
ant(k).Cost=ant(k).Cost+model.D(tour(i),tour(i+1));
end
if ant(k).Cost<BestSol.Cost
BestSol=ant(k);
end
end
for k=1:nAnt
tour=ant(k).Tour;
tour=[tour tour(1)];
for l=1:nVar
i=tour(l);
j=tour(l+1);
tau(i,j)=tau(i,j)+Q/ant(k).Cost;
end
end
tau=(1-rho)*tau;
BestCost(it)=BestSol.Cost;
figure(1);
tour=BestSol.Tour;
tour=[tour tour(1)]; %MatLab recommends preallocation here
plot(model.x(tour),model.y(tour),'g.-');
end
If you change the size of an array, that means copying it to a new location in memory. This is not a huge problem for small arrays but for large arrays this slows down your code immensely. The tour arrays you're using are fixed size (51 or n+1 in this case) so you should preallocate them as zero arrays. the only thing you do is add the first element of the tour again to the end so all you have to do is set the last element of the array.
Here is what you should change:
x = 10*rand(50,1);
y = 10*rand(50,1);
n=numel(x);
D=zeros(n,n);
for i=1:n-1
for j=i+1:n
D(i,j)=sqrt((x(i)-x(j))^2+(y(i)-y(j))^2);
D(j,i)=D(i,j);
end
end
model.n=n;
model.x=x;
model.y=y;
model.D=D;
nVar=model.n;
MaxIt=1000;
nAnt=50;
Q=1;
tau0=10*Q/(nVar*mean(model.D(:)));
alpha=1;
beta=5;
rho=0.6;
eta=1./model.D;
tau=tau0*ones(nVar,nVar);
BestCost=zeros(MaxIt,1);
empty_ant.Tour=zeros(n, 1);
empty_ant.Cost=[];
ant=repmat(empty_ant,nAnt,1);
BestSol.Cost=inf;
for it=1:MaxIt
for k=1:nAnt
ant(k).Tour=randi([1 nVar]);
for l=2:nVar
i=ant(k).Tour(end);
P=tau(i,:).^alpha.*eta(i,:).^beta;
P(ant(k).Tour)=0;
P=P/sum(P);
r=rand;
C=cumsum(P);
j=find(r<=C,1,'first');
ant(k).Tour=[ant(k).Tour j];
end
tour = zeros(n+1,1);
tour(1:n) = ant(k).Tour;
n=numel(ant(k).Tour);
tour(end) = tour(1); %MatLab recommends preallocation here
ant(k).Cost=0;
for i=1:n
ant(k).Cost=ant(k).Cost+model.D(tour(i),tour(i+1));
end
if ant(k).Cost<BestSol.Cost
BestSol=ant(k);
end
end
for k=1:nAnt
tour(1:n)=ant(k).Tour;
tour(end) = tour(1);
for l=1:nVar
i=tour(l);
j=tour(l+1);
tau(i,j)=tau(i,j)+Q/ant(k).Cost;
end
end
tau=(1-rho)*tau;
BestCost(it)=BestSol.Cost;
figure(1);
tour(1:n) = BestSol.Tour;
tour(end) = tour(1); %MatLab recommends preallocation here
plot(model.x(tour),model.y(tour),'g.-');
end
I think that the warning that the MATLAB Editor gives in this case is misplaced. The array is not repeatedly resized, it is just resized once. In principle, tour(end+1)=tour(1) is more efficient than tour=[tour,tour(1)], but in this case you might not notice the difference in cost.
If you want to speed up this code you could think of vectorizing some of its loops, and of reducing the number of indexing operations performed. For example this section:
tour = ant(k).Tour;
n=numel(tour);
tour=[tour tour(1)]; %MatLab recommends preallocation here
ant(k).Cost=0;
for i=1:n
ant(k).Cost=ant(k).Cost+model.D(tour(i),tour(i+1));
end
if ant(k).Cost<BestSol.Cost
BestSol=ant(k);
end
could be written as:
tour = ant(k).Tour;
ind = sub2ind(size(model.D),tour,circshift(tour,-1));
ant(k).Cost = sum(model.D(ind));
if ant(k).Cost < BestSol.Cost
BestSol = ant(k);
end
This rewritten code doesn't have a loop, which usually makes things a little faster, and it also doesn't repeatedly do complicated indexing (ant(k).Cost is two indexing operations, within a loop that will slow you down more than necessary).
There are more opportunities for optimization like these, but rewriting the whole function is outside the scope of this answer.
I have not tried running the code, please let me know if there are any errors when using the proposed change.

Ruby elegant alternative to ++ in nested loops?

Before anything, I have read all the answers of Why doesn't Ruby support i++ or i—? and understood why. Please note that this is not just another discussion topic about whether to have it or not.
What I'm really after is a more elegant solution for the situation that made me wonder and research about ++/-- in Ruby. I've looked up loops, each, each_with_index and things alike but I couldn't find a better solution for this specific situation.
Less talk, more code:
# Does the first request to Zendesk API, fetching *first page* of results
all_tickets = zd_client.tickets.incremental_export(1384974614)
# Initialises counter variable (please don't kill me for this, still learning! :D )
counter = 1
# Loops result pages
loop do
# Loops each ticket on the paged result
all_tickets.all do |ticket, page_number|
# For debug purposes only, I want to see an incremental by each ticket
p "#{counter} P#{page_number} #{ticket.id} - #{ticket.created_at} | #{ticket.subject}"
counter += 1
end
# Fetches next page, if any
all_tickets.next unless all_tickets.last_page?
# Breaks outer loop if last_page?
break if all_tickets.last_page?
end
For now, I need counter for debug purposes only - it's not a big deal at all - but my curiosity typed this question itself: is there a better (more beautiful, more elegant) solution for this? Having a whole line just for counter += 1 seems pretty dull. Just as an example, having "#{counter++}" when printing the string would be much simpler (for readability sake, at least).
I can't simply use .each's index because it's a nested loop, and it would reset at each page (outer loop).
Any thoughts?
BTW: This question has nothing to do with Zendesk API whatsoever. I've just used it to better illustrate my situation.
To me, counter += 1 is a fine way to express incrementing the counter.
You can start your counter at 0 and then get the effect you wanted by writing:
p "#{counter += 1} ..."
But I generally wouldn't recommend this because people do not expect side effects like changing a variable to happen inside string interpolation.
If you are looking for something more elegant, you should make an Enumerator that returns integers one at a time, each time you call next on the enumerator.
nums = Enumerator.new do |y|
c = 0
y << (c += 1) while true
end
nums.next # => 1
nums.next # => 2
nums.next # => 3
Instead of using Enumerator.new in the code above, you could just write:
nums = 1.upto(Float::INFINITY)
As mentioned by B Seven each_with_index will work, but you can keep the page_number, as long all_tickets is a container of tuples as it must be to be working right now.
all_tickets.each_with_index do |ticket, page_number, i|
#stuff
end
Where i is the index. If you have more than ticket and page_number inside each element of all_tickets you continue putting them, just remember that the index is the extra one and shall stay in the end.
Could be I oversimplified your example but you could calculate a counter from your inner and outer range like this.
all_tickets = *(1..10)
inner_limit = all_tickets.size
outer_limit = 5000
1.upto(outer_limit) do |outer_counter|
all_tickets.each_with_index do |ticket, inner_counter|
p [(outer_counter*inner_limit)+inner_counter, outer_counter, inner_counter, ticket]
end
# some conditional to break out, in your case the last_page? method
break if outer_counter > 3
end
all_tickets.each_with_index(1) do |ticket, i|
I'm not sure where page_number is coming from...
See Ruby Docs.

Returning multiple ints and passing them as multiple arguements in Lua

I have a function that takes a variable amount of ints as arguments.
thisFunction(1,1,1,2,2,2,2,3,4,4,7,4,2)
this function was given in a framework and I'd rather not change the code of the function or the .lua it is from. So I want a function that repeats a number for me a certain amount of times so this is less repetitive. Something that could work like this and achieve what was done above
thisFunction(repeatNum(1,3),repeatNum(2,4),3,repeatNum(4,2),7,4,2)
is this possible in Lua? I'm even comfortable with something like this:
thisFunction(repeatNum(1,3,2,4,3,1,4,2,7,1,4,1,2,1))
I think you're stuck with something along the lines of your second proposed solution, i.e.
thisFunction(repeatNum(1,3,2,4,3,1,4,2,7,1,4,1,2,1))
because if you use a function that returns multiple values in the middle of a list, it's adjusted so that it only returns one value. However, at the end of a list, the function does not have its return values adjusted.
You can code repeatNum as follows. It's not optimized and there's no error-checking. This works in Lua 5.1. If you're using 5.2, you'll need to make adjustments.
function repeatNum(...)
local results = {}
local n = #{...}
for i = 1,n,2 do
local val = select(i, ...)
local reps = select(i+1, ...)
for j = 1,reps do
table.insert(results, val)
end
end
return unpack(results)
end
I don't have 5.2 installed on this computer, but I believe the only change you need is to replace unpack with table.unpack.
I realise this question has been answered, but I wondered from a readability point of view if using tables to mark the repeats would be clearer, of course it's probably far less efficient.
function repeatnum(...)
local i = 0
local t = {...}
local tblO = {}
for j,v in ipairs(t) do
if type(v) == 'table' then
for k = 1,v[2] do
i = i + 1
tblO[i] = v[1]
end
else
i = i + 1
tblO[i] = v
end
end
return unpack(tblO)
end
print(repeatnum({1,3},{2,4},3,{4,2},7,4,2))

Unexpected $end after while loop and nested if

I have this program that I am working on that is supposed to find the sum of the first 1000 prime numbers. Currently all I am concerned with is making sure that the program is finding the first 1000 prime numbers, I will add the functionality for adding them later. Here is what I have:
#!/usr/bin/ruby
def prime(num)
is_prime = true
for i in 2..Math.sqrt(num)
if (num % i) == 0
is_prime = false
else
is_prime = true
end
end
return is_prime
end
i = 2
number_of_primes = 0
while number_of_primes < 1000
prime = prime(i)
if prime == true
number_of_primes++
end
i++
end
When i try to run the program I get the following feedback:
sumOfPrimes.rb:32: syntax error, unexpected keyword_end
sumOfPrimes.rb:34: syntax error, unexpected keyword_end
what gives? Any direction is appreciated.
Ruby doesn't have ++ operator, you need to do += 1
number_of_primes += 1
Unasked for, but a few pieces of advice if you're interested:
One of the cool things about Ruby is that question marks are legal in method names. As such you'll often find that 'predicate' methods (methods that test something and return true or false) end with a question mark, like this: odd?. Your prime method is a perfect candidate for this, so we can rename it prime?.
You use a local variable, is_prime, to hold whether you have found a factor of the number you're testing yet - this is the kind of thing you'd expect to do in an imperative language such as java or C - but Ruby has all sorts of cool features from functional programming that you will gain great power and expressiveness by learning. If you haven't come across them before, you may need to google what a block is and how the syntax works, but for this purpose you can just think of it as a way to get some code run on every item of a collection. It can be used with a variety of cool methods, and one of them is perfectly suited to your purpose: none?, which returns true if no items in the collection it is called on, when passed to the code block you give, return true. So your prime? method can be rewritten like this:
def prime? num
(2..Math.sqrt(num)).none? { |x| num % x == 0 }
end
Apart from being shorter, the advantage of not needing to use local variables like is_prime is that you give yourself fewer opportunities to introduce bugs - if for example you think the contents of is_prime is one thing but it's actually another. It's also, if you look carefully, a lot closer to the actual mathematical definition of a prime number. So by cutting out the unnecessary code you can get closer to exposing the 'meaning' of what you're writing.
As far as getting the first 1000 primes goes, infinite streams are a really cool way to do this but are probably a bit complex to explain here - definitely google if you're interested as they really are amazing! But just out of interest, here's a simple way you could do it using just recursion and no local variables (remember local variables are the devil!):
def first_n_primes(i = 2, primes = [], n)
if primes.count == n then primes
elsif prime? i then first_n_primes(i + 1, primes + [i], n)
else first_n_primes(i + 1, primes, n)
end
end
And as far as summing them up goes all I'll say is have a search for a ruby method called inject - also called reduce. It might be a bit brain-bending at first if you haven't come across the concept before but it's well worth learning! Very cool and very powerful.
Have fun!

Problem with Ruby recursive lambda call

I have the following code that correctly traverses all the nodes in a graph like so:
seen = {}
dfs = lambda do |node|
return if seen[node]
seen[node] = true
$edges[node].each {|n| dfs.call n}
end
dfs.call 0
However, I would like to write it this way, which I understand is correct:
$edges[node].each &dfs
However, when I do this it appears that dfs is only being called on the first element of the list of nodes in $edge[node]. What gives?
Surprisingly enough, your problem is not in the recursion! It actually is because of the shared seen collection among all calls in $nodes[node].each &dfs.
Let's go through the operation: the call to $nodes[node].first should not have any problems, because we know the snippet works for any one node. There is a problem however: seen is not reset, and you are already going to the next node! You already saw all the nodes, so when you try even one on the next cycle, it will immediately return out of the proc because of the condition. The same will happen for every other call as you loop through $nodes. It only seems that the calls to the rest of the nodes never happened!
To solve your problem, isolate seen to the scope of each call of dfs, which we can still do in functional programming:
dfs = lambda do |node|
seen = []
sub_dfs = lambda do |sub_node|
return if seen.include? sub_node
seen << sub_node
$edges[sub_node].each &sub_dfs
end
sub_dfs.call node
end
$edges[some_node].each &dfs
Now seen is safely isolated in each call to dfs.
Another way to make recursive lambdas:
fac = lambda{|n, &context| n.zero? ? 1 : n * eval("fac.call(#{n-1}) {}",context.binding)}
But have to be called with empty block though
fac.call(2){} = 2
fac.call(3){} = 6
fac.call(4){} = 24
binding is used to evaluate the code outside of lambda scope

Resources