I get an error
Stack limit exceeded
in Prolog using append
I am writing a Spanish grammar and need to use append to separate clauses (to separate subject and predicate, get verbs, etc...).
Here I write a simplified use of append that I do not understand why is overflow. I get the answers that I want, but then loops to infinity and beyond!
complejidad(1,0).
complejidad(2,0).
complejidad(2,1).
lista(['A'], 0).
lista(['B'], 0).
lista(Frase, J) :-
lista(Prefijo, 0),
lista(Sufijo, Js),
append(Prefijo, Sufijo, Frase),
complejidad(J, Js).
If I query:
lista(X,_)
I get:
Y put the comlejidad statement to limit the recursion a 2 steps. But something happens with append.
Here is the live example: https://swish.swi-prolog.org/p/stack-limit-exceeded-with-append.pl
And here is my grammar: https://swish.swi-prolog.org/p/mini-gramatica-castellana.v2.pl
I show my grammar to illustrate why I need append!
As you see the error message shows the totally allocated stack 0.2G, which is smaller than in the standalone version, where one gets 1.0G by default. This is to assure that more pengines can run simultaneously on the SWISH server.
The SWI-Prolog stack needs a window for parameter passing. The error message does not show the window size. This is a hard coded parameter, SWI-Prolog needs to be recompiled. Maybe for SWISH the window size should be also reduced.
The stackover flow happens because Js=2 has no solution in complejidad(J, Js). So its backtracking again and again, producing larger and larger intermediate derivations without showing an answer. Until it gets into a stack overflow.
This is not a problem of append/3, rather of lista/2 and depth first search. Try using a Prolog debugger to see what happens.
To limit depth of recursion programatically I add a number parameter:
lista(['A'], N) :- N>=0.
lista(['B'], N) :- N>=0.
lista(Frase, N) :-
N > 0,
N_1 is N-1,
lista(Prefijo, N_1),
lista(Sufijo, N_1),
append(Prefijo, Sufijo, Frase).
In the atomic clauses y put N with N>=0 (that is because I want than the depth was a limit not an exact number; previously I try with N equal to 0).
In the recursive clause y add N>0 and define a N_1 variable to N-1
Related
Looking at the code below:
multiple(X,0).
multiple(X,Y) :- lt(0,X), lt(0,Y), diff(Y,X,D), multiple(X,D).
There happens to be something wrong. For your reference:
lt/2 is whether the first argument is less than the second.
diff/3 is whether the third argument is equal to the first argument minus the second.
lt/2 and diff/3 are defined correctly.
Is there a logical mistake in the definition? Is assuming that 0 is the multiple of every number problematic or is the logical mistake somewhere else? I get correct answers but the query goes to infinite loop I think.
EDIT:
here are the other definitions.
natNum(0).
natNum(s(X)) :- natNum(X).
lt(0,s(X)) :- natNum(X).
lt(s(X),s(Y)) :- lt(X,Y).
sum(0,X,X).
sum(s(X),Y,s(Z)) :- sum(X,Y,Z).
diff(X,Y,Z) :- sum(Z,Y,X).
?- multiple(X, s(s(s(s(s(s(0))))))).
where s(0) is 1, s(s(0)) is 2 etc. It gives all the desired answers for X but after the last answer, it gets stuck. I assume in an infinite recursive loop?
What is happening in your program? Does it loop forever, or does it only take some time since you haven't updated your hardware in recent decades? We cannot tell. (Actually, we could tell by looking at your program, but that is much too complex for the moment).
What we can do with ease is narrow down the source of this costly effort. And this, without a deep understanding of your program. Let's start with the query:
?- multiple(X, s(s(s(s(s(s(0))))))).
X = s(0)
; X = s(s(0))
; X = s(s(s(0)))
; X = s(s(s(s(s(s(0))))))
; loops. % or takes too long
Isn't there an easier way to do this? All this semicolon typing. Instead, simply add false to your query. In this manner the solutions found are no longer shown and we can concentrate on this annoying looping. And, if we're at it, you can also add false goals into your program! By such goals the number of inferences might be reduced (or stays the same). And if the resulting fragment (called a failure-slice) is looping, then this is a reason why your original program loops:
multiple(_X,0) :- false.
multiple(X,Y) :- lt(0,X), false, lt(0,Y), diff(Y,X,D), multiple(X,D).
natNum(0) :- false.
natNum(s(X)) :- natNum(X), false.
lt(0,s(X)) :- natNum(X), false.
lt(s(X),s(Y)) :- false, lt(X,Y).
?- multiple(X, s(s(s(s(s(s(0))))))), false.
loops.
Do your recognize your program? Only those parts remained that are needed for a loop. And, actually in this case, we have an infinite loop.
To fix this, we need to modify something in the remaining, visible part. I'd go for lt/2 whose first clause can be generalized to lt(0, s(_)).
But wait! Why is it OK to generalize away the requirement that we have a natural number? Look at the fact multiple(X,0). which you have written. You have not demanded that X is a natural number either. This kind of over-generalizations often appears in Prolog programs. They improve termination properties at a relatively low price: Sometimes they are too general but all terms that additionally fit into the generalization are not natural numbers. They are terms like any or [a,b,c], so if they appear somewhere you know that they do not belong to the solutions.
So the idea was to put false goals into your program such that the resulting program (failure-slice) still loops. In the worst case you put false at a wrong place and the program terminates. By trial-and-error you get a minimal failure-slice. All those things that are now stroked through are irrelevant! In particular diff/3. So no need to understand it (for the moment). It suffices to look at the remaining program.
I am currently trying to understand the basics of prolog.
I have a knowledge base like this:
p(a).
p(X) :- p(X).
If I enter the query p(b), the unification with the fact fails and the rule p(X) :- p(X) is used which leads the unification with the fact to fail again. Why is the rule applied over and over again after that? Couldn't prolog return false at this point?
After a certain time I get the message "Time limit exceeded".
I'm not quite sure why prolog uses the rule over and over again, but since it is, I don't understand why I get a different error message as in the following case.
To be clear, I do understand that "p(X) if p(X)" is an unreasonable rule, but I would like to understand what exactly happens there.
If I have a knowledge base like this:
p(X) :- p(X).
p(a).
There is no chance to come to a result even with p(a) because the fact is below the rule and the rule is called over and over again. For this variant I receive a different error message almost instantly "ERROR: Out of local stack" which is comprehensible.
Now my question - what is the difference between those cases?
Why am I receiving different error messages and why is prolog not returning false after the first application of the rule in the above case? My idea would be that in the above case the procedure is kind of restarted each time the rule gets called and in the below case the same procedure calls the rule over and over again. I would be grateful if somebody could elaborate this.
Update: If I query p(a). to the 2nd KB as said I receive "Out of local stack", but if I query p(b). to the same KB I get "Time limit exceeded". This is even more confusing to me, shouldn't the constant be irrelevant for the infinite loop?
Let us first consider the following program fragment that both examples have in common:
p(X) :- p(X).
As you correctly point out, it is obvious that no particular solutions are described by this fragment in isolation. Declaratively, we can read it as: "p(X) holds if p(X) holds". OK, so we cannot deduce any concrete solution from only this clause.
This explains why p(b) cannot hold if only this fragment is considered. Additionally, p(a) does not imply p(b) either, so no matter where you put the fact p(a), you will never derive p(b) from these two clauses.
Procedurally, Prolog still attempts to find cases where p(X) holds. So, if you post ?- p(X). as a query, Prolog will try to find a resolution refutation, disregarding what it has "already tried". For this reason, it will try to prove p(X) over and over. Prolog's default resolution strategy, SLDNF resolution, keeps no memory of which branches have already been tried, and also for this reason can be implemented very efficiently, with little overhead compared to other programming languages.
The difference between an infinite deduction attempt and an out of local stack error error can only be understood procedurally, by taking into account how Prolog executes these fragments.
Prolog systems typically apply an optimization that is called tail call optimization. This is applicable if no more choice-points remain, and means that it can discard (or reuse) existing stack frames.
The key difference between your two examples is obviously where you add the fact: Either before or after the recursive clause.
In your case, if the recursive clause comes last, then no more choice-points remain at the time the goal p(X) is invoked. For this reason, an existing stack frame can be reused or discarded.
On the other hand, if you write the recursive clause first, and then query ?- q(X). (or ?- q(a).), then both clauses are applicable, and Prolog remembers this by creating a choice-point. When the recursive goal is invoked, the choice-point still exists, and therefore the stack frames pile up until they exceed the available limits.
If you query ?- p(b)., then argument indexing detects that p(a) is not applicable, and again only the recursive clause applies, independent of whether you write it before or after the fact. This explains the difference between querying p(X) (or p(a)) and p(b) (or other queries). Note that Prolog implementations differ regarding the strength of their indexing mechanisms. In any case, you should expect your Prolog system to index at least on the outermost functor and arity of the first argument. If necessary, more complex indexing schemes can be constructed manually on top of this mechanism. Modern Prolog systems provide JIT indexing, deep indexing and other mechanisms, and so they often automatically detect the exact subset of clauses that are applicable.
Note that there is a special form of resolution called SLG resolution, which you can use to improve termination properties of your programs in such cases. For example, in SWI-Prolog, you can enable SLG resolution by adding the following directives before your program:
:- use_module(library(tabling)).
:- table p/1.
With these directives, we obtain:
?- p(X).
X = a.
?- p(b).
false.
This coincides with the declarative semantics you expect from your definitions. Several other Prolog systems provide similar facilities.
It should be easy to grasp the concept of infinite loop by studying how standard repeat/0 is implemented:
repeat.
repeat :- repeat.
This creates an infinite number of choice points. First clause, repeat., simply allows for a one time execution. The second clause, repeat :- repeat. makes it infinitely deep recursion.
Adding any number of parameters:
repeat(_, _, ..., _).
repeat(Param1, Param2, ..., ParamN) :- repeat(Param1, Param2, ..., ParamN).
You may have bodies added to these clauses and have parameters of the first class having meaningful names depending on what you are trying to archive. If bodies won't contain cuts, direct or inherited from predicates used, this will be an infinite loop too just as repeat/0.
I just have a little question about the following code I've just written.
permu([],[]).
permu([H|T],R) :- same_length([H|T],R), select(H,R,Rz), permu(T,Rz).
Result: gives out all permutation (doesn't delete redundancies, but that's not the matter; by using findall and defining another predicate I can fix that easily)
My initial code:
permu([],[]).
permu([H|T],R) :- select(H,R,Rz), permu(T,Rz).
Result: gives out the first result and ends up in endless loop, when clicking "Next". But when I decided to precisely define their length (with same_length as shown above) it suddenly worked.
I have already traced both codes, but I still don't understand why the length of both lists HAS to be defined under any circumstances.
Looking at my initial code, both lists lose one element in each step of the recursion. The list on the left gets its head cut off each run and my R variable is being shortened because of select. The recursion process ends, when both lists run out of elements in the same recursion run => due to select, both lists need to share the same number and type of elements.
What does prevent Prolog from realizing it? What makes him fall into the endless loop?
Taking this code as an example I don't have to define the length but it still works:
permu([], []).
permu([H|T], R) :- permu(T, Rz), select(H, R, Rz).
In contrast to my initial code here the select is just put after the recursion call (permu).
I'm working on some experiments for comparing different labeling heuristics in Sicstus Prolog.
But I keep getting into 'Resource error: insufficient memory'.
I'm pretty sure I'm doing something wrong in my testcode.
The following code will replicate my problem:
:- use_module(library(clpfd)).
:- use_module(library(lists)).
atest( R, C ):-
X is R * C,
length( M, X),
domain( M, 0, X),
all_distinct( M ),
statistics(walltime, [_,_SinceLast]),
labeling( [],M ),
statistics(walltime, [_,SinceLast]),
write('Labeling time: '), write(SinceLast),write('ms'),nl.
% Testcode for looping through alot of variations and run one test for each variant
t1:-
Cm = 1000,
Cr = 1000,
(
for(C,0, Cm),
param([Cm,Cr])
do
(
for(R, 0, Cr ),
param([C])
do
atest( C, R )
)
).
A short time after I call the t1 predicate, I get a 'Resource error: insufficient memory' exception.
I guess I should do something after I call atest to release resources?
Also: Is this the correct way to measure labeling-time? Any better way to do this?
You are not doing anything strictly wrong, but you are trying to run
length(Xs,N), domain(Xs,0,N), all_distinct(Xs), labeling([],Xs).
for N up to 1000000. The system constructs a search tree with depth N, and has to store intermediate states of the variables and the constraint system for each level. This takes a lot of memory for large N, and it is quite possible that you get a memory overflow already for a single run with large N.
The second issue is that you are running your benchmarks in a recursive loop, i.e. you are effectively creating a conjunction
atest(0,0), ..., atest(1000,1000)
and since each call to atest/2 succeeds with its first solution and keeps its search state around, this means you are trying to create a search tree with 250500250000 levels...
The simplest improvement is to cut each search after the first solution by changing your code to once(atest(C,R)). A further improvement is to run the benchmarks in a failure-driven loop
(
between(0,Cm,C),
between(0,Cr,R),
once(atest(C,R)),
fail
;
true
)
which will free memory sooner and faster, and reduce distortion of your measurements due to garbage collection.
Toplevel hides alternate answers
If you are testing t1 on SICStus' toplevel shell with some smaller values you might get the wrong impression that t1 has exactly one answer/solution. However, this is not the case! So the toplevel hides from you the other answers. This is a special behavior in the SICStus toplevel which does not show further answers if the query does not contain variables. But there is a total of x! many solutions for your labeling, x! for each test case and the timing for the other solutions is some random value. You are out of memory because for each test case, Prolog keeps a record to continue producing the next solution for each test case.
Looping
I do not recommend using a failure driven loop for testing, instead, use the following loop which is very similar but much safer:
\+ (
between(0, Cm, C),
between(0, Cr, R),
\+ atest(C, R)
).
The big difference to a failure driven loop is when atest/2 accidentally fails for some C and R. In a failure driven loop this will go essentially unnoticed, whereas above construct will fail.
Some systems provide a predicate forall/2 for this purpose.
Timing
If you do timing, better only use the first element of the list and compute the difference:
statistics(walltime, [T0|_]),
Goal,
statistics(walltime, [T1|_]),
D is T1 - T0.
In this manner alternate answers to Goal will give you a more meaningful value.
I am new to prolog, and I have to write a program about the water
jugs. My problem is regarding the initial state of jugs and the query
formation. The query will be of the form:
?- myPredicate(args), filled(j1,1)
Meaning j1 filled with 1 gallon of water. j1 represents one of the jugs; the other is j2. Initally, I have
filled(j1,0)
filled(j2,5)
capacity(j1,2)
capacity(j2,5)
I would really be grateful if you provide me with information regarding the following:
Question A: Do I have to declare initial state of the j1 inside my program? filled(j1,0)
Question B: I need to make my program find a solution for
filled(j1,1). For that I have some ideas, but what I am not sure
about, is how to update filled(J,Volume) from query and
myPredicate.
I am very confused since I have the initial state filled(j1,0) and now
I have to create a filled(j1,1) in myPredicate. So I should have some form of filled(J,Volume) in myPredicate, so the query returns true instead of false.
How do I incorporate filled(J,Voume) inside myPredicate so when the above query is run, I can show correct answer?
Example program with a passed-in parameter, an initial fact and an iterative task to perform. Iteration is by means of recursion. Before each reentry the value related to certain a parameter can be effectively updated for the next pass.
my_loop(N) :- N > 10.
my_loop(N) :- N =< 10,
write(N), nl,
Nextin is N + 1,
my_loop(Nextin).
:- my_loop(1).
A:
The given information (facts) are needed by
the program. They can be made available from keyboard input, as initial
arguments to some predicate, or as facts in the database as you
suggested. Also, only suggested for special situations, the facts
can be hard-coded into some predicate or rule.
Below and above are example of passing the initial information in as parameters: :- my_predicate(args...).
If there are a lot of facts, the database is best. A few facts that
need to change every time is best gotten from the keyboard. Otherwise,
it probably doesn't matter.
:- my_predicate([fill(j1,0),fill(j2,5)], Answer),
write(Answer),
nl.
B:
See the my_ loop example:
In my_loop, the task of counting [1..10] is solved iteratively.
The givens: 1 is passed in as a parameter, mainly because the
program does the same thing over-and-over:
1. take a number (N); quit if it is too big. Otherwise...
2. print it.
3. calculate the next number (N+1)
4. repeat
10 is hard-coded. It could have been a fact: stop_after(10).
Now the data to be manipulated, the variable N in my_loop, and
{ j1,j2 } in myPredicate doesn't actually need to be re-assigned over-and-over again: See my_loop. Just re-enter
the computation when it's time to do the same
thing again, but with different parameters:
cap(j1,2).
cap(j2,5).
my_predicate(Status, Answer) :-
got_juice(Status,0),
Answer=Status.
%%% Instead of changing values, rerun comp. with new values
%%% based on a computation made from the old ones.
my_predicate([filled(j1,J1), filled(j2,J2)], Answer) :-
Used is J1 + J2,
got_juice(Used, J), J \= 0,
cap(j1,C1), cap(C2),
%% Use cap and filled to add more to filled..
NextJ1 is J1 + ...,
NextJ2 is J2 + ...,
my_predicate(filled(j1,NextJ1), filled(..., Answer).
NOTE:
The predicate above is just to demonstrate iteration in Prolog, using the parameters of "myProgaram". For an actual implementation see the program suggested in the comment from matcheek.