There is a very detailed Draft proposal for setup_call_cleanup/3.
Let me quote the relevant part for my question:
c) The cleanup handler is called exactly once; no later than upon failure of G. Earlier moments are:
If G is true or false, C is called at an implementation dependent moment after the last solution and after the last observable effect of G.
And this example:
setup_call_cleanup(S=1,G=2,write(S+G)).
Succeeds, unifying S = 1, G = 2.
Either: outputs '1+2'
Or: outputs on backtracking '1+_' prior to failure.
Or (?): outputs on backtracking '1+2' prior to failure.
In my understanding, this is basically because a unification is a
backtrackable goal; it simply fails on reexecution. Therefore, it is
up to the implementation to decide whether to call the cleanup just
after the fist execution (because there will be no more observable
effects of the goal), or postpone until the second execution of the
goal which now fails.
So it seems to me that this cannot be used to detect determinism
portably. Only a few built-in constructs like true, fail, !
etc. are genuinely non-backtrackable.
Is there any other way to check determinism without executing a goal twice?
I'm currently using SWI-prolog's deterministic/1, but I'd certainly
appreciate portability.
No. setup_call_cleanup/3 cannot detect determinism in a portable manner. For this would restrict an implementation in its freedom. Systems have different ways how they implement indexing. They have different trade-offs. Some have only first argument indexing, others have more than that. But systems which offer "better" indexing behave often quite randomly. Some systems do indexing only for nonvar terms, others also permit clauses that simply have a variable in the head - provided it is the last clause. Some may do "manual" choice point avoidance with safe tests prior to cuts and others just omit this. Brief, this is really a non-functional issue, and insisting on portability in this area is tantamount to slowing systems down.
However, what still holds is this: If setup_call_cleanup/3 detects determinism, then there is no further need to use a second goal to determine determinism! So it can be used to implement determinism detection more efficiently. In the general case, however, you will have to execute a goal twice.
The current definition of setup_call_cleanup/3 was also crafted such as to permit an implementation to also remove unnecessary choicepoints dynamically.
It is thinkable (not that I have seen such an implementation) that upon success of Call and the internal presence of a choicepoint, an implementation may examine the current choicepoints and remove them if determinism could be detected. Another possibility might be to perform some asynchronous garbage collection in between. All these options are not excluded by the current specification. It is unclear if they will ever be implemented, but it might happen, once some applications depend on such a feature. This has happened in Prolog already a couple of times, so a repetition is not completely fantasy. In fact I am thinking of a particular case to help DCGs to become more determinate. Who knows, maybe you will go that path!
Here is an example how indexing in SWI depends on the history of previous queries:
?- [user].
p(_,a). p(_,b). end_of_file.
true.
?- p(1,a).
true ;
false.
?- p(_,a).
true.
?- p(1,a).
true. % now, it's determinate!
Here is an example, how indexing on the second argument is strictly weaker than first argument indexing:
?- [user].
q(_,_). q(0,0). end_of_file.
true.
?- q(X,1).
true ; % weak
false.
?- q(1,X).
true.
As you're interested in portability, Logtalk's lgtunit tool defines a portable deterministic/1 predicate for 10 of its supported backend Prolog compilers:
http://logtalk.org/library/lgtunit_0.html
https://github.com/LogtalkDotOrg/logtalk3/blob/master/tools/lgtunit/lgtunit.lgt (starting around line 1051)
Note that different systems use different built-in predicates that approximate the intended functionality.
Related
Main features
I recently have been looking to make a Prolog meta-interpreter with a certain set of features, but I am starting to see that I don't have the theoretical knowledge to work on it.
The features are as follows :
Depth-first search.
Interprets any non-recursive Prolog program the same way a classic interpreter would.
Guarantees breaking out of any infinite recursion. This most-likely means breaking Turing-completeness, and I'm okay with that.
As long as each step of the recursion reduces the complexity of the expression, keep evaluating it. To be more specific, I want predicates to be allowed to call themselves, but I want to prevent a clause to be able to call a similarly or more complex version of itself.
Obviously, (3) and (4) are the ones I am having problems with. I am not sure if those 2 features are compatible. I am not even sure if there is a way to define complexity such that (4) makes logical sense.
In my researches, I have come across the concept of "unavoidable pattern", which, I believe, provides a way to ensure feature (3), as long as feature (4) has a well-formed definition.
I specifically want to know if this kind of interpreter has been proven impossible, and, if not, if theoretical or concrete work on similar interpreters has been done in the past.
Extra features
Provided the above features are possible to implement, I have extra features I want to add, and would be grateful if you could enlighten me on the feasibility of such features as well :
Systematically characterize and describe those recursions, such that, when one is detected, a user-defined predicate or clause could be called that matches this specific form of recursion.
Detect patterns that result in an exponential number of combinatorial choices, prevent evaluation, and characterize them in the same way as step (5), such that they can be handled by a built-in or user-defined predicate.
Example
Here is a simple predicate that obviously results in infinite recursion in a normal Prolog interpreter in all but the simplest of cases. This interpreter should be able to evaluate it in at most PSPACE (and, I believe, at most P if (6) is possible to implement), while still giving relevant results.
eq(E, E).
eq(E, F):- eq(E,F0), eq(F0,F).
eq(A + B, AR + BR):- eq(A, AR), eq(B, BR).
eq(A + B, B + A).
eq(A * B, B * A).
eq((A * B) / B, A).
eq(E, R):- eq(R, E).
Example of results expected :
?- eq((a + c) + b, b + (c + a)).
true.
?- eq(a, (a * b) / C).
C = b.
The fact that this kind of interpreter might prove useful by the provided example hints me towards the fact that such an interpreter is probably impossible, but, if it is, I would like to be able to understand what makes it impossible (for example, does (3) + (4) reduce to the halting problem? is (6) NP?).
If you want to guarantee termination you can conservatively assume any input goal is nonterminating until proven otherwise, using a decidable proof procedure. Basically, define some small class of goals which you know halt, and expand it over time with clever ideas.
Here are three examples, which guarantee or force three different kinds of termination respectively (also see the Power of Prolog chapter on termination):
existential-existential: at least one answer is reached before potentially diverging
universal-existential: no branches diverge but there may be an infinite number of them, so the goal may not be universally terminating
universal-universal: after a finite number of steps, every answer will be reached, so in particular there must be a finite number of answers
In the following, halts(Goal) is assumed to correctly test a goal for existential-existential termination.
Existential-Existential
This uses halts/1 to prove existential termination of a modest class of goals. The current evaluator eval/1 just falls back to the underlying engine:
halts(halts(_)).
eval(Input) :- Input.
:- \+ \+ halts(halts(eval(_))).
safe_eval(Input) :-
halts(eval(Input)),
eval(Input).
?- eval((repeat, false)).
C-c C-cAction (h for help) ? a
abort
% Execution Aborted
?- safe_eval((repeat, false)).
false.
The optional but highly recommended goal directive \+ \+ halts(halts(eval(_))) ensures that halts will always halt when run on eval applied to anything.
The advantage of splitting the problem into a termination checker and an evaluator is that the two are decoupled: you can use any evaluation strategy you want. halts can be gradually augmented with more advanced methods to expand the class of allowed goals, which frees up eval to do the same, e.g. clause reordering based on static/runtime mode analysis, propagating failure, etc. eval can itself expand the class of allowed goals by improving termination properties which are understood by halts.
One caveat - inputs which use meta-logical predicates like var/1 could subvert the goal directive. Maybe just disallow such predicates at first, and again relax the restriction over time as you discover safe patterns of use.
Universal-Existential
This example uses a meta-interpreter, adapted from the Power of Prolog chapter on meta-interpreters, to prune off branches which can't be proven to existentially terminate:
eval(true).
eval((A,B)) :- eval(A), eval(B).
eval((A;_)) :- halts(eval(A)), eval(A).
eval((_;B)) :- halts(eval(B)), eval(B).
eval(g(Head)) :-
clause(Head, Body),
halts(eval(Body)),
eval(Body).
So here we're destroying branches, rather than refusing to evaluate the goal.
For improved efficiency, you could start by naively evaluating the input goal and building up per-branch sets of visited clause bodies (using e.g. clause/3), and only invoke halts when you are about to revisit a clause in the same branch.
Universal-Universal
The above meta-interpreter rules out at least all the diverging branches, but may still have an infinite number of individually terminating branches. If we want to ensure universal termination we can again do everything before entering eval, as in the existential-existential variation:
...
:- \+ \+ halts(halts(\+ \+ eval(_))).
...
safe_eval(Input) :-
halts(\+ \+ eval(Input)),
eval(Input).
So we're just adding in universal quantification.
One interesting thing you could try is running halts itself using eval. This could yield speedups, better termination properties, or qualitatively new capabilities, but would of course require the goal directive and halts to be written according to eval's semantics. E.g. if you remove double negations then \+ \+ would not universally quantify, and if you propagate false or otherwise don't conform to the default left-to-right strategy then the (goal, false) test for universal termination (PoP chapter on termination) also would not work.
Edit: This question assumes you enabled the occurs check. It's not about setting Prolog flags.
There was a bunch of papers 30 years ago about optimizing away the occurs check automatically, when it's safe (about 90% of predicates, in typical codebases). Different algorithms were proposed. Do modern Prolog compilers (such as SWI Prolog) do anything like that (when the occurs check is turned on)? What algorithms do they favor?
As an example, would they remove the occurs check when compiling a predicate such as this:
less(0, s(_)).
less(s(X), s(Y)) :- less(X, Y).
(from the discussion below this answer)
There are a variety of optimizations that can optimize away the occurs check when the occurs check flag is set to "true". This is necessary to make sound unification feasible for common cases. A typical optimization is the detection of linearity of a head:
linear(Head) :-
term_variables(Head, Vars),
term_singletons(Head, Singletons),
Vars == Singletons.
In this case the occurs check can be omitted when invoking a clause. We can make the test for the less/2 example whether the heads are linear or not. It turns out that both heads are linear:
?- linear(less(0, s(_))).
true.
?- linear(less(s(X), s(Y))).
true.
So a Prolog system can totally omit the occurs check for the predicate less/2 and nevertheless produce sound unification. This optimization is for example seen in Jekejeke Prolog when inspecting the intermediate code. The instruction unify_linear is used:
?- vm_list(less/2).
less(0, s(A)).
0 unify_linear _0, 0
1 unify_linear _1, s(A)
less(s(X), s(Y)) :-
less(X, Y).
0 unify_linear _0, s(X)
1 unify_linear _1, s(Y)
2 goal_last less(X, Y)
In contrast to the instruction unify_term, the instruction unify_linear does not perform an occurs check, even when the occurs check flag is set to true.
Ritu Chadha; David A. Plaisted (1994). "Correctness of unification without occur check in prolog". The Journal of Logic Programming. 18 (2): 99–122. doi:10.1016/0743-1066(94)90048-5.
SWI Prolog does not switch on the occurs check unless you specifically ask it to:
By using unify_with_occurs_check/2 instead of = locally. (Note that this means head unification is not affected, i.e. still runs without occurs check - I think?)
By switching on occurs check globally thorugh setting the flag occurs_check: set_prolog_flag(occurs_check,true)
Is this a problem? I don't think so.
Consider the related case of having assertions in other programming languages (or even in Prolog for that matter: assertion/1, I heartily recommend it).
When you develop, you will have those "switched on" to verify constraints at runtime at some CPU cost. Once you are sure your program works (by consruction, prrof, and testing), you will "switch them off". Interfaces between your program and any caller (who may be malicious, confused or buggy) may still be guarded by must_be/2 checks so that interface contracts are enforced.
The case of the occurs check would be similar: If you suspect that cyclic data structures may occur and lead to problems, switch 'occurs check' on. Code you program so that everything works right (and you are sure that it works right, preferably by construction and proof). Then switch it off again.
Whats also popular, using a distinction inside the variables, like "fresh" and "stale" variables. "fresh" variables typically don't need an occurs check.
The distinction can be made dynamically. Here are some results, the dynamic distinction ("Freshness") can be also combined with some static analysis ("Linear"):
The above graph shows the result of a heuristic based on reference counting. The below paper discusses using one bit, i.e. tagging variables as UNBOUND or NEW_UNBOUND.
The Occur-Check Problem Revisited - Beer, 1988
https://core.ac.uk/download/pdf/82747799.pdf
I am currently trying to understand the basics of prolog.
I have a knowledge base like this:
p(a).
p(X) :- p(X).
If I enter the query p(b), the unification with the fact fails and the rule p(X) :- p(X) is used which leads the unification with the fact to fail again. Why is the rule applied over and over again after that? Couldn't prolog return false at this point?
After a certain time I get the message "Time limit exceeded".
I'm not quite sure why prolog uses the rule over and over again, but since it is, I don't understand why I get a different error message as in the following case.
To be clear, I do understand that "p(X) if p(X)" is an unreasonable rule, but I would like to understand what exactly happens there.
If I have a knowledge base like this:
p(X) :- p(X).
p(a).
There is no chance to come to a result even with p(a) because the fact is below the rule and the rule is called over and over again. For this variant I receive a different error message almost instantly "ERROR: Out of local stack" which is comprehensible.
Now my question - what is the difference between those cases?
Why am I receiving different error messages and why is prolog not returning false after the first application of the rule in the above case? My idea would be that in the above case the procedure is kind of restarted each time the rule gets called and in the below case the same procedure calls the rule over and over again. I would be grateful if somebody could elaborate this.
Update: If I query p(a). to the 2nd KB as said I receive "Out of local stack", but if I query p(b). to the same KB I get "Time limit exceeded". This is even more confusing to me, shouldn't the constant be irrelevant for the infinite loop?
Let us first consider the following program fragment that both examples have in common:
p(X) :- p(X).
As you correctly point out, it is obvious that no particular solutions are described by this fragment in isolation. Declaratively, we can read it as: "p(X) holds if p(X) holds". OK, so we cannot deduce any concrete solution from only this clause.
This explains why p(b) cannot hold if only this fragment is considered. Additionally, p(a) does not imply p(b) either, so no matter where you put the fact p(a), you will never derive p(b) from these two clauses.
Procedurally, Prolog still attempts to find cases where p(X) holds. So, if you post ?- p(X). as a query, Prolog will try to find a resolution refutation, disregarding what it has "already tried". For this reason, it will try to prove p(X) over and over. Prolog's default resolution strategy, SLDNF resolution, keeps no memory of which branches have already been tried, and also for this reason can be implemented very efficiently, with little overhead compared to other programming languages.
The difference between an infinite deduction attempt and an out of local stack error error can only be understood procedurally, by taking into account how Prolog executes these fragments.
Prolog systems typically apply an optimization that is called tail call optimization. This is applicable if no more choice-points remain, and means that it can discard (or reuse) existing stack frames.
The key difference between your two examples is obviously where you add the fact: Either before or after the recursive clause.
In your case, if the recursive clause comes last, then no more choice-points remain at the time the goal p(X) is invoked. For this reason, an existing stack frame can be reused or discarded.
On the other hand, if you write the recursive clause first, and then query ?- q(X). (or ?- q(a).), then both clauses are applicable, and Prolog remembers this by creating a choice-point. When the recursive goal is invoked, the choice-point still exists, and therefore the stack frames pile up until they exceed the available limits.
If you query ?- p(b)., then argument indexing detects that p(a) is not applicable, and again only the recursive clause applies, independent of whether you write it before or after the fact. This explains the difference between querying p(X) (or p(a)) and p(b) (or other queries). Note that Prolog implementations differ regarding the strength of their indexing mechanisms. In any case, you should expect your Prolog system to index at least on the outermost functor and arity of the first argument. If necessary, more complex indexing schemes can be constructed manually on top of this mechanism. Modern Prolog systems provide JIT indexing, deep indexing and other mechanisms, and so they often automatically detect the exact subset of clauses that are applicable.
Note that there is a special form of resolution called SLG resolution, which you can use to improve termination properties of your programs in such cases. For example, in SWI-Prolog, you can enable SLG resolution by adding the following directives before your program:
:- use_module(library(tabling)).
:- table p/1.
With these directives, we obtain:
?- p(X).
X = a.
?- p(b).
false.
This coincides with the declarative semantics you expect from your definitions. Several other Prolog systems provide similar facilities.
It should be easy to grasp the concept of infinite loop by studying how standard repeat/0 is implemented:
repeat.
repeat :- repeat.
This creates an infinite number of choice points. First clause, repeat., simply allows for a one time execution. The second clause, repeat :- repeat. makes it infinitely deep recursion.
Adding any number of parameters:
repeat(_, _, ..., _).
repeat(Param1, Param2, ..., ParamN) :- repeat(Param1, Param2, ..., ParamN).
You may have bodies added to these clauses and have parameters of the first class having meaningful names depending on what you are trying to archive. If bodies won't contain cuts, direct or inherited from predicates used, this will be an infinite loop too just as repeat/0.
I have a somewhat complex predicate with four arguments that need to work when both the first and last arguments are ground/not ground, not ground/ground or ground/ground, and the second and third arguments are ground.
i.e. predicate(A,B,C,D).
I can't provide my actual code since it is part of an assignment.
I have it mostly working, but am receiving instantiation errors when A is not ground, but D is. However, I have singled out a line of code that is causing issues. When I change the goal order of the predicate, it works when D is ground and A is not, but in doing so, it no longer works for when A is ground and D is not. I'm not sure there is a way around this.
Is there a way to use both lines of code so that if the A is ground for instance it will use the first line, but if A is not ground, it will use the second, and ignore the first? And vice versa.
You can do that, but, almost invariably, you will break the declarative semantics of your programs if you do that.
Consider a simple example to see how such a non-monotonic and extra-logical predicate already breaks basic assumptions and typical declarative properties of well-known predicates, like commutativity of conjunction:
?- ground(X), X = a.
false.
But, if we simply exchange the goals by commutativity of conjunction, we get a different answer:
?- X = a, ground(X).
X = a.
For this reason, such meta-logical predicates are best avoided, especially if you are just beginning to learn the language.
Instead, better stay in the pure and monotonic subset of Prolog. Use constraints like dif/2 and CLP(FD) to make your programs usable in all directions, increasing generality and ease of understanding.
See logical-purity, prolog-dif and clpfd for more information.
Common Lisp allows exception handling through conditions and restarts. In rough terms, when a function throws an exception, the "catcher" can decide how/whether the "thrower" should proceed. Does Prolog offer a similar system? If not, could one be built on top of existing predicates for walking and examining the call stack?
The ISO/IEC standard of Prolog provides only a very rudimentary exception and error handling mechanism which is - more or less - comparable to what Java offers and far away from Common Lisp's rich mechanism, but there are still some points worth noting. In particular, beside the actual signalling and handling mechanism, many systems provide a mechanism similar to unwind-protect. That is, a way to ensure that a goal will be executed, even in the presence of otherwise unhandled signals.
ISO throw/1, catch/3
An exception is raised/thrown with throw(Term). First a copy of Term is created with copy_term/2 lets call it Termcopy and then this new copy is used to search for a corresponding catch(Goal, Pattern, Handler) whose second argument unifies with Termcopy. When Handler is executed, all unifications caused by Goal are undone. So there is no way for the Handler to access the substitutions present when throw/1 is executed. And there is no way to continue at the place where the throw/1 was executed.
Errors of built-in predicates are signaled by executing throw(error(Error_term, Imp_def)) where Error_term corresponds to one of ISO's error classes and Imp_def may provide implementation defined extra information (like source file, line number etc).
There are many cases where handling an error locally would be of great benefit but it is deemed by many implementors to be too complex to implement.
The additional effort to make a Prolog processor handle each and every error locally is quite considerable and is much larger than in Common Lisp or other programming languages. This is due to the very nature of unification in Prolog. The local handling of an error would require to undo unifications performed during the execution of the built-in: An implementor has thus two possibilities to implement this:
create a "choice point" at the time of invoking a built-in predicate, this would incur a lot of additional overhead, both for creating this choice point and for "trailing" subsequent bindings
go through each and every built-in predicate manually and decide on a case-by-case basis how to handle errors — while this is the most efficient in terms of runtime overheads, this is also the most costly and error-prone approach
Similar complexities are caused by exploiting WAM registers within built-ins. Again, one has the choice between a slow system or one with significant implementation overhead.
exception_handler/3
Many systems, however, provide internally better mechanisms, but few offer them consistently to the programmer. IF/Prolog provides exception_handler/3 which has the same arguments as catch/3 but handles the error or exception locally:
[user] ?- catch((arg(a,f(1),_); Z=ok), error(type_error(_,_),_), fail).
no
[user] ?- exception_handler((arg(a,f(1),_); Z=ok), error(type_error(_,_),_), fail).
Z = ok
yes
setup_call_cleanup/3
This built-in offered by quite a few systems. It is very similar to unwind-protect but requires some additional complexity due to Prolog's backtracking mechanism. See its current definition.
All these mechanisms need to be provided by the system implementor, they cannot be built on top of ISO Prolog.
You can use hypothetical reasoning, to implement what you want. Lets say
a Prolog system that allows hypothetical reasoning supports the following
inference rule:
G, A |- B
----------- (Right ->)
G |- A -> B
There are some Prolog systems that support this, for example lambda Prolog.
You can now use hypothetical reasoning to implement for example restart/2
and signal_condition/3. Assume the hypothetical reasoning is done via
(-:)/2, we could then have:
restart(Goal,Handler) :-
(handler(Handler) -: Goal).
signal_condition(Condition, Restart) :-
handler(Handler), call(Handler,Condition,Restart), !.
signal_condition(Condition, _) :-
throw(Condition).
The solution will not for nothing traverse the whole stack trace, but
directly query for a handler. But it begs the question whether I need
a special Prolog or whether I can do hypothetical reasoning by myself.
As a first approximation the (-:)/2 can be implemented as follows:
(Clause -: Goal) :- assume(Clause), Goal, retire(Clause).
assume(Clause) :- asserta(Clause).
assume(Clause) :- once(retact(Clause)).
retire(Clause) :- once(retract(Clause)).
retire(Clause) :- asserta(Clause).
But the above will not work correctly if Goal issues a cut or an
exception. So a better solution available for example in Jekejeke
Minlog 0.6 would be:
(Clause -: Goal) :- compile(Clause, Ref), assume_ref(Ref), Goal, retire_ref(Ref).
assume_ref(Ref) :- sys_atomic((recorda(Ref), sys_unbind(erase(Ref)))).
retire_ref(Ref) :- sys_atomic((erase(Ref), sys_unbind(recorda(Ref)))).
The sys_unbind/1 predicate schedules an undo goal on the binding list. It
corresponds to the undo/1 from SICStus. The binding list is resilient to
cuts. The sys_atomic/1 assures that the undo goal is always schedule, even
if an external signal happens during the execution, such as for example
an end-user issued abort. It corresponds to how for example the first argument
of setup_call_cleanup/3 is handled.
The advantage of using clause references here is that the clause is only compiled
once, even if backtracking happens between the goal and the continuation after
the (-:)/2. But otherwise the solution is most likely slower than putting
a goal on the stack trace via calling it. But one could imagine further refinements
of a Prolog system, for example (-:)/2 as a primitive and appropriate compile
techniques.
ISO prolog defines these predicates:
throw/1 which throws an exception. The argument is the exception to be thrown (any term)
catch/3 which executes a goal and catches certain exceptions in which case it executes the exception handler. First argument is the goal to be called, second argument is the exception template (if exception thrown by throw/1 unifies with this template the handler goal is executed), and the third argument is the handler goal is executed.
Example usage:
test:-
catch(my_goal, my_exception(Args), (write(exception(Args)), nl)).
my_goal:-
throw(my_exception(test)).
Regarding your note "If not, could one be built on top of existing predicates for walking and examining the call stack?" i don't think there is a general way to do this. Maybe look at the documentation of the prolog system you are using to see if there is some way to walk through the stack.
As false mentioned in his answer, ISO Prolog doesn't allow this. However, some experimentation shows that SWI-Prolog has provided a mechanism on which conditions and restarts could be built. A very rough proof of concept follows.
The "catcher" invokes restart/2 to call a goal and supplies a predicate for choosing among available restarts should a condition be raised. The "thrower" invokes signal_condition/2. The first argument is the condition to raise. The second argument will be bound to a chosen restart. If no restart is chosen, the condition becomes an exception.
restart(Goal, _) :- % signal condition finds this predicate in the call stack
call(Goal).
signal_condition(Condition, Restart) :-
prolog_current_frame(Frame),
prolog_frame_attribute(Frame, parent, Parent),
signal_handler(Parent, Condition, Restart).
signal_handler(Frame, Condition, Restart) :-
( prolog_frame_attribute(Frame, goal, restart(_, Handler)),
call(Handler, Condition, Restart)
-> true
; prolog_frame_attribute(Frame, parent, Parent)
-> signal_handler(Parent, Condition, Restart)
; throw(Condition) % reached top of call stack
).