Here is a simple CLPFD relation:
1 #= 81 mod X.
This returns:
X in inf.. -1\/1..sup,
81 mod X#=1.
Unless my math is completely incorrect, shouldn't the domain of X be -80.. -1\/1..80?
First things first:
Is there a mistake in the constraint solver?
No(t necessarily), because all admissible solutions are still contained in the domain that is deduced, and no wrong solutions are reported. The domain that you deduced is a proper subset of what the solver reports.
Does the constraint solver propagate as well as it could in this case?
No, obviously not, as your stronger bounds already show. In fact, the admissible domain is even smaller than what you deduced: X in 2..80 would also be valid, because X can definitely not be negative, and it also cannot be equal to 1.
Exercise: Is X in 2..80 the smallest (with respect to set inclusion) domain that can be deduced in this case? Why (not)? And in what, if any, sense is it minimal?
So, what is going on here?
The explanation is rather easy: Implementing (mod)/2, (rem)/2, (div)/2 and—to somewhat lesser extent—even (*)/2 in such a way that they propagate as much as possible in all cases is brutally difficult to get right, and was clearly not done in this case.
Do we have to live with this shortcoming?
No, because we can improve the constraint solver to handle such cases too! That is, we can add logic so that it propagates more strongly! Doing this elegantly and correctly is an unsolved problem in general, and subject of active research. See for example Finite Domain Constraint Solver Learning, the references included therein, and several other papers. Of course, the dream would be to somehow derive the propagation directly from the specification of these operations, which is at least decades away. Until then, such issues are found and improved rather ad hoc.
Disclaimer: I don't know what I am talking about, I just wanted to see for myself what happens, and thought it might be useful to share.
With SWI-Prolog and library(clpfd):
?- use_module(library(clpfd)).
true.
?- 1 #= 81 rem X.
X in inf.. -1\/1..sup,
81 rem X#=1.
?- 1 #= 81 mod X.
X in inf.. -1\/1..sup,
81 mod X#=1.
?- 1 #= 81 - X * (81 // X ).
X in -80.. -1\/1..80,
X*_G1053#=80,
81//X#=_G1053,
_G1053 in -80.. -1\/1..80.
Strange, isn't the expression in the last example supposed to be how modulo division is defined?
If you take Gnu-Prolog (mod is not directly supported):
| ?- 1 #= 81 rem X.
X = _#4(2..80)
yes
| ?- 1 #= 81 - X * (81 // X).
no
Hmm. What if you just reorder the expression:
| ?- 1 #= 81 - (81 // X) * X.
X = _#4(2..80)
yes
Conclusion: yes, it seems that writing a good CLP(FD) library is indeed not easy. One could even be left with the impression that these libraries exhibit emergent behaviors that the authors are not always fully aware of.
Related
In my Prolog course the past semester, I fell a little behind around the time CLP was introduced. Now I'm trying to catch up, and have tried my hand at a past exam that the professor supplied to all students.
In particular, there was this question:
What is the domain of the decision variable Z in CLP(FD) after the following query:
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y - X.
It seems to me that the answer should be
Z in 1..99
but when I ran it in my SWI-Prolog installation to double-check, I got
Z in -5.. -1\/1..99
which seems to be based on a naive comparison of the maximum and minimum values of X & Y, without regard for the constraint linking them (Y #> X).
I realize that concessions to feasibility have to be made here and the domains returned will sometimes be less restrictive than they could be, but I'm surprised to see it fail on such a simple example.
My questions
I assume that this has to do with how CLP chooses to propagate (or not to propagate) various constraints internally, but I don't understand how it does that - it's all something of a black box for me. How, exactly (or failing that, approximately), does CLP propagate its constraints?
Is there any way to make CLP(FD) apply the constraint appropriately, perhaps by reordering? I've already tried tacking on an extra Y #> X at the end, but that didn't change any of the variables domains.
It seems to me that the answer should be
Z in 1..99
How can you be so sure that you are right? This is one of the nice properties of constraints: You can verify this most easily:
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y -X.
X in 1..7,
Z+X#=Y,
X#=<Y+ -1,
Z in -5.. -1\/1..99,
Y in 2..100.
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y -X, Z #< 0.
false.
OK, now I believe what you said.
So you have discovered here an inconsistency which is present also in SICStus' native library(clpfd) as well as library(clpz). First please note that the answer given was not incorrect! It said: Yes, there are solutions provided X in 1..7, Z+X#=Y, X#=<Y+ -1, Z in -5.. -1\/1..99, Y in 2..100. is true. Helas, this is not true.
So that answer is a bit like the legalese in many insurance contracts where they say, yes we will pay, provided all that tiny unreadable print holds, but in reality you could replace that wall of microtext by a big fat false.
In general, such inconsistencies are inevitable since CLP(FD)/CLP(Z) as defined in above systems permits to formulate undecidable problems. Thus, no matter how evolved your constraint solver is, we have the guarantee that there will be always cases that we cannot solve. That's a scientific, mathematical law, much more reliable than empirical laws like gravity or that speed limit.
The inconsistency here is effectively an engineering tradeoff. As long as nobody complains and doesn't have a convincing use case, the developers of such systems will not see a reason to improve. After all, such an improvement might slow down existing use cases.
How, exactly (or failing that, approximately), does CLP propagate its constraints?
Actually, for any problem of realistic size, nobody knows. But this is not necessary either. In the case of CLP(FD), the fundamental element are the domains attached to the logical variables. You see them as (in)/2 goals like Z in -5.. -1\/1..99. Connected between them are the actual constraints. In your case Y #> X and Z #= Y-X. These constraints now only see the domains of the variables and try to maintain consistency between them. As an even coarser approximation, the domains are seen as intervals thus Z in -5 .. 99 instead of above. What (most of them) do not see are the other constraints. In this case, there is no direct connection between Y #> X and Z #= Y-X. And thus the inconsistency. Such limited consistency checks are much easier to implement and also quite fast and often outperform more complete algorithms. With the discovery of better algorithms things evolve. A nice example is all_distinct/1 which maintains consistency between all variables using Regin's algorithm, whereas all_different/1 only maintains consistency between each pair of variables. But in any case: these things evolve and it is a bit of a surprise that this is an exam question.
Is there any way to make CLP(FD) apply the constraint appropriately ...?
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y -X, clpfd:contracting([X,Y,Z]).
X in 1..7,
Z+X#=Y,
X#=<Y+ -1,
Z in 1..99,
Y in 2..100.
But most will ignore this issue and just add labeling([],[X,Y])
What is the domain of Z?
That is an ambiguous question. Give both as an answer.
I wrote a quick predicate in Prolog trying out CLP(FD) and its ability to solve systems of equations.
problem(A, B) :-
A-B #= 320,
A #= 21*B.
When I call it in SWI, I get:
?- problem(A,B).
320+B#=A,
21*B#=A.
Whereas in GNU, I get the correct answer of:
| ?- problem(A,B).
A = 336
B = 16
What's going on here? Ideally I'd like to get the correct results in SWI as it's a much more robust environment.
This is a good observation.
At first glance, it will no doubt appear to be a shortcoming of SWI that it fails to propagate as strongly as GNU Prolog.
However, there are also other factors at play here.
The core issue
To start, please try the following query in GNU Prolog:
| ?- X #= X.
Declaratively, the query can be read as: X is an integer. The reasons are:
(#=)/2 only holds for integers
X #= X does not constrain the domain of the integer X in any way.
However, at least on my machine, GNU Prolog answers with:
X = _#0(0..268435455)
So, in fact, the domain of the integer X has become finite even though we have not restricted it in any way!
For comparison, we get for example in SICStus Prolog:
?- X #= X.
X in inf..sup.
This shows that the domain of the integer X has not been restricted in any way.
Replicating the result with CLP(Z)
Let us level the playing field. We can simulate the above situation with SWI-Prolog by artificially restricting the variables' domains to, say, the finite interval 0..264:
?- problem(A, B),
Upper #= 2^64,
[A,B] ins 0..Upper.
In response, we now get with SWI-Prolog:
A = 336,
B = 16,
Upper = 18446744073709551616.
So, restricting the domain to a finite subset of integers has allowed us to replicate the result we know from GNU Prolog also with the CLP(FD) solver of SWI-Prolog or its successor, CLP(Z).
The reason for this
The ambition of CLP(Z) is to completely replace low-level arithmetic predicates in user programs by high-level declarative alternatives that can be used as true relations and of course also as drop-in replacements. For this reason, CLP(Z) supports unbounded integers, which can grow as large as your computer's memory allows. In CLP(Z), the default domain of all integer variables is the set of all integers. This means that certain propagations that are applied for bounded domains are not performed as long as one of the domains is infinite.
For example:
?- X #> Y, Y #> X.
X#=<Y+ -1,
Y#=<X+ -1.
This is a conditional answer: The original query is satisfiable iff the so-called residual constraints are satisfiable.
In contrast, we get with finite domains:
?- X #> Y, Y #> X, [X,Y] ins -5000..2000.
false.
As long as all domains are finite, we expect roughly the same propagation strength from the involved systems.
An inherent limitation
Solving equations over integers is undecidable in general. So, for CLP(Z), we know that there is no decision algorithm that always yields correct results.
For this reason, you sometimes get residual constraints instead of an unconditional answer. Over finite sets of integers, equations are of course decidable: If all domains are finite and you do not get a concrete solution as answer, use one of the enumeration predicates to exhaustively search for solutions.
In systems that can reason over infinite sets of integers, you will sooner or later, and necessarily, encounter such phenomena.
I am trying my hand at relational prolog. Part of my program needs to deal with bitmasks. It however seems that prolog code handles bit makes, such as to set a bit or clear a bit doesn't work relationally -- i.e. it only works in setting a bit, but not in the other direction, identifying what bit is set.
For example:
setbit(X, N, V) :-
N1 #= 1<< N,
V #= X \/ N1.
this code only work in one direction, where X and N are given and V is calculated. If one provides V and N, then X is not derived, but rather its left as an uninstantiated expression.
Does this mean that calculating with bit maps and masks is out of scope of relational prolog.
?- setbit(0,1,X).
X = 2.
?- setbit(X, 1, 2).
2#=X\/2.
the latter doesn't bind X to 0.
thank you,
Daniel
Edit: based on the comments below, the following code works very well:
setbit(X, N, V) :-
X in 0..1,
label([X]),
N1 #= 1<< N,
V #= X \/ N1.
clearbit(X, N, V) :-
X in 0..1,
label([X]),
current_prolog_flag(max_tagged_integer, MTI),
N1 #= MTI /\ \(1<<N),
% N1 #= 0xffffffffffffff /\ \(1<<N),
V #= X /\ N1
Note, the current_prolog_flag -- it retrieves the maximum integer fitting into one word on the current machine architecture -- on 64 bit its 54 bits, the rest of the bits are used for housekeeping.
From a given solution, you cannot conclude that it is the only solution. That is, from
?- setbit(0,1,X).
X = 2.
you cannot conclude that
?- setbit(X, 1, 2).
has X = 0 as the only solution. In fact, there is another solution, namely
?- setbit(2, 1, 2).
true.
Ideally, all constraints would maintain domain-consistency. In this ideal world we would have:
?- setbit(X, 1, 2).
X in 0\/2, % idealiter
2#=X\/2.
instead of
?- setbit(X, 1, 2).
2#=X\/2. % realiter
But first of all, let us realize that both answers are correct! The second answer says precisely the same as the ideal one. However, finding a concrete solution may be more costly in the second case. In particular, since the following query has an answer:
?- setbit(X, 1, 2), X #> 2.
X in 3..sup,
2#=X\/2. % inconsistency
This answer reads like the notice of winning a lottery ticket you never heard of:
Yes, congratulation! There is a solution, provided all this fine print, this X in 3..sup, 2#=/2 has a solution, otherwise it does not have any solution. So don't complain, we told you so.
That is, an answer may very well contain exactly zero solutions. To be absolutely sure about a solution, you have to eliminate all constraints. The easiest way to do so, is using labeling/2. However, labeling/2 is defined only for finite domains (that's where the FD in CLPFD stems from). But in this case X is not constrained to a finite domain - would that be the case, we would have an extra constraint like X in 0..2.
The degree of consistency in clpfd-systems heavily depends on the actual use cases. After all, full consistency is undecidable. So there will always be cases where we would expect a more precise outcome. It is rather a question of tradeoffs for both run- and development time. Should you have convincing use cases contact the system developer.
In this particular case, you are probably better off using modulo arithmetics and addition.
I am not great at clpfd, but I think the problem here is that you haven't given X a finite domain or asked for its values to be enumerated. This works:
?- setbit(X, 1, 2), X in 0..1, label([X]).
X = 0 ;
false.
The second expression there, X in 0..1 says you want X to be zero or one, and the third says, "give me the values X can obtain."
In SWI-Prolog, the following query gives this result:
?- X mod 2 #= 0, X mod 2 #= 0.
X mod 2#=0,
X mod 2#=0.
While correct, there is obviously no need for the second constraint
Similarly:
?- dif(X,0), dif(X,0).
dif(X, 0),
dif(X, 0).
Is there no way to avoid such duplicate constraints? (Obviously the most correct way would be to not write code that leads to that situation, but it is not always that easy).
You can either avoid posting redundant constraints, or remove them with a setof/3-like construct. Both are very implementation specific. The best interface for such purpose is offered by SICStus. Other implementations like YAP or SWI more or less copied that interface, by leaving out some essential parts. A recent attempt to overcome SWI's deficiencies was rejected.
Avoid posting constraints
In SICStus, you can use frozen/2 for this purpose:
| ?- dif(X,0), frozen(X,Goal).
Goal = prolog:dif(X,0),
prolog:dif(X,0) ? ;
no
| ?- X mod 2#=0, frozen(X, Goal).
Goal = clpfd:(X in inf..sup,X mod 2#=0),
X mod 2#=0,
X in inf..sup ? ;
no
Otherwise, copy_term/3 might be good enough, provided the constraints are not too much interconnected with each other.
Eliminate redundant constraints
Here, a setof-like construct together with call_residue_vars/1 and copy_term/3 is probably the best approach. Again, the original implementation is in SICStus....
For dif/2 alone, entailment can be tested without resorting to any internals:
difp(X,Y) :-
( X \= Y -> true
; dif(X, Y)
).
Few constraint programming systems implement contraction also related to factoring in resolution theorem proving, since CLP labeling is not SMT. Contraction is a structural rule, and it reads as follows, assuming constraints are stored before the (|-)/2 in negated form.
G, A, A |- B
------------ (Left-Contraction)
G, A |- B
We might also extend it to the case where the two A's are derivably equivalent. Mostlikely this is not implemented, since it is costly. For example Jekejeke Minlog already implements contraction for CLP(FD), i.e. finite domains. We find for queries similar to the first query from the OP:
?- use_module(library(finite/clpfd)).
% 19 consults and 0 unloads in 829 ms.
Yes
?- Y+X*3 #= 2, 2-Y #= 3*X.
3*X #= -Y+2
?- X #< Y, Y-X #> 0.
X #=< Y-1
Basically we normalize to A1*X1+..+An*Xn #= B respectively A1*X1+..+An*Xn #=< B where gcd(A1,..,An)=1 and X1,..,Xn are lexically ordered, and then we check whether there is already the same constraint in the constraint store. But for CLP(H), i.e. Herbrand domain terms, we have not yet implemented contraction. We are still deliberating an efficient algorithm:
?- use_module(library(term/herbrand)).
% 2 consults and 0 unloads in 35 ms.
Yes
?- neq(X,0), neq(X,0).
neq(X, 0),
neq(X, 0)
Contraction for dif/2 would mean to implement a kind of (==)/2 via the instantiation defined in the dif/2 constraint. i.e. we would need to apply a recursive test following the pairing of variables and terms defined in the dif/2 constraint against all other dif/2 constraints already in the constraint store. Testing subsumption instead of contraction would also make more sense.
It probably is only feasible to implement contraction or subsumption for dif/2 with the help of some appropriate indexing technique. In Jekejeke Minlog for example for CLP(FD) we index on X1, but we did not yet realize some indexing for CLP(H). What we first might need to figure out is a normal form for the dif/2 constraints, see also this problem here.
I'm new to prolog and every single bit of code I write turns into an infinite loop.
I'm specifically trying to see if X is in the range from 0 to K - 1.
range(X,X).
range(X,K) :- K0 is K - 1, range(X,K0).
My idea behind the code is that I decrement K until K0 equals X, then the base case will kick in. I'm getting an infinite loop though, so something with the way I'm thinking is wrong.
Welcome to the wondrous world of Prolog! It seems you tried to leapfrog several steps when learning Prolog, and (not very surprisingly) failed.
Ideally, you take a book like Art of Prolog and start with family relations. Then extend towards natural numbers using successor-arithmetics, and only then go to (is)/2. Today, (that is, since about 1996) there is even a better way than using (is)/2 which is library(clpfd) as found in SICStus or SWI.
So let's see how your program would have been, using successor-arithmetics. Maybe less_than_or_equal/2 would be a better name:
less_than_or_equal(N,N).
less_than_or_equal(N,s(M)) :-
less_than_or_equal(N,M).
?- less_than_or_equal(N,s(s(0))).
N = s(s(0))
; N = s(0)
; N = 0.
It works right out of the box! No looping whatsoever. So what went wrong?
Successor arithmetics relies on the natural numbers. But you used integers which contain also these negative numbers. With negative numbers, numbers are no longer well ordered (well founded, or Noetherian), and you directly experienced that consequence. So stick with the natural numbers! They are all natural, and do not contain any artificial negative ingredients. Whoever said "God made the integers, all else is the work of man." must have been wrong.
But now back to your program. Why does it not terminate? After all, you found an answer, so it is not completely wrong. Is it not? You tried to reapply the notions of control flow you learned in command oriented languages to Prolog. Well, Prolog has two relatively independent control flows, and many more surprising things like real variables (TM) that appear at runtime that have no direct counterpart in Java or C#. So this mapping did not work. I got a little bit suspicious when you called the fact a "base case". You probably meant that it is a "termination condition". But it is not.
So how can we easily understand termination in Prolog? The best is to use a failure-slice. The idea is that we will try to make your program as small as possible by inserting false goals into your program. At any place. Certain of the resulting programs will still not terminate. And those are most interesting, since they are a reason for non-termination of the original program! They are immediately, causally connected to your problem. And they are much better for they are shorter. Which means less time to read. Here are some attempts, I will strike through the parts that are no longer relevant.
range(X,X).
range(X,K) :-
K0 is K - 1, false,
range(X,K0).
Nah, above doesn't loop, so it cannot tell us anything. Let's try again:
range(X,X) :- false.
range(X,K) :-
K0 is K - 1,
range(X,K0), false.
This one loops for range(X,1) already. In fact, it is the minimal failure slice. With a bit of experience you will learn to see those with no effort.
We have to change something in the visible part to make this terminate. For example, you might add K > 0 or do what #Shevliaskovic suggested.
I believe the simplest way to do this is:
range(X,X).
range(X,K) :- X>0, X<K-1.
and here are my results:
6 ?- range(4,4).
true .
7 ?- range(5,8).
true.
8 ?- range(5,4).
false.
The simple way, as has been pointed out, if you just want to validate that X lies within a specified domain would be to just check the condition:
range(X,K) :- X >= 0 , X < K .
Otherwise, if you want your range/2 to be generative, would be to use the built-in between/3:
range(X,K) :- integer(K) , K1 is K-1 , between(0,K1,X).
If your prolog doesn't have a between/3, it's a pretty simple implementation:
%
% the classic `between/3` wants the inclusive lower and upper bounds
% to be bound. So we'll make the test once and use a helper predicate.
%
between(Lo,Hi,N) :-
integer(Lo),
integer(Hi),
_between(Lo,Hi,N)
.
_between(Lo,Hi,Lo) :- % unify the lower bound with the result
Lo =< Hi % - if we haven't yet exceeded the inclusive upper bound.
. %
_between(Lo,Hi,N) :- % otherwise...
Lo < Hi , % - if the lower bound is less than the inclusive upper bound
L1 is Lo+1 , % - increment the lower bound
_between(L1,Hi,N) % - and recurse down.
. %