In my Prolog course the past semester, I fell a little behind around the time CLP was introduced. Now I'm trying to catch up, and have tried my hand at a past exam that the professor supplied to all students.
In particular, there was this question:
What is the domain of the decision variable Z in CLP(FD) after the following query:
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y - X.
It seems to me that the answer should be
Z in 1..99
but when I ran it in my SWI-Prolog installation to double-check, I got
Z in -5.. -1\/1..99
which seems to be based on a naive comparison of the maximum and minimum values of X & Y, without regard for the constraint linking them (Y #> X).
I realize that concessions to feasibility have to be made here and the domains returned will sometimes be less restrictive than they could be, but I'm surprised to see it fail on such a simple example.
My questions
I assume that this has to do with how CLP chooses to propagate (or not to propagate) various constraints internally, but I don't understand how it does that - it's all something of a black box for me. How, exactly (or failing that, approximately), does CLP propagate its constraints?
Is there any way to make CLP(FD) apply the constraint appropriately, perhaps by reordering? I've already tried tacking on an extra Y #> X at the end, but that didn't change any of the variables domains.
It seems to me that the answer should be
Z in 1..99
How can you be so sure that you are right? This is one of the nice properties of constraints: You can verify this most easily:
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y -X.
X in 1..7,
Z+X#=Y,
X#=<Y+ -1,
Z in -5.. -1\/1..99,
Y in 2..100.
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y -X, Z #< 0.
false.
OK, now I believe what you said.
So you have discovered here an inconsistency which is present also in SICStus' native library(clpfd) as well as library(clpz). First please note that the answer given was not incorrect! It said: Yes, there are solutions provided X in 1..7, Z+X#=Y, X#=<Y+ -1, Z in -5.. -1\/1..99, Y in 2..100. is true. Helas, this is not true.
So that answer is a bit like the legalese in many insurance contracts where they say, yes we will pay, provided all that tiny unreadable print holds, but in reality you could replace that wall of microtext by a big fat false.
In general, such inconsistencies are inevitable since CLP(FD)/CLP(Z) as defined in above systems permits to formulate undecidable problems. Thus, no matter how evolved your constraint solver is, we have the guarantee that there will be always cases that we cannot solve. That's a scientific, mathematical law, much more reliable than empirical laws like gravity or that speed limit.
The inconsistency here is effectively an engineering tradeoff. As long as nobody complains and doesn't have a convincing use case, the developers of such systems will not see a reason to improve. After all, such an improvement might slow down existing use cases.
How, exactly (or failing that, approximately), does CLP propagate its constraints?
Actually, for any problem of realistic size, nobody knows. But this is not necessary either. In the case of CLP(FD), the fundamental element are the domains attached to the logical variables. You see them as (in)/2 goals like Z in -5.. -1\/1..99. Connected between them are the actual constraints. In your case Y #> X and Z #= Y-X. These constraints now only see the domains of the variables and try to maintain consistency between them. As an even coarser approximation, the domains are seen as intervals thus Z in -5 .. 99 instead of above. What (most of them) do not see are the other constraints. In this case, there is no direct connection between Y #> X and Z #= Y-X. And thus the inconsistency. Such limited consistency checks are much easier to implement and also quite fast and often outperform more complete algorithms. With the discovery of better algorithms things evolve. A nice example is all_distinct/1 which maintains consistency between all variables using Regin's algorithm, whereas all_different/1 only maintains consistency between each pair of variables. But in any case: these things evolve and it is a bit of a surprise that this is an exam question.
Is there any way to make CLP(FD) apply the constraint appropriately ...?
?- X in 1..7, Y in -3..100, Y #> X, Z #\= 0, Z #= Y -X, clpfd:contracting([X,Y,Z]).
X in 1..7,
Z+X#=Y,
X#=<Y+ -1,
Z in 1..99,
Y in 2..100.
But most will ignore this issue and just add labeling([],[X,Y])
What is the domain of Z?
That is an ambiguous question. Give both as an answer.
Related
I wrote a quick predicate in Prolog trying out CLP(FD) and its ability to solve systems of equations.
problem(A, B) :-
A-B #= 320,
A #= 21*B.
When I call it in SWI, I get:
?- problem(A,B).
320+B#=A,
21*B#=A.
Whereas in GNU, I get the correct answer of:
| ?- problem(A,B).
A = 336
B = 16
What's going on here? Ideally I'd like to get the correct results in SWI as it's a much more robust environment.
This is a good observation.
At first glance, it will no doubt appear to be a shortcoming of SWI that it fails to propagate as strongly as GNU Prolog.
However, there are also other factors at play here.
The core issue
To start, please try the following query in GNU Prolog:
| ?- X #= X.
Declaratively, the query can be read as: X is an integer. The reasons are:
(#=)/2 only holds for integers
X #= X does not constrain the domain of the integer X in any way.
However, at least on my machine, GNU Prolog answers with:
X = _#0(0..268435455)
So, in fact, the domain of the integer X has become finite even though we have not restricted it in any way!
For comparison, we get for example in SICStus Prolog:
?- X #= X.
X in inf..sup.
This shows that the domain of the integer X has not been restricted in any way.
Replicating the result with CLP(Z)
Let us level the playing field. We can simulate the above situation with SWI-Prolog by artificially restricting the variables' domains to, say, the finite interval 0..264:
?- problem(A, B),
Upper #= 2^64,
[A,B] ins 0..Upper.
In response, we now get with SWI-Prolog:
A = 336,
B = 16,
Upper = 18446744073709551616.
So, restricting the domain to a finite subset of integers has allowed us to replicate the result we know from GNU Prolog also with the CLP(FD) solver of SWI-Prolog or its successor, CLP(Z).
The reason for this
The ambition of CLP(Z) is to completely replace low-level arithmetic predicates in user programs by high-level declarative alternatives that can be used as true relations and of course also as drop-in replacements. For this reason, CLP(Z) supports unbounded integers, which can grow as large as your computer's memory allows. In CLP(Z), the default domain of all integer variables is the set of all integers. This means that certain propagations that are applied for bounded domains are not performed as long as one of the domains is infinite.
For example:
?- X #> Y, Y #> X.
X#=<Y+ -1,
Y#=<X+ -1.
This is a conditional answer: The original query is satisfiable iff the so-called residual constraints are satisfiable.
In contrast, we get with finite domains:
?- X #> Y, Y #> X, [X,Y] ins -5000..2000.
false.
As long as all domains are finite, we expect roughly the same propagation strength from the involved systems.
An inherent limitation
Solving equations over integers is undecidable in general. So, for CLP(Z), we know that there is no decision algorithm that always yields correct results.
For this reason, you sometimes get residual constraints instead of an unconditional answer. Over finite sets of integers, equations are of course decidable: If all domains are finite and you do not get a concrete solution as answer, use one of the enumeration predicates to exhaustively search for solutions.
In systems that can reason over infinite sets of integers, you will sooner or later, and necessarily, encounter such phenomena.
I am trying my hand at relational prolog. Part of my program needs to deal with bitmasks. It however seems that prolog code handles bit makes, such as to set a bit or clear a bit doesn't work relationally -- i.e. it only works in setting a bit, but not in the other direction, identifying what bit is set.
For example:
setbit(X, N, V) :-
N1 #= 1<< N,
V #= X \/ N1.
this code only work in one direction, where X and N are given and V is calculated. If one provides V and N, then X is not derived, but rather its left as an uninstantiated expression.
Does this mean that calculating with bit maps and masks is out of scope of relational prolog.
?- setbit(0,1,X).
X = 2.
?- setbit(X, 1, 2).
2#=X\/2.
the latter doesn't bind X to 0.
thank you,
Daniel
Edit: based on the comments below, the following code works very well:
setbit(X, N, V) :-
X in 0..1,
label([X]),
N1 #= 1<< N,
V #= X \/ N1.
clearbit(X, N, V) :-
X in 0..1,
label([X]),
current_prolog_flag(max_tagged_integer, MTI),
N1 #= MTI /\ \(1<<N),
% N1 #= 0xffffffffffffff /\ \(1<<N),
V #= X /\ N1
Note, the current_prolog_flag -- it retrieves the maximum integer fitting into one word on the current machine architecture -- on 64 bit its 54 bits, the rest of the bits are used for housekeeping.
From a given solution, you cannot conclude that it is the only solution. That is, from
?- setbit(0,1,X).
X = 2.
you cannot conclude that
?- setbit(X, 1, 2).
has X = 0 as the only solution. In fact, there is another solution, namely
?- setbit(2, 1, 2).
true.
Ideally, all constraints would maintain domain-consistency. In this ideal world we would have:
?- setbit(X, 1, 2).
X in 0\/2, % idealiter
2#=X\/2.
instead of
?- setbit(X, 1, 2).
2#=X\/2. % realiter
But first of all, let us realize that both answers are correct! The second answer says precisely the same as the ideal one. However, finding a concrete solution may be more costly in the second case. In particular, since the following query has an answer:
?- setbit(X, 1, 2), X #> 2.
X in 3..sup,
2#=X\/2. % inconsistency
This answer reads like the notice of winning a lottery ticket you never heard of:
Yes, congratulation! There is a solution, provided all this fine print, this X in 3..sup, 2#=/2 has a solution, otherwise it does not have any solution. So don't complain, we told you so.
That is, an answer may very well contain exactly zero solutions. To be absolutely sure about a solution, you have to eliminate all constraints. The easiest way to do so, is using labeling/2. However, labeling/2 is defined only for finite domains (that's where the FD in CLPFD stems from). But in this case X is not constrained to a finite domain - would that be the case, we would have an extra constraint like X in 0..2.
The degree of consistency in clpfd-systems heavily depends on the actual use cases. After all, full consistency is undecidable. So there will always be cases where we would expect a more precise outcome. It is rather a question of tradeoffs for both run- and development time. Should you have convincing use cases contact the system developer.
In this particular case, you are probably better off using modulo arithmetics and addition.
I am not great at clpfd, but I think the problem here is that you haven't given X a finite domain or asked for its values to be enumerated. This works:
?- setbit(X, 1, 2), X in 0..1, label([X]).
X = 0 ;
false.
The second expression there, X in 0..1 says you want X to be zero or one, and the third says, "give me the values X can obtain."
I am trying to figure out what is bounds propagation in clpfd, but cannot seem to find a good explanation anywhere.
I am revising for Prolog and clpfd and came across this question, but looking at the lecture notes it does not make sense to me. Could someone please explain the actual meaning of bounds propagation and what it is used for.
Here is the question I am referring to:
When the following Prolog program
:- use_module(library(clpfd)).
bounds(X, Y, Z) :-
X in 1..5,
Y in 1..2,
Z in 3..5,
X #= Y + Z.
is queried it gives the answer:
?- bounds(X, Y, Z).
X in 4..5,
Y in 1..2,
Z in 3..4.
Explain how bounds propagation can be applied to infer this answer.
Bounds propagation is a form of propagation that the constraint solver automatically applies for you. The key point, for users, is that they need not understand the algorithm behind it, but can simply rely on the constraint solver to do the work for them. In the result you show, the solver has already applied this form of propagation.
To understand what the constraint solver is doing for you, here is a start:
We know:
Y is at least 1
Z is at least 3
X is the sum of Y and Z.
Hence (exercise: Why?): X is at least 4.
Then, repeat this reasoning for all other variables, for both upper and lower bounds!
Repeat this until no more domain elements can be removed from any of the variables, which is called a fix point of the propagation. When this is done, you have established bounds consistency.
Here is a simple CLPFD relation:
1 #= 81 mod X.
This returns:
X in inf.. -1\/1..sup,
81 mod X#=1.
Unless my math is completely incorrect, shouldn't the domain of X be -80.. -1\/1..80?
First things first:
Is there a mistake in the constraint solver?
No(t necessarily), because all admissible solutions are still contained in the domain that is deduced, and no wrong solutions are reported. The domain that you deduced is a proper subset of what the solver reports.
Does the constraint solver propagate as well as it could in this case?
No, obviously not, as your stronger bounds already show. In fact, the admissible domain is even smaller than what you deduced: X in 2..80 would also be valid, because X can definitely not be negative, and it also cannot be equal to 1.
Exercise: Is X in 2..80 the smallest (with respect to set inclusion) domain that can be deduced in this case? Why (not)? And in what, if any, sense is it minimal?
So, what is going on here?
The explanation is rather easy: Implementing (mod)/2, (rem)/2, (div)/2 and—to somewhat lesser extent—even (*)/2 in such a way that they propagate as much as possible in all cases is brutally difficult to get right, and was clearly not done in this case.
Do we have to live with this shortcoming?
No, because we can improve the constraint solver to handle such cases too! That is, we can add logic so that it propagates more strongly! Doing this elegantly and correctly is an unsolved problem in general, and subject of active research. See for example Finite Domain Constraint Solver Learning, the references included therein, and several other papers. Of course, the dream would be to somehow derive the propagation directly from the specification of these operations, which is at least decades away. Until then, such issues are found and improved rather ad hoc.
Disclaimer: I don't know what I am talking about, I just wanted to see for myself what happens, and thought it might be useful to share.
With SWI-Prolog and library(clpfd):
?- use_module(library(clpfd)).
true.
?- 1 #= 81 rem X.
X in inf.. -1\/1..sup,
81 rem X#=1.
?- 1 #= 81 mod X.
X in inf.. -1\/1..sup,
81 mod X#=1.
?- 1 #= 81 - X * (81 // X ).
X in -80.. -1\/1..80,
X*_G1053#=80,
81//X#=_G1053,
_G1053 in -80.. -1\/1..80.
Strange, isn't the expression in the last example supposed to be how modulo division is defined?
If you take Gnu-Prolog (mod is not directly supported):
| ?- 1 #= 81 rem X.
X = _#4(2..80)
yes
| ?- 1 #= 81 - X * (81 // X).
no
Hmm. What if you just reorder the expression:
| ?- 1 #= 81 - (81 // X) * X.
X = _#4(2..80)
yes
Conclusion: yes, it seems that writing a good CLP(FD) library is indeed not easy. One could even be left with the impression that these libraries exhibit emergent behaviors that the authors are not always fully aware of.
I am trying to figure out what is bounds propagation in clpfd, but cannot seem to find a good explanation anywhere.
I am revising for Prolog and clpfd and came across this question, but looking at the lecture notes it does not make sense to me. Could someone please explain the actual meaning of bounds propagation and what it is used for.
Here is the question I am referring to:
When the following Prolog program:- use_module(library(clpfd)).bounds(X, Y, Z) :-
X in 1..5,
Y in 1..2,
Z in 3..5,
X #= Y + Z.
is queried it gives the answer:
?- bounds(X, Y, Z).
X in 4..5,
Y in 1..2,
Z in 3..4.
Explain how bounds propagation can be applied to infer this answer.
I give you a start:
After all constraints are posted, one thing is immediately clear: X is at least 4. Why? Because Y is at least 1, and Z is at least 3, and X is the sum of Y and Z.
Given this knowledge, go through the posted constraints again, and see if you can adjust any boundaries.
This is what bounds propagation does: going through the boundaries of all variables and seeing if any of them can be adjusted due to the posted constraints. This is repeated until there are no more possible domain reductions.