Probability calculation for bayesian network - probability

I am struggling to understand how Bayesian probabilities are calculated for the following network:
I don't understand how the probability of 0.69 is calculated for the P(C=true|A=T)?
Also, how to calculate the initial probability of 0.19 for P(C=true) when there is no any evidence?

If you are missing a variable in the evidence, you need to sum over all the possible values of that variable (this is called marginalization).
P(C | A, B) = P(A, B, C) / P(A, B)
=>
P(C | A) = P (C, B=T | A) + P(C, B=F | A)
= P(C | B=T, A) * P(B=T | A) + P(C | B=F, A) * P(B=F | A)
And the same way if you want to have just P(C).
See also this answer

Related

A specific push down automaton for a language (PDA)

I'm wondering how could I design a pushdown automaton for this specific language.
I can't solve this..
L2 = { u ∈ {a, b}∗ : 3 ∗ |u|a = 2 ∗ |u|b + 1 }
So the number of 'a's multiplied by 3 is equals to number of 'b's multiplied by 2 and added 1.
The grammar corresponding to that language is something like:
S -> ab | ba |B
B -> abB1 | baB1 | aB1b | bB1a | B1ab | B1ba
B1 -> aabbbB1 | baabbB1 | [...] | aabbb | baabb | [...]
S generates the base case (basically strings with #a = 1 = #b) or B
B generates the base case + B1 (in every permutation)
B1 adds 2 'a' and 3 'b' to the base case (in fact if you keep adding this number of 'a' and 'b' the equation 3#a = 2#b + 1 will always be true!). I didn't finish writing B1, basically you need to add every permutation of 2 'a' and 3 'b'. I think you'll be able to do it on your own :)
When you're finished with the grammar, designing the PDA is simple. More info here.
3|u|a = 2|u|b + 1 <=> 3|u|a - 2|u|b = 1
The easiest way to design a PDA for this is to implement this equation directly.
For any string x, let f(x) = 3|x|a - 2|x|b. Then design a PDA such that, after processing any string x:
The stack depth is always equal to abs( floor( f(x)/3 ) );
The symbol on the top of the stack (if any), reflects the sign of floor( f(x)/3 ). You only need 2 kinds of stack symbols
The current state number = f(x) mod 3. Of course you only need 3 states.
From the state number and the symbol on top of the stack, you can detect when f(x) = 1, and at that condition the PDA accepts x as a string in the language.

Probability of a disjunction on N dependent events in Prolog

Does anybody know where to find a Prolog algorithm for computing the probability of a disjunction for N dependent events? For N = 2 i know that P(E1 OR E2) = P(E1) + P(E2) - P(E1) * P(E2), so one could do:
prob_disjunct(E1, E2, P):- P is E1 + E2 - E1 * E2
But how can this predicate be generalised to N events when the input is a list? Maybe there is a package which does this?
Kinds regards/JCR
The recursive formula from Robert Dodier's answer directly translates to
p_or([], 0).
p_or([P|Ps], Or) :-
p_or(Ps, Or1),
Or is P + Or1*(1-P).
Although this works fine, e.g.
?- p_or([0.5,0.3,0.7,0.1],P).
P = 0.9055
hardcore Prolog programmers can't help noticing that the definition isn't tail-recursive. This would really only be a problem when you are processing very long lists, but since the order of list elements doesn't matter, it is easy to turn things around. This is a standard technique, using an auxiliary predicate and an "accumulator pair" of arguments:
p_or(Ps, Or) :-
p_or(Ps, 0, Or).
p_or([], Or, Or).
p_or([P|Ps], Or0, Or) :-
Or1 is P + Or0*(1-P),
p_or(Ps, Or1, Or). % tail-recursive call
I don't know anything about Prolog, but anyway it's convenient to write the probability of a disjunction of a number of independent items p_m = Pr(S_1 or S_2 or S_3 or ... or S_m) recursively as
p_m = Pr(S_m) + p_{m - 1} (1 - P(S_m))
You can prove this by just peeling off the last item -- look at Pr((S_1 or ... or S_{m - 1}) or S_m) and just write that in terms of the usual formula, writing Pr(A or B) = Pr(A) + Pr(B) - Pr(A) Pr(B) = Pr(B) + Pr(A) (1 - Pr(B)), for A and B independent.
The formula above is item C.3.10 in my dissertation: http://riso.sourceforge.net/docs/dodier-dissertation.pdf It is a simple result, and I suppose it must be an exercise in some textbooks, although I don't remember seeing it.
For any event E I'll write E' for the complementary event (ie E' occurs iff E doesn't).
Then we have:
P(E') = 1 - P(E)
(A union B)' = A' inter B'
A and B are independent iff A' and B' are independent
so for independent E1..En
P( E1 union .. union En ) = 1 - P( E1' inter .. inter En')
= 1 - product{ i<=i<=n | 1 - P(E[i])}

Is there a fast algorithm to determine the godel number of a term of a context free language?

Suppose we have a simple grammar specification. There is a way to enumerate terms of that grammar that guarantees that any finite term will have a finite position, by iterating it diagonally. For example, for the following grammar:
S ::= add
add ::= mul | add + mul
mul ::= term | mul * term
term ::= number | ( S )
number ::= digit | digit number
digit ::= 0 | 1 | ... | 9
You can enumerate terms like that:
0
1
0+0
0*0
0+1
(0)
1+0
0*1
0+0*0
00
... etc
My question is: is there a way to do the opposite? That is, to take a valid term of that grammar, say, 0+0*0, and find its position on such enumeration - in that case, 9?
For this specific problem, we can cook up something fairly simple, if we allow ourselves to choose a different enumeration ordering. The idea is basically the one in Every Bit Counts, which I also mentioned in the comments. First, some preliminaries: some imports/extensions, a data type representing the grammar, and a pretty-printer. For the sake of simplicity, my digits only go up to 2 (big enough to not be binary any more, but small enough not to wear out my fingers and your eyes).
{-# LANGUAGE TypeSynonymInstances #-}
import Control.Applicative
import Data.Universe.Helpers
type S = Add
data Add = Mul Mul | Add :+ Mul deriving (Eq, Ord, Show, Read)
data Mul = Term Term | Mul :* Term deriving (Eq, Ord, Show, Read)
data Term = Number Number | Parentheses S deriving (Eq, Ord, Show, Read)
data Number = Digit Digit | Digit ::: Number deriving (Eq, Ord, Show, Read)
data Digit = D0 | D1 | D2 deriving (Eq, Ord, Show, Read, Bounded, Enum)
class PP a where pp :: a -> String
instance PP Add where
pp (Mul m) = pp m
pp (a :+ m) = pp a ++ "+" ++ pp m
instance PP Mul where
pp (Term t) = pp t
pp (m :* t) = pp m ++ "*" ++ pp t
instance PP Term where
pp (Number n) = pp n
pp (Parentheses s) = "(" ++ pp s ++ ")"
instance PP Number where
pp (Digit d) = pp d
pp (d ::: n) = pp d ++ pp n
instance PP Digit where pp = show . fromEnum
Now let's define the enumeration order. We'll use two basic combinators, +++ for interleaving two lists (mnemonic: the middle character is a sum, so we're taking elements from either the first argument or the second) and +*+ for the diagonalization (mnemonic: the middle character is a product, so we're taking elements from both the first and second arguments). More information on these in the universe documentation. One invariant we'll maintain is that our lists -- with the exception of digits -- are always infinite. This will be important later.
ss = adds
adds = (Mul <$> muls ) +++ (uncurry (:+) <$> adds +*+ muls)
muls = (Term <$> terms ) +++ (uncurry (:*) <$> muls +*+ terms)
terms = (Number <$> numbers) +++ (Parentheses <$> ss)
numbers = (Digit <$> digits) ++ interleave [[d ::: n | n <- numbers] | d <- digits]
digits = [D0, D1, D2]
Let's see a few terms:
*Main> mapM_ (putStrLn . pp) (take 15 ss)
0
0+0
0*0
0+0*0
(0)
0+0+0
0*(0)
0+(0)
1
0+0+0*0
0*0*0
0*0+0
(0+0)
0+0*(0)
0*1
Okay, now let's get to the good bit. Let's assume we have two infinite lists a and b. There's two things to notice. First, in a +++ b, all the even indices come from a, and all the odd indices come from b. So we can look at the last bit of an index to see which list to look in, and the remaining bits to pick an index in that list. Second, in a +*+ b, we can use the standard bijection between pairs of numbers and single numbers to translate between indices in the big list and pairs of indices in the a and b lists. Nice! Let's get to it. We'll define a class for Godel-able things that can be translated back and forth between numbers -- indices into the infinite list of inhabitants. Later we'll check that this translation matches the enumeration we defined above.
type Nat = Integer -- bear with me here
class Godel a where
to :: a -> Nat
from :: Nat -> a
instance Godel Nat where to = id; from = id
instance (Godel a, Godel b) => Godel (a, b) where
to (m_, n_) = (m + n) * (m + n + 1) `quot` 2 + m where
m = to m_
n = to n_
from p = (from m, from n) where
isqrt = floor . sqrt . fromIntegral
base = (isqrt (1 + 8 * p) - 1) `quot` 2
triangle = base * (base + 1) `quot` 2
m = p - triangle
n = base - m
The instance for pairs here is the standard Cantor diagonal. It's just a bit of algebra: use the triangle numbers to figure out where you're going/coming from. Now building up instances for this class is a breeze. Numbers are just represented in base 3:
-- this instance is a lie! there aren't infinitely many Digits
-- but we'll be careful about how we use it
instance Godel Digit where
to = fromIntegral . fromEnum
from = toEnum . fromIntegral
instance Godel Number where
to (Digit d) = to d
to (d ::: n) = 3 + to d + 3 * to n
from n
| n < 3 = Digit (from n)
| otherwise = let (q, r) = quotRem (n-3) 3 in from r ::: from q
For the remaining three types, we will, as suggested above, check the tag bit to decide which constructor to emit, and use the remaining bits as indices into a diagonalized list. All three instances necessarily look very similar.
instance Godel Term where
to (Number n) = 2 * to n
to (Parentheses s) = 1 + 2 * to s
from n = case quotRem n 2 of
(q, 0) -> Number (from q)
(q, 1) -> Parentheses (from q)
instance Godel Mul where
to (Term t) = 2 * to t
to (m :* t) = 1 + 2 * to (m, t)
from n = case quotRem n 2 of
(q, 0) -> Term (from q)
(q, 1) -> uncurry (:*) (from q)
instance Godel Add where
to (Mul m) = 2 * to m
to (m :+ t) = 1 + 2 * to (m, t)
from n = case quotRem n 2 of
(q, 0) -> Mul (from q)
(q, 1) -> uncurry (:+) (from q)
And that's it! We can now "efficiently" translate back and forth between parse trees and their Godel numbering for this grammar. Moreover, this translation matches the above enumeration, as you can verify:
*Main> map from [0..29] == take 30 ss
True
We did abuse many nice properties of this particular grammar -- non-ambiguity, the fact that almost all the nonterminals had infinitely many derivations -- but variations on this technique can get you quite far, especially if you are not too strict on requiring every number to be associated with something unique.
Also, by the way, you might notice that, except for the instance for (Nat, Nat), these Godel numberings are particularly nice in that they look at/produce one bit (or trit) at a time. So you could imagine doing some streaming. But the (Nat, Nat) one is pretty nasty: you have to know the whole number ahead of time to compute the sqrt. You actually can turn this into a streaming guy, too, without losing the property of being dense (every Nat being associated with a unique (Nat, Nat)), but that's a topic for another answer...

Algorithm: Triangle with two constraints, each corner on a given line

Some time ago I asked a question on math.stackexchange and got an answer. I have difficulties deriving an algorithm from that answer because my background is in design and hope some of you can help me.
The original question with visual sketch and possible answer are here:
https://math.stackexchange.com/questions/667432/triangle-with-two-constraints-each-corner-on-a-given-line
The question was: Given 3 3-dimensional lines (a, b and c) that coincide in a common point S and a given Point B on b, I'm looking for a point A on a and a point C on c where AB and BC have the same length and the angle ABC is 90 degrees.
I will have to implement this algorithm in an imperative language, any code in C++, Java, imperative pseudo-code or similar is fine.
Also, different approaches to this problem are equally welcome. Plus: Thanks for any hints, if the complete solution is indeed too time-consuming!
The two key formulas are
(I've replied the derivation for the formulas in the mathematics stack exchange site)
Substituting the first in the second gives in the end a 4th degree equation that is quite annoying to solve with a closed form. I've therefore used instead a trivial numerical solver in Python:
# function to solve (we look for t such that f(t)=0)
def f(t):
s = (t*cB - B2) / (t*ac - aB)
return s*s - 2*s*aB - t*t + 2*t*cB
# given f and an interval to search generates all solutions in the range
def solutions(f, x0, x1, n=100, eps=1E-10):
X = [x0 + i*(x1 - x0)/(n - 1) for i in xrange(n)]
Y = map(f, X)
for i in xrange(n-1):
if (Y[i]<0 and Y[i+1]>=0 or Y[i+1]<0 and Y[i]>=0):
xa, xb = X[i], X[i+1]
ya, yb = Y[i], Y[i+1]
if (xb - xa) < eps:
# Linear interpolation
# 0 = ya + (x - xa)*(yb - ya)/(xb - xa)
yield xa - ya * (xb - xa) / (yb - ya)
else:
for x in solutions(f, xa, xb, n, eps):
yield x
The search algorithm samples the function in the interval and when it finds two adjacent samples that are crossing the f=0 line repeats the search recursively between those two samples (unless the interval size is below a specified limit, approximating the function with a line and computing the crossing point in that case).
I've tested the algorithm generating random problems and solving them with
from random import random as rnd
for test in xrange(1000):
a = normalize((rnd()-0.5, rnd()-0.5, rnd()-0.5))
b = normalize((rnd()-0.5, rnd()-0.5, rnd()-0.5))
c = normalize((rnd()-0.5, rnd()-0.5, rnd()-0.5))
L = rnd() * 100
B = tuple(x*L for x in b)
aB = dot(a, B)
cB = dot(c, B)
B2 = dot(B, B)
ac = dot(a, c)
sols = list(solutions(f, -1000., 1000.))
And there are cases in which the solutions are 0, 1, 2, 3 or 4. For example the problem
a = (-0.5900900304960981, 0.4717596600172049, 0.6551614908475357)
b = (-0.9831451620384042, -0.10306322574446096, 0.15100848274062748)
c = (-0.6250439408232388, 0.49902426033920616, -0.6002456660677057)
B = (-33.62793897729328, -3.5252208930692497, 5.165162011403056)
has four distinct solutions:
s = 57.3895941365 , t = -16.6969433689
A = (-33.865027354189415, 27.07409541837935, 37.59945205363035)
C = (10.436323283003153, -8.332179814593692, 10.022267893763457)
|A - B| = 44.5910029061
|C - B| = 44.5910029061
(A - B)·(C - B) = 1.70530256582e-13
s = 43.619078237 , t = 32.9673082734
A = (-25.739183207076163, 20.5777215193455, 28.577540327140607)
C = (-20.606016281518986, 16.45148662649085, -19.78848391300571)
|A - B| = 34.5155582156
|C - B| = 34.5155582156
(A - B)·(C - B) = 1.13686837722e-13
s = -47.5886624358 , t = 83.8222109697
A = (28.08159526800866, -22.450411211385674, -31.17825902887765)
C = (-52.39256507303229, 41.82931682916268, -50.313918854788845)
|A - B| = 74.0747844969
|C - B| = 74.0747844969
(A - B)·(C - B) = 4.54747350886e-13
s = 142.883074325 , t = 136.634726869
A = (-84.31387768560096, 67.4064705656035, 93.61148799140805)
C = (-85.40270813540043, 68.1840435123674, -82.01440263735996)
|A - B| = 124.189861967
|C - B| = 124.189861967
(A - B)·(C - B) = -9.09494701773e-13
Write two quadratic equations for lambda, mu unknowns (just above matrix forms).
Solve this system with paper, pen and head, or with any mathematical software like Maple, Mathematica, Matlab, Derive etc. You will have 4th order equation. It has closed-form solution - apply Ferrari or Kardano method and get real roots, find mu, lambda, then point coordinates.

Help with algorithm for compute columns sum of a (quadtree) matrix?

Given this definition and a test matrix:
data (Eq a, Show a) => QT a = C a | Q (QT a) (QT a) (QT a) (QT a)
deriving (Eq, Show)
data (Eq a, Num a, Show a) => Mat a = Mat {nexp :: Int, mat :: QT a}
deriving (Eq, Show)
-- test matrix, exponent is 2, that is matrix is 4 x 4
test = Mat 2 (Q (C 5) (C 6) (Q (C 1) (C 0) (C 2) (C 1)) (C 3))
| | |
| 5 | 6 |
| | |
-------------
|1 | 0| |
|--|--| 3 |
|2 | 1| |
I'm trying to write a function that will output a list of columns sum, like: [13, 11, 18, 18]. The base idea is to sum each sub-quadtree:
If quadtree is (C c), then output the a repeating 2 ^ (n - 1) times the value c * 2 ^ (n - 1). Example: first quadtree is (C 5) so we repeat 5 * 2^(2 - 1) = 10, 2 ^ (n - 1) = 2 times, obtaining [5, 5].
Otherwise, given (Q a b c d), we zipWith the colsum of a and c (and b and d).
Of course this is not working (not even compiling) because after some recursion we have:
zipWith (+) [[10, 10], [12, 12]] [zipWith (+) [[1], [0]] [[2], [1]], [6, 6]]
Because I'm beginning with Haskell I feel I'm missing something, need some advice on function I can use. Not working colsum definition is:
colsum :: (Eq a, Show a, Num a) => Mat a -> [a]
colsum m = csum (mat m)
where
n = nexp m
csum (C c) = take (2 ^ n) $ repeat (c * 2 ^ n)
csum (Q a b c d) = zipWith (+) [colsum $ submat a, colsum $ submat b]
[colsum $ submat c, colsum $ submat d]
submat q = Mat (n - 1) q
Any ideas would be great and much appreciated...
Probably "someone" should have explained to who is worried about the depth of the QuadTree that the nexp field in the Matrix type is exactly meant to be used to determine the real size of a (C _).
About the solution presented in the first answer, ok it works. However it is quite useless to construct and deconstruct Mat, this could be easily avoided. Moreover the call to fromIntegral to "bypass" the type checking problem coming from the use of replicate can be solved without forcing to first going to Integral and then coming back, like
let m = 2^n; k=2^n in replicate k (m*x)
Anyway, the challenge here is to avoid the quadratical behavior due to the ++, that is what I would expect.
Cheers,
Let's consider your colsum:
colsum :: (Eq a, Show a, Num a) => Mat a -> [a]
colsum m = csum (mat m)
where
n = nexp m
csum (C c) = take (2 ^ n) $ repeat (c * 2 ^ n)
csum (Q a b c d) = zipWith (+) [colsum $ submat a, colsum $ submat b]
[colsum $ submat c, colsum $ submat d]
submat q = Mat (n - 1) q
It is almost correct, except the line where you define csum (Q a b c d) = ....
Let think about types. colsum returns a list of numbers. ZipWith (+) sums two lists elementwise:
ghci> :t zipWith (+)
zipWith (+) :: Num a => [a] -> [a] -> [a]
This means that you need to pass two lists of numbers to zipWith (+). Instead you create two lists of lists of numbers, like this:
[colsum $ submat a, colsum $ submat b]
The type of this expression is [[a]], not [a] as you need.
What you need to do is to concatenate two lists of numbers to obtain a single list of numbers (and this is, probably, what you intended to do):
((colsum $ submat a) ++ (colsum $ submat b))
Similarly, you concatenate lists of partial sums for c and d then your function should start working.
Let's go more general, and come back to the goal at hand.
Consider how we would project a quadtree into a 2n×2n matrix. We may not need to create this projection in order to calculate its column sums, but it's a useful notion to work with.
If our quadtree is a single cell, then we'd just fill the entire matrix with that cell's value.
Otherwise, if n ≥ 1, we can divide the matrix up into quadrants, and let the subquadtrees each fill one quadrant (that is, have each subquadtree fill a 2n-1×2n-1 matrix).
Note that there's still a case remaining. What if n = 0 (that is, we have a 1×1 matrix) and the quadtree isn't a single cell? We need to specify some behaviour for this case - maybe we just let one of the subquadtrees populate the entire matrix, or we fill the matrix with some default value.
Now consider the column sums of such a projection.
If our quadtree was a single cell, then the 2n column sums will all be 2n
times the value stored in that cell.
(hint: look at replicate and genericReplicate on hoogle).
Otherwise, if n ≥ 1, then each column overlaps two distinct quadrants.
Half of our columns will be completely determined by the western quadrants,
and the other half by the eastern quadrants, The sum for a particular column
can be defined as the sum of the contribution to that column
from its northern half (that is, the column sum for that column in the northern quadrant),
and its southern half (likewise).
(hint: We'll need to append the western column sums to the eastern column sums
to get all the column sums, and combien the northern and southern demi-column sums
to get the actual sums for each column).
Again, we have a third case, and the column sum here depends on how
you project four subquadtrees onto a 1×1 matrix. Fortunately, a 1×1 matrix means
only a single column sum!
Now, we only care about a particular projection - the projection onto a matrix of size 2dd×2d
where d is the depth of our quadtree. So you'll need to figure the depth too. Since a
single cell fits "naturally" into a matrix of size 1×1, that implies that it has a
depth of 0. A quadbranch must have depth great enough to allow each of its subquads to fit
into their quadrant of the matrix.

Resources