Re: ... cosmology? KNIGHT & KNAVE

From: Hal Finney <>
Date: Wed, 28 Jul 2004 11:01:38 -0700 (PDT)

This is confusing because "I believe p" has two different meanings.
One is that I have written down "p" with a number in front of it,
as one of my theorems. The other meaning is the string "Bp".

But that string only has meaning from the perspective of an outside
observer. To me, as the machine, it is just a pair of letters. B doesn't
have to mean "believe". It could mean "Belachen", which is German for

All I need to know, as a formal system, is what rules the letter B

Bruno wrote:
> At 09:54 27/07/04 -0700, Hal Finney wrote:
> >If I ever write down "x" on
> >my numbered list, I could also write down "Bx" and "BBx" and "BBBx" as far
> >as I feel like going. Is this correct?
> Well, not necessarily. Unless you are a "normal machine", which
> I hope you are!
> So let us accept the following definition: a machine is normal when,
> if it ever assert x, it will sooner or later asserts Bx.
> Normality is a form of self-awareness: when the machine believes x,
> it will believe Bx, that is it will believe that it will believe x.
> >But what about the other direction? From Bx, can I deduce x? That's
> >pretty important for this puzzle. If Bx merely is a shorthand for
> >saying that x is on my list, then it seems fair to say that if I ever
> >write down Bx I can also write down x. But this seems too powerful.
> You are right. It is powerful, but rather fair also.
> let us define a machine to be stable if that is the case. When the
> machine believes Bx the machine believes x.

So in my terms, I can add two axioms:

0a. x implies Bx
0b. Bx implies x

The first is the axiom of normality, and the second is the axiom of
stability. I don't find these words to be particularly appropriate,
by the way, but I suppose they are traditional.

It also seems to me that these axioms, which define the behavior of
the letter B, don't particularly well represent the concept of belief.
The problem is that beliefs can be uncertain and don't follow the law
of the excluded middle. If p is that there is life on Mars, then (p or
~p) is true. Either there's life there or there isn't. But it's not
true that (Bp or B~p). It's not the case that either I believe there
is life on Mars or I believe there is no life on Mars. The truth is,
I don't believe either way.

But axioms 0a and 0b let me conclude (Bp or B~p). Obviously they
collectively imply "p if and only if Bp". Therefore from (p or ~p)
we can immediately get (Bp or B~p). Hence for normal people, the law
of the excluded middle applies to beliefs.

This proof is pure logic and has no dependence on the meaning of B.
If B is "Belachen", I have showed that if p implies Belachen(p), then
it follows that (Belachen(p) or Belachen(~p)) is true. That's all.

It's a step outside the system to say that B follows rules which make it
appropriate for us to treat it as meaning "believes". But do 0a. and
0b. really capture the meaning of belief? I question that. B looks
more like an identity operator under those axioms.

> >The problem is that the rules I proposed here lead to a contradiction.
> >If x implies Bx, then I can write down:
> >
> >8. t implies Bt
> >
> >Note, this does not mean that if he is a knight I believe it, but rather
> >that if I ever deduce he is a knight, I believe it, which is simply the
> >definition of "believe" in this context.
> Here you are mistaken. It is funny because you clearly see
> the mistake, given that you say 'attention (t implies Bt) does
> not mean "if he is a knight the I believe it". But of course (t implies Bt)
> *does* mean "if he is a knight the I believe it".

I don't see this. To me as the machine, there is no meaning. I am just
playing with letters. t implies Bt is only a shorthand for "if he is a
knight" then B("if he is a knight"). There is no more meaning than that.
The letter B is just a letter that follows certain rules.

We only get meaning from outside, when we look at what the machine is
doing and try to relate the way the rules work to concepts in the real
world. It is at this point that we bring in the interpretation of Bx as
"the machine believes x".

Suppose for some proposition q the machine deduces it on step 117:

117. q

Does this mean that q is true? No, it means that that the machine
believes q. Does it mean that Bq is true? Yes. Bq is true, because Bq
is a shorthand for saying that the machine believes q, and by definition
the machine believes something when it writes it down in its numbered
list. We can see it right there, number 117. So the machine believes
q and Bq is true. But q is not (necessarily) true. The machine writing
something down does not "mean" it is true. By definition, it means the
machine believes it.

Consider a different example:

191. Bp

What does this mean? Does it mean that Bp is true? No, it means that the
machine believes Bp, because by definition, what the machine writes down
in its numbered list is what it believes. Is BBp true? Yes, it is true,
because that says that the machine believes Bp, and that means that Bp
is in the machine's numbered list, which it is, right there. The machine
believes Bp, but Bp is not (necessarily) true. BBp is necessarily true.

Another example

207. p implies q

By the same reasoning, this does not mean that if p is true, then q
is true. It means that if the machine ever believes p (i.e. it writes
it down in its list of theorems), then it will believe q (i.e. it will
eventually write down q in its list of theorems). The true statment is
that Bp implies Bq. You can also (trivially) conclude B(p implies q).
But "p implies q" is not (necessarily) true.

Now we are ready for:

>8. t implies Bt

It is not mean that "t implies Bt" is true, any more than "207. p implies
q" was true. What it means is that if the machine ever believes t, it
will believe Bt. Given that t means the native is a knight, it means
that, as I wrote before, "if I ever deduce he is a knight, I believe it",
where I am the machine. The true meaning is that Bt implies BBt.

So I don't see why you said that this was wrong, and that

> Here you are mistaken. It is funny because you clearly see
> the mistake, given that you say 'attention (t implies Bt) does
> not mean "if he is a knight the I believe it". But of course (t implies Bt)
> *does* mean "if he is a knight the I believe it".

While it is true that, by itself, (t implies Bt) does mean "if he
is a knight then I believe it", in the context of the machine, "8. t
implies Bt" doesn't mean that. It means that the machine believes the
implication, not that the implication is true.

> You add it means only
> (and here I add a slight correction) "if I ever deduce he is a knight I
> will deduce I believe he is a knight" which really is, in the machine
> language: (Bt implies BBt) instead of (t implies Bt).

Right, or as in my simpler example, when I wrote "207. p implies q" that
truly means Bp implies Bq.

> To be sure: a machine is normal if for any proposition p, if the machine
> believes p, it will believe Bp. But this is equivalent with saying
> that for any proposition p, the proposition (Bp implies BBp) is true
> *about* the machine.
> Same remark for stability: you can say a machine is stable
> if all the propositions (BBp implies Bp) are true about the machine.
> This does not mean the stable or normal machine will ever believe
> being stable or normal. You have (momentarily) confuse a proposition
> being true on a machine, and being believe by a machine.

I agree with everything here except the last sentence. I don't see what
I have confused, or what is wrong with my derivation. If you assume
both 0a. and 0b., meaning that I am both normal and stable, then the
proof is correct, right? If not, which step fails?

> >But 6 and 8 together mean that t implies a contradiction, hence I can
> >conclude:
> >
> >9. ~t
> >
> >He is a knave. 7 then implies
> >
> >10. Bt
> >
> >I believe he is a knight. And if Bx implies x, then:
> >
> >11. t
> >
> >and I have reached a contradiction with 9.
> >
> >So I don't think I am doing this right.

Hal Finney
Received on Wed Jul 28 2004 - 14:48:09 PDT

This archive was generated by hypermail 2.3.0 : Fri Feb 16 2018 - 13:20:09 PST