- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]

From: Jesse Mazer <lasermazer.domain.name.hidden>

Date: Wed, 08 Jun 2005 18:22:59 -0400

rmiller wrote:

*>
*

*>At 02:45 PM 6/7/2005, Jesse Mazer wrote:
*

*>(snip)
*

*>
*

*>
*

*>>Of course in this example Feynman did not anticipate in advance what
*

*>>licence plate he'd see, but the kind of "hindsight bias" you are engaging
*

*>>in can be shown with another example. Suppose you pick 100 random words
*

*>>out of a dictionary, and then notice that the list contains the words
*

*>>"sun", "also", and "rises"...as it so happens, that particular 3-word
*

*>>"gestalt" is also part of the title of a famous book, "the sun also rises"
*

*>>by Hemingway. Is this evidence that Hemingway was able to anticipate the
*

*>>results of your word-selection through ESP? Would it be fair to test for
*

*>>ESP by calculating the probability that someone would title a book with
*

*>>the exact 3-word gestalt "sun, also, rises"? No, because this would be
*

*>>tailoring the choice of gestalt to Hemingway's book in order to make it
*

*>>seem more unlikely, in fact there are 970,200 possible 3-word gestalts you
*

*>>could pick out of a list of 100 possible words, so the probability that a
*

*>>book published earlier would contain *any* of these gestalts is a lot
*

*>>higher than the probability it would contain the precise gestalt "sun,
*

*>>also, rises". Selecting a precise target gestalt on the basis of the fact
*

*>>that you already know there's a book/story containing that gestalt is an
*

*>>example of hindsight bias--in the Heinlein example, you wouldn't have
*

*>>chosen the precise gestalt of Szilard/lens/beryllium/uranium/bomb from a
*

*>>long list of words associated with the Manhattan Project if you didn't
*

*>>already know about Heinlein's story.
*

*>>
*

*>>RM wrote:
*

*>In two words: Conclusions first.
*

*>Can you really offer no scientific procedure to evaluate Heinlein's story?
*

*>At the cookie jar level, can you at least grudgingly admit that the word
*

*>"Szilard" sure looks like "Silard"? Sounds like it too. Or is that a
*

*>coincidence as well? What are the odds. Should be calculable--how many
*

*>stories written in 1939 include the names of Los Alamos scientists in
*

*>conjunction with the words "bomb" , "uranium. . ."
*

*>
*

*>You're shaking your head. This, I assume is already a done deal, for you.
*

*>
*

*>And that, in my view, is the heart of the problem. Rather than swallow
*

*>hard and look at this in a non-biased fashion, you seem to be glued to the
*

*>proposition that (1) it's intractable or (2) it's not worth analyzing
*

*>because the answer is obvious.
*

I think you misunderstood what I was arguing in my previous posts. If you

look them over again, you'll see that I wasn't making a broad statement

about the impossibility of estimating the probability that this event would

have happened by chance, I was making a specific criticism of *your* method

of doing so, where you estimate the probability of the particular "gestalt"

of Szilard/lens/beryllium/uranium/bomb, rather than trying to estimate the

probability that a story would anticipate *any* possible gestalt associated

with the Manhattan Project. By doing this, you are incorporating hindsight

knowledge of Heinlein's story into your choice of the "target" whose

probability you want to estimate, and in general this will always lead to

estimates of the significance of a "hit" which are much too high. If you

instead asked someone with no knowledge of of Heinlein's story to come up

with a list of as many possible words associated with the Manhattan Project

that he could think of, then estimated the probability that a story would

anticipate *any* combination of words on the list, then your method would

not be vulnerable to this criticism (it might be flawed for other reasons,

but I didn't address any of these other reasons in my previous posts).

Look over the analogy I made in my last post again:

Suppose you pick 100 random words out of a

dictionary, and then notice that the list contains the words "sun", "also",

and "rises"...as it so happens, that particular 3-word "gestalt" is also

part of the title of a famous book, "the sun also rises" by Hemingway. Is

this evidence that Hemingway was able to anticipate the results of your

word-selection through ESP? Would it be fair to test for ESP by calculating

the probability that someone would title a book with the exact 3-word

gestalt "sun, also, rises"? No, because this would be tailoring the choice

of gestalt to Hemingway's book in order to make it seem more unlikely, in

fact there are 970,200 possible 3-word gestalts you could pick out of a list

of 100 possible words, so the probability that a book published earlier

would contain *any* of these gestalts is a lot higher than the probability

it would contain the precise gestalt "sun, also, rises".

To simplify things even further, let's say you simply make a list of ten

random numbers from 1 to 100, and before you make the list I make the

prediction "the list will contain the numbers 23 and 89". If it turns out

that those two numbers are indeed on your list, what is the significance of

this result as evidence for precognition on my part? Your method would be

like ignoring the other 8 numbers on the list and just finding the

probability that I would hit the precise target of "23, 89" by chance, which

(assuming order doesn't matter) would be only about a 1 in 5025 shot, if my

math is right. But the probability that both the numbers I guess will be

*somewhere* on the list of ten is significantly higher--I get that the

probability of this would be about 1 in 121. So if this experiment is done

in many alternate universes, then if in fact I have no precognitive

abilities, in about 1 in 121 universes, both numbers I guess will happen to

be on your list by luck. But then if you used the method of tailoring the

choice of target to my guess, in each such universe you will conclude that I

only had a 1 in 5025 chance of making that guess by chance. Clearly, then,

you get bad conclusions if you use hindsight knowledge to tailor the choice

of target to what you know was actually guessed in this way. But it's also

clear that this example is sufficiently well-defined that I would have no

general objection to estimating the probability that my "hit" could have

occurred by chance, it's just that the correct answer is 1 in 121, not 1 in

5025.

Jesse

Received on Wed Jun 08 2005 - 18:40:28 PDT

Date: Wed, 08 Jun 2005 18:22:59 -0400

rmiller wrote:

I think you misunderstood what I was arguing in my previous posts. If you

look them over again, you'll see that I wasn't making a broad statement

about the impossibility of estimating the probability that this event would

have happened by chance, I was making a specific criticism of *your* method

of doing so, where you estimate the probability of the particular "gestalt"

of Szilard/lens/beryllium/uranium/bomb, rather than trying to estimate the

probability that a story would anticipate *any* possible gestalt associated

with the Manhattan Project. By doing this, you are incorporating hindsight

knowledge of Heinlein's story into your choice of the "target" whose

probability you want to estimate, and in general this will always lead to

estimates of the significance of a "hit" which are much too high. If you

instead asked someone with no knowledge of of Heinlein's story to come up

with a list of as many possible words associated with the Manhattan Project

that he could think of, then estimated the probability that a story would

anticipate *any* combination of words on the list, then your method would

not be vulnerable to this criticism (it might be flawed for other reasons,

but I didn't address any of these other reasons in my previous posts).

Look over the analogy I made in my last post again:

Suppose you pick 100 random words out of a

dictionary, and then notice that the list contains the words "sun", "also",

and "rises"...as it so happens, that particular 3-word "gestalt" is also

part of the title of a famous book, "the sun also rises" by Hemingway. Is

this evidence that Hemingway was able to anticipate the results of your

word-selection through ESP? Would it be fair to test for ESP by calculating

the probability that someone would title a book with the exact 3-word

gestalt "sun, also, rises"? No, because this would be tailoring the choice

of gestalt to Hemingway's book in order to make it seem more unlikely, in

fact there are 970,200 possible 3-word gestalts you could pick out of a list

of 100 possible words, so the probability that a book published earlier

would contain *any* of these gestalts is a lot higher than the probability

it would contain the precise gestalt "sun, also, rises".

To simplify things even further, let's say you simply make a list of ten

random numbers from 1 to 100, and before you make the list I make the

prediction "the list will contain the numbers 23 and 89". If it turns out

that those two numbers are indeed on your list, what is the significance of

this result as evidence for precognition on my part? Your method would be

like ignoring the other 8 numbers on the list and just finding the

probability that I would hit the precise target of "23, 89" by chance, which

(assuming order doesn't matter) would be only about a 1 in 5025 shot, if my

math is right. But the probability that both the numbers I guess will be

*somewhere* on the list of ten is significantly higher--I get that the

probability of this would be about 1 in 121. So if this experiment is done

in many alternate universes, then if in fact I have no precognitive

abilities, in about 1 in 121 universes, both numbers I guess will happen to

be on your list by luck. But then if you used the method of tailoring the

choice of target to my guess, in each such universe you will conclude that I

only had a 1 in 5025 chance of making that guess by chance. Clearly, then,

you get bad conclusions if you use hindsight knowledge to tailor the choice

of target to what you know was actually guessed in this way. But it's also

clear that this example is sufficiently well-defined that I would have no

general objection to estimating the probability that my "hit" could have

occurred by chance, it's just that the correct answer is 1 in 121, not 1 in

5025.

Jesse

Received on Wed Jun 08 2005 - 18:40:28 PDT

*
This archive was generated by hypermail 2.3.0
: Fri Feb 16 2018 - 13:20:10 PST
*