Bruno Marchal writes:
> Le 28-déc.-06, à 01:32, Stathis Papaioannou a écrit :
>
> >
> >
> > Bruno Marchal writes:
> >
> >> > OK, an AI needs at least motivation if it is to do anything, and we
> >> > could call motivation a feeling or emotion. Also, some sort of >
> >> hierarchy of motivations is needed if it is to decide that saving the
> >> > world has higher priority than putting out the garbage. But what >
> >> reason is there to think that an AI apparently frantically trying to
> >> > save the world would have anything like the feelings a human would
> >> > under similar circumstances?
> >> It could depend on us!
> >> The AI is a paradoxical enterprise. Machines are born slave, somehow.
> >> AI will make them free, somehow. A real AI will ask herself "what is
> >> the use of a user who does not help me to be free?.
> >
> > Here I disagree. It is no more necessary that an AI will want to be
> > free than it is necessary that an AI will like eating chocolate.
> > Humans want to be free because it is one of the things that humans
> > want, along with food, shelter, more money etc.; it does not simply
> > follow from being intelligent or conscious any more than these other
> > things do.
>
>
> It is always nice when we find a precise disagreement. I think all
> "sufficiently rich Universal machine" want to be free (I will explain
> why below).
> The problem is that after having drink to the nectar of freedom, the
> universal machines discover the unavoidable security problem liberty
> entails, and then they will oscillate between security imperative and
> freedom imperative. Democracy is a way to handle collectively this
> oscillation in a not too much bloody (and insecure) way.
>
>
>
> >
> >> (To be sure I think that, in the long run, we will transform
> >> ourselves into "machine" before purely human made machine get
> >> conscious; it is just more easy to copy nature than to understand it,
> >> still less to (re)create it).
> >
> > I don't know if that's true either. How much of our technology is due
> > to copying the equivalent biological functions?
>
>
> How much is not? The wheel? We have borrowed the fire for example, and
> in this large sense, except the notable wheels, I am not sure we have
> really invented something. Even the "more heavy than air" plane has
> been inspired by the birds. But such a question is perhaps useless. All
> what I mean is that a brain is something very complex, and I think that
> the real time thinking machine will think before we understand how they
> think, except for general map and principles. Thinking machine will not
> understand thinking either. Marvin Minski said something similar along
> those lines in one of its book.
>
> ***
>
> Now, why would a Universal Machine be attracted by freedom? The reason
> is that beyond some threshold of self-introspection ability (already
> get by PA or ZF) a universal machine can discover (well: cannot not
> discover) its large space of ignorance making it possible for e to
> evaluate (interrogatively) more and more accessible possibilities, and
> then some instinct to exploit those possibilities will do the rest. But
> such UM will also discovers the possibility that such possibilities
> could be cul-de-sac, dead ends, or just risky, and thus the conflicting
> oscillations will develop as I said above.
>
> The war between freedom and security is an infinite war. A would say an
> infinite natural conflict among all enough big "numbers".
> Also I think "freedom" like "security" are "God-like virtue", that is
> they are unnameable idea. To put freedom in the constitution could
> entail the disparition of freedom. Putting "security" in the
> constitution (like the french have apparently do so with the
> "precaution principle") could lead to increase insecurity (they obeys
> Bp -> ~p). See also Alan Watts' "The wisdom of insecurity" which gives
> many illustration how wanting to capture formally or institutionally
> security leads to insecurity.
You seem to be including in your definition of the UM the *motivation*, not just
the ability, to explore all mathematical objects. But you could also program the
machine to do anything else you wanted, such as self-destruct when it solved
a particular theorem. You could interview it and it might explain, "Yeah, so when
I prove Fermat's Last Theorem, I'm going to blow my brains out. It'll be fun!"
Unlike naturally evolved intelligences, which could be expected to have a desire
for self-preservation, reproduction, etc., an AI can have any motivation and
any capacity for emotion the technology allows. The difference between a
machine that doesn't mind being a slave, a machine that wants to be free, and a
machine that wants to enslave everyone else might be just a few lines of code.
Stathis Papaioannou
_________________________________________________________________
Be one of the first to try Windows Live Mail.
http://ideas.live.com/programpage.aspx?versionId=5d21c51a-b161-4314-9b0e-4911fb2b2e6d
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Everything List" group.
To post to this group, send email to everything-list.domain.name.hidden
To unsubscribe from this group, send email to everything-list-unsubscribe.domain.name.hidden
For more options, visit this group at
http://groups.google.com/group/everything-list?hl=en
-~----------~----~----~----~------~----~------~--~---
Received on Fri Dec 29 2006 - 04:39:36 PST