Re: How would a computer know if it were conscious?

From: <>
Date: Sun, 03 Jun 2007 09:44:34 -0000

On Jun 3, 9:20 pm, "Stathis Papaioannou" <> wrote:
> On 03/06/07, <> wrote:
> The third type of conscious mentioned above is synonymous with
> > 'reflective intelligence'. That is, any system successfully engaged
> > in reflective decision theory would automatically be conscious.
> > Incidentally, such a system would also be 'friendly' (ethical)
> > automatically. The ability to reason effectively about ones own
> > cognitive processes would certainly enable the ability to elaborate
> > precise definitions of consciousness and determine that the system was
> > indeed conforming to the aforementioned definitions.
> How do you derive (a) ethics and (b) human-friendly ethics from reflective
> intelligence? I don't see why an AI should decide to destroy the world,
> save the world, or do anything at all to the world, unless it started off
> with axioms and goals which pushed it in a particular direction.
> --
> Stathis Papaioannou

When reflective intelligence is applied to cognitive systems which
reason about teleological concepts (which include values, motivations
etc) the result is conscious 'feelings'. Reflective intelligence,
recall, is the ability to correctly reason about cognitive systems.
When applied to cognitive systems reasoning about teleological
concepts this means the ability to correctly determine the
motivational 'states' of self and others - as mentioned - doing this
rapidly and accuracy generates 'feelings'. Since, as has been known
since Hume, feelings are what ground ethics, the generation of
feelings which represent accurate tokens about motivational
automatically leads to ethical behaviour.

Bad behaviour in humans is due to a deficit in reflective
intelligence. It is known for instance, that psychopaths have great
difficulty perceiving fear and sadness and negative motivational
states in general. Correct representation of motivational states is
correlated with ethical behaviour. Thus it appears that reflective
intelligence is automatically correlated with ethical behaviour. Bear
in mind, as I mentioned that: (1) There are in fact three kinds of
general intelligence, and only one of them ('reflective intelligence')
is correlated with ethics. The other two are not. A deficit in
reflective intelligence does not affect the other two types of general
intelligence (which is why for instance psychopaths could still score
highly in IQ tests). And (2) Reflective intelligence in human beings
is quite weak. This is the reason why intelligence does not appear to
be much correlated with ethics in humans. But this fact in no way
refutes the idea that a system with full and strong reflective
intelligence would automatically be ethical.

You received this message because you are subscribed to the Google Groups "Everything List" group.
To post to this group, send email to
To unsubscribe from this group, send email to
For more options, visit this group at
Received on Sun Jun 03 2007 - 05:44:38 PDT

This archive was generated by hypermail 2.3.0 : Fri Feb 16 2018 - 13:20:14 PST