Re: How would a computer know if it were conscious?

From: Brent Meeker <meekerdb.domain.name.hidden>
Date: Tue, 05 Jun 2007 10:41:46 -0700

Stathis Papaioannou wrote:
>
>
> On 05/06/07, *marc.geddes.domain.name.hidden <mailto:marc.geddes.domain.name.hidden.com>*
> <marc.geddes.domain.name.hidden <mailto:marc.geddes.domain.name.hidden.com>> wrote:
>
> Self-improvement requires more than just extra hardware. It also
> requires the ability to integrate new knowledge with an existing
> knowledge base in order to create truly orginal (novel) knowledge.
> But this appears to be precisely the definition of reflective
> intelligence! Thus, it seems that a system missing reflective
> intelligence simply cannot improve itself in an ordered way. To
> improve, a current goal structure has to be 'extrapolated' into a new
> novel goal structure which none-the-less does not conflict with the
> spirit of the old goal structure. But nothing but a *reflective*
> intelligence can possibly make an accurate assessment of whether a new
> goal structure is compatible with the old version! This stems from
> the fact that comparison of goal structures requires a *subjective*
> value judgement and it appears that only a *sentient* system can make
> this judgement (since as far as we know, ethics/morality is not
> objective). This proves that only a *sentient* system (a *reflective
> intelligence*) can possibly maintain a stable goal structure under
> recursive self-improvement.
>
>
> Why would you need to change the goal structure in order to improve
> yourself?

Even more problematic: How would you know the change was an improvement? An improvement relative to which goals, the old or the new?

Brent Meeker


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Everything List" group.
To post to this group, send email to everything-list.domain.name.hidden
To unsubscribe from this group, send email to everything-list-unsubscribe.domain.name.hidden
For more options, visit this group at http://groups.google.com/group/everything-list?hl=en
-~----------~----~----~----~------~----~------~--~---
Received on Tue Jun 05 2007 - 13:41:52 PDT

This archive was generated by hypermail 2.3.0 : Fri Feb 16 2018 - 13:20:14 PST