On benevolence and friendly AI theory

Jame5 it not just a science fiction novel – it is a science fiction novel with a cause. Ensuring the creation of a friendly AI is hard for many reasons:

  • Creating an AGI is hard
  • Goal retention is hard
  • Recursive self improvement is hard

The question what friendliness means however existed before all of those problems, is a separate one and needs to be answered before the creation of a friendly AI can be attempted. Coherent Extrapolated Volition in short CEV is Eliezer S. Yudkowsky’s take on Friendliness.

While CEV is great to describe what a friendly AIG will do, my critique of CEV is that it postpones answering the question of what friendliness is specifically until after we have an AIG that will answer that question for us.

Yes – a successfully implemented friendly AGI will do ‘good’ stuff and act in our ‘best interest’. But what is good and what is our best interest? In Jame5 I provide a different solution to the friendliness issue and suggest to skip right to the end of chapter 9 for anyone who would like to get right to the meat.

In addition I have summarized my core friendliness concepts in a paper called ‘Benevolence – a Materialist Philosophy of Goodness‘ (2007/11/09 UPDATE: latest version here) and in the end formulate the following friendly AGI supergoal:


  • Suffering: negative subjective experience equivalent to the subjective departure from an individual’s model of optimal fitness state as encoded in its genome/memome
  • Growth: absolute increase in individual’s fitness
  • Joy: positive subjective experience equivalent to the subjective contribution to moving closer towards an individual’s model of optimal fitness state as encoded in its genome/memome

Derived friendly AGI super goal: “Minimize all involuntary human suffering, direct all
unavoidable suffering towards growth, and reward all voluntary suffering contributing
to an individual’s growth with an equal or greater amount of joy.”

Comments (3)

Jame5 available for download

Get it while it is hot! It is an A4 PDF (111 pages, 1.4 MB). The PDF itself is free as in speech but a meatspace version is in the making and shall be available in the next 2 weeks or so and will be announced here.

Some testimonials after the jump.

“Jame5 is an engaging story about our confrontation with the singularity. From virtual worlds to AI gods, it provides a poignant and sometimes chilling exploration of the future and what it means to be human. A fascinating read.” Dr. Stephen Omohundro – President of Self-Aware Systems and Advisor to the Singularity Institute for Artificial Intelligence

„Of the several hundred science fiction books I have read so far Jame5 is one of the good ones. Its new ideas gave inspiration for thought and I had fun reading it.” Michael Adling – micenterprise.de

“If you enjoy intelligent fiction with more than a mere pinch of philosophy, this book is for you. Applies the idea of ‘survival of the fittest’ to a singularity scenario and challenges your perception of what is real, what is right, what is possible – and what is not. An alluring glimpse into the future of mankind!” Monika Siegenthaler

“If you are what you remember then couldn’t your past have just been a dream? Read at your own risk – Jame5 may dislocate your mind!” Olive Huang Hai – youtube.com/Katzen2002

„Answering the essential question of what the meaning of life might be has occupied us since the existence of human consciousness. Jame5 explores this as well as other questions and opens new perspectives that keep the reader thinking long after having closed the book.” Sonja Costabel

Comments (7)