"Presents a decision procedure that can, in principle, observe people's behavior and from that infer what they want."

Short introduction to the concept of recursive self-improvement and the resulting "friendliness problem".

"A trick to building a safe super intelligence would be to require that its goals always consist of two parts: Firstly a specification of what is to be achieved, and secondly a specification of what resources can be used."

"Trying to prove that an AI is friendly is hard, trying to define “friendly” is hard, and trying to prove that you can’t prove friendliness is also hard. Although it is not the desired possibility, I suspect that the latter is actually the case."

Informal essay by Ben Goertzel. A result of his conversation with Shane Legg.

J. Andrew Rogers' argument against add-on Friendliness (the self-modeling Gödel problem).