On mindcrime

One of the interesting ideas in Bostrom’s Superintelligence (a book not short on interesting ideas) is that of mindcrime.

Suppose a superintelligence could simulate human beings in order to learn more about their psychology. The crime arises when these simulations lead to cruel experiments which cause the simulated human beings harm. Because a superintelligence is likely to be made of electronics rather than old-fashioned biology, the harm can be caused at light-speed, far more efficiently than any harm humans could directly cause.

A good book raises more questions than it answers. I’ve been thinking about the following questions and possible answers ever since:

How could humans be simulated at all?

Discussed in the book is the idea of whole brain emulation as one possible avenue of simulation. One could slice, scan and map the brain and then emulate its operation inside a computer. The emulation should, if correct, behave as though the brain were still operational.

How could simulated human feel pain if they are only simulated?

This seems to be the problem of other minds. How can you know anyone else feels pain? Perhaps they are all faking. Bostrom also argues that it is likely that we are already living in a simulation. So to find a simulated human who feels pain, you might only have to look to yourself. But certainly if the humans were simulated with whole brain emulation, I see no reason why they wouldn’t be able to feel pain.

I can imagine people in pain. How do I know I am not committing mind crime?

It seems unlikely to me that my imaginations are capable of subjective experience. Partly because it seems impossible for my brain to simulate the entirety of another brain and also because imaginings don’t tend to behave of their own accord.

I’m still reeling from the book, and recommend it to anyone wanting to learn more about the inherent danger in what is potentially humanity’s last invention.