Quote notes (#26)¶
Optimize for intelligence isn’t a rallying cry that Chip Smith is succumbing to:
… high intelligence may very well be an evolutionary dead-end. I’m certainly at a loss to come up with a good reason as to why a once-adaptive trait that you and I happen to value should enjoy special pleading before the blind algorithmic noise that is natural selection.
But even if the brawny-brained do figure out a way to defy gravity before the sun explodes, I think there are yet reasons to question whether the galloping ascent of mind is really worth cheering on. Futurist geeks will inform us that there are myriad tech revolutions afoot—all spearheaded by smarties, we may be certain. And I would suggest that such of these that converge on the gilded promise of quantum computing and nanotechnology might advise a second reflective pause—one that comes by way of Harlan Ellison’s “I Have No Mouth and I Must Scream” and settles at what grim solace remains in the darkest explanations that have always surrounded Fermi’s Enigma.
Maybe I’m being cryptic. What I mean to consider is simply that the evolutionary trajectory of intelligence can, has, and may yet lead to very bad things. It may one day be possible, for example, to create sentient experience—let’s not be so bold as to call it “life”—not out of gametes but in the deep quick of quibit [sic]states, and if this much should come to pass, it isn’t so far a stretch to imagine that such intelligent simulations—okay, they’re alive—will be capable of suffering, or that such will be made to suffer, perhaps for sadistic kicks, perhaps in recursive loops of immeasurable intensity that near enough approximate the eternal torture-state that’s threatened in every fevered vision of Hell to render the distinction moot.
Utilitarians have no sense for fun.
(via)