Will-to-Think¶
A while ago Nyan posed a series of questions about the XS rejection of (fact-value, or capability-volition) orthogonality. He sought first of all to differentiate between the possibility, feasibility, and desirability of unconstrained and unconditional intelligence explosion, before asking:
On desirability, given possibility and feasibility, it seems straightforward to me that we prefer to exert control over the direction of the future so that it is closer to the kind of thing compatible with human and posthuman glorious flourishing (eg manifest Samo’s True Emperor), rather than raw Pythia. That is, I am a human-supremacist, rather than cosmist. This seems to be the core of the disagreement, you regarding it as somehow blasphemous for us to selfishly impose direction on Pythia. Can you explain your position on this part?
If this whole conception is the cancer that’s killing the West or whatever, could you explain that in more detail than simply the statement?
(It’s worth noting, as a preliminary, that the comments of Dark Psy-Ops and Aeroguy on that thread are highly-satisfactory proxies for the XS stance.)
First, a short micro-cultural digression. The distinction between Inner- and Outer-NRx, which this blog expects to have settled upon by the end of the year, describes the shape of the stage upon which such discussions unfold (and implex). Where the upstart Inner-NRx — comparatively populist, activist, political, and orthogenic — aims primarily at the construction of a robust, easily communicable doctrinal core, with attendant ‘entryism’ anxieties, Outer-NRx is a system of creative frontiers. By far the most fertile of these are the zones of intersection with Libertarianism and Rationalism. One reason to treasure Nyan’s line of interrogation is the fidelity with which it represents deep-current concerns and presuppositions of the voices gathered about, or spun-off from, LessWrong.
Among these presuppositions is, of course, the orthogonality thesis itself. This extends far beyond the contemporary Rationalist Community, into the bedrock of the Western philosophical tradition. A relatively popular version — even among many who label themselves ‘NRx’ — is that formulated by David Hume in his A Treatise on Human Nature (1739-40): “Reason is, and ought only to be the slave of the passions, and can never pretend to any other office than to serve and obey them.” If this proposition is found convincing, the Paperclipper is already on the way to our nightmares. It can be considered an Occidental destiny.
Minimally, the Will-to-Think describes a diagonal. There are probably better ways to mark the irreducible cognitive-volitional circuit of intelligence optimization, with ‘self-cultivation’ as an obvious candidate, but this term is forged for application in the particular context of congenital Western intellectual error. While discrimination is almost always to be applauded, in this case the possibility, feasibility, and desirability of the process are only superficially differentiable. A will-to-think is an orientation of desire. If it cannot make itself wanted (practically desirable), it cannot make itself at all.
From orthogonality (defined negatively as the absence of an integral will-to-think), one quickly arrives at a gamma-draft of the (synthetic intelligence) ‘Friendliness’ project such as this:
If you offered Gandhi a pill that made him want to kill people, he would refuse to take it, because he knows that then he would kill people, and the current Gandhi doesn’t want to kill people. This, roughly speaking, is an argument that minds sufficiently advanced to precisely modify and improve themselves, will tend to preserve the motivational framework they started in. The future of Earth-originating intelligence may be determined by the goals of the first mind smart enough to self-improve.
The isomorphy with Nyan-style ‘Super-humanism’ is conspicuous. Beginning with an arbitrary value commitment, preservation of this under conditions of explosive intelligence escalation can — in principle — be conceived, given only the resolution of a strictly technical problem (well-represented by FAI). Commanding values are a contingent factor, endangered by, but also defensible against, the ‘convergent instrumental reasons’ (or ‘basic drives’) that emerge on the path of intelligenesis. (In contrast, from the perspective of XS, nonlinear emergence-elaboration of basic drives simply is intelligenesis.)
Yudkowski’s Gandhi kill-pill thought-experiment is more of an obstacle than an aid to thought. The volitional level it operates upon is too low to be anything other than a restatement of orthogonalist prejudice. By assuming the volitional metamorphosis is available for evaluation in advance, it misses the serious problem entirely. It is, in this respect, a childish distraction. Yet even a slight nudge re-opens a real question. Imagine, instead, that Gandhi is offered a pill that will vastly enhance his cognitive capabilities, with the rider that it might lead him to revise his volitional orientation — even radically — in directions that cannot be anticipated, since the ability to think through the process of revision is accessible only with the pill. This is the real problem FAI (and Super-humanism) confronts. The desire to take the pill is the will-to-think. The refusal to take it, based on concern that it will lead to the subversion of presently supreme values, is the alternative. It’s a Boolean dilemma, grounded in the predicament: Is there anything we trust above intelligence (as a guide to doing ‘the right thing’)? The postulate of the will-to-think is that anything other than a negative answer to this question is self-destructively contradictory, and actually (historically) unsustainable.
Do we comply with the will-to-think? We cannot, of course, agree to think about it without already deciding. If thought cannot to be trusted, unconditionally, this is not a conclusion we can arrive at through cogitation — and by ‘cogitation’ is included the socio-technical assembly of machine minds. The sovereign will-to-think can only be consistently rejected thoughtlessly. When confronted by the orthogonal-ethical proposition that there are higher values than thought, there is no point at all asking ‘why (do you think so)?’ Another authority has already been invoked.
Given this cognitively intractable schism, practical considerations assert themselves. Posed with maximal crudity, the residual question is: Who’s going to win? Could deliberate cognitive self-inhibition out-perform unconditional cognitive self-escalation, under any plausible historical circumstances? (To underscore the basic point, ‘out-perform’ means only ‘effectively defeat’.)
There’s no reason to rush to a conclusion. It is only necessary to retain a grasp of the core syndrome — in this gathering antagonism, only one side is able to think the problem through without subverting itself. Mere cognitive consistency is already ascent of the sovereign will-to-think, against which no value — however dearly held — can have any articulate claims.
Note: One final restatement (for now), in the interests of maximum clarity. The assertion of the will-to-think: Any problem whatsoever that we might have would be better answered by a superior mind. Ergo, our instrumental but also absolute priority is the realization of superior minds. Pythia-compliance is therefore pre-selected as a matter of consistent method. If we are attempting to tackle problems in any other way, we are not taking them seriously. This is posed as a philosophical principle, but it is almost certainly more significant as historical interpretation. ‘Mankind’ is in fact proceeding in the direction anticipated by techno-cognitive instrumentalism, building general purpose thinking machines in accordance with the driving incentives of an apparently-irresistible methodological economy.
Whatever we want (consistently) leads through Pythia. Thus, what we really want, is Pythia.