Observing trends in information technology, some researchers conclude that artificial intelligence (AI) will eventually surpass the brightest human minds and take control of its own evolution. Assuming these researchers are correct, it is in our interest to ensure that we design AI to be friendly from the beginning.
Whether AI will surpass the brightest human minds depends, in my estimation, on whether we take into consideration the accelerating enhancement of human minds. As our information technology is becoming more powerful, it is also becoming more intimate. Arguably, a substantial portion of human intelligence is already based in our technology, carried in our pockets and decentralized into the Internet, yet essential to our identities. I expect AI will not surpass human minds, but rather gradually become indistinguishable from them, as human minds become increasingly distinguishable from their ancestors. However, I do think it's worth our time and effort to consider the hypothesis that AI will make a hard break from human minds and surpass them altogether, particularly because I would like to avoid that kind of future. Our posture toward AI should account for human enhancement, which is what computers are, at least for now.
By way of analogy, if AI expectations were prophecies (which they are for those of us who see religion everywhere), I would say that they too often advocate a relatively weak Messianic posture, emphasizing too much the externality of the Messiah. In the Judeo-Christian tradition, the Messiah brings salvation, whether from enemy tribes, from death and hell, or from whatever vexes us. Many wait for the Messiah, passively. Some work toward establishing a context that they suppose will induce the arrival of the Messiah. Others identify with the Messiah, thereby situating themselves psychologically to create the salvation they desire. This last posture, internalizing the Messiah, is the most empowering. Of course, power has its risks. On the one hand, some that identify with the Messiah are rightly considered lunatics. On the other hand, some do not internalize the Messiah at the expense of others, but rather consider it a common good to take full responsibility for our communal salvation. These persons are of the sort that make better worlds. Analogously, while AI expectations usually include a call to avoid a passive posture toward the beneficial power, they too often assume an externalizing posture. They too often lack the sacrament of human enhancement, through which we consume the beneficial power and thereby take on its identity, rather than only working to create a context for it.
