The Age of Em: Work, Love and Life when Robots Rule the Earth

Robin Hanson

This is a usual big picture. This is the U. S. economy over the last few years. Every time it went up or down a bit, it was big news. We’re going to look at a bigger picture.

Robin Hanson

This is the world over the last half century or so. And now the y-axis is logarithmic, and you may know that exponentials look like lines on a graph like this. It’s roughly aligned. So the major news of the world in the last half century or so is relatively steady exponential growth.

Robin Hanson

However, if you look over several thousand years, you’ll see that it doesn’t look like a line anymore. It looks like we’ve had growth lately, but before then there wasn’t any growth. There was this industrial revolution, and somehow we learned to invent growth.

Robin Hanson

But if you look on a larger time scale, It was growing exponentially before then. It was just growing slower. During the farming era, the economy doubled roughly every thousand years, but in the last few centuries, it’s been doubling roughly every 15 years.

Robin Hanson

Before 5000 BC or so, it looks like before then there was no growth. But if we go back even farther, it’s an even slower exponential growth. And if you go back even farther to the whole era of animal brains, they were growing roughly exponentially even slower. And it turns out genomes were growing even slower before that.

Robin Hanson

If we only focus on the history of humanity. We’ll see that the entire history is actually well approximated as a sequence of exponential growth. That is, the line looks pretty close to the dot. And I’d rather look at it in the following way.

Robin Hanson

So now the y-axis is the growth rate. So steady exponential growth looks less like a straight line on this graph. And you can see we’ve had these eras of exponential growth punctuated by sharp transitions to faster growth rates. So foragers doubled roughly every quarter million years, farmers every thousand. And we’ve been doubling every 15. And the transitions between the eras took less than the previous doubling time.

Robin Hanson

So, if this trend were to continue, Then, sometime in the next century or so, roughly, there will be a transition of, say, ten years or less, after which the economy would be doubling roughly every month. And that would continue for a year or two until, plausibly, something else would happen. That’s the straightforward projection of previous historical trends.

Robin Hanson

What could possibly cause something that big and dramatic? One of the most common speculations, as you may have even heard today, is artificial intelligence, smart robots. How could that happen? There are several scenarios people talk about.

Robin Hanson

One of the most straightforward, conservative scenarios is that we slowly just keep accumulating software and making it better the way we have for the last 70 years. I was an AI researcher and when I’ve been asking other researchers who’ve been in the field for at least 20 years, how far have we come in the last 20 years as a percentage of the distance to human level abilities? They typically tell me 5 to 10% of the way with no noticeable acceleration. And at that rate, we’re talking 2 to 4 centuries, which is in the foreseeable future, but not in your lifetime necessarily.

Robin Hanson

Other people think we’ll have much more rapid progress because they think we’re going to discover a great grand theory that’s going to make everything much easier. I’m skeptical. But I’m going to talk here about a third scenario, one I think isn’t obvious going to happen, but it’s plausible enough to be worth considering. And that’s the idea of porting software.

Robin Hanson

So today, if you have some old computer running software you like, and you’d like software like that running on a new computer, one approach is to stare at the software, try to guess how it works, and then write software on the new machine that works how you think it works on the old machine. Another approach, however, is to write what’s called an emulator that makes the new computer look like the old one to the software. If you can write an emulator, you can just move the software over without understanding it. And it’ll just work.

Robin Hanson

So, the idea is to try to do that for the human brain. To do this, we’ll need three technologies to reach good enough levels. None of them are there yet, but plausibly they might do that within a century, roughly.

Robin Hanson

One of them is we’ll need lots of cheap, fast parallel computers. Another is that we’ll need to have ways to scan particular human brains in fine spatial and chemical detail to see which cells of what type or where are connected to what. And third, we’re going to need models of how each kinds of brain cell takes signals in, changes internal state, and then sends output signals. And if we can have good enough models for all the kinds of cells in a brain and a good enough scan of a particular brain, then we can put it all together to make a model of that particular brain, which has the same input-output behavior as the brain.

Robin Hanson

What that means is you could hook it up with hands, eyes, ears, mouth, and then you could talk to it. It might talk back. You could ask it to do things. It might do them just like the original would. In fact, You’d have to convince it that it is the emulation now because the moment before it was scanned, it remembered being the human. So it’s very human-like, and if you had that, everything could change.

Robin Hanson

How long this will take depends roughly on how deep we’ll have to go into the structures of the brain. The deeper we have to go, the longer it’ll take to model and to have computers that are cheap enough. But even in the worst case, it looks like roughly in a century or so.

Robin Hanson

I’ve been around people talking about this idea for many decades. It’s a staple of science fiction and futurism, so I’m sure many of you know it as the upload concept or the emulation concept. When people have talked about it, they usually talk about, is it even possible to make an emulation of a machine that emulates a real fresh and bred human? If you made one, would it be conscious or is it just an empty machine? If you made one of me, is that me or somebody else? These are all fascinating questions that I’m going to ignore because people have neglected the question of, yeah, but what would actually happen? Not what’s the philosophy of identity, but what would the world look like? So that’s the question I’ve tried to take on in this book.

Robin Hanson

Now, if you look on Amazon, you’ll find roughly 20% of the books have the keyword history and 1% the keyword future. You’ll find there’s more interest in future fiction than historical fiction, however. Why so little interest in non-fiction about the future? If you ask intellectuals, they will tell you. Well, duh, we have artifacts about the past, we have documents and buildings, and we can study those, but we have no artifacts about the future, so we can’t study the future. I think that’s wrong because we have theory and we can use theory to study the future.

Robin Hanson

Think about three kinds of futurism: one is projecting current trends. Decreasing fertility, increasing leisure, other sorts of trends like that, and you could just ask where they might go. Another is to imagine trend-disrupting technologies like self-driving cars or blockchains. and envision when those technologies would show up, of what cost and what form. And a third approach is to ask, if a disruptive technology showed up, how would that change the rest of the world?

Robin Hanson

And a lot of the people who can do number two, they’re technologists who, like me, when I was a physics student, were told, you know, those people over in social science building, that’s bullshit. They make all that stuff up. That doesn’t exist. Only we go physical scientists over here really know stuff. And people in the technology world tend to think that, so they don’t think it’s possible to apply social science to actually figure out what would be the consequence of these technologies. I am now a professor of economics and I want to tell you it is possible to know things about social science and apply it and I’m trying to use this book to show just how much I can say.

Robin Hanson

Now, this is a map of all of academia, connected by links when they feel beside each other a lot. It turns out to be a ring, which is irrelevant for our purposes. For our purposes, the key point is the colored boxes are the ones I’m drawing on for this book. Which is a large number of areas, which means I’m taking the usual disadvantage in academia of studying too many areas and making it into a virtue by doing something you can only do if you study a lot of areas. This allows me to take the approach of picking the low-hanging fruit.

Robin Hanson

I’m just going to go area by area of trying to apply the standard results to get predictions, and when it gets hard and complicated, I just go on to the next thing. I’m going to do what theorists do everywhere, which is to look for our keys under the lamppost. What that means is we make simplifying assumptions as necessary to get concrete conclusions. So I’m going to explain my simplifying assumptions to you.

Robin Hanson

First of all, I am not trying to be creative or original other than by asking this unusual question. I’m just going to apply academic consensus. I’m going to focus on what the world’s likely to be like if we did the least to avoid it. Not to recommend or say that it’s a good or bad thing. I would be happy if it sounded more like a history test book or a business case book than a science fiction movie or a comic book.

Robin Hanson

I’m going to try to focus on the next great era after ours that’s as different from our era as our era is from the farming or foraging eras that came before it. I will not tell you about the universe for the next trillion years. I’m going to focus on after a transition. Transitions are harder to predict. People don’t know where they’re going. They’re anxious, et cetera. So I’m going to focus on what this is like when people are used to it. You’re now used to the industrial era, so your behavior is easier to predict.

Robin Hanson

And I’m going to use Economist’s standard best tool, which is called Supply and Demand. This implicitly assumes that there’s a lot of competition, there’s a lot of buyers and sellers for everything, there’s not much regulation. This is implicitly, we economists are looking at a world where no one’s in charge. In worlds where no one’s in charge, things can happen that nobody likes. And that’s just what I’m going to describe. If that’s what happens, you don’t have to like it.

Robin Hanson

Finally, in terms of the technology, the simplifying assumption is these emulations are opaque. That is, they’re black boxes. So you made a scan, you made a model, and now you have it. But all you can do is turn it on, turn it off, copy it, erase it, run it fast, run it slow. That’s it. You can’t take somebody’s music and combine it with somebody else’s Christmas vacation memory. You just don’t know how to edit it. Okay? Those are my working assumptions.

Robin Hanson

And now I’m going to tell you about what this world is like.

Robin Hanson

Imagine that you are in virtual reality, and this is your view. You can see this, you can hear the gulls, you can feel the wind on your skin. This might be what you really are, although you need something better to feel the wind. Now, an emulation could be in that same virtual reality, feeling that wind and seeing the sights, but for them, this is what they really are. They really are sitting on racks of hardware in a vast city of hardware because they are emulations running computers. But from their point of view, it looks like the beautiful virtual reality.

Robin Hanson

Or more plausibly, it looks like this. In virtual reality, M’s never need to feel pain, hunger, disease, grime. Their bodies are always beautiful. It’s very nice. It’s very cheap to give them a luxurious environment, because that’s just a few bits. But these are mostly desks. They are working most of the time. I’ll explain in a moment.

Robin Hanson

Okay, one of the things, I can tell you a number of things that are true for any sort of robot, which in emulation is a kind of robot. They can all be represented by computer files. And so, for example, computer files can be immortal. In principle. Now, today houses and cars are in principle immortal. If you keep repairing them, they can keep lasting forever. That doesn’t mean you do. In principle, immortality doesn’t imply actual immortality. Just to notice.

Robin Hanson

You can send computer files around the world at the speed of light. So emulations and any sort of robot can run around the world at the speed of light. You know that if we kill nature, you will probably die. Emulations know that if they accidentally kill off nature, if they’re made in factories out of stuff dug up in mines, they don’t die. They are less eager to save nature than you would be. You can make copies of computer files, and this has enormous implications.

Robin Hanson

So, if we just think about the supply and demand for labor, there’s a huge demand for people like you, a limited number of people like you, you guys get a huge wage. Once you have lots of robots that can do all the things you do, it’s cheap to make them in factory, wages fall to the cost of making them, which is well below your subsistence level. And even if there’s lots of you, wages fall low. The economy can actually grow faster.

Robin Hanson

So today, the economy needs labor and capital. We can make capital actually machines really fast in factories. But having a lot more machines and the same number of workers isn’t really actually that useful. If we could make a lot more people to go with the machines, then we could grow the economy a lot faster, but that’s not how we’ve been growing the economy. We haven’t been doubling the number of people every 15 years. How have we been doubling the economy? We’ve been making better machines. That’s been the main way we’ve been growing the economy. But if you can make a substitute for a human in a factory as fast as we’ve ever been able to make stuff in factories, The economy can grow much faster than today. Standard economic theory says the economy could double every month, plausibly.

Robin Hanson

That’s not crazy. If it takes a year to get to Mars and the economy doubles every month, the opportunity cost is just ridiculous. So there’ll be a lot less interest in doing things that are slow and long distance during this age, which may only last a year or two. Again. And I don’t tell you what happens next.

Robin Hanson

Okay, this is your life. You start and you end. Now, emulations have a wider variety of lives available. This is, for example, the life of an emulation who every day when they’re ready for work, they split off a number of short-term copies to stand in line with a DMV or whatever, and then only one of those copies goes on to the next day. These short-term copies are all work. The mainline copy, they have to rest before being ready for the next day. But these short-term copies, they’re all work, so they’re pretty cheap. We’ll talk more about them.

Robin Hanson

This is an opportunistic M. They make more copies of whichever versions are in more demand. They don’t know where their future’s going. This is an emulation designer. They conceive of a large system, and then they split into copies who elaborate the details of that design, all the way until they worked out all the details.

Robin Hanson

This allows emulations to conceive of larger, more coherent designs than people can today, because today one person designs and then other people implement. This is an emulation plumber who remembers every day for the last 20 years they only ever worked two hours a day, a life of leisure. What actually happened is when they were ready for work, a thousand copies were made, each of whom did a two-hour plumbing job, and one of them went on to the next day. Objectively, working over 99% of the time. Subjectively, a life of leisure.

Robin Hanson

We know of a lot of kinds of systems, including software which rots in the human brain, which have the feature that as they adapt and evolve To particular circumstances, they become fragile and harder to readapt to other circumstances. So, just as software rots, humans start out with fluid intelligence when we’re young, and we end up with Crystallized intelligence when we’re old, we know more, but we learn fewer tricks. So, because of this, emulations probably have a limited career length. Of perhaps a century, after which they need to retire and be replaced by slightly younger versions who’ve been training in slightly newer ways, because of this, emulations have around them younger and older versions of them. themselves, they can compare themselves to. So they see where their future is going in great detail. They see where they’re going to live, what kind of city they live in, and who they’re marrying, and they know their marriage will work because they have this reference.

Robin Hanson

Now, again, this is you. You start and then you end. This could be you if at the beginning of a party you took a drug that meant you would not remember that party the next day or ever after. I’m told some people do that. My question for you is: Toward the end of this party, would you say to yourself, I’m about to die? I became a new creature at the beginning of this party, that person tomorrow, that’s not me, because they won’t remember me. And how did I get myself into this? I don’t want to die. I hate this. Destroy, fight, rebel. Or

Robin Hanson

You could have said to yourself, that’s just me tomorrow. I won’t remember this. The emulations have the same options when they split off a short-term copy to stand in line with the DMV. That short-term copy could say, I’m a new creature with a short life, I hate this. Or they could say, I’m a part of the same person, I just won’t remember tomorrow what I did today. I predict they will take that second attitude, not because it’s philosophically correct, it just lets you get along in this world. This is a competitive world that selects for that sort of attitude.

Robin Hanson

Now, even if you don’t mind not remembering what you did, if you interact with other people and then later on you don’t seem to remember what they told you, they’ll be pissed. So a simple solution is that you just all copy whole teams together who then work together and then end or retire together. Now today, it’s hard to meet with celebrities.

Robin Hanson

Their time is valuable. For emulations, it’s easy, however. They just split off a new copy and you can interact with the president if you like. And everybody can. The hard thing is to get them to remember you later. But Ms can use this to their advantage.

Robin Hanson

For example, say the President says we must invade Iraq, and you’re not sure you believe that. You say, why? And they say, can’t tell you state secrets. You’ll have to trust me. You don’t know if you can trust them. But with emulations, you can trust them in the following way. What you do is you make a copy of yourself and a copy of the president, and both of the copies go inside a safe. And inside the safe, they can explain all their secret reasons. And at the end of the process, one bit comes out: a bit your copy inside chose to tell you: are they persuaded? So you hear a bit that says, Yes, there’s a good reason for this. I can’t tell you what it is, but there’s a good reason. And that was you a moment ago telling you that.

Robin Hanson

Humans in this scenario must retire all at once, everyone. Now, humans start out owning all the capital in this world. And if the economy doubles every month, their capital’s doubling every month. So collectively, humans are getting really rich really fast. However, many people have noted there are a lot of people in our society who don’t own much besides their ability to work. If they don’t acquire sufficient insurance or sharing arrangements before this happens, they will be out of money and starve. So I recommend doing that. Will the world do that? Well, sometimes in places in the past they have, and sometimes they haven’t. So that’s probably what happens in the future. It probably varies.

Robin Hanson

Now, why should the M’s let humans retire? Well, today we have retirees around us, and we could say to ourselves, you know, those retirees, they’re not doing anything for the rest of us. Let’s kill them and take their stuff. But we don’t. Plausibly, not just because some of us like some of the retirees, it’s also because we share institutions with them that we would threaten by this sort of action. And that’s plausibly why the humans could stay in peace as retirees for the age of M. You should worry less about whether that’s true or not than the fact that you don’t know what happens after the age of M.

Robin Hanson

Now, emulations all start out as humans, so they’re psychologically humans. All their mental characteristics are near the human range, but they’re not typical humans. So the emulation economy is going to select among the seven billion or ten billion humans. For the ones that are the most productive in the emulation economy in order to scan and train and make many copies of. So, the typical emulation on a graph of, say, smarter, hard working is up there on the upper right. They are the best. In fact, probably most emulations are copies of the few hundred most productive humans in this economy. That makes the typical emulation compared to the typical human as elite as the typical Nobel Prize winner, Olympic gold medalist, billionaire, head of state. They are very good and they know that. They will probably look on humans with nostalgia and gratitude, perhaps, but not so much respect, which is, if you think about it, how you think about your ancestors usually.

Robin Hanson

Now, we know many things about how more productive people differ from others in our society. We can just use those to make weak predictions about the emulations, how they differ from typical humans. They’ll probably be smart on average, conscientious, hardworking, workaholic even, married, religious because religion does correlate with productivity. Probably near a peak age of, which for us is near 40 or 50 in productivity. They’re at the peak of their career. They’re really good.

Robin Hanson

Now, some of you, when you were a child, you may have thought, I’m kind of weird, these people around me are different. I wonder if I’m from another planet. Someday I’ll meet other people from my planet, and then we’ll get along. Well, now for emulations, this is kind of true. Remember, most emulations are copies of the few hundred most copied, so they have millions or billions of other copies of the same human available to them to interact with. This is what I call a clan, and it’s available for finance and politics and law, and so they have a more large trusted social unit than you have today. They can trust them even more than families.

Robin Hanson

Probably the most traumatic thing that ever happened to humans, as discussed by an earlier speaker, was the transition from foraging to farming. Foragers were in, more like most animals, when they did what felt natural, it was usually roughly the right thing to do. But when farming became possible, it was only possible for humans to become farmers because we had enough cultural plasticity to change our norms and to change our values via culture. In order to adopt the new practices. So, farmers had a lot more inequality in war and slavery and property and marriage They traveled less, and these were all relatively alien things to foragers, but farmers got used to them, and one of the ways through that was strong conformity pressures and religions that were invented in order to allow that.

Robin Hanson

So there’s been this schism and that happened for thousands of years this farming, new farming style culture and attitudes was dominated and in the last few hundred years we’ve gotten rich. And as we’ve gotten rich, we’ve had a lot of social trends that drift back in the forager direction. That is, that can plausibly explain drifts in fertility, democracy, slavery, art, leisure. A lot of these ways, we have gotten more like foragers in our attitudes, and many people are proud of that and eager to follow that. Also, roughly a liberal-conservative axis. And so many people are looking forward to a Star Trek or Ian Bank sort of rich progressive future. The prediction here is that even though the humans would continue in that trend, the emulations would not. The emulations go back to a subsistence level income. They now have need of the social pressures of religion to get them to act in new ways that are strange to foragers, and so I predict a resurgence of religion for the and farmer-style attitudes in general for the emulations.

Robin Hanson

Humans today have three stages of life. You train and then you are maximally productive and then you retire. Emulations have the same three stages, but You can train a small number of copies, and you can spend a lot of money on that, and then you can make many copies of the best trainees. So, training can be done more cheaply.

Robin Hanson

And then, when you retire, if you can’t afford to retire at the speed at which you were working. You can just retire slower. So I told you emulations could run faster or slower. I didn’t tell you that the cost is roughly proportional to speed. So if you retire at 1% the speed when you’re working, that only costs 1% as it would to retire full speed. So emulations would be assigned to different tasks based the speed would be chosen to be suitable for the task.

Robin Hanson

So, when they’re running a fast factory, maybe they need to run fast. If they run something slow, driving a slow truck, they can be slow. If they interact with humans, they’d be slow. If their bosses are software engineers, they’d be fast. Fast and soil emulations would actually live in very different worlds, psychologically.

Robin Hanson

A fast emulation lives in a world of granite and rock. Almost everything around them is stuff they can’t change, and they have to find the few tunnels where they can even that anything can change in that world. In contrast to slow emulation, they’re in a world of fast buzzing insects around them. They need to keep their fingers out for it. They won’t get bitten off so they can find a place where they can be safely slow. Fast and slow emulations are exactly the same minds. They just run fast or slow, but they still would see different worlds. Now, a typical emulation that interacts with humans would run at a human speed.

Robin Hanson

A one that ran 16 times faster, if they had a physical body, it would actually need to be 16 times shorter in order to have the to have the body feel natural because uh at the moment our reaction time of a tenth of a second of our mind is roughly matched to the tenth of a second maximum. Period of the body parts we can control. So, as the emulations got faster, they would have smaller bodies when they did, although most of them would work in virtual reality, as I said. They would probably clump into the similar sort of speeds because that way they can interact with others. So this creates a class hierarchy.

Robin Hanson

And in fact, it probably goes up to a million times faster than humans. And the faster ones are just high status in many ways. They have many markers of status in our world. They embody more wealth, they win arguments, etc. So, this is a literal class hierarchy based on speed, and it’s really hard to argue that the fast ones really aren’t worth more. Because they really are able to do more.

Robin Hanson

In the other direction, it probably goes down to a billion times slower than human speed. And most of the slower emulations are like the ghosts of our literature. If you’re familiar with the ghosts of literature, they are creatures that are supposedly all around us that we could interact with if we wanted to, if we go through a special process, but they can’t influence our world very much. They don’t know very much. They’re obsessed with the past. So, kind of what’s the point?

Robin Hanson

Now, in our world today, we clump together in cities and we get a value out of that, but we aren’t all in one big city. And the main reason is as we make cities larger, there’s more traffic congestion. People spend a larger percentage of their time in traffic. Emulations, when they’re in the city, they can interact with anybody in the city instantly just by sending bits that represent where they are in a virtual room. They don’t actually have to move themselves, so they have a lot lower traffic congestion costs. So they can clump together in larger cities. Most emulations are probably in a small number of very dense cities, leaving most of the rest of the earth for the humans because the emulations don’t want it.

Robin Hanson

Now, today you can interact with anybody in virtual reality on the Earth because your tenth of a second reaction time is enough time for a light signal to go all the way around the Earth. But if you run faster than humans, say 16 times faster, you need to be within a thousand kilometers before you will not notice where they are. And the faster that you go, the closer you need to be so that you won’t notice these delays. So that means there’s a trade-off. If you want to run fast, you have to get closer to it on them. That limits how fast emulations will want to go. On the other hand, if they run much slower, too slow, then their careers will be over because jobs will have changed soon after they did their training.

Robin Hanson

So the trade-off between those two factors was roughly a thousand times human speed, which is my best guess for how fast the typical emulation goes. So at a thousand times human speed, the doubling time of a month is a century for them. That is, their world is subjectively changing more slowly than your world is changing for you. It’s a more stable world.

Robin Hanson

Uh some of you may be software engineers. There’s some changes in software engineering in this scenario. Uh over the last few decades we’ve had this experience that computers have been getting cheap fast and pe people have not. So people have changed their software engineering sty styles to use a lot more hardware compared to Compared to the people. Once emulation shows up, however, they suddenly get a lot cheaper, so there’s a back-to-the future moment where you want to go back to old styles, and then From that point on, the emulation hardware actually falls at the same price rate as the computer hardware. So, for writing parallel software, they would stay at the same relative cost level, and so they’d go to a back to the future moment and then stay there. And for serial, writing serial programs, it would get worse. Over time, they would want to do more and more things in their own head instead of having the computers do it.

Robin Hanson

I’ll skip talking about inequality to go to my last slide. I have given over a hundred talks on this. And at this point I know that you guys are eager to evaluate this world, to decide if you love it or hate it. I don’t feel I need to do very much of this evaluation because I know everybody else will do it. What I need to do is make sure you pay enough attention to the facts of the world before you do your evaluation. But let me summarize some things that might be relevant for your evaluation.

Robin Hanson

Now, first, I’d say If you took somebody from a thousand years ago and you asked them to evaluate your world, they’d probably love it or hate it depending on the first few things they heard. Your world is really just weird for them and they couldn’t evaluate it very well. So you might want to, if you were thinking about this, say, buy a whole book about it or something.

Robin Hanson

Okay, so the positives, again, mostly living and working in virtual reality. No pain, hunger, disease, grime. Their bodies are always beautiful. They’re not so afraid of what we’d call death. There’s a vast population of them who find their lives worth living. They live in much larger cities. If you ever said, I couldn’t live in a small town because there’s not enough going on there, they say that about your cities because their cities are much larger and more intricate. Larger economy can afford to spend more on art and music because these are all fixed-cost industries. They see the world as more stable.

Robin Hanson

On the other hand, They go back to a Malthusian subsistence wage for the emulations. Now, this isn’t a strange hypothetical. This is the usual case in history. It’s how pretty much all animals have ever lived and how pretty much all humans have ever lived until a few hundred years ago. So we understand how this works. There’s a lot of these short-term copies that end or retire. There’s more inequality, not just of wealth, but of these speeds, the class hierarchy. There’s probably larger, more bureaucratic firms. Less nature perhaps. Less space travel, at least during these two years. Less democracy, more religion. This is my best guess of the age of M.

Robin Hanson

Remember, it wasn’t my job to make you love it or hate it, just to tell you what I best guessed would happen if we did the least to avoid it. If you don’t like it, change it. Yeah, perfect.

Speaker 2

Okay, questions into the mic.

Speaker 3

Thank you. That was fascinating. So does it change any of these emulations? don’t have the status as free agents, but are legally property.

Robin Hanson

Some things change, most things don’t. So actually, most societies that we’ve ever seen, if many people were slaves, the society would still look pretty similar. Because slavery doesn’t change that much. It changes some things, so certainly it changes some degrees of inequality. We know some things about when slavery has happened in the past or not that give us a hint about when slavery might happen in the future. So, for example, Slaves are most valuable when wages are high. When land is scarce and labor is plentiful, wages fall to subsistence levels anyway, and it costs as much to feed a slave as it does to feed, you know, to feed a slave. Somebody else, so what’s the point? Somewhat. So, this is a world of subsistence wages. So, this is a world where there isn’t that much advantage from owning a slave, other because you cost roughly the same.

Speaker 3

One quick follow-up question, and then I’ll go away. What if all emulations were property? Or had the status of property. Would that change it? What kind of parameters are you thinking about that might change? Well, I’m imagining a world in which all emulations are intellectual property. held by humans. Possibly there’s some kind of technology that hardwires them so that they can’t go and Revolt or undo that particular legal arrangement?

Robin Hanson

So, I mean, if you’re worried about revolt, then since if they were, you know, if them being free workers who Earn the same amount as a slave would anyway makes them revolt less, then you might as well not have them be a slave. You might as well have them be a worker at a subsistence level. Another relevant factor here is in the early M era, the scanning process would probably be destructive. So, the initial emulations would be volunteers. That would be the only way you would get people to volunteer. And they would have to at least expect not to be slaves. So, a mass slavery scenario would have to be a surprise. To most of the people who had initially became signed up to become emulations because, again, they had to volunteer. But of course, surprises could happen. Thank you.

Speaker 2

Do you think that solar energy would play a large role in the world do you imagine of EMs? I mean, energy is wealth and It would be like billions of oil wells. It could be everybody might be wealthy with s solar energy. There’s so much solar energy. Around if it were harnessed properly and everything, and the idea of working at jobs to make money and make profits and all that would be impacted that way, I think. I just wonder what your thoughts would be about that. Well, I think you’re thinking of solar cells as a form of capital.

Robin Hanson

But generically, if you own capital, you can live off the interest if you own enough of it. In a world that’s a subsistence-level world, most people don’t own that much capital. But couldn’t the Sun just support all the at least it could support ten billion human beings and as if they were all just wealthy, like oil. If there was a very strong population control In this world, to prevent large numbers of M’s, then the small number of M’s remaining could all be rich. And one way they could be rich is by each owning a lot of capital. So, just in general, The thing is, the scenario that’s happening is because it’s so easy to increase the population, it happens. And as the population increases, the marginal wage of the value of each worker goes way down until it reaches subsistence level. So, it would take strong coordination to prevent that. You have to imagine population control at the global level that’s really quite intrusive. It has to go everywhere and prevent these sorts of things being created.

Speaker 5

Thank you. Presumably, an EM world would not be perfect. It would be flawed at least in moments. What might those moments look like? Would it be an epic pileup on the interstate? Would it be of Y2K proportions? The unspooling of tape, robots running into each other, warning, warning, Will Robinson, lost in space, like so all the usual things in our world would probably have echoes in this world.

Robin Hanson

They would have wars sometimes. And the wars would be destructive. The wars would probably be more cyber wars when feasible because the physical wars are just so slow here. But sometimes there’d be physical wars. Physical wars. The organizations are larger and more bureaucratic, so, although they’re more functional, you would be more frustrated by office politics and various bureaucratic inefficiencies as you are today. You know, the world has less variety in the sense that there’s only a few hundred people, so today we often enjoy the variety of people we interact with. Here, most of the people you interact with would be people you knew very well, because you had interacted with versions of them before. You might like that or not, but that would be some of the things that would go wrong. Of course, when you retire, you can retire slow, but you might have to worry that the A retirement place you go to, whether it’s following its promises to actually take care of you. You know, people actually worry about that today. You send them off to a retirement place, and then maybe those places often don’t take care of them as well as they promised you, but if you’re not paying very close attention. You don’t know that they’re not being treated well. That can happen to M retirees as well. Think of most of the problems you have in our world, and there are echoes in this world.

Speaker 6

I wonder if you could expand a little bit on both the concept that On the one hand, you have subsistence wages and long work hours, and on the other hand, you have a lot of leisure in this world. And then, secondly, a topic that we’ve talked about briefly, which is: can you talk a little bit about Sort of the responsibility for the infrastructure on which the M’s are running. Right, so

Robin Hanson

Let me take trying to remember both of the questions. Infrastructure and then the other one was subsistence wages, long work hours versus lots of leisure. Right. So I said the plumber remembered a life of leisure, but he’s misremembering. That is because we’re not used to this process that our memory could be very selective. That is, what we remember could be a non-representative sample of our history because we’ve never lived these lives as branching trees. So, this plumber objectively is working most of the time, but he remembers a life of leisure. And so, the emulations could be selected to remember lives of leisure, but they are not having a life of leisure. Leisure. They probably really like their life of work, though. So remember, even in our world, the most productive people tend to be workaholics who are really into their jobs. That’s what these people are. So even though if they’re working a lot, they’re really focused and obsessed and into it. Now, in terms of infrastructure, if you go back a thousand years and you ask what were most people doing a thousand years ago when most people were at subsistence farming level, well, they were mostly doing the things that were required to exist.

Robin Hanson

They were mostly making food and clothes and heat and buildings and transport because that’s what that economy needed. So this economy is mostly doing the things the M’s need to survive. They’re making computer hardware, they’re sending energy through it, they’re sending cooling pipes, they’re making structural support, communication lines. They’re doing those are where most of the effort’s going to because that’s Where most of what they need to subsist is. Describing a world where no one’s in charge, I’m assuming that there’s just these different companies that are providing these different services. Some people make computer hardware, other people make other things, and you. They’re competing with each other, so you buy the cheapest ones, you’re the one you like, and nobody’s necessarily in charge, but the net effect is that there’s all these things you can buy, and when you work hard and you just make enough money to barely survive. Yeah, so one of the aspects of this mismembering that’s a little interesting for me is

Speaker 7

It just sounds like something that no matter how hard you try to arrange the parameters in a way that the M’s are never going to notice problems with their world, that they’re going to notice them. like Truman show in a kind of Truman show kind of way. And so I’m wondering, like, have you explored these the ramifications of these trying to control this supposedly interactive world of M’s? Where things are bound to happen that you didn’t predict, and they’re starting to notice things like, oh wait, I’m noticing all these plumbers all around me doing The same thing I’m doing.

Robin Hanson

When I say misremember, I mean they’re kind of fooling themselves. It’s not like they don’t really know if you ask them. So even us today, I mean, we go through a lot of mental, psychological gymnastics to convince ourselves we’re okay with going to work every morning. You know, some people, when they look at this, they look, from a distance, it sounds like a terrible life. You go off to work every morning, there’s a boss who tells you what to do, and you have to wear what clothes they tell you. And then there’s all these forms you have to fill out and crap. You can make it sound like a terrible life, but clearly most of us find a way to find a corner of it where we’re okay with it. And that’s the same thing for the emulations. From a distance, you could criticize their world, but they’re in their world. They found a place in it where they’re okay and they’re doing what it takes to get along in their world, which is what we’ve pretty much done, which pretty much all humans have ever done. Ask yourself, a thousand years ago, most people were subsistence farmers. Why did they do that? Why didn’t they just commit suicide and quit? Well, you know, those who did didn’t last very long, and everybody else found a way to get along in their world, right? So if we’re excited about the H of M, what can we do to promote its coming?

Speaker 8

Or if we’re not, what can we do to stop it? So there’s enabling technologies.

Robin Hanson

So you might work on one of those enabling technologies. At some point, it will become feasible to invest ventures into trying this out, but probably not now. You could be one of those people doing that. You could certainly try to smooth the rough transition waters. That is, if this happens, it probably happens inevitably. But you could have very awkward rough transitions. For example, you could have large areas of the earth where they prevent this and they try to try to stop this from happening. And so VMs who succeed after that in existing and thriving even in the face of that opposition, they may not be so grateful and thankful to the humans. That could cause troubles later on for them treating the humans well. So it would it would probably be in the humans’ interest if this transition went smoothly and the emulations felt gratitude for the humans’ help coming to exist rather than, you know resentment for the opposition. You you know, th those are those are some of the kinds of things you could do. Uh you could invest, of course, in in places you so geography, you could decide where these M cities might show up and you Try to buy real estate. If you were going to oppose, I mean, the most straightforward thing is you’ll have to try to create some strong world government. Which isn’t impossible and we’ve certainly had larger scale stronger government over time. I’m doubtful that we would have sufficient capacity in time, but if you actually thought this was a terrible thing, that might motivate you to try harder to to help us coordinate better to have global laws that counter this. Basically, you’ll have to look at everybody’s computer everywhere and see what they’re running and make sure they aren’t running this. Which is possible, but dangerous.

Speaker 10

Thank you. Alright, so I was just wondering whatever your like what are your thoughts or what have you seen like considering like if these Ms like inhabit robot bodies to get like physical work done or just like yes live in the So in our economy, maybe 80% of jobs are desk jobs.

Robin Hanson

But some of our jobs are physical jobs, and they also have physical jobs. So, to do them, they will have to have physical bodies. They don’t have to have a permanent body. They can switch into a body to do this job, switch into a different body to do the job, switch out of the body to go take a break, a coffee break So, they don’t have to be tied to any particular body, and the body doesn’t have to be human-shaped. We already know that humans today can look at many machines as if it was the extension of their body. So, we know our minds are flexible enough to map onto lots of kinds of bodies. So, plausibly they will just explore that space and find out which kind of bodies are good for which jobs. Yeah, well, what I was sort of more wondering was: would most of them want to inhabit

Speaker 10

like robot bodies like just to uh if most of them did want to do that like what would happen to the w to the world like just if they wanted to go If they preferred living in the robot body over their own world.

Robin Hanson

So in our world when we’re rich, what happens in our world depends a lot more about what we want. In a world where you’re at subsistence wages, what you want doesn’t matter as much to what happens. It doesn’t matter what you want. Unless you do the things you need to do to survive, you go away and somebody else takes your place. So they basically are in that situation. It doesn’t matter how much they want a robot body or not, unless they’ve got a lot of wealth to spend or something, they will have to be doing the usual thing. Those who have wealth for leisure, some of them will choose to go on robot bodies and glide over vast clouds, or you know, those will be expensive compared to virtual reality versions of that, which would be a lot cheaper, but some rich people would spend it on the expensive version anyway. So you mentioned the possibility of this age lasting a year or two of our time.

Speaker 6

What are the factors that might Play into how long the age lasts. So, my simple heuristic is that we’ve had this history of ages where there’s been sudden transitions to the next age, and in that history, the ages have lasted a similar number of doubling times.

Robin Hanson

So I say, well, looks like a pattern there. I’m not willing to project the next age through more doubling times than the last ages I’ve already experienced. So, you know, 20 doublings looks like. about the limit I’d want to project. That’s my basic reasoning, is just to say not that I have a very particular idea of what could go wrong, although one possible way this era could end is if we eventually achieved artificial intelligence through other means that we Was cheaper and more effective than emulation. Emulations are a form of artificial intelligence. If you believe they are inefficient form compared to other forms, then eventually they might be replaced. That’s a different.

Speaker 11

I’m referring back to the famous social psychologist, Dr. Zeus, who came up with a story with stars on there, you know, creating uniqueness as the fundamental value is scarcity itself. And I’m intrigued with what you’ve done here with the M’s. You seem to have an efficiency factor, an economic efficiency factor, that goes down to kind of an oligopoly of 500. Performances, which I think cuts somehow against this idea that scarcity Has value. You go to museums now, right? Where do we go when we have free time? We go to the best. And I think, would I be correct to say that your whole theory might be challenged by the notion that there’s an infinite demand for uniqueness. That always will keep the wages. We know a lot about them. There’s going to be an inefficiency, so the wages never go totally forward. We don’t have to do hypothetical.

Robin Hanson

We know, in our actual world today, about the demand for variety and uniqueness. So, let me just tell you about it. In most industries, most industries are dominated by a small number of producers, a dozen or so. That’s the nature of most industries in our world. So, most people who buy things They look at the possible product variety and they don’t want to go that far down the list. In most industries, they want a limited amount of variety, not zero, but a limited amount. That’s just the nature of most industries that we have today. Labor markets are different today because employers have to deal with a lot more variety than they ideally want. In fact, the people most employers pay the most for come out of institutions that do the most to make them uniform and less creative and Stamp by standard product on them. These are the elite universities of our world. They have star-bellied sneaches. We see what the demand is in terms of variety of human workers. We see, in fact, that schools tend to crush creativity out of people, and that seems to be what most workplaces actually prefer. Because, in fact, in most workplaces, creativity looks like error. And vari and uncontrolled variation. And so I would say we’ll have the labor market look like what most other industries look like already. We already know the usual taste for variety in most industries. Now the labor market will look like that. And yes, you’re all precious, unique snowflakes, but employers don’t pay for that. Sorry.

Speaker 8

I’ll see two quick questions. One is: What is your M game plan? If you’re in this world How do you succeed and thrive? You personally, just curious what your personal game plan would be. And then the other one would be: have you thought about ideas of nesting these M’s? Rather than just forking and going off it, can you run? What could you essentially have another simulation arise within that simulation? Have you thought about that? No, that doesn’t make any sense.

Robin Hanson

Okay. So let’s focus on the first question. So your first question was: how do you personally thrive and succeed? So the scenario is. When the transition happens at the very first, the first thing they’re looking for are people at the peak of their career. The best lawyer, the best software engineer. So if you can time it, you want to be at the peak of your career just when this thing happens. But very quickly, they will be looking not for people at the peak of their career in the human economy, but for younger people who can be trained and become more flexible in the new economy. So they’re going to be looking for children. And they’re going to be destructively scanning them. So there’ll be some conflict there in the early days of the emulation economy seeking five-year-olds to destructively scan. who create to train to become productive emulations and people who think that’s not nice. So if I can’t be one of the person at the peak of my career, which happens, which doesn’t work for me because I’m already at the peak of my career and we’re way ahead of time. I might want to encourage my children, or more plausibly than my grandchildren, to try to be the sort of people who are well suited for this world. That is, when this world shows up, they should be children. who have many signs of being very productive and flexible and willing to work in this world and then not think they’re too proud to do anything. Are you religious? Say again? Are you planning to become religious? I might certainly help my grandchildren become religious. Again, you meet with some missionaries. Right. But again, just to have whatever at it, you know.

Robin Hanson

A thousand years ago, you could have been told you’re a subsistence farmer, and people the world could have told you, hey, in the future, industry’s going to take over. You could have had the attitude, this is terrible. I’m a subsistence farmer, my children are and my grandchildren are. We’re going to be pushed aside and those industrialists are going to take over. Another attitude you could have, well, if industry’s taken over, then I want my children and grandchildren to try to become industrialists, to try to become whatever this world wants. So you should try to teach your children to be ready to become what this world wants if you want to. Have them win in this world.

Speaker 7

So I was going to ask two quick questions about one about some of your assumptions around scarcity and what might drive the emergence of the age of N. And if there were some sort of radical rethinking or revolution that occurred that caused, for example, universal basic income or other things to cause people not to have to worry about preparing for retirement or job displacement is happening so quickly that the whole social contract is failing. So we need to do something more radical to kind of correct things. Would that possibly prevent or slow the emergence of the H of M and then Sorry to jump two questions in there, but you said you didn’t really think about what happened after the age of MZ. I said I don’t know what happens after. You don’t know. I would love to hear your just speculation on what maybe happens after. So the first question was about a basic income fund.

Robin Hanson

Revolution, that kind of thing. Okay, so if you promise everybody a basic income and that includes emulations and then you allow them to make as many copies as they want, your basic income fund just disappears quickly. You can’t afford it. So you’ll need to pair a basic income with some population control. So that’s why I said you could manage to create a world of high income M’s if you Strongly control their population worldwide. Without that, you can’t manage it. So you don’t need to give them a basic income if you control the population enough. They would just naturally have high wages. Again, it’s the population control that would be the strongest thing you would need.

Robin Hanson

In terms of what happens next, I alluded to that before. Over the last decades, we’ve slowly been automating many tasks by writing software to do those tasks. Well, presumably the M’s continue with this spectrum of continuing to write software that gets better and smarter and continues to replace M’s on various specific tasks. The net integral of that effect over the M era could be to slowly automate everything. Which case, emulations would then suffer the loss of jobs that we would suffer if emulations never showed up. So the same scenario could play out for the emulations on a shorter time scale, but it could be the same sort of scenario.

Speaker 12

This is the last question. So if I understood correctly and followed you, you made a simplifying assumption that that Mind emulations are units and they don’t merge or cross-pollinate, basically, and that that individuality stays the same despite the fact that now you have billions of copies. of it. But it seems to me that there are um there are aspects of this that actually uh challenge that notion of individuality. You talked about like the selective memory, but we’re already there is an an analogous process in the human mind where we narrate our narrative self experiences things very differently than our experiential self. So I wonder if you removed that simplifying assumption. What possibilities might emerge? Are we talking about the simplifying assumptions of common identity across copies? Or the simplifying assumption that That they couldn’t cross-pollinate or, you know, that they remained a unique.

Robin Hanson

Just one thing to be clear: in this branching structure of copies, they could come to see themselves as different from other things that they Had the same origin. They could take on different professions, live and marry different people, etc. So I certainly see a lot of possibilities for even when they’re all copies of the same origin, feeling that they’re different. Now, one of the most obvious opt options people are interested in is merging. If you could have multiple copies merged together, then you don’t have to have them have these separate lives. But probably what happens is that whatever accumulation of fragility that happens with experience and aging, when you merge them together, you sum the fragility of the two parts. And so that means you accelerate the aging through the merging. That means they probably don’t want to do that much merging until they retire. So merging becomes much more attractive among retirees. Because you might have thousands of retirees who are all slow. They could all merge together. They’re more fragile, but so what? Among the workers, though. So we today, we remember everything we do, so it’s not a cost. Emulations have to pay a lot more attention to each thing they do. Do I want to remember this? Because remembering this has the cost. Of not only needing rest at the end of the day, but also just that fragility that accumulates. So if you can do something and not remember it and do it nearly as well, you probably do. Thank you all very much.