Monthly Archives: March 2012

Unthinkable as it may be, humanity, every last person, could someday be wiped from the face of the Earth. We have learned to worry about asteroids and supervolcanoes, but the more-likely scenario, according to Nick Bostrom, a professor of philosophy at Oxford, is that we humans will destroy ourselves.

Bostrom, who directs Oxford's Future of Humanity Institute, has argued over the course of several papers that human extinction risks are poorly understood and, worse still, severely underestimated by society. Some of these existential risks are fairly well known, especially the natural ones. But others are obscure or even exotic. Most worrying to Bostrom is the subset of existential risks that arise from human technology, a subset that he expects to grow in number and potency over the next century.

Despite his concerns about the risks posed to humans by technological progress, Bostrom is no luddite. In fact, he is a longtime advocate of transhumanism---the effort to improve the human condition, and even human nature itself, through technological means. In the long run he sees technology as a bridge, a bridge we humans must cross with great care, in order to reach new and better modes of being. In his work, Bostrom uses the tools of philosophy and mathematics, in particular probability theory, to try and determine how we as a species might achieve this safe passage. What follows is my conversation with Bostrom about some of the most interesting and worrying existential risks that humanity might encounter in the decades and centuries to come, and about what we can do to make sure we outlast them.

Some have argued that we ought to be directing our resources toward humanity's existing problems, rather than future existential risks, because many of the latter are highly improbable. You have responded by suggesting that existential risk mitigation may in fact be a dominant moral priority over the alleviation of present suffering. Can you explain why?

Bostrom:Well suppose you have a moral view that counts future people as being worth as much as present people. You might say that fundamentally it doesn't matter whether someone exists at the current time or at some future time, just as many people think that from a fundamental moral point of view, it doesn't matter where somebody is spatially---somebody isn't automatically worth less because you move them to the moon or to Africa or something. A human life is a human life. If you have that moral point of view that future generations matter in proportion to their population numbers, then you get this very stark implication that existential risk mitigation has a much higher utility than pretty much anything else that you could do. There are so many people that could come into existence in the future if humanity survives this critical period of time---we might live for billions of years, our descendants might colonize billions of solar systems, and there could be billions and billions times more people than exist currently. Therefore, even a very small reduction in the probability of realizing this enormous good will tend to outweigh even immense benefits like eliminating poverty or curing malaria, which would be tremendous under ordinary standards.

You have argued that we underrate existential risks because of a particular kind of bias called observation selection effect. Can you explain a bit more about that?

Bostrom: The idea of an observation selection effect is maybe best explained by first considering the simpler concept of a selection effect. Let's say you're trying to estimate how large the largest fish in a given pond is, and you use a net to catch a hundred fish and the biggest fish you find is three inches long. You might be tempted to infer that the biggest fish in this pond is not much bigger than three inches, because you've caught a hundred of them and none of them are bigger than three inches. But if it turns out that your net could only catch fish up to a certain length, then the measuring instrument that you used would introduce a selection effect: it would only select from a subset of the domain you were trying to sample. Now that's a kind of standard fact of statistics, and there are methods for trying to correct for it and you obviously have to take that into account when considering the fish distribution in your pond. An observation selection effect is a selection effect introduced not by limitations in our measurement instrument, but rather by the fact that all observations require the existence of an observer. This becomes important, for instance, in evolutionary biology. For instance, we know that intelligent life evolved on Earth. Naively, one might think that this piece of evidence suggests that life is likely to evolve on most Earth-like planets. But that would be to overlook an observation selection effect. For no matter how small the proportion of all Earth-like planets that evolve intelligent life, we will find ourselves on a planet that did. Our data point-that intelligent life arose on our planet-is predicted equally well by the hypothesis that intelligent life is very improbable even on Earth-like planets as by the hypothesis that intelligent life is highly probable on Earth-like planets. When it comes to human extinction and existential risk, there are certain controversial ways that observation selection effects might be relevant.

Bostrom: Well, one principle for how to reason when there are these observation selection effects is called the self-sampling assumption, which says roughly that you should think of yourself as if you were a randomly selected observer of some larger reference class of observers. This assumption has a particular application to thinking about the future through the doomsday argument, which attempts to show that we have systematically underestimated the probability that the human species will perish relatively soon. The basic idea involves comparing two different hypotheses about how long the human species will last in terms of how many total people have existed and will come to exist. You could for instance have two hypothesis: to pick an easy example imagine that one hypothesis is that a total of 200 billion humans will have ever existed at the end of time, and the other hypothesis is that 200 trillion humans will have ever existed.

Let's say that initially you think that each of these hypotheses is equally likely, you then have to take into account the self-sampling assumption and your own birth rank, your position in the sequence of people who have lived and who will ever live. We estimate currently that there have, to date, been 100 billion humans. Taking that into account, you then get a probability shift in favor of the smaller hypothesis, the hypothesis that only 200 billion humans will ever have existed. That's because you have to reason that if you are a random sample of all the people who will ever have existed, the chance that you will come up with a birth rank of 100 billion is much larger if there are only 200 billion in total than if there are 200 trillion in total. If there are going to be 200 billion total human beings, then as the 100 billionth of those human beings, I am somewhere in the middle, which is not so surprising. But if there are going to be 200 trillion people eventually, then you might think that it's sort of surprising that you're among the earliest 0.05% of the people who will ever exist. So you can see how reasoning with an observation selection effect can have these surprising and counterintuitive results. Now I want to emphasize that I'm not at all sure this kind of argument is valid; there are some
deep methodological questions about this argument that haven't been resolved, questions that I have written a lot about. See I had understood observation selection effects in this context to work somewhat differently. I had thought that it had more to do with trying to observe the kinds of events that might cause extinction level events, things that by their nature would not be the sort of things that you could have observed before, because you'd cease to exist after the initial observation. Is there a line of thinking to that effect? Bostrom: Well, there's another line of thinking that's very similar to what you're describing that speaks to how much weight we should give to our track record of survival. Human beings have been around for roughly a hundred thousand years on this planet, so how much should that count in determining whether we're going to be around another hundred thousand years? Now there are a number of different factors that come into that discussion, the most important of which is whether there are going to be new kinds of risks that haven't existed to this point in human history---in particular risks of our own making, new technologies that we might develop this century, those that might give us the means to create new kinds of weapons or new kinds of accidents. The fact that we've been around for a hundred thousand years wouldn't give us much confidence with respect to those risks.But, to the extent that one were focusing on risks from nature, from asteroid attacks or risks from say vacuum decay in space itself, or something like that, one might ask what we can infer from this long track record of survival. And one might think that any species anywhere will think of themselves as having survived up to the current time because of this observation selection effect. You don't observe yourself after you've gone extinct, and so that complicates the analysis for certain kinds of risks. A few years ago I wrote a paper together with a physicist at MIT named Max Tegmark, where we looked at particular risks like vacuum decay, which is this hypothetical phenomena where space decays into a lower energy state, which would then cause this bubble propagating at the speed of light that would destroy all structures in its path, and would cause a catastrophe that no observer could ever see because it would come at you at the speed of light, without warning. We were noting that it's somewhat problematic to apply our observations to develop a probability for something like that, given this observation selection effect. But we found an indirect way of looking at evidence having to do with the formation date of our planet, and comparing it to the formation date of other earthlike planets and then using that as a kind of indirect way of putting a bound on that kind of risk. So that's another way in which observation selection effects become important when you're trying to estimate the odds of humanity having a long future.

Read this article:
We're Underestimating the Risk of Human Extinction