by Jamais Cascio

2007

from Kurzweilai Website

I was reminded, earlier this year, of an observation made by polio vaccine pioneer Dr. Jonas Salk. He said that the most important question we can ask of ourselves is,

"are we being good ancestors?"

This is a particularly relevant question for those of us here at the Summit.

In our work, in our policies, in our choices, in the alternatives that we open and those that we close, are we being good ancestors? Our actions, our lives have consequences, and we must realize that it is incumbent upon us to ask if the consequences we're bringing about are desirable.

It's not an easy question to answer, in part because it can be an uncomfortable examination. But this question becomes especially challenging when we recognize that even small choices matter.

It's not just the multi-billion dollar projects and unmistakably world-altering ideas that will change the lives of our descendants. Sometimes, perhaps most of the time, profound consequences can arise from the most prosaic of topics.

Which is why I'm going to talk a bit about video games.

Well, not just video games, but video games and camera-phones and Google Earth and the myriad day-to-day technologies that, individually, may attract momentary notice, but in combination, may actually offer us a new way of grappling with the world. And just might, along the way, help to shape the potential for a safe Singularity.

Earlier this year, I co-authored a document that I know some of you in the audience have seen: the Metaverse Roadmap Overview (metaverse is a portmanteau of the prefix "meta" - meaning "beyond" - and "universe".)

In this work, along with my colleagues John Smart and Jerry Paffendorf, I sketch out four scenarios of how a combination of forces driving the development of immersive, richly connected information technologies may play out over the next decade. But what has struck me more recently about the Roadmap scenarios is that the four worlds could also represent four pathways to a Singularity.

Not just in terms of the technologies, but - more importantly - in terms of the social and cultural choices we make while building those technologies.

The four metaverse worlds emerged from a relatively commonplace scenario structure. We arrayed two spectra of possibility against each other, thereby offering four outcomes.

Specialists sometimes refer to this as the "four-box" method, and it's a simple way of forcing yourself to think through different possibilities.

This is probably the right spot to insert my first disclaimer: scenarios are not predictions, they're provocations. They're ways of describing different future possibilities not to demonstrate what will happen, but to suggest what could happen. They offer a way to test out strategies and assumptions - what would the world look like if we undertook a given action in these four futures?

To construct our scenario set we selected two themes likely to shape the ways in which the Metaverse unfolds:

• the spectrum of technologies and applications ranging from augmentation tools that add new capabilities to simulation systems that model new worlds

   

• the spectrum ranging from intimate technologies, those that focus on identity and the individual, to external technologies, those that provide information about and control over the world around you

These two spectra collide and contrast to produce four scenarios.

1. The first, Virtual Worlds, emerges from the combination of Simulation and Intimate technologies.

   These are immersive representations of an environment, one where the user has a presence within that reality, typically as an avatar of some sort. Today, this means World of Warcraft, Second Life, Sony Home and the like.
   
   Over the course of the Virtual Worlds scenario, we'd see the continued growth and increased sophistication of immersive networked environments, allowing more and more people to spend substantial amounts of time engaged in meaningful ways online.

    

   The ultimate manifestation of this scenario would be a world in which the vast majority of people spend essentially all of their work and play time in virtual settings, whether because the digital worlds are supremely compelling and seductive, or because the real world has suffered widespread environmental and economic collapse.
    

2. The next scenario, Mirror Worlds, comes from the intersection of Simulation and Externally-focused technologies.

   These are information-enhanced virtual models or “reflections” of the physical world, usually embracing maps and geo-locative sensors. Google Earth is probably the canonical present-day version of an early Mirror World.
   
   While undoubtedly appealing to many individuals, in my view, the real power of the Mirror World setting falls to institutions and organizations seeking to have a more complete, accurate and nuanced understanding of the world's transactions and underlying systems.

    

   The capabilities of Mirror World systems is enhanced by a proliferation of sensors and remote data gathering, giving these distributed information platforms a global context. Geospatial, environmental and economic patterns could be easily represented and analyzed.

    

   Undoubtedly, political debates would arise over just who does, and does not, get access to these models and databases.
    

3. Thirdly, Augmented Reality looks at the collision of Augmentation and External technologies.

   Such tools would enhance the external physical world for the individual, through the use of location-aware systems and interfaces that process and layer networked information on top of our everyday perceptions.
   
   Augmented Reality makes use of the same kinds of distributed information and sensory systems as Mirror Worlds, but does so in a much more granular, personal way. The AR world is much more interested in depth than in flows: the history of a given product on a store shelf; the name of the person waving at you down the street (along with her social network connections and reputation score); the comments and recommendations left by friends at a particular coffee shop, or bar, or bookstore.

    

   This world is almost vibrating with information, and is likely to spawn as many efforts to produce viable filtering tools as there are projects to assign and recognize new data sources.
    

4. Lastly, we have Life-logging, which brings together Augmentation and Intimate technologies.

   Here, the systems record and report the states and life histories of objects and users, enhancing observation, recall, and communication. I've sometimes talked about one version of this as the "participatory panopticon."
   
   Here, the observation tools of an Augmented Reality world get turned inward, serving as an adjunct memory.

    

   Life-logging systems are less apt to be attuned to the digital comments left at a bar than to the spoken words of the person at the table next to you. These tools would be used to capture both the practical and the ephemeral, like where you left your car in the lot and what it was that made your spouse laugh so much. Such systems have obvious political implications, such as catching a candidate's gaffe or a bureaucrat's corruption.

    

   But they also have significant personal implications: what does the world look like when we know that everything we say or do is likely to be recorded?

This underscores a deep concern that crosses the boundaries of all four scenarios: trust.

"Trust" encompasses a variety of key issues:

• protecting privacy and being safely visible

• information and transaction security

• critically, honesty and transparency

It wouldn't take much effort to turn all four of these scenarios into dystopias.

The common element of the malevolent versions of these societies would be easy to spot: widely divergent levels of control over and access to information, especially personal information. The ultimate importance of these scenarios isn't just the technologies they describe, but the societies that they create.

So what do these tell us about a Singularity?

Second disclaimer time: although I worked with John and Jerry on the original Metaverse scenarios, they should not be blamed for any of what follows.

Across the four Metaverse scenarios, we can see a variety of ways in which the addition of an intelligent system would enhance the user's experience. Dumb non-player characters and repetitive bots in virtual worlds, for example, might be replaced by virtual people essentially indistinguishable from characters controlled by human users.

Efforts to make sense of the massive flows of information in a Mirror World setting would be enormously enhanced with the assistance of sophisticated machine analyst. Augmented Reality environments would thrive with truly intelligent agent systems, knowing what to filter and what to emphasize.

In a life-logging world, an intelligent companion in one's mobile or wearable system would be needed in order to figure out how to index and catalog memories in a personally meaningful way; it's likely that such a system would need to learn how to emulate your own thought processes, becoming a virtual shadow.

None of these systems would truly need to be self-aware, self-modifying intelligent machines - but in time, each could lead to that point.

But if the potential benefits of these scenaric worlds would be enhanced with intelligent information technology, so too would the dangers. Unfortunately, avoiding dystopian outcomes is a challenge that may be trickier than some may expect - and is one with direct implications for all of our hopes and efforts for bringing about a future that would benefit human civilization, not end it.

It starts with a basic premise: software is a human construction.

That's obvious when considering code written by hand over empty pizza boxes and stacks of paper coffee cups.

But even the closest process we have to entirely computer-crafted software - emergent, evolutionary code - still betrays the presence of a human maker: evolutionary algorithms may have produced the final software, and may even have done so in ways that remain opaque to human observers, but the goals of the evolutionary process, and the selection mechanism that drives the digital evolution towards these goals, are quite clearly of human origin.

To put it bluntly, software, like all technologies, is inherently political. E

ven the most disruptive technologies, the innovations and ideas that can utterly transform society, carry with them the legacies of past decisions, the culture and history of the societies that spawned them. Code inevitably reflects the choices, biases and desires of its creators.

This will often be unambiguous and visible, as with digital rights management. It can also be subtle, as with operating system routines written to benefit one application over its competitors (I know some of you in this audience are old enough to remember "DOS isn't done 'til Lotus won't run").

Sometimes, code may be written to reflect an even more dubious bias, as with the allegations of voting machines intentionally designed to make election-hacking easy for those in the know. Much of the time, however, the inclusion of software elements reflecting the choices, biases and desires of its creators will be utterly unconscious, the result of what the coders deem obviously right.

We can imagine parallel examples of the ways in which metaverse technologies could be shaped by deeply-embedded cultural and political forces:

• the obvious, such as life-logging systems that know to not record digitally-watermarked background music and television

• the subtle, such as augmented reality filters that give added visibility to sponsors, and make competitors harder to see

• the malicious, such as mirror world networks that accelerate the rupture between the information haves and have-nots - or, perhaps more correctly, between the users and the used

• and, again and again, the unintended-but-consequential, such as virtual world environments that make it impossible to build an avatar that reflects your real or desired appearance, offering only virtual bodies sprung from the fevered imagination of perpetual adolescents.

So too with what we today talk about as a "singularity."

The degree to which human software engineers actually get their hands dirty with the nuts & bolts of AI code is secondary to the basic condition that humans will guide the technology's development, making the choices as to which characteristics should be encouraged, which should be suppressed or ignored, and which ones signify that "progress" has been made.

Whatever the degree to which post-singularity intelligences would be able to reshape their own minds, we have to remember that the first generation will be our creations, built with interests and abilities based upon our choices, biases and desires.

This isn't intrinsically bad; emerging digital minds that reflect the interests of their human creators is a lever that gives us a real chance to make sure that a "singularity" ultimately benefits us.

But it holds a real risk. Not that people won't know that there's a bias: we've lived long enough with software bugs and so-called "computer errors" to know not to put complete trust in the pronouncements of what may seem to be digital oracles.

The risk comes from not being able to see what that bias might be.

Many of us rightly worry about what might happen with "Metaverse" systems that analyze our life logs, that monitor our every step and word, that track our behavior online so as to offer us the safest possible society - or best possible spam. Imagine the risks associated with trusting that when the creators of emerging self- aware systems say that they have our best interests in mind, they mean the same thing by that phrase that we do.

For me, the solution is clear.

• Trust depends upon transparency

• Transparency, in turn, requires openness

• We need an Open Singularity

At minimum, this means expanding the conversation about the shape that a singularity might take beyond a self-selected group of technologists and philosophers. An "open access" singularity, if you will.

Dr. Kurzweil's books are a solid first step, but the public discourse around the singularity concept needs to reflect a wider diversity of opinion and perspective.

If the singularity is as likely and as globally, utterly transformative as many here believe, it would be profoundly unethical to make it happen without including all of the stakeholders in the process - and we are all stakeholders in the future.

World-altering decisions made without taking our vast array of interests into account are intrinsically flawed, likely fatally so. They would become catalysts for conflicts, potentially even the triggers for some of the "existential threats" that may arise from transformative technologies.

Moreover, working to bring in diverse interests has to happen as early in the process as possible. Balancing and managing a global diversity of needs won't be easy, but it will be impossible if democratization is thought of as a bolt-on addition at the end.

Democracy is a messy process. It requires give-and-take, and an acknowledgement that efficiency is less important than participation.

We may not have an answer now as to how to do this, how to democratize the singularity. If this is the case - and I suspect that it is - then we have added work ahead of us.

The people who have embraced the possibility of a singularity should be working at least as hard on making possible a global inclusion of interests as they do on making the singularity itself happen.

All of the talk of "friendly AI" (AI = Artificial Intelligence) and "positive singularities" will be meaningless if the only people who get to decide what that means are the few hundred of us in this room.

My preferred pathway would be to "open source" the singularity, to bring in the eyes and minds of millions of collaborators to examine and co-create the relevant software and models, seeking out flaws and making the code more broadly reflective of a variety of interests. Such a proposal is not without risks. Accidents will happen, and there will always be those few who wish to do others harm.

But the same is true in a world of proprietary interests and abundant secrecy, and those are precisely the conditions that can make effective responses to looming disasters difficult. With an open approach, you have millions of people who know how dangerous technologies work, know the risks that they hold, and are committed to helping to detect, defend and respond to crises.

That these are, in Bill Joy's term, "knowledge-enabled" dangers means that knowledge also enables our defense; knowledge, in turn, grows faster as it becomes more widespread.

This is not simply speculation; we've seen time and again, from digital security to the global response to SARS, that open access to information-laden risks ultimately makes them more manageable.

The metaverse roadmap offers a glimpse of what the next decade might hold, but does so recognizing that the futures it describes are not end-points, but transitions.

The choices we make today about commonplace tools and everyday technologies will shape what's possible, and what's imaginable, with the generations of technologies to come.

If the singularity is in fact near, the fundamental tools of information, collaboration and access will be our best hope for making it happen in a way that spreads its benefits and minimizes its dangers - in short, making it happen in a way that lets us be good ancestors.

If we're willing to try, we can create a future, a singularity, that's wise, democratic and sustainable - a future that's open.

• Open as in transparent

• Open as in participatory

• Open as in available to all

• Open as in filled with an abundance of options

The shape of tomorrow remains in our grasp, and will be determined by the choices we make today.

Choose wisely...