Viewing entries in
Synthetic Worlds

Motion Gaming Technology For Everyone

Omek Interactive wants to put you in the game…and in the TV…and in the computer. The Israel based company has developed Shadow SDK, a middlewarepackage that enables 3D gesture technology for all types of home media. With Shadow, third party developers can create realistic video games where your body becomes the controller, or it can be used to create gesture controlled TV/media centers, or computer interfaces. Omek Interactive demoed some great applications fueled by Shadow at Techonomy 2010. Check out them out along with CEO Janine Kutliroff’s presentation in the video below.

It looks like the human computer interface of the future could be the open air. Ive seen some pretty cool gesture systems that only require a camera and a person’s body to control various media devices. The incredible interface from Minority Report is going to arrive in the next few years, gesture TVs are coming to the market soon (“the end of 2010″), and Microsoft’s Project Natal should be available at about the same time. Because Shadow enabled applications can work with video games, it’s often compared to Natal. Both can give you real-time control of an avatar, as you’ll see in the following:

Kutliroff’s speech ends around 5:40 followed by a media room gesture control application, a demonstration of an avatar (7:43), and a pretty neat-looking boxing game (8:43).

Of course one of the big differences between Project Natal and Shadow is that you’ll only ever see Natal on the Xbox or other Microsoft platforms. Shadow might be popping up everywhere. At least, that’s what Kutliroff and Omek seemed to be aiming for. Other companies in the gesture control business are focusing on a single application (Toshiba/Hitachi for TVs and home media, g-speak for computers, and Project Natal for video games). Omek Interactive isn’t married to one particular kind of hardware and they’re definitely trying to court a plurality of application developing firms. While they’ve created some interesting demo games and applications, Kutliroff’s presentation clings to the middleware status. Shadow is, after all, a SDK. Omek is poised to enable third party developers to build the next generation of gesture controlled technologies. Probably in video games, but possibly for TVs and computers as well.

The only question I have is whether the products that would sandwich Shadow (the 3D cameras on one side, and the gesture enabled applications on the other) are actually ready. We’ve seen some depth-perceptive cameras on the market (such as the 3D stereoscopic webcam from Minoru) but they are far from ubiquitous. Likewise, there’s been some good buzz surrounding gesture TVs and Project Natal’s video games but neither is actually on sale yet. This is an emerging market, and while the possibilities for gesture controls are very promising there’s no guarantee they’ll be popular. Omek could be caught as the middleman between two types of products that never get off the ground.

I must admit that part of my skepticism stems from the fact that gesture controls are not my favorite of the technologies contending to be the next major human-computer interface. As fun as it may be to play a movie with the flip of a wrist, or use your entire body to play a virtual boxing match, these applications lack tactile feedback. There’s nothing to hold. Nothing physical to let you know that you’re actually interacting with something. To me, for gesture controls to really succeed they’ll need some sort of haptics. I’d be totally cool with flailing my limbs through the open air if I could actually feel when my virtual self was hitting something.

Still, my personal preferences aside, the entire body monitoring control scheme seems to be grabbing a lot of attention. Omek Interactive is making a great move by racing to become the definitive middleware solution in the field. If the public does become interested in gesture technology, the Shadow SDK could get some major use. It would let companies that are good at making hardware, and companies that are good at making applications (i.e. games) focus on their strengths while Omek knits them together. That’s a smart strategy and a sure way to enable innovation. It will likely take several years before we know whether gesture controls are here to stay, but Omek is certainly a name to watch while we figure it all out.

Moving Towards An Open Singularity

Recently, I had a dialogue with some colleagues (Tina and RJ), about technology and the future. The focus of our discussion was the Metaverse and The Singularity. Although, my colleagues were unfamiliar with these exact terms. I believe the dialog important enough to want to share some thoughts about that discussion and the singularity prior to the Singularity Summit (which is happening in NYC on October 3-4). And I encourage anyone reading this to attend.

 

Yes, this post is long, but worthwhile, if for no other reason than to share the ideas of The Singularity and the Metaverse as well some new thoughts I had on those subjects.

 

So, the conversation with my colleagues when like this (paraphrasing):

 

- "What happens when.. virtual worlds meet geospatial maps of the planet?"

- "When simulations get real and life and business go virtual?"

- "When you use a virtual Earth to navigate the physical Earth, and your avatar becomes your online agent?"

-- "What happens then," I said, "is called the Metaverse."

I recall an observation made by polio vaccine pioneer Dr. Jonas Salk. He said that the most important question we can ask of ourselves is, "are we being good ancestors?"

 

This is a particularly relevant question for those of us that will be attending the Singularity Summit this year. In our work, in our policies, in our choices, in the alternatives that we open and those that we close, are we being good ancestors? Our actions, our lives have consequences, and we must realize that it is incumbent upon us to ask if the consequences we're bringing about are desirable.

 

This question was a big part of the conversation with my colleagues. Although, that is not an easy question to answer, in part because it can be an uncomfortable examination. But this question becomes especially challenging when we recognize that even small choices matter. It's not just the multi-billion dollar projects and unmistakably world-altering ideas that will change the lives of our descendants. Sometimes, perhaps most of the time, profound consequences can arise from the most prosaic of topics.

 

Which is why I'm going to write a bit here about video games.

 

Well, not just video games, but video games and camera phones (which many of my readers know - I happen to know quite a bit about), and Google Earth and the myriad day-to-day technologies that, individually, may attract momentary notice, but in combination, may actually offer us a new way of grappling with the world. And just might, along the way, help to shape the potential for a safe Singularity.

 

In the Metaverse Roadmap Overview the authors sketch out four scenarios of how a combination of forces driving the development of immersive, richly connected information technologies may play out over the next decade. But what has struck me more recently about the roadmap scenarios is that the four worlds could also represent four pathways to a Singularity. Not just in terms of the technologies, but—more importantly—in terms of the social and cultural choices we make while building those technologies.

 

The four metaverse worlds emerged from a relatively commonplace scenario structure. The authors arrayed two spectra of possibility against each other, thereby offering four outcomes. Analysts sometimes refer to this as the "four-box" method, and it's a simple way of forcing yourself to think through different possibilities.

 

This is probably the right spot to insert this notion: scenarios are not predictions, they're provocations. They're ways of describing different future possibilities not to demonstrate what will happen, but to suggest what could happen. They offer a way to test out strategies and assumptions—what would the world look like if we undertook a given action in these four futures?

 

To construct the scenario set the authors selected two themes likely to shape the ways in which the Metaverse unfolds: the spectrum of technologies and applications ranging from augmentation tools that add new capabilities to simulation systems that model new worlds; and the spectrum ranging from intimate technologies, those that focus on identity and the individual, to external technologies, those that provide information about and control over the world around you. These two spectra collide and contrast to produce four scenarios.

 

The first, Virtual Worlds, emerges from the combination of Simulation and Intimate technologies. These are immersive representations of an environment, one where the user has a presence within that reality, typically as an avatar of some sort. Today, this means World of Warcraft, Second Life, PlayStation Home and the like.

 

Over the course of the Virtual Worlds scenario, we'd see the continued growth and increased sophistication of immersive networked environments, allowing more and more people to spend substantial amounts of time engaged in meaningful ways online. The ultimate manifestation of this scenario would be a world in which the vast majority of people spend essentially all of their work and play time in virtual settings, whether because the digital worlds are supremely compelling and seductive, or because the real world has suffered widespread environmental and economic collapse.

 

The next scenario, Mirror Worlds, comes from the intersection of Simulation and Externally-focused technologies. These are information-enhanced virtual models or “reflections” of the physical world, usually embracing maps and geo-locative sensors. Google Earth is probably the canonical present-day version of an early Mirror World.

 

While undoubtedly appealing to many individuals, in my view, the real power of the Mirror World setting falls to institutions and organizations seeking to have a more complete, accurate and nuanced understanding of the world's transactions and underlying systems. The capabilities of Mirror World systems is enhanced by a proliferation of sensors and remote data gathering, giving these distributed information platforms a global context. Geospatial, environmental and economic patterns could be easily represented and analyzed. Undoubtedly, political debates would arise over just who does, and does not, get access to these models and databases.

 

Thirdly, Augmented Reality looks at the collision of Augmentation and External technologies. Such tools would enhance the external physical world for the individual, through the use of location-aware systems and interfaces that process and layer networked information on top of our everyday perceptions.

 

Augmented Reality makes use of the same kinds of distributed information and sensory systems as Mirror Worlds, but does so in a much more granular, personal way. The AR world is much more interested in depth than in flows: the history of a given product on a store shelf; the name of the person waving at you down the street (along with her social network connections and reputation score); the comments and recommendations left by friends at a particular coffee shop, or bar, or bookstore. This world is almost vibrating with information, and is likely to spawn as many efforts to produce viable filtering tools as there are projects to assign and recognize new data sources.

 

Lastly, we have Lifelogging, which brings together Augmentation and Intimate technologies. Here, the systems record and report the states and life histories of objects and users, enhancing observation, recall, and communication. I've sometimes discussed one version of this as the "participatory panopticon."

Here, the observation tools of an Augmented Reality world get turned inward, serving as an adjunct memory. Lifelogging systems are less apt to be attuned to the digital comments left at a bar than to the spoken words of the person at the table next to you. These tools would be used to capture both the practical and the ephemeral, like where you left your car in the lot and what it was that made your spouse laugh so much. Such systems have obvious political implications, such as catching a candidate's gaffe or a bureaucrat's corruption. But they also have significant personal implications: what does the world look like when we know that everything we say or do is likely to be recorded?

 

This underscores a deep concern that crosses the boundaries of all four scenarios: trust.

 

"Trust" encompasses a variety of key issues: protecting privacy and being safely visible; information and transaction security; and, critically, honesty and transparency. It wouldn't take much effort to turn all four of these scenarios into dystopias. The common element of the malevolent versions of these societies would be easy to spot: widely divergent levels of control over and access to information, especially personal information. The ultimate importance of these scenarios isn't just the technologies they describe, but the societies that they create.

 

So what do these tell us about a Singularity?

 

Across the four Metaverse scenarios, we can see a variety of ways in which the addition of an intelligent system would enhance the audience's experience. Dumb non-player characters and repetitive bots in virtual worlds, for example, might be replaced by virtual people essentially indistinguishable from characters controlled by human users. Efforts to make sense of the massive flows of information in a Mirror World setting would be enormously enhanced with the assistance of sophisticated machine analyst. Augmented Reality environments would thrive with truly intelligent agent systems, knowing what to filter and what to emphasize. In a lifelogging world, an intelligent companion in one's mobile or wearable system would be needed in order to figure out how to index and catalog memories in a personally meaningful way; it's likely that such a system would need to learn how to emulate your own thought processes, becoming a virtual shadow.

 

None of these systems would truly need to be self-aware, self-modifying intelligent machines—but in time, each could lead to that point.

 

But if the potential benefits of these scenarist worlds would be enhanced with intelligent information technology, so too would the dangers. Unfortunately, avoiding dystopian outcomes is a challenge that may be trickier than some may expect—and is one with direct implications for all of our hopes and efforts for bringing about a future that would benefit human civilization, not end it.

 

It starts with a basic premise: software is a human construction. That's obvious when considering code written by hand over empty pizza boxes and stacks of paper coffee cups. But even the closest process we have to entirely computer-crafted software—emergent, evolutionary code—still betrays the presence of a human maker: evolutionary algorithms may have produced the final software, and may even have done so in ways that remain opaque to human observers, but the goals of the evolutionary process, and the selection mechanism that drives the digital evolution towards these goals, are quite clearly of human origin.

 

To put it bluntly, software, like all technologies, is inherently political. Even the most disruptive technologies, the innovations and ideas that can utterly transform society, carry with them the legacies of past decisions, the culture and history of the societies that spawned them. Code inevitably reflects the choices, biases and desires of its creators.

 

This will often be unambiguous and visible, as with digital rights management. It can also be subtle, as with operating system routines written to benefit one application over its competitors (I know some of you reading this are old enough to remember "DOS isn't done 'til Lotus won't run"). Sometimes, code may be written to reflect an even more dubious bias, as with the allegations of voting machines intentionally designed to make election-hacking easy for those in the know. Much of the time, however, the inclusion of software elements reflecting the choices, biases and desires of its creators will be utterly unconscious, the result of what the coders deem obviously right.

 

We can imagine parallel examples of the ways in which metaverse technologies could be shaped by deeply-embedded cultural and political forces: the obvious, such as lifelogging systems that know to not record digitally-watermarked background music and television; the subtle, such as augmented reality filters that give added visibility to sponsors, and make competitors harder to see; the malicious, such as mirror world networks that accelerate the rupture between the information haves and have-nots—or, perhaps more correctly, between the users and the used; and, again and again, the unintended-but-consequential, such as virtual world environments that make it impossible to build an avatar that reflects your real or desired appearance, offering only virtual bodies sprung from the fevered imagination of perpetual adolescents.

 

So too with what we today talk about as a "singularity." The degree to which human software engineers actually get their hands dirty with the nuts & bolts of AI code is secondary to the basic condition that humans will guide the technology's development, making the choices as to which characteristics should be encouraged, which should be suppressed or ignored, and which ones signify that "progress" has been made. Whatever the degree to which post-singularity intelligences would be able to reshape their own minds, we have to remember that the first generation will be our creations, built with interests and abilities based upon our choices, biases and desires.

 

This isn't intrinsically bad; emerging digital minds that reflect the interests of their human creators is a lever that gives us a real chance to make sure that a "singularity" ultimately benefits us. But it holds a real risk. Not that people won't know that there's a bias: we've lived long enough with software bugs and so-called "computer errors" to know not to put complete trust in the pronouncements of what may seem to be digital oracles. The risk comes from not being able to see what that bias might be.

 

Many of us rightly worry about what might happen with "Metaverse" systems that analyze our life logs, that monitor our every step and word, that track our behavior online so as to offer us the safest possible society—or best possible spam. Imagine the risks associated with trusting that when the creators of emerging self- aware systems say that they have our best interests in mind, they mean the same thing by that phrase that we do.

 

For me, the solution is clear. Trust depends upon transparency. Transparency, in turn, requires openness.

 

We need an Open Singularity.

 

At minimum, this means expanding the conversation about the shape that a singularity might take beyond a self-selected group of technologists and philosophers. An "open access" singularity, if you will. Ray Kurzweil's books and lectures are a solid first step, but the public discourse around the singularity concept needs to reflect a wider diversity of opinion and perspective.

 

If the singularity is as likely and as globally, utterly transformative as many here believe, it would be profoundly unethical to make it happen without including all of the stakeholders in the process—and we are all stakeholders in the future.

 

World-altering decisions made without taking our vast array of interests into account are intrinsically flawed, likely fatally so. They would become catalysts for conflicts, potentially even the triggers for some of the "existential threats" that may arise from transformative technologies. Moreover, working to bring in diverse interests has to happen as early in the process as possible. Balancing and managing a global diversity of needs won't be easy, but it will be impossible if democratization is thought of as a bolt-on addition at the end.

 

Democracy is a messy process. It requires give-and-take, and an acknowledgement that efficiency is less important than participation.

 

We may not have an answer now as to how to do this, how to democratize the singularity. If this is the case—and I suspect that it is—then we have added work ahead of us. The people who have embraced the possibility of a singularity should be working at least as hard on making possible a global inclusion of interests as they do on making the singularity itself happen. All of the talk of "friendly AI" and "positive singularities" will be meaningless if the only people who get to decide what that means are the few hundred who read and understand this blog posting.

 

My preferred pathway would be to "open source" the singularity, to bring in the eyes and minds of millions of collaborators to examine and co-create the relevant software and models, seeking out flaws and making the code more broadly reflective of a variety of interests. Such a proposal is not without risks. Accidents will happen, and there will always be those few who wish to do others harm. But the same is true in a world of proprietary interests and abundant secrecy, and those are precisely the conditions that can make effective responses to looming disasters difficult. With an open approach, you have millions of people who know how dangerous technologies work, know the risks that they hold, and are committed to helping to detect, defend and respond to crises. That these are, in Bill Joy's term, "knowledge-enabled" dangers means that knowledge also enables our defense; knowledge, in turn, grows faster as it becomes more widespread. This is not simply speculation; we've seen time and again, from digital security to the global response to influenza, that open access to information-laden risks ultimately makes them more manageable.

 

The Metaverse Roadmap offers a glimpse of what the next decade might hold, but does so recognizing that the futures it describes are not end-points, but transitions. The choices we make today about commonplace tools and everyday technologies will shape what's possible, and what's imaginable, with the generations of technologies to come. If the singularity is in fact near, the fundamental tools of information, collaboration and access will be our best hope for making it happen in a way that spreads its benefits and minimizes its dangers—in short, making it happen in a way that lets us be good ancestors.

 

If we're willing to try, we can create a future, a singularity, that's wise, democratic and sustainable—a future that's open. Open as in transparent. Open as in participatory. Open as in available to all. Open as in filled with an abundance of options.

 

The shape of tomorrow remains in our grasp, and will be determined by the choices we make today. Choose wisely.

ESPN and synthetic worlds

Synthetic worlds are a big deal. Sure you expect me to tell you that. I spend most of time thinking about them and a good portion of my time inhabiting them. But, no really, I told people the Internet was a big deal and I'm telling you now - synthetic world's are a big deal. We will start to see them oozing their way into our lives more and more each day.

Case in point, ESPN, the cable powerhouse that calls itself “Worldwide Leader in Sports,” is looking to extend its domain in synthetic worlds by merging video game graphics with real-life sports anchors. The network, which is owned by the Walt Disney Company, has spent the last year working on a new technology with Electronic Arts, the leading game publisher, that would allow ESPN commentators to interact live with realistic-looking, three-dimensional virtual players as they pontificate about coming matches during broadcasts.

“It’s a way for us to remain relevant,” said John Skipper, ESPN’s executive vice president for content. “We want to make sure we remain connected to lots and lots of fans, and using the language that gamers understand is one way.” Boiled down, the technology, which will make its debut this Sunday on ESPN’s popular “NFL Countdown” program, involves using an Electronic Arts’ title — say Madden NFL 09 — with specialized digital camera equipment in the studio. Presto: Both real and virtual people move around the ESPN set to demonstrate plays and possible situations. And the sports behemoth has more ambitious plans down the road. Instead of using the technology, called EA Sports Virtual Playbook, to tell viewers what to look for before games, ESPN wants to use it in reverse to play the ultimate Monday morning quarterback. Using real information from a game, ESPN anchors could reprogram an actual sequence to show, for example, what would have happened had Peyton Manning thrown right instead of left.

I have written often about how various forms of media — television, the Internet, radio — are all converging. And while television content has converged into video games, Virtual Playbook offers an example of convergence moving in the opposite direction. ESPN is bringing the look and feel of a video game to television for the sake of interactivity, flexibility and visual aid. Television and movie executives have struggled for years to attract young consumers who play video games to more traditional forms of entertainment. At the same time, ESPN is on a mission to tap new areas of growth as it faces challenges in its core operations.

ESPN, three decades old, remains one of the media industry’s biggest gold mines, with successful magazine and Internet operations to complement its suite of cable channels. Analysts estimate that ESPN represents about a quarter of Disney’s annual operating income. But ESPN must also battle the ever-increasing number of Web sites offering sports video and maintain growth as cable operators resist paying higher subscription fees to carry its programming. ESPN charges cable companies about $3.50 a month for each subscriber; the vast majority of cable channels charge well under $1.

So far this year, ESPN’s household rating is up a modest 10 percent compared with the same period last year, largely because of more interest in basketball and the X Games. But household ratings for “Sunday NFL Countdown” have stagnated over the last two years. Ratings for “Monday Night Football” dropped 13 percent in 2007 over the year earlier, according to Nielsen Media Research data. Among ESPN’s programming improvements are an expansion of “SportsCenter,” the network’s flagship program, into daytime and an ambitious push into high-definition programming. But the network is also banking on technological advancements like Virtual Playbook. “We think this will wow our viewers,” said Stephanie Druley, a senior ESPN producer who oversees N.F.L. studio programming. “No one has seen this before.”

EA Sports, a division of Electronic Arts, had a business goal of its own in developing the technology. The unit is actively moving toward a broader sports and entertainment enterprise to speed up growth. Licensing technology is a main part of the strategy. There is also the matter of increasing sales for the division’s bread-and-butter products. Its longtime strategy of churning out annual sequels for its Madden football game — the series is in its 20th sequel — is showing strain. With heavy promotion on ESPN via Virtual Playbook, sales might improve. “We’ve got to keep swimming,” said Peter Moore, president of EA Sports. “Part of that effort is to bring the consumer more into our world. Virtual Playbook is going to give us a lot of opportunity to talk to football fans.”

ESPN will also showcase Virtual Playbook on programs like “Monday Night Countdown,” “NFL Live,” and, occasionally, “SportsCenter.” The network hopes to expand the feature to analysis of other sports like basketball and soccer, said Anthony Bailey, vice president for emerging technology at ESPN. In 2005, Electronic Arts reached a 15-year deal with ESPN to publish sports games that use the ESPN brand name and content. Analysts value the agreement at $850 million to ESPN over the contract’s duration. Neither ESPN nor EA Sports said whether Virtual Playbook was part of the 15-year contract or if the two companies reached a new licensing agreement. An ESPN spokesman said current contracts with the National Football League allow it to use the likenesses of players to enhance its programming.

Integrating human anchors with virtual-reality players might be new, but Virtual Playbook is likely to look familiar to avid ESPN viewers. The network has sometimes shown video game simulations of match-ups that look similar. Still, new media analysts gave ESPN early praise for the effort. “If ESPN wants to gain more exposure to the gamer audience, this seems like a smart way to go about it,” said Michael Dowling, the chief executive of Interpret, a new media consultancy based in Santa Monica, Calif. “It adds an element of coolness and realism that gamers really want.”

Anyway, I think is good news. I am waiting for CNN to adopt a similar approach so they can render up non-sports avatar driven events in a similar fashion. I don't believe that is far off.

"...And now the news for Avatars."

CNN has just joined Reuters and the BBC by opening a news outlet in Second Life (SL). The Cable News Network will create a variation of its "i-Reports," which is what CNN terms its method for the average citizen to contribute eyewitness reports. CNN will equip Second Life citizens with kits enabling them to transmit copy and photos. In-world, you will be able to get the latest news via kiosks scattered throughout the synthetic landscape.

second_life_logo.gifAs I have said, there have been many lessons for marketers to learn as they take their first steps into synthetic worlds. American Apparel and Starwood Hotels, two brands that established a presence in Second Life, have closed them down. Visitors at other marketing spots are sparse. As for whether Second Life will generate news events, I would not be worried. Many of the visitors to the synthetic world include regular TV news making personalities like Bono, Elvis Costello, Judge Richard Posner, Mia Farrow, U..S. military strategist Thomas Barnett and Newt Gingrich to name a few.

Meanwhile, research company The Yankee Group recently produced a report on Second Life detailing a "pronounced" decline in usage. Senior analyst Christopher Collins said Second Life's claims of more than 10 million avatars skewed perceptions of its popularity. "The people that are active there tend to be very active," Collins said. On the other hand, many users come to Second Life and are "frustrated, confused and don't find it immediately compelling," he added.

I am not certain of the total value of the Yankee Group's report. I don't think they have caused any serious revelations on the part of academics or financial analysts. Today, synthetic worlds serve certain social media needs. And media outlets may be over-hyping audience expectations for their experiences in these destinations. The real value of Synthetic world's have yet to be realized from a socio-economic and cultural perspective. But, the presence of these news outlets means more adoption and as a result the further convergence of meaning for events in the both the synthetic and natural world. Cool stuff.