Todd Richmond is the director of advanced prototypes at the University of Southern California Institute for Creative Technologies (ICT), as well as a member of the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems Mixed Reality Committee and IEEE Future Directions Symbiotic Autonomous Systems Initiative. He also is a musician and visual artist. Todd’s areas of work are far-reaching: emerging, disruptive technologies and their implications and applications for training, learning and operations; future environments for communication and collaboration; immersive technologies; interactive education; visualization, and analytics. He works closely with the USC School of Cinematic Arts and has studied with Grammy-winning bassist Victor Wooten.
How do you describe the long-term impact of mixed reality on humanity?
Todd Richmond: Mixed reality is going to change the way that we interact with each other; it’s going to change the way that we interact with information, and it’s going to change the way that we live our lives.
Think about how the telephone changed society and what industries the telephone changed. Think about how television changed industries, and how the internet changed society. Mixed reality will have a similarly profound effect because it is a new medium for communication and collaboration. It starts to blur the lines between the physical and the virtual, and that means that mixed reality starts to redefine reality for humans and what it means to be human.
Given this, how do you assess the ethical considerations around the rollout of mixed reality?
Todd Richmond: I think that any new technology has an ethical component. Look back to Robert Oppenheimer’s struggles when the Manhattan Project was successful and they detonated the first atomic bomb. Virtual humans and artificial intelligence (AI) is the Manhattan Project of the 21st century. The power and the breadth of AI and virtualized spaces are almost incalculable. Mixed reality is really about trying to bring together virtual things with physical things and alter the human experience. You can’t alter the human experience without considering the ethical ramifications. We can either have those discussions up front, or we can wait until something bad happens and then react to it.
Mixed reality enables us to be places and do things that we couldn’t otherwise do. The problem is that technology is agnostic, and I firmly believe in yin-yang. So if something gives you good capabilities, then it’s going to also give you bad capabilities. Mixed reality will enable healthcare to reach people who have never been reached before. It will allow people to connect in ways that they never connected before. It will also allow people to harass people in ways that they never could before, and it will allow people to torture people in ways that they never could before. It’s important that we have the conversation about both the light and the dark sides of technologies.
Let’s talk through a couple of specific areas of life—what are some of the ways that mixed reality will impact industry, for example?
Todd Richmond: Augmented reality is already being used on production floors and manufacturing floors, but there’s going to be an even more profound shift. We’ve been working on a process called mixed-reality prototyping, where we’re trying to understand human-machine teaming. Specifically, we’re looking at how autonomous drones can react to human behavior. With this, we’ve been looking at how to get people who are at different locations to work as if they are all at the same place. When you’re trying to figure out autonomous algorithms for drones following human beings, you don’t want to work with real drones and real people. So, we’ve made virtual versions where we can solve some of those problems and start to mix reality. This idea of prototyping in a virtual environment and then mixing reality is the way industry is going to design products 10, 20 years from now. This allows the end user to be a part of the virtual environment and part of the design process before you ever build a physical prototype. Mixed-reality prototyping is going to radically transform the way that products are designed, manufactured and sold.
How might mixed reality benefit healthcare?
Todd Richmond: Mixed reality is going to allow us to diagnose in ways we’ve never been able to do before. Wearable sensors could allow real-time telemetry to be shared with doctors. Virtual physicians could basically allow your physician to live in your phone, with 24/7 access. So, if you’re freaking out at 3 in the morning, you don’t have call and get your doctor’s answering service. Instead, you can interact with a virtual version of that doctor, who has the data from your wearable sensors and can help to determine if you actually need to go to the emergency room.
Despite all of these advancements, the biggest problem in healthcare remains how to change patient behavior—how do you get the patient to adhere to their treatment regimen? So, people are already working on applications where a patient wears a device which knows what medications a person is supposed to be on, can see the pill containers to know when they’re taking it and reminds them when they do not.
The one thing that we still haven’t cracked is how we truly engage the patient. To do this, we must tell a story. Humans still respond to narrative. You can collect all the data in the world and display it in bar graphs and pie charts, but people need a story. Traditional narrative structures created for film and the flat screen do not work in virtual reality and mixed reality. We have to come up with new ways to tell stories, so we’ve been working closely with the cinema school to help us with this challenge.
What benefit does IEEE provide in preparing for the implications of mixed reality?
Todd Richmond: IEEE helps to serve as a neutral, third-party ground where industry, academia and government can work together to understand the implications of mixed reality.
For me, it’s always about applications and implications. What the commercial sector usually focuses on are the applications, and they don’t consider the implications. The problem is that digital and mixed reality is going to become so pervasive that the implications will be incredibly profound and we need to get out in front of those discussions.
IEEE tries to stay ahead of the technology curve by putting out position papers and convening groups. The leadership of IEEE really is about trying to occupy the space between industry, academia and government and being somewhat of a glue to bring those things together. They ask questions that industry either won’t or aren’t able to ask.
Standards can have a lot more power than they used to, and it’s because of the breadth, depth and reach that mixed reality is going to have. I think one of our challenges moving forward is how IEEE will be a global force for mixed reality. How do we craft platforms and organizational structures that enable conversations so that we all can move forward with development in a thoughtful way?
Policy and standards are important, and, unfortunately, they oftentimes lag behind technology adoption. Drones are a perfect example. As drones became commoditized, policy really didn’t get rendered until there were some catastrophic near-misses. Finding the balance between standards and policies, and the needs of commerce is a very difficult calculation.
For more information on these emerging technology areas, please watch “Mixed Reality—The Future of Our World,” see The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems’ “Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems” and get involved with IEEE Future Directions and the IEEE Standards Association (IEEE SA). Also, please visit the IEEE SA at Augmented World Expo (AWE).