The way we interact with software is evolving. Advances in the fields of Artificial Intelligence (AI) and Machine Learning (ML) are merging with rapidly evolving software interfaces such as Virtual Reality (VR) and Augmented Reality (AR), creating a paradigm shift in the way we use and experience the digital realm. We’re now entering an era of ‘Mixed Reality’ (MR). This technology is set to impact the way we work at Deloitte – improving connection across locations and changing the way we share information. By 2019, some 20% of enterprise organisations will have experimented with Virtual or Augmented Reality technology. Deloitte Digital has already built deep experience across a number of projects, including the ANZ Virtual Garden installation and the 2017 Australian Open ANZ tennis activation, ANZ Breakpoint. At a recent Innovating with Impact* session we were joined by Fabio Zambetta, Associate Professor at RMIT. An expert in the field of Mixed Reality, he’s fascinated by the way we interact with software. Dr Zambetta explained how Artificial Intelligence and Mixed Reality technologies will revolutionise the way software works, and outlined the impact this may have on our lives. We debrief with him below, reflecting back on the session. Fabio, tell us a bit about yourself? I am an Associate Professor at RMIT University where I research on Artificial Intelligence, Mixed Reality and Computer Graphics – and collaborate with colleagues in Computer Science, Design, Neuroscience and Engineering. My research includes the use of machine learning techniques to solve complex navigation problems in videogames or on physical devices (e.g. drones). I have also been teaching games development courses at RMIT for more than 13 years. What will be the human impact of Mixed Reality? Software is a vastly intangible artefact, which has been confined to flat, 2D screens for many decades. The advent of devices such as the Microsoft HoloLens has allowed us to leverage AI algorithms and Mixed Reality visualization to make software “pop out” of the screen – and take a place in familiar spaces. This will have a very profound impact on our lives – both work and leisure. The technology also creates some interesting legal and ethical questions. Mixed Reality is not a completely novel concept. As human beings we have always used our imagination to create “alternate” realities. Our own perception of the world is indeed a form of mixed reality, as our brain approximates the “true” form of reality (a very interesting philosophical question in itself), super imposing our dreams and visions on top of it. However, it is now possible to make some of those dreams and visions more tangible – by creating 3D characters who can converse with humans, or by visualizing data in 3D in our own office space. This was indeed unthinkable until recently and probably the domain of sci-fi show such as Star Trek, which interestingly featured a “Holodeck” machine capable of creating holographic experiences out of thin air. Coming back to the real world though, what will gradually happen is that our conceptualization of reality will start to shift, integrating more and more accurate “virtual” content in our daily lives. This is not new either, as the introduction of the web first and of smart phones second has indeed made our lives much more virtual than they used to be. Again, this creates lots of opportunities as well as raising ethical, health and moral concerns, which will need to be addressed properly. How will Mixed Reality impact the way we work? Going back to the original point of software popping out of a 2D screen, it is worth noting that operating systems for personal computers have always adopted spatial metaphors. We speak of a “desktop”, a “file”, of “folders”, and obviously “windows”. My prediction is that once some of the core technical and engineering obstacles are overcome, we will feel that interacting in our workplace will be eerily comfortable. We will not need to manage data on an artificial, flat desktop on a small screen but rather move virtual representations of files and folders on our physical desk in the office! One problem stemming from that, is that all our current UI/UX design methodologies are suddenly a bad fit. We are currently in a primordial stage of defining a design language for mixed reality and we do not know what we do not know. However, cracking hard problems has always been attractive to me and the challenge makes working in this area even more exciting! What does Mixed Reality mean for businesses? Current forecasts indicate that business arising from MR technology will be worth several dozen AU$ billion by early 2020. In some sense, this should not surprise us as the Internet has generated much business and created new forms of communication and services. A lot of the early work in MR will be around creating more and easier to use technology, which in turn will facilitate the creation of new services and applications. What will be interesting however is that, technical implications aside, every industry leveraging training will be able to make use of very immersive forms of simulation, at an unprecedented level of detail. Also, virtual assistants will reach new heights and provide more efficiency in every industry, including health, law, government and indeed anywhere customer support is required. Finally, the entertainment industry will witness more blurred lines between episodic movie series, videogames and theatre, giving rise to novel forms of storytelling that will be played, in real 3D in front of our eyes, in our own living rooms. Peter Bauld, Deloitte Digital Reality lead gives his point of view on the VR and AR world: “Virtual Reality is starting to get huge traction within the learning and training arena providing users the ability to fully immerse themselves and learn at a much faster speed through proven increased retention rates. Augmented Reality is still about 4 to 5 years from mainstream adoption but companies like Magic Leap and other new technology developments will see it fly ahead of VR in the coming years having more commercial use in a range of industries.” *About Innovating with Impact Innovating with Impact is a monthly event hosted by Deloitte’s Innovation team. The sessions provide a forum for Deloitte staff to hear from leading innovators across a broad spectrum of fields, as they present inspiring stories and mind-expanding ideas.