Interaction between Real and Virtual Worlds

Virtual reality (VR) technology associated for a long time solely with video games, has since experienced a major boom, particularly in the field of training. The «democratisation» of VR headsets is no stranger to this. The number of headsets sold has exploded from 5 million units in 2014 to 68 million in 2020, their cost has dropped and the technology itself has evolved. We are now talking about immersive technologies including virtual, augmented and mixed realities (VR/AR/MR). UTC has been a pioneer since it introduced, as early as 2001, teaching in virtual reality and launched, within its Heudiasyc laboratory, research on both the fundamental and application levels. The interaction between the real and virtual worlds opens up immense fields of application, particularly in relation to robotics. For example, we can interact with a drone that maps the damage caused by a natural disaster in places that have become inaccessible. Obviously, these new possibilities can be used for malicious purposes, and this raises several ethical issues. UTC’s academics are aware of this.

Interaction between Real and Virtual Worlds

From real to virtual drones

Pedro Castillo is a CNRS research scientist who works at the UTV-Heudiasyc Lab. and a member of the Robotic Systems in Interaction (SyRI) team. He is specialized in automatic control applied to robotics. His research focuses, in particular, on the automatic control of UAVs, autonomous UAVs but also, more recently, virtual UAVs.

Arriving from Mexico with a scholarship, Pedro Castillo started a PhD thesis, in 2000, in automatic control, more particularly on the automatic control of drones, at UTC. His thesis won him the national prize for the best thesis in automatic control in early 2004. During these years, he did a series of post-docs in the United States at Massachusetts Institute of Technology (MIT), in Australia and in Spain, then applied for a position with the CNRS, which he joined in 2005, appointed to the Heudiasyc mixed research unit, where he continued his research on the control of miniature drones. "As early as 2002, during my thesis, we conducted the initial tests. We were the first team in France to work on this subject at that time and one of the first to develop an autonomous four rotor drone," he explains.

This explains the recognition that the laboratory enjoys in this field, both in terms of theoretical and experimental research. "UTC-Heudiasyc is known for having developed fundamental platforms. And it was during my thesis that we started to develop experimental platforms dedicated to flight modes in order to validate the theoretical research that we were carrying out. As early as 2005, we worked on setting up a common platform for the validation of aerial drone control systems, which was completed in 2009", he explains.

While many researchers are placing their bets on autonomous drones, Heudiasyc has adopted a different gamble. "We realised that we cannot leave all the work to a drone. There may be situations that it will not be able to handle. In this case, the human must be able to take over. In robotics, we talk about a control loop where the human can interact with the robot at any time," says Pedro Castillo.

Until recently, in the field of human-machine interaction, rather classical approaches have dominated. Those that use joysticks or remote operation, which, thanks to feedback from the system to the operator, allow the control of a remote robot.

The idea of introducing virtual reality? "It's to introduce visual and audio feedback. In a word: to see what the robot sees by equipping it with cameras and to be able to hear, for example, in the event of a motor problem. This will make it easier to control and therefore navigate the drone," he explains. But Pedro Castillo and Indira Thouvenin decided to go further and explore a new theme: virtual robotics. "We decided to represent our aerial drone test room in the Cave, i.e., with highly immersive technology. We also created a small virtual drone that can be manipulated, which can be given different trajectories and carry out different missions. It is a sort of assistant to the real drone, since the latter will then carry out all the tasks that the operator indicates to the virtual drone, "explains Pedro Castillo.

Concrete applications? "We are interested in civil applications and there are many. We can mention the inspection of buildings, for example, or the occurrence of any natural disaster. There may be inaccessible places and drones allow us to take stock of the material or human damage and act quickly and in the right place," he concludes.