Interaction between Real and Virtual Worlds

Virtual reality (VR) technology associated for a long time solely with video games, has since experienced a major boom, particularly in the field of training. The «democratisation» of VR headsets is no stranger to this. The number of headsets sold has exploded from 5 million units in 2014 to 68 million in 2020, their cost has dropped and the technology itself has evolved. We are now talking about immersive technologies including virtual, augmented and mixed realities (VR/AR/MR). UTC has been a pioneer since it introduced, as early as 2001, teaching in virtual reality and launched, within its Heudiasyc laboratory, research on both the fundamental and application levels. The interaction between the real and virtual worlds opens up immense fields of application, particularly in relation to robotics. For example, we can interact with a drone that maps the damage caused by a natural disaster in places that have become inaccessible. Obviously, these new possibilities can be used for malicious purposes, and this raises several ethical issues. UTC’s academics are aware of this.

Interaction between Real and Virtual Worlds

Kiva - training in gestural expertise

Sébastien Destercke is a CNRS research scientist and head of the Knowledge, Uncertainty and Data (CID) team at UTCHeudiasyc. His field of research concerns modelling and reasoning under uncertain conditions, in particular in the presence of high or severe levels of uncertainty.

Concretely? "Strong uncertainties aredefined as missing or imprecise data, poor or qualitative information. What are the underlying ideas? "The main idea is to model this type of information in a mathematical language in order to carry out reasoning tasks. This can be automatic learning, i.e., learning from examples, or making decisions in the face of uncertainty, "explains Sébastien Destercke.

Is there a transition to virtual reality? "At Heudiasyc, we have strong expertise in theories that generalise probabilities, such as theories of evidence or imprecise probabilities. These are rich mathematical languages that allow uncertainty and incompleteness of information to be modelled in a very accurate manner. Such expressiveness is particularly useful in certain applications of virtual reality, especially in the case of designing training aids. Among the uncertainties requiring fine modelling, we can cite those concerning the learner's competence profile or even his emotional states. Taking uncertainty into account in the reasoning will make it possible to better adapt training scenarios, which can be better personalised for each profile," he adds.

This work on uncertainty has, among other things, led to the Kiva project, which is built around an "informed virtual environment for training in technical gestures in the field of aluminium cylinder head manufacturing"; a project that won an award in the "Training and Education" category at the Laval Virtual trade fair. What is Kiva's objective? "We focused on a special training gesture: how to blow impurities off the surface of aluminium alloy cylinder heads that have just been cast? The aim is get the trainee to reproduce the 'expert' gesture. However, in this situation, we have at least two sources of uncertainty: on the one hand, the "expert" gesture can change from one expert to another and, also, the recognition of the gesture which cannot be done perfectly. The trainee is equipped with sensors; he or she will make movements that are not necessarily regular or collected in a continuous manner. This gives us partial information on the basis of which we must define the recognition of the trainee's gesture and try to measure how this gesture matches the 'expert' gesture. Now, in the system that will guide the learner, it is necessary to include these sources of uncertainty" , adds Sébastien Destercke.

"An 'expert' gesture that the learner is asked to reproduce in a Cave. The use of virtual reality in companies for training purposes is recent. Until now, it has often been limited to improving the ergonomics of workstations, where issues related to man-machine interaction are less critical. With these training objectives, they have become fundamental," he concludes.