Using VR, AR and MR in simulations and training

The industry is ready to accept with open arms virtual reality (VR) and its immediate relatives, augmented reality (AR), and the newest promising direction, mixed reality (MR) or, according to another terminology, blended reality (this technology includes elements of augmented reality in addition to physical presence). However, is this technology really needed in training? Does it give something new except for the wow effect? Is it necessary to move technology just for the sake of technology? The answer is obvious - yes.

When designing training systems, it is necessary to take into account which technologies have matured to their use and how they can be applied in the educational process. Why do you need a full-sized model or even a real sample, what can be done on a simulator using AR and VR, what is enough for a simple iPad? The choice of technology is part of the design of the training systems. And VR is just another tool in the developers bank.

Experiments with the use of virtual reality in training and production have been going on for quite some time. Specialists from Oculus Rift and surgeons from the Children's Hospital in Los Angeles created a simulator that allows doctors to work out the skills to save newborns with anaphylactic shock. Western construction companies are widely using VR for training workers in areas such as safety instructions, working on a loader and other construction equipment, working on scaffolding and welding. Airbus uses a mixed reality in the production of the A350 XWB and A380 to assist in the assembly of the fuselage; according to them, after the introduction of this technology, the time required to check how 80'000 connecting brackets are installed has been reduced from three weeks to three days. Google compared the use of VR and training videos when training a barista - as a result, VR won in all respects even here.

Nevertheless, it is possible to say that technology has gained recognition only when it is widely used in military simulations and training, which are currently the most numerous and advanced. Atul Patel, director of technology and innovation at Lockheed Martin, believes that the VR market will grow extremely aggressively over the next 5-10 years. The entire VR-AR-MR market is estimated at $ 60-120 billion over the next 5 years, of which more than $ 5-10 billion of this amount may fall to military applications.

Virtual reality is most closely associated with the traditional simulation for land, sea and air technology. The entire image is fully rendered on a computer, and most often displayed on virtual reality glasses, or, more rarely, projected on a panoramic screen. At the moment, the most popular glasses are Oculus Rift and Samsung Gear2, sometimes companies use their own development.

A startup from London called Immerse and defense company QinetiQ created the first "multiplayer" VR-simulator of submarine crews for the Royal Navy. It is a detailed recreated interior of a submarine with working displays and instruments for training and practicing actions in emergency situations without putting the crew at risk.

Augmented and mixed reality has so far been used less frequently. The most interesting here is the MR-development from Rockwell Collins, called Coalescence. This system is the first one that can truly convey sensations when training in virtual reality. This mixed system uses specially modified real equipment and equipment, instead of gloves traditional in such systems or clothes with vibration motors, to create tactile sensations, and is aimed at training in the field of handling weapons and controlling equipment.

From the technical side, the equipment for the new "realities" is still in its infancy. Like any emerging technology, VR currently has flaws that seriously limit its use. Some of them are purely technical: narrow field of view, low image quality, lack of tactile sensations. Before VR glasses become a mass standard for training and simulations, you need to bring the image resolution to 4K, the refresh rate to 60 Hz and more, and the field of view to at least 90 degrees.

Another problem is the cables and the weight of the helmet / glasses. Ideally, all this should be wireless, so that nothing restricts the movement of the student and does not confuse. It is quite natural that the weight of the equipment is a big problem, because due to the strain on the neck, the time of use can be seriously limited.

However, the main problem is nausea and dizziness when using virtual reality. Oculus Rift even created a special “comfort scale” for applications, so that users can represent the degree of discomfort.

This problem is similar to the one that manifested 50 years ago with the appearance of the first flight simulators. This is due to the characteristics of the organism, and more precisely, the vestibular apparatus of a person. If a rapidly changing picture flashes before your eyes, but the body remains stationary, then the effect of motion sickness arises. This side effect can be overcome with the help of "teleportation" inside the game to reduce the number of movements, however, for those simulators where the simulation should be as seamless as possible, this option is not suitable.

Another physiological problem is the problem of eye accommodation conflict.

In VR-eye glasses, one flat image with one focus point. This is not the same as in the real world, where our eyes take the same image from many different angles and combine them together to form a scene with more depth, allowing our eyes to move freely. This leads to a double image, and, much more unpleasantly, to severe eye strain and headache.

However, a solution to this problem can be found. In August, scientists in the image processing group at Stanford University, led by Dr. Gordon Wetzstein, said they had found a workaround. The solution is to recreate the natural visible light source in the VR headset, simulating the use of multiple perspectives of the same image. Ultimately, the image becomes similar to a hologram, which is less burdensome for the muscles and accommodation of the eyes.