“Gosh, a giant bug has flown over me”: SDK release for surround sound
Google introduced the SDK Resonance Audio - a tool for sound design of games, 360-degree video, VR and AR applications.
This is not the first development of the company in this area. In January 2016, the Google Cardboard SDK with surround sound was released. His revised version was included in the Daydream 2.0 VR platform, which is currently open to third-party developers.
A novelty called “Resonance Audio” brings together Google's work in this area and allows you to implement them on a variety of platforms, including mobile.
Below we tell about the features of Resonance Audio.
Photo by Jennifer Boyer CC BY
What does it give
- The effect of full presence . The average viewing angle of human vision is 160-200 degrees, but with the help of hearing capabilities a person can form a complete image of space for himself. By synchronizing virtual vision and virtual sound, it will be possible to achieve a greater effect of presence in virtual reality.
- Economical use of CPU . Ambisonics technology works here - a 3D version of Mid-Side recording. In simple terms, it saves computational resources in the process of "mixing" hundreds of virtual sound sources. In addition, for Unity, it is possible to precompute the reverb effect depending on the acoustic properties of the virtual environment.
- Cross platform Resonance Audio runs on Android, iOS, Windows, MacOS, Linux operating systems; on Unity and Unreal Engine; with FMOD audio library; Wwise software package; digital sound workstations. Google also provides APIs for C / C ++, Java and Objective-C.
- Formation of complex sound environments . Using the SDK, you can work with a number of parameters that affect the spatial perception of sounds. For example, set the direction of propagation of the sound wave. The developers bring the situation with the sound source in the form of a virtual guitar: it will sound louder if your game character stands before it and is quieter if behind it.
In addition, the SDK provides for a different perception of sound depending on environmental conditions (for example, in an open and closed space). Through the use of these features, developers can create more realistic scenes.
How Resonance Audio Works
The technology allows you to set the position of the sound source along with the nature of its distribution and simulates the interaction of sound waves with virtual objects. To do this, Resonance Audio introduces three types of components of the general perception of a sound effect.
- Direct sound - comes from the source to the listener directly without interacting with obstacles. It has the greatest intensity in comparison with other components and calms down as it moves away from the source.
- Primary reflections from surfaces - have a lower intensity, which also depends on the surface properties. They allow you to estimate the size and shape of the virtual space.
- This is followed by secondary and subsequent reflections , which arrive with increasing density and overlap with direct sound. They build a more complete picture of the surrounding space.
As Chris Hegstrom, sound designer at Microsoft HoloLens, emphasizes , the human brain processes sound information faster than visual information. The instinct of self-preservation taught a person to determine the distance to the sound source, and on this principle is built the process of forming the presence effect, for which the audio component is largely responsible for:
The effect of presence is the difference between “Wow, in this game is a very cool animation of a huge beetle” and “Gosh, a giant beetle flew over me!”
Using the capabilities of Resonance Audio ( video ), a person immersed in the virtual world forms a kind of "map" of the space in which he is located. And the person determines the approximate location of objects that he cannot see with the help of hearing.
What else do you need to know
The importance of work on surround sound technologies is confirmed by experiments, one of which was conducted by G'Audio Lab. People were given two demo versions of the game - one with surround sound and the other with normal. After the completion of the gameplay, respondents openly said that they did not even think that the sound could make such a big difference in the sensations from the gameplay.
Joel Susal, the head of the Dolby division, which deals with virtual and augmented reality, reminds us that it is realistic sound effects like the sharp creaking of a door or the sound of approaching steps that make us instinctively turn our heads to see the source of the sound and find out that - so it happens. Surround sound technology will enrich the gameplay of VR games and expand the area of interaction with the gaming environment.
Surround sound can be used not only in games and movies, but also in music videos , during live VR broadcasts of concerts and sports matches, so that the person, being at home, feels like a part of the event. Surround sound will help you to feel what is not so easy to feel even in the real world - for example, not just to go to a concert, but to be right on the stage, next to your favorite performer.
Other materials from our “Hi-Fi World”:
All Articles