What Is Spatial Audio? A Deep Dive.
Spatial audio is an immersive technology for experiencing sound.
Discover the history, science, and tools of this field.
Since the history of recorded sound, humans have constantly been trying to expand the realism of listening experiences.
From mono to stereo, to surround sound and beyond - listening setups and audio formats have gradually evolved, enabling new means of creation and expression.
The field of spatial audio is surprisingly deep and diverse, with many formats, setups, speakers, and types to explore, and thanks to the power of technology today, it achieves highly realistic results.
This guide explains what spatial audio is, dives into the history, and explores the best options available for creating and experiencing this new format of immersive listening.
TL;DR - Spatial Audio
Spatial audio is a technology designed to make listening experiences more immersive and realistic by virtually placing sounds in different directions and distances from the head.
Unlike stereo, spatialized audio uses advanced technology to create three-dimensional soundscapes.
Many platforms (like Netflix, Apple Music, VR, video games, and AudioCube) offer spatial audio modes.
AudioCube is a new Windows/Mac app designed to create and experience spatial audio.
Defined: What is Spatial Audio?
Spatial audio is a technology that aims to create a more immersive and realistic listening experience, making it seem as if the sound is coming from different directions and distances.
Compared to the common stereo format, which is limited to two audio channels (left and right), spatial audio uses advanced processing to create fully 3D soundscapes - often giving the user the impression that they are inside the sonic environment.
Spatial audio is more immersive and realistic than traditional audio for the following reasons:
Directionality: Spatial audio provides a sense of directionality, meaning you can perceive sounds as coming from specific locations around you - not just from the left or right but also from front, back, above, and below.
Distance and Depth: It can simulate distance and depth, making some sounds seem closer and others farther away, which adds a layer of realism to the audio experience.
Acoustic Simulation: Many forms of Spatial Audio simulate the acoustic environment of the presented scene, including reverb, reflections, and absorption - which help to create a more realistic experience.
Movement Synchronization: When you move your head or your device (in the case of smartphones or VR headsets with dynamic head tracking), spatial audio can adjust the sound in real time to match your new orientation or position, enhancing the sense of immersion.
Typically you need specific hardware, like multi-speaker systems, or special headphones, for the best spatial audio experience. You'll also need some to decode the spatial audio signals, depending on the format used. However, with software like AudioCube, you can create and experience spatial audio on a stereo setup.
The History of Spatial Audio and Audio Formats
By looking at the history of spatial audio and related audio formats, you can get a good idea of what this technology is, and how it differs from its ancestors.
Audio formats have evolved drastically over time, with each successive generation offering a richer and more realistic audio experience.
Each step in this evolution brought a new dimension to how we experience sound, and opened up new doors for audio formats.
1) Monaural (Mono) Sound
Era: Early 20th century
Characteristics: Mono sound is the earliest form of audio recording and playback. It uses a single channel of audio, meaning all sounds are mixed together and played through one speaker or set of headphones. The sound appears to come from a single point.
Usage: Mono was standard for radio broadcasts, early films, and initial music recordings. It is still common in some cheaper smartphones.
2) Stereophonic (Stereo) Sound
Era: Popularized in the 1950s
Characteristics: Stereo sound uses two separate audio channels (left and right) to create a more natural listening experience. It provides a sense of width and space, allowing listeners to perceive the directionality of sounds, though primarily in the horizontal plane.
Usage: Stereo became the standard for music recordings, FM radio broadcasting, and home audio systems. It's now the most common format for TVs, laptops, and other consumer audio devices.
3) Quadraphonic Sound
Era: 1970s
Characteristics: An early attempt at surround sound, using four audio channels (front left, front right, rear left, rear right) to create a more immersive and encompassing sound field than stereo.
Usage: Limited to some music recordings and specialized audio setups. It didn't gain widespread popularity due to technical complexities and a lack of standardized formats.
4) Ambisonics
Era: Conceptualized in the 1970s, refined, and more broadly applied in the 21st century.
Characteristics: Ambisonics is a full-sphere surround sound format that goes beyond traditional channel-based audio. It captures and replays sound from all directions, including above and below the listener. The technique uses a set of audio signals called B-format, which represent the sound field at a point in space. This allows for a more accurate and immersive reproduction of the sound environment. Ambisonics is highly flexible and scalable, supporting various playback configurations from headphones to multi-speaker arrays.
Usage: Initially used in experimental music and sound installations, Ambisonics has found wider applications with the advent of VR and 360-degree video, where audio can adapt to the listener's orientation. It's also used in professional audio production, spatial audio research, and increasingly in consumer applications for a more immersive listening experience.
5) Surround Sound (5.1, 7.1, etc.)
Era: Gained traction in the 1980s and 1990s
Characteristics: Surround sound systems use multiple audio channels (5, 7, or more) and speakers placed around the listener to create an enveloping sound experience. It adds depth and a three-dimensional aspect to audio, allowing sounds to come from all directions, including behind the listener.
Usage: Became a standard for home theater systems, cinemas, and professional audio production, especially for movies and games.
6) Dolby Atmos, DTS:X (Object-Based Spatial Audio)
Era: 2010s onwards
Characteristics: These technologies represent a leap from channel-based to object-based audio. Sounds can be placed and moved in a three-dimensional space, including height channels, providing a more realistic and immersive soundscape.
Usage: Used in cinemas, home theaters, music production, and streaming services. It's also increasingly used in headphones for personal listening, often through virtualization techniques.
7) Binaural and Spatial Audio
Era: Development through the 20th century, but popularized and widely accessible in the 2010s and 2020s.
Characteristics: Binaural audio uses two microphones to simulate human hearing, creating a 3D stereo sound sensation. Spatial audio takes this further by adding advanced processing to simulate how sound behaves in a three-dimensional environment, including the vertical axis.
Usage: Used in VR/AR, gaming, and with certain music and video streaming services. It's especially popular in headphone-based listening experiences where head-tracking can be used to adjust the audio dynamically as the listener's head moves.
From mono to spatial audio, each format brought a new dimension to the listening experience, transforming how we create, share, and enjoy sound.
The evolution of audio formats reflects the ongoing pursuit of more immersive and lifelike soundscapes, using technological advancements to bring us closer to how we naturally hear and perceive the world around us.
Where is Spatial Audio Used?
Spatial audio is found in a range of services. One of the most obvious is in cinemas with surround sound (and the fancier Dolby Atmos systems), although you'll find spatial audio support in a whole bunch of apps, devices, and platforms.
Cinema and Home Theater:
Movies are increasingly being produced with spatial audio formats like Dolby Atmos and DTS:X, offering viewers an immersive audio experience.
Home theater systems (even soundbars) are adopting these formats, allowing users to enjoy cinema-like surround sound at home.
Music Production and Streaming Services:
Artists and producers are creating music tracks in spatial audio formats to provide a more immersive listening experience.
Streaming services like Apple Music, Tidal, and Amazon Music offer spatial audio tracks, often labeled as Dolby Atmos or Sony 360 Reality Audio. Apple Spatial Audio is a new feature found in some Apple devices, that creates a personalized spatial audio experience.
Gaming:
Video games use spatial audio to create a more immersive and realistic gaming environment, helping players pinpoint the direction of sounds like footsteps or gunfire.
Gaming consoles and PCs, along with gaming headphones, often support spatial audio technologies to enhance the gaming experience.
Virtual Reality (VR) and Augmented Reality (AR):
Spatial audio is crucial in VR and AR to create a believable and immersive environment, matching the audio experience with the visual one.
VR content, including games, educational content, and simulations, often uses binaural audio or Ambisonics for a realistic audio experience.
Concerts and Live Performances:
Some live performances and concerts use spatial audio technology to provide a unique and immersive listening experience for the audience.
Headphones and Personal Audio Devices:
Many modern headphones and earbuds are equipped with spatial audio capabilities, offering an immersive listening experience for music, movies, and games. these compatible headphones usually have an option to turn spatial audio on. As do some wireless speakers.
Devices like the Apple AirPods Pro and Max, some Beats Headphones, and certain models from Sony, Bose, and others, support spatial audio. Even some devices like Apple TV and certain iPad Mini models offer this feature.
These are just some of the most relevant situations where spatial audio technology is used.
The use of spatial audio is becoming more popular as people learn how powerful it is. Spatial audio is the future of sound, and will soon become the norm - just how stereo is today.
How Does Spatial Audio Work? The Deep Science
Spatial audio uses psychoacoustic principles and advanced signal-processing techniques to create the illusion of a three-dimensional sound environment.
It operates on the premise that the human auditory system interprets sounds differently based on their spatial characteristics, such as the direction of arrival, the time delay between ears (interaural time differences, ITD), and the level difference between ears (interaural level differences, ILD).
These cues, along with the spectral information altered by the head-related transfer function (HRTF), provide the brain with the necessary data to localize sound sources in three-dimensional space.
In spatial audio systems, sounds are encoded with spatial information that can be channel-based, object-based, or scene-based (I explain these types in more detail below).
In channel-based systems like 5.1 or 7.1 surround sound, audio is played through multiple speakers arranged around the listener.
Object-based systems, such as Dolby Atmos and DTS:X, dynamically render sound objects in a 3D space, adapting to the speaker setup for optimal spatial effect.
Scene-based systems like Ambisonics capture and reproduce the entire sound field, allowing for high precision in sound source localization and movement.
The synthesis of these techniques, underpinned by psychoacoustic principles, allows spatial audio to deliver a more natural, lifelike, and immersive auditory experience.
Experience Spatial Audio Software: AudioCube
If you want to experience how spatial audio works and start to create and experiment with sound in a virtual sonic environment - AudioCube is the best place to start.
AudioCube.app is an emerging spatial audio platform. It's still in the early days of development (mostly due to being made by a single guy, me, Noah), but it's posed to become an exciting tool in this field.
It's a 3D DAW (digital audio workstation), that gives you a new way of interacting with sound and music in a 3D space. You can spatialize and audio easily, by importing it and adding it to a 3D audio device. You can move the sources around however you choose, and move the listening position freely.
It features a powerful mixer with volume control, eq, and effects for each channel, and a mastering rack.
See it in action with this video!
AudioCube.app is currently in closed beta. If you want to test it, or sign up for early access, please register with the form or email below.
