Have you ever been in a cafe or clothing store when a song comes on that you just have to know the name of? You pull out your phone, open Shazam, and in seconds you have the song title, artist, and options to save it or open it in Spotify. But how does the magic of Shazam work? In this post, we’ll unpack the technology behind the popular music identification app so you can better understand how it’s able to detect songs with just a short sample.
Shazam is an application that can identify music, movies, advertising, and television shows based on a short sample played and using the microphone on the device. For music, this sample needs to be just a few seconds long. Once Shazam captures an audio sample, it creates an acoustic fingerprint based on the sample’s spectrogram and uses algorithms to match it against a central database of fingerprinted songs.
This technology allows Shazam to rapidly recognize music in a matter of seconds. The app has been popular with users since its launch in 2002 and has continued improving its music recognition capabilities over time.
How Does Shazam Work?
Shazam’s core technology is based on audio fingerprinting. An audio fingerprint is a compact digital summary of an audio sample, created with a hash function.
Here is an overview of how Shazam creates audio fingerprints and identifies matches:
- A user captures a short sample of audio using Shazam on their smartphone, usually about 5-10 seconds of a song.
- Shazam analyzes the spectrogram of the sample by converting it to a fast Fourier transform (FFT) and examining the frequencies present over time.
- The spectrogram is made into a hash fingerprint by an algorithm. This creates a digital summary of the sound sample.
- The fingerprint is compared to Shazam’s database of song fingerprints to find an exact or close match.
- If a match is found, Shazam returns the song title, artist, album, and other metadata to the user. Users can save the song, listen to it on streaming services, or share it.
The key technology that makes this possible is the ability to create compact audio fingerprints that can be efficiently matched against Shazam’s massive database. Next, we’ll dive deeper into fingerprinting.
Audio Fingerprinting, Explained
Audio fingerprinting allows Shazam to recognize and match short snippets of songs based on the audio itself, even in noisy environments. Let’s break this down step-by-step:
1. Analyze the spectrogram
Shazam first converts the audio sample to a spectrogram, which visually represents frequencies in the sample over time. This reveals the spectral content and patterns that give each song a unique identity.
2. Create a hash fingerprint
The data from the spectrogram then passes through a hash function, which converts it to a fingerprint. This is a compact digital summary of the audio sample.
3. Generate a bit vector
The hash fingerprint is further processed into a bit vector, which is a very short binary representation of the audio sample. For example, the bit vector might be 1024 bits long.
4. Compare to other fingerprints
Shazam efficiently compares the new fingerprint to millions of fingerprints already in its database to look for a match.
5. Return song data
If there is a match, Shazam immediately returns the song title, artist name, album, and more.
This process happens nearly instantaneously because the fingerprints are small, the matching algorithms are optimized for speed, and the song database is indexed intelligently. Next we’ll look at the types of algorithms Shazam uses.
Key Algorithms Used by Shazam
Shazam relies on several audio signal processing techniques and algorithms to create fingerprints and identify matches, including:
- Fast Fourier Transform (FFT) – Converts the audio sample into the frequency domain so Shazam can analyze the spectral components.
- Peak frequency detection – Detects frequencies that contain the most energy to generate a signature.
- Time delta detection – Measures time between signal peaks to capture rhythmic components.
- Spectral analysis – Studies how energy is distributed across different frequencies.
- Acoustic fingerprinting algorithms – Converts audio samples to compact digital fingerprints that can be compared quickly.
- Locality-sensitive hashing – An algorithm that indexes data in a way that similar fingerprints are located near each other for fast matching.
- AI matching – Statistical modeling and machine learning to detect distorted, cropped, or noisy samples.
By combining these techniques, Shazam creates robust fingerprints that contain identifying features of songs while also supporting ultra-fast matching against the fingerprint database.
Inside Shazam’s Massive Acoustic Fingerprint Database
Shazam’s database is at the heart of its matching capabilities. It contains acoustic fingerprints for over 11 million songs as of 2018.
To build this database, Shazam uses audio data from various sources:
- Direct partnerships with record labels and distributors
- Crawl and index publicly available song sources
- Users submitting unknown songs that get added
Shazam needs to have high-quality audio samples to analyze each song. The goal is to capture all the unique acoustic elements that make a fingerprint for every track.
The database also needs powerful indexing to support blazing fast searches. Billions of fingerprints can be quickly scanned to find matches.
As the database expands, the accuracy and speed of Shazam’s matching improves. Machine learning helps refine matches and correct errors over time too.
How Shazam Matches and Returns Results to You
When you use Shazam to identify a song, how does it find and return the match to your device so quickly?
The Matching Process
- Shazam instantly creates a fingerprint of the audio sample captured by your phone’s microphone.
- This fingerprint is compared in real-time to the fingerprints in Shazam’s database.
- Because the database is heavily optimized, Shazam scans through millions of fingerprints in seconds.
- If a solid match is found, Shazam immediately sends the song information back to your phone.
- If no match is found, Shazam keeps trying with larger audio samples and by comparing to similarly indexed fingerprints.
Returning the Match
Once a song match is determined, the Shazam app displays the results on your phone, including:
- Song title and artist
- Album name and album artwork
- Links to stream or buy the song
- Options to share the song via social media or text
- Lyrics, videos, concert information, etc
Shazam integrates with streaming services like Spotify and Apple Music so you can easily listen to the full song. The whole matching process takes 3-10 seconds in most cases.
The Evolution of Shazam’s Recognition Technology
Shazam has been continually evolving its audio recognition technology since the first version launched in 2002.
Here is a quick overview of Shazam’s technical milestones:
- 2002 – The original algorithm analyzed spectrogram data to construct fingerprints. It matched against a small local database.
- 2004 – Switched to analyzing peak frequencies and timing patterns as the basis for fingerprinting.
- 2005 – Introduced faster matching algorithms and a centralized song database.
- 2008 – Added auto-tagging of tracks to expand the fingerprint database.
- 2009 – Launched a song recognition speed of under 3 seconds.
- 2010 – Updated apps to run recognition in the background, not just on-demand.
- 2014 – Added Alberta Machine Learning Group’s (AMLG) matching algorithms.
- 2018 – App could identify songs in real world from TV, radio, films, ads, games, etc.
- Today – Shazam relies on audio fingerprinting, spectral analysis, machine learning, and statistical modeling to achieve industry-leading speed and accuracy. The fingerprint database expands continuously.
Advancements in smartphone processing power, fingerprinting techniques, big data capabilities, and machine learning have allowed Shazam to stay at the forefront of music recognition after 20 years.
Frequently Asked Questions about Shazam
Here are answers to some common questions about how Shazam works its magic:
How does Shazam identify songs so quickly?
Shazam can identify songs in seconds thanks to audio fingerprinting algorithms that create compact acoustic signatures. These fingerprints are matched against Shazam’s extensive database using locality-sensitive hashing for ultra-fast lookups.
What are acoustic fingerprints and how are they created?
Acoustic fingerprints are digital summaries of a song’s spectral content over time. They capture identifying elements like frequencies, rhythms, harmonics, etc. Audio signal processing converts sound samples into fingerprints.
How large is Shazam’s song database?
As of 2018, Shazam’s database contains more than 11 million fingerprinted songs. Their database expands continuously as new music is released.
Does Shazam need to listen to the entire song to identify it?
No. Shazam only needs to capture about 5-10 seconds of a song. The acoustic fingerprint contains enough detail to match the full track.
Can Shazam identify songs over background noise?
Yes. Shazam uses filtering techniques to reduce the impact of ambient noise. The key spectral components are still detectable for creating a usable fingerprint.
How does Shazam make money?
Shazam earns revenues via advertising, commissions on referrals to music services, in-app purchases, and data analytics. It was purchased by Apple for a reported $400 million in 2018.
Conclusion
In summary, Shazam uses audio fingerprinting to rapidly detect which song is playing based on just a short sample. By transforming sound waves into compact acoustic fingerprints and matching them against a huge song database optimized for speed, Shazam can return results in seconds.
Continuous technical improvements in fingerprinting algorithms, machine learning, and A/B testing of song matching allow Shazam to keep pace as music evolves. While its core technology remains audio fingerprinting, Shazam keeps finding ways to do it faster, more accurately, and across more sources of audio input.
After reading this article, you now have an inside look at how Shazam taps advanced signal processing, spectral analysis, hashing techniques, big data capabilities, and machine learning to create an amazing music recognition experience on millions of devices worldwide.
So the next time you want to figure out the name of a song playing, remember it takes some complex technology working in the background for Shazam to work its magic almost instantly!