It's something you see at every live music event, people holding their phone above their heads to record the sights and sounds of the gig in progress. Often people are capturing video and almost completely indecipherable audio on a device that just wasn’t designed for this kind of recording - a fact they will only discover when they choose to rewatch the video later. And usually... only once before deleting.
While the video portion of a recording might be acceptable quality - depending on where you’re standing and what is happening with the lighting - any audio recorded on your phone from a concert is going to be terrible. Cameras on devices have emerged as a key selling point for phones in the last decades, with huge amounts of time and research put into making them smaller and better, year after year. However, both speakers and microphones on phones have received nowhere near as much attention. But this isn’t just due to lack of interest, it's due to fundamental design limitations that are much harder to work around.
The difference between light and sound is specifically that the miniaturization of photography is much easier considering the tiny wavelengths of light itself. Sound wavelengths in the lower hertz range on the other hand can extend over meters, and so creating an input that can capture them and be extremely small is an enormous challenge.
(and never will be).
Most smartphones currently use microelectricalmechanical systems (MEMS) microphones, which are small. Sound is captured by two charged silicon membranes, one of which vibrates in response to pressure waves, and these vibrations are then converted into an electronic response. Though high-quality MEMS microphones are available, the ones on mobile phones tend to only have their frequency range defined within 100Hz and 10kHz, with a resonant peak between 4kHz and 6kHz - ideal for capturing the human voice, and easily mass produced, but not ideal for high-quality musical recordings.
Live music on stage is most likely to be captured using professional dynamic mics, designed to have a directional polar pattern and each microphone made to capture an optimal frequency range for its specific purpose: low-frequency mics for a kick drum and bass (e.g., AKG D112), and an SM57 for guitars or snares. A range of different mics are mixed by an engineer to produce the sound that we all hear at a gig. Compare this to the mass-produced smartphone mic previously described, and you see the problem.
The wall we’ve hit in terms of what mass-produced mics can record is as much about their purpose as it is about physical or design limitations. Simply put, phones are not designed to record anything more than a person’s voice. It can be easy to forget with all the features that have been added over the years, but phones are still... phones. Designed to make calls. To send and receive verbal communications. Why would anyone want to enhance the recording and playback qualities of a phone beyond what they can do currently?
A big reason is the huge rise in user generated content (UGC), which brands, artists, and organizations of all kinds have been quick to harness as part of their marketing strategies - through both influencers and by turning their fans into brand ambassadors. While images and videos have always been center stage in influencer culture, when it comes to live music and live events in particular sound is just as, if not more, important. The low quality content created by fans and influencers from live events is largely down to limitations in readily available audio technology. If you want an example of just how effective the right mix of audio and video can be in UGC, look no further than TikTok.
Moving past the limitations of the most commonly available microphones, anyone with even a passing audio understanding will tell you that no recording done from the middle of a crowd is going to turn out well. There is a reason that artists record their songs in specially designed studios. Sound is limited to exactly where you are in physical dimensions. In the context of a live event you are not in the ideal position to be recording anything. Sounds will be coming at you from every direction, at high volume, and reverberating around the space blending with the roar of the crowd - so it’s not going to work. Even the best microphone in the world won’t overcome those environmental factors, and this doesn’t even touch on the issue of distortion in very loud environments.
The biggest problem here is blind source separation. If you are talking to someone or listening to music in a crowded room, you are able to focus in on that sound while filtering out other noise in order to hear it better. This is a neat feature of the human brain that we call “cocktail party effect” - the brain's ability to focus your auditory attention on a particular stimulus - that unfortunately isn’t shared by audio technology.
With microphones being unable to “focus” on a specific set of sounds in the same way that we can, it absorbs all sound almost equally. In a loud environment with dozens or even hundreds of different sources of sound, microphones simply can’t differentiate between what we want to hear and what we don’t.
When it comes to images and videos, there are no end of apps or software options that will allow you to touch up your content to a professional level. So where is "Photoshop for audio"? Audio spectra from dirty audio leads to overlap in the frequency domain, making it impossible to unpick like a human brain would manage to naturally. The technology does exist, but for highly specific applications, such as speech extraction from music. The problem is that each application is so specialized that it wouldn't be practical to put in a consumer-grade device like a mobile phone.
With all of this in mind the only way to get decent audio for live events from the devices available to consumers is by circumventing the issue of phone microphones entirely. The best way to sync audio quality with video quality is to take the audio directly from the mixer - with its multiple array of professional microphones - the same audio that would be going to the live speakers or broadcast. Take the live audio before it is audible.
TagMix accomplishes this by taking this audio from the mixer, uploading it to the cloud directly, and “swapping out” the audio when it is detected in the user's video content. In practical terms, this means users are able to have concert quality audio in their UGC without the overwhelming sound of the crowd, or being limited by the microphones they have available. In order to enhance the impact that UGC can have for the venue or artist, the Tagmix app also helps users to share their moments frictionlessly. It also makes the music used in UGC traceable, ensuring artists and other rights holders are compensated wherever their music is played.
Ultimately, the whole music ecosystem will benefit from enhanced, shareable UGC videos. With the user being able to post content across multiple social media platforms including Facebook, Instagram, Twitter, and Snapchat simultaneously, TagMix goes far beyond boosting social media impact for the end user. TagMix allows venues, promoters, and artists to benefit through increased SEO and meaningful alignment via Spotify calls to action being embedded directly into the finished content.
Our technology has already been made available across multiple stages at music festivals such as Ultra, Tomorrowland, and Hideout, while we have also been present at conferences including Ibiza's International Music Summit, Miami Music Week, and the Amsterdam Dance event. Further expansion of the use of our technology is planned in the United Kingdom later in the summer.
The importance of UCG is only going to continue growing as time goes on, and we’re likely to see a spike in its importance in the short term as events, venues, and artists attempt to coax people back into attending following the devastating impact the coronavirus pandemic has had on live events. It is a perfect tool for building brand awareness, building a community, and maximizing organic growth, and a tool that is only enhanced by making it easy for people to represent your event in the best possible way using the best quality video and audio available.
www.tagmix.me
This article was originally published in The Audio Voice newsletter (#339), August 2021.
