True Wireless Stereo (TWS) earbuds have seen an incredible growth these past few years and have evolved from poorly connected gadgets to reliable must-have accessories for virtually any activity: listening to music, consuming other media, calling on the go, and teleconferencing, with new use cases coming (e.g., vital signs monitoring and hearing enhancement).
After connectivity markedly improved over the past years, the primary differentiating factors for TWS earbuds now include:
• Music and media playback quality (with ANC and 3D audio)
• Voice calling quality (both transmitting TX and receiving Rx)
• Battery life
Many brands have been fighting to achieve the greatest amount of noise cancellation and the best playback sound—and there is still a lot of work invested into delivering proper acoustics via design, material choices, and software tuning. All mid- and high-end devices claim great sound quality with renowned acoustic brands sound signatures. Alas, it is still very common to find devices with large drivers or multiple balanced armature (BA) speakers and mediocre sound reproduction. Properly tuning an ANC engine not to degrade music has also proven hard to achieve even on premium devices.
However, battery life was extended with latest chipsets and the introduction of Bluetooth 5. And the industry is making more effort on codecs (LC3), while extra sensors are also being integrated for ease of use and battery savings pursued (e.g., with optical proximity sensors for off-ear detection). Despite some improvements since the launch of the first TWS earbuds, the voice calling use case on current devices is also not exempt from flaws. New technologies offer opportunities for significant improvements.
Beyond mastering those must-have features, there is also a clear need for differentiation as all brands offer more or less the same features in each price range. TWS earbuds have great potential to move from being a mostly passive gadget to becoming an always-on active device offering real personal sensing enhancements.
Crystal-Clear Voice Calling
Voice Calling has improved in the mid- and high-end segments, but there is still a very significant margin to do much better even on premium reference models, such as Apple AirPods Pro and Huawei Freebuds Pro that still let too much noise pass through while not always preserving your own voice well. And there is nothing worse than being on a conference call and feeling that your audience does not properly hear you and your message does not come through.
On mid- and low-end devices, voice calling quality has not improved much since the first TWS earbuds because pre-integrated standard platform solutions are used for cost reasons in that segment, leaving the owner with a pair of earbuds that can barely be used to make a phone call.
In this pandemic era, when so many of us work from home, a long-lasting and dependable device is crucial as we spend our days talking with our teams and customers on Microsoft Teams, Zoom, Slack, and others. Users must be able to rely on their earbuds to pick up their voice and filter out background noise, including the most transient sounds such as background voices.
Areas of Improvement for Voice Calling
For every new pair of earbuds, reviewers and consumers have accepted that there is no easy solution, that background noise is always a problem when trying to be properly heard while using earbuds. When outdoors, wind noise is one of the biggest challenges, especially if no protective mesh or mechanical solution was implemented on the earbud. Wind noise may be one of the most problematic noise types during a voice call (or while listening to music) outside.
Front ends for voice assistants have improved greatly on smartphones and smart speakers. However, consistent voice capture is still rare in TWS earbuds, making it a challenge to reliably interact with your device in all environments. Today, voice calling is still one of the most power-hungry applications on an earbud, so footprints are critical to leave room for new use cases and the processing of sensor data for new differentiating features.
7 Sensing’s Pure Voice
Compared to existing voice calling solutions, the novelty of Pure Voice is to perfectly leverage the in-ear inward facing microphones and/or bone vibration sensors present in mid- and high-end earbuds. Pure Voice offers excellent voice preservation and dramatically removes background noise (including wind) and noise in speech while keeping very low footprints. Figure 1 details how the Pure Voice technology works.
The Pure Voice technology uses in-ear microphones and/or Bone Vibration Sensors for voice pick-up. The in-ear microphones are used for feedback noise cancellation and are getting more popular with the increasing presence of hybrid ANC. Bone vibration sensors are also used on high-end earbuds of brands such as Apple, Huawei, and Samsung.
The Pure Voice algorithm automatically adapts to all existing/known bone vibration sensors and in-ear microphones. Neither technologies nor sensors are easy to deal with, as they capture sound in a very nonlinear and distorted way. They also capture all sorts of other noises (e.g., scratching or teeth sounds), and their integration can sometimes create a lot of complexity. Bone sensors are particularly adapted to open fit or leaky designs with large vents although Pure Voice supports all earbud designs with any of the two technologies.
Pure Voice Performance Benchmarks
7 Sensing performed multiples tests to confirm the performance improvements brought by Pure Voice (using devices with two external microphones with one in-ear microphone) over standard devices. Tests were performed with car noise and pub noise, comparing Apple AirPods Pro and Huawei Freebuds Pro earbuds. During the benchmark tests, one of 7 Sensing’s development earbuds was used on one ear, while a commercially available earbud was used on the other ear (Figure 2).
The results are very significant. Voice preservation was near perfect while offering almost full and instantaneous noise suppression. In the first test, 7 Sensing pitted Apple AirPods Pro vs. 7 Sensing Pure Voice. The Apple AirPods Pro have two external microphones, one Bone Vibration Sensor, and one in-ear microphone per earbud.
Figure 3 shows that while the AirPods Pro were not able to remove more noise without degrading speech, Pure Voice removed most of the background noise while still having much more speech content preserved. Figure 4 shows that with pub noise, which is renowned for being difficult, AirPods Pro maintained a good amount of speech at the cost of not removing much noise. Pure Voice, without any convergence time, removed significantly more background noise while still preserving all the speech content.
In this next test, 7 Sensing pitted Huawei Freebuds Pro vs. 7 Sensing Pure Voice. The Huawei Freebuds Pro have two external microphones, one Bone Vibration Sensor, and one in-ear microphone per earbud. Figure 5 shows the Freebuds Pro did a good job at removing noise in speech; however, Pure Voice on the right removed all the noise with no convergence required for the noise suppression algorithm. Pure Voice also shows much more speech preservation in the mid-range.
Figure 6 shows that with pub noise, the Freebuds Pro were not able to remove background noise enough in the low frequencies while heavily smoothing noise and speech in the higher bands. Pure Voice was able to clean the low frequencies efficiently and still maintaining high frequency content.
Super Hearing: Beyond ANC and Transparency
Due to the fact that the comfort of TWS earbuds has improved and they have become trendy fashion accessories, many users now wear them for a large part of their day, opening up the opportunity for a new use case: better hearing.
Some earbud products provide a “transparency” or “hear through” feature, however, even normal hearing people regularly face very noisy environments where conversations are challenging. Without transforming them into medical hearing aids, earbuds can greatly improve listening conditions to make conversations crisp and clear.
ANC seeks maximum isolation for immersion — thanks to progress in hardware latency, design, and tuning, it is offering ever stronger noise cancellation. However, users sometimes desire the opposite effect of being able to hear and socialize while keeping their earbuds in. Premium earbuds today offer adaptive noise cancellation, setting the sound scene somewhere between transparency and isolation.
Today, transparency, or hear through, is at best compensating the occlusion effect to offer almost normal hearing. Transparency has been present since the first hearables (Bragi Dash/Headphone), but seldom reaches a real natural feeling of not wearing earbuds. In transparency mode, no distinction is made between noise and information/content — thus yielding no improvement of a user’s existing hearing capabilities.
What if a TWS earbud could actually enhance the wearer’s capabilities and give him or her super-hearing? By combining techniques used in expensive medical-grade hearing aids with modern highly integrated and powerful consumer TWS chips, this has become a real possibility.
Amplification, noise reduction, and speech enhancements add high-end differentiating features to an earbud product in a crowded market — yet without the stigma of wearing a medical device.
Market Traction for Augmented Hearing Features
Some key devices have already started implementing features to hear better. Among the top contenders are the Nuheara IQ Buds2 Max, the Apple AirPods Pro, and the Huawei Freebuds Pro — however, each one comes with its own limitations.
The Nuheara IQ Buds2 Max offer beamforming, noise reduction, and amplification. However, they offer limited conversation and speech enhancement features, and they don’t offer any voice perception improvement, which is crucial when wearing that kind of device nearly all day. They remain the most compelling offer feature-wise on the market, albeit they are positioned in the hearing assistance segment, or so-called up-and-coming over-the-counter (OTC) hearing devices.
With the accessibility features on iPhone released with iOS 14, Apple AirPods Pro offer tailored amplification without speech enhancement, beamforming, or noise reduction. AirPods Pro can thus be turned into personal sound amplifiers without any stigma of wearing an assistance device. With their Voice Mode, the Huawei Freebuds Pro offer a conversation mode with limited noise reduction. However, they don’t offer beamforming or any kind of amplification.
7 Sensing Hearing Augmentation (HA)
To take the capabilities of TWS earbuds to the next level, 7 Sensing offers its Hearing Augmentation (HA) software solution. This suite of signal processing algorithms makes it possible to not just block out the world — as with ANC — but instead control exactly what the users want to hear for extra content awareness and elevated conversations, whenever they wish.
An earbud with 7 Sensing’s HA allows for new designs with super hearing capabilities, where users are able to comfortably have a conversation with other people in a noisy restaurant. Likewise, wearing the same design, users can enjoy their favorite music while jogging and still be able to hear sounds around them for safety awareness, tune in to a lecture for total concentration, or enjoy a music concert at safe levels.
HA automatically switches from one mode to the other. If the earbud picks up the voice of the user indicating the start of a conversation, it can automatically pause music playback and switch to Hearing Augmentation.
Combining passive isolation and noise cancellation, HA is layered upon and mixed into the existing audio path and takes advantage from maximum isolation available on the earbud. This is usually a combination of ANC in low frequencies (< 1kHz) and passive attenuation (or insertion loss) above. Figure 7 demonstrates how the HA process works.
7 Sensing offers two HA modes. The first mode is Super Hearing for extended perception with controllable amplification, beamforming and realtime noise reduction. The second mode is Voice Focus, which is ideal for conversations. It enhances the speech spectrum to narrow down your listening sphere for absorbed discussions. To demonstrate HA’s capabilities, 7 Sensing compared its features and benefits to that of Apple’s AirPods Pro and Huawei’s Freebuds Pro (Figure 8).
In a quest to keep Hearing Augmentation simple to use, 7 Sensing also implemented an automatic mode change with a dedicated Voice Detect feature. Voice Detect makes any earbud mode—Music (ANC on/off), Transparency, or Super Hearing—automatically switch to Voice Focus as the owner starts speaking. And after some time without speaking, the previous mode automatically resumes. An example of 7 Sensing’s Hearing Augmentation mode changes is shown in Figure 9.
Future Uses
Combined with its research on audio features for TWS earbuds, 7 Sensing is also working in augmented sensing. The company has a strong team dedicated to Machine Learning — exploring applications with vision, depth, spectral and light sensing (Figure 10). The technology complementarity of AI/Machine Learning, Vision, and Audio ideally positions 7 Sensing to explore the exciting future of Augmented Sensing in earbuds, headsets, and augmented reality devices. aX
This article was originally published in audioXpress April 2021.
About the AuthorJean-Marc Luneau is an audio DSP engineer focusing on engineering and product management. With more than 15 years of experience in research and development, he has worked on speech, music compression and voice enhancement, and biomedical imaging at Fraunhofer IIS, EPFL, Aalborg University and NXP Software, before co-founding 7 Sensing Software in 2017 in Leuven, Belgium.
7 Sensing Software
7SensingSoftware.com is an independent software vendor enabling technology companies to build new and exciting sensing solutions by providing algorithm IP licensing, software development and integration services. 7 Sensing Software is a Qualcomm Extension Program member, providing its low footprint audio solutions on QCC5100 series.
