A Computer in Every Ear. The Rise of Headphone 3.0

September 1 2022, 19:00
As Apple keeps reminding us, in just a few days, we will have the opportunity to learn all about the iPhone 14 and whatever it is that the company has in store for us through the end of the year. In the most likely of the predictions, we can expect Apple to also announce a new model of its market-leading Apple Watch, and in the less likely category, new AirPods. Until that scheduled event takes place, the whole consumer electronics industry will be waiting to learn what they will be competing with.

With a market share between 29% and as much as 56%, depending on who does the research and how, Apple leads the overall true wireless earbuds market globally. And even if the different AirPods models (and not forgetting those from Beats) continue to lose market share given the number of competitors that have entered the segment, Apple still has increased sales volumes, as the market overall continues to grow. As it happens with the iPhone, more important than the units sold, Apple gains more revenue in the product category than all the profits of all its competitors added up.

As everyone trying to compete in the space, in the AirPods, as with any other product segment that Apple enters, its products combine unique custom hardware, a dedicated OS and a software ecosystem. And since no other player in the market combines all those fundamental pieces, no one can optimize products to the level Apple does.
 
Apple is the market leader in a category that no one predicted it would ever enter. And once again it combined complete optimization of hardware, OS, and software ecosystem. Shazam was acquired by Apple in 2018 because it added value as a platform that changes the way people engage with music by making song identification accessible to everyone. Today, Shazam has more than 225 million global monthly users.
App Marketplace
I've been recently reminded about an anniversary that tells us a lot about how important Apple is in the overall audio and music space: Shazam, the music identification app, recently turned 20.

Launched in the summer of 2002, Shazam has now officially surpassed 70 billion song recognitions. Originally, Shazam was launched as a text message service in August 2002. At the time, users in the UK could identify songs by dialing “2580” on their phones and holding it up as a song played. They were then sent an SMS message telling them the song title and the name of the artist. In July 2008 Shazam launched on the new App Store and later launched an Android version. In April 2015 Shazam became available on the first Apple Watch, and in September 2018 Shazam was acquired by Apple.

It's just another app, but I know I use it in Apple Watch all the time - I have my own watch face with Shazam always ready to start listening. It's a really brilliant way to discover new songs and artists, and keep memories of great moments, wherever I go.

As it is painfully clear to any manufacturer or brand that intends to enter the still-thriving true wireless stereo (TWS) space - which very soon promises to evolve in completely new directions with hearing augmentation and hearables, it's really hard to compete with Apple. The elements of technology that make such products are only partially available and do not offer the desirable level of integration and optimization – never mind the possibility to update features over-the-air. 

A brief look at all the products just being announced at IFA 2022 in Berlin, Germany, shows how constrained even the most powerful manufacturers are feeling when designing new products.

Low-power wireless and DSP platforms - even edge-AI processors that are able to process natural language - are now starting to become available, specifically optimized for TWS earbuds. Qualcomm is pursuing that strategy with its Snapdragon Sound platform. Likewise, the software development kits for such products are available from a variety of vendors. 

And obviously, there are companies now offering a variety of software that manufacturers can use to differentiate new models, including hearing personalization. Bragi, the category pioneer, is now trying to build a complete software platform for the category, with its Bragi OS that includes multiple service integrations like Amazon Alexa and embedded AI. This effectively offers a sort of "marketplace" for product developers and manufacturers to "shop" and implement in their products. But that choice, App Store style, is still not available to the consumer.
 
For developers and manufacturers, designing new true wireless earbuds can be an exercise in frustration, given the tremendous platform constraints. Even when working from a blank canvas, it’s hard to predict all the different combinations of features they can put into a single product.
Enter Sonical Headphone 3.0
I have recently met a new company that is working to solve this dilemma for manufacturers by creating a fully integrated approach that could revolutionize the product development of headphones and earbuds, while at the same helping consumers to gain more choice. To start, Sonical intends to create an environment that will allow consumers to pick and choose apps for their hearables, in the same way they use apps on smartphones. 

To make this possible the San Mateo, California-based company is designing and building the world’s first "ear computer" that has the capabilities and performance for next-generation headphones, earbuds, and all ear worn products. And to make the platform accessible they are creating a dedicated operating system to unlock possibilities for thousands of existing and new developers. At the core, those developers will be able to leverage a powerful AI and advanced audio processing platform that runs a real-time operating system (based on FreeRTOS).

I spoke with Gary Spittle, the CEO and founder of Sonical, which has been working to fulfill this vision since early 2020. Gary is a technology and product strategy expert, with a long and remarkable career in the audio industry, where he lead companies in the music, professional audio, consumer electronics, and mobile segments. And he has worked directly with most of the world’s leading audio product brands for more than two decades, defining, designing, building, and shipping products on a large scale. Before Sonical, he led his own audio consultancy business in the San Francisco, Bay Area, and before that he worked with Knowles, Dolby, CSR (before the acquisition by Qualcomm), CEDAR Audio, and others, piling up multiple patents in the process.
 
Developers, manufacturers, and users all want more. And that is why Sonical is working to create a foundation with an optimized chip and operating system solution.
The full interview I did with Gary Spittle will be published in audioXpress magazine. I'll just summarize here the points that Gary highlighted about Sonical and its market approach:
- Hearables should not be defined by the functions of a chip or the Bluetooth profiles, there is a lot more that product makers want to do.
- Instead, the industry needs an open platform that is software defined, by the end user (just like laptops, tablets, smartphones, smart watches).
- The products will no longer be categorized as a "phone accessory."
- Having just a chip is not enough, developers don't want to spend weeks down in the weeds of chip level firmware. They want to write high level apps and plugins and rapidly deploy them.
- Having just a software framework is not enough, it will always be restricted by existing silicon performance.
- The optimal solution (proven by CSR) is a combination of hardware and software that is well supported for product manufacturers and developers to innovate.
- The next generation of products will not only have improvements for how we listen to audio, they will also listen to the world around us and present an enhanced version using augmented reality algorithms, and will also listen to our bodies to monitor our health and wellness using AI.

Gary is very familiar with all the challenges of developing Bluetooth products and designing products and solutions for headphones, hearing aids, and hearables. That led him to understand that the industry "needs a new chip." And that was even before Apple did just that with the H1 processor. More importantly, he believes that manufacturers need to work with software and hardware. And most important, he believes in the power of an open platform for the category, so anything can be driven through apps. 

Obviously, there is a lot more to learn about what Sonical is doing. I can just share that the Sonical team absolutely has the vision to complete the puzzle and more importantly, they have had significant progress in their efforts.
 
"We see that the product manufacturers actually want to build independent products that can be used when not attached to a phone. And that means a completely new system and a different way of thinking about hearables, making the product the center of the use case rather than an accessory," says Gary Spittle, Sonical's founder and CEO.
I have just one more thing to share here.
Sonical and Segotia, an advanced audio and neuro technology company from Ireland, have established a partnership to promote this Headphone 3.0 strategy, leveraging the strengths of each team. The immediate focus is to expand CosmOS, the embedded operating system developed by Sonical, with a growing ecosystem of plugins and apps for hearable devices to transition to Headphone 3.0.

Segotia (www.segotia.xyz) is a Galway, Ireland-based company that specializes in fusing audio processing with the sensing of user brain activity and the decoding of Neuro Markers, which indicate the nature and extent of user attention, engagement, and response.

The ability to download personalized neuro-sensing applications to consumer devices via CosmOS is seen as a game changer, enabling a new generation of applications including the ability to monitor listener attention and help with clarity of an audio signal in a loud environment - aka helping to solve “the Cocktail Party Effect.” This includes features such as Audio Attention Decoding to bring consumer Headphone 3.0 products into the realm of professional hearing devices.

"CosmOS will be the tool that delivers flexibility to Hearables,” says Gary Spittle. "We are on the cusp of the era of Headphones 3.0, which puts configurability of Headsets into the hands of the device owners and removes the constraints imposed by the platform vendors.

Hope this excites your curiosity. You can visit Sonical’s work-in-progress website at www.sonical.ai
 
"We need an ear computer. And the main difference is that when you build a computer, you don't know what the end user is going to do with it. It's completely blank. Rather than an application-specific chip, where you're trying to define what the user can do and you optimize for that use case, whether that's playing music or making a phone call, a computing platform is completely empty, and the operating system needs to allow people to do whatever they want. And that is driven by apps or as we call them plugins. The prototypes that we've developed now, already demonstrate how this is possible. It also needs access to a lot more data, not just speakers and microphones, and the other piece that's needed is different types of connectivity, not just Bluetooth,” Gary Spittle adds.
Page description
About Joao Martins
Since 2013, Joao Martins leads audioXpress as editor-in-chief of the US-based magazine and website, the leading audio electronics, audio product development and design publication, working also as international editor for Voice Coil, the leading periodical for... Read more

related items