The Practical Goal of Manufacturing Tests
It is very easy to lose sight of the goal of any quality control process, but the goal of a manufacturing test is to ensure that the products manufactured and delivered to customers are essentially identical to the product design specifications and performance requirements.
Standing in the way of this goal is uncontrolled variability or tolerances in materials and manufacturing processes. The question then becomes, what is the most effective way to test product to ensure it meets an organization’s quality standard. The gold standard for quality control is a 100% full-functional test. This approach can best be summarized by the statement “If it’s not tested, it doesn’t work.” In this approach, the test strategy is dictated by a functional specification for the product and all manufactured devices are tested to ensure every user feature is functional to the specification.
In the context of an audio device, all connectors, speakers, and microphones are exercised according to their performance specifications and verified to work correctly. Almost by definition, a full-functional test is quite expensive: It requires extensive equipment and jigs for the various audio interfaces, as well as a significant amount of time during manufacturing.
At the other end of the spectrum is simply not testing at all. While perhaps a strange concept to discuss in an article about audio tests, there are some devices that are so inexpensive that there is simply no margin for quality control. Imagine the free earphones that most airlines distribute to passengers. These are essentially disposable items and must have a manufacturing cost literally measured in pennies. With these products, end-customers are essentially the quality control mechanism. Customer rejections or field failures become the signaling mechanism that there is a quality problem.
The spectrum in between 100% full-functional tests and no tests at all is characterized by an analysis of the cost of tests or conversely the value of quality. Depending on the value proposition of a product, it may be critical to ensure that no defective products reach end customers.
Conversely, there is a market space for products where quality or performance is not a core value proposition. These considerations should coalesce into a quality standard for the manufacturer, and that standard should directly inform test strategy.
When and Where to Test
Most modern devices come together as components from many suppliers, which are combined into sub-assemblies and then assembled into the final product. If testing is reserved until the device is fully assembled, so called end-of-line tests, any failures can result in complete scrappage of a finished product. If components and sub-assemblies are tested but a final check is not performed, a defective product may escape. How can a product assembled entirely from good components fail?
Unfortunately, one of the leading causes of defects for audio devices is mis-assembly and damage during manufacturing. Again, there is a spectrum from manufacturers who test at every point in the manufacturing supply chain to companies that choose not to test at all. If an organization’s quality standard suggests that some manufacturing test must be performed, “Where to test?” is often best suggested by the statement, “A test that never fails is not useful.”
Many organizations start out performing extensive testing during the development process for a product, so called Engineering, Design, and Production Validation stages (EVT, DVT, and PVT) can be used to optimize where, and for what, audio tests should be performed. The goal of this testing is not just to confirm the performance of the design or even to qualify components, but to discover if manufacturing tolerances call for more, or less, testing on specific components or sub-assemblies.
Manufacturing Tests vs. R&D Measurements
One common area of tension is the desire to directly correlate audio measurements made on a manufacturing line with those made in a product development lab. This is an inescapable tension because measurements made during product development almost always take much more time to conduct than is tolerable in a manufacturing environment. Whether it is trying to compare the speaker’s free-field frequency response to the response inside a compact test cell or a high-resolution stepped sweep of a power amplifier to a few discreet test points, it is often simply not possible to directly compare R&D measurements to QC measurements, at least not without incurring a high cost of testing.
The single best approach to bridging this discontinuity is the development of a test standard and/or the use of golden (or reference) unit (see Figure 1). With this strategy, one or more known good units are measured in the development lab and at each manufacturing site. The results can then be compared and correlated (see Figure 2). In cases where direct measurement results cannot be compared (e.g., because the acoustic test fixtures are different), it may be possible to develop transfer functions that adjust for the difference between different fixtures. Note, a transfer function for comparison between test fixtures is only valid for a given device.
What to Test: Electronic Devices
When testing a purely electronic device (e.g., a power amplifier, headphone amplifier, DAC, ADC, or really any device with an electrical input or output), one thing to note is that even the simplest test will yield valuable data. That is, just measuring level, distortion, and noise with a single tone (e.g., 1 kHz) effectively yields a lot of information.
A single-tone test can be performed easily and quickly, likely in as little as one second, and will functionally verify most of an electronic signal chain. It verifies that the power supply is functional, that the signal path and all electronic components in it are correctly installed and, by measuring the added spurious noise, verifies that many adjacent devices are not defective. Adding a few more test tones or points for practical purposes provides almost complete validation.
A classic approach for a full-bandwidth electronic device is a test with points at 20 Hz, 1 kHz, and 20 kHz. Testing at additional frequency points will verify that output coupling filters are correct. In electronic devices, beyond basic functionality, a classic cause of fallout is variability or tolerance in the passive analog components (i.e., the capacitors, resistors, and inductors), particularly in the output filters used in power and headphone amplifiers. By choosing test frequencies at the corner points of the filters, you can easily verify that these components are correct without having to perform a complete frequency sweep of the device. In addition, if you offset the frequencies in each channel (e.g., test at 19.5 kHz in one channel while testing at 20 kHz in the other), you can simultaneously identify any crosstalk.
What to Test: Speakers
For speakers, earphones (see Photo 1), and balanced armature drivers, it is usually only minimally useful to test the device at one frequency. Usually these devices are swept across a range of frequencies of interest. However, since uncorrelated noise is not a useful measurement on a speaker driver, the sweeps can be very fast, with a few second-long sweep usually enough. When testing a speaker driver, it is useful to examine the acoustic response of the device and the impedance function. Combined, the two will essentially reveal whether the system is performing correctly or not.
For an easy incoming inspection of raw speaker drivers, I would like to suggest the value of an electrical-only measurement of impedance. Because any deviation in mass of the cone or compliance of the suspension will cause a shift in the impedance curve—particularly at the frequency of the resonant peak—it can be used to quickly weed out devices with common assembly errors. In addition, measuring just impedance considerably simplifies the test requirement. No acoustic test box or measurement microphone is required, considerably simplifying the test setup.
Finally, for the best measurement repeatability, impedance should be measured at small signal levels, that is a signal level where the driver’s motion is within the linear, pistonic range of motion.
This leads to a common disconnect between manufacturing and R&D. In manufacturing, speaker drivers are usually measured at high signal levels, to test the driver through their full range of motion. However, the impedance curve of a driver at high signal levels cannot be compared to the typical small signal used when calculating Thiele-Small (T-S) parameters, the more common measurement made in R&D.
Modern audio analyzers can usually measure all the important parameters of a speaker in a single fast sweep. With a measurement microphone and electrical connection, a speaker’s acoustic frequency response and distortion can be verified along with the impedance and resonance, Rub & Buzz, or high-order harmonic distortion, and even polarity can all be checked in a single test.
What to Test: Microphones
A microphone is often tested just for frequency response, usually with a swept signal across the frequency range of interest. This can be used to digitally calibrate the sensitivity of the microphone for noise cancelling application and phase data can also be captured for microphone beam forming application.
A second test is a seal test. For this test, the microphone is swept twice, with the acoustic port for the microphone occluded the second time. This is to verify that the microphone is appropriately acoustically isolated from other parts of the device.
Tracking Data
One area that is often overlooked in manufacturing tests is the tracking and analysis of test results beyond simply recording pass and fail results. If measured data is collected and stored, it is possible to examine it for several purposes (see Figure 3). First, a field failure can be audited or traced against the measurements made during manufacturing. Perhaps the manufacturing tests need to be modified to check for a failure mode that is otherwise being missed. Or perhaps the device passed QC, but marginally, and the pass/fail limits should be adjusted. Perhaps the device passed but only after extensive retesting (i.e., an operator simply retested a device until it passed), indicating the device had erratic behavior and policies on retest need to be evaluated. Even as devices pass, it is useful to examine trend data to detect drift in materials and tolerances in order to preempt future failures.
Conclusion
While it may seem like an abstract connection, the technical details of the measurements made in manufacturing are directly related to an organization’s overall quality standard and a product’s value proposition. How is the product priced? Is it positioned as a high-end or luxury item? Or a value-oriented product? What is the cost of a field failure? (Either the direct cost of managing return and replacement or in damage to the brand.)
From these questions, a budget can be set for test and quality control. Will every unit manufactured be subject to full functional test, will basic functionality be checked, or will only items subject to common failure be inspected? Will every unit be tested or will manufacturing just be sampled?
With the test objectives and the budget set, it is then possible to outline the technical objectives of the test. These may be to just probe a device with a single tone and record output level and THD+N. With a larger test budget, it may be to use multitone, continuous sweep, or another fast measurement technique to characterize a device over a more complete operational envelope. Ultimately, the technical requirements for quality control tests are the product of overall quality standards. aX
This article was originally published in audioXpress, November 2020.
About the Author
As Vice President of Product Development at Audio Precision (AP), Daniel Knighten is focused on the definition of the features, functions, and price of new products while managing AP’s engineering team and shepherding products through their complete life cycle. He also supervises the AP technical support team, a rich source of firsthand experience on how customers use, and aspire to use, AP analyzers, software, and test accessories.
