On-Device Speech Separation With Mic Arrays

ISEF Category: Embedded Systems

Subcategory: Signal Processing  ·  Difficulty: Advanced  ·  Setup: School Lab  ·  Time: Full Year

The Hook

Three people talking at once can sound like one messy wall of sound. Your ears do a better job than most computers in a noisy room. That makes speech separation a real engineering problem. If you can split voices on a tiny device, you are working on the same kind of challenge used in hearing tech and voice interfaces.

What Is It?

This project asks you to separate overlapping voices from a small microphone array, then run the whole system on a low-power device. The array gives you multiple views of the same sound. The algorithm uses tiny timing and phase differences between microphones to estimate who said what. Think of it like trying to follow several flashlight beams crossing in a dark room, but each mic sees the beams from a slightly different angle.

Independent vector analysis, or IVA, is one method for blind source separation. “Blind” means you do not hand the system labeled examples of each speaker during the main separation step. Instead, the algorithm looks for patterns that stay grouped together across the microphone channels. The goal is to pull apart mixed audio into cleaner speaker streams without sending data to the cloud.

Why This Is a Good Topic

This is a strong science fair topic because you can change one part of the system and measure the result. You can test microphone spacing, room noise, number of speakers, or algorithm settings. The project connects to hearing aids, smart assistants, and meeting transcription, so the real-world use is obvious. You can learn signal processing, embedded programming, and experimental design in one project.

Research Questions

• How does microphone spacing affect speech separation accuracy in a four-mic array??
• What is the effect of background noise level on source separation quality??
• Does on-device independent vector analysis outperform a simple beamforming baseline for two or three talkers??
• To what extent does room echo reduce speaker isolation on a Pi Zero 2??
• Which microphone layout gives the best separation for speakers seated around a table??
• How does the number of active speakers change the signal-to-distortion ratio of the output??

Basic Materials

• Pi Zero 2 or similar single-board computer.
• Four INMP441 digital microphones.
• Microphone breakout boards or wiring connectors.
• Breadboard and jumper wires.
• MicroSD card with operating system installed.
• USB power supply rated for the board.
• Headphones or small speakers for playback checks.
• Laptop for coding, testing, and data analysis.
• Quiet and noisy room setups for recording tests.
• Printed seating marks or tape for fixed speaker positions.
• Stopwatch or notes app for logging trials.
• Spreadsheet software for organizing results.

Advanced Materials

• Pi Zero 2 or similar edge device.
• Four INMP441 digital microphones with matched sampling setup.
• Audio interface or logic analyzer for debugging digital microphone timing.
• Calibrated speaker or reference audio playback system.
• Measurement microphone for ground-truth recording comparison.
• Sound level meter.
• Acoustic foam or movable panels for controlled room tests.
• Reference workstation for running offline comparison algorithms.
• MATLAB or Python environment for offline signal analysis.
• External storage for logging raw audio and processed outputs.
• Small tripods or custom 3D-printed mic mounts.
• FFT or oscilloscope tools for signal verification.

Software & Tools

• Python: Processes audio files, runs analysis, and compares separation metrics.
• NumPy: Handles array math for multi-channel signal processing.
• SciPy: Supports filtering, transforms, and numerical analysis.
• librosa: Helps inspect and visualize audio features and waveforms.
• ImageJ: Not for audio itself, but useful if you document mic geometry, test rigs, or setup photos.

Experiment Steps

1. Define the target use case, such as two or three speakers around a table in a noisy room.
2. Choose the measurement metrics you will use, such as separation quality, delay, and runtime on the device.
3. Design a baseline system so you can compare IVA against a simpler method like channel selection or beamforming.
4. Plan a repeatable recording setup with fixed speaker positions, fixed mic positions, and matched test scenes.
5. Build a validation plan that compares separated outputs to clean reference recordings or known spoken scripts.
6. Decide how you will test tradeoffs between accuracy, compute load, and real-time performance on the Pi Zero 2.

Common Pitfalls

• Letting the microphones drift out of sync, which breaks the phase clues the algorithm needs to separate voices.
• Changing speaker positions between trials, which mixes geometry changes with algorithm performance.
• Testing only in one quiet room, which hides how fast separation falls apart with echo and background noise.
• Judging success by ear alone, which makes the results subjective and hard to compare.
• Skipping a simple baseline, which makes it hard to prove that IVA beats a less complex method.

What Makes This Competitive

A competitive version of this project does more than make cleaner audio. It measures performance with clear metrics, compares against a fair baseline, and tests more than one room or speaker layout. Strong projects also study the tradeoff between separation quality and on-device speed. If you can explain when the system fails and why, your work looks much more like research than a demo.

Project Variations

• Test the same algorithm with two, three, and four speakers to see how separation degrades as the scene gets crowded.
• Compare a dining room, classroom, and car interior to see how room acoustics change source separation quality.
• Swap IVA for another method, such as beamforming or independent component analysis, and compare runtime and output clarity.

Learn More

• MIT OpenCourseWare: Search for signal processing and digital speech processing lecture notes for background on filtering, spectra, and audio features.
• NIH PubMed: Search for review articles on blind source separation, speech enhancement, and hearing assistive technology.
• IEEE Xplore: Search for papers on microphone arrays, independent vector analysis, and embedded speech separation.
• NASA Open Data and Tutorials: Use signal analysis examples and data handling practices to strengthen your workflow.
• SciPy Documentation: Read the signal processing and FFT sections for practical Python tools you can apply to audio data.