Deepfake Voice Liveness for Phone Banking

ISEF Category: Systems Software

Subcategory: Cybersecurity  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A voice can sound real and still be fake. Cheap speakers, cloned speech, and replay attacks can all fool a phone system if it listens the wrong way. Your project asks a sharp question, can a hidden audio challenge prove a caller is live? That kind of test matters for banking, account recovery, and any service that trusts a voice.

What Is It?

This project studies voice liveness, which means checking whether a caller is a real person speaking live or a recording, clone, or replay. Think of it like asking for a secret hand signal during a phone call. The system sends a tiny audio challenge, then listens for the reply. If the sound changes in a way that matches a live microphone path, the caller passes. If the signal looks copied, delayed, or distorted in a fake way, the system flags it.

Phase-modulated tones are a clever kind of challenge. Phase is the timing position of a wave, and small phase changes can carry information without sounding obvious to people. A laptop can play these tones, while a phone microphone captures the response. Your job is to see whether different replay devices, cheap speakers, and voice-clone setups leave measurable traces in that response. You are not just asking whether the audio sounds fake. You are testing whether the system can measure a hidden fingerprint in the signal path.

Why This Is a Good Topic

This is a strong science fair topic because you can test a real security problem with measurable audio data. It connects to fraud prevention, identity verification, and deepfake defense, all problems people care about now. You can vary the attack type, the speaker, the playback device, or the room, then compare how well the system detects each case. That gives you a clear independent variable, a clear output score, and room for solid statistics.

Research Questions

• How does speaker quality affect the system’s ability to detect replay attacks?
• What is the effect of playback device type on phase response features used for liveness detection?
• Does room background noise reduce the accuracy of a microphone-side challenge-response check?
• To what extent can the system separate live speech from cloned speech when both use the same device?
• Which acoustic features most strongly distinguish a live microphone path from a replayed signal?
• How does distance between the speaker and microphone change detection performance?

Basic Materials

• Laptop with a microphone and headphone output.
• Smartphone or second phone for recording test calls.
• Cheap consumer speaker or small Bluetooth speaker.
• Headphones for monitoring audio playback.
• Quiet room for baseline tests.
• Audio editing or analysis software on a laptop.
• Spreadsheet software for logging results.
• Notebook for trial conditions and labels.

Advanced Materials

• Measurement microphone or calibrated USB microphone.
• Audio interface with low-noise input.
• Access to multiple consumer speakers with different drivers and enclosures.
• Oscilloscope or audio spectrum analyzer if available.
• Collection of recorded live, replayed, and synthetic voice samples.
• Controlled acoustic test space or sound-treated room.
• Computer with Python installed for signal processing.
• Reference playback chain for repeatable challenge delivery.

Software & Tools

• Python: Processes audio files, extracts features, and runs classification tests.
• Audacity: Lets you inspect waveforms, compare recordings, and check noise artifacts.
• ImageJ: Can help visualize spectrogram snapshots and compare signal patterns if you export images.
• Jupyter Notebook: Keeps your analysis, plots, and notes in one place.
• Excel: Organizes trial labels, attack conditions, and summary statistics.

Experiment Steps

1. Define the exact attack types you will test, such as live speech, replay, and cloned speech.
2. Choose the signal features you will measure, such as timing shifts, phase response, or spectral differences.
3. Build a baseline dataset from clean live calls so you know what normal looks like.
4. Plan a matched set of spoof trials so each attack type uses the same script, device class, and listening setup.
5. Decide how you will score detection, such as false accept rate, false reject rate, and overall accuracy.
6. Preplan the statistics you will use to compare conditions and avoid cherry-picking the best runs.

Common Pitfalls

• Using recordings from only one voice, which makes the system look better than it really is.
• Testing replay attacks in one quiet room only, which hides how noise changes the signal.
• Changing speaker volume between trials, which adds a confounding variable to the phase measurement.
• Mixing live and spoof samples in the same folder without strict labels, which breaks later analysis.
• Judging success by audio quality alone instead of accuracy, false accepts, and false rejects.

What Makes This Competitive

A class-level version of this project shows that the idea works. A stronger version tests several attack types, several speakers, and several rooms, then reports error rates with real statistical comparisons. You can raise the level further by comparing your method to a simple baseline, like energy or spectral features, and by checking how well it holds up across devices. Strong projects also explain failure cases, not just wins.

Project Variations

• Test the same challenge-response idea against voice-clone samples from different text-to-speech systems.
• Compare wired speakers, Bluetooth speakers, and laptop speakers as replay devices.
• Measure whether your detector still works when the caller uses a different room or background noise level.

Learn More

• NOAA Acoustic Resources: Search NOAA for educational material on sound, waves, and spectral analysis concepts.
• MIT OpenCourseWare Signals and Systems: Find free lecture notes and problem sets on audio signals and filtering.
• NIST Digital Identity Guidelines: Search the NIST site for guidance on authentication and spoofing risks.
• PubMed: Search for review articles on voice spoofing detection, replay attacks, and speaker verification.
• IEEE Xplore abstracts: Search for recent papers on replay attack detection and voice liveness, then read abstracts and open-access versions when available.