Smartphone Heart Rate and SpO2 Inference

ISEF Category: Embedded Systems

Subcategory: Optics  ·  Difficulty: Advanced  ·  Setup: School Lab  ·  Time: Full Year

The Hook

Your phone camera can pick up tiny color changes in skin that match your pulse. That means your next science fair project could turn video frames into health data. Add motion data from the phone’s IMU, and you can test whether motion correction really improves accuracy. This is a strong mix of computer vision, signal processing, and real-world sensing.

What Is It?

This project asks a simple question with a tricky answer, can a phone camera estimate heart rate and blood oxygen well enough to compete with a sensor made for the job? The camera looks at subtle changes in color across RGB frames. Those changes come from blood volume pulsing under the skin. A convolutional neural network, or CNN, is a model that learns patterns in images. In this case, it tries to map video and motion data to a pulse estimate.

The IMU, or inertial measurement unit, is the phone’s motion sensor. It tracks acceleration and rotation. That matters because hand movement can fool the camera into reading noise as signal. Think of it like trying to hear a whisper in a moving car. The video carries the useful signal, but motion can bury it. Your job is to test whether combining RGB frames with IMU data makes the estimate more stable and more accurate.

Why This Is a Good Topic

This is a strong science fair topic because you can test a real sensor problem, measure performance with clear numbers, and compare two approaches head-to-head. You can ask whether motion compensation, model size, or lighting conditions change accuracy. The project connects to telehealth, fitness wearables, and low-cost screening tools. You can learn data collection, model training, calibration, and error analysis without needing a hospital lab.

Research Questions

• How does adding IMU motion data change heart-rate prediction error from smartphone RGB video?
• What is the effect of different lighting conditions on CNN-based pulse estimation accuracy?
• Does a small CNN perform better than a simple signal-processing baseline for heart-rate inference?
• To what extent does skin tone affect SpO2 prediction error from phone-camera data?
• Which video frame window length gives the lowest mean absolute error for heart-rate estimation?
• How does motion intensity change the gap between the smartphone estimate and the MAX30102 reference?

Basic Materials

• Smartphone with camera and IMU sensors.
• $20 MAX30102 pulse oximeter module or reference device.
• Tripod or phone stand.
• Simple test targets or mounting setup to keep camera distance fixed.
• Consent forms for any human data collection.
• Notebook or spreadsheet for logging trial conditions.
• Controlled indoor light source.
• Laptop for data transfer and analysis.

Advanced Materials

• Smartphone with raw camera access and IMU logging.
• MAX30102 sensor with microcontroller interface.
• Microcontroller board such as Arduino or ESP32 for synchronized reference logging.
• Calibration chart or pulse-oximeter comparison setup.
• Computer with GPU access if available.
• Colored filters or neutral-density filters for lighting tests.
• Repeatable motion rig or metronome-based movement protocol.
• Skin-tone reference cards or a diverse participant set for bias testing.

Software & Tools

• Python: Processes video, sensor logs, and error metrics for model training and evaluation.
• OpenCV: Extracts frames, handles color channels, and supports image preprocessing.
• NumPy: Organizes numeric data and builds feature arrays for modeling.
• pandas: Stores trial metadata and compares model outputs across conditions.
• ImageJ: Checks frame quality and helps inspect color changes across sample clips.

Experiment Steps

1. Define one target first, such as heart rate only, before you add SpO2 or multiple outputs.
2. Choose a reference method and plan how you will align camera data with the MAX30102 readings.
3. Decide which inputs the model will see, RGB frames only, RGB plus IMU, or a baseline feature set.
4. Plan a data split that keeps the same person or recording session from leaking into both training and testing.
5. Build a calibration and evaluation plan that reports error, bias, and consistency under motion.
6. Design a stress test for lighting, movement, and skin tone so you can see where the model fails.

Common Pitfalls

• Using shaky hand-held video, which adds motion blur and hides the color signal you need.
• Collecting reference and camera data without tight synchronization, which makes the model learn the wrong target.
• Training on clips from one lighting setup only, which causes the model to fail as soon as the room changes.
• Mixing training and test samples from the same recording session, which inflates accuracy and hides overfitting.
• Ignoring skin tone, motion level, or participant differences, which can make the model look good on average but poor in edge cases.

What Makes This Competitive

A competitive version does more than report one accuracy number. You compare multiple model designs, show how motion correction changes error, and test whether the method holds up across lighting and skin tone. Strong projects also use clean validation, separate people in train and test sets, and clear statistics, not just screenshots of predictions. If you can explain where the model breaks and why, your work looks much stronger.

Project Variations

• Test heart-rate estimation during walking versus sitting to measure how motion changes model error.
• Compare a CNN that uses RGB frames only with one that also uses IMU features.
• Focus on SpO2 estimation under different lighting colors and camera distances to study calibration drift.

Learn More

• NIH PubMed: Search review articles on photoplethysmography, remote heart-rate sensing, and SpO2 estimation.
• NASA Open Data Portal: Explore free sensor-data examples and data-handling ideas for time-series analysis.
• MIT OpenCourseWare: Look for computer vision and machine learning course materials to build your modeling background.
• NOAA Earth System Research Laboratories: Review signal-processing and time-series analysis examples that transfer well to noisy sensor data.
• Journal of Biomedical Optics: Search for peer-reviewed papers on camera-based pulse monitoring and remote photoplethysmography.