Emotion-Mirroring Companion Robot for Seniors

ISEF Category: Robotics and Intelligent Machines

Subcategory: Cognitive Systems  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

Older adults can spend long stretches without a calm, friendly voice nearby. A soft robot that listens for emotion and answers with light and movement tries to fill that gap. The hard part is not building the plush shell. It is testing whether people read the signals the way you expect.

What Is It?

This project asks you to build a companion robot that senses emotion in speech and changes its behavior in response. If the speaker sounds stressed, the robot might glow a warm color or make a gentle purr. If the speaker sounds calm, it might stay steady and quiet. The core idea is simple, like a mood ring that listens instead of feeling heat.

Speech-emotion recognition means a model looks at voice features such as pitch, energy, and rhythm, then guesses an emotion class like calm, sad, or upset. Wav2Vec2 small is a prebuilt speech model that can be adapted for this job. On-device means the model runs on the robot itself or a nearby local device, not on a cloud server. That matters because older adults may need better privacy, faster response, and less setup hassle.

Why This Is a Good Topic

This topic works well for a science fair because you can test both the machine learning side and the human interaction side. You can measure whether the model classifies emotion well, whether the robot’s signals are easy to understand, and whether people prefer one feedback style over another. It also connects to a real problem, which is social support for older adults and better emotional sensing in assistive robots. You can learn about data labeling, model evaluation, user studies, and ethical design.

Research Questions

• How does the choice of emotion class set affect speech-emotion recognition accuracy for older adult voices?
• What is the effect of running the model on-device versus on a laptop on response delay and battery use?
• Does LED color mapping change how accurately seniors interpret the robot’s mood response?
• To what extent do gentle purr patterns improve perceived comfort compared with light-only feedback?
• Which voice features contribute most to correct emotion predictions for calm, stressed, and sad speech?
• What is the effect of personalized calibration on recognition accuracy across different speakers?

Basic Materials

• Plush toy shell or soft robot enclosure
• Single-board computer or laptop for local inference
• USB microphone or headset microphone
• RGB LED strip or RGB LED module
• Small vibration motor or haptic purr motor
• Motor driver or transistor module
• Battery pack or power bank
• Breadboard and jumper wires
• Basic sewing kit or craft tools
• Consent forms and simple diary sheets
• Spreadsheet for logging responses

Advanced Materials

• Microcontroller or single-board computer with offline inference support
• USB microphone with known frequency response
• RGB LED hardware with diffuser materials
• Small speaker or vibration actuator for purr feedback
• IMU or pressure sensor for interaction logging
• 3D-printed or laser-cut internal mounts
• External battery monitor
• Audio recording interface for labeled speech samples
• Laptop with Python and model training libraries
• Secure local storage device for participant data

Software & Tools

• R or Google Sheets: Helps you graph accuracy, response time, and diary data.

Experiment Steps

1. Define the exact emotion labels you will test and how each label will map to a robot response.
2. Choose whether you will compare raw model outputs, hand-built speech features, or both.
3. Plan a small validation set with balanced speakers, recording conditions, and emotion classes.
4. Design a response scheme for the LEDs and purr motor that keeps feedback clear, gentle, and consistent.
5. Build a pilot test that checks model accuracy, latency, and whether people understand the robot’s signals.
6. Design the diary study so you can compare comfort, clarity, and trust across users and sessions.

Common Pitfalls

• Ignoring privacy and consent for voice recordings, which can block the study or make the data unusable.

What Makes This Competitive

A strong version of this project goes beyond making a cute robot. You would compare multiple emotion models, test calibration across different speakers, and measure both technical performance and user trust. You could also analyze which feedback channel, light, vibration, or both, actually helps seniors read the robot’s state. Careful statistics and a real pilot study make the work feel like research, not a demo.

Project Variations

• Test the same robot with caregiver speech instead of senior speech to compare recognition performance across age groups.
• Swap the plush form factor for a tabletop companion and see whether touchable design changes comfort ratings.
• Compare emotion mapping strategies, such as color-only, purr-only, and combined feedback, to see which one users understand best.

Learn More

• Google Scholar: Search for recent papers on Wav2Vec2, speech emotion recognition, and on-device inference.