Haptic Vests for Orchestral Music Experience

ISEF Category: Technology Enhances the Arts

Subcategory: Other  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

What if you could feel a symphony instead of hearing it? A vest with 16 vibration motors can turn music into a body map of rhythm, pitch, and texture. That could make concerts more accessible for deaf and hard-of-hearing listeners. Your project can test whether that feeling actually improves the concert experience.

What Is It?

This project turns music into touch. A haptic vest uses small vibration motors, called LRAs, or linear resonant actuators, to send signals across the body. Each motor can stand for a different sound source, such as violins, brass, percussion, or vocals. The user feels a pattern of taps, pulses, and motion instead of only hearing sound.

The key idea is translation. Demucs is a machine learning tool that separates a song into stems, which are individual instrument tracks. You can map those stems to different motors, then decide how strong, fast, and where each vibration should appear. Think of it like subtitles for the body. You are not copying the music perfectly. You are creating a new sensory version that may help people follow the structure, energy, and emotion of a performance.

Why This Is a Good Topic

This makes a strong science fair topic because you can test both the device and the user experience. You can measure whether different vibration maps help people recognize musical changes, follow instruments, or report higher enjoyment. The project connects to accessibility, assistive technology, and performance design. You can learn signal mapping, user testing, and basic statistics without needing a full lab.

Research Questions

• How does a stem-based vibration map affect deaf or hard-of-hearing listeners' reported enjoyment of orchestral music?
• What is the effect of spatial motor placement on a listener's ability to identify musical sections?
• Does adding separate vibration channels for percussion improve beat detection compared with a single-channel vest?
• To what extent does vibration intensity mapping change comfort and fatigue during a performance simulation?
• Which stem assignment strategy leads to the best recognition of instrument changes from vibration alone?
• How does a haptic vest compare with no vest control in ratings of immersion and emotional response?

Basic Materials

• Wearable vest or strap system with mounting points for 16 LRA motors.
• 16 LRA motors with driver boards.
• Microcontroller such as Arduino or Raspberry Pi Pico.
• Battery pack rated for safe wearable use.
• Smartphone or laptop for playback and control.
• Headphones or speakers for test audio setup.
• Printed survey forms or online survey tool.
• Stopwatch or timing app.
• Tape measure for consistent motor placement.
• Basic sewing kit or fabric adhesive for assembly.

Advanced Materials

• Vest frame with adjustable motor mounts and washable outer shell.
• 16 LRA motors with calibrated driver circuits.
• Microcontroller with multichannel PWM or serial control.
• Audio separation pipeline for Demucs stem processing.
• Data logging device for motor timing and trigger validation.
• Heart rate or skin response sensor for optional comfort or arousal measures.
• Acoustic or vibration meter for validation of output consistency.
• Motion capture or inertial sensor system for posture and movement checks.
• Computer with Python for analysis and visualization.
• IRB or school human-subjects approval materials if required.

Software & Tools

• Demucs: Separates songs into instrument stems that you can map to different motors.
• Python: Processes audio, controls test conditions, and analyzes survey results.
• Audacity: Lets you inspect audio tracks and confirm stem timing by ear and waveform.
• ImageJ: Can help measure and compare vibration marker patterns in simple prototype tests.
• Google Forms: Collects participant ratings and short responses in a consistent format.

Experiment Steps

1. Define the exact user experience you want to improve, such as beat tracking, instrument awareness, or overall enjoyment.
2. Choose one vibration mapping rule and one control condition so you can compare them fairly.
3. Plan how each stem will connect to motor position, intensity, and timing before you build the vest.
4. Design a small pilot test to check whether people can tell different musical events apart through vibration.
5. Set up a rating system for enjoyment, comfort, clarity, and fatigue so your results are measurable.
6. Decide in advance how you will analyze differences between the vest condition and the no-vest control.

Common Pitfalls

• Mapping too many musical elements at once, which makes the vibration pattern feel random instead of readable.
• Letting motor placement drift between participants, which changes the meaning of each vibration zone.
• Using songs with muddy stem separation, which blurs the signal and weakens the comparison.
• Measuring only enjoyment without checking comprehension or comfort, which leaves the result too vague.
• Ignoring volume, posture, or seat location in the control condition, which can confound the user ratings.

What Makes This Competitive

A stronger version of this project would test a clear design hypothesis, not just build a cool vest. You could compare multiple stem-mapping strategies, then use statistics to see which one helps users most. A competitive project also needs tight controls, such as matched songs, fixed motor layouts, and blind or randomized trials. If you add objective measures, like beat recognition accuracy or response time, your results become much more persuasive.

Project Variations

• Test the vest with pop music instead of orchestral music to see whether simpler stems improve pattern recognition.
• Compare a 16-motor layout with a smaller 8-motor version to find the minimum hardware needed for useful musical detail.
• Add a lip-reading or caption condition so you can measure whether combined cues improve accessibility more than haptics alone.

Learn More

• Demucs documentation and papers: Search for the official Demucs GitHub repository and the linked paper to learn how source separation works.
• NIH PubMed: Search for review articles on haptic feedback, deaf accessibility, and music perception.
• IEEE Xplore: Search for papers on wearable haptics and music-to-vibration mapping, then use abstracts and accessible PDFs when available.
• MIT OpenCourseWare: Look for courses on digital signal processing to build intuition for audio features and timing.
• The Journal of the Acoustical Society of America: Search the journal for studies on vibrotactile music perception and accessibility.