Museum Audio Guide App for Visitor Recall

ISEF Category: Technology Enhances the Arts

Subcategory: Other  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

Museums often lose visitors in the first minute. A label on the wall is like a flashlight with weak batteries, it gives you some light, but not much direction. Your app could act like a smart guide that notices the room, picks the right context, and speaks to the visitor at the right moment. Then you can test whether that changes what people remember.

What Is It?

This project studies a museum guide app that listens to ambient sound, checks location with GPS, and uses that context to generate spoken commentary about an artwork or exhibit. Think of it like a librarian who knows which shelf you are standing in front of, then whispers the right summary instead of reading a random page. The technical idea is context-aware audio guidance, which means the app uses signals from the environment to decide what content to present.

You also measure the human side. Do people stay longer near the exhibit? Do they remember more facts later? Those outcomes turn your app from a cool demo into a research project. You are not just building software, you are testing whether context changes how people learn from art.

Why This Is a Good Topic

This is a strong science fair topic because you can test clear variables, like guidance mode, audio style, or context detection accuracy, and measure real outcomes such as dwell time and recall. The project connects to museums, accessibility, and visitor experience, so it has a real-world use. You can learn app design, audio classification, evaluation metrics, and human-subject study planning without needing a wet lab.

Research Questions

• How does ambient-sound classification accuracy change when the app moves between different exhibit spaces?
• What is the effect of context-aware spoken commentary on visitor dwell time compared with printed labels alone?
• Does personalized audio generated from a small language model improve immediate recall of exhibit facts?
• To what extent does GPS-based exhibit detection reduce the number of wrong commentary prompts?
• Which audio style, short facts or story-like narration, leads to better delayed recall?
• How does background noise level affect the usability of on-device speech output in a museum setting?

Basic Materials

• Smartphone with microphone and GPS access.
• Headphones or small speaker for playback testing.
• Test exhibit images or printed mock exhibits.
• Printed labels for the control condition.
• Survey forms for recall questions.
• Stopwatch or timestamp logging app.
• Notebook for observing visitor behavior.
• Consent forms for any human-subject testing.

Advanced Materials

• Android phone with local model support.
• Laptop for model development and evaluation.
• Publicly available audio dataset for ambient-sound classification.
• VGGish-compatible feature extraction pipeline.
• Python environment for training and analysis.
• GPS or indoor-positioning test logs.
• Raspberry Pi or similar edge device for deployment comparisons.
• Voice synthesis testing setup for Piper TTS.
• Statistical analysis package for visitor study results.

Software & Tools

• Python: Runs feature extraction, model testing, and statistical analysis.
• TensorFlow Lite: Supports on-device inference for compact classification models.
• Librosa: Extracts and visualizes audio features from sound clips.
• ImageJ: Can help if you also analyze exhibit image salience or screen captures.
• R: Helps compare dwell time and recall results across visitor groups.

Experiment Steps

1. Define the exact visitor problem you want to solve, such as better recall, longer dwell time, or faster exhibit identification.
2. Choose one context signal to test first, then decide how you will compare it against a simpler baseline.
3. Design a fallback path for when audio recognition or GPS confidence is low, so the app does not guess blindly.
4. Plan your outcome measures before you build, including how you will score recall and log visitor behavior.
5. Build a fair comparison between printed labels, static audio, and context-aware audio, with the same content length where possible.
6. Select statistical tests that match your data type, then predefine how you will handle unclear responses or missed detections.

Common Pitfalls

• Using visitor dwell time as the only success metric, which can hide confusion or distraction.
• Letting the app generate different amounts of content for each exhibit, which makes recall results hard to compare.
• Testing in a quiet room only, which misses the impact of museum background noise on audio detection and playback.
• Treating GPS as accurate indoors, which can cause the app to attach commentary to the wrong exhibit.
• Mixing up content quality with delivery mode, which makes it unclear whether people liked the facts or the speaking style.

What Makes This Competitive

A stronger version of this project would not just build the app, it would test why the app works. You could compare multiple context signals, measure accuracy under noisy conditions, and separate content quality from delivery quality. Strong studies also use clean controls, enough participants, and statistics that match the data. A novel angle, like accessibility for low-vision visitors or museum-specific indoor localization, could make the project much more compelling.

Project Variations

• Test the same idea in a science museum instead of an art museum, then compare whether visitors remember factual labels better than narrative labels.
• Replace GPS with Bluetooth beacons or QR-based room detection, then measure whether indoor positioning improves prompt accuracy.
• Compare synthesized voices, such as a neutral voice versus a warm voice, and measure which one keeps visitors engaged longer.

Learn More

• PubMed: Search for review articles on museum learning, visitor engagement, and audio-based education studies.
• NIH PubMed Central: Find free full-text papers on human-computer interaction, speech synthesis, and user studies.
• NASA Open Data Portal: Explore public datasets and examples of on-device classification workflows for sensor-based projects.
• MIT OpenCourseWare: Search for courses on machine learning, signal processing, and user interface design.
• Computers & Education: Search the journal for studies on digital guides, learning outcomes, and attention in informal education.