Blind-Friendly Shelf Finder With Depth Audio

ISEF Category: Robotics and Intelligent Machines

Subcategory: Cognitive Systems  ·  Difficulty: Advanced  ·  Setup: School Lab  ·  Time: 1 to 2 Months

The Hook

A store aisle can turn into a maze when you cannot see price tags or shelf labels. Your phone can talk, but it cannot always point to the right item fast. A cane-mounted system that estimates depth and speaks distances could close that gap. Your project can measure whether that extra spatial cue really saves time.

What Is It?

This project asks a simple question, can a camera and an AI model help you find items on a shelf faster than speech alone? Depth estimation means turning a flat image into a rough map of which objects are nearer and which are farther away. Think of it like a very fast guess about who is standing in the front row and who is in the back row.

The system uses a camera on a cane handle, a depth model such as DepthAnything, and bone-conduction audio. Bone-conduction headphones send sound through your cheekbones, so your ears stay open to the world around you. That matters in a store, where you still need to hear carts, voices, and warnings. Your output can rank the nearest three items and give each one a distance estimate, so the user gets a simple spatial cue instead of a long speech prompt.

Why This Is a Good Topic

This makes a strong science fair topic because you can test a clear, measurable outcome, search time. You can compare your system against a verbal-only phone app and see whether spatial audio plus depth estimates helps users find targets faster or with fewer errors. The project connects to accessibility, assistive robotics, and human-computer interaction, which gives it real-world value. You can also learn computer vision, model evaluation, and experiment design without needing a full university lab.

Research Questions

• How does a depth-estimation audio aid change shelf-search time compared with a verbal-only smartphone baseline? ?
• What is the effect of item spacing on how accurately the system identifies the nearest three products? ?
• Does the system reduce search errors more for high-contrast packaging than for similar-looking packaging? ?
• To what extent does bone-conduction audio improve target-finding speed compared with standard earbuds? ?
• Which display angle produces the most stable distance estimates across repeated shelf scans? ?
• How does shelf clutter affect the agreement between model-ranked items and the actual nearest items? ?

Basic Materials

• Smartphone or small camera module with video capture.
• Cane handle or handheld mount for the camera.
• Bone-conduction headphones.
• Laptop with internet access for model setup and analysis.
• Tape measure or ruler for ground-truth distances.
• Printed shelf targets or grocery items with varied packaging.
• Stopwatch or phone timer.
• Data sheet or spreadsheet for recording trials.

Advanced Materials

• Depth camera or standard RGB camera with stable mounting rig.
• Single-board computer or compact processor for on-device inference.
• External battery pack for field testing.
• Bone-conduction audio device with adjustable output levels.
• Calibration markers for distance validation.
• Reference targets for depth accuracy checks.
• Quiet test aisle or mock shelf setup.
• Access to Python development tools and model inference libraries.

Software & Tools

• Python: Runs the depth estimation pipeline and logs test results.
• OpenCV: Handles image capture, resizing, and basic computer vision tasks.
• DepthAnything: Estimates relative depth from shelf images.
• ImageJ: Helps measure item spacing and compare detected positions with ground truth.
• Google Sheets: Organizes trial data and calculates summary statistics.

Experiment Steps

1. Define the user task, such as finding one target item or the nearest three items on a shelf.
2. Choose the signal you will compare, including spoken-only guidance, depth-audio guidance, or a third control.
3. Plan how you will measure success, such as time, accuracy, and number of wrong picks.
4. Build a ground-truth method for shelf distance and item identity so you can score the model fairly.
5. Design test scenes that vary clutter, lighting, item height, and packaging similarity.
6. Set up a repeatable analysis plan that compares performance across trial types and user conditions.

Common Pitfalls

• Letting the camera angle change between trials, which shifts the depth map and breaks comparison across shelves.
• Testing only one easy shelf, which hides how the system fails on cluttered or similar-looking packaging.
• Using spoken prompts that give too much detail, which makes the baseline unfairly weak.
• Ignoring latency from capture to audio output, which matters because slow feedback can erase speed gains.
• Skipping a ground-truth check for item distance, which makes it hard to tell whether errors came from the model or the experiment setup.

What Makes This Competitive

A competitive version of this project does more than show that the tool works once. You would test several shelf layouts, compare at least two guidance styles, and report both speed and error rate. Strong entries also check calibration, latency, and failure cases, not just average improvement. If you can show when the system helps, when it hurts, and why, your project becomes much stronger.

Project Variations

• Test the same system on a home pantry shelf instead of a grocery shelf to see how clutter changes depth accuracy.
• Replace the cane mount with a chest mount and compare whether the camera angle improves item ranking.
• Add a vibration cue for the nearest item and compare tactile feedback with bone-conduction audio alone.

Learn More

• NIH PubMed: Search review articles on assistive vision, object detection, and human-computer interaction for blind users.
• NASA Open Source Software: Look for free computer vision and embedded AI tools that can support real-time inference.
• MIT OpenCourseWare, Introduction to Computer Vision: Use lecture notes to review image processing and depth concepts.
• OpenCV Documentation: Find free guides for video capture, image preprocessing, and camera calibration.
• IEEE Xplore abstract search: Read abstracts on wearable assistive vision and depth estimation to see common evaluation methods.