Smart Stage Spotlight Tracking for School Plays

ISEF Category: Technology Enhances the Arts

Subcategory: Engineering Effects  ·  Difficulty: Advanced  ·  Setup: School Lab  ·  Time: Full Year

The Hook

A stage spotlight can miss its mark in seconds, and the audience notices. A small tracking system can keep up with actors without a human operator. That makes this a real engineering problem, not just a cool gadget. You can test whether the machine tracks better, faster, and with less crew effort than a person.

What Is It?

This project builds an automated spotlight that points at the most active performer on stage. The system uses a camera to detect faces and a microphone signal to detect who is speaking. Then it fuses those signals, which means it combines two sources of information to make one decision.

Think of it like a goalie watching both the ball and the player’s body. If one clue gets messy, the other can help. A face detector can find where actors are, and voice activity detection, or VAD, can show who is speaking. Your pan-tilt servo head then turns the light toward that target.

The main science fair question is not just, “Can it work?” You also want to know how steady the tracking looks, how fast it reacts, and whether it reduces the work of a lighting crew during a live rehearsal or school play.

Why This Is a Good Topic

This is a strong science fair topic because you can measure real engineering performance. You can track objective numbers like jitter, response time, target loss, and operator workload. The project connects to theater, live events, accessibility, and low-cost automation. You can learn about computer vision, signal fusion, control systems, and experimental design without needing a professional lab.

Research Questions

• How does audio-visual fusion affect spotlight tracking accuracy compared with camera-only tracking?
• What is the effect of actor movement speed on tracking jitter?
• Does adding voice-activity detection reduce target switching during overlapping scenes?
• To what extent does ambient stage noise change the system's ability to identify the main speaker?
• Which face-detection confidence threshold gives the best balance between steady tracking and missed targets?
• How does automated tracking change the number of manual adjustments a lighting operator must make?

Basic Materials

• Used Jetson Nano board with power supply.
• USB camera with tripod or stage mount.
• Pan-tilt servo mount or small gimbal head.
• High-intensity LED spotlight or small theater light.
• Microphone or USB microphone.
• Breadboard and jumper wires.
• Servo controller or compatible interface board.
• Laptop for setup, logging, and data review.
• Tape measure or floor markers for stage positions.
• Notebook or spreadsheet for observations.

Advanced Materials

• Jetson Nano with compatible cooling and storage.
• Wide-angle camera with known field of view.
• Directional microphone or microphone array.
• Pan-tilt mount with encoder feedback, if available.
• Lighting control relay or DMX interface for integration tests.
• Calibration target for camera alignment.
• Sound level meter for scene noise profiling.
• High-speed reference camera for ground-truth motion tracking.
• Reference actor marks and stage layout map.
• Access to a school theater or rehearsal space.

Software & Tools

• Python: Runs the tracking pipeline, logs outputs, and supports analysis.
• OpenCV: Handles video capture, face detection support, and frame measurements.
• ImageJ: Measures spotlight position error and frame-to-frame jitter from recordings.
• Jupyter Notebook: Organizes plots, summaries, and comparison tables.
• YOLOv8: Detects people or faces when you test vision-based target selection.

Experiment Steps

1. Define the tracking goal, including what counts as the target performer and what counts as a miss.
2. Build a baseline version, then compare it with the audio-visual fusion version under the same stage setup.
3. Choose the main performance metrics, such as jitter, target acquisition time, and manual intervention count.
4. Plan a calibration method that maps camera coordinates to spotlight movement on stage.
5. Design test scenes that vary one factor at a time, such as motion, noise, or overlapping speech.
6. Prepare a logging plan so you can match each spotlight move to the actor's position and voice signal.

Common Pitfalls

• Training on rehearsal footage from only one lighting setup, which makes the detector fail when the stage looks different.
• Letting microphone pickup dominate the target choice, which can steer the spotlight toward off-stage noise or the wrong actor.
• Measuring success only by visual smoothness, which hides cases where the light tracks the wrong person very smoothly.
• Skipping coordinate calibration between the camera view and the pan-tilt head, which creates constant pointing offset.
• Testing with too few scene types, which makes the system look good in one scene and weak in real play conditions.

What Makes This Competitive

A stronger project goes beyond a simple demo. You can compare separate sensor modes, test multiple scenes, and report real metrics like tracking error distribution and false target switches. You can also study how different fusion rules change performance under noise, occlusion, and overlapping speech. If you add a careful human-factor measure, like how much work the lighting operator saves, the project becomes much stronger.

Project Variations

• Test whether the system works better with face landmarks only, voice activity only, or fused signals during scenes with blocking changes.
• Compare spotlight tracking on small classroom stages versus a full auditorium to see how distance and camera angle change performance.
• Measure whether different decision rules, such as majority vote or confidence weighting, reduce false target switches in ensemble scenes.

Learn More

• NOAA Theater and stage lighting guides: Search NOAA and university extension theater safety resources for basics on lighting angles, glare, and stage visibility.
• NASA Open Source Robotics and vision resources: Search NASA technical reports and code repositories for examples of vision-based tracking and sensor fusion.
• NIH PubMed: Search for review articles on audiovisual perception, speaker localization, and human attention in group settings.
• MIT OpenCourseWare, Computer Vision: Search MIT OpenCourseWare for free lecture notes on image processing, object detection, and tracking.
• OpenCV documentation: Find the free official docs for camera capture, detection, and video analysis examples.
• IEEE Xplore and arXiv: Search for papers on audio-visual fusion, face tracking, and spotlight or camera auto-follow systems.