Head-Tracked 3D Phone Display Project

ISEF Category: Technology Enhances the Arts

Subcategory: Display Technology  ·  Difficulty: Advanced  ·  Setup: School Lab  ·  Time: 1 to 2 Months

The Hook

Your phone screen is flat, but your brain can still read it as deep. That happens because your eyes use motion parallax, the way nearby objects seem to move faster than far ones when you shift your head. If you track the viewer and update the image in real time, a simple display can feel much more three-dimensional. That makes this project part art, part optics, and part human perception.

What Is It?

This project asks whether a flat display can feel 3D when it reacts to your head position. The core trick is motion parallax. That means the image changes as you move, just like a real object would. Instead of drawing one fixed picture, the system redraws the scene from the viewer's current eye position.

Think of it like peeking through a window. If the window view shifts as you move left and right, your brain treats the scene as having depth. A Pepper's-ghost setup can strengthen that effect by combining a screen, reflective optics, and a rendered scene that updates in real time. You are not trying to fool everyone the same way. You are testing whether tracking one viewer makes the illusion stronger than a static version.

Why This Is a Good Topic

This is a strong science fair topic because you can test a real perception effect with clear variables. You can compare tracked versus static displays, different scene types, and different viewing distances. The result is measurable with user ratings or forced-choice responses, so you are not stuck with a vague art opinion. You also learn computer vision, human factors, and experimental design, which gives the project real depth.

Research Questions

• How does head tracking change perceived depth compared with a static display?
• What is the effect of viewing distance on the strength of the 3D illusion?
• Does a scene with strong foreground and background separation produce higher depth ratings than a simple scene?
• To what extent does tracking error reduce perceived realism in the display?
• Which display condition leads to more correct forced-choice answers about which image looks deeper?
• How does the speed of head movement affect the stability of the illusion?

Basic Materials

• Smartphone or small monitor with a bright display.
• Raspberry Pi or similar small computer.
• Pi Camera or webcam.
• Mirror or partially reflective acrylic sheet for a Pepper's-ghost style setup.
• Tripod, clamp, or simple stand to hold the camera steady.
• Black cardboard or foam board to block stray light.
• Notebook or spreadsheet for recording user responses.
• Consent form and survey sheet for human subject testing.
• Ruler or measuring tape for viewing distance setup.

Advanced Materials

• Raspberry Pi Camera module or calibrated USB camera.
• Mini display or smartphone with adjustable brightness.
• Partially reflective beam splitter or optical acrylic.
• Computer with Python and OpenCV for camera processing.
• MediaPipe FaceMesh for facial landmark tracking.
• Unity, Godot, or Three.js for real-time scene rendering.
• ImageJ for analyzing captured display frames and alignment error.
• Statistical software such as R or Python SciPy for forced-choice analysis.
• Eye-tracking system, if your lab has one, for validation of gaze alignment.

Software & Tools

• Python: Processes camera input, runs tracking logic, and logs viewer position or response data.
• MediaPipe FaceMesh: Estimates facial landmarks so you can infer eye position in real time.
• OpenCV: Helps you capture video, detect the face, and measure image alignment.
• R: Runs statistical tests on user study results and compares tracked versus static conditions.
• ImageJ: Measures visual alignment and display geometry from screenshots or camera captures.

Experiment Steps

1. Define the exact illusion you want to test, such as perceived depth, realism, or forced-choice accuracy.
2. Choose one variable to change first, such as tracking on versus tracking off, while keeping the scene and brightness fixed.
3. Build a display geometry plan that keeps camera position, screen angle, and viewer position consistent across trials.
4. Design a user study that forces a comparison, not just a yes-or-no opinion.
5. Decide how you will turn perception data into numbers, such as accuracy rates, rating scales, or response times.
6. Plan controls that rule out glare, brightness differences, and simple image quality effects.

Common Pitfalls

• Tracking the wrong part of the face, which makes the image shift in the opposite direction of the viewer's head.
• Changing the room lighting between trials, which alters screen glare and changes perceived depth.
• Using a scene with too little depth separation, which makes the tracked version look almost the same as the static version.
• Letting the display lag behind head movement, which breaks the illusion and adds motion smear.
• Mixing up optical alignment and software effects, which makes you unable to tell whether the illusion came from the rendering or the mirror setup.

What Makes This Competitive

A stronger project would not stop at, 'tracked looks better.' You would measure how much better, under which conditions, and why. That could mean a carefully designed forced-choice test, a clean comparison across multiple scene types, or a statistical model that separates tracking quality from visual realism. You could also test whether different viewers respond differently, which adds a human perception layer that feels much more research-like.

Project Variations

• Test the same tracking idea with different scene types, such as landscapes, geometric shapes, or animated characters.
• Compare webcam-based eye tracking with face-centroid tracking to see whether better eye estimates improve the illusion.
• Measure how the illusion changes when the viewer stands at different distances or angles from the display.

Learn More

• PubMed: Search for review articles on motion parallax, depth perception, and stereopsis to understand the vision science behind the illusion.
• NIH: Look for neuroscience and human perception resources that explain how the brain estimates depth from visual cues.
• NASA: Search for display and human factors research on how people perceive synthetic scenes and motion cues.
• MIT OpenCourseWare: Use open lectures on computer vision, graphics, and perception to learn the technical pieces behind tracking and rendering.
• OpenCV Documentation: Read the free docs for face detection, camera calibration, and real-time video processing examples.
• R Tutorials on CRAN: Use free documentation and examples for analyzing forced-choice data and comparing conditions.