Optical Flow Attack Defense for Rovers

ISEF Category: Robotics and Intelligent Machines

Subcategory: Other  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A rover can look fine to you and still fail on the road. A small printed patch can trick a vision system into missing an obstacle. That makes this project a real test of how fragile robot perception can be. You get to measure the weakness, then try to fix it.

What Is It?

This project looks at optical flow, which is the motion pattern a camera sees as the rover moves. A safe rover uses that motion to judge distance, spot obstacles, and decide when to stop or turn. If a printed pattern confuses the flow estimate, the rover may think the path is clear when it is not.

Think of optical flow like watching raindrops slide across a window while a car moves. The robot turns those streaks into a map of motion. An adversarial patch is a carefully designed image that acts like a fake visual cue. Even though the patch is just ink on paper, it can bend the robot's perception enough to cause a missed obstacle.

Why This Is a Good Topic

This is a strong science fair topic because you can test a clear input, measure a clear output, and compare a baseline against a defense. You can vary patch shape, size, placement, lighting, and motion, then track miss-rate and false-stop rate. The topic connects to self-driving cars, warehouse robots, and drones, so the real-world stakes are easy to explain. You also learn computer vision, experiment design, and how to judge whether a defense actually helps.

Research Questions

• How does patch size affect the miss-rate of an optical-flow obstacle avoidance system?
• How does patch placement on the obstacle change the rover's detection accuracy?
• What is the effect of lighting level on the patch's ability to fool the vision system?
• To what extent does input randomization reduce miss-rates without raising false stops?
• Which patch pattern creates the largest drop in obstacle detection across repeated trials?
• How does rover speed change the system's sensitivity to adversarial patches?

Basic Materials

• Wheeled rover platform with a forward-facing camera.
• Laptop with Python and OpenCV installed.
• Printed high-contrast test patches on matte paper.
• Flat test track with simple obstacle objects.
• Measuring tape for fixed camera and obstacle placement.
• Tripod or rigid mount for the camera or rover sensor.
• Consistent indoor lighting setup.
• Notebook or spreadsheet for trial logs.

Advanced Materials

• Rover platform with programmable camera pipeline.
• High-resolution camera with adjustable exposure.
• 3D-printed or laser-cut patch holders.
• Calibrated lighting setup with dimmable sources.
• GPU-capable workstation for computer vision experiments.
• ImageJ or Python image-processing tools for patch verification.
• Access to a wind-free indoor test lane or robotics lab space.
• Screened obstacle set with known reflectance and texture.

Software & Tools

• Python: Processes video frames, runs optical flow, and logs detection outcomes.
• OpenCV: Computes optical flow and supports image pre-processing experiments.
• ImageJ: Checks printed patch contrast, scale, and repeatability before testing.
• Google Sheets: Organizes trial data and calculates miss-rate summaries.
• Jupyter Notebook: Keeps analysis, plots, and notes in one reproducible place.

Experiment Steps

1. Define the failure mode you want to measure, such as missed obstacle detection, delayed stopping, or wrong turn decisions.
2. Choose one baseline rover vision pipeline and document exactly how it judges motion and obstacles.
3. Design a patch family with controlled changes in shape, contrast, scale, and placement.
4. Plan a fair test matrix that separates patch effects from lighting, distance, and rover speed.
5. Add a simple input-randomization defense and decide how you will compare it against the baseline.
6. Select metrics that capture both attack success and defense cost, such as miss-rate, false-stop rate, and detection latency.

Common Pitfalls

• Changing camera exposure between runs, which makes optical flow differences look like patch effects.
• Testing patches only one time each, which hides run-to-run variation in the rover's perception.
• Printing patches on glossy paper, which adds reflections that confound the attack.
• Mixing up obstacle texture and patch texture, which makes it hard to tell what caused the failure.
• Judging the defense only by lower miss-rate, while ignoring that it may also cause many false stops.

What Makes This Competitive

A strong version of this project uses tight controls and a clean evaluation plan. You would compare several patch designs, not just one, and report confidence intervals or significance tests, not only raw percentages. You could also test whether the defense still works when you change lighting, speed, or patch placement. That kind of multi-factor analysis shows that you understand both the attack and the limits of the fix.

Project Variations

• Test whether motion blur makes the patch attack stronger on a faster rover.
• Compare optical-flow attacks against a color-based obstacle detector to see which perception method fails first.
• Evaluate whether adding randomized cropping, noise, or frame skipping improves defense under different lighting conditions.

Learn More

• OpenCV Documentation: Search the official OpenCV docs for optical flow, video processing, and image pre-processing methods.
• MIT OpenCourseWare, Introduction to Computer Science and Programming Using Python: Search the course materials for Python image-processing practice.
• Berkeley Artificial Intelligence Research, adversarial examples papers: Search for Eykholt-style physical attack papers and related vision attacks.
• PubMed: Search review articles on adversarial machine learning and safety in autonomous systems.
• NASA Technical Reports Server: Search for robotics perception, obstacle avoidance, and vision reliability reports.