Phone-Camera Ball-On-Plate Control

ISEF Category: Robotics and Intelligent Machines

Subcategory: Control Theory  ·  Difficulty: Advanced  ·  Setup: School Lab  ·  Time: 1 to 2 Months

The Hook

A small delay can make a ball roll off the plate, even when the code looks perfect. That is the whole challenge here. Your camera sees the ball, your controller reacts, and the gimbal has to move fast enough to catch up. This project lets you measure where fast becomes unstable.

What Is It?

A ball-on-plate system is a classic control problem. You tilt a flat plate, the ball rolls, and your job is to keep it near the center. The camera acts like your eyes. The software tracks the ball’s position in each frame, then tells the servos how to tilt the plate. That loop is called image-based visual servoing, which means control based on images instead of direct touch sensors.

Think of it like trying to hold a tray level while watching the tray in a mirror. If your eyes, brain, and hands lag too much, the tray wobbles or tips. In this project, the lag comes from camera exposure, image processing, and servo response. A Kalman filter is a prediction tool. It estimates where the ball is likely to be next, which can help the controller act before the ball drifts too far.

Why This Is a Good Topic

This topic works well for a science fair because you can change one thing at a time and measure the result. Latency, overshoot, settling time, and steady-state error all give you real numbers to compare. The project connects to drones, self-balancing robots, factory automation, and camera-based tracking systems. You can learn control theory, computer vision, and data analysis without needing a full research lab.

Research Questions

• How does camera frame rate affect the stability of a phone-camera ball-on-plate control loop?
• What is the effect of added image-processing delay on overshoot and settling time?
• Does a Kalman-filter-augmented loop reduce ball position error compared with a basic visual servo loop?
• To what extent does servo update rate change the largest stable tilt command before oscillation begins?
• Which tracking method gives lower closed-loop error, centroid tracking or threshold-based ball detection?
• How does lighting variation affect tracking accuracy and control stability?

Basic Materials

• Smartphone camera with manual video settings, if available.
• Hobby-servo gimbal or two-axis tilt platform.
• Microcontroller board, such as Arduino or ESP32.
• Ball bearings or marbles of known size and mass.
• Flat plate surface with a high-contrast background.
• External power supply for servos.
• Laptop for coding and data logging.
• Tape measure or ruler for calibration.
• Tripod or fixed phone mount.
• Colored markers or printed fiducial dots for coordinate reference.

Advanced Materials

• High-speed camera or phone with high frame-rate video mode.
• Precision servo drivers with feedback-capable hobby servos.
• Optical bench or rigid frame for reduced vibration.
• Calibration grid or checkerboard target.
• IMU module for optional comparison sensing.
• Laser cut or 3D printed plate mount.
• Computer with MATLAB or Python for control analysis.
• Force sensors or load cells for plate response testing.
• ND filter or controlled lighting rig for consistent imaging.
• Data acquisition hardware for synchronized timing measurements.

Software & Tools

• Python: Processes video frames, tracks the ball, and logs control performance.
• OpenCV: Detects the ball position and supports image-based tracking.
• ImageJ: Measures pixel movement and helps check tracking quality frame by frame.
• MATLAB or GNU Octave: Simulates control loops and compares stability metrics.
• LibreOffice Calc: Organizes trial data and calculates summary statistics.

Experiment Steps

1. Define the control signal you will measure, such as ball position error, overshoot, or settling time.
2. Choose one camera and processing pipeline, then lock down how you will convert pixels into plate coordinates.
3. Design a baseline feedback loop with no prediction so you have a clean control case.
4. Add a Kalman filter and decide which noisy measurements and state estimates it will use.
5. Plan a latency test that separates camera delay, processing delay, and servo response delay.
6. Set up repeatable stability tests that let you compare the largest disturbance each loop can recover from.

Common Pitfalls

• Using automatic exposure, which changes brightness between frames and confuses ball tracking.
• Mounting the phone at a slight angle, which warps the plate geometry and makes position estimates wrong.
• Letting servo backlash dominate the motion, which hides the effect of the controller itself.
• Testing only one disturbance size, which makes the stability comparison too weak to interpret.
• Comparing runs without synchronized timing, which mixes camera delay, code delay, and servo lag into one unclear result.

What Makes This Competitive

A strong version of this project separates sensing delay, computation delay, and actuator delay instead of treating them as one blob. That lets you answer a real control question, not just show that the ball moves. You can also compare multiple filters or controllers under the same disturbance pattern and use metrics like phase lag, RMS error, and recovery time. If you test across lighting, frame rate, and servo speed, your results start to look like a real systems study.

Project Variations

• Test how different ball materials, such as steel, glass, or plastic, change tracking stability and recovery time.
• Compare centroid tracking, edge detection, and color thresholding for ball localization under the same camera setup.
• Add intentional vibration to the plate base and measure whether prediction helps more than simple feedback control.

Learn More

• MIT OpenCourseWare, Feedback Control Systems: Search MIT OpenCourseWare for control systems lectures and problem sets that explain feedback, stability, and response time.
• NASA Glenn Research Center, Control Systems: Read the free educational pages on feedback control and dynamic response in engineered systems.
• OpenCV Documentation: Use the official docs for camera capture, image thresholding, contour detection, and tracking basics.
• PubMed: Search for review articles on vision-based control, Kalman filtering, and state estimation in robotics.
• IEEE Xplore Abstracts: Search for recent ball-and-plate and image-based visual servoing papers, then read abstracts and free author versions when available.