Safe Cart-Pole RL Control With Barrier Shielding

ISEF Category: Robotics and Intelligent Machines

Subcategory: Control Theory  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A cart-pole can fail in one move. A small control mistake can send the cart into the rail and end the run. That makes it a perfect test bed for safe AI control. You can compare a shielded controller with an unconstrained one and measure how often each stays inside the track.

What Is It?

A cart-pole is a classic control system. A pole balances on a moving cart, and the controller tries to keep the pole upright while also keeping the cart on the track. If the cart gets too close to the rail, the episode fails.

Your project adds a safety layer called a control barrier function, or CBF. Think of it like a guardrail for the AI. The learning controller can suggest an action, but the shield checks that action and blocks unsafe moves before they happen. That lets you compare two ideas, raw reinforcement learning and reinforcement learning plus safety filtering.

This setup matters because a controller can look good in simulation and still fail on real hardware. A safe controller tries to keep performance high without crossing the line on safety. That makes it a strong project for testing how well modern learning methods transfer from a computer model to a real robot.

Why This Is a Good Topic

This is a strong science fair topic because you can measure clear outcomes, like rail hits, success rate, and balance time. You can also compare two controllers under the same conditions, which makes the experiment fair and easy to analyze. The work connects to robotics safety, autonomous systems, and real-world control, where one bad action can break hardware. You can learn simulation, controller design, statistics, and how to test whether a method still works outside the computer.

Research Questions

• How does adding a control-barrier-function shield change the rail violation rate compared with unconstrained PPO?
• What is the effect of different safety thresholds on balance success and rail violations?
• Does the shield reduce performance when the cart-pole starts from larger initial angles?
• To what extent does the safe controller transfer from simulation to the desktop track without extra tuning?
• Which failure modes appear most often in the unconstrained controller versus the shielded controller?
• What is the effect of sensor noise on the shielded controller's episode success rate?

Basic Materials

• Desktop cart-pole track system or similar teaching kit
• Laptop or desktop computer for training and analysis
• Physics or robotics simulator that models cart-pole dynamics
• USB data cable or wireless link for controller communication
• Camera or phone for recording runs
• Spreadsheet software for logging results
• Digital stopwatch or run log sheet for manual backup
• Spare batteries or power supply for the hardware.

Advanced Materials

• Desktop cart-pole apparatus with encoder feedback
• Real-time control computer or microcontroller interface
• Simulator environment for PPO training and testing
• Control-barrier-function implementation in Python or MATLAB
• Motion capture or high-speed video system for validation
• Data acquisition hardware for logging position, angle, and control signal
• Calibration tools for sensor alignment and track limits
• Optional force sensor or current sensor for actuator stress analysis.

Software & Tools

• Python: Runs the simulation, trains the controller, and analyzes outcome metrics.
• PyTorch: Supports reinforcement-learning model training and policy evaluation.
• OpenAI Gym or Gymnasium: Provides a cart-pole simulation environment for controller testing.
• MATLAB or Simulink: Helps model dynamics, design the shield, and compare trajectories.
• ImageJ: Measures motion from video if you need an independent check on cart and pole position.

Experiment Steps

1. Define the safety boundary you care about, such as track limits and pole angle limits.
2. Choose one baseline controller and one shielded controller so your comparison stays clean.
3. Build a simulation test plan that includes many starting states and random disturbances.
4. Decide how you will score each run, including violations, balance time, and tracking error.
5. Plan a transfer test that keeps the hardware setup fixed while you compare both controllers under the same conditions.
6. Select statistical tests before you collect data so you can compare failure rates and performance fairly.

Common Pitfalls

• Training only in easy simulation states, which can hide rare crashes that appear on the real track.
• Ignoring actuator limits, which can make the shield look safe in code but fail on hardware.
• Comparing controllers with different starting angles, which turns a controller test into a setup test.
• Tuning the safety shield after seeing the real-world results, which biases the comparison.
• Counting near-misses as successes, which can hide a controller that barely avoids the rail.

What Makes This Competitive

A competitive version does more than report that one controller failed less often. You need clean baselines, many repeated trials, and a metric that captures both safety and control quality. Strong entries also test distribution shift, like noisier sensors, new initial states, or changed track friction. If you can explain why the shield helps and where it hurts, your project starts to look like real control research.

Project Variations

• Compare a control-barrier-function shield with a simple action clamp to see which one preserves performance better.
• Test the same safe controller under different sensor noise levels to see how fragile the safety guarantee becomes.
• Replace cart-pole with a self-balancing robot or inverted pendulum if your lab has a different platform.

Learn More

• MIT OpenCourseWare, under Control Systems: Free lecture notes and problem sets for feedback control and stability, found by searching MIT OpenCourseWare control systems.
• OpenAI Gym/Gymnasium documentation: Describes the cart-pole environment and how to run reinforcement-learning experiments, found by searching the official Gymnasium docs.
• PubMed: Search for review articles on safe reinforcement learning and control barrier functions to see how researchers frame safety guarantees.
• IEEE Xplore: Search for papers on safe RL, cart-pole control, and control barrier functions to find recent methods and benchmarks.
• NASA NTRS: Search the NASA Technical Reports Server for control, autonomy, and safety papers that discuss practical robotics testing.
• NIST Digital Library of Mathematical Functions and control resources: Search NIST resources for math references that support stability and estimation work.