Continual Learning Garden Rover Classifier

ISEF Category: Robotics and Intelligent Machines

Subcategory: Machine Learning  ·  Difficulty: Advanced  ·  Setup: School Lab  ·  Time: Full Year

The Hook

A robot can look smart on day one and get worse after new training. That slip, called catastrophic forgetting, is a real problem in machine learning. Your garden rover can test whether a small edge computer can keep learning new weeds without losing old ones. That makes this project feel like a mini version of a real farm robot problem.

What Is It?

This project asks a simple question with a hard twist, can a robot keep learning new plant species without forgetting the ones it already knows? You train a camera-based classifier to tell weeds from crops, then keep adding new species as your backyard plot changes over time. The challenge is that many models “overwrite” older knowledge when they learn new data. That problem is called catastrophic forgetting.

Think of it like learning vocabulary in a new language. If you only study the latest words, you may start losing the first ones. Continual learning tries to stop that by mixing old knowledge with new examples, replaying past data, or protecting parts of the model from changing too much. In your project, the Jetson Nano acts like the robot brain, and the garden rover becomes the moving camera platform that gathers new images over weeks.

Why This Is a Good Topic

This is a strong science fair topic because you can measure it with real numbers, like accuracy on old species, accuracy on new species, and how much forgetting happens after each update. It also connects to a real problem in agriculture, where robots must recognize plants that change with season, location, and growth stage. You can learn image collection, model training, evaluation, and experimental design, all in one project. You do not need a university lab, but you do need careful planning and enough patience to build a good dataset.

Research Questions

• How does continual learning method affect classification accuracy on previously seen weed species after new species are added?
• What is the effect of replay buffer size on catastrophic forgetting in a weed-vs-crop classifier?
• Does mixing old and new images during fine-tuning improve retention of earlier plant classes?
• To what extent does model performance change when images come from different days, lighting, or growth stages?
• Which continual learning strategy keeps the best balance between learning new weeds and remembering old ones?
• How does a Jetson Nano model compare with naive fine-tuning in memory use, speed, and accuracy?

Basic Materials

• Jetson Nano or similar small edge AI computer.
• Raspberry Pi camera module or USB camera.
• Wheeled rover chassis with motors and battery pack.
• MicroSD card with operating system and storage.
• Laptop for data transfer and training.
• Phone or camera for collecting labeled plant images.
• Notebook or spreadsheet for tracking plant labels and collection dates.
• Assorted backyard or school garden plants, including crops and weeds.

Advanced Materials

• Jetson Nano with heatsink and stable power supply.
• Robot rover chassis with motor controller and wheel encoders.
• Camera with adjustable mount and fixed focus settings.
• External USB drive or SSD for image storage.
• Color card or reference target for photo standardization.
• Optional depth camera or wide-angle camera for multi-view data.
• Access to a larger plant image dataset for pretraining or comparison.
• GPU-capable workstation for model development and ablation testing.

Software & Tools

• Python: Runs data collection scripts, model training, and evaluation code.
• PyTorch: Builds and tests image classifiers and continual learning methods.
• OpenCV: Preprocesses plant images and helps with camera calibration.
• ImageJ: Measures image quality and helps inspect plant photos before training.
• LabelImg: Creates bounding boxes or labels for plant image datasets.

Experiment Steps

1. Define the plant classes you want the rover to recognize first, and decide how many new classes you will add later.
2. Plan a fixed image capture setup so your training photos stay as similar as possible across weeks.
3. Choose one continual learning strategy to compare with naive fine-tuning, and decide what counts as forgetting.
4. Build a dataset plan that keeps old-class test images separate from new-class training images.
5. Design evaluation metrics that track old accuracy, new accuracy, and overall balance after each update.
6. Set up a fair benchmark so both methods see the same data splits, the same model size, and the same evaluation order.

Common Pitfalls

• Changing camera angle or distance between collection days, which makes the model learn background changes instead of plant features.
• Using too few images for each weed or crop, which makes class imbalance dominate the results.
• Testing on images that were also used for training, which hides catastrophic forgetting.
• Adding new species without saving a fixed old test set, which makes it impossible to measure memory loss.
• Comparing methods after different amounts of training data, which turns the benchmark into an unfair comparison.

What Makes This Competitive

A stronger version of this project goes beyond a simple accuracy score. You can compare multiple continual learning methods, measure forgetting on each old class, and test whether results hold across different lighting or plant growth stages. You can also report memory use, inference speed on the Jetson Nano, and tradeoffs between learning new classes and keeping old ones. That kind of full evaluation makes the project feel like real robotics research, not just a model demo.

Project Variations

• Compare rehearsal-based learning with regular fine-tuning on the same garden dataset.
• Test whether adding nighttime or shaded images changes forgetting and new-class accuracy.
• Use leaf close-ups instead of full-plant images to see whether the model retains older classes better.

Learn More

• MIT OpenCourseWare, Introduction to Deep Learning: Free course material that helps you understand neural networks and training basics, found by searching MIT OpenCourseWare deep learning.
• PyTorch Tutorials: Free guides for building image classifiers and custom training loops, found on the PyTorch website.
• NASA Harvest: Research and data on crop monitoring and agricultural AI, found by searching NASA Harvest.
• USDA National Agricultural Library: Background on weeds, crops, and agricultural imaging, found by searching the USDA National Agricultural Library.
• PubMed: Search review articles on catastrophic forgetting and continual learning in neural networks.