Diffusion Policy Grasping for Soft Objects

ISEF Category: Robotics and Intelligent Machines

Subcategory: Machine Learning  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A sponge can slip, bend, and twist faster than a robot gripper can react. That makes soft-object grasping a hard test for machine learning. If you can train a robot to pick up plush toys and sponges, you are testing how well it learns control, not just memorized moves. That is a real robotics problem, not a toy demo.

What Is It?

This project asks whether a diffusion policy can learn better grasping behavior than simpler imitation-learning methods. A diffusion policy is a model that turns noisy guesses into a smooth action plan. In plain language, it starts with rough ideas about what the robot should do, then refines them until it finds a good next move.

That matters for deformable objects, because soft items do not behave like rigid blocks. A sponge squishes. A plush toy bends. A two-finger gripper has to predict how the object will shift as the fingers close. Your project compares this approach with behavior cloning, which copies expert actions directly, and implicit behavior cloning, which learns a latent action style instead of direct commands.

Why This Is a Good Topic

This is a strong science fair topic because you can measure it with a clear outcome, grasp success rate. You can also test generalization, which means whether the model works on new soft objects it never saw during training. That gives you a real research question instead of a simple demo. You also get to learn data collection, model comparison, and statistical analysis, which are all useful for robotics research.

Research Questions

• How does a diffusion policy compare with behavior cloning on grasp success for unseen soft objects?
• What is the effect of object deformation level on grasp success rate across different learned policies?
• Does training on more teleoperated demonstrations improve generalization to held-out plush toys and sponges?
• To what extent does a diffusion policy reduce failed grasps caused by slip or crushing compared with baseline methods?
• Which object features, such as size, softness, or surface texture, most affect policy performance?
• How does the number of training demos change the gap between diffusion policy and implicit behavior cloning?

Basic Materials

• Two-finger robot gripper with position control and force feedback.
• Robotic arm or fixed gripper mount with repeatable positioning.
• Teleoperation interface for recording human demonstration trajectories.
• Set of deformable test objects, such as sponges, plush toys, and foam shapes.
• Set of held-out objects that are not used during training.
• RGB camera or overhead webcam for recording grasp attempts.
• Tripod or fixed camera mount for consistent views.
• Laptop with enough memory to store videos and trajectories.
• Notebook or spreadsheet for logging trial outcomes.
• Measuring tape or ruler for documenting object dimensions.

Advanced Materials

• Robot arm with a two-finger parallel gripper and programmable control.
• Force-torque sensor or gripper force logging system.
• Depth camera or multi-view camera setup for object pose tracking.
• High-accuracy motion capture system, if available.
• Large storage drive for trajectory and video datasets.
• Workstation with a GPU for policy training.
• ROS or similar robot middleware for data collection and deployment.
• Calibration target for camera and robot alignment.
• Set of deformable benchmark objects with varied stiffness and geometry.
• Data annotation tool for labeling grasp outcomes and failure modes.

Software & Tools

• Python: Organizes data, trains models, and runs analysis scripts.
• PyTorch: Trains diffusion policies and baseline imitation models.
• ROS: Connects the robot, sensors, and teleoperation pipeline.
• OpenCV: Processes camera images and measures visual grasp outcomes.
• ImageJ: Helps inspect and compare object deformation or contact zones in recorded frames.

Experiment Steps

1. Define the exact grasp task, the object set, and the success metric you will use across all models.
2. Collect a small, balanced dataset of teleoperated grasps that covers different soft-object shapes and failure modes.
3. Train one diffusion policy and at least two baseline imitation models on the same demonstration set.
4. Design a held-out evaluation set that includes objects with different softness, shape, and surface texture.
5. Plan a fair testing protocol with the same starting conditions, the same number of trials, and the same scoring rules for every model.
6. Choose analysis methods that compare success rate, stability, and failure type, not just one final score.

Common Pitfalls

• Training and testing on the same objects, which inflates success rates and hides weak generalization.
• Letting camera placement drift between sessions, which changes the visual input and confuses the policy.
• Collecting demonstrations that are too similar, which teaches the model one grasp style instead of a flexible strategy.
• Scoring only whether the object leaves the table, which misses slip, partial lifts, and crushed grasps.
• Comparing models with different trial counts or different start positions, which makes the baseline results unfair.

What Makes This Competitive

A competitive project would go past a simple success comparison. You would test whether the diffusion policy generalizes better under a harder shift, like new softness levels, new object shapes, or cluttered starts. You could also break failures into categories, such as slip, crush, and missed contact, then use statistics to show where one method really helps. Strong calibration, fair baselines, and clean evaluation make this feel like research instead of a class demo.

Project Variations

• Compare policies on objects with different stiffness, such as foam, sponge, and plush fabric.
• Test whether vision-only input or vision plus force feedback gives better grasp generalization.
• Measure whether a smaller training set changes diffusion policy performance more than it changes baseline imitation models.

Learn More

• MIT OpenCourseWare: Search for robotics, manipulation, and machine learning courses with lecture notes and assignments.
• Papers with Code: Search for diffusion policy and imitation learning benchmarks to compare methods and metrics.
• arXiv: Search for recent preprints on diffusion policies, imitation learning, and deformable object manipulation.
• PubMed: Search for human-robot interaction review articles if you want to connect grasping to usability and safety.
• ROS Documentation: Read the free tutorials for robot control, sensor integration, and data logging.