Photo Restoration With Diffusion and Scratch Detection

ISEF Category: Technology Enhances the Arts

Subcategory: Music and Image Manipulation  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

Old photos fade, crack, and lose detail, but computers can help bring them back. Your project asks a sharper question, though, can a custom restoration pipeline beat the commercial tools people already use? That makes this more than a photo fix. It becomes a test of how well AI can recover visual history without making up the wrong details.

What Is It?

This project studies image restoration, which means repairing damaged images so they look closer to the original. Think of it like digital conservation work. One part of the pipeline finds scratches and missing spots, and the other part fills those gaps with a model called diffusion in-painting, which predicts missing pixels from the surrounding image.

A scratch-mask detector is a model trained to spot damage areas. A mask is just a black-and-white map that marks where the photo is broken. The diffusion model then uses that mask to decide where to reconstruct texture, faces, clothing, and backgrounds. Your goal is not only to make the photos look better, but also to measure whether your method preserves detail more faithfully than off-the-shelf restoration tools.

Why This Is a Good Topic

This is a strong science fair topic because you can test it with real images, clear metrics, and a meaningful comparison. You can measure restoration quality, damage detection accuracy, and how often the model invents details that were never in the original image. That connects to museums, family archives, journalism, and digital history. A student can learn computer vision, model evaluation, dataset design, and how to compare AI tools in a fair way.

Research Questions

• How does a custom scratch-mask detector change restoration quality compared with diffusion in-painting alone?
• What is the effect of training set size on scratch-mask detection accuracy for historical photographs?
• Does combining mask detection with diffusion reduce visible artifact creation compared with commercial photo restoration tools?
• To what extent does image age or damage type affect restoration quality scores?
• Which evaluation metric, such as PSNR, SSIM, or a human rating rubric, best matches perceived photo fidelity?
• How does domain-specific training on archival photographs compare with training on mixed modern and historical images?

Basic Materials

• Computer with a modern GPU or cloud access for model training.
• Small curated dataset of historical photographs from public archives.
• Image annotation tool for drawing scratch masks.
• Public benchmark set of 100 historical photographs with permission to use.
• Image editing software for viewing and cropping samples.
• Spreadsheet or notebook for logging image scores and model settings.

Advanced Materials

• High-memory workstation or university GPU server.
• Larger archival image dataset with varied damage types.
• Professional-grade annotation software for mask labeling.
• Version control system for code and experiment tracking.
• Python machine learning environment with PyTorch or TensorFlow.
• Quantitative image-quality evaluation package for PSNR, SSIM, and perceptual metrics.
• Human evaluation interface for pairwise ranking or fidelity ratings.

Software & Tools

• Python: Runs the training, inference, and evaluation scripts for the restoration pipeline.
• PyTorch: Builds and trains the scratch-mask detector and diffusion-based model components.
• OpenCV: Preprocesses archival images and helps inspect damage regions.
• ImageJ: Measures image quality features and compares restored and original regions.
• Label Studio: Lets you draw and manage scratch masks for your custom dataset.

Experiment Steps

1. Define the damage types you want your pipeline to handle, such as scratches, stains, tears, and missing patches.
2. Build a labeled dataset that separates damaged regions from clean regions, and keep a clear train, validation, and test split.
3. Choose the restoration baseline you will compare against, including at least one commercial tool and one simpler model without mask detection.
4. Decide which quality metrics will matter most, including pixel-level scores, perceptual similarity, and human judgment of realism.
5. Plan a fair comparison method that keeps the same test photos, output size, and evaluation rubric across every tool.
6. Set up error analysis so you can explain which photo types improve, which fail, and whether the model hallucinates new details.

Common Pitfalls

• Labeling scratches too loosely, which teaches the detector to mark normal texture as damage.
• Mixing highly damaged and lightly damaged photos in the same split, which makes the test results look better than they are.
• Comparing tools on different image sizes or crops, which breaks fairness across methods.
• Trusting one quality score alone, which can miss fake details that still score well numerically.
• Using archival photos with mismatched lighting or contrast, which makes the model confuse fading with actual damage.

What Makes This Competitive

A competitive project goes beyond making photos look nicer. You need a careful benchmark, a clean comparison against strong baselines, and a clear story about where the pipeline succeeds or fails. Strong projects also separate restoration quality from authenticity, since a sharp image can still invent faces, textures, or edges that were not there. If you analyze error patterns by damage type, photo age, and training set size, your project starts to look like real research.

Project Variations

• Focus only on facial portraits and test whether the pipeline preserves identity details better than commercial tools.
• Compare scratch-mask training on hand-labeled masks versus automatically generated masks to see which gives cleaner restoration.
• Test the same pipeline on newspaper photos, film scans, and family portraits to see how image source changes performance.

Learn More

• PyTorch tutorials: Free documentation and beginner examples for building image models, found on the official PyTorch site.
• OpenCV documentation: Free guides for image preprocessing, masking, and pixel-level inspection, found on the official OpenCV site.
• PubMed: Search for review articles on image restoration, digital heritage, and computer vision evaluation methods.
• IEEE Xplore: Search for peer-reviewed papers on image in-painting, scratch detection, and archival photo restoration.
• Library of Congress digital collections: Find public-domain and historical photographs for dataset planning and benchmarking.
• MIT OpenCourseWare: Search for computer vision and machine learning course materials to learn model evaluation and training basics.