GitHub Accessibility Linter for Frontend PRs

ISEF Category: Systems Software

Subcategory: Other  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A tiny code change can break a website for millions of users. One missing label, one color change, or one shifted button can make a page hard to use with a screen reader or keyboard. Your project can help catch those problems before the code ships. That gives you a real software tool, not just a demo.

What Is It?

This project asks you to build a checker for GitHub pull requests. A pull request is a proposed code change. Your checker looks at the page before and after the change, then flags possible accessibility problems, which are issues that make a site harder to use for people with disabilities.

Think of it like a spell-checker, but for website accessibility. One part of your system can run axe-core, a tool that scans pages for common WCAG, or Web Content Accessibility Guidelines, violations. The other part can compare the DOM, the page structure that browsers read, before and after the change. You then combine those signals so the tool can guess which frontend diffs introduced new problems.

The hard part is not finding any issue. The hard part is finding the new issue that came from the change. That makes this a systems software project with real engineering depth.

Why This Is a Good Topic

This is a strong science fair topic because you can test it with real code, real pull requests, and clear metrics like precision, recall, and false alarm rate. It connects to a real problem in software engineering, since accessibility bugs often slip through code review. You can learn how static checks, browser automation, and machine learning signals work together in one pipeline. You also get room to compare different models, feature sets, and evaluation methods.

Research Questions

• How does combining axe-core with DOM-diff features change the accuracy of accessibility violation detection?
• What is the effect of training on one set of open-source projects and testing on a different set of projects?
• Does a small DOM-diff model detect more newly introduced WCAG issues than axe-core alone?
• To what extent does adding code-change metadata improve prediction of accessibility regressions?
• Which types of frontend changes most often create false positives in a PR accessibility linter?
• How does model performance change when you predict violation type instead of only predicting whether a PR is risky?

Basic Materials

• Laptop or desktop computer with a modern browser.
• GitHub account with access to public repositories.
• Python 3 and a code editor such as VS Code.
• Node.js for running frontend test scripts.
• axe-core and a browser automation tool such as Playwright or Selenium.
• Dataset of public pull requests from open-source web projects.
• Spreadsheet software or a notebook for logging results.
• Git or a GitHub API client for collecting diffs and metadata.

Advanced Materials

• Access to a university or lab workstation with more RAM and storage.
• Docker for reproducing target web app environments.
• Large-scale GitHub dataset or archived pull request corpus.
• Labeled accessibility violation dataset, if available through prior papers or public benchmarks.
• Python machine learning stack such as scikit-learn, PyTorch, or TensorFlow.
• Browser automation infrastructure for repeated page renders.
• Evaluation scripts for precision, recall, F1 score, and calibration.
• Static analysis or program differencing tools for frontend code.

Software & Tools

• GitHub API: Collects pull request metadata, diffs, and repository history from public projects.
• axe-core: Scans rendered pages for common WCAG issues during automated testing.
• Playwright: Loads pages in a real browser so you can compare prechange and postchange states.
• Python: Cleans data, builds features, trains models, and computes evaluation metrics.
• scikit-learn: Trains baseline classifiers and helps you test feature sets quickly.

Experiment Steps

1. Define exactly what counts as a new accessibility regression in a pull request.
2. Choose your project sample, then decide how you will pair each changed page with its prechange version.
3. Design a feature set that includes axe-core findings, DOM-diff signals, and code-change metadata.
4. Build a baseline model first, then compare it with your combined signal model.
5. Plan your evaluation so you can separate easy wins from hard cases and measure false positives.
6. Set up error analysis so you can see which change types confuse the linter most.

Common Pitfalls

• Treating every axe-core warning as a new regression, which inflates your labels and weakens the evaluation.
• Comparing pages that do not match the changed component, which makes the DOM diff meaningless.
• Training and testing on nearby commits from the same project, which can hide overfitting.
• Using only final page screenshots, which misses keyboard, semantic, and screen-reader failures.
• Ignoring class imbalance, which can make a model look good while it misses rare accessibility bugs.

What Makes This Competitive

A class-level version of this project just reports whether a page has accessibility issues. A stronger version predicts whether a specific pull request introduced the issue, which is a much harder problem. Competitive projects also test across many repositories, not just one app, and they report precision, recall, and error patterns by bug type. If you compare rule-based checks, DOM features, and a learned model, you get much deeper analysis.

Project Variations

• Try a version that predicts only keyboard accessibility regressions instead of all WCAG issues.
• Try a version that compares React components, Vue components, or plain HTML diffs as separate feature groups.
• Try a version that ranks pull requests by accessibility risk instead of making a yes-or-no prediction.

Learn More

• W3C Web Accessibility Initiative: Read the WCAG guidelines and supporting materials on the W3C site.
• axe-core Documentation: Learn how the accessibility engine works from the official axe-core docs.
• GitHub Docs API section: Find pull request and repository data fields in the GitHub documentation.
• NIH PubMed: Search for review articles on automated web accessibility detection and software defect prediction.
• MIT OpenCourseWare, Introduction to Algorithms: Use the free course materials to strengthen your understanding of model evaluation and data structures.