Android News Credibility Checker

ISEF Category: Systems Software

Subcategory: Mobile Apps  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A phone can already help you order food, track steps, and translate text. Now imagine it also flags a news story that sounds persuasive but leans on weak evidence. That idea sits right at the edge of AI, mobile systems, and media literacy. You can test whether a small model on a phone can spot low-credibility framing fast enough to matter.

What Is It?

This project builds a news-literacy app that runs completely on an Android phone. The app uses a small language model, which is a model trained to predict and generate text. Your job is not to make the model write articles. Your job is to make it score or flag parts of an article that look shaky, such as loaded wording, missing sources, or one-sided framing.

Think of it like a pocket editor with a fact-checking habit. It does not replace a human reader, and it does not prove a story is true or false by itself. Instead, it gives a quick credibility signal. You can then test whether that signal matches labeled examples from datasets like LIAR and PolitiFact, and whether students reading on a phone notice and trust the warning.

Why This Is a Good Topic

This is a strong science fair topic because you can measure both software performance and human usefulness. You can compare accuracy, latency, battery use, and user trust, which gives you real data instead of a yes-or-no demo. The topic connects to misinformation, student media literacy, and private on-device AI, so it has real-world relevance. You can also make it more original by testing whether personalized feedback helps high school readers spot weak framing better than a generic warning.

Research Questions

• How does model quantization affect on-device credibility classification accuracy on Android?
• What is the effect of personalized feedback on high school readers' ability to identify low-credibility framing?
• Does an on-device model reduce response latency enough to improve user engagement compared with a cloud-based baseline?
• To what extent does the app's warning format change how users rate article trustworthiness?
• Which article features, such as sensational words, missing citations, or emotional framing, most often trigger false alarms?
• How does article length affect the model's ability to flag weak framing consistently?

Basic Materials

• Android phone or tablet with developer access and enough storage for a local model
• Computer with Android Studio installed
• USB cable for device debugging
• Sample news articles from LIAR and PolitiFact labels
• Spreadsheet software for results tracking
• Digital timer or screen recording for latency checks
• Headphones or quiet space for user testing
• Consent forms and survey draft for student readers.

Advanced Materials

• Android device with a modern processor and at least 6 GB RAM
• Laptop or workstation for model conversion and app development
• Quantized 1B-parameter model compatible with mlc-llm
• Android Studio with emulator support
• LIAR dataset and PolitiFact article set
• Python environment for evaluation scripts
• USB power meter for battery draw testing
• Screen logging or telemetry tools for response-time analysis
• Structured interview guide for user study sessions.

Software & Tools

• Android Studio: Builds and tests the Android app, and lets you debug device behavior.
• mlc-llm: Runs a quantized language model locally on Android for on-device inference.
• Python: Cleans datasets, runs evaluation scripts, and compares model outputs.
• ImageJ: Not needed for this topic, so leave it out and use text analysis instead.
• Google Sheets: Tracks article labels, user responses, and summary statistics.

Experiment Steps

1. Define the exact output your app will produce, such as a credibility score, a warning label, or highlighted risky phrases.
2. Choose one baseline model setup, then decide what changes you will test, such as quantization level, prompt design, or personalization rules.
3. Build a labeled test set from LIAR and PolitiFact examples so you can compare app output against known ground truth.
4. Plan evaluation metrics for both software and people, including accuracy, latency, battery use, and user trust.
5. Design a user study with high school readers that compares your app against a no-warning or generic-warning condition.
6. Set up controls that separate model quality from interface effects, so you know whether the warning itself helps, not just the layout.

Common Pitfalls

• Using an unbalanced article set, which can make the app look accurate by guessing the majority class.
• Letting prompt wording change between trials, which confounds model performance with instructions.
• Testing only on polished fact-check examples, which hides how the app handles messy real news text.
• Measuring latency on one device only, which can make the system look faster or slower than it really is.
• Asking readers if they liked the app instead of checking whether the warning actually changed their trust judgments.

What Makes This Competitive

A stronger project goes beyond a simple app demo. You can compare multiple prompt strategies, test different quantization settings, and separate article-level accuracy from user-level impact. You can also analyze which kinds of framing trigger the model most often, then explain why those errors happen. That mix of system design, evaluation, and human factors makes the work feel much more serious than a basic prototype.

Project Variations

• Test whether the app works better on opinion columns, straight news, or social media style posts.
• Compare a generic credibility warning with a personalized warning tuned to a student's reading history.
• Measure whether highlighting loaded language, missing sources, or emotional framing changes user trust more than a single score.

Learn More

• PubMed: Search for review articles on media literacy, misinformation, and how warnings affect reader trust.
• NIH: Look for public materials on health misinformation and science communication.
• NASA Open Data Portal: Use it as a model for how government data sets are documented and shared.
• MIT OpenCourseWare, Natural Language Processing: Study text classification basics and evaluation ideas from free course notes.
• Proceedings of the ACM on Human-Computer Interaction: Search for user study methods and mobile interface evaluation papers.
• arXiv: Search for recent papers on mobile large language models, quantization, and on-device inference.