Handwriting Emotion to Calligraphy Art Project

ISEF Category: Technology Enhances the Arts

Subcategory: Human Information Exchange  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

Your handwriting can reveal more than your words. Small tremors, pauses, and pressure changes can act like a hidden mood trace. If you can measure that trace, you can turn a plain note into art that still feels like the person who wrote it. That mix of signal processing, machine learning, and design makes this a rare science fair topic.

What Is It?

This project asks whether a wearable sensor system can read emotional style from handwriting motion, then redraw the text in a matching calligraphic style. An IMU measures motion, like tiny changes in angle and acceleration. EMG sensors measure muscle activity, which can act like a proxy for grip force and tension. You are not reading feelings like a mind reader. You are looking for patterns in how a hand moves while writing.

Think of it like turning a singer’s voice into sheet music, then remixing that music into a new style. Your model first classifies the handwriting into labels such as calm, urgent, or hesitant. Then a generative font or style transfer model changes the appearance of the written words while trying to keep them readable. The core question is whether you can preserve the words and still carry over the emotional feel.

Why This Is a Good Topic

This is a strong science fair topic because you can test a real signal pipeline from sensor data to classification to visual output. You can compare features, models, and design choices with clear metrics, like accuracy, readability, and user ratings of emotional match. The project connects to assistive tech, expressive design, and human-computer interaction, so it has a real-world angle. You also get room to be original, since the best version will depend on your own data collection, feature engineering, and evaluation design.

Research Questions

• How does adding EMG data change emotion-classification accuracy compared with IMU data alone?
• What is the effect of handwriting speed variation on the system’s ability to separate calm, urgent, and hesitant samples?
• Does a model trained on pressure rhythm generalize better across writers than a model trained on raw motion signals?
• To what extent does preserving stroke order improve human recognition of the original words after calligraphic re-rendering?
• Which feature set, micro-tremor metrics, pause timing, or pressure rhythm, best predicts perceived urgency in handwriting?
• How does the style-transfer output affect user ratings of emotion fidelity and text readability?

Basic Materials

• Pen or stylus with a secure grip area.
• IMU sensor module with accelerometer and gyroscope.
• EMG sensor kit with electrodes and leads.
• Microcontroller board compatible with sensor logging.
• Breadboard and jumper wires.
• Laptop for data collection and analysis.
• Spreadsheet software for organizing trials.
• Consent forms for any human-subject handwriting study.
• Paper, pens, and a consistent writing surface.
• Digital camera or smartphone for backup documentation.

Advanced Materials

• Custom wearable bracelet housing for the IMU and EMG sensors.
• High-resolution data acquisition system for synchronized sensor streams.
• Adjustable force or pressure sensor array for pen grip analysis.
• Reference tablet or digitizer for ground-truth stroke capture.
• Calibrated handwriting capture setup with controlled lighting and camera angle.
• GPU-enabled workstation for training sequence models or generative font models.
• Annotation software for labeling writing tasks and emotion prompts.
• Statistical analysis package for mixed-effects modeling or repeated-measures tests.
• 3D printer or laser cutter for rapid prototype enclosures.
• Secure storage system for participant data and consent records.

Software & Tools

• Python: Processes sensor signals, builds features, and trains classification models.
• Jupyter Notebook: Lets you test analysis steps and keep code, plots, and notes together.
• ImageJ: Measures stroke width, shape features, and visual changes in rendered samples.
• OpenCV: Tracks handwriting images and extracts visual metrics from scanned or photographed output.
• Audacity: Checks audio if you add synchronized verbal prompts or timing cues during trials.

Experiment Steps

1. Define the writing tasks and the emotion labels you will test, then keep the prompt set consistent across writers.
2. Decide which sensor signals matter most, such as acceleration, rotation, muscle activity, pressure rhythm, or pause timing.
3. Plan a clean data format so each sample links writer, prompt, label, and sensor stream without confusion.
4. Build a baseline classifier first, then compare it with models that add more features or more sensor channels.
5. Design a rendering method that preserves the letter shapes, then add emotional style changes without destroying readability.
6. Plan human testing for both word recognition and perceived emotion, then choose metrics before you collect responses.

Common Pitfalls

• Mixing writers in one training set without checking personal style differences, which can make the model learn the person instead of the emotion.
• Letting sensor placement shift between sessions, which changes the signal more than the handwriting does.
• Using vague emotion prompts, which gives samples that do not clearly separate calm, urgent, and hesitant writing.
• Testing the generated font only by eye, which misses whether readers can still identify the words accurately.
• Collecting too few trials per writer, which makes the classifier look better or worse because of random noise.

What Makes This Competitive

A stronger version of this project will separate the sensing problem from the rendering problem and test both with careful metrics. You can go beyond simple accuracy and measure cross-writer generalization, recognition rate, and human-rated emotion match. You can also compare multiple model choices, then explain why one works better for motion, pressure, or style transfer. That kind of clean analysis makes the project feel like research, not just a demo.

Project Variations

• Try the same pipeline with left-handed and right-handed writers to test whether handedness changes emotion signals.
• Replace EMG with a pen-force sensor and compare whether grip pressure or muscle activity better predicts emotion labels.
• Test the rendering step on printed quotes, short messages, or signatures to see which text type keeps emotion best while staying readable.

Learn More

• PubMed: Search for review articles on handwriting analysis, EMG sensing, and affective computing to find human studies and signal-processing methods.
• NIH PubMed Central: Read full-text papers on physiological signal classification and human-computer interaction.
• IEEE Xplore: Search for papers on handwriting recognition, wearable sensing, and style transfer, then use abstracts and accessible manuscripts when available.
• MIT OpenCourseWare: Look for courses on signal processing, machine learning, and computer vision to strengthen your analysis plan.
• NIST Digital Library of Mathematical Functions? No. Search the NIST website for measurement and signal analysis references if you need calibration and data-quality guidance.
• arXiv: Search for preprints on handwriting synthesis, sequence models, and emotion-aware generative design, then verify methods with peer-reviewed sources.