Emotion-Conditioned Folk Melody Re-Harmonization

ISEF Category: Technology Enhances the Arts

Subcategory: Music and Image Manipulation  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

A melody can feel happy, tense, or sad even when the notes stay the same. Harmony does a lot of the emotional heavy lifting. If you can steer that harmony on purpose, you can turn one tune into several different emotional versions. That gives you a real research question, not just a cool music trick.

What Is It?

This project studies how harmony changes the emotion people hear in a melody. You start with a folk tune, then create new harmonic versions that aim for a specific Plutchik emotion, like joy, fear, or sadness. Plutchik emotions are a common way to group basic feelings. Think of them like a color wheel for mood.

The key idea is style transfer. In plain English, that means you keep the melody recognizable while changing the musical features that affect mood. Rule-based voice-leading helps the notes move smoothly from chord to chord. Voice-leading is just the path each note takes between harmonies. A learned reranker then picks the version that sounds most like the target emotion, based on data from listeners or labeled examples.

Why This Is a Good Topic

This is a strong science fair topic because you can measure both musical structure and human response. You can test whether certain chord choices, smoothness rules, or melody settings push listeners toward one emotion more than another. It connects to real problems in music generation, accessibility, film scoring, and interactive art. A student can learn signal design, annotation, data analysis, and basic machine learning without needing to invent a new instrument or build custom hardware.

Research Questions

• How does rule-based voice-leading affect listener accuracy when they identify a target Plutchik emotion?
• What is the effect of harmonic density on whether a folk melody sounds joyful, sad, or tense?
• Does a learned reranker improve emotion classification more than rule-based harmonization alone?
• To what extent does preserving the original melody limit the range of emotions listeners hear?
• Which chord progressions most often shift a neutral folk tune toward a chosen target emotion?
• How does listener musical training change forced-choice accuracy across four emotion targets?
• What is the effect of tempo control versus harmony control on perceived emotion in the same melody?

Basic Materials

• Laptop or desktop computer with enough memory for audio work.
• Headphones for careful listening tests.
• MIDI keyboard or computer piano roll editor.
• Digital audio workstation such as MuseScore, GarageBand, or FL Studio trial.
• Spreadsheet software for recording listener responses.
• Online survey tool for collecting forced-choice emotion ratings.
• Small set of public-domain folk melodies in MIDI or notation form.
• Consent form template for human-subject listening tests, if required by your school.

Advanced Materials

• University workstation or cloud GPU access for model training.
• Python environment with music processing libraries.
• Annotated dataset of melodies with emotion labels.
• MIDI synthesis tools for generating audio examples.
• Audio analysis library for extracting harmony, pitch, and timing features.
• Statistical software for mixed-effects models or significance testing.
• IRB or school human-subject approval materials, if listener studies require them.
• High-quality reference monitors or calibrated headphones for playback checks.

Software & Tools

• Python: Builds the data pipeline, trains the reranker, and runs analysis scripts.
• MuseScore: Creates and edits the melody and harmony examples in notation form.
• PrettyMIDI: Reads, writes, and edits MIDI files for model input and output.
• Librosa: Extracts audio features for comparison and analysis.
• Google Forms: Collects forced-choice listener responses and basic demographics.

Experiment Steps

1. Define the emotion set you want to test and decide how you will label each musical example.
2. Choose a small group of folk melodies that stay recognizable after reharmonization.
3. Design the rule set for voice-leading so each generated harmony follows the same musical constraints.
4. Build a baseline system, then add the learned reranker so you can compare simple versus improved output.
5. Plan your listening test so each participant hears enough examples to compare four emotion targets fairly.
6. Decide how you will score accuracy, agreement, and statistical significance before you collect responses.

Common Pitfalls

• Using melodies that are too short or too famous, which makes emotion judgments depend on memory instead of harmony.
• Letting the reranker change too many musical features at once, which hides whether harmony or melody caused the effect.
• Collecting listener responses with uneven audio playback, which makes some examples sound louder or clearer than others.
• Comparing emotion labels without a balanced set of four targets, which can inflate accuracy for one easy category.
• Ignoring participant musical background, which can confound whether the model or the listener caused the result.

What Makes This Competitive

A stronger version of this project would separate harmony effects from melody, rhythm, and loudness effects. You could test whether the model works on new folk tunes, not just the training set. You could also compare listener accuracy across different groups and use a harder statistical test than simple vote counts. The best projects explain why the system works, not just whether it works.

Project Variations

• Test the same emotion-conditioning method on children’s songs instead of folk melodies.
• Compare MIDI-only output with human-arranged harmonic versions to see which one listeners prefer.
• Swap forced-choice ratings for a continuous emotion slider to measure intensity, not just category choice.

Learn More

• MIT OpenCourseWare: Search for free courses on music perception, machine learning, and computational audio analysis.
• PubMed: Search for review articles on music emotion perception and listener studies.
• Journal of New Music Research: Search recent articles on computational music generation and emotion mapping.
• Frontiers in Psychology: Search for studies on music emotion, cognition, and forced-choice listening experiments.
• NIH PubMed Central: Find full-text papers on emotion recognition, audio modeling, and behavioral experiments.