RL Wi-Fi Coexistence on ESP32

ISEF Category: Embedded Systems

Subcategory: Networking and Data Communications  ·  Difficulty: Advanced  ·  Setup: University Lab  ·  Time: Full Year

The Hook

Your phone and your smart devices are fighting for airtime all the time. On a crowded 2.4 GHz band, one device can slow everyone else down. A smart scheduler can learn when to send, wait, or back off. That makes this a great project if you like wireless systems and data-driven design.

What Is It?

This project studies how a device can learn to share radio time with Wi-Fi instead of using a fixed rule like CSMA/CA, which stands for carrier sense multiple access with collision avoidance. Think of it like a traffic light that does not follow one timer. It watches the road, learns the pattern, and changes the signal to keep cars moving.

Reinforcement learning is a method where a system tries actions, sees what happens, and gets rewarded for better results. In this case, the device can learn which transmission choices lead to higher throughput, lower delay, or fewer collisions while still being a good neighbor on the same band. The ESP32 gives you a practical test platform because it has wireless hardware, a microcontroller, and enough support to prototype network behavior.

Why This Is a Good Topic

This is a strong science fair topic because you can measure it, compare it, and improve it step by step. You are not just building a gadget, you are testing a decision system under real wireless congestion. That connects to smart homes, sensor networks, and crowded public Wi-Fi. You can learn wireless metrics, experiment design, and basic machine learning in one project.

Research Questions

• How does a reinforcement-learning scheduler affect throughput when the shared 2.4 GHz channel gets busier?
• What is the effect of reward function design on packet delivery success in an ESP32 wireless testbed?
• Does a learned MAC-layer policy reduce latency more than a fixed CSMA/CA policy under the same traffic load?
• To what extent does channel congestion change the scheduler's decisions over time?
• Which traffic pattern produces the biggest performance gain from learning-based coexistence?
• How does adding a fairness term to the reward change the balance between throughput and Wi-Fi coexistence?

Basic Materials

• ESP32 development boards, at least two.
• Wi-Fi-capable laptop for configuration and logging.
• USB cables for each ESP32 board.
• Router or access point for creating a controlled Wi-Fi environment.
• Basic network monitoring software.
• Digital notebook or spreadsheet for logging results.
• Stable power supply or powered USB hub.
• Signal timing notes for documenting each test run.

Advanced Materials

• Spectrum analyzer or software-defined radio for observing band activity.
• Packet sniffer hardware or Wireshark-compatible capture setup.
• University access point or controlled wireless test chamber.
• Oscilloscope or logic analyzer for timing transmission events.
• Python environment with reinforcement-learning libraries.
• Microcontroller debug tools for firmware profiling.
• External antennas with known characteristics for controlled testing.
• Network emulator or traffic generator for repeatable congestion patterns.

Software & Tools

• Python: Trains reinforcement-learning agents and analyzes throughput, delay, and fairness data.
• Wireshark: Captures wireless traffic for packet-level comparison of learned and fixed schedules.
• Arduino IDE: Builds and flashes ESP32 firmware for the testbed.
• Jupyter Notebook: Organizes plots, tables, and statistical comparisons in one place.
• ImageJ: Not needed for the radio itself, but useful if you document hardware setups with annotated photos.

Experiment Steps

1. Define the performance metric you care about most, such as throughput, latency, or fairness.
2. Choose the exact wireless situation your device will face, such as light, medium, or heavy channel contention.
3. Design a baseline system so you can compare learning against a fixed CSMA/CA approach.
4. Build a reward rule that turns network outcomes into numbers the agent can improve.
5. Plan how you will collect repeatable measurements without changing the environment between runs.
6. Decide which statistics will show whether the learned policy really beats the baseline.

Common Pitfalls

• Testing in a noisy wireless space, which makes it impossible to tell your scheduler from nearby network traffic.
• Changing hardware placement between trials, which alters signal strength and hides the effect of your algorithm.
• Training on one traffic pattern and then claiming the policy works for all network loads.
• Using only average throughput, which can hide packet loss, delay spikes, and unfairness.
• Letting firmware timing drift or logging bugs corrupt the action history, which breaks your reinforcement-learning data.

What Makes This Competitive

A strong version of this project does more than compare two throughput numbers. You can test several reward functions, multiple traffic loads, and more than one fairness metric. You can also show where the learned policy helps and where it fails, which makes the work look more like real systems research. Careful controls and repeatable data matter as much as the algorithm itself.

Project Variations

• Compare a learned scheduler against CSMA/CA under video, sensor, and bursty traffic patterns.
• Test whether adding a fairness penalty changes how well the ESP32 coexists with a busy Wi-Fi access point.
• Swap throughput for latency and jitter, then see whether the learned policy still beats the baseline.

Learn More

• NIH PubMed: Search review articles on reinforcement learning, wireless coexistence, and medium access control to find background papers and experimental methods.
• IEEE Xplore: Search for papers on learning-based MAC scheduling and Wi-Fi coexistence, then use abstracts and author keywords to map the field.
• MIT OpenCourseWare: Look for networking and machine learning course materials that explain packet scheduling, feedback, and optimization.
• Arduino Documentation: Find ESP32 board guides, firmware examples, and serial logging references for device setup.
• Wireshark User’s Guide: Learn how to capture and inspect packet traffic so you can compare learned and fixed network behavior.