Drones Learn to Circle Wildfires in a Virtual Training Ground for Emergency Response

Why Wildfire Response Needs Drones That Can Adapt on Their Own

Wildfires do not wait for plans to be finished. In the 2025 Los Angeles fires, crews faced delays, broken communications, and limited resources. The flames spread faster than operational orders could reach them. Plans took time to write and send out. By the time they reached crews, conditions could have changed. Firefighters also had to watch for flare-ups, which pulled people away from active suppression. Many endured 24-hour shifts in hazardous terrain. Fatigue slowed reactions and raised the risk of injury. Helicopters needed frequent maintenance, so they were not always available when a fire jumped ahead.

Fire lines move quickly. Wind can push embers into new trees. Terrain can channel flames up slopes or block them in valleys. A drone that follows one preset path may fly over the wrong spot at the worst moment. It needs to decide, second by second, where to look next. It must stay safe while still getting close enough to see heat and smoke.

Traditional control methods often assume a stable problem. They work well when the target is known and the rules do not change. A wildfire is neither. The fire front can split, jump ahead, or die down in one place while growing elsewhere. A controller that must rebuild its plan for every new situation may be too slow to matter. Reinforcement learning offers a different route. A drone learns from experience how to adjust its path as conditions change.

Today, many wildfire drones still depend on people at the controls. That limits uptime. It also makes it hard to scale a fleet across a large fire. Radio links can be weak in rugged terrain. A drone that can adapt on its own can keep working when communication fades. Such drones are meant to add extra eyes where human coverage thins out. They can spot hotspots early. They can keep a moving picture of the fire edge. That helps commanders see where resources are needed next.

Building a Virtual Fire Range and Its Safety Rules

The training ground is a small three-dimensional box, twenty units long, ten wide, and ten high. It is not a forest map. It is a clean test space where fire behavior can be controlled. Two drones move inside it as simple spheres, each with a radius of 0.25 units. For collision checks, each drone also carries a cube-shaped box half a unit on every side. That box makes crashes easier to detect.

Fires are modeled as tall cylinders, not flames with flickering edges. A normal fire has a radius of 0.25 units and rises ten units high. The full height matters because it imitates the worst case: real fires can climb very high through trees. In one larger scenario, the central fire grows to a radius of 2.5 units.

The range has hard and soft rules. A direct hit on fire ends the episode immediately. That is the hardest rule in the system. Drones are also punished for drifting into danger zones near a flame. They earn credit when they hold an optimal gap from the fire. The ideal distance acts like a safe corridor around the burning edge. The reward for that gap follows a smooth curve. Small mistakes cost little. Large intrusions cost much more.

The range also discourages wall-hugging. A drone that stays too close to the outer boundary loses reward. It is meant to keep drones from hiding against the box while a fire spreads nearby. Energy use is checked at every step. Sudden bursts of thrust are penalized more than steady motion. This keeps flight patterns smooth.

A drone must move at least 0.75 units to count as actively exploring. If it moves less, a small penalty applies. If it moves more, it receives a bonus. The range is built around several fire layouts. One layout places a single fire in the center, like a campfire. Another starts central and lets new fires appear nearby over time. A third puts eight fires in a line across the map. Other layouts scatter fires randomly or create an impassable wall of flame.

When fires spread, each existing flame has a very small chance of creating a new one nearby. It appears within five units of the parent fire. The new flame must stay at least 1.5 units from all others. This prevents the map from becoming too crowded. In one harsh layout, drones can suffer a random crash with a tiny chance each step, about 0.06 percent. This mimics embers or failures in a crown fire.

Safety rules tighten as training progresses. A built-in safety margin shrinks across five levels. Early on, drones get a wider buffer around danger. Later, they must fly closer and control their path more precisely.

Following the Drones Through Thousands of Training Steps

The drones were trained in repeated trials called episodes. Each episode lasted up to 300 steps. A step is one small decision about how to move. The whole run covered 1,000 episodes. It was split into five difficulty levels. Each level had 200 episodes. This staged plan let the drones start with more safety margin and later fly closer to fire.

At first, performance looked messy. Episode length often dropped below 300 steps. That meant a drone hit something or ended early. The success rate also fell sharply. It swung between zero and twenty percent around episode 200. Success required surviving 300 steps without crashing and detecting fire. Early drones were still learning what danger felt like.

Two internal error measures also improved. One tracked how well the movement network matched its own plan. It fell from about one tenth to one ten-thousandth. The other tracked how well a judging network predicted future payoff. It settled near six hundredths. These numbers are small because they measure prediction errors, not flight quality.

Then the curves turned upward. After about 400 episodes, most runs reached the full 300-step length. The success rate climbed back to near 100 percent and stayed there. This was not just luck. It showed that the drones had learned to avoid collisions while still finding fire.

The reward score told a similar story. It started near zero and rose quickly during the first hundred episodes. Later it flattened, spiked between episodes 700 and 800, then settled around 250,000. That large number is not a distance or speed. It is an accumulated score from many small rewards and penalties across steps. A higher score means the drone was better at balancing safe distance, movement, and fire detection.

Another measure tracked how far each drone stayed from active fire. At the start, the average gap was 7.5 units. Over the first 200 episodes, that gap shrank fast. By episode 400, it had settled near 1 unit. The drones were not simply flying into flames. They learned to hover close enough to watch the edge and far enough to stay safe.

The flight paths changed too. Early paths were short and erratic. Some ended at walls or looped away from fire. Later paths became longer arcs around the burning area. Drones made repeated passes along the perimeter. This pattern suggests a useful habit. The drone kept fire in view. It adjusted as the fire grew. It also avoided obstacles.

Which Rewards Shaped the Best Fire-Watching Behavior

A drone’s score was built from ten small nudges. Some pushed it toward fire, some away, and others rewarded fresh ground. To see which nudge mattered, each one was removed while the rest stayed in place. The full set of signals produced the most balanced flying. It kept drones close enough to watch flames without crashing into them.

The safest-looking behavior came from distance-based rewards. When proximity or optimal-distance terms were left out, drones still found fires often, but their spacing became less reliable. In several layouts, versions missing those signals averaged about 1.8 fires per run, while the full score did as well. The key difference was not just finding fire. It was staying at a useful watching range.

Exploration rewards changed how much ground was covered. Removing them made drones visit fewer tiles. In one layout, the exploration variant saw 150 tiles. That was about 60 more than baseline and about 20 more than proximity-focused flying. That shows coverage can be bought with extra wandering. But wandering alone did not always mean better fire watching.

The discovery bonus had a sharp effect. Without it, drones found fewer fires in one simple case, averaging only 0.6 instead of about 1. Yet they visited more squares, around 200. They were searching wider but missing the point: new flames mattered more than empty ground.

Edge penalties kept drones from hugging walls. When that penalty was removed, coverage rose in some cases. It reached about 150 tiles instead of 135 under the full score. More area was seen, but the behavior drifted away from the fire’s edge. Movement and stationary terms also mattered. They encouraged active patrol instead of hovering. Lateral patrol rewards made drones sweep around a perimeter more often, viewing about 140 tiles in one layout versus 110 for the full set.

The best fire-watching came not from one giant reward but from a balanced score. Distance signals kept drones safe and useful. Exploration and discovery signals pushed them to find new flames. Edge and movement signals stopped lazy or wall-bound paths. Together, they shaped a habit: circle the fire, check fresh ground, and keep a steady watch.

What the Fire-Smart Drones Still Have to Prove

The drones learned a useful habit. They stopped crashing into flames. They began circling fire edges instead of fleeing or hovering blindly. That behavior emerged from repeated trial and error inside a virtual range. The scoring rules pushed them toward safe distance, fresh ground, and steady movement. Staged difficulty helped them gain confidence near danger.

The bigger promise is real-time awareness. Wildfires do not wait for plans to be written. A drone that can adjust its path as the fire shifts could give crews earlier warnings of flare-ups, spotting, or new fronts. It could also reduce the need for exhausted humans to patrol uncertain ground. In a future response system, several such drones might share observations and keep watch while people focus on suppression.

The obstacles are just as clear. The virtual world left out wind, rough terrain, sensor noise, and broken radio links. Those factors can change how a drone sees fire and moves through air. More varied fire shapes and longer training runs would test whether the learned behavior holds up. Energy use and even coverage also need closer measurement. A policy that works in one clean simulation may fail when conditions become messy.

The lesson is not that autonomous wildfire drones are ready for deployment. It is that a practical path now exists. The next step is to make the training world less forgiving. Can the drones still keep their cool?