Game AI could learn many human-like play styles from adjustable targets

A player’s style as a short list of numbers

A player’s style can be reduced to a small list of numbers. Each number sits between 0 and 1. A value near 0 means the trait is barely present. A value near 1 means it is strongly present. The traits are chosen in advance. They might include aggressiveness, cooperativeness, mobility, competitiveness, and risk-taking. In one test setup, six such numbers were used. Each number tracked a different way of playing.

The list is called a behavior vector. It does not record every move. It summarizes the player’s measured tendencies. A high aggressiveness value means more pressure on opponents. A low cooperativeness value means less help to teammates. A high mobility value means more movement across the map. The numbers can be read as dials. Turning one dial changes the style.

This encoding uses ordinary in-game statistics rather than recorded actions. A profile can be built from a single completed game. That is useful when little data exists. Still, several games are better to average. Motivation and performance change between sessions. Averaging smooths those swings. The result is a stable description of how someone tends to play.

This representation also supports styles that no human may use. Values can be pushed toward extremes. That creates unusual combinations for testing software. A style can sit anywhere on a continuum. Human-like profiles are one part of it. Strange or extreme profiles form another part.

Measuring those styles in a simple two-versus-two arena

The test arena is a small grid world built for two teams of two agents. Each agent can move, pick up objects, and fight opponents. Safe coins give steady points. Diamonds sit in riskier places and pay more. Killing an enemy also adds score. The game gives the system a clean way to measure style from ordinary play statistics.

Six numbers describe what an agent does. Mobility is the share of visitable cells that the agent has visited. A high value means wide roaming. A low value means tight movement near one area. Teammate distance measures how close an agent stays to its partner. It can show a player who sticks together or one who plays alone. The coin score ratio is the share of total points from safe coins. The diamond score ratio is the share from risky diamonds. The kill score ratio is the share from opponent kills. Dominance divides total score by the theoretical maximum possible score.

All six values are scaled from 0 to 1. That makes them easy to compare across agents and episodes. A value near zero means little of that behavior. A value near one means strong expression of it. The metrics also separate related but different choices. An agent can be aggressive without being dominant if it kills often but scores little overall. Another can be safe and mobile, collecting coins while covering much ground. These measurements turn visible actions into a compact behavioral profile.

Training one shared model to chase many targets at once

A single policy learns to play many styles at once. During training, each agent receives a target style for that episode. It also sees its own current style and the normal game state. The reward is based on how much the gap between those two signals shrinks. If an action moves behavior closer to the target, it earns credit. If it pushes behavior away, it loses credit. This turns style control into a continuous steering problem.

The shared network sees three kinds of input. A map view passes through convolutional layers. Simple numbers such as health and time remain available. The current and target style signals are added to the same representation. From there, one branch chooses actions and another estimates value for training. All agents use the same weights. Each agent still gets its own target. Matches therefore contain many different style goals at once.

The reward is scaled by the size of the target signal. Without that scaling, large targets could promise more total return than small ones. The model would drift toward extreme styles instead of ordinary ones. Scaling also changes what an action means. A kill may be normal under an aggressive goal but unusual under a calm one.

Training does not need every sampled target to be realistic. If a target is hard or impossible, the agent still learns from movement toward it and away from it. This avoids needing recorded human play to define the allowed style region. Some odd combinations are included during training, which helps the policy learn broad steering behavior.

The final run used about 200 million time steps. That is roughly 5,500 hours of gameplay. Sixteen game instances ran in parallel on consumer hardware with an RTX 3080 GPU, 32 GB of RAM, and an Intel i5 processor. Earlier smaller networks were trained for 100 million steps each. Their reward curves improved steadily. The best configuration was chosen for the longer run.

The dense distance-based reward kept training stable without changing standard settings. Both action selection and value estimation saw the target signal. This let the network treat the goal as part of the situation, not as random noise.

Checking whether the agents really cover different play styles

To check whether the agents really cover different play styles, their behavior was measured over 1,250 completed episodes. The goal was to see if one shared policy could produce a wide band of styles instead of collapsing into one safe way of playing. A win-only baseline was used as a contrast. That baseline learned only how to win. It tended to make similar choices in similar situations. Its behavior occupied a narrow region of the style space. The new method produced significantly greater behavioral diversity than that baseline.

The agents were also asked to match many target styles. They did best on score-based dimensions, where the numbers are close to direct game outcomes. Cooperation and mobility showed weaker or moderate degradation. That pattern makes sense because those traits depend on movement and team positioning over time. A two-dimensional view of the behavior space was used to compare coverage. In that view, the new agents spread across a much broader area than the win-only policy. The CARI baseline also produced different styles by changing reward weights. It could push an agent toward more aggression or more coin collection. But it could not aim at an exact point such as making kills 40 percent of total score.

The episode data showed real links between behaviors. Diamond share and kill share moved strongly in opposite directions, with a correlation near minus 0.60. That is partly a game rule, because coin, diamond, and kill shares must add to one. Kill share and overall dominance were almost uncorrelated, about 0.10. Mobility was mildly linked to scoring dimensions, since collecting items and fighting both require moving around the map. Errors also had structure. When an agent failed to get enough kills, its total score often fell too. The error correlation was near plus 0.47. Mistakes were spread across several dimensions rather than piling up in one place. That pattern supports stable training dynamics.

What the method gives us, and what it still cannot do

The main result is a way to make game AI choose its personality after training. A player’s style becomes a small list of numbers between zero and one. Each number sets a trait, such as aggression, teamwork, movement, or risk. One shared policy can be pointed at many different lists. It does not need a separate model for each bot type. That is the practical gain. Game makers can ask for a cautious collector, a reckless attacker, or a teammate who stays close to another player. The same trained network can move toward those targets by changing its behavior during play.

This matters because games are hard to test when every AI opponent plays the same way. A win-focused bot may always choose one winning path. It may miss bugs that only appear under unusual human habits. The adjustable style space opens a wider testing ground. It also gives a rough model of how different players might behave, using ordinary in-game statistics rather than long recordings of their moves.

The limits are real. The method depends on metrics that can be measured at every step. Six hand-made numbers cannot capture everything about human play. Motion smoothness, timing under pressure, and tactical patience may need new dimensions. More dimensions make the target space grow quickly. Training may need longer runs or smarter sampling to keep coverage good. Some sampled targets are unrealistic, even though they still help learning. The generated styles should be read as coarse approximations, not full copies of individual players.

If style can become a dial on an AI bot, what happens when game testers stop asking whether an agent wins and start asking which human habit it imitates?