FM27 match engine: what we know so far

[caiograco]

I'm creating this thread to collect and summarize what I've learned about the FM27's match engine using the tool to run the engine outside the game. The idea is so people can find the results in one place instead of across several threads.

First, how the test works: All the tool does is to run FM27's own match engine outside the game, so its like doing an instant results in game but we can manipulate everything about a match. Every test plays the same matches twice on the same random seeds, once with a change and once without, so the difference comes from the change and not from luck. Most tests (not all) use neutral ground: the same synthetic Premier League mid-table squad on both sides (invented players modelled on the FM27 database), four preset opponents, home and away, tactical familiarity held equal. Results are in expected points (xPts, worked out from both sides' xG, steadier than points) per game; multiply by 38 for a season. I also rely on statistical tests to make conclusion. In general I use a t value of 3 or more to decide if the difference is real.

Here are the things I've tested so far:
1. Player attributes: values per point, thresholds, styles, positions, hidden attributes, CA, goalkeepers
2. Tactical familiarity
3. Fitness, positions and match conditions
4. Tactics, team instructions and Tactical Tweaks
5. Odds and ends: in-match management and what the engine ignores

You can find the original write ups, with full tables and more details here: Fm-Match-Lab/docs/studies at main · caiograco289/Fm-Match-Lab. The repo will keep getting updated as I do more tests.

Also, credit to lqf0624, whose Windows FM Tactics Lab made this possible, and to everyone that did similar tests before and those that asked questions that shaped the tests.

 

[FMMaster5]

Is anyone else facing this issue? For me the tactics have stopped working after the patch. Im leaking goals and losing matches?

 

[caiograco]

Pace and Acceleration are worth about twice as much per point as anything else, and most technical attributes are worth close to nothing inside a match. The table below summarizes the main learnings. It reports the value of one point change in a given attribute (holding the others constant) for the whole squad on expected points (xPts) per game. The report (Fm-Match-Lab/docs/studies/01-attribute-values.md at main · caiograco289/Fm-Match-Lab) also describes the effect on shots, Xg for and against. This was done on neutral ground,

Attribute	Per point	Shape from 1 to 20
Acceleration	+0.068	0.082 below 11, 0.049 above
Pace	+0.062	0.073 below 13, 0.038 above
Jumping Reach	+0.035	0.019 up to 9, 0.047 above: pays at the top
Stamina	+0.034	steep below 4 (0.092), then 0.026
Strength	+0.009 in xPts, but about +0.036 in real points	works through goals, not chances (see FAQ)
Anticipation	+0.024	straight line
Work Rate	+0.022	a floor: 1 costs half a point a game, 3-6 a little, 9-20 nothing
Concentration	+0.021	straight line
Balance	+0.017	straight line
Dribbling	+0.017	straight line
Decisions	+0.005	about nothing at every level
Passing	+0.003	about nothing at every level, even at 1
Finishing	0.000 in xPts	converts chances: +0.3 goals a game from 1 to 20

Not in the table, from the +5 tests on real matches: Composure and Vision show a little; Technique, First Touch, Crossing, Off the Ball, Tackling, Marking and Positioning show nothing clear; Natural Fitness is exactly zero inside a match.

I also did a bunch of additional tests that are interesting. Let me try to summarize them in a questions and answer format:

Is every point worth the same? Is 16 to 20 bigger than 8 to 12?
For most attributes, yes, each point is worth about the same, with no threshold. Four exceptions: Pace and Acceleration are worth more below about 12 than above. Jumping Reach is the opposite, flat up to 9 and then the steepest gain of any attribute (12 to 18 is worth +0.34 a game). Work Rate only hurts below about 9. Stamina hurts most below 4. There are reported in study 12

Do physical attributes only win because tests use meta tactics?
No. Physical +5 (Pace, Acceleration, Stamina, Work Rate, Aggression) is worth +0.38 to +0.48 a game under all five styles tested, possession styles included. Passing attributes help direct styles (Direct Counter-Attack +0.15, Gegenpress +0.11) and do nothing for Tiki-Taka or Control Possession, whose players already complete 90-92% of passes. Stamina pays at every intensity, most with a high press. These are reported in study 9

Do attributes combine, so a group is worth more than its parts?
No. Groups of four (playmaking, ball carrying, defending, aerial) raised together are worth exactly what their members are worth one at a time. Each group's value comes from one or two members: Jumping Reach is the whole aerial group, Anticipation the whole defending group. These are in study 10

Are weights different by position? Is Pace worth more for a winger than a midfielder?
I couldn't find any statistical differences. Giving +5 to one pair of players at a time (centre-backs, full-backs, DMs, wide AMs, strikers) gives about 0.03-0.06 a game for all positions. There are some small differences but statistically it could be just from chance. I didn't test all attributes and this is something I want to revisit at some point, but so far the conclusion is that there are no clear differences. These are reported in study 13

Is one player worth one-eleventh of the whole team?
This surprised me and the answer is not always. The results show that +5 Jumping Reach on just three players gives two-thirds of the whole team's gain. The first players you improve seems to matter most. These are reported in study 13.

Do attribute values change for a strong or weak team?
It doesn't seem so (at least among premier league level teams). The ranking is the same for top, middle and bottom sides, favourite or underdog (rank correlation 0.84). An underdog may gain a little less overall, mostly at the back, but that this is not statistically different. These are reported in study 14

Why do Strength and Finishing look weak in xG-based tables?
It seems the answer is that they act on conversion, which xG don't measure. From 1 to 20, Strength adds 0.5 goals a game above xG and +0.69 real points. It also hurt a lot below 6. Finishing adds about 0.3 goals. These are reported in study 12

Which hidden attributes matter? Is it only Consistency?
It doesn't seem so. Consistency matters across matches in a way that is not capture by the ME. In the ME, it barely matters and isn't even read (need to understand why) in many matches. Pressure is the one with a clear effect. A a squad at Pressure 1 loses 0.15 a game (about 6 points a season). Important Matches doesn't seem to be worth more on European games. Dirtiness and Temperament change cards but not results. These are reported on study 11

Does Current Ability play a role in the match?
No. With every attribute fixed, a squad at CA 40 does as well as one at CA 200, and Passing or Decisions +5 is worth the same at CA 60 as at 180. CA is read during the match, but it doesn't seem to change chances or results. Better players matter through their attributes. These are reported in study 20

What about goalkeepers?
This surprised me and I think I need to revist this since it goes against what we previously knew. In my tests, no single goalkeeper attribute +5 has a clear effect alone (Positioning and Concentration lead). All of them +5 together raise the save rate from 75% to 80%, about +0.26 points a game. The keeper's position rating matters far more: at 10 his save rate drops to 60%, conceding 0.7 more goals a game. xG-based tables miss keepers entirely, because saves change goals, not xG. These are reported in study 21

Why do these numbers differ from HarvestGreen's?
As far as I know we use the same engine (lqf0624's tool), with different setups. The order at the top agrees. On the Windows tool's sample match with everyone at 12, most effects shrink to about 40% of their size on real matches and some vanish. I think it has to do with the type of match it was trained on. I did most of my simulations on real matches (loaded from my save) and then modified. On these real matches, even levelled to 12, mental attributes like Anticipation and Concentration clearly count. These are reported in study 2

There are a lot of caveats and different ways to have done these. So let me know if you spot any issues or any follow ups that might be useful.

 

[caiograco]

Now, some results on tactical familiarity.

Tactical familiarity is worth about half a point a game from low to fully learned. Most of it comes from training, and it stops paying once the tactic is reasonably well learned. Most of these results come from study 3.

The table below summarizes the main numbers. It reports the effect on points per game of different familiarity levels in all eight tactical areas, compared with the same matches. This was mostly done on real matches from my save.

Familiarity in all eight areas	Effect
Untrained	about −0.4 points
Low (20) to high (80)	+0.52 points, about the same as +7 Pace for the whole squad
Around 50 and above	the full gain is already there; 90 and 100 play identical matches

Here are the other things I found, in the same questions and answer format:

What is tactical familiarity in FM27?
It's the Tactical Understanding part of the Tactical Effectiveness screen. Inside the engine every player has a record with a value for each of the eight tactical areas (In Possession Shape, Passing Style, Build Up, Chance Creation, Out Of Possession Shape, Line Of Engagement, Pressing Intensity, Pressing Trap). Each area combines match exposure and training, plus the player's tactical ability and how simple the tactic is. These are reported in study 3

What matters more, training or matches?
Training. The engine seems to weight training twice as much as match exposure (0.6 against 0.3), and tactical ability and simplicity count 0.25 each. @nyutso found the same with his own tool. These are reported in study 3

Does any single area matter most?
I doesn't seem so.Raising one area alone does almost nothing, the effect is spread over all eight, so a tactic has to be learned as a whole. For my pressing tactic, an untrained Pressing Intensity hurt the most.

Does player tactical ability matter?
Yes, but less than training. It seems to be the median of Anticipation, Decisions, Composure, Concentration and Versatility (this matched the engine exactly), and it counts a quarter in each area.

And tactic complexity?
This one is still open. Complexity is in the same formula: my tactic (complexity 78 out of 100) costs every player about 20 points of understanding in every area before training is added. But changing complexity right before a match does nothing, so my guess is that it works while familiarity is built up in training, i.e. a simpler tactic seems to be learned faster. I can't test that with the tool yet.

Are good players worth more with a familiar tactic?
It doesn't seem so. Attributes and familiarity just add up, they don't interact. These are reported in study 4

Do some tactics suffer more when they are unfamiliar?
Yes, that seems to be the case. I tested 23 tactics at low familiarity and some lost 0.3-0.4 a game while others lost about 0.2. The top of the ranking hardly changed though. One interesting thing is that the effect of familiarity was larger for the preset tactics than for the community (Knap's and ZaZ's amazing tactics). I think this goes against most people's intutiion. These are reported in study 6

One big caveat: the tool runs matches, not training. So how fast familiarity grows, how fast it fades and what tweaking sliders costs is not something I can measure here. @nyutso posted some interesting findings on that from the game data in the other thread.

Let me know if you see any issues or if there is something about familiarity you would like me to test.

 

[caiograco]

On Fitness, positions and match conditions.
Match sharpness and condition are the biggest effects I've measured so far, bigger than anything you can change on the tactics screen. Home advantage is real, morale at kick-off doesn't seem to matter. I think these are already know but its good to document and test them on the same tool.
The table below summarizes the main numbers, in points per game compared with the same matches where everyone is as fresh and sharp as saved.

Change	Effect
Match sharpness 50% vs 100%	−1.03
Match sharpness 75% vs 100%	−0.37
Condition 80% vs 100%	−0.80
Condition 90% vs 100%	−0.28
Every outfield starter at 10 in his own positions	−0.10 xPts
Every position rating at 10 (his own and all others)	−0.30 xPts
Goalkeeper at 10 in goal	−0.53, conceding 0.7 more goals
Team cohesion, lowest to highest	+0.2 to +0.3
Playing at home (identical sides)	1.54 against 1.18 points
Morale 5 to 20 at kick-off	no measurable effect

Here are the questions I think people care about:
Should I rotate my players?
I think the results support that. Starting players at 75% sharpness or 90% condition costs about a third of a point a game, and at 50% sharpness or 80% condition close to a full point. Nothing I tested on the tactics screen comes close to that. These are reported in study 5.

How bad is playing someone out of position?
In my first tests, it seemed that position ratings at 10 cost 0.85 points a game, but a follow-up showed most of that comes from the ratings of the positions players are not playing. With only their own positions at 10, all outfield starters together cost about 0.1 a game. The exception is the goalkeeper, his position rating is basically his shot-stopping. These are reported in study 17

Does an unfamiliar out-of-possession position cost more than an in-possession one?
Neither costs anything I could measure, for the three or four players whose positions change between phases. These are reported in study 17

How big is home advantage?
About the size on real football, which I think is a good thing. With identical squads and tactics, the home side gets +0.21 xG, +0.28 goals and +1.3 shots a game. It varies by ground (different matches - one of twelve gave about +0.5 xG and one gave none) but not with the club's level. These are reported in study 15

Does morale matter?
Not at kick-off. Morale 5, 10 and 20 against the usual 15 are all within ±0.04 a game. It may still matter over a season through other things (training, form, player happiness), which the tool can't see. These are reported in study 15

Note that sharpness, condition and cohesion were tested on real matches from my own save, the rest on neutral ground. I am moving towards the tests on neutral ground since I don't want they to rely on my real matches going forward.

 

[FMMaster5]

Which is the best tactic atm?

 

[Purity]

Home advantage is based on crowd and number of spectators for home team and away team. More spectators = more advantage, neutral = 50/50 but even with 50/50 results differs depending on number of spectators, I am feeling it depends based on Pressure. It is important to maximise stadium seats to have more spectators and bigger difference bewteen home/away spectator %

 

[caiograco]

Home advantage is based on crowd and number of spectators for home team and away team. More spectators = more advantage, neutral = 50/50 but even with 50/50 results differs depending on number of spectators, I am feeling it depends based on Pressure. It is important to maximise stadium seats to have more spectators and bigger difference bewteen home/away spectator %

That makes sense. I can try a few different tests to prove it and document the size.

 

[caiograco]

Now on tactics, team instructions and tactical tweaks

I used the tool to test what are the good tactics, and explored changing team instructions and tweaks.

The main test is the tactic league where I pitted 8 community tactics (mainly Knap's and ZaZ's against pre-sets) on a league where every team had the same squad (home and away) over 2 seasons. Results here: Fm-Match-Lab/docs/studies/07-tactic-league.md at main · caiograco289/Fm-Match-Lab

The table below shows how the presets did in a league where every team had the same synthetic squad, home and away, over 20 seasons (points per game).

Preset	Points per game
Gegenpress	1.39
Fluid Counter-Attack	1.38
Catenaccio	1.03
Park the Bus	0.92

The community tactics in the same league finished between 1.44 and 1.78, so every one of them beat every preset. The full table is in study 7.

Here are some learnings:
Why are the defensive presets so bad?
They create about 1 xG less per game and still concede slightly more. Sitting deep didn't help even as an underdog on my real matches. These are reported in study 6

Is there rock-paper-scissors between styles? Do counter tactics work?
That would be cool, but unfortunately that doesn't seem to be the case. A tactic's overall strength predicts almost all its head to head results. In an 18 tactic league only one of 306 pairings was clearly different from that. These are reported in study 7b

Do individual team instructions matter?
Mostly not, at least one at a time on a tactic that already works. I changed every team instruction on one of Knap's tactics, 300 paired matches each, and reran the promising ones on 900 more, and most turned out to be chance. I did find that one step more attacking mentality cost 0.12 points a game, and showing the opponent inside or outside on the pressing trap cost about 0.2. Small effects (under about 0.1) can't be ruled out though. Takeaway, the top community tactics are likely already close to optimal, but the tool can be used to further optimize if anyone wants to do that. These are reported in study 5. T

What do Tactical Tweaks do and what can we test?
What I did was apply it for the whole match from kick-off on a bunch of presets as well as one of Knap's tactics. Doing so, only Lock It Down has a clear effect everywhere: about −0.2 expected points a game, with fewer goals at both ends. Goalkeeper Up For Set Pieces concedes more in every test. The rest average out near zero over the presets, although Throw The Kitchen Sink, Go Long and Play With More Urgency cost points on my high press, high tempo tactic. But the way I'm doing the test is like the team instructions above. What I want to do but need to change the code is test them late in match after a goal has been scored, which is how they are meant to be used. These are reported in study 8

 

[caiograco]

Some other random learnings.
Things that have no measurable effect inside a match (note on inside, a lot of them matter outside)

• Current Ability (40 to 200, attributes fixed) - i.e. attributes matter but CA by itself does not
• Weight (exactly zero), height, reputation, age - my understand is that height affects jumping reach but it has no direct effect on the ME
• Morale at kick-off
• Natural Fitness (exactly zero)
• Most personality fields: Consistency, Important Matches, Versatility, Injury Proneness, Loyalty, Adaptability, Professionalism, Ambition - again, all of these matter, just not at the match
• Turning opposition instructions off - and this could be something about how I'm doing the test, please let me know if you have any ideas.

Small effects that may matter:

• Pressure at 1: −0.15 expected points a game
• Dirtiness 20: one more card a game but no effect on results
• Temperament 1: half a card but no effect on results

Some other issues points around the tool:
The code allows me to have the assistant or AI for in-match management. As best as I understand, assistant is like doing instant replay and AI is like how the other teams in the game run their matches. If I turn AI on, it changes the tactic and player selection. In terms of points, letting the AI manage during the match cost 0.56 points a game against the assistant on my matches. There are a bunch of reasons this could be the case, so it needs more testing. These are reported in study 5

Did the 27.0.1 patch change anything? Not for attribute values as far as I can tell. The values before and after the patch are highly correlated (r=0.98). I haven't re-done these formally comparing with the hotfix. These are reported in study 1

 

[squirrel_plays]

The GKs one is surprising. I would have expected GKs testing to say agility and reflexes are king

 

[Purity]

The GKs one is surprising. I would have expected GKs testing to say agility and reflexes are king

In my testing they still are. I'll run another batch tonight specifically on GKs to confirm my findings in another environment.

 

[caiograco]

I'm glad you checked. Thank you. Can you post it here when you have them?

 

[Purity]

Source: a campaign of 72,000 simulated matches (38 fixtures): the 48 visible attributes of the goalkeeper vary (real level + random shift), the 10 outfield players keep their real levels. Measure: real points per season (not goals scored, not goal difference). The numbers below are gains in points per season over each range of levels: for example Agility +2.63 in "8→12" means going from 8 to 12 gains 2.63 points per season, i.e. about +0.66 per level. The last column is the average per level between 8 and 20.

Attribute	Gain 1→8	Gain 8→12	Gain 12→16	Gain 16→20	Total 1→20	Average per level (8→20)
Agility	+5.96	+2.63	+2.05	+2.21	+12.85	+0.57
Reflexes	+7.06	+2.08	+1.65	+1.79	+12.58	+0.46
AerialReach	+1.35	+0.72	+0.32	+0.73	+3.12	+0.15
Concentration	+0.78	+0.45	+0.45	+0.45	+2.13	+0.11
Handling	+1.56	+0.09	+0.44	-0.24	+1.85	+0.02
InjuryProneness	-0.27	-0.27	-0.43	-0.54	-1.51	-0.10
RushingOut	-0.47	-0.27	-0.27	-0.27	-1.28	-0.07
CommandOfArea	+0.41	+0.23	+0.23	+0.23	+1.10	+0.06
Pace	+0.24	+0.14	+0.14	+0.14	+0.66	+0.04
Consistency	+0.22	+0.13	+0.13	+0.13	+0.61	+0.03
Decisions	+0.19	+0.11	+0.11	+0.11	+0.52	+0.03
Balance	+0.19	+0.11	+0.11	+0.11	+0.52	+0.03
Anticipation	+0.19	+0.11	+0.11	+0.11	+0.52	+0.03
Acceleration	+0.18	+0.11	+0.11	+0.11	+0.51	+0.03
Stamina	-0.18	-0.10	-0.10	-0.10	-0.48	-0.03
Dirtiness	+0.17	+0.10	+0.10	+0.10	+0.47	+0.03
Eccentricity	+0.14	+0.08	+0.08	+0.08	+0.38	+0.02
Positioning	+0.12	+0.07	+0.07	+0.07	+0.33	+0.02
WorkRate	+0.10	+0.06	+0.06	+0.06	+0.28	+0.01
Strength	+0.05	+0.03	+0.03	+0.03	+0.14	+0.01
OneOnOnes	+0.04	+0.02	+0.02	+0.02	+0.10	+0.01

Attributes with no measurable effect (total within ±0.05): Communication, Kicking, Punching, Throwing, FirstTouch, Passing, Technique, Aggression, Teamwork, Bravery, OffTheBall, Determination, Flair, Leadership, Composure, Vision, JumpingReach, NaturalFitness, ImportantMatches, Versatility, Adaptability, Ambition, Loyalty, Pressure, Professionalism, Sportsmanship, Temperament.

Reading: Agility and Reflexes dominate (about 0.5 points per season per level, much more below level 8), then Aerial Reach, Concentration and Command of Area. +5 on every attribute of the goalkeeper is worth about +7 points per season (the forum figure is +9.9). Positioning shows about 0 in my runs while it is near the top in other testers' one-attribute-at-a-time tests (not significant there).

Caveats: this goalkeeper campaign is old. A new goalkeeper campaign with new settings is planned.