FM27 Match Lab (macOS): Tool to simulate matches outside the game

First of all, great to have FM-arena back. I'm a long time lurker and user of the findings here.

For the past week I've been working on a tool to run FM27's actual match engine outside the game and on your own saved matches (so you can see how your team would have done with a different tactic or better tactical understanding, or with a better player). You take a match FM27 saved, replay it (it comes out exactly the same - same score, same xG, same goal minutes), then change one thing and replay it a few hundred times on the same seeds. Because the seeds are the same, whatever difference you get comes from your change, not from luck. The tool and method can also be used to simulate matches with different seeds and different conditions (i.e. we can test player attributes, tactical familiarity, etc...)

It's on GitHub, free and open source: GitHub - caiograco289/Fm-Match-Lab: Run Football Manager 27's own match engine on macOS, outside the game: replay your saved matches exactly, change tactics, attributes or instructions, and simulate thousands of matches to see what works. Unofficial; needs your own FM27.

First, credit where it's due: none of this would exist without the original windows based FM Tactics Lab by lqf0624, a tool from the Chinese FM community (see GitHub - lqf0624/fm-tactics-lab: FM 战术实验室 · Batch tactical experiments with the original FM27 detailed match engine. Windows desktop app and MIT-licensed tool source.). They did the hard work of figuring out how to drive FM27's engine outside the game in the first place, and the browser UI and a big chunk of the server still come from their project. I started out just wanting to port it to Mac, but then decided to play with it and make a bunch of changes (all with Claude code of course). It basically has the engine host for macOS, and allows us to replay your own save matches, as well as do some cool simulations. So it isn't the same tool anymore, but it only exists because theirs did. If you're on Windows, go use theirs. Mine is Mac only.

I've also used it to run a bunch of simulations to try to understand how FM27 works. So far I've I've thrown about 190,000 matches at it. Initially with the 27.0.0 version, but now also with the newest release 27.0.1. You can see the study results summarized by Claude here: Fm-Match-Lab/docs/studies at main · caiograco289/Fm-Match-Lab. It was mostly done with a save I was on 2037 with a team I moved up the rankings to the premier league. It would be interesting to re-do them with a different team/league.

Most of the results are not surprising. Let me try to organize them below with the help of AI (sorry it writes better than I do).

Looking forward to see what others learn from it!

--- AI summary of findings below ---

Everything below comes from ten real matches of one club (a newly promoted PL side, so mostly underdog games). Each test changes one thing for my team and replays all ten matches 200-300 times against the unchanged version. +0.1 points per game is roughly 4 points over a season, for scale.

Pace and Acceleration are still king

No surprise. I bumped each attribute by 5 for the whole squad, and Pace and Acceleration came out on top by a distance, about +0.07 ppg per point. +5 Pace across the squad is worth around 13 points a season. After those: Jumping Reach, Stamina, Anticipation, Dribbling, Concentration. So the FM27 engine hasn't changed the old order much.

Finishing does what you'd expect: no extra chances (xG stays the same), it just puts more of them away. Passing, Technique, First Touch, Crossing, Off the Ball, Tackling, Marking and Positioning showed nothing I could measure, so any effect they have is under about 0.1 ppg. The one odd result was Goalkeeper Kicking, which actually made things slightly worse. I don't have an explanation for that yet.

Fitness and selection matter more than tactics

Everyone knows rotation matters. I just hadn't seen how much put in points. Starting players at 50% match sharpness cost a full point per game; 75% still cost about 0.37. Condition at 80% cost 0.8 ppg. Starters out of position (position ratings capped at 10) cost 0.85 ppg. Nothing I changed on the tactics screen came anywhere near those.

Mental attributes do matter - on real matches

This is the one result that might go against some testing. On a generic test match with every attribute at 12, mental attributes looked close to worthless. On real matches, even with every player levelled to 12, Anticipation and Concentration clearly matter. On the generic match most effects shrank to about 40% of their real-match size, and some disappeared completely. That doesn't make test leagues wrong, but the test match itself seems to shape the results more than I expected.

Familiarity: about half a point a game, then it stops

People already know familiarity matters. The numbers: in FM27 it's the Tactical Understanding part of the Tactical Effectiveness screen, and training counts about twice as much as match exposure. Untrained costs around 0.4 ppg; well trained (around 75 per area) gains about 0.2. Above that there's nothing more to gain, because each area is capped. That's worth knowing if you're grinding tactical training past that point. Low to high in every area was worth about +0.5 ppg, about the same as +7 Pace for the whole squad. Attributes and familiarity just add up: good players aren't worth more with a familiar tactic.

Tactics

I ran 23 tactics on the same ten matches: the 10 FM27 presets plus 13 tactics I'd downloaded, a lot of them from here. My club's own tactic, STORMBRINGER 27 BBMIW 424, came out best. HIGHWAY STAR 27 BBM 4231 P90 and ZaZ Ace of Spades were basically level with it. So the tactics that test well here also held up on real matches.

Every preset was worse, and the defensive ones were awful, which again matches what testing here has shown. Park the Bus got 0.86 ppg against 1.80 for STORMBRINGER; Catenaccio got 1.20. They create about a full xG less per game and still concede slightly more. Even as an underdog, sitting deep doesn't pay in this engine.

Big caveat: every tactic used the same XI in the same slots, which is unfair on shapes very different from a 4-2-4. Please don't read this as a general ranking of anyone's tactic.

Random bits

- Letting the AI manager run the match instead of your assistant cost 0.56 ppg. I hadn't seen that one measured before.
- Most individual team instructions made no clear difference on their own.
- One step more attacking mentality cost a bit of xGD; points didn't clearly change.
- Weight has exactly zero effect. CA, reputation and height showed nothing either. The engine plays on attributes, not on the CA number - which people have suspected for a long time.

Caveats

All of this comes from one club, mostly losing to better teams. A title-chasing side could look different, and that's the main reason I'm posting this. Also, the instruction/mentality/personality results are still being rerun with more seeds, so treat the small ones as unconfirmed for now. The big ones (fitness, positions, AI manager) aren't going anywhere.

Full write-ups with all the tables are in the repo under docs/FINDINGS.md and docs/studies/ if you want the details.

How to set it up

You need a Mac, FM27 from Steam (27.0.0 or 27.0.1 - it refuses other versions until I've mapped them, so expect a delay after patches), Steam running and logged in, Python 3.12 and the Xcode command line tools. It loads the engine from your own install and doesn't touch Steam or DRM. Nothing from the game is in the repo.

Play at least one match first (instant result is fine), then:


Then go to http://127.0.0.1:18765 in your browser. From there you can edit pretty much everything the engine reads (attributes, lineups, roles, instructions, set pieces, who manages each side) and run batches. It imports and exports .fmf files too. Older FM24/FM26 tactics often won't load - re-saving them in FM27 seems to fix that.

To run the same kind of study on your own club:


It does about 3 seconds per match per worker, so around 6k matches an hour on six workers. FM27 has to be closed while it runs.

Also let me know if there are any tests that people want me to run. Or things to add to the tool.

caiograco said:

Also let me know if there are any tests that people want me to run. Or things to add to the tool.

Click to expand...

It would be nice to know how the tweaks perform in goals scored, goals conceded, and points per match. Some of those tweaks could also be more useful against different tactics, so having results separated by different presets could also be useful.

Thanks! Goals scored, goals conceded and points per match are all recorded for every simulated match, so I can add them to the results. You can already give the opponent any tactic in the app (import their .fmf) and compare a change against it. What's missing is running a change against many opposition tactics at once, for example every preset, and that's easy to add.

Quick question: by tweaks, do you mean FM27's in-match Tactical Tweaks (Press Aggressively, Go Long, Lock It Down…) or instruction changes to a tactic? Instruction changes can already be tested. The in-match Tactical Tweaks would need some work to support first.

I also wrote on your thread on the SI forums but the post is pending:

"One thing you might find interesting: when I dropped tactical familiarity to low, the Aces seemed to lose more than most, especially Hearts and Clubs. Spades held up best, so it might be the one to start with on a new save. The full results are here: Fm-Match-Lab/docs/studies/06-tactics-x-familiarity.md at main · caiograco289/Fm-Match-Lab"

" It might be good for tactical testing. See also what I did here: https://github.com/caiograco289/Fm-Match-Lab/blob/main/docs/studies/05-everything-else.md, by changing a bunch of team instructions one by one to see which drove XG more."

I did too the exact same analysis using Claude—I even created different test leagues and came up with a tactic using this tool. The tool I developed was built on the exact same logic; it simulates about 7 matches per second, but I haven’t designed a page to display the results yet. Since I’m the only one who’s tested the tactic so far, I haven’t had the chance to fully test how consistent it really is in the game. Maybe I’ll share it too I’ll post updates below this post.

I think there are some interesting results around tactical familiarity that I would love someone to dig deeper. I made a start here: Fm-Match-Lab/docs/studies/03-tactical-familiarity.md at main · caiograco289/Fm-Match-Lab but it seems that the main thing that matters is the training part of the tactical familiarity and less so match exposure and tactical ability. Player ability and tactical complexity didn't seem to matter, which I think is a new finding if it holds up. And makes training tactic even more important than people realize.

caiograco said:

Thanks! Goals scored, goals conceded and points per match are all recorded for every simulated match, so I can add them to the results. You can already give the opponent any tactic in the app (import their .fmf) and compare a change against it. What's missing is running a change against many opposition tactics at once, for example every preset, and that's easy to add.

Quick question: by tweaks, do you mean FM27's in-match Tactical Tweaks (Press Aggressively, Go Long, Lock It Down…) or instruction changes to a tactic? Instruction changes can already be tested. The in-match Tactical Tweaks would need some work to support first.

Click to expand...

Yeah, I mean those in game tactic tweaks. They are basically a set of team instructions, so you can simply pre-apply them directly to the tactic and test to see the results.

caiograco said:

I also wrote on your thread on the SI forums but the post is pending:

"One thing you might find interesting: when I dropped tactical familiarity to low, the Aces seemed to lose more than most, especially Hearts and Clubs. Spades held up best, so it might be the one to start with on a new save. The full results are here: Fm-Match-Lab/docs/studies/06-tactics-x-familiarity.md at main · caiograco289/Fm-Match-Lab"

" It might be good for tactical testing. See also what I did here: https://github.com/caiograco289/Fm-Match-Lab/blob/main/docs/studies/05-everything-else.md, by changing a bunch of team instructions one by one to see which drove XG more."

Click to expand...

I answered there, and recommended allowing people to use the tool in multiplayer games, so they can edit both teams. That could allow them to do many stuff like testing counter tactics or even running entire test leagues by simply switching the tactics of the teams.

ZaZ said:

Yeah, I mean those in game tactic tweaks. They are basically a set of team instructions, so you can simply pre-apply them directly to the tactic and test to see the results.

I answered there, and recommended allowing people to use the tool in multiplayer games, so they can edit both teams. That could allow them to do many stuff like testing counter tactics or even running entire test leagues by simply switching the tactics of the teams.

Click to expand...

Both things should be doable now that we have everything de-constructed. I will set Claude on it.

FM27: what I found about Tactical Familiarity

I saw the Fm-Match-Lab study saying that the training part of tactical familiarity matters most. I've been testing the same thing in FM27 with my own tool, and my results point the same way. Here is a short summary.


HOW I TESTED

I built a tool that runs FM27's own match engine outside the game and plays thousands of matches in identical conditions. Every match has a "seed" that fixes all the luck, so I can play the exact same match twice: once with a change and once without. Any difference comes from the change, not from luck.


WHAT xPts MEANS

Real results are noisy, so I mostly compare xPts (expected points). From both teams' xG I work out the chance of a win, a draw and a loss:
xPts = 3 × win chance + 1 × draw chance
Example: a 50% chance to win and a 25% chance to draw = 1.75 xPts. xPts needs far fewer matches than real points to show an effect.


1. FAMILIARITY HAS A BIG EFFECT

I set my team's tactical understanding (all 8 areas on the Tactical Efficiency screen) to fixed levels and played about 750 matches at each level:
- 50 → +0.16 xPts and +0.26 real points per match
- 65 → +0.19 xPts and +0.30 real points per match
- 100 → up to +0.28 xPts and +0.40 real points per match (a shorter test of about 190 matches, so read it as an upper estimate)

Over a 34-match league season (simple multiplication, compared with a team whose familiarity is near zero):
- 50 → about +5.6 xPts and +8.7 real points
- 65 → about +6.5 xPts and +10 real points
- 100 → up to about +9.5 xPts and +13.6 real points

Raising only one area did almost nothing. The effect comes from all 8 areas together.


2. HOW THE GAME BUILDS IT

- Every Tactical Understanding session on a non-match day gives every player in the squad one "record" for that area.
- Playing matches also raises familiarity.
- Records fade with age: full value in the first week, half after about 3 weeks, a sixth after about 3 months, zero after about 8 months. If you stop training, familiarity melts away.
- Each record remembers the setting you trained with. Moving a related slider one notch cuts its value by 10%; ten notches wipe it out.
- Player tactical ability, coach quality and time at the club are only multipliers (game data: player 0.70–1.0, coach 0.75–1.0, time at club 0.60–1.0). So how much you train matters far more than who you train.
- Training two tactics splits the effect 80/20 (game data; I didn't test this in matches).


WHAT I DO IN MY OWN GAME

- 12 Tactical Understanding sessions a week, covering all 8 areas. In my test save this reaches about 60 in 17 weeks and about 80 after a season.
- One tactic, no slider tweaking.
- Training intensity: Half for every fitness band. In my tests, Normal and Double lowered match-day condition without a clear gain, and intensity also feeds the game's injury risk.
- I haven't fully optimised the weekly schedule or the intensity yet; this is the best setup I have so far.

These are lab numbers from one squad against one opponent, so the size may differ in your game. The direction is clear, though: train it, cover all 8 areas, and leave the tactic alone.

ZaZ said:

Yeah, I mean those in game tactic tweaks. They are basically a set of team instructions, so you can simply pre-apply them directly to the tactic and test to see the results.

I answered there, and recommended allowing people to use the tool in multiplayer games, so they can edit both teams. That could allow them to do many stuff like testing counter tactics or even running entire test leagues by simply switching the tactics of the teams.

Click to expand...

I've tried to measure this too. Here's what I found, and why single tweaks are so hard to measure properly.

A quick note on the numbers: I use two measures.
- Real points: the actual result of the match (3 for a win, 1 for a draw, 0 for a loss).
- xPts (expected points): the points a team "deserved" based on the chances in the match. I take both teams' xG, work out how likely a win, a draw and a loss were, and calculate 3 × win chance + 1 × draw chance. Example: a 50% chance to win and a 25% chance to draw = 1.75 xPts. xPts is much less affected by luck than real points, so it shows small effects with far fewer matches.

What I found (about 780 paired matches per test; per match, and over a 34-match season):
- A set of 5 instruction changes on one of my tactics: +0.11 xPts (+3.6 per season) and +0.09 real points (+3.2 per season). Goals conceded fell by about 0.17 per match (about 6 fewer per season). Goals scored didn't clearly change.
- A community instruction package on another tactic: +0.075 xPts (+2.6 per season) and +0.04 real points (+1.4 per season, too small to separate from luck). Goals conceded fell by about 0.22 per match (about 7 fewer per season). Goals scored didn't clearly change.
- For my other formations, none of the extra changes I tried beat the starting instructions.
- Formation and roles mattered much more than any single instruction.

Why it's hard:
1. The effect is tiny compared with the luck in a match. One match gives 0, 1 or 3 real points, so results jump around a lot, while a good instruction tweak is worth only about 0.05–0.1 xPts per match. In the tests above, 780 matches were enough to see the effect in xPts, but not clearly in real points. To prove a +0.04 real-points gain on its own, you'd need about 5,000 matches for that single comparison.
2. There are a lot of single tweaks. Counting every card and every setting, there are 79 single changes you could make. At 1,000–5,000 matches each, that's 80,000 to 400,000 matches, for one tactic only.
3. Splitting by opponent tactic multiplies that again: 8 opponent presets means 8 times the matches. The AI also changes its tactic during the match, so "against preset X" isn't a clean group.
4. Tweaks don't add up. The effect of one card depends on the other cards, the formation and the roles. In my test the five changes worked together, but only one of them clearly helped on its own.
5. Testing many tweaks always produces some lucky winners, so every winner has to be re-tested on new matches.

So I can say which whole package works, and that it mainly cuts goals conceded, but not reliably what a single card is worth. Maybe @caiograco can find a better way to study this.

caiograco said:

Also let me know if there are any tests that people want me to run. Or things to add to the tool.

Click to expand...

Hello, everybody do tests of sharpness and condition, but I miss energy test. Could you please test it?