TL;DR: I randomised all 60 attributes of every player in a 20-club league every 75 seconds and let the game holiday through ~16,000 full-engine matches. Pace and Acceleration are worth ~3x anything else. Everything technical measured at or below zero. According to these resutls no attribute has a threshold, they pay linearly 1-20. Once you divide by what a point costs in CA, the best buys are Work Rate, Jumping Reach and the hidden personality attributes, which are (mostly) free.
Method FM's engine is C++ with MSVC RTTI, so every object carries its class name behind the vtable pointer, so you can find every player and scoreline by name instead of hunting offsets. Loop: write a random attribute vector into every squad, sleep 75s while the game holidays, read back and harvest scorelines, repeat every 75s to take into account promoted teams.
Warning that cost me 120,000 matches Match Detail Level decides which simulator runs. On None everything resolves through the Quick Match Engine and effects come out 10x too small in the wrong order. Set the competition to All Competitive Matches. The quick path has its own QME_STAT_MANAGER sitting beside Poisson and Normal distribution objects, so it samples statistics rather than simulating football.
Dataset 15,951 full-engine matches. 47 visible + 5 hidden + 8 personality attributes randomised. CA and PA pinned at 150, morale pinned at 15. Attributes are stored 1-100 and displayed /5; personality attributes sit on a different record at plain 1-20.
Reading the numbers CA is pinned, so ability is a fixed budget and every weight answers "given fixed ability, where is it best spent?" Positioning at -9.6 does not mean Positioning harms you, rather it means CA spent there is not spent on Pace, and that trade costs about 10 points a season.
Validation Fitted on 5,011 matches, tested on 5,011 never used for fitting. Correlation with actual goal difference: Measured weights 0.392; Pace+Acceleration only 0.365; per-position 0.329; CA only 0.318; everything except the speed pair -0.043 Favourable quarter of the test set averages +1.79 goal difference, unfavourable -0.73. A two-segment fit searching every knee from 6 to 16 found no thresholds anywhere.
Global weights, CA cost and value Format: attribute, points per season, (CA cost of one point, points per CA point). CA cost measured separately by ramping a single attribute on an isolated player, R2=0.93. Hidden and personality attributes cost no CA at all. Ratios for near-free attributes are huge because the denominator is tiny, and read those as "free", not as literal multiples. Work Rate is the standout: nearly as valuable as Pace per point of ability, at a third of the cost. Decisions is the worst buy in the game, while the second most expensive attribute and negative. Pressure is worth 14.9 points a season and costs nothing.
Positional weights (0-20 within each group) Acceleration outranks Pace for CB and FB and the order flips further forward which seems plausible. The positional tables predicted worse out of sample than the pooled one (0.329 vs 0.392).
For goalkeepers, read the top two or three only. Reflexes and Agility separate cleanly; below that it is noise, and Handling ranking under Corners is not a finding. One keeper per match against ten outfielders means that column has about a tenth of the statistical power.
Yes, that seems to be my conclusion on it as well. With this method of reading out data through the code however it's just easier to get a bigger data set. This data set was from an about 8-hour overnight session, but since the output was pretty much in line with the rest of the findings published before, I didn't even bother to let it run for more to be honest.
One thing that my research contradicts though is that I didnt't seem to find any 'knee' for attributes. Where a certain attribute would perform much better than just 1 point lower. I've read some other threads and research that suggest for several attributes a 8 minimum, or you get much less results. Not saying that's not valid, but that type of result didn't really show up for me.
Does your simulation engine has any way of randomizing positional proficiency? (like having your ST being somewhere between ineffective and natural ST)
We have an early view of how it has an impact, but your simulation engine seems very efficient so we might be able to properly (finally!) isolate that.
TL;DR: I randomised all 60 attributes of every player in a 20-club league every 75 seconds and let the game holiday through ~16,000 full-engine matches.
Pace and Acceleration are worth ~3x anything else.
Everything technical measured at or below zero.
According to these resutls no attribute has a threshold, they pay linearly 1-20.
Once you divide by what a point costs in CA, the best buys are Work Rate, Jumping Reach and the hidden personality attributes, which are (mostly) free.
Method
FM's engine is C++ with MSVC RTTI, so every object carries its class name behind the vtable pointer, so you can find every player and scoreline by name instead of hunting offsets.
Loop: write a random attribute vector into every squad, sleep 75s while the game holidays, read back and harvest scorelines, repeat every 75s to take into account promoted teams.
Warning that cost me 120,000 matches
Match Detail Level decides which simulator runs.
On None everything resolves through the Quick Match Engine and effects come out 10x too small in the wrong order. Set the competition to All Competitive Matches.
The quick path has its own QME_STAT_MANAGER sitting beside Poisson and Normal distribution objects, so it samples statistics rather than simulating football.
Dataset
15,951 full-engine matches.
47 visible + 5 hidden + 8 personality attributes randomised.
CA and PA pinned at 150, morale pinned at 15.
Attributes are stored 1-100 and displayed /5;
personality attributes sit on a different record at plain 1-20.
Reading the numbers
CA is pinned, so ability is a fixed budget and every weight answers "given fixed ability, where is it best spent?"
Positioning at -9.6 does not mean Positioning harms you, rather it means CA spent there is not spent on Pace, and that trade costs about 10 points a season.
Validation
Fitted on 5,011 matches, tested on 5,011 never used for fitting.
Correlation with actual goal difference:
Measured weights 0.392; Pace+Acceleration only 0.365; per-position 0.329; CA only 0.318; everything except the speed pair -0.043
Favourable quarter of the test set averages +1.79 goal difference, unfavourable -0.73.
A two-segment fit searching every knee from 6 to 16 found no thresholds anywhere.
Global weights, CA cost and value
Format: attribute, points per season, (CA cost of one point, points per CA point).
CA cost measured separately by ramping a single attribute on an isolated player, R2=0.93.
Hidden and personality attributes cost no CA at all.
Ratios for near-free attributes are huge because the denominator is tiny, and read those as "free", not as literal multiples.
Work Rate is the standout: nearly as valuable as Pace per point of ability, at a third of the cost. Decisions is the worst buy in the game, while the second most expensive attribute and negative.
Pressure is worth 14.9 points a season and costs nothing.
Positional weights (0-20 within each group)
Acceleration outranks Pace for CB and FB and the order flips further forward which seems plausible.
The positional tables predicted worse out of sample than the pooled one (0.329 vs 0.392).
For goalkeepers, read the top two or three only.
Reflexes and Agility separate cleanly; below that it is noise, and Handling ranking under Corners is not a finding.
One keeper per match against ten outfielders means that column has about a tenth of the statistical power.
Shout out
to mavarobli / mavali / mark17072
@https://fm-arena.com/thread/20087-fmsuperscout-free-open-source-scouting-tool-for-fm26-with-a-meta-score-built-on-fm-arena-attribute-testing/
I've had a quick look on the result and it's inline with the FM-Arena Attribute testing for FM24 - https://fm-arena.com/thread/14009-attribute-testing-football-manager-24/
Lapidus said: I've had a quick look on the result and it's inline with the FM-Arena Attribute testing for FM24 - https://fm-arena.com/thread/14009-attribute-testing-football-manager-24/
Yes, that seems to be my conclusion on it as well.
With this method of reading out data through the code however it's just easier to get a bigger data set.
This data set was from an about 8-hour overnight session, but since the output was pretty much in line with the rest of the findings published before, I didn't even bother to let it run for more to be honest.
One thing that my research contradicts though is that I didnt't seem to find any 'knee' for attributes.
Where a certain attribute would perform much better than just 1 point lower.
I've read some other threads and research that suggest for several attributes a 8 minimum, or you get much less results.
Not saying that's not valid, but that type of result didn't really show up for me.
Does your simulation engine has any way of randomizing positional proficiency? (like having your ST being somewhere between ineffective and natural ST)
We have an early view of how it has an impact, but your simulation engine seems very efficient so we might be able to properly (finally!) isolate that.
Amazing work buddy.