A more appropriate example to compare to Bastoni is Illia Zabarnyi, whom Exellent castigated, as Zabarnyi also has very low dirtiness (2) & injury proneness (3), but has an equal rating to Bastoni in FMST26 (both ~78%)
78% | Illia Zabarnyi (DC) - 3rd, 5th, 4th, 5th, 2nd = 3.8 position 82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position 77% | Alessandro Bastoni (DC) [Exellent version] - 3rd, 5th, 5th, 3rd, 3rd, 7th = 4.333 position 75% | Kurt Zouma (DC) - 7th, 5th, 6th, 3rd = 5.6 position 73% | Shota Fukuoka (DC) [Exellent version] - 7th, 8th, 7th, 8th, 8th, 11th = 8.166 position 71% | Sean Goldberg (DC) - 5th, 16th (sacked), 8th (sacked), 5th = 8.5 position
Note: I capped Zabarnyi's PA to just above CA so that he wouldn't grow during the season.
You can see here why I automatically assumed Fukuoka was better than Bastoni, on the basis of his very low dirtiness and injury proneness.
I didn't set dirtiness at such a high negative weighting for the heck of it.
I know it's unintuitive but I'm just reflecting what the data shows. And actually this is the most clear cut example I've seen, for with Maeda you have stuff like work rate and anticipation clouding it a bit. But here, Zabaryni literally has no other advantage than +3 composure, and many disadvantages (including in meaningful attributes!).
I challenge anyone to try and explain why Zabarnyi finishes higher than Bastoni looking at the comparison above, without concluding dirtiness and injury proneness has a massive impact.
Now I have had the thought that it could be that low dirtiness is the necessary cherry on top at the highest level, but diminishes in relative importance at lower levels. I lean towards no, but we'll cross this bridge when we come to it - I figure that the most common, or final goal, is to win at a high level in the game, and so that is what I have remained focused on for now.
82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position 77% | Alessandro Bastoni (DC) [Exellent version] - 3rd, 5th, 5th, 3rd, 3rd, 7th = 4.333 position 75% | Kurt Zouma (DC) - 7th, 5th, 6th, 3rd = 5.6 position 73% | Shota Fukuoka (DC) [Exellent version] - 7th, 8th, 7th, 8th, 8th, 11th = 8.166 position 71% | Sean Goldberg (DC) - 5th, 16th (sacked), 8th (sacked), 5th = 8.5 position
I withdraw saying that Fukuoka is better than Bastoni. My mistake was to assume that the GS rating reflected the FMST26 rating.
But we see here that the rating for both players in FMST26 aligns correctly.
I got ahead of myself because I know that Maeda & Furuhashi are slightly better than Lautaro Martinez, even though Martinez has better attributes in literally almost every way - except dirtiness and injury proneness, and the case looked at a glance to be the same here.
So there is a translation to GS problem here.
Now if I compare DC % between FMST26 and GS, the following players have significant discrepancies (2% or more, when both are adjusted to 77.9% max):
Declan Rice - 77.1% (FMST26) vs. 74.76% (GS) = -2.34% Fiyako Tomori - 71.9% (FMST26) vs. 74.33% (GS) = +2.43% Nacho - 70.9% (FMST26) vs. 73.51% (GS) = +2.61% Levi Colwill - 71.1% (FMST26) vs. 73.27% (GS) = +2.17% Shota Fukuoka [Exellent version] - 69.3% (FMST26) vs. 73.32% (GS) = +4.02% Pantelis Hatzidiakos - 69.1% (FMST26) vs. 72.10% (GS) = +3.00% Cristian Romero - 69.1% (FMST26) vs. 71.96% (GS) = +2.86% Pascal Struijk - 68.8% (FMST26) vs. 71.88% (GS) = +3.08% Federico Dimarco - 74.0% (FMST26) vs. 71.72% (GS) = -2.28% Maxence Lacroix - 68.7% (FMST26) vs. 71.69% (GS) = 2.99%
That's 10 out of the top 70 players (14.3%), and he happened to find the worst one of the bunch, so overall I'm not too concerned. Bastoni has a 1.6% underrating in GS.
Maeda, who benefits from the high negative dirtiness and injury proneness values (he has '2' and '3' respectively) already had to be compromised down from 4th in FMST26 to 16th in GS, so it's a bit of a damned if you do, damned if you don't situation with GS. This is just the limit of GS. No doubt it could be improved a bit, but it's not going to be able to match FMST26.
While I'm doing Bastoni I do notice that he is actually 77.9% in FMST26, for FM24 (where he has largely the same stats). Anyone can verify this by loading up FMST26 with my weights and checking Bastoni.
So based on the rankings you see above, as well as 78% | Jules Kounde (DR), I would expect Bastoni to come ~4.8 position on average, but we'll see.
Exellent said: Hey, I tested your latest rating build, and it’s a complete disaster.
your super rating says that Shota Fukuoka is better than Bastoni. Spoiler Expand
I've inputted the stats shown for Fukuoka into FM24. I ended up with 131 instead of 135 CA, probably because of some lacking DL/DR/weak foot proficiency, but that's no issue. The only other difference is that I've changed Fukuoka's nation to England, so that his low adaptability doesn't mess with things.
Here is the result:
82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position 75% | Kurt Zouma (DC) - 7th, 5th, 6th, 3rd = 5.6 position 73% | Shota Fukuoka (DC) [Exellent version] = 7th, 8th, 7th, 8th, 8th, 11th = 8.166 position 71% | Sean Goldberg (DC) - 5th, 16th (sacked), 8th (sacked), 5th = 8.5 position
I'm using the FMST26 % as that is more accurate and what I always refer to, but you can see the difference is only 73% vs 75% anyway.
Kurt Zouma is quite similar to Bastoni if you compare the stats:
But I will also do a clone of Bastoni for comparison so that there's no doubt about it.
This is all a good test of my claims, because this way no one can say it's only valid for cherrypicked players.
Rain said: Yeah it doesn't have to be 1 mil exactly was just an idea. Why did you do 60th ranked? I figured give each team or file that is being tested the top 18 or so guys by rating and positions at 1 mil or under and every team use the same exact tactic. Could either replace the lowest prem teams or just run it in whatever test league is used. I could set it up I'm sure I just need to know which files to even test I'm losing track at this point. Expand I think I chose rank 60th partly to help avoid what you mentioned, every team selecting the same best handful of players, but also to make the results clearer (i.e. 8th vs 14th, instead of 1st vs 1st). I expected there to be too much overlap anyway, since I was selecting 1-2 backups as well, but actually about 50% of the team ended up different with each GS file, and there were significant differences in the results.
You could use the Premier League 2.1 GS file, and other's GS files of your choice.
Rain said: We could really solve this "which file is best?" problem pretty easily. Take whichever files say your original, pure performance, prem 2.1, keith, etc. and just use each file to make a team in the league based on the best player you can buy for 1 mil and under (Should filter out teams using a bunch of the same top players) and if they do have the same player just clone them. Then just run the league a good amount of times and take the median results. Expand That would require them to actually do a test though. 😂
I did exactly this with 2 of my files and Orion's as a control. The search isn't working right now, so I can't find the exact results right now, both of my files did something like 6 positions better.
I think I used $1mil budget per player, and picked the 60th ranked player in position + backups just beneath them of course.
It was tedious (even for me!), and unfortunately when it came to revisiting it for testing the new file that came after that, I hadn't tracked everything properly so I would have to redo the selection for all 3 files in addition again so I didn't bother and used a different method of assessment instead.
But it's probably worth revisiting in the future.
Although no doubt we'll still hear in response 'I don't want players under $1mil! I want the best players! If they have an extra 6 passing for an extra $50mil, I don't care, I want it!' from Yarema, treath, and friends.
Exellent said: Hey, I tested your latest rating build, and it’s a complete disaster. It ranks Trevoh Chalobah and Illia Zabarnyi as the top CBs in the game
You’ve been working on this rating for 9 months with zero positive results. Maybe it's time to just give up?
Also, your super rating says that Shota Fukuoka is better than Bastoni. Spoiler Expand On what basis, other than your own incredulity, do you find that Fukuoka is inferior to Bastoni?
To me, it's clear as day that the reason Fukuoka is superior is his low dirtiness and injury proneness. You may not like it, but this is what peak performance looks like.
I do not know if you are using FM26 or FM24, but in a FM24 save I have that is a few years in, Chalobah's last season performance was 7.11 average for A. Madrid, and Zabarnyi's was 7.17 for 31 games in the English Premier League (7.23 for 38 games in the season before that).
Zighnetar said: @GeorgeFloydOverdosed Try this experiment. Set all physical attributes to 20, and dribbling, anticipation, and work rate to 20. With these attributes, you win everything in FM 24 and 26. In the 2017 version, you win nothing. But remember to set all other attributes to 1. Not 8, not 6, just 1. Expand I was thinking to mention that I am not disputing your findings. Nor even the main conclusion. It's just that I am aware of some likely cavaets from having done the same kind of tests with FM24.
So with the 20 drib/work/ant test, the problem is that it won't work without a couple of other key ones such as concentration. As you saw, adding composure (which is one of the essential attributes), improved the position from 19th to 6th. You say you win everything with it in FM24.. I guess it might be possible, since you do have anticipation added not to mention every physical at 18.. but just be aware that it'd be very much on the knife-edge.
So that one for me is easily explained away.
Winning with 15 physicals is also no problem in FM24, so long as you have the adequate mentals and drib, which you have provided 20 for.
The 20 mental/technical 10 physical one is somewhat more compelling. I've tested a player who just about matches this template, Dybala, in FM24, and the result was very poor. I'm sure that if I played with a full squad of Dybalas, they'd be relegated in FM24.
However it could be one of these two things:
1) Dybala has one key difference, which is having just 9 work rate. Kane and Lewandowski do very well in FM24 which shows that certain high mentals/technicals can make up for even ~13 physicals. But even then, 10 should be a bridge too far
2) It could be that you not done your test on 'full detail', which is something easy to overlook
Nonetheless let us just assume it is valid:
Your test shows that the physical requirement has dropped from ~13 in FM24 to 10 in FM17. My test shows that what comes 5th in FM24, with an emphasis on pace/acc, comes 14th in FM17 - though increasing pace/acc slightly bumps that up to 8th.
From your 20 ant/comp/drib/work 18 physicals test, it's clear that as in FM24, most mental/technical attributes don't matter at all in FM17. My team coming 8th-14th instead of relegated doubles the evidence for this. This isn't any argument against you at all whatsoever, I just think this is a relevant finding that I want to highlight to others.
I agree and share in with your conclusion that 'speed is still important, but not as crucial as in the new Football Manager'. It seems as though the weighting of speed in FM17 is say a third or a half less, and that this is likely shifted not to all technicals/mentals in general, but to drib/ant/work/comp/conc and so forth.
I inputted my player template that finished ~5th in the Premier League in FM24, into FM17, and got 14th with the 'GOODBYE 442' tactic. But that meant the good results were within reach, so I increased pace & acc from 15 to 16, and got 8th:
So it seems that most attributes not mattering is true in FM17 as it is in FM24. Perhaps not exactly the same, but not far off in any case.
I suspect the difference between 14th/8th for FM17 and 5th for FM24 is actually the quality of the tactic. And perhaps it is the case that the introduction of gegenpress, and/or just the incompetence of the new match engine team (Nic Madden who joined SI for FM16 replaced Paul Collyer as head of match engine/match ai from FM18 onwards according to ChatGPT?), permitted for more overpowered tactics.
So based on this, it seems like player attributes in FM have always been misleading BS, but perhaps at least in some earlier versions, it was adequately obscured by the constraints of the tactic, which is overall more consequential than attributes themselves (though it won't save a dud team, and the ratio seems something like 70/30), and were either more balanced before or it simply wasn't yet exploited by people like Knap fully. Personally I reckon it's both things together.
This may actually be quite useful to know, because if the underlying attribute mechanics have always been the same, and they only have some degree of control over the tactic impact, it seems to me to mean that FM26, FM27, FM28, etc. are either going to have strong exploit tactics, in which case the attribute meta works the same and we can predict that just by looking at the exploit tactic degree of success, or tactics have been flattened down, in which case the attribute meta still applies but now you need more of those lesser attributes in the meta (i.e. anticipation, composure, dribbling, etc.) in order to actually win the season. And they can't go too far down the 2nd route, because then players would complain about not being able to win anything. That's my theory anyway.
A4 said: I only heard that traits don’t really matter much in FM, but I’ve always noticed how good players with “tries tricks” are. I even started teaching it to everyone in my youth academy. Of course, it’s anecdotal evidence, but when youngsters learn it—or even just try to do it—their impact and match ratings seem to be so much higher.
I’ve also noticed the same thing with the Light-Hearted personality. Somehow, good players with that personality seem to perform so much better for me than players with other personalities.
Is this something anyone has ever tested before? Expand
So I've tested now Osimhen having both 'shoots with power' and 'plays one-twos' as ST, as these are the two preferred moves that supposedly had the best benefit in Stybb's testing.
Victor Osimhen default - 3rd, 3rd, 7th, 3rd, 3rd, 4th, 3rd, 4th = 3.833 position Osimhen w/ shoots with power, plays one-twos - 5th, 6th, 6th, 3rd, 5th = 5 position
I don't think there's enough samples to conclude the preferred moves made him worse. I think it's unlikely, but it is possible since it could be overriding his role in the Knap tactic.
What we can be sure of is that it definitely hasn't made him better.
I picked Osimhen because I already have test data on him in a fairly controlled team, he's one of more consistent players of the players I have data on, he's a good ST but not overly good (we want some headroom for any improvement to shine through), and he's a well rounded player that excels in pace & acc & finishing.
bf3metro said: any FMSS update? please, thanks mate! Expand Busy in life for a few days right now, but I'll get round to it probably after that
OpticFawn said: what values would you put for Injury Proneness, natural fitness, dirtiness and aggression as simple values on other platforms? Expand You could input the values I give for Genie Scout
The relative importance of those attributes are:
Dirtiness = Very important Injury proneness = Moderate importance Natural fitness & Aggression = Minor importance
LightningFlik said: Why is the left foot worth more than the right? Expand It's just something I added in to toy around with later.
To compare the relative impacts:
left20/right6 vs right20/left6 = 20 difference 1 loyalty vs 20 loyalty = 19 difference
That's ~0.2 pace.
Thinking about it now, I should probably remove the left foot preference for the next version. 0.2 pace is small, but not exactly nothing.
I tried thinking of a valid reason to keep it:
Even if left vs. right makes no performance difference, when it does in real life, there are fewer left-footed players in the database, and we need some left-footed players for set pieces so we should skew slightly towards them.. but then again, set piece attributes are proven to be useless.
I guess the only real testing I can think of when it comes to preferred foot performance is HarvestGreen's test, and while his results show that foot side matters depending on position, I don't think we have any real information on left vs right in a central position such as ST or DC.
I'm not quite sure what to make of Yarema's above 2 posts to be honest. Seemed like a jab to me. I figured he is passive aggressively saying, if this is the unrefined method with which you handle this aspect of your data, then how can we trust your handling of the supposedly 'objective' position data? In other words, conflating the assessment of error of the weightings, with the assessment of error of the position data. He seems contented in having muddied the waters.
Maybe I'm over-interpreting his vaguery, but nonetheless I'm going to clarify it if he won't.
For the position data, ChatGPT gives me the following 95% confidence intervals:
Things get hairy for the handful below Dybala.
It surmizes it all overall as:
Minimum ±0.38 1st quartile (Q1) ±0.82 Median ±1.38 Mean ±1.52 3rd quartile (Q3) ±1.69 90th percentile (approx.) ±2.50
Now the color coded system I used for the % figures, which represents the best fit of the attribute weightings to the position data, is just to show which players evidently require further attribute weight adjustments in order to be reasonably consistent with the position data. There will likely always be outliers, so I also wanted to show just large this group is - it's currently ~12-15%, which I think isn't too bad.
As for the confidence interval of the % results, it took a while in ChatGPT and I'm not entirely sure I've done it correctly, but it should be this:
95% prediction interval is ±9–12 percentage points 80% prediction interval is ±6.3-7.7 percentage points
This should take into account both the position data uncertainty and the best fit uncertainty of the % figures.
If the % was fitted to the average position data perfectly, it would have a 95% confidence interval of ±3.6 percentage points. And that itself could be improved with more samples taken.
Yarema said: I was planning on staying out of this thread, but ... you're actually calculating error rate by "I checked 32 players and 4 didn't fit my model"? Expand Statistical accuracy of my player rating system
I compared my 60–90% player ratings against the objective average positions from 32 players.
Results:
Pearson correlation: −0.80 — strong relationship between rating and objective position R²: 63.8% — the rating explains approximately 64% of the variation in position Unexplained variation: 36.2% Regression slope: −0.241 positions per percentage point — higher ratings predict better positions MAE: 0.73 positions — the rating misses the objective position by ~0.73 places on average RMSE: ~1.05 positions — larger errors are penalised more heavily Normalised MAE: ~15.5% — average error relative to the mean objective position
In short: the rating system has a strong relationship with the objective rankings, with an average error of around 0.73 positions.
For your forum, I'd describe this as:
Using a ±5 percentage-point tolerance, 5 of 32 players (15.6%) fall outside the expected range, meaning 84.4% of the ratings fall within ±5% of the regression-derived expected value.
Thanks ChatGPT!
2017 called, they want their 'Do you even have a PhD in Statistics?' line back.
mytreds said: Would any of the advice in the OP work in older FM titles? Expand For attributes, too hard to say, but I think it's more likely things have stayed the same than that they have changed. I suspect it would largely hold true going back to ~FM18 at least, but that's just my guess.
For everything else, it should be true going back a long way.
Ndour17 said: Reading back through this thread's evolution, a few points seem worth flagging clearly — not to dismiss the overall contribution (the physical-attributes-dominate thesis holds up well against Zippo's isolated A/B test and skawkclsrn's large-scale randomized study), but because several specific claims here don't hold up as well as they're presented:
The "position proficiency caps at 18" claim (repeated with confidence early on) turned out to be wrong once tested — worth flagging since it may still be floating around in older posts/quotes. The Genie Scout personality-weighting bug wasn't caught internally — it took LightningFlik's independent tool to demonstrate the Osimhen-over-Haaland inconsistency. Worth being more explicit that GS output shouldn't be trusted at face value for hidden attributes even now. The "Blended" file and the "15 pace/acc realistic template" contradict each other on your own numbers (the Newcastle-vs-synthetic-squad comparison) — this was acknowledged but never really resolved, just reframed as "the file is a rough guesstimate." That's a fair characterization, but it should probably be stated up front rather than discovered by a reader cross-checking the math. The "attribute thresholds are relative to competition level" theory, used to explain away discrepancies with ykykyk05251/Zippo/Orion/HarvestGreen22, has never actually been tested — and skawkclsrn's much larger randomized dataset found no thresholds anywhere, which cuts against it rather than for it. A lot of the decimal-precision weights (Pressure=52, Determination=56, etc.) come from single small-sample runs, some explicitly labeled "pure speculation" or "I don't remember why I set this" in your own posts. That's fine as a starting point, but the precision of the numbers implies a confidence level the underlying data doesn't really support.
None of this undercuts the value of the overall synthesis — it's clearly the most complete one on the forum. But the granular numbers probably deserve a "confidence level" caveat next to them, given how often they've had to be walked back or reframed. Expand Yes, these are good points, and most of them I still keep in mind, but it can be difficult to remember everything. I can see you've done a close reading, because I know some things here that I think everybody would have skipped over.
I will clarify a few things:
1) The hiddens bug with Genie Scout indeed remains unresolved, so I highly recommend to use FMST26 or FMSS. I choose to put out a GS file anyway because there are still useless features it has that the other two lack, plus its the established tool. I halved the personality weightings in my GS file to try and temper whatever inaccuracies arise, as personality turns out to be relatively unimportant for performance, but they remain included because I have to keep injury proneness and dirtiness in there anyway - there's no way around it. I'm no longer doing the ranking-to-weights match up work in GS, but I was initially, so this should bypass the bug issue to a degree, though now I do the weight work in FMST26 so it will probably drift more and more away from being accurate in GS. Long story short though, I think it's viable to continue using GS.
2) Pressure, determination, I think perhaps aggression, and no doubt a couple of others that are mainly 0 CA attributes were never properly refined in my template tests. So I tested determination at 14, but unlike many/most other attributes, I didn't get round to trying it at 13, 12, and so on. So in theory an attribute like aggression could be anywhere between '1' and '11' as a requirement. It's set at 11 because I believe it matters, unlike say passing, based on either other people's tests, or my own previous tests (say the 1CA tests). I figured at the time that since these are 0 CA cost attributes, or necessary for player growth anyway (i.e. determination), then it's not a priority to pin them down like I pinned down agility to 10. I ended up never getting around to completing it thoroughly.
3) I am pretty dead certain there are attribute thresholds relative to competition level. HarvestGreen22's data shows this clearly. It's what I have observed, even now with the real player rank data albeit its more convincing than definitive - Haaland comes no.1, but he has work rate of 13. Luis Suarez has godly technicals & mentals, but very low acc/pace/sta, and we see he is one of the worst performers. Those are the obvious examples, but I've seen many more subtle ones that are nonetheless identifiable. All that said, work rate does appear to be of importance even beyond ~11-13, and we see similar things for stamina and anticipation and whatnot, so while I'm sure there are attribute thresholds, the understanding of the curvature could do with refinement. Part of it could be that certain things kick in at certain levels. My observation with GKs is that at a mid level, reflexes matter less, but reflexes seem to take precedence at a high level (~15+) while aerial reach is essential at mid level but is significantly less essential at a high level (~15+).
4) I don't recall the Blended file vs template problem. And I'm not sure which came first, so I can't assume it's that the template invalidated the Blended file. All I can say is that when I did my best attempt at an objective comparative test of all the files, the Blended one did poorly. From memory, the Blended file was mainly based on HarvestGreen22's data + consideration of CA weights + consideration of player growth. I was also still using positional differences at that time, which now turns out to be bunk. As a point of history now, we can go back and ask 'Well, was the Blended file actually a step backwards from even the first file, and therefore do you regret putting it out there?'. At least that's the question I ask myself. And for the other files, I can say, 'they were better than the first file, but perhaps only marginally in hindsight'. Actually was striking to me was how although the improvement was marginal, the improvements of my files and those of others appear remarkably gradual and consistent. I would have expected things to be more over the place. And I could be wrong, but I think this latest method and the weightings derived from it are a big step forward.
5) I think it needs to be restated the current precision of the attribute weights does not reflect the actual real influence/value of the attribute. I'm merely moving numbers up and down to get a best fit of the data. It's like wedging a piece of wood inbetween two metal joints to make the leaning tower stable. It's not the right material, but if it works, it works, and we can replace it with the nuts and bolts later.
Recall that 87% for ST is 2.571 and 73% for ST is 6.111, and that positions other than ST appear to be consistently ~1 position lower.
Every position for the Knap 424 formation has been sampled now, and I'm satisfied that the same set of required attributes apply to all outfield positions in the same way.
I've done some readjustments to the weightings.
I've dropped Sterling the equivalent of ~1 position, but this means he should we should expect no worse than ~4 position, yet his average is 5.375 if we give +1 for not being ST. This is the best I've been able to do.
Non-ST positions seem more comparable to ST now (less unexplained position difference), and stamina is now a more reasonable 40 instead of 94 weight - stamina seemed to be the partial cause of Sterling's overrating. Quite a few players have pleasant coincidental changes, such as Maeda going higher and Gyokeres lower, which is more line with the real rankings.
However some changes had to go with it, such as Vlahovic dropping a little below Messi. I figure it's more likely that Sterling should drop 1 position, than Vlahovic should not drop 0.5 position. And given most players seem more to be more accurate now, I'm not too concerned about Vlahovic.
Next step is translating to Genie Scout weights, then I can upload the updated file. Main reason for updated file will be adding the new GK weights of course.
Rain said: What am I missing here why do you think Mbappe and Sterling should be closer in terms of results? I would assume the +5 Pace, +3 Acceleration, +3 Dribbling, and +5 Determination alone would speak for itself. Expand The method I use is that I compare two players in conjunction with a few others, to rule out most of the attributes.
So for instance, let's say Mbappe is better because he has 20 pace/acc, then why does Haaland actually perform better? Then we find someone with same pace/acc as Haaland, but performs significantly worse. This player has -5 ant, -3 det, -3 work, -7 tack. We can rule out tackling off the bat, but how do we know how the remaining 3 attributes contribute? We compare two other completely different players to each other and one has +4 ant -2 work and the other has -4 ant +2 work, and find the +4 ant -2 work one performs much better. So now we know it's likely ant is the key missing factor, or at least contributes that precise position difference. We then verify that this would line up the rankings more correctly instead of no change or worse.
So that process gets us to the final weights, which we know are close to valid because almost all the players line up well without contradictions, and according to the weights Sterling should be closer to Mbappe.
If you compare Sterling to Mbappe, it's no problem, but what if you compare Sterling (6.6) to Saka (2.75):
There are a few differences, but nothing major. We know all the technicals are useless, aside from dribbling. That leaves a handful of minor to moderate differences in mentals, balance, and pressure (hidden), as the possible causes.
And the thing is, he's pretty well rounded in every attribute, so there's not obvious non-linear threshold being crossed for a certain attribute. I thought perhaps composure, but my template says 11 is the minimum, and he has 12. Thinking perhaps pressure right now.
A4 said: There are some weird results in this video, I don't know the guy and how trusted his test is but it's way different than pretty much all the actual results I saw
Expand Coincidentally I just made a comment on this video before coming here and seeing this post.
I was actually going to bring him up in my response concerning traits to you, and I'm guessing you asked it perhaps because you saw his video on traits too.
The jist of my post was that either this guy has something seriously wrong with him, or I'm missing something. Hence, another reason to give traits another test.
I first came across him ages ago, about 3 years ago, in which he claimed to do a proper test that showed scout JPA has a strong effect on newgens, which I knew from my own testing to be absolutely false. I left a comment about this, but he didn't respond.
Fast forward to now, and these past few months he's supposedly running and pumping out all these in-depth controlled tests with a discord community to back him, but if you go to it, it's a ghost town. Strange. And the tests themselves, obviously fly in the face of the results derived by others doing similar testing.
But I realized today it could be that it's because he's assessing by goals conceded and goals scored, which is not an adequate measure. I keep forgetting myself that final season position is the only reliable measure, at least that I know of. Still.. the scout JPA result can't be explained by this, so I simply don't trust the guy on anything. Maybe when I do the trait test, I'll also check to see if it affects goals difference as he claims it does.
I decided to leave all this out of my post before in the end, because obviously it sounds obsessive and veering off from what you asked, but its what came to mind. Occasionally youtubers can be a source of fresh good info.
78% | Illia Zabarnyi (DC) - 3rd, 5th, 4th, 5th, 2nd = 3.8 position
82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position
77% | Alessandro Bastoni (DC) [Exellent version] - 3rd, 5th, 5th, 3rd, 3rd, 7th = 4.333 position
75% | Kurt Zouma (DC) - 7th, 5th, 6th, 3rd = 5.6 position
73% | Shota Fukuoka (DC) [Exellent version] - 7th, 8th, 7th, 8th, 8th, 11th = 8.166 position
71% | Sean Goldberg (DC) - 5th, 16th (sacked), 8th (sacked), 5th = 8.5 position
Note: I capped Zabarnyi's PA to just above CA so that he wouldn't grow during the season.
You can see here why I automatically assumed Fukuoka was better than Bastoni, on the basis of his very low dirtiness and injury proneness.
I didn't set dirtiness at such a high negative weighting for the heck of it.
I know it's unintuitive but I'm just reflecting what the data shows. And actually this is the most clear cut example I've seen, for with Maeda you have stuff like work rate and anticipation clouding it a bit. But here, Zabaryni literally has no other advantage than +3 composure, and many disadvantages (including in meaningful attributes!).
I challenge anyone to try and explain why Zabarnyi finishes higher than Bastoni looking at the comparison above, without concluding dirtiness and injury proneness has a massive impact.
Now I have had the thought that it could be that low dirtiness is the necessary cherry on top at the highest level, but diminishes in relative importance at lower levels. I lean towards no, but we'll cross this bridge when we come to it - I figure that the most common, or final goal, is to win at a high level in the game, and so that is what I have remained focused on for now.
77% | Alessandro Bastoni (DC) [Exellent version] - 3rd, 5th, 5th, 3rd, 3rd, 7th = 4.333 position
75% | Kurt Zouma (DC) - 7th, 5th, 6th, 3rd = 5.6 position
73% | Shota Fukuoka (DC) [Exellent version] - 7th, 8th, 7th, 8th, 8th, 11th = 8.166 position
71% | Sean Goldberg (DC) - 5th, 16th (sacked), 8th (sacked), 5th = 8.5 position
I withdraw saying that Fukuoka is better than Bastoni. My mistake was to assume that the GS rating reflected the FMST26 rating.
But we see here that the rating for both players in FMST26 aligns correctly.
I got ahead of myself because I know that Maeda & Furuhashi are slightly better than Lautaro Martinez, even though Martinez has better attributes in literally almost every way - except dirtiness and injury proneness, and the case looked at a glance to be the same here.
So there is a translation to GS problem here.
Now if I compare DC % between FMST26 and GS, the following players have significant discrepancies (2% or more, when both are adjusted to 77.9% max):
Declan Rice - 77.1% (FMST26) vs. 74.76% (GS) = -2.34%
Fiyako Tomori - 71.9% (FMST26) vs. 74.33% (GS) = +2.43%
Nacho - 70.9% (FMST26) vs. 73.51% (GS) = +2.61%
Levi Colwill - 71.1% (FMST26) vs. 73.27% (GS) = +2.17%
Shota Fukuoka [Exellent version] - 69.3% (FMST26) vs. 73.32% (GS) = +4.02%
Pantelis Hatzidiakos - 69.1% (FMST26) vs. 72.10% (GS) = +3.00%
Cristian Romero - 69.1% (FMST26) vs. 71.96% (GS) = +2.86%
Pascal Struijk - 68.8% (FMST26) vs. 71.88% (GS) = +3.08%
Federico Dimarco - 74.0% (FMST26) vs. 71.72% (GS) = -2.28%
Maxence Lacroix - 68.7% (FMST26) vs. 71.69% (GS) = 2.99%
That's 10 out of the top 70 players (14.3%), and he happened to find the worst one of the bunch, so overall I'm not too concerned. Bastoni has a 1.6% underrating in GS.
Maeda, who benefits from the high negative dirtiness and injury proneness values (he has '2' and '3' respectively) already had to be compromised down from 4th in FMST26 to 16th in GS, so it's a bit of a damned if you do, damned if you don't situation with GS. This is just the limit of GS. No doubt it could be improved a bit, but it's not going to be able to match FMST26.
So based on the rankings you see above, as well as 78% | Jules Kounde (DR), I would expect Bastoni to come ~4.8 position on average, but we'll see.
your super rating says that Shota Fukuoka is better than Bastoni.
I've inputted the stats shown for Fukuoka into FM24. I ended up with 131 instead of 135 CA, probably because of some lacking DL/DR/weak foot proficiency, but that's no issue. The only other difference is that I've changed Fukuoka's nation to England, so that his low adaptability doesn't mess with things.
Here is the result:
82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position
75% | Kurt Zouma (DC) - 7th, 5th, 6th, 3rd = 5.6 position
73% | Shota Fukuoka (DC) [Exellent version] = 7th, 8th, 7th, 8th, 8th, 11th = 8.166 position
71% | Sean Goldberg (DC) - 5th, 16th (sacked), 8th (sacked), 5th = 8.5 position
I'm using the FMST26 % as that is more accurate and what I always refer to, but you can see the difference is only 73% vs 75% anyway.
Kurt Zouma is quite similar to Bastoni if you compare the stats:
But I will also do a clone of Bastoni for comparison so that there's no doubt about it.
This is all a good test of my claims, because this way no one can say it's only valid for cherrypicked players.
I figured give each team or file that is being tested the top 18 or so guys by rating and positions at 1 mil or under and every team use the same exact tactic. Could either replace the lowest prem teams or just run it in whatever test league is used. I could set it up I'm sure I just need to know which files to even test I'm losing track at this point.
I think I chose rank 60th partly to help avoid what you mentioned, every team selecting the same best handful of players, but also to make the results clearer (i.e. 8th vs 14th, instead of 1st vs 1st). I expected there to be too much overlap anyway, since I was selecting 1-2 backups as well, but actually about 50% of the team ended up different with each GS file, and there were significant differences in the results.
You could use the Premier League 2.1 GS file, and other's GS files of your choice.
That would require them to actually do a test though. 😂
I did exactly this with 2 of my files and Orion's as a control. The search isn't working right now, so I can't find the exact results right now, both of my files did something like 6 positions better.
I think I used $1mil budget per player, and picked the 60th ranked player in position + backups just beneath them of course.
It was tedious (even for me!), and unfortunately when it came to revisiting it for testing the new file that came after that, I hadn't tracked everything properly so I would have to redo the selection for all 3 files in addition again so I didn't bother and used a different method of assessment instead.
But it's probably worth revisiting in the future.
Although no doubt we'll still hear in response 'I don't want players under $1mil! I want the best players! If they have an extra 6 passing for an extra $50mil, I don't care, I want it!' from Yarema, treath, and friends.
You’ve been working on this rating for 9 months with zero positive results.
Maybe it's time to just give up?
Also, your super rating says that Shota Fukuoka is better than Bastoni.
On what basis, other than your own incredulity, do you find that Fukuoka is inferior to Bastoni?
To me, it's clear as day that the reason Fukuoka is superior is his low dirtiness and injury proneness. You may not like it, but this is what peak performance looks like.
I do not know if you are using FM26 or FM24, but in a FM24 save I have that is a few years in, Chalobah's last season performance was 7.11 average for A. Madrid, and Zabarnyi's was 7.17 for 31 games in the English Premier League (7.23 for 38 games in the season before that).
Try this experiment. Set all physical attributes to 20, and dribbling, anticipation, and work rate to 20. With these attributes, you win everything in FM 24 and 26. In the 2017 version, you win nothing.
But remember to set all other attributes to 1. Not 8, not 6, just 1.
I was thinking to mention that I am not disputing your findings. Nor even the main conclusion. It's just that I am aware of some likely cavaets from having done the same kind of tests with FM24.
So with the 20 drib/work/ant test, the problem is that it won't work without a couple of other key ones such as concentration. As you saw, adding composure (which is one of the essential attributes), improved the position from 19th to 6th. You say you win everything with it in FM24.. I guess it might be possible, since you do have anticipation added not to mention every physical at 18.. but just be aware that it'd be very much on the knife-edge.
So that one for me is easily explained away.
Winning with 15 physicals is also no problem in FM24, so long as you have the adequate mentals and drib, which you have provided 20 for.
The 20 mental/technical 10 physical one is somewhat more compelling. I've tested a player who just about matches this template, Dybala, in FM24, and the result was very poor. I'm sure that if I played with a full squad of Dybalas, they'd be relegated in FM24.
However it could be one of these two things:
1) Dybala has one key difference, which is having just 9 work rate. Kane and Lewandowski do very well in FM24 which shows that certain high mentals/technicals can make up for even ~13 physicals. But even then, 10 should be a bridge too far
2) It could be that you not done your test on 'full detail', which is something easy to overlook
Nonetheless let us just assume it is valid:
Your test shows that the physical requirement has dropped from ~13 in FM24 to 10 in FM17.
My test shows that what comes 5th in FM24, with an emphasis on pace/acc, comes 14th in FM17 - though increasing pace/acc slightly bumps that up to 8th.
From your 20 ant/comp/drib/work 18 physicals test, it's clear that as in FM24, most mental/technical attributes don't matter at all in FM17. My team coming 8th-14th instead of relegated doubles the evidence for this. This isn't any argument against you at all whatsoever, I just think this is a relevant finding that I want to highlight to others.
I agree and share in with your conclusion that 'speed is still important, but not as crucial as in the new Football Manager'. It seems as though the weighting of speed in FM17 is say a third or a half less, and that this is likely shifted not to all technicals/mentals in general, but to drib/ant/work/comp/conc and so forth.
I inputted my player template that finished ~5th in the Premier League in FM24, into FM17, and got 14th with the 'GOODBYE 442' tactic. But that meant the good results were within reach, so I increased pace & acc from 15 to 16, and got 8th:
So it seems that most attributes not mattering is true in FM17 as it is in FM24. Perhaps not exactly the same, but not far off in any case.
I suspect the difference between 14th/8th for FM17 and 5th for FM24 is actually the quality of the tactic. And perhaps it is the case that the introduction of gegenpress, and/or just the incompetence of the new match engine team (Nic Madden who joined SI for FM16 replaced Paul Collyer as head of match engine/match ai from FM18 onwards according to ChatGPT?), permitted for more overpowered tactics.
So based on this, it seems like player attributes in FM have always been misleading BS, but perhaps at least in some earlier versions, it was adequately obscured by the constraints of the tactic, which is overall more consequential than attributes themselves (though it won't save a dud team, and the ratio seems something like 70/30), and were either more balanced before or it simply wasn't yet exploited by people like Knap fully. Personally I reckon it's both things together.
This may actually be quite useful to know, because if the underlying attribute mechanics have always been the same, and they only have some degree of control over the tactic impact, it seems to me to mean that FM26, FM27, FM28, etc. are either going to have strong exploit tactics, in which case the attribute meta works the same and we can predict that just by looking at the exploit tactic degree of success, or tactics have been flattened down, in which case the attribute meta still applies but now you need more of those lesser attributes in the meta (i.e. anticipation, composure, dribbling, etc.) in order to actually win the season. And they can't go too far down the 2nd route, because then players would complain about not being able to win anything. That's my theory anyway.
I’ve also noticed the same thing with the Light-Hearted personality. Somehow, good players with that personality seem to perform so much better for me than players with other personalities.
Is this something anyone has ever tested before?
So I've tested now Osimhen having both 'shoots with power' and 'plays one-twos' as ST, as these are the two preferred moves that supposedly had the best benefit in Stybb's testing.
Victor Osimhen default - 3rd, 3rd, 7th, 3rd, 3rd, 4th, 3rd, 4th = 3.833 position
Osimhen w/ shoots with power, plays one-twos - 5th, 6th, 6th, 3rd, 5th = 5 position
I don't think there's enough samples to conclude the preferred moves made him worse. I think it's unlikely, but it is possible since it could be overriding his role in the Knap tactic.
What we can be sure of is that it definitely hasn't made him better.
I picked Osimhen because I already have test data on him in a fairly controlled team, he's one of more consistent players of the players I have data on, he's a good ST but not overly good (we want some headroom for any improvement to shine through), and he's a well rounded player that excels in pace & acc & finishing.
Busy in life for a few days right now, but I'll get round to it probably after that
OpticFawn said: what values would you put for Injury Proneness, natural fitness, dirtiness and aggression as simple values on other platforms?
You could input the values I give for Genie Scout
The relative importance of those attributes are:
Dirtiness = Very important
Injury proneness = Moderate importance
Natural fitness & Aggression = Minor importance
It's just something I added in to toy around with later.
To compare the relative impacts:
left20/right6 vs right20/left6 = 20 difference
1 loyalty vs 20 loyalty = 19 difference
That's ~0.2 pace.
Thinking about it now, I should probably remove the left foot preference for the next version. 0.2 pace is small, but not exactly nothing.
I tried thinking of a valid reason to keep it:
Even if left vs. right makes no performance difference, when it does in real life, there are fewer left-footed players in the database, and we need some left-footed players for set pieces so we should skew slightly towards them.. but then again, set piece attributes are proven to be useless.
I guess the only real testing I can think of when it comes to preferred foot performance is HarvestGreen's test, and while his results show that foot side matters depending on position, I don't think we have any real information on left vs right in a central position such as ST or DC.
Maybe I'm over-interpreting his vaguery, but nonetheless I'm going to clarify it if he won't.
For the position data, ChatGPT gives me the following 95% confidence intervals:
Things get hairy for the handful below Dybala.
It surmizes it all overall as:
Minimum ±0.38
1st quartile (Q1) ±0.82
Median ±1.38
Mean ±1.52
3rd quartile (Q3) ±1.69
90th percentile (approx.) ±2.50
Now the color coded system I used for the % figures, which represents the best fit of the attribute weightings to the position data, is just to show which players evidently require further attribute weight adjustments in order to be reasonably consistent with the position data. There will likely always be outliers, so I also wanted to show just large this group is - it's currently ~12-15%, which I think isn't too bad.
As for the confidence interval of the % results, it took a while in ChatGPT and I'm not entirely sure I've done it correctly, but it should be this:
95% prediction interval is ±9–12 percentage points
80% prediction interval is ±6.3-7.7 percentage points
This should take into account both the position data uncertainty and the best fit uncertainty of the % figures.
If the % was fitted to the average position data perfectly, it would have a 95% confidence interval of ±3.6 percentage points. And that itself could be improved with more samples taken.
Statistical accuracy of my player rating system
I compared my 60–90% player ratings against the objective average positions from 32 players.
Results:
Pearson correlation: −0.80 — strong relationship between rating and objective position
R²: 63.8% — the rating explains approximately 64% of the variation in position
Unexplained variation: 36.2%
Regression slope: −0.241 positions per percentage point — higher ratings predict better positions
MAE: 0.73 positions — the rating misses the objective position by ~0.73 places on average
RMSE: ~1.05 positions — larger errors are penalised more heavily
Normalised MAE: ~15.5% — average error relative to the mean objective position
In short: the rating system has a strong relationship with the objective rankings, with an average error of around 0.73 positions.
For your forum, I'd describe this as:
Using a ±5 percentage-point tolerance, 5 of 32 players (15.6%) fall outside the expected range, meaning 84.4% of the ratings fall within ±5% of the regression-derived expected value.
Thanks ChatGPT!
2017 called, they want their 'Do you even have a PhD in Statistics?' line back.
For attributes, too hard to say, but I think it's more likely things have stayed the same than that they have changed. I suspect it would largely hold true going back to ~FM18 at least, but that's just my guess.
For everything else, it should be true going back a long way.
FMST26 weights (copy paste this into statistics > custom metrics (advanced)):
Outfield:
(acceleration * 90.5 + pace * 94 + jumping_reach * 43 + pressure * 17 + dribbling * 14.3 + work_rate * 66.5 + agility * 16.0 + anticipation * 40 + composure * 26 + stamina * 50 + consistency * 14.05 + determination * 43.5 + balance * 14.25 + strength * 5.0 + concentration * 45 + finishing * 2.0 + important_matches * 10.1 + natural_fitness * 6.45 + professionalism * 15.0 + ambition * 2.0 + loyalty * 1.0 + aggression * 10 + vision * 0.5 + left_foot * 6 + right_foot * 5 - (injury_proneness * (48 / (natural_fitness * 0.75))) - (dirtiness * 32 * (aggression * 0.1))) / 113.2
GK:
((aerial_reach * 28.68 + command_of_area * 15.69 + first_touch * 16.99 + passing * 10.46 + reflexes * 86 + concentration * 82.35 + determination * 88.89 + work_rate * 38.68 + acceleration * 15.03 + balance * 10.46 + agility * 32.68 + jumping_reach * 14.30 + natural_fitness * 10.46 + pace * 28.30 + stamina * 32.68 + strength * 10.46 + technique * 15.69 + pressure * 15.03 + professionalism * 9.80 + flair * 49.02 - injury_proneness * 49.02 + vision * 25 - dirtiness * 16.34 - ca * 9.80)) / 84.5
Genie Scout file: https://files.catbox.moe/9n18q0.grf
I translated it to GS as best I could.
Outfield:
90% | Erling Haaland - 1st, 1st, 4th, 1st, 2nd, 1st = 1.666 position
88% | Kylian Mbappe - 2nd, 3rd, 3rd, 3rd, 5th, 2nd, 1st, 1st = 2.5 position
87% | Mohamed Salah* - 1st, 3rd, 5th, 3rd, 3rd, 2nd, 1st = 2.571 position
88% | Kylian Mbappe (AML) - 1st, 2nd, 3rd, 5th = 2.75 position
82% | Bukayo Saka (AMR) - 2nd, 3rd, 3rd, 3rd = 2.75 position
82% | Heung-Min Son - 3rd, 2nd, 2nd, 2nd, 2nd, 6th, 4th = 3 position
83% | Daizen Maeda - 2nd, 3rd, 4th, 4th, 2nd, 3rd, 3rd = 3 position
80% | Kyogo Furuhashi - 4th, 2nd, 3rd, 5th, 3rd, 2nd, 2nd = 3 position
82% | Vinicius Junior - 5th, 4th, 1st, 3rd = 3.25 position
83% | Eduardo Camavinga (DM) = 3rd, 5th, 2nd, 3rd, 4th = 3.4 position
82% | Harry Kane - 5th, 2nd, 3rd, 3rd, 1st, 7th = 3.5 position
83% | Robert Lewandowski - 2nd, 2nd, 2nd, 3rd, 5th, 2nd, 8th, 5th, 3rd = 3.555 position
77% | Dusan Vlahovic - 1st, 4th, 4th, 5th, 1st, 6th, 4th, 4th = 3.625
82% | Lautaro Martinez - 7th, 3rd, 1st, 5th, 2nd, 5th, 2nd, 5th = 3.75 position
80% | Victor Osimhen - 3rd, 3rd, 7th, 3rd, 3rd, 4th, 3rd, 4th = 3.833 position
78% | Lionel Messi - 4th, 2nd, 1st, 5th, 9th, 3rd = 4 position
79% | Viktor Gyokeres - 6th, 4th, 3rd, 4th, 5th, 5th, 2nd, 5th, 2nd = 4 position
81% | Giovanni Di Lorenzo (DR) - 3rd, 3rd, 2nd, 9th, 2nd, 5th, 5th = 4.143 position
82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position
82% | Ronald Araujo (DR) - 4th, 8th, 3rd, 1st, 7th = 4.6 position
73% | Donyell Malen - 2nd, 2nd, 7th, 5th, 5th, 7th = 4.666 position
78% | Jules Kounde (DR) - 5th, 4th, 7th 5th, 3rd = 4.8 position
75% | Kurt Zouma (DC) - 7th, 5th, 6th, 3rd = 5.6 position
72% | Robert Glatzel - 5th, 7th, 6th, 3rd, 12th, 6th, 2nd, 6th, 8th = 6.111 position
81% | Raheem Sterling (AMR) - 6th, 7th, 7th, 7th, 6th, 6th, 6th, 6th = 6.375 position
69% | Paulo Dybala - 5th, 5th, 6th, 10th, 3rd, 12th, 6th, 5th, 6th, 6th = 6.4 position
72% | Andrian Kraev (DM) = 9th, 8th, 5th, 5th = 6.75 position
62% | Adam Le Fondre - 8th, 11th (sacked), 5th, 7th (sacked), 6th (sacked), 8th (sacked) = 7.5 position
66% | Kingsley Schindler (DR) - 4th, 11th, 6th, 11th = 8 position
63% | Luis Suarez - 7th, 9th, 10th, 8th, 8th = 8.4 position
71% | Sean Goldberg (DC) - 5th, 16th (sacked), 8th (sacked), 5th = 8.5 position
77% | Sacha Boey (DR) - 7th (sacked), 12th (sacked), 12th (sacked) = 10.333 position
Error rate = 4/32 = 12.5%
Vlahovic = 6.5% off
Malen = 6.8% off
Sterling = 12.3% off
Boey = 22.1% off
GK:
90% | Alisson - 6th, 3rd, 4th, 4th, 3rd = 4 position
84% | Thibaut Courtois - 3rd, 7th, 2nd, 5th, 8th = 5 position
82% | Jan Oblak - 5th, 8th, 6th, 4th, 3rd = 5.2 position
82% | Walter Benitez - 4th, 9th, 4th, 6th, 5th = 5.6 position
82% | Ugurcan Cakir - 7th, 9th, 2nd = 6 position
79% | Ederson - 6th, 6th, 4th, 8th, 7th, 7th = 6.333 position
81% | Marc-Andre ter Stegen - 10th, 11th, 4th, 4th, 3rd = 6.4 position
84% | Gregor Kobel - 4th, 12th, 2nd, 10th (sacked), 3rd, 10th (sacked), 5th = 6.571 position
78% | Anatolii Trubin - 15th (sacked), 6th, 4th (sacked), 5th = 7.5 position
70% | Davy Roef - 13th (sacked), 4th, 14th (sacked), 6th (sacked), 1st = 7.6 position
61% | Jack Stevens - 13th (sacked), 4th, 9th (sacked), 8th (sacked), 8th (sacked) = 8.4 position
75% | Sinan Bolat - 4th, 2nd, 19th (sacked), 5th, 14th (sacked), 9th (sacked), 12th (sacked), 5th = 8.75 position
67% | Ben Winterbottom - 10th (sacked), 5th (sacked), 12th (sacked), 18th (sacked) = 11.25 position
Harder to tell who is inaccurate and who is error for GK..
Error rate = 3/13 = 23.1%
Kobel = 7.1% off
Roef = 7.1%(?) off
Stevens = 18.0%(?) off
The "position proficiency caps at 18" claim (repeated with confidence early on) turned out to be wrong once tested — worth flagging since it may still be floating around in older posts/quotes.
The Genie Scout personality-weighting bug wasn't caught internally — it took LightningFlik's independent tool to demonstrate the Osimhen-over-Haaland inconsistency. Worth being more explicit that GS output shouldn't be trusted at face value for hidden attributes even now.
The "Blended" file and the "15 pace/acc realistic template" contradict each other on your own numbers (the Newcastle-vs-synthetic-squad comparison) — this was acknowledged but never really resolved, just reframed as "the file is a rough guesstimate." That's a fair characterization, but it should probably be stated up front rather than discovered by a reader cross-checking the math.
The "attribute thresholds are relative to competition level" theory, used to explain away discrepancies with ykykyk05251/Zippo/Orion/HarvestGreen22, has never actually been tested — and skawkclsrn's much larger randomized dataset found no thresholds anywhere, which cuts against it rather than for it.
A lot of the decimal-precision weights (Pressure=52, Determination=56, etc.) come from single small-sample runs, some explicitly labeled "pure speculation" or "I don't remember why I set this" in your own posts. That's fine as a starting point, but the precision of the numbers implies a confidence level the underlying data doesn't really support.
None of this undercuts the value of the overall synthesis — it's clearly the most complete one on the forum. But the granular numbers probably deserve a "confidence level" caveat next to them, given how often they've had to be walked back or reframed.
Yes, these are good points, and most of them I still keep in mind, but it can be difficult to remember everything. I can see you've done a close reading, because I know some things here that I think everybody would have skipped over.
I will clarify a few things:
1) The hiddens bug with Genie Scout indeed remains unresolved, so I highly recommend to use FMST26 or FMSS. I choose to put out a GS file anyway because there are still useless features it has that the other two lack, plus its the established tool. I halved the personality weightings in my GS file to try and temper whatever inaccuracies arise, as personality turns out to be relatively unimportant for performance, but they remain included because I have to keep injury proneness and dirtiness in there anyway - there's no way around it. I'm no longer doing the ranking-to-weights match up work in GS, but I was initially, so this should bypass the bug issue to a degree, though now I do the weight work in FMST26 so it will probably drift more and more away from being accurate in GS. Long story short though, I think it's viable to continue using GS.
2) Pressure, determination, I think perhaps aggression, and no doubt a couple of others that are mainly 0 CA attributes were never properly refined in my template tests. So I tested determination at 14, but unlike many/most other attributes, I didn't get round to trying it at 13, 12, and so on. So in theory an attribute like aggression could be anywhere between '1' and '11' as a requirement. It's set at 11 because I believe it matters, unlike say passing, based on either other people's tests, or my own previous tests (say the 1CA tests). I figured at the time that since these are 0 CA cost attributes, or necessary for player growth anyway (i.e. determination), then it's not a priority to pin them down like I pinned down agility to 10. I ended up never getting around to completing it thoroughly.
3) I am pretty dead certain there are attribute thresholds relative to competition level. HarvestGreen22's data shows this clearly. It's what I have observed, even now with the real player rank data albeit its more convincing than definitive - Haaland comes no.1, but he has work rate of 13. Luis Suarez has godly technicals & mentals, but very low acc/pace/sta, and we see he is one of the worst performers. Those are the obvious examples, but I've seen many more subtle ones that are nonetheless identifiable. All that said, work rate does appear to be of importance even beyond ~11-13, and we see similar things for stamina and anticipation and whatnot, so while I'm sure there are attribute thresholds, the understanding of the curvature could do with refinement. Part of it could be that certain things kick in at certain levels. My observation with GKs is that at a mid level, reflexes matter less, but reflexes seem to take precedence at a high level (~15+) while aerial reach is essential at mid level but is significantly less essential at a high level (~15+).
4) I don't recall the Blended file vs template problem. And I'm not sure which came first, so I can't assume it's that the template invalidated the Blended file. All I can say is that when I did my best attempt at an objective comparative test of all the files, the Blended one did poorly. From memory, the Blended file was mainly based on HarvestGreen22's data + consideration of CA weights + consideration of player growth. I was also still using positional differences at that time, which now turns out to be bunk. As a point of history now, we can go back and ask 'Well, was the Blended file actually a step backwards from even the first file, and therefore do you regret putting it out there?'. At least that's the question I ask myself. And for the other files, I can say, 'they were better than the first file, but perhaps only marginally in hindsight'. Actually was striking to me was how although the improvement was marginal, the improvements of my files and those of others appear remarkably gradual and consistent. I would have expected things to be more over the place. And I could be wrong, but I think this latest method and the weightings derived from it are a big step forward.
5) I think it needs to be restated the current precision of the attribute weights does not reflect the actual real influence/value of the attribute. I'm merely moving numbers up and down to get a best fit of the data. It's like wedging a piece of wood inbetween two metal joints to make the leaning tower stable. It's not the right material, but if it works, it works, and we can replace it with the nuts and bolts later.
73% | Andrian Kraev (DM) = 9th, 8th, 5th, 5th = 6.75 position
Recall that 87% for ST is 2.571 and 73% for ST is 6.111, and that positions other than ST appear to be consistently ~1 position lower.
Every position for the Knap 424 formation has been sampled now, and I'm satisfied that the same set of required attributes apply to all outfield positions in the same way.
I've done some readjustments to the weightings.
I've dropped Sterling the equivalent of ~1 position, but this means he should we should expect no worse than ~4 position, yet his average is 5.375 if we give +1 for not being ST. This is the best I've been able to do.
Non-ST positions seem more comparable to ST now (less unexplained position difference), and stamina is now a more reasonable 40 instead of 94 weight - stamina seemed to be the partial cause of Sterling's overrating. Quite a few players have pleasant coincidental changes, such as Maeda going higher and Gyokeres lower, which is more line with the real rankings.
However some changes had to go with it, such as Vlahovic dropping a little below Messi. I figure it's more likely that Sterling should drop 1 position, than Vlahovic should not drop 0.5 position. And given most players seem more to be more accurate now, I'm not too concerned about Vlahovic.
Next step is translating to Genie Scout weights, then I can upload the updated file. Main reason for updated file will be adding the new GK weights of course.
The method I use is that I compare two players in conjunction with a few others, to rule out most of the attributes.
So for instance, let's say Mbappe is better because he has 20 pace/acc, then why does Haaland actually perform better? Then we find someone with same pace/acc as Haaland, but performs significantly worse. This player has -5 ant, -3 det, -3 work, -7 tack. We can rule out tackling off the bat, but how do we know how the remaining 3 attributes contribute? We compare two other completely different players to each other and one has +4 ant -2 work and the other has -4 ant +2 work, and find the +4 ant -2 work one performs much better. So now we know it's likely ant is the key missing factor, or at least contributes that precise position difference. We then verify that this would line up the rankings more correctly instead of no change or worse.
So that process gets us to the final weights, which we know are close to valid because almost all the players line up well without contradictions, and according to the weights Sterling should be closer to Mbappe.
If you compare Sterling to Mbappe, it's no problem, but what if you compare Sterling (6.6) to Saka (2.75):
There are a few differences, but nothing major. We know all the technicals are useless, aside from dribbling. That leaves a handful of minor to moderate differences in mentals, balance, and pressure (hidden), as the possible causes.
And the thing is, he's pretty well rounded in every attribute, so there's not obvious non-linear threshold being crossed for a certain attribute. I thought perhaps composure, but my template says 11 is the minimum, and he has 12. Thinking perhaps pressure right now.
Coincidentally I just made a comment on this video before coming here and seeing this post.
I was actually going to bring him up in my response concerning traits to you, and I'm guessing you asked it perhaps because you saw his video on traits too.
The jist of my post was that either this guy has something seriously wrong with him, or I'm missing something. Hence, another reason to give traits another test.
I first came across him ages ago, about 3 years ago, in which he claimed to do a proper test that showed scout JPA has a strong effect on newgens, which I knew from my own testing to be absolutely false. I left a comment about this, but he didn't respond.
Fast forward to now, and these past few months he's supposedly running and pumping out all these in-depth controlled tests with a discord community to back him, but if you go to it, it's a ghost town. Strange. And the tests themselves, obviously fly in the face of the results derived by others doing similar testing.
But I realized today it could be that it's because he's assessing by goals conceded and goals scored, which is not an adequate measure. I keep forgetting myself that final season position is the only reliable measure, at least that I know of. Still.. the scout JPA result can't be explained by this, so I simply don't trust the guy on anything. Maybe when I do the trait test, I'll also check to see if it affects goals difference as he claims it does.
I decided to leave all this out of my post before in the end, because obviously it sounds obsessive and veering off from what you asked, but its what came to mind. Occasionally youtubers can be a source of fresh good info.