ZaZ said: If you want people to respect the results of your tests, you need to respect the results of other people too. Saying stuff like "it is all wrong" is not very scientific. Their results are valid for their methodology and data set, and their interpretation is based on the method of their analysis. It is not up to you to decide which of the thesis are more trustworthy. Expand I think you've got the picture wrong.
As I referred to in my post, the OP of that thread said 'For goalkeepers, read the top two or three only.'
I didn't do the full quote, which follows with:
'Reflexes and Agility separate cleanly; below that it is noise, and Handling ranking under Corners is not a finding. One keeper per match against ten outfielders means that column has about a tenth of the statistical power.'
I'm not saying his other findings are 'completely wrong' - only the GK weights.
I'm using my own data to refute the data he got on GKs, that's the sound basis for me saying 'it is all wrong'. And it is up to me to decide which of the thesis are more trustworthy.
Panneton0 said: I'd be interested to see how this specific attribute ends up being even more important than pace/acc. Expand BTW I'll tell you how I got to such a high value for stamina, and the other values in general, since I didn't really explain.
I start with the existing weights, changed a few weights until the players roughly aligned properly, then I carefully compared players who would remain stubbornly out of alignment (fixing one, ruins another).
Maeda was key to this process. He's near the top, but he only has greens in aggression, determination, teamwork, work rate, acceleration, natural fitness, pace, stamina.
I start with ruling out teamwork, cause I know from testing before that that's useless.
Long story short, moving everything up and down except stamina and work rate were a no go. Stamina ended up being an indispensable part of the puzzle. And it worked for other players too. So this is how I ended up at such a high figure for stamina.
And as I said, I tried to revisit and lower this later, because I wanted the weights to more closely match HarvestGreen's data, but couldn't find a way of doing it.
Panneton0 said: Do you force a fixed starting XI so no rotation is possible? That might explain the abnormaly high Stamina required to fit the season result. I'd be interested to see how this specific attribute ends up being even more important than pace/acc. I feel like it would have this importance for a club with a poor backup, but not as much if the backup is still decent. Anyway, interesting! Expand Good question
I anticipate this by having 4 of the same player as the position I'm testing. The rest of the team also has extras for each position.
So it's not that fatigue is leading to stamina being overpowered. I was scratching my head over stamina until I thought about it, realizing that stamina is actually compensating for the 2 key attributes of pace and acc, not just one, so it makes sense in that way that it's so strongly weighted.
I did try and get it to work with say 25 stamina. Couldn't get it working. Other attributes, I found you could use 14 or 50 for it and it'd work either way, just shifting the other weights around. I think composure was one such attribute.
I actually paid some close attention to the game time the 4 players were getting, because obviously I wanted them to get adequate games (for match fitness & reliable scoring) without being overworked. I also didn't want a situation where injury leads to being a player short. I've got the precise data, but generally speaking all of the two most played players each got ~28-35 Premier League games/season. That doesn't include cup games and whatnot, which totaled all combined to 41-50 games/season for the assistant's most picked player of the four.
I did not force XI selection, just let the assistant select who he thought was best each game.
OpticFawn said: the same weightings for each position? Expand Yes
I was hesitant to release this before testing all positions, but these combined team results gave me enough confidence to believe it should be the same or about the same for all positions.
I did do another DR and a DC
82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position
compare to:
83% | Giovanni Di Lorenzo (DR) - 3rd, 3rd, 2nd, 9th, 2nd, 5th, 5th = 4.143 position
82% | Daizen Maeda - 2nd, 3rd, 4th, 4th, 2nd, 3rd, 3rd = 3 position 81% | Kyogo Furuhashi - 4th, 2nd, 3rd, 5th, 3rd, 2nd, 2nd = 3 position 82% | Vinicius Junior - 5th, 4th, 1st, 3rd = 3.25 position 83% | Harry Kane - 5th, 2nd, 3rd, 3rd, 1st, 7th = 3.5 position 83% | Lautaro Martinez - 7th, 3rd, 1st, 5th, 2nd, 5th, 2nd, 5th = 3.75 position 81% | Victor Osimhen - 3rd, 3rd, 7th, 3rd, 3rd, 4th, 3rd, 4th = 3.833 position 81% | Viktor Gyokeres - 6th, 4th, 3rd, 4th, 5th, 5th, 2nd, 5th, 2nd = 4 position
Both DR and DC appear a bit ineffectual compared to ST, but it's not off by much.
I had two options: Release what I have based on the above results, or test each position (so at least 100 samples) which would probably take a week or two. I could also end up getting burn out by that point. So I decided to just release it now, since I've got some stuff to do these next few days anyway. Testing of other positions will continue to be done. If it turns out it doesn't work for other positions properly, it'll be adjusted.
The weights for GK in that thread are completely wrong. For instance, Aerial Reach is 1.8 while Reflexes is 20. In fact Aerial Reach is the most important attribute while reflexes is the least important of the big three, and less important than has always been believed. I didn't comment on this in that thread, because the OP does denote 'For goalkeepers, read the top two or three only.'
HarvestGreen's GK data is also wrong. The problem both these people are having is low sample size leading to invalid results.
Here are the ideal minimum weights I derived from brute force testing:
Of those, Aerial Reach is the least flexible in terms how low you can go and get away with it.
Traits do not seem to effect performance at all for any player.
I can't remember if height itself plays any role, but jumping reach in indeed important, so not just Aerial Reach, and jumping reach is semi-determined by height I think (for starting database players it is, and I reckon they probably coded it for newgens too).
One attribute that matters for GK that I think pretty much everybody would be unaware of, and isn't shown on that image, is technique. Technique actually matters quite a lot for GK. It's useless for outfield players, but important for GK.
- Designed to align with the objective results of real players within the English Premier League - GK not yet done, weights from Premier League 1.0 are used which are decent - Do not read too much into the specific values for each attribute, I used HarvestGreen's and my own data as a basis to start with, but it's largely just plugging in numbers until I get the output I'm looking for, which is alignment with the rankings. Individual attribute weights will be inaccurate or even superfluous, but it's what they sum up to that matters. - GS file is a bit less accurate than the FMST26 weights. Highly recommend FMST26 over GS if possible.
I could keep working on this for forever and a day, but I feel like it's good enough right now for a first version.
You can make your own adjustments, add in non-linear formulas, etc. to line up with the rankings better if you like, since the ranking data is really what's solid.
bf3metro said: and btw, may i ask if you still refer to your ratings, or did you changed/tweaked them?
Pace: 6, Acceleration: 6, JumpingReach: 5.2, Dribbling: 5.2, Balance: 4.8, Concentration: 5.2, Anticipation: 5.2, Determination: 5.6, Agility: 4, Stamina: 5.2, Composure: 4.4, WorkRate: 4.4, Aggression: 4.4, ImportantMatches: 5.2, NaturalFitness: 4.8, Consistency: 5.2, Pressure: 5.2, Professionalism: 5.2, Expand Yes, those are the most recent. It's simply my findings for the balance of attributes that achieves 5th in the Premier League at minimal cost/requirements.
But now I have a bunch of objective player rankings to work with, so I'm currently working a set of weights that lines up with this objective ranking list. And I'm close to done on the first go at this.
bf3metro said: what's better in your opinion? FMST26 or FMSS? Expand I haven't used either enough to form an opinion on it. I'll be using FMST26 simply because it supports FM24, but FMSuperScout doesn't.
Note that an edited Man City was left in. All four teams played in the same league together. GKs were controlled players, as I haven't done GK yet.
The best players in each position are ~85-90% for reference.
I'm actually surprised with how consistent the results turned out, as when I was testing Sacha Boey (5th best DR, 80.2%) I got first 2 results of 7th and 12th. Perhaps it's that some positions contribute more than others.
This is also giving an idea now what % difference 1 position difference is. ~1% = 1 position difference.
88% = 1st?, Premier League 80% = 8th, Premier League 74% = 14th, Premier League 70% = 18th?, relegated, Premier League
Indeed for DR, at 71.1% I see a player from Bournemouth who finished 18th in the season I'm looking at.
I think I'll be testing a few players in the DC position, and if they align neatly if the current weights, I'll be posting them soon after.
The meta team players are 71.8%. Based on the ST % values, we would expect the meta team to finish ~6.2, which is almost correct (~5th average from prior testing). It makes Knap's tactic worth 11%..
Panneton0 said: Just FYI, I checked the FM Scouting Tool26 and I realize I was doing the exact thing by modifying the code within FMSuperScout (being open-source paves the way to fully customizable score). Expand Yes, it would be good for more people to come up with their own weights for the rankings, as there's so many possibilities it could be right now.
I'm using a couple of formula weights, but I'm leaving the non-linear adjustments until after I have some confidence all outfield positions are the same and get a set of weights out.
OpticFawn said: I actually appreciate your tests fr, look forward to your files as this as a good baseline to apply to the upcoming game imo Expand It's always nice to have one's work appreciated, but I carry on regardless.
As I've said before, being beholden to praise or criticism paralyzes and retards the creator. We see that with the state of FM itself. They have no balls, and likely no genuine interest in the game either.
Someone said in this thread earlier 'I liked X file, but now you've gone off the rails with Y file!'. The deeper one gets, the less likely others are able to understand or sympathize.
I know I'm upsetting a fair few people by constantly coming up with new conclusions that contradict or confound the previous ones at times, not least because they may take it as painting them a fool for believing them once, but this is the process genuine progress necessitates. It's keithb who is always right from the get-go, not me.
Rain said: I understand what you're saying, but you also can't assume your player will be playing the same teams every time there will be different players playing depending on who the managers decide to play anyway. Expand You are saying either one of two things: That my method is unreliable because the opposition teams are in flux, or otherwise that deducing weightings more precisely is pointless because in reality the opposition teams are always in flux.
Look, my view is that I don't need teams to be frozen, because I actually think having teams in flux replicates the realism better. I don't need to be able to work out precisely how good a particular player is. If I can just improve the reasonable certainty from +/- 15% to +/- 8%, then that's what worth doing in my opinion. Of course, data from controlled testing is often useful, but it's not the be all or end all.
Rain said: Also we know what is important so why are you still trying to get this to an exact science 3 years later? Expand Because it is more enjoyable than playing the game itself. The difficulty makes it an enjoyable challenge, and it's one of the few things I know of that hasn't been done to death by a million people.
You say 'we know what is important', but it has become apparent that the assertion pace/acc is everything isn't quite true anymore. I'm still discovering new things that surprise me. And judging by quite a few of the recent replies in this thread, many don't have a clue what's important in the game and what's not.
Now you could say 'well, we've worked out the game well enough to win everything with ease', and that would have been true already back in 2019 - before EBFM, HarvestGreen, FM Arena, etc. - because we had Knap tactics. If other people want to persist with the game vanilla or whatnot, good for them - the problem I have with it is when people tell me what to do with my time based on that attitude.
Yarema said: And that was my whole point. That other attributes aren't completely useless and shouldn't be valued at 0. Expand They are, as demonstrated by the Messi vs Furuhashi example I showed you in detail, but you have seemingly chosen to overlook.
I have almost completed another example which is even better (Luis Suarez vs Le Fonde). Suarez has green in almost everything technical/mental, but has similar low physicals to Le Fonde. Suarez is finishing consistently 7th to 10th, making him the worst player tested so far.
I don't know why you want to die on this hill. I was talking about sta, pace, acc which are attributes that matter. A majority of attributes do not matter at all, and should be valued at 0, even when other attributes are identical, due to the CA cost.
Rain said: If you really want to test strikers and rank them should probably just give all the teams in the league including your own the exact same attributes besides the player you are testing. That way you will have consistent data and confirm your player is always playing in the same circumstances. Expand HarvestGreen did that and got results that don't align with the real rankings. You could say my rankings aren't real because the rest of my team is artificial, but it would be closer to the reality than HarvestGreen's because all of the opposition is real.
The reason for the discrepancy between HarvestGreen's data and the rankings in reality, is that he's testing against opposition that is either all 10 or 20. If I put an ST with 12 jump reach in a league where every player has 1 jump reach, guess how many aerial attempts he's going to win. But each Premier League team in reality has players with at least ~15 jump reach.
Nonetheless I'm currently plugging in HarvestGreen's 10 > 18 data into FMST26 as weights to see how it fares compared to the weights I ended up with so far, because the more accurate the starting set of weights, the less likely we get a set of weights that 'functions' properly most of the time but isn't actually using the right weights that will get us to near 100% accuracy. One problem that appears with HarvestGreen's weights is that Maeda and Furuhashi are underrated.
What I'm doing at the moment is using HarvestGreen's 10 > 18 & 6 > 18 data as a baseline, then changing just the minimum of what needs to be changed to make things line up.
One example of a radical change, and what I think is going on:
In my weightings attempt before this, I found stamina ~90 (a bit higher than acc/pace) is necessary. HarvestGreen says stamina 10 > 18 is 21.6% of acc 10 > 18. I'm saying it's ~110%.
From my own testing, I know the following is ideal: 20 acc/pace 13 sta | 18 acc/pace 14 sta | 14 acc/pace 15 sta. Something like that anyway, you get the picture. So we see sta is 1/3 of acc, or it's higher at 1 to 1 or a bit more.
Acc & Pace are 2 attributes it's conserving, so sta @ 1/3 acc is actually 2/3 of acc or pace alone. So we know it's in a range of ~67%-200%. Hence I think why something like 110% sta weighting of acc works and HarvestGreen's 21.6% doesn't.
But why did HarvestGreen get 21.6%? Well in his test his players have 10 in every attribute except the one tested, so he's doing 10 acc/pace 18 sta vs 10 acc/pace/sta. That translates to roughly ~9.7 acc/pace vs. 8.3 acc/pace. That's +17% for the 18 sta team.
I've just started having a look at this tool called FMST 26, but it looks like the solution to my problems.
In spite of the name, it works for both FM24 and FM26.
It reads direct from memory, so we've got access to all the hiddens now for FM24, and it was instant to load for me too.
It can measure and sort by match ratings and goals.
And the cherry on top is that not only does it allow for a custom set of weightings, but it seems like it's possible to do more sophisticated mathematical and logical operations with the weights too. If I'm correct in that, then that opens up the possibility of accounting for non-linear attributes and attribute interactions further down the track.
It does lack a few things Genie Scout has or had (player attribute compare, team overall % comparison), but it has the things I need most.
I've had no choice but to continue working with Genie Scout until now, because it's the only tool that can weight the hiddens for FM24. It doesn't calculate things properly, but I decided to just do the best job I could do with it.
So now I can see what my weightings are actually ranking the players accurately, and I see that Maeda is finally 3rd after Haaland and Mbappe as he should be, which is a good sign. I now have to readjust the weightings slightly, but now I can be certain they're being calculated accurately.
I suppose I'll furnish an example, to demonstrate how dirtiness doesn't just solve Maeda, it hits a number of birds with one stone.
If you look at Lautaro Martinez, it's very hard to explain why he finishes 3.75. He has lower pace/acc than Mbappe, lower key mentals than Lewandowski, but if you get familiar with all the pros and cons, the conclusion is that he should be better.
But it's the comparison between Martinez and Maeda that really makes this clear:
Looking at this you've got to ask, how in the hell does Maeda do better than Martinez. It's one reason I've already taken 8 samples on Martinez. If you emphasize pace/acc to an extreme level, that drops the bomb on Kane & Lewandowski in particular. But with the dirtiness factor, it suddenly works out - not just for Maeda vs Martinez, but also Martinez vs Mbappe, Lewandowski, etc.
Still a fair while to go on getting a robust set of weightings, but I wanted to share something I've discovered so far.
Dirtiness seems to be far more important than I ever gave it credit for, and injury proneness isn't far off either.
How do I conclude this?
1) High weight of dirtiness is the only thing I can see that allows Maeda to as highly ranked as his position. More highly valuing any other attribute he has (of which there are few he has high), such as work rate or determination, would mess up the rank of other players. Maeda has very low dirtiness of '2'.
2) High weight of dirtiness also just so happens to fix up a lot of the other ranking problems. So, more circumstantial evidence it's real.
3) I compare not just the rankings, but the relative %, and the only one that was particularly off was Dybala. So I looked into Dybala and noticed the unique thing he does have is high injury proneness. I had not added injury proneness as a weight so far, so I added it, and it largely fixed up Dybala's %. Then I looked at my test saves of Dybala to compare.. indeed what I found that when Dybala finished 3rd he had minimal injuries, whereas when he finished 12, there were a bunch of moderate and major injuries. I added 4 of each player to the team, so I didn't think this would become an issue, but it still does. That's not a problem for the validity at all in my opinion, because in reality you're not going to have more than 4 first team STs anyway - if anything you'd have less of that caliber, which only emphasizes injury proneness more if anything.
4) I already reasoned long ago that dirtiness is worse than injury proneness, because with dirtiness you get a red card and lose a player, whereas with injury proneness you get an injury but the player gets immediately replaced with one that is only ~10% worse. On the other hand, injury proneness also leads to extended periods of low match fitness, which has a strong effect on performance, and so injury proneness is more serious than just the in-match effect.
All that said, it could just be that dirtiness and/or injury proneness are just acting as 'functional' weightings at the moment, that could be disproved later by certain other players. I'd say the probability of this is only say ~20% though, mainly because it seems impossible to explain Maeda doing so well without it.
Yarema said: Now add 40 CA to them in nonessential attributes and sim again. If possible a fair assessment, not all CA into the absolute worst attributes you can think of.
Also the not engineered distribution is sort of true but not really. Using the meta schedules to boost physicals to levels that were obviously not intended is kind of engineering them, although they do happen "naturally" as opposed to straight up editing them. Expand Here you go lad
Not exactly the same in what I would consider the meaningful attributes, but hey, what two players in reality are? And I think it's fair to say that one would assume Messi's +10 dribbling would offset Furuhashi's +8 work rate.
In a team of controlled players but also in a real league, simulated for multiple seasons on full detail.
Training was left as default, no meta training or quickness focus. Positions were also pre-edited to be the same: Only 20 in ST.
Hiddens are close to the same, Messi perhaps has the edge, but judge for yourself:
I think you've got the picture wrong.
As I referred to in my post, the OP of that thread said 'For goalkeepers, read the top two or three only.'
I didn't do the full quote, which follows with:
'Reflexes and Agility separate cleanly; below that it is noise, and Handling ranking under Corners is not a finding.
One keeper per match against ten outfielders means that column has about a tenth of the statistical power.'
I'm not saying his other findings are 'completely wrong' - only the GK weights.
I'm using my own data to refute the data he got on GKs, that's the sound basis for me saying 'it is all wrong'. And it is up to me to decide which of the thesis are more trustworthy.
BTW I'll tell you how I got to such a high value for stamina, and the other values in general, since I didn't really explain.
I start with the existing weights, changed a few weights until the players roughly aligned properly, then I carefully compared players who would remain stubbornly out of alignment (fixing one, ruins another).
Maeda was key to this process. He's near the top, but he only has greens in aggression, determination, teamwork, work rate, acceleration, natural fitness, pace, stamina.
I start with ruling out teamwork, cause I know from testing before that that's useless.
Long story short, moving everything up and down except stamina and work rate were a no go. Stamina ended up being an indispensable part of the puzzle. And it worked for other players too. So this is how I ended up at such a high figure for stamina.
And as I said, I tried to revisit and lower this later, because I wanted the weights to more closely match HarvestGreen's data, but couldn't find a way of doing it.
Good question
I anticipate this by having 4 of the same player as the position I'm testing. The rest of the team also has extras for each position.
So it's not that fatigue is leading to stamina being overpowered. I was scratching my head over stamina until I thought about it, realizing that stamina is actually compensating for the 2 key attributes of pace and acc, not just one, so it makes sense in that way that it's so strongly weighted.
I did try and get it to work with say 25 stamina. Couldn't get it working. Other attributes, I found you could use 14 or 50 for it and it'd work either way, just shifting the other weights around. I think composure was one such attribute.
I actually paid some close attention to the game time the 4 players were getting, because obviously I wanted them to get adequate games (for match fitness & reliable scoring) without being overworked. I also didn't want a situation where injury leads to being a player short. I've got the precise data, but generally speaking all of the two most played players each got ~28-35 Premier League games/season. That doesn't include cup games and whatnot, which totaled all combined to 41-50 games/season for the assistant's most picked player of the four.
I did not force XI selection, just let the assistant select who he thought was best each game.
Yes
I was hesitant to release this before testing all positions, but these combined team results gave me enough confidence to believe it should be the same or about the same for all positions.
I did do another DR and a DC
82% | Kim Min Jae (DC) - 3rd, 3rd, 6th, 6th, 3rd, 4th, 5th = 4.286 position
compare to:
83% | Giovanni Di Lorenzo (DR) - 3rd, 3rd, 2nd, 9th, 2nd, 5th, 5th = 4.143 position
82% | Daizen Maeda - 2nd, 3rd, 4th, 4th, 2nd, 3rd, 3rd = 3 position
81% | Kyogo Furuhashi - 4th, 2nd, 3rd, 5th, 3rd, 2nd, 2nd = 3 position
82% | Vinicius Junior - 5th, 4th, 1st, 3rd = 3.25 position
83% | Harry Kane - 5th, 2nd, 3rd, 3rd, 1st, 7th = 3.5 position
83% | Lautaro Martinez - 7th, 3rd, 1st, 5th, 2nd, 5th, 2nd, 5th = 3.75 position
81% | Victor Osimhen - 3rd, 3rd, 7th, 3rd, 3rd, 4th, 3rd, 4th = 3.833 position
81% | Viktor Gyokeres - 6th, 4th, 3rd, 4th, 5th, 5th, 2nd, 5th, 2nd = 4 position
Both DR and DC appear a bit ineffectual compared to ST, but it's not off by much.
I had two options: Release what I have based on the above results, or test each position (so at least 100 samples) which would probably take a week or two. I could also end up getting burn out by that point. So I decided to just release it now, since I've got some stuff to do these next few days anyway. Testing of other positions will continue to be done. If it turns out it doesn't work for other positions properly, it'll be adjusted.
HarvestGreen's GK data is also wrong. The problem both these people are having is low sample size leading to invalid results.
Here are the ideal minimum weights I derived from brute force testing:
Of those, Aerial Reach is the least flexible in terms how low you can go and get away with it.
Traits do not seem to effect performance at all for any player.
I can't remember if height itself plays any role, but jumping reach in indeed important, so not just Aerial Reach, and jumping reach is semi-determined by height I think (for starting database players it is, and I reckon they probably coded it for newgens too).
One attribute that matters for GK that I think pretty much everybody would be unaware of, and isn't shown on that image, is technique. Technique actually matters quite a lot for GK. It's useless for outfield players, but important for GK.
Use FMST26 (works for FM24 & FM26) and copy paste this into statistics > custom metrics (advanced):
(acceleration * 86.8 + pace * 86.2 + jumping_reach * 46.2 + pressure * 16.7 + dribbling * 14.6 + work_rate * 63.0 + agility * 17.0 + anticipation * 58.2 + composure * 25.6 + stamina * 93.9 + consistency * 14.1 + determination * 30.0 + balance * 13.5 + strength * 1.0 + concentration * 47.3 + finishing * 2.0 + important_matches * 10.2 + natural_fitness * 7.9 + professionalism * 15.0 + ambition * 2.0 + loyalty * 1.0 + aggression * 6.3 + vision * 0.5 + left_foot * 6 + right_foot * 5 - (injury_proneness * (50 / (natural_fitness * 0.85))) - (dirtiness * 32 * (aggression * 0.1))) / 118
Genie Scout file:
https://files.catbox.moe/z7hsqg.grf
Notes:
- Designed to align with the objective results of real players within the English Premier League
- GK not yet done, weights from Premier League 1.0 are used which are decent
- Do not read too much into the specific values for each attribute, I used HarvestGreen's and my own data as a basis to start with, but it's largely just plugging in numbers until I get the output I'm looking for, which is alignment with the rankings. Individual attribute weights will be inaccurate or even superfluous, but it's what they sum up to that matters.
- GS file is a bit less accurate than the FMST26 weights. Highly recommend FMST26 over GS if possible.
I could keep working on this for forever and a day, but I feel like it's good enough right now for a first version.
You can make your own adjustments, add in non-linear formulas, etc. to line up with the rankings better if you like, since the ranking data is really what's solid.
Pace: 6, Acceleration: 6, JumpingReach: 5.2, Dribbling: 5.2, Balance: 4.8, Concentration: 5.2, Anticipation: 5.2, Determination: 5.6, Agility: 4, Stamina: 5.2, Composure: 4.4, WorkRate: 4.4, Aggression: 4.4, ImportantMatches: 5.2, NaturalFitness: 4.8, Consistency: 5.2, Pressure: 5.2, Professionalism: 5.2,
Yes, those are the most recent. It's simply my findings for the balance of attributes that achieves 5th in the Premier League at minimal cost/requirements.
But now I have a bunch of objective player rankings to work with, so I'm currently working a set of weights that lines up with this objective ranking list. And I'm close to done on the first go at this.
I haven't used either enough to form an opinion on it. I'll be using FMST26 simply because it supports FM24, but FMSuperScout doesn't.
Results:
80% | Sheffield Utd - 9th, 9th, 7th, 6th, 7th = 7.6 position
78% | Luton - 12th, 7th, 11th, 11th, 12th = 10.6 position
76% | Burnley - 20th, 11th, 12th, 12th, 14th = 13.8 position
74% | Brentford - 10th, 17th, 16th, 14th, 15th = 14.4 position
Note that an edited Man City was left in. All four teams played in the same league together. GKs were controlled players, as I haven't done GK yet.
The best players in each position are ~85-90% for reference.
I'm actually surprised with how consistent the results turned out, as when I was testing Sacha Boey (5th best DR, 80.2%) I got first 2 results of 7th and 12th. Perhaps it's that some positions contribute more than others.
This is also giving an idea now what % difference 1 position difference is. ~1% = 1 position difference.
88% = 1st?, Premier League
80% = 8th, Premier League
74% = 14th, Premier League
70% = 18th?, relegated, Premier League
Indeed for DR, at 71.1% I see a player from Bournemouth who finished 18th in the season I'm looking at.
I think I'll be testing a few players in the DC position, and if they align neatly if the current weights, I'll be posting them soon after.
The meta team players are 71.8%. Based on the ST % values, we would expect the meta team to finish ~6.2, which is almost correct (~5th average from prior testing). It makes Knap's tactic worth 11%..
ST
90% | Erling Haaland - 1st, 1st, 4th, 1st, 2nd, 1st = 1.666 position
87% | Kylian Mbappe - 2nd, 3rd, 3rd, 3rd, 5th, 2nd, 1st, 1st = 2.5 position
87% | Mohamed Salah* - 1st, 3rd, 5th, 3rd, 3rd, 2nd, 1st = 2.571 position
84% | Heung-Min Son - 3rd, 2nd, 2nd, 2nd, 2nd, 6th, 4th = 3 position
82% | Daizen Maeda - 2nd, 3rd, 4th, 4th, 2nd, 3rd, 3rd = 3 position
81% | Kyogo Furuhashi - 4th, 2nd, 3rd, 5th, 3rd, 2nd, 2nd = 3 position
82% | Vinicius Junior - 5th, 4th, 1st, 3rd = 3.25 position
83% | Harry Kane - 5th, 2nd, 3rd, 3rd, 1st, 7th = 3.5 position
84% | Robert Lewandowski - 2nd, 2nd, 2nd, 3rd, 5th, 2nd, 8th, 5th, 3rd = 3.555 position
79% | Dusan Vlahovic - 1st, 4th, 4th, 5th, 1st, 6th, 4th, 4th = 3.625
83% | Lautaro Martinez - 7th, 3rd, 1st, 5th, 2nd, 5th, 2nd, 5th = 3.75 position
81% | Victor Osimhen - 3rd, 3rd, 7th, 3rd, 3rd, 4th, 3rd, 4th = 3.833 position
78% | Lionel Messi - 4th, 2nd, 1st, 5th, 9th, 3rd = 4 position
81% | Viktor Gyokeres - 6th, 4th, 3rd, 4th, 5th, 5th, 2nd, 5th, 2nd = 4 position
74% | Donyell Malen - 2nd, 2nd, 7th, 5th, 5th, 7th = 4.666 position
73% | Robert Glatzel - 5th, 7th, 6th, 3rd, 12th, 6th, 2nd, 6th, 8th = 6.111 position
69% | Paulo Dybala - 5th, 5th, 6th, 10th, 3rd, 12th, 6th, 5th, 6th, 6th = 6.4 position
64% | Adam Le Fondre - 8th, 11th (sacked), 5th, 7th (sacked), 6th (sacked), 8th (sacked) = 7.5 position
63% | Luis Suarez - 7th, 9th, 10th, 8th, 8th = 8.4 position
DR
83% | Giovanni Di Lorenzo - 3rd, 3rd, 2nd, 9th, 2nd, 5th, 5th = 4.143 position
66% | Kingsley Schindler - 4th, 11th, 6th, 11th = 8 position
It's worth mentioning again that all players were set to 20 ST and nothing else. This will affect a few players slightly.
* I forgot to duplicate Salah first before moving him, so there was no Salah for the opposition. Could redo, but shouldn't make much difference.
Yes, it would be good for more people to come up with their own weights for the rankings, as there's so many possibilities it could be right now.
I'm using a couple of formula weights, but I'm leaving the non-linear adjustments until after I have some confidence all outfield positions are the same and get a set of weights out.
It's always nice to have one's work appreciated, but I carry on regardless.
As I've said before, being beholden to praise or criticism paralyzes and retards the creator. We see that with the state of FM itself. They have no balls, and likely no genuine interest in the game either.
Someone said in this thread earlier 'I liked X file, but now you've gone off the rails with Y file!'. The deeper one gets, the less likely others are able to understand or sympathize.
I know I'm upsetting a fair few people by constantly coming up with new conclusions that contradict or confound the previous ones at times, not least because they may take it as painting them a fool for believing them once, but this is the process genuine progress necessitates. It's keithb who is always right from the get-go, not me.
You are saying either one of two things: That my method is unreliable because the opposition teams are in flux, or otherwise that deducing weightings more precisely is pointless because in reality the opposition teams are always in flux.
Look, my view is that I don't need teams to be frozen, because I actually think having teams in flux replicates the realism better. I don't need to be able to work out precisely how good a particular player is. If I can just improve the reasonable certainty from +/- 15% to +/- 8%, then that's what worth doing in my opinion. Of course, data from controlled testing is often useful, but it's not the be all or end all.
Rain said: Also we know what is important so why are you still trying to get this to an exact science 3 years later?
Because it is more enjoyable than playing the game itself. The difficulty makes it an enjoyable challenge, and it's one of the few things I know of that hasn't been done to death by a million people.
You say 'we know what is important', but it has become apparent that the assertion pace/acc is everything isn't quite true anymore. I'm still discovering new things that surprise me. And judging by quite a few of the recent replies in this thread, many don't have a clue what's important in the game and what's not.
Now you could say 'well, we've worked out the game well enough to win everything with ease', and that would have been true already back in 2019 - before EBFM, HarvestGreen, FM Arena, etc. - because we had Knap tactics. If other people want to persist with the game vanilla or whatnot, good for them - the problem I have with it is when people tell me what to do with my time based on that attitude.
Adam Le Fondre - 8th, 11th (sacked), 5th, 7th (sacked), 6th (sacked), 8th (sacked) = 7.5 position
Luis Suarez - 7th, 9th, 10th, 8th, 8th = 8.4 position
Note that Suarez isn't (significantly) superior in the key mentals of anticipation, concentration, work rate.
The argument that useless technicals/mentals should be counted even a little is in the toilet.
They are, as demonstrated by the Messi vs Furuhashi example I showed you in detail, but you have seemingly chosen to overlook.
I have almost completed another example which is even better (Luis Suarez vs Le Fonde). Suarez has green in almost everything technical/mental, but has similar low physicals to Le Fonde. Suarez is finishing consistently 7th to 10th, making him the worst player tested so far.
I don't know why you want to die on this hill. I was talking about sta, pace, acc which are attributes that matter. A majority of attributes do not matter at all, and should be valued at 0, even when other attributes are identical, due to the CA cost.
HarvestGreen did that and got results that don't align with the real rankings. You could say my rankings aren't real because the rest of my team is artificial, but it would be closer to the reality than HarvestGreen's because all of the opposition is real.
The reason for the discrepancy between HarvestGreen's data and the rankings in reality, is that he's testing against opposition that is either all 10 or 20. If I put an ST with 12 jump reach in a league where every player has 1 jump reach, guess how many aerial attempts he's going to win. But each Premier League team in reality has players with at least ~15 jump reach.
Nonetheless I'm currently plugging in HarvestGreen's 10 > 18 data into FMST26 as weights to see how it fares compared to the weights I ended up with so far, because the more accurate the starting set of weights, the less likely we get a set of weights that 'functions' properly most of the time but isn't actually using the right weights that will get us to near 100% accuracy. One problem that appears with HarvestGreen's weights is that Maeda and Furuhashi are underrated.
What I'm doing at the moment is using HarvestGreen's 10 > 18 & 6 > 18 data as a baseline, then changing just the minimum of what needs to be changed to make things line up.
One example of a radical change, and what I think is going on:
In my weightings attempt before this, I found stamina ~90 (a bit higher than acc/pace) is necessary.
HarvestGreen says stamina 10 > 18 is 21.6% of acc 10 > 18. I'm saying it's ~110%.
From my own testing, I know the following is ideal: 20 acc/pace 13 sta | 18 acc/pace 14 sta | 14 acc/pace 15 sta. Something like that anyway, you get the picture. So we see sta is 1/3 of acc, or it's higher at 1 to 1 or a bit more.
Acc & Pace are 2 attributes it's conserving, so sta @ 1/3 acc is actually 2/3 of acc or pace alone. So we know it's in a range of ~67%-200%. Hence I think why something like 110% sta weighting of acc works and HarvestGreen's 21.6% doesn't.
But why did HarvestGreen get 21.6%? Well in his test his players have 10 in every attribute except the one tested, so he's doing 10 acc/pace 18 sta vs 10 acc/pace/sta. That translates to roughly ~9.7 acc/pace vs. 8.3 acc/pace. That's +17% for the 18 sta team.
In spite of the name, it works for both FM24 and FM26.
It reads direct from memory, so we've got access to all the hiddens now for FM24, and it was instant to load for me too.
It can measure and sort by match ratings and goals.
And the cherry on top is that not only does it allow for a custom set of weightings, but it seems like it's possible to do more sophisticated mathematical and logical operations with the weights too. If I'm correct in that, then that opens up the possibility of accounting for non-linear attributes and attribute interactions further down the track.
It does lack a few things Genie Scout has or had (player attribute compare, team overall % comparison), but it has the things I need most.
I've had no choice but to continue working with Genie Scout until now, because it's the only tool that can weight the hiddens for FM24. It doesn't calculate things properly, but I decided to just do the best job I could do with it.
So now I can see what my weightings are actually ranking the players accurately, and I see that Maeda is finally 3rd after Haaland and Mbappe as he should be, which is a good sign. I now have to readjust the weightings slightly, but now I can be certain they're being calculated accurately.
If you look at Lautaro Martinez, it's very hard to explain why he finishes 3.75. He has lower pace/acc than Mbappe, lower key mentals than Lewandowski, but if you get familiar with all the pros and cons, the conclusion is that he should be better.
But it's the comparison between Martinez and Maeda that really makes this clear:
Looking at this you've got to ask, how in the hell does Maeda do better than Martinez. It's one reason I've already taken 8 samples on Martinez. If you emphasize pace/acc to an extreme level, that drops the bomb on Kane & Lewandowski in particular. But with the dirtiness factor, it suddenly works out - not just for Maeda vs Martinez, but also Martinez vs Mbappe, Lewandowski, etc.
Dirtiness seems to be far more important than I ever gave it credit for, and injury proneness isn't far off either.
How do I conclude this?
1) High weight of dirtiness is the only thing I can see that allows Maeda to as highly ranked as his position. More highly valuing any other attribute he has (of which there are few he has high), such as work rate or determination, would mess up the rank of other players. Maeda has very low dirtiness of '2'.
2) High weight of dirtiness also just so happens to fix up a lot of the other ranking problems. So, more circumstantial evidence it's real.
3) I compare not just the rankings, but the relative %, and the only one that was particularly off was Dybala. So I looked into Dybala and noticed the unique thing he does have is high injury proneness. I had not added injury proneness as a weight so far, so I added it, and it largely fixed up Dybala's %. Then I looked at my test saves of Dybala to compare.. indeed what I found that when Dybala finished 3rd he had minimal injuries, whereas when he finished 12, there were a bunch of moderate and major injuries. I added 4 of each player to the team, so I didn't think this would become an issue, but it still does. That's not a problem for the validity at all in my opinion, because in reality you're not going to have more than 4 first team STs anyway - if anything you'd have less of that caliber, which only emphasizes injury proneness more if anything.
4) I already reasoned long ago that dirtiness is worse than injury proneness, because with dirtiness you get a red card and lose a player, whereas with injury proneness you get an injury but the player gets immediately replaced with one that is only ~10% worse. On the other hand, injury proneness also leads to extended periods of low match fitness, which has a strong effect on performance, and so injury proneness is more serious than just the in-match effect.
All that said, it could just be that dirtiness and/or injury proneness are just acting as 'functional' weightings at the moment, that could be disproved later by certain other players. I'd say the probability of this is only say ~20% though, mainly because it seems impossible to explain Maeda doing so well without it.
Also the not engineered distribution is sort of true but not really. Using the meta schedules to boost physicals to levels that were obviously not intended is kind of engineering them, although they do happen "naturally" as opposed to straight up editing them.
Here you go lad
Kyogo Furuhashi - 4th, 2nd, 3rd, 5th, 3rd, 2nd, 2nd = 3 position
Lionel Messi - 4th, 2nd, 1st, 5th, 9th, 3rd = 4 position
Furuhashi vs Messi - 136 CA vs 185 CA
Not exactly the same in what I would consider the meaningful attributes, but hey, what two players in reality are? And I think it's fair to say that one would assume Messi's +10 dribbling would offset Furuhashi's +8 work rate.
In a team of controlled players but also in a real league, simulated for multiple seasons on full detail.
Training was left as default, no meta training or quickness focus. Positions were also pre-edited to be the same: Only 20 in ST.
Hiddens are close to the same, Messi perhaps has the edge, but judge for yourself: