Given there seems to be a bit of interest I thought I owed it to you all to give an update. I have finished the 5 runs of the base testing and I will finish the 5 runs of Passion4FM training schedule tonight. I have downloaded all the data I am interested in from the base testing, and there is an awful lot of it. I have build the structure for the analysis tool and have put the original, pre test, data in. It is more time consuming than I had hoped but I think the worst of it is behind me.
After running the base testing a couple of things were thrown up. Firstly, I think I can use Man City as they did keep the selections, they just changed tactics and selected their best players for games. The U18s were the ones I nominated. Also, they are using whatever training I put in.
The second thing is more of a problem. Chelsea and Spurs seem to be reverting back to when I first did the squads so they are not using the players I want in the correct squads. It is consistent though so will be the same scenario for each tactic. It just means we have less players in the grouping of EPL players and a new group who play both EPL and U18s right through the season.
Another minor hiccup has been the nomenclature between the Genie Scout downloads the the ones directly out of the game. But I am only using the ones out of the game now to ensure nothing much changes between each run and have stopped trying to read it into the analysis.
These oddities mean we have a few more groupings now. Overall we have 176 players we are looking at. Each test will cover around 40 games for EPL/cups and for U18s. So players will be playing 30-40 games a test, with 5 tests per training schedule or 150-200 games to analyze each schedule. We will have them all training for 10 months per test run or around 43 weeks each test, 215 weeks each training schedule.
Of the 176 players, 11 will be playing both U18 and EPL games, so mostly over 50 games per season, 16 will be playing the full EPL season of around 40 games, 66 will be playing the full U18 season with a bit of bench time and the odd run in the EPL, and 61 will be doing all the training in the main squad and only get the odd bench spot and maybe replace an injured player once or twice a season. The remaining 22 are the Man City players who are using a different tactic, and are split equally into the last two groups and will probably be included in those groups unless their data is dramatically different.
I should have the first bit of analysis done in the next day or so and the tests will just keep running regardless.
I have hit a few problems so changing tack slightly. Most of the clubs seem to override my selections, with only Chelsea, Spurs and Newcastle adhering to the players preferred. Manchester City throw everything out so I will not waste my time with them.
The approach now is to reduce the main squad to 22 players with the most potential to improve. Select the first eleven and make the second eleven available for the U18 team. I will still control the training of all these players. All other players will be in the U23 squad. I am hoping the second eleven will at least get game time albeit at a much lower level. For the 3 sides that seem to abide by the rules I will have a comparison between levels of playing experience on development. For the other 4 sides I would expect to have the second eleven still playing at U18 level, so a full 7 teams with comparable experience. With the first eleven for non complying 4 sides I doubt they will get much playing time so any improvement will be all down to training.
So we will have 3 interesting groupings to analyze. I will start running the control 5 runs now, leaving all the training to the AM. I will just recheck all the selections etc prior to the first full run.
Out of interest the player with the most potential in the Newcastle squad played all the games in the trial and improved from a current ability of 55.2% (potential ability 77.4%) to current ability of 64.4% in one season. The whole first eleven made pretty impressive amounts of improvement.
Nikolas said: Hello Mark and ZaZ! Congrats on the great work. You are providing invaluble analysis and insight to the game.
Regarding this whole conversation about important attributes, i've been using excel to mimic the work of genie scout, because i dont want to see the players CA, PA and hidden stats. I just want to compare my own players and find the best of them for each position and the best position for each one of them. Of course, feel free to suggest me a different way, with hidden CA and PA.
For now I export player attributes from fm print screen option, and then paste them in an specificly made excel file, and then i have the positional ratings for each on of them nice and fast. Few seconds for the whole team for every postion.
The question I have is regarding weight for each position. How do you actually calulate that?
In my mind my assumption is that a player with, lets say 10 in each attribute, should have an equal score in every position. And then, with the purpose to have numbers in the range of 0-100, i make the appropiate divisons to balance the numbers.
I.e. lets take DC position with Chinese version and zaz -25 for under 50 modifications. The total sum of the weights is 995. To make it 100 in need to devide by 9.95. So 9.95 is the balance number for this position. For FST is 7.85 etc.
Then I take players attributes and multiply them with the approriate weights for each position. Then take the sum of the result and divide it with the weight number for the position. In case of DC, i divide by 9.95 and so on. Then i divide by 20, so i can have a range of 0-100 in the result.
And this way i get the positional rating.
As a result a player with 10 in each attribute, would get 50% for all positions. A player with 20 in every attribute would get 100% for all positions.
Is this the correct way to do this? Expand
That is an interesting question. I think our Chinese colleagues gave us the answer to this very question. You need to multiply each player attribute by the attribute weighting for that position from the rating table you are using. Then total of all these calculations and divide by the total value of all the weightings for that position to give you the weighted average value of attributes. Multiply the weighted average value of attributes by 20 and subtract 121. This will give you the Tactical True Current Ability for that position for that player and allow you to compare them with other players of the same position.
You do need to remember the analysis done by FM Arena on position ability though. If a player is less than accomplished at the position you shouldn't be using them in that position.
I have downloaded the Genie Scout data from the base save and selected all the squads. Each team will have 28 players, all from their U18 and U23 squads. All squad players were selected by their Ability Difference, that is the difference between Potential Ability and Current Ability, with the player having the highest gap getting the first position in the squad. I take the best side I can get as the run on eleven and then pick a reserve for each position but only a single DC and STC. This makes up 9 reserves. I am not sure how the AM will select players outside my first 11 but I have added 3 reserves, selecting a further GK, DC and STC. And then there are 5 additional players. I am also hoping to include Man City this time.
My theory is that the AM will choose from the reserves next but time will tell if this works. I have set all the managers up as unsackable. I am now importing all training schedules that I will be testing. I have added RDF and Hook on top of Passion4FM, FC Cadoni and ZaZ, plus of course mine. It should take about a day per 5 tests for each of the training schedules which will make it a week roughly. Probably do it quicker but that's what I am allowing.
If anyone has thoughts on this let me know. I am hoping to finish the set up this evening and start the testing tomorrow afternoon.
ZaZ said: Remember to run default training for reference too! It would also be nice to see how much the training affects results (if possible), since some training schedules are more focused in match bonuses. Expand
I will definitely run with the AM training first as a base.
CBP87 said: In your initial assessment, was the AM in charge of general and individual training? Expand
Yes, I didn't touch training in the previous runs as I thought the AM would change anything I did anyhow.
The intent with the 7 sides is to make the managers unsackable, and then drop their main players to reserves. I will bring in their best 22 best potential players by position. Lock in play the selected players if possible, and see how they develop over a season. The best development is the winner.
The second part will be to test the individual focus for the best training tactics. Once again it will be based on improvement.
Please let me know if you think I should adjust the methodology.
Today I ran a test to see if the Assistant Manager would change your training set up on holiday tests. Turns out they don't. They don't even change the individual position or focus training. So I am now keen to test training set up ups and their impact on player development. Our recent advice from our friends from China on their observations using machine learning have fueled this.
Firstly I will use the same league I used for the Genie Scout rating test. I will use the same 7 teams. Testing will involve a base test, and then use 4 training schedules to ascertain the best two. I have selected ZaZ, FC Cadoni, Passion FC and a hybrid that I use - lets call it MDW. If you have another training schedule that is working really well please let me know and I will consider including it. After selecting the best two I am going to test individual focus using my old approach and the one suggested by the machine learning guys.
It might take me a while to get my act together but hang in there and keep your eyes on this for the results. It might be a game changer.
You have 4 strikers in your side and Huanca would be your 5th. I have run the ratings against all 5 strikers using each of the ratings files. I would be tempted to buy him. He doesn't rate well in my MDW ratings file but goes well in the Y50 and even the ykykyk files.
I think from the analysis I have, done that the machine learning works. I personally will be using the full ykykyk ratings file as I think it will be more stable longer term. I will now balance that ratings file and derive the unrated positions as best I can.
@ZaZ prefers the chopped down versions because you get better value players. I prefer to try and get players you can keep as you progress. And I certainly move between the approaches and value what ZaZ does immensely.
My focus beyond changing my ratings file is to upgrade my approach to individual training focus. There were some notes in the machine learning example relating to that. I am exploring how I improve on that aspect in my games. At some stage I will provide my learnings on that front.
OpticFawn said: Hey, @Mark and @ZaZ This is weird, this player according to genie under the zaz 25 under 50 rating has the guy rated better than all of my strikers but looks the worst? what do you guys think I'm seeing this on other players.
José Huanca rating = 73.1
Dennis = 69.65
Junior Arias = 71.18
As you can see Jose is the highest by a fair bit and looks the worst out of the 3. Expand
I am sorry mate but I cant replicate the issue. I have done the manual calculations and also set 3 identical players up in my test save and get different results and orders to what you have.
All I can think of is it hasn't loaded properly or it has somehow changed.
If you could get a copy of your save linked I would happily investigate further if you want. But as it stands I really don't know what is going on.
In the balanced version I added positions not covered by the original and then made the top rated player from each position roughly even which allows you to find players across positions. This is done by adjusting the overall weighting for the position and doesn't impact ratings from within the position itself because all the attribute weightings remain the same.
For example the top rated GK was previously 62.4 on the save I have, and the top rated Fast Striker was 87.5. All the other positions were somewhere in between. Now the top rated player for each position is around the 91 or 92 mark.
ZaZ said: Thank you for the results! I'm a researcher from the field of optimization, which is closely related to machine learning, so I have no doubts their experiment holds lots of value. I'll try to boost the goalkeeper scores of my best version and add to my tactic page, as well as linking here so they can also see your filters. Looking forward for your next improvements or any suggestions.
P.S.: Just to be clear, Y50 is the one that subtracts 25 from all under 50 using chinese as base, right? Expand
Mark said: I am now running more thorough testing to see which GS Ratings work best. I have taken the ZaZ EPL Test League and downloaded the ratings for EPL players for each ratings filter including the GS default. I set up coaches for the top 9 teams and set all tactics to ZaZ Blue DM. I will do at least 3 runs and then look at the predictions for each position versus the end player ratings for each position.
It should be interesting to see performance against this data set. Expand
So my work here is now complete. There were some issues. There were 2 of the managers that didn't work because I hadn't taken control of them, That left us with 7 sides we have control of. One of course was Man City whose Manager was sacked every test as they couldn't compete against the ZaZ Blue tactics.
The methodology was to take the best rated XI for each ratings and compare the average ratings for each side. I ran 5 tests and then dropped any player that had less than 50 games over the 5 runs or 190 games. I only selected players rated best from the 7 teams we were working with who all used ZaZ Blue DM tactics.
So they were playing the same role for the majority of the time and played substantial games in the role.
I have to say the results very much favor the Chinese machine learning research we were all interested in.
I tested 6 ratings files plus the default Genie Scout rating file. The main ratings files were ZaZ, my MDW, ykykyk the one based on the Chinese machine learning research, and 3 variants of that proposed by @ZaZ.
I used either end of season team of the year for the squad or the highest positional rating to determine the position for each player. Players with less than 50 games of the 5 tests were excluded. Only players in the 7 controlled teams were included, Arsenal, Chelsea, Leicester, Liverpool, Man Utd, Newcastle, Spurs.
First we have the control test which is the default Genie Scout ratings. Here is their side:
So of the main 3 ratings files, MDW, ykykyk and ZaZ here are the results:
And here are the 3 test tactics from ZaZ using the Chinese machine learning technology.
The Analysis from our Chinese friends has been very impressive,
Petrades said: Sorry for the Dutch, Seuntjes is both 7 on pace and acc Expand
I therefore assume jumping reach is the one that is 16 and natural fitness moves as well. If that is the case then the Chinese ratings and my ratings would both have the first one in front.
What ratings are you using and can you translate the attributes so I can check the calculations
ZaZ said: I would never get that SS with 7 acceleration and pace. Lower league is full of those low speed players, which is why it's so easy to win there. Expand
I cant translate this language but I thought from the placement of the numbers that the first player has 14 Acc and 9 Pace and the second has 7 Acc and 16 Pace. If that is right then it is more understandable the the ratings are close. I make it just favoring the second player. If the attributes don't align with the English version then I understand.
After running the base testing a couple of things were thrown up. Firstly, I think I can use Man City as they did keep the selections, they just changed tactics and selected their best players for games. The U18s were the ones I nominated. Also, they are using whatever training I put in.
The second thing is more of a problem. Chelsea and Spurs seem to be reverting back to when I first did the squads so they are not using the players I want in the correct squads. It is consistent though so will be the same scenario for each tactic. It just means we have less players in the grouping of EPL players and a new group who play both EPL and U18s right through the season.
Another minor hiccup has been the nomenclature between the Genie Scout downloads the the ones directly out of the game. But I am only using the ones out of the game now to ensure nothing much changes between each run and have stopped trying to read it into the analysis.
These oddities mean we have a few more groupings now. Overall we have 176 players we are looking at. Each test will cover around 40 games for EPL/cups and for U18s. So players will be playing 30-40 games a test, with 5 tests per training schedule or 150-200 games to analyze each schedule. We will have them all training for 10 months per test run or around 43 weeks each test, 215 weeks each training schedule.
Of the 176 players, 11 will be playing both U18 and EPL games, so mostly over 50 games per season, 16 will be playing the full EPL season of around 40 games, 66 will be playing the full U18 season with a bit of bench time and the odd run in the EPL, and 61 will be doing all the training in the main squad and only get the odd bench spot and maybe replace an injured player once or twice a season. The remaining 22 are the Man City players who are using a different tactic, and are split equally into the last two groups and will probably be included in those groups unless their data is dramatically different.
I should have the first bit of analysis done in the next day or so and the tests will just keep running regardless.
The approach now is to reduce the main squad to 22 players with the most potential to improve. Select the first eleven and make the second eleven available for the U18 team. I will still control the training of all these players. All other players will be in the U23 squad. I am hoping the second eleven will at least get game time albeit at a much lower level. For the 3 sides that seem to abide by the rules I will have a comparison between levels of playing experience on development. For the other 4 sides I would expect to have the second eleven still playing at U18 level, so a full 7 teams with comparable experience. With the first eleven for non complying 4 sides I doubt they will get much playing time so any improvement will be all down to training.
So we will have 3 interesting groupings to analyze. I will start running the control 5 runs now, leaving all the training to the AM. I will just recheck all the selections etc prior to the first full run.
Out of interest the player with the most potential in the Newcastle squad played all the games in the trial and improved from a current ability of 55.2% (potential ability 77.4%) to current ability of 64.4% in one season. The whole first eleven made pretty impressive amounts of improvement.
Here are the training schedules I will be using:
Passion4FM
FMS
Hoods
ZaZ
MDW
RDF
FC Cadoni
Regarding this whole conversation about important attributes, i've been using excel to mimic the work of genie scout, because i dont want to see the players CA, PA and hidden stats. I just want to compare my own players and find the best of them for each position and the best position for each one of them. Of course, feel free to suggest me a different way, with hidden CA and PA.
For now I export player attributes from fm print screen option, and then paste them in an specificly made excel file, and then i have the positional ratings for each on of them nice and fast. Few seconds for the whole team for every postion.
The question I have is regarding weight for each position. How do you actually calulate that?
In my mind my assumption is that a player with, lets say 10 in each attribute, should have an equal score in every position. And then, with the purpose to have numbers in the range of 0-100, i make the appropiate divisons to balance the numbers.
I.e. lets take DC position with Chinese version and zaz -25 for under 50 modifications. The total sum of the weights is 995. To make it 100 in need to devide by 9.95. So 9.95 is the balance number for this position. For FST is 7.85 etc.
Then I take players attributes and multiply them with the approriate weights for each position. Then take the sum of the result and divide it with the weight number for the position. In case of DC, i divide by 9.95 and so on. Then i divide by 20, so i can have a range of 0-100 in the result.
And this way i get the positional rating.
As a result a player with 10 in each attribute, would get 50% for all positions.
A player with 20 in every attribute would get 100% for all positions.
Is this the correct way to do this?
That is an interesting question. I think our Chinese colleagues gave us the answer to this very question. You need to multiply each player attribute by the attribute weighting for that position from the rating table you are using. Then total of all these calculations and divide by the total value of all the weightings for that position to give you the weighted average value of attributes. Multiply the weighted average value of attributes by 20 and subtract 121. This will give you the Tactical True Current Ability for that position for that player and allow you to compare them with other players of the same position.
You do need to remember the analysis done by FM Arena on position ability though. If a player is less than accomplished at the position you shouldn't be using them in that position.
Sure I will get to that in the next 24 hours.
My theory is that the AM will choose from the reserves next but time will tell if this works. I have set all the managers up as unsackable. I am now importing all training schedules that I will be testing. I have added RDF and Hook on top of Passion4FM, FC Cadoni and ZaZ, plus of course mine. It should take about a day per 5 tests for each of the training schedules which will make it a week roughly. Probably do it quicker but that's what I am allowing.
If anyone has thoughts on this let me know. I am hoping to finish the set up this evening and start the testing tomorrow afternoon.
I will definitely run with the AM training first as a base.
CBP87 said: In your initial assessment, was the AM in charge of general and individual training?
Yes, I didn't touch training in the previous runs as I thought the AM would change anything I did anyhow.
I definitely am. And thanks for asking.
The second part will be to test the individual focus for the best training tactics. Once again it will be based on improvement.
Please let me know if you think I should adjust the methodology.
Firstly I will use the same league I used for the Genie Scout rating test. I will use the same 7 teams. Testing will involve a base test, and then use 4 training schedules to ascertain the best two. I have selected ZaZ, FC Cadoni, Passion FC and a hybrid that I use - lets call it MDW. If you have another training schedule that is working really well please let me know and I will consider including it. After selecting the best two I am going to test individual focus using my old approach and the one suggested by the machine learning guys.
It might take me a while to get my act together but hang in there and keep your eyes on this for the results. It might be a game changer.
Will do
You have 4 strikers in your side and Huanca would be your 5th. I have run the ratings against all 5 strikers using each of the ratings files. I would be tempted to buy him. He doesn't rate well in my MDW ratings file but goes well in the Y50 and even the ykykyk files.
I think from the analysis I have, done that the machine learning works. I personally will be using the full ykykyk ratings file as I think it will be more stable longer term. I will now balance that ratings file and derive the unrated positions as best I can.
@ZaZ prefers the chopped down versions because you get better value players. I prefer to try and get players you can keep as you progress. And I certainly move between the approaches and value what ZaZ does immensely.
My focus beyond changing my ratings file is to upgrade my approach to individual training focus. There were some notes in the machine learning example relating to that. I am exploring how I improve on that aspect in my games. At some stage I will provide my learnings on that front.
José Huanca rating = 73.1
Dennis = 69.65
Junior Arias = 71.18
As you can see Jose is the highest by a fair bit and looks the worst out of the 3.
I am sorry mate but I cant replicate the issue. I have done the manual calculations and also set 3 identical players up in my test save and get different results and orders to what you have.
All I can think of is it hasn't loaded properly or it has somehow changed.
If you could get a copy of your save linked I would happily investigate further if you want. But as it stands I really don't know what is going on.
For example the top rated GK was previously 62.4 on the save I have, and the top rated Fast Striker was 87.5. All the other positions were somewhere in between. Now the top rated player for each position is around the 91 or 92 mark.
I hope this makes sense.
Hope this helps someone
ZaZ 25 under 50 Balanced
P.S.: Just to be clear, Y50 is the one that subtracts 25 from all under 50 using chinese as base, right?
That is correct ZaZ.
It should be interesting to see performance against this data set.
So my work here is now complete. There were some issues. There were 2 of the managers that didn't work because I hadn't taken control of them, That left us with 7 sides we have control of. One of course was Man City whose Manager was sacked every test as they couldn't compete against the ZaZ Blue tactics.
The methodology was to take the best rated XI for each ratings and compare the average ratings for each side. I ran 5 tests and then dropped any player that had less than 50 games over the 5 runs or 190 games. I only selected players rated best from the 7 teams we were working with who all used ZaZ Blue DM tactics.
So they were playing the same role for the majority of the time and played substantial games in the role.
I have to say the results very much favor the Chinese machine learning research we were all interested in.
I tested 6 ratings files plus the default Genie Scout rating file. The main ratings files were ZaZ, my MDW, ykykyk the one based on the Chinese machine learning research, and 3 variants of that proposed by @ZaZ.
I used either end of season team of the year for the squad or the highest positional rating to determine the position for each player. Players with less than 50 games of the 5 tests were excluded. Only players in the 7 controlled teams were included, Arsenal, Chelsea, Leicester, Liverpool, Man Utd, Newcastle, Spurs.
First we have the control test which is the default Genie Scout ratings. Here is their side:
So of the main 3 ratings files, MDW, ykykyk and ZaZ here are the results:
And here are the 3 test tactics from ZaZ using the Chinese machine learning technology.
The Analysis from our Chinese friends has been very impressive,
I therefore assume jumping reach is the one that is 16 and natural fitness moves as well. If that is the case then the Chinese ratings and my ratings would both have the first one in front.
What ratings are you using and can you translate the attributes so I can check the calculations
I cant translate this language but I thought from the placement of the numbers that the first player has 14 Acc and 9 Pace and the second has 7 Acc and 16 Pace. If that is right then it is more understandable the the ratings are close. I make it just favoring the second player. If the attributes don't align with the English version then I understand.
MDW Part Time Training
if you you are full time you will need to research more to find ZaZ's