mmigueis said: It seems pretty conclusive, IMO! (Note that in comparing the "technique = 20" team to the "technique = 1" teams, you could average across all the "technique = 1" teams, as they are set up exactly the same).
Have you ever done similar tests for other attributes? It would be very interesting to see the results. Expand
This is true. For the mean/std metrics I could've averaged everyone that isn't the Team 1. I'll keep this in mind for other analysis. But I think it's good to have everyone separated for the other tests.
I've done similar testing for Decisions. While I didn't have time to generate all graphs for it (yet), the results seem similar regarding position (this means, it won the league just as much as Technique), however the correlation between GD, Won and Pts is a bit lower (0.1 lower). Either this is just a result of a small dataset, or it might suggest that while Decisions do matter positively, it does have lower influence in this scenario.
svonn said: Looking at the initial tests from @harvestgreen22, it is clear that not all attributes have a linear effect. Attributes like work rate seem to have a "saturation point", so anything above 10 has diminishing returns. There may be something similar at play here - testing with 1 technique might have a high effect, 10 might be some kind of sweet spot, while the drop above might be due to something like statistical noise or some other threshold that allows actions that aren't helpful, especially if the whole team is doing them. Expand
For sure, I think it's clear that the influence of the attributes varies per attribute and the combination of attributes.
That's one of the reasons why I think it's important to test for correlation instead of pure GD, and why I think it's useful to test extremes(1 vs 20). Extremes show us that the stats do have some influence, and that it's positive. Correlation shows us how much should we expect every attribute to matter relation to every stat in specific scenarios.
harvestgreen22 said: Thanks for your work, I'm also slowly trying a new initial condition, Four teams use replicants, different 4-3-3, then test what the goal difference is without changing the attributes, then test what the goal difference is with -10 Technique 1,500 games(match) tested now. I expect to increase it to about 5000. Expand
Thanks mate, I'll be keeping an eye out in your tests! I wish I had that patience to do 5000 matches, hahaha.
Also, sorry for stealing your post. I'll probably make a separate ones for my analysis.
As before, I did 10 seasons (this means 300 matches). The configuration is the same as the other comment.
Databattle 1 has ALL attributes on 20. The other teams have ALL attributes on 20, EXCEPT Technique which is on 1.
Hypothesis: IF Technique doesn't matter, what we expect is to that results of the league will be random or close to random. IF Technique is NEGATIVE, what we expect is that Databattle 1 will be WORSE than the other teams overall.
Results: Databattle 1 overperformed all other teams in most seasons, having better GD, Position, winrate and more points most of the time. It won the league 7 times (7/10), and has higher mean points, GD, wins, and position. See images for more info (On the correlation matrix, the "Technique 20" data point is simply 1 if the team is Databattle 1, and 0 if it isn't. Also, remember that POSITION is "inversed", so having lower position is actually better). Even in the seasons it did not win, it came second by AT MOST 4 points.
Conclusion: When everything else is at 20, is seems to be better to have technique at 20 than it is to have it at 1. Eventually I'll add more data to this analysis, as I feel 10 tournaments/300 matches is a small amount, but for now I'll test other attributes in the same manner.
harvestgreen22 said: Thank you. If I have time I'll keep expanding the sample to see if it's random, if it's due to certain conditions, or if it's true Expand
I feel there is somepositive correlation using technique, but haven't been able to point exactly what is it.
I've updated my testing database. So 4 teams, all playing a 4-3-3 without any instructions (FB (Su), CD (De), DM (De), BBM (Su), CM (Su), W (Su), AF (At)). Each team plays 30 matches. Injuries are disabled. Attributes are frozen, and testing is done on holiday.
Teams 2, 3, 4 have technique on 1 for all line players, and all other attributes on 20. Team 1 has everything at 20.
The GK for all teams have everything at 10.
I'm still running the tests but right now the difference seems too big for technique to be negative or to not matter.
I did some tests with flair but didn't really notice a negative correlation. 4 team league, teams B, C, D have all players with 20 in all normal attributes. Team A players have everything on 20 except flair, which is on 1. All attributes are frozen (this means that the attributes stay on the same level all season). All players are proficient in all positions. The formation used was a 4-1-1-3-1, for all teams (FB (De), CD (De), DM (Su), CM (Su), W (At), AM (Su), AF (At)).
I ran 10 seasons (the first season 10 times, to be precise). Team A won the league twice, got third a couple of times and second a couple too. The goal difference wasn't really relevant either: 19, 15, -6,-1, -16, -10, -2, 15, -6, 12.
Didn't test combinations or other attributes, though.
Of course, 10 seasons amounts to a very small number of matches in a league with 4 teams (300 matches per team), so more testing is surely required.
Have you ever done similar tests for other attributes? It would be very interesting to see the results.
This is true. For the mean/std metrics I could've averaged everyone that isn't the Team 1. I'll keep this in mind for other analysis. But I think it's good to have everyone separated for the other tests.
I've done similar testing for Decisions. While I didn't have time to generate all graphs for it (yet), the results seem similar regarding position (this means, it won the league just as much as Technique), however the correlation between GD, Won and Pts is a bit lower (0.1 lower). Either this is just a result of a small dataset, or it might suggest that while Decisions do matter positively, it does have lower influence in this scenario.
svonn said: Looking at the initial tests from @harvestgreen22, it is clear that not all attributes have a linear effect. Attributes like work rate seem to have a "saturation point", so anything above 10 has diminishing returns. There may be something similar at play here - testing with 1 technique might have a high effect, 10 might be some kind of sweet spot, while the drop above might be due to something like statistical noise or some other threshold that allows actions that aren't helpful, especially if the whole team is doing them.
For sure, I think it's clear that the influence of the attributes varies per attribute and the combination of attributes.
That's one of the reasons why I think it's important to test for correlation instead of pure GD, and why I think it's useful to test extremes(1 vs 20). Extremes show us that the stats do have some influence, and that it's positive. Correlation shows us how much should we expect every attribute to matter relation to every stat in specific scenarios.
harvestgreen22 said: Thanks for your work, I'm also slowly trying a new initial condition,
Four teams use replicants, different 4-3-3,
then test what the goal difference is without changing the attributes,
then test what the goal difference is with -10 Technique
1,500 games(match) tested now. I expect to increase it to about 5000.
Thanks mate, I'll be keeping an eye out in your tests! I wish I had that patience to do 5000 matches, hahaha.
Also, sorry for stealing your post. I'll probably make a separate ones for my analysis.
As before, I did 10 seasons (this means 300 matches). The configuration is the same as the other comment.
Databattle 1 has ALL attributes on 20. The other teams have ALL attributes on 20, EXCEPT Technique which is on 1.
Hypothesis: IF Technique doesn't matter, what we expect is to that results of the league will be random or close to random. IF Technique is NEGATIVE, what we expect is that Databattle 1 will be WORSE than the other teams overall.
Results: Databattle 1 overperformed all other teams in most seasons, having better GD, Position, winrate and more points most of the time. It won the league 7 times (7/10), and has higher mean points, GD, wins, and position. See images for more info (On the correlation matrix, the "Technique 20" data point is simply 1 if the team is Databattle 1, and 0 if it isn't. Also, remember that POSITION is "inversed", so having lower position is actually better). Even in the seasons it did not win, it came second by AT MOST 4 points.
Conclusion: When everything else is at 20, is seems to be better to have technique at 20 than it is to have it at 1. Eventually I'll add more data to this analysis, as I feel 10 tournaments/300 matches is a small amount, but for now I'll test other attributes in the same manner.
I feel there is some positive correlation using technique, but haven't been able to point exactly what is it.
I've updated my testing database. So 4 teams, all playing a 4-3-3 without any instructions (FB (Su), CD (De), DM (De), BBM (Su), CM (Su), W (Su), AF (At)). Each team plays 30 matches. Injuries are disabled. Attributes are frozen, and testing is done on holiday.
Teams 2, 3, 4 have technique on 1 for all line players, and all other attributes on 20.
Team 1 has everything at 20.
The GK for all teams have everything at 10.
I'm still running the tests but right now the difference seems too big for technique to be negative or to not matter.
4 team league, teams B, C, D have all players with 20 in all normal attributes. Team A players have everything on 20 except flair, which is on 1. All attributes are frozen (this means that the attributes stay on the same level all season). All players are proficient in all positions. The formation used was a 4-1-1-3-1, for all teams (FB (De), CD (De), DM (Su), CM (Su), W (At), AM (Su), AF (At)).
I ran 10 seasons (the first season 10 times, to be precise). Team A won the league twice, got third a couple of times and second a couple too. The goal difference wasn't really relevant either: 19, 15, -6,-1, -16, -10, -2, 15, -6, 12.
Didn't test combinations or other attributes, though.
Of course, 10 seasons amounts to a very small number of matches in a league with 4 teams (300 matches per team), so more testing is surely required.