First of all, for those who may not be aware, RNG is an acronym for Random Number Generator. RNG is something that produces randomness, variability, or luck in digital games.
Here's the test result of "4132 Extreme Diamond Undervolt II" tactic - The 1st Test 5,760 matches played / PTS = 61.9 / G.D. = +16.1 / FOR = 81.3 / AG = 65.2
Here's the result of re-test - RE-TEST 5,760 matches played / PTS = 59.5 / G.D. = +13.0 / FOR = 80.4 / AG = 67.4
Some facts if we take a single test ( 320 matches played ):
The highest PTS = 65.7 The lowest PTS = 54.7 The highest goals scored = 85.1 The lowest goals scored = 73.3 The highest goals conceded = 72.0 The lowest goals conceded = 62.3
If we take all the 36 tests:
Points range from 54.7 to 65.7 Goals Scored range from 73.3 to 85.1 Goals Conceded range from 62.3 to 72.0
So as you can see if we tested tactics only for 320 matches ( 1 Test ) then depending on the RNG(luck) "4132 Extreme Diamond Undervolt II" tactic might hit 54.7 points or 65.7 points.
Please note, in our tactic testing league:
- every player gets 100% boost of Conditions and Morale before each match - all the teams use the same starting eleven for each match, no rotation happens - no injures happen - no transfers happen - no manager moves happen ... and there are many other features that reduce the RNG
Now, just imagine what RNG you have in normal game when the factors above aren't eliminated and they add an additional RNG.
Of course, in normal game it's possible to get very consistent results with almost no RNG but to achive that your team must be much stronger comparing to your opponents, something like PSG in French Legue 1 you'll be winning almost 100% of the matches with any tactic so no RNG.
CBP87 said: Sorry @Zippo slightly confused, so are you saying that even playing 5760 matches that RNG swing is greater than initially thought (+1 either way) Expand
One can go high, the other low, then that makes two points of difference.
ZaZ said: One can go high, the other low, then that makes two points of difference. Expand
Correct me if I'm wrong but in this case the initial result was 62, so surely it should only drop to 61? I understand your point that there can be a potential swing of 2 either, e.g if the lowest RNG was got then say 61 was the return but the next test it hit the highest RNG and got a 63 but that is still within the range of the initial RNG range
CBP87 said: Sorry @Zippo slightly confused, so are you saying that even playing 5760 matches that RNG swing is greater than initially thought (+1 either way) Expand
Let's assume that the true score of some tactic is 60 pts.
You test it for 5,760 matches and get 61 pts.
Then you re-rest it for 5,760 matches again and get 59 pts.
So you can conclude the RNG swing of a 5,760 matches test can be as high as 1 point ( -/+ 1 point away from the true score).
Btw, our RNG measure was for FM22 and it seems for FM23 the RNG has slight increased so it could be 1.5 points for a 5,760 matches test. Expand
Have you thought about using something like a Monte Carlo sampling on the tests? Basically, take smaller samples of the 5760 matches repeated times, then make statistics based on the statistic of those samples (for example, taking the medians of those subsamples, and calculating the median of them).
It's not hard to implement, and it can increase the precision of tests without increasing the computational effort used. There are also other types of subsampling, including bootstrapping and other techniques.
ZaZ said: Have you thought about using something like a Monte Carlo sampling on the tests? Basically, take smaller samples of the 5760 matches repeated times, then make statistics based on the statistic of those samples (for example, taking the medians of those subsamples, and calculating the median of them).
It's not hard to implement, and it can increase the precision of tests without increasing the computational effort used. There are also other types of subsampling, including bootstrapping and other techniques. Expand
I don't think anything of that can be easily applied on our automated testing. Also, anyway there won't be enough subsamples when most of time we do 2-6 tests.
Mostly important, there's no need for such accuracy. Look at this test - https://fm-arena.com/thread/3293-what-it-takes-to-dominate-epl-with-bournemouth/ so to dominate the league and have consistent results your team must be the strongest ( or close to that ) in the league and when it happens then the difference between a 61 points tactic and a 43 points tactic reduces to 7 points and if you look at the test it happens when the team is far from being the stronger team in the league, so when finally your team is the strongest team in the league then there won't be any difference between a 61 points tactic and a 55 points tactic.
I mean even -/+ 4 points accuracy is totally acceptable for FM.
Also, it's funny that the test #1 of this tactic also hit very high 68.2 points
So if we tested tactics only 1 run ( 320 matches ) or 2 runs ( 640 matches ) then "424 Extreme Chainsaw" tactic would be sitting the top with 69 points!
But after 18 tests ( 5,760 Matches ) the average points per 38 matches dropped from 69 to 59 points.
That's all what you need to know about the FM RNG...
Hey there,
First of all, for those who may not be aware, RNG is an acronym for Random Number Generator. RNG is something that produces randomness, variability, or luck in digital games.
Here's the test result of "4132 Extreme Diamond Undervolt II" tactic - The 1st Test
5,760 matches played / PTS = 61.9 / G.D. = +16.1 / FOR = 81.3 / AG = 65.2
Here's the result of re-test - RE-TEST
5,760 matches played / PTS = 59.5 / G.D. = +13.0 / FOR = 80.4 / AG = 67.4
Some facts if we take a single test ( 320 matches played ):
The highest PTS = 65.7
The lowest PTS = 54.7
The highest goals scored = 85.1
The lowest goals scored = 73.3
The highest goals conceded = 72.0
The lowest goals conceded = 62.3
If we take all the 36 tests:
Points range from 54.7 to 65.7
Goals Scored range from 73.3 to 85.1
Goals Conceded range from 62.3 to 72.0
So as you can see if we tested tactics only for 320 matches ( 1 Test ) then depending on the RNG(luck) "4132 Extreme Diamond Undervolt II" tactic might hit 54.7 points or 65.7 points.
Please note, in our tactic testing league:
- every player gets 100% boost of Conditions and Morale before each match
- all the teams use the same starting eleven for each match, no rotation happens
- no injures happen
- no transfers happen
- no manager moves happen
... and there are many other features that reduce the RNG
Now, just imagine what RNG you have in normal game when the factors above aren't eliminated and they add an additional RNG.
Of course, in normal game it's possible to get very consistent results with almost no RNG but to achive that your team must be much stronger comparing to your opponents, something like PSG in French Legue 1 you'll be winning almost 100% of the matches with any tactic so no RNG.
RNG, you are so CRUEL!
Sorry @Zippo slightly confused, so are you saying that even playing 5760 matches that RNG swing is greater than initially thought (+1 either way)
CBP87 said: Sorry @Zippo slightly confused, so are you saying that even playing 5760 matches that RNG swing is greater than initially thought (+1 either way)
One can go high, the other low, then that makes two points of difference.
ZaZ said: One can go high, the other low, then that makes two points of difference.
Correct me if I'm wrong but in this case the initial result was 62, so surely it should only drop to 61? I understand your point that there can be a potential swing of 2 either, e.g if the lowest RNG was got then say 61 was the return but the next test it hit the highest RNG and got a 63 but that is still within the range of the initial RNG range
CBP87 said: Sorry @Zippo slightly confused, so are you saying that even playing 5760 matches that RNG swing is greater than initially thought (+1 either way)
As @ZaZ said.
Let's assume that the true score of some tactic is 60 pts.
You test it for 5,760 matches and get 61 pts.
Then you re-rest it for 5,760 matches again and get 59 pts.
So you can conclude the RNG swing of a 5,760 matches test can be as high as 1 point ( -/+ 1 point away from the true score).
Btw, our RNG measure was for FM22 and it seems for FM23 the RNG has slight increased so it could be 1.5 points for a 5,760 matches test.
Zippo said: As @ZaZ said.
Let's assume that the true score of some tactic is 60 pts.
You test it for 5,760 matches and get 61 pts.
Then you re-rest it for 5,760 matches again and get 59 pts.
So you can conclude the RNG swing of a 5,760 matches test can be as high as 1 point ( -/+ 1 point away from the true score).
Btw, our RNG measure was for FM22 and it seems for FM23 the RNG has slight increased so it could be 1.5 points for a 5,760 matches test.
Have you thought about using something like a Monte Carlo sampling on the tests? Basically, take smaller samples of the 5760 matches repeated times, then make statistics based on the statistic of those samples (for example, taking the medians of those subsamples, and calculating the median of them).
It's not hard to implement, and it can increase the precision of tests without increasing the computational effort used. There are also other types of subsampling, including bootstrapping and other techniques.
ZaZ said: Have you thought about using something like a Monte Carlo sampling on the tests? Basically, take smaller samples of the 5760 matches repeated times, then make statistics based on the statistic of those samples (for example, taking the medians of those subsamples, and calculating the median of them).
It's not hard to implement, and it can increase the precision of tests without increasing the computational effort used. There are also other types of subsampling, including bootstrapping and other techniques.
I don't think anything of that can be easily applied on our automated testing. Also, anyway there won't be enough subsamples when most of time we do 2-6 tests.
Mostly important, there's no need for such accuracy. Look at this test - https://fm-arena.com/thread/3293-what-it-takes-to-dominate-epl-with-bournemouth/ so to dominate the league and have consistent results your team must be the strongest ( or close to that ) in the league and when it happens then the difference between a 61 points tactic and a 43 points tactic reduces to 7 points and if you look at the test it happens when the team is far from being the stronger team in the league, so when finally your team is the strongest team in the league then there won't be any difference between a 61 points tactic and a 55 points tactic.
I mean even -/+ 4 points accuracy is totally acceptable for FM.
Here's a funny fact.
The highest ever score of a single test was achieved by "424 Extreme Chainsaw" tactic - https://fm-arena.com/tactic/4025-4132-extreme-diamond-undervolt-ii/
The Test #2 hit crazy 69,4 points
Also, it's funny that the test #1 of this tactic also hit very high 68.2 points
So if we tested tactics only 1 run ( 320 matches ) or 2 runs ( 640 matches ) then "424 Extreme Chainsaw" tactic would be sitting the top with 69 points!
But after 18 tests ( 5,760 Matches ) the average points per 38 matches dropped from 69 to 59 points.
That's all what you need to know about the FM RNG...
Zippo said: Here's a funny fact.
The highest ever score of a single test was achieved by 424 Extreme Chainsaw tactic - https://fm-arena.com/tactic/4025-4132-extreme-diamond-undervolt-ii/
The Test #2 hit crazy 69,4 points
Also, it's funny that the test #1 of this tactic also hit a very high 68.2 points
So if we tested tactics only 1 run ( 320 matches ) or 2 runs ( 640 matches ) then "424 Extreme Chainsaw" would be sitting the top with 69 points!
But after 18 runs ( 5,760 Matches ) the average points per 38 matches dropped from 69 to 59 points.
That's all what you need to know about the FM RNG...
yep, I've noticed really giant margin between runs for some tactics
didn't see chainsaw results run by run tho
opq said: yep, I've noticed really giant margin between runs for some tactics
didn't see chainsaw results run by run tho
Yeah, that's the FM RNG... untamed and wild!
If you look at the result of our RNG measurement test - https://fm-arena.com/thread/2713-10-944-matches-tested-fm-rng-measured/ then you'll notice that if you test a tactic only for 38 matches then the RNG might be as high as crazy 26 points ( away from the true score ) !