opq said: yep, I've noticed really giant margin between runs for some tactics
didn't see chainsaw results run by run tho Expand

Yeah, that's the FM RNG... untamed and wild!

If you look at the result of our RNG measurement test - https://fm-arena.com/thread/2713-10-944-matches-tested-fm-rng-measured/ then you'll notice that if you test a tactic only for 38 matches then the RNG might be as high as crazy 26 points ( away from the true score ) !

Here's a funny fact.

The highest ever score of a single test was achieved by "424 Extreme Chainsaw" tactic - https://fm-arena.com/tactic/4025-4132-extreme-diamond-undervolt-ii/

The Test #2 hit crazy 69,4 points




Also, it's funny that the test #1 of this tactic also hit very high 68.2 points

So if we tested tactics only 1 run ( 320 matches ) or 2 runs ( 640 matches ) then "424 Extreme Chainsaw" tactic would be sitting the top with 69 points!

But after 18 tests ( 5,760 Matches ) the average points per 38 matches dropped from 69 to 59 points.

That's all what you need to know about the FM RNG...

ZaZ said: Have you thought about using something like a Monte Carlo sampling on the tests? Basically, take smaller samples of the 5760 matches repeated times, then make statistics based on the statistic of those samples (for example, taking the medians of those subsamples, and calculating the median of them).

It's not hard to implement, and it can increase the precision of tests without increasing the computational effort used. There are also other types of subsampling, including bootstrapping and other techniques. Expand

I don't think anything of that can be easily applied on our automated testing. Also, anyway there won't be enough subsamples when most of time we do 2-6 tests.

Mostly important, there's no need for such accuracy. Look at this test - https://fm-arena.com/thread/3293-what-it-takes-to-dominate-epl-with-bournemouth/ so to dominate the league and have consistent results your team must be the strongest ( or close to that ) in the league and when it happens then the difference between a 61 points tactic and a 43 points tactic reduces to 7 points and if you look at the test it happens when the team is far from being the stronger team in the league, so when finally your team is the strongest team in the league then there won't be any difference between a 61 points tactic and a 55 points tactic.

I mean even -/+ 4 points accuracy is totally acceptable for FM.

CBP87 said: Sorry @Zippo slightly confused, so are you saying that even playing 5760 matches that RNG swing is greater than initially thought (+1 either way) Expand

As @ZaZ said.

Let's assume that the true score of some tactic is 60 pts.

You test it for 5,760 matches and get 61 pts.

Then you re-rest it for 5,760 matches again and get 59 pts.

So you can conclude the RNG swing of a 5,760 matches test can be as high as 1 point ( -/+ 1 point away from the true score).

Btw, our RNG measure was for FM22 and it seems for FM23 the RNG has slight increased so it could be 1.5 points for a 5,760 matches test.

More Info - https://fm-arena.com/thread/3578-some-sad-facts-about-fm-rng/

Hey there,

First of all, for those who may not be aware, RNG is an acronym for Random Number Generator. RNG is something that produces randomness, variability, or luck in digital games.

Here's the test result of "4132 Extreme Diamond Undervolt II" tactic - The 1st Test
5,760 matches played / PTS = 61.9 / G.D. = +16.1 / FOR = 81.3 / AG = 65.2

Here's the result of re-test - RE-TEST
5,760 matches played / PTS = 59.5 / G.D. = +13.0 / FOR = 80.4 / AG = 67.4

Some facts if we take a single test ( 320 matches played ):

The highest PTS = 65.7
The lowest PTS = 54.7
The highest goals scored = 85.1
The lowest goals scored = 73.3
The highest goals conceded = 72.0
The lowest goals conceded = 62.3

If we take all the 36 tests:

Points range from 54.7 to 65.7
Goals Scored range from 73.3 to 85.1
Goals Conceded range from 62.3 to 72.0

So as you can see if we tested tactics only for 320 matches ( 1 Test ) then depending on the RNG(luck) "4132 Extreme Diamond Undervolt II" tactic might hit 54.7 points or 65.7 points.

Please note, in our tactic testing league:

- every player gets 100% boost of Conditions and Morale before each match
- all the teams use the same starting eleven for each match, no rotation happens
- no injures happen
- no transfers happen
- no manager moves happen
... and there are many other features that reduce the RNG

Now, just imagine what RNG you have in normal game when the factors above aren't eliminated and they add an additional RNG.

Of course, in normal game it's possible to get very consistent results with almost no RNG but to achive that your team must be much stronger comparing to your opponents, something like PSG in French Legue 1 you'll be winning almost 100% of the matches with any tactic so no RNG.  


RNG, you are so CRUEL!

@Delicious, look at this - https://fm-arena.com/find-comment/19243/

Sometime the data in FM might be misleading, for example, a corner setup might score a lot of goals that the game doesn't count as corner goals ( it could be after second balls or something like that )

Please note, neither am I saying that's the case, just a heads-up, to be careful with that data.

At the end the overall goals is what is important the most.

Sane said: @Zippo there are statistics, which cd scores more? running into the nearest one or standing there? In other words, where do you put the cd with the best heading? Expand

Hi,

You can check it yourself.

Near Post Corner - https://drive.google.com/file/d/1Q3rGWz9ucfGKXZUWU28HTyX72DDS7VIA/view?usp=sharing

Far Post Corner - https://drive.google.com/file/d/1FjiZLwVwAmnBQcXlSg-vVZMrxUSxK6Gs/view?usp=sharing


It's weird that the "Far Post" Corner setup scores more goals from corners if you look at the team stats at the end of the season but the "Near Post Corner" setup scores more goals overall.

It could be that the "Near Post" Corner setup scores a lot of goals after second balls, which might not to be count as corner goals... or I have no idea

424 Alhamdulillah II
https://fm-arena.com/tactic/3913-424-alhamdulillah-ii/




Far Post Corner






Near Post Corner

Steelwood said: using quite a few of the instructions tested by @Zippo as well as the reliable set-pieces. Expand

Just a heads-up, the instructions were tested separately, which means, for example, the "Much Lower D-Line" instruction was showing positive signs ( it still might be the RNG ) and the "Drop Off More" instruction also was showing positive signs ( it also might be the RNG ) BUT both instructions added might produce a negative effect.

The synergy between instruction hasn't been tested but there might be some and it must be taken into consideration.

Sane said: It was an interesting test anyway, any plans to test low crosses and mixed? Because at the moment there is no clear understanding of the crosses... Expand

Probably, we'll test that at some point.

Sane said: @Zippo Be More Disciplined is it for every position to hold a position? Expand

Honestly, I have no idea how it works under the hood.

It might be that that the "Be More Disciplined" TI makes the players to stick to their duties more tightly, which means the "Attacking" duties become even more attacking and the "Support" duties become even more supportive and so no.

But it's only my guess.

Cherknam said: So it's better with 'much lower' or 'much higher'? I think you might have made a mistake on the table at the bottom. Expand

That was just a typo. I've fixed it.

Please note, the RNG of 5,760 matches played still might be as high as 1 point.