Sunday, May 10, 2015

AFK / 4v5 Games - An Analysis Using the Kaplan Meier Estimator

This is a common complaint on reddit. So let's talk about "4v5". I will demonstrate how often "4v5" occurs and the chance of winning a "4v5". I will also show the probability of a player reconnecting once he has been AFK for a significant amount of time.

The analysis will employ a commonly used statistical model in biomedical research - the Kaplan-Meier Estimator - the details of which I will provide at the end.

I am going to define "4v5" strictly as a game where one player chose a champion but could not immediately enter the game. This causes the player to be "AFK" and idle within the game as soon as the game starts, creating a scenario which one team has only four  players. Anyone with at least some experience with the game knows that "4v5" is an uphill battle since the numerical advantage of the opposing team is very difficult to overcome.

Detecting this kind of scenario is very simple using the Riot API. Since nearly all players will start the game by buying some starting items, the time required for a player to connect (or not connect at all) can be measured simply by the time which a player buys his first item. While this is not the perfect solution, this is about the best we can do using the data that are available.

I want to be very clear that this study is only about players who are "AFK" from the beginning of the game. It does not consider scenarios which a player plays for (say) 15 minutes, then decides to leave the game for any reason.


1. How Often Can You Win a 4v5? How Big Is the 4v5 Problem?

First, let's address the effect of 4v5. Unfortunately, if a player takes a long time to connect, the effect on the team is detrimental - even if he eventually connects to the game. Using 220k Silver-level ranked solo queue games played on Patch 5.7 on the NA server, I find the following (if you are interested in other Elo brackets, click here):

Time Needed to Connect Win Rate Occurrence
0 - 1 minutes 50.12% 95.617%
1 - 2 minutes 49.14% 3.605%
2 - 5 minutes 43.12% 0.552%
5 - 10 minutes 35.55% 0.117%
10 - 15 minutes 29.69% 0.030%
15 - 20 minutes 24.05% 0.011%
20 - 25 minutes 27.78% 0.004%
25+ minutes 41.18% 0.003%
never connects 11.55% 0.060%

As the table demonstrates, in a pure 4v5 scenario where a player never connects, the chance of his team winning the game is around 11.6%. This is somewhat expected, since 4v5 is an uphill battle. But perhaps more surprisingly is when a player connects late: if a player connects within 5 to 10 minutes after the game starts, the chance of him winning is only about 35.3%

Additionally, we see that a "pure" 4v5 where one player never connects is actually pretty rare - at 0.06% per player per game. This translates to about only about 0.6% chance of seeing one such case per game since there are 10 players in a game. In most cases, the player will reconnect at certain point of the game - provided the game doesn't finish quickly enough. Once the missing player does connect, the outlook of the game recovers somewhat. On the other hand, it can be shown that if it takes at least 5 minutes to connect (including not connecting at all) for a player to connect, the team of that player has only an overall 27.3% chance of winning, which is still very low.

The take-home message here is as follows. A "pure" 4v5 is fairly rare; however, even a late connect (5+ minutes) can be detrimental to the team. Therefore, the study on the 4v5 should focus on how much time a player needs to connect to the game.

I will try to answer two questions in the sections below:

1. Are there any differences in the frequency of AFK in terms of time needed to reconnect between different Elo brackets?

2. Suppose you are currently in a game and your teammate has been AFK for 15 minutes since the beginning of the game. What is the probability of him coming back in the next 5 minutes?


2. AFK Length and Elo brackets

While the analysis above used Silver-level games, it is interesting to compare the occurrence of AFK with other Elo brackets. As it turns out, it seems that Bronze-level players are far more often to AFK than players of other Elo brackets, as the following table will demonstrate:

AFK Length(min)5+10+15+20+25+30+35+40+
Bronze0.417%0.207%0.146%0.124%0.114%0.108%0.104%0.100%
Silver0.225%0.108%0.078%0.066%0.060%0.056%0.054%0.053%
Gold0.156%0.077%0.058%0.049%0.044%0.040%0.038%0.037%
Platinum0.120%0.060%0.046%0.039%0.035%0.034%0.033%0.033%
Diamond0.090%0.045%0.034%0.030%0.025%0.024%0.022%0.022%

For example, we see that 0.417% of Bronze players will take at least 5 minutes after game starts to connect (keep in mind this includes players who never connects to the game). On the other hand, for Silver players this probability is only 0.225%, about half as often as for Bronze players. Overall, we see that as we move up in Elo brackets, AFK frequency decreases dramatically.

The following is the Kaplan-Meier Estimator (Curve) used to compute the table above.


3. My Teammate Didn't Connect. Is He Coming Back?

Say you are 15 minutes into a game and one of your teammates  remain disconnected. What is the chance of him coming back in the next 5 minutes? It turns out this can be fairly easily calculated.

Time Already Spent AFK (minutes) Probability of Returning in the Next 5 Minutes
5 50.8%
10 25.5%
15 14.5%
20 11.0%
25 6.1%
These probabilities are surprisingly similar across Elo brackets.

Therefore, when a player has already disconnected for 15 minutes, the chance of him returning in the next 5 minutes is only 14.5%. With this in mind, it is my opinion that allowing players to surrender in 15 minutes instead of 20 in a 4v5 game is reasonable and can be a good thing for the game.


4. The Kaplan-Meier Estimator

As mention above, the focus of this study should be on the time needed for a player to connect to a game. There is a small problem with this, however: if, say, a particular player needs 25 minutes to reconnect, but the game finishes in only 20 minutes, then through the data available it is impossible to pinpoint the exact time needed for re-connection.

Essentially, the data at hands have issues with censoring: we are interested in time needed for re-connection, but a lot of the data we are interested in are missing due to games finishing too fast. To overcome this difficulty, the Kaplan-Meier Estimator is used for the analysis of the data in Section 2 and 3.

The Kaplan Meier Estimator is a non-parametric statistic often used to study time-to-event data with right censoring. In the usual context of biomedical sciences, it's often used for survival analysis where the event in "time-to-event" is death. For our purposes, however, the event of interest isn't "death" - it's when a player connects to the game.

EDIT:
1.  KM Curve with 95% CI

Data is retrieved using the Riot API. 

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.