Activision has published a 25-page white paper exploring the impact of skill-based matchmaking (SBMM) on its multiplayer lobbies, determining that SBMM is better for all players.
As spotted by indie game developer and consultant Rami Ismail, the report – which can be read in full on Activision's official website – outlines an "amazing A/B test" where Activision "secretly progressively turned off SBMM and monitored retention… and turns out everyone hated it, with more quitting, less playing, and more negative blowouts". Cover image for YouTube videoPlayStation live-service Concord won't sell you a battle pass - would it be better if it did? PlayStation live-service Concord won't sell you a battle pass - but would it be more successful if it did?Watch on YouTube
Activision announced plans to launch the series of white papers back in April, and has already considered the impact connections and Time to Match has on online play.
The most recent paper, however – entitled "Matchmaking Series: The Role of Skill in Matchmaking" – "shares a look at matchmaking inside Call of Duty multiplayer, including how we always work to create a balanced system for the benefit of all players. While skill is one of many factors in matchmaking; it is not the driving force behind Call of Duty’s matchmaking system".
"Playing a better opponent may push players to become more proficient over time, but being severely outperformed in their matches, our testing has shown, instead leads players to quit matches in progress or to not play multiplayer," Activision surmises.
It also confirmed that it whilst it was "considering a single core multiplayer playlist that does not use skill as a factor in matchmaking", Activision believes "historical testing reveals that low- and mid-skill players would be unlikely to participate in such a playlist. As a match can only be created from players actively searching for a playlist, the likely result would be matches composed solely of high-skill players."
The experiment, Activision opines, shows that the existing SBMM system allows a broadest range of players to participate in "all key parts of Call of Duty's game design in core multiplayer".
This "includes games where players can contribute meaningfully to their team, not only with wins, but for their own personal records and achievements".
"The system strives for team balance, where players are not consistently on the wrong side of blowout matches. Players who frequently experience blowouts are seen to quit matches in progress and the game itself, which negatively impacts the experience for players of all skill levels," Activision concluded.
"We will always strive to improve our approach to matchmaking to ensure that players of all skill levels are having the best possible experience. A healthy player population is good for everyone in the community."
Further research papers into ranked played and "experimental methods" are expected later this year.