2th

He went on to post the top 10 for NA which were (formatted for better view)

From 2019, the top 10 most frustrating champs in NA were:

• Pyke
• Fizz
• Zoe
• Akali
• Teemo
• Shaco
• Vladimir
• Rengar
• Yasuo
• Master Yi

https://twitter.com/blaustoise/status/1301578621593681920

EDIT -- Check my comment history for full details on the survey

grabbing the top comment to say:

data is from October 2019. It's why in another thread I remarked that Korea is more likely to find champions frustrating if they have high pro-play presence (they disproportionately found Pantheon, Renekton, and Leblanc frustrating relative to other regions.

We typically run this survey around end-of-year, so we'll update data for 2020.

Champions not included:

- Samira, Yone, Lillia, Sett, Aphelios, Senna, Voli + Fiddle updates (only old versions)

GaggedAndDrooling

lol it's blaustoise. He never shares the source of his comments. He just cherry picks points of data that only he has access to and abuses his privilege to make a point that no one can say anything against.

The source is from our Mass Champion Survey (many devs have referenced it before, myself included. We've talked about it in official blog posts too). It's a survey we run in some of our largest regions that asks players their opinions about champions regarding everything from being frustrating to play against, their visual design, to their VO and lore.

The survey is distributed via the client (it gives a link to a Qualtrics survey). It's a pretty massive sample size, typically 30-40 thousand players per region. There's actually a high likelihood that players within this thread have taken it before.

As for cherry-picking, it was an interesting regional disparity that I wanted to highlight. I highlighted some other disparities too, but Yuumi in NA vs CN was one of the biggest once I found. I also pointed out that a bunch of champions are pretty close in rank for "frustrating to play against" across regions surveyed to demonstrate that it's not just all champions that NA and CN view differently.

Hope that helps.

DarkLeoDude

Depends on the sample size of data. How many people are actually filling out these surveys and giving a rundown of EVERY champion in the game?

Most likely it's small fractions of a fraction of a percent, which leads to wild swings like this.

From the most recent survey:

NA, 30,000 players each evaluate 3 champions randomly selected.

Total of 90,000 champion evaluations from 30,000 players

For the ~145 champions on this survey, that comes out to over 600 players per champion per region.

It's difficult because given the number of champions we have (now 150), and if we want to slice the data by region (let's say we do 5 regions), the survey gets pretty massive. Around ~150,000 total players surveyed. Hence "Mass" Champion survey

MasterOfBinary

Agreed. As long as Riot isn't releasing full datasets on this stuff, I find these tweets from Blaustoise shitting on community sentiment frustrating at best and antagonistic at worst. Want to complain that only NA finds something frustrating? Fine. Release the question asked, who was asked it, sample size, answers received, etc. He constantly hides behind cherrypicked stats that the community doesn't have access to, and everyone acts like it's totally fine. So long as they're not releasing the methodology and data behind these talking points, it's not a discussion with the community - it's shifting blame.

Question asked:

Next, consider these specific aspects of [CHAMPION]'s gameplay in League of Legends and rate your agreement with the following statements:  

 [CHAMPION] is frustrating to play against.

Answers: 5 point likert (Strongly disagree to Strongly agree)

---

Who was asked:

Active League of Legends players throughout the month of October, 2019, in 5 major regions: North America, Brazil, EUW, Korea, and China.

---

Sample Size:~150k for the total survey, ~30k per region. Respondents randomly see 3 champions to evaluate, 90,000 total evaluations per region, ~600 per champion per region. So "NA Yuumi" data is n~600, "CN Yuumi" data is n~600.

---

Answers received:

As stated above, it was a likert score. Scores are then ranked per demographic or split that you want (so Pyke had the highest or rank 1 likert score in this question for NA, Yuumi had the 13th highest)

HubblePie

I'm kind of curious to see where this data's coming from

From our Mass Champion survey (giant survey we run in a bunch of major regions to assess champions on a number of different things).

It's a nice addition to playrate and winrate data (those being behavioral data, Mass Champion being sentiment data)

If you check my comments I responded more in depth about the details of the survey to some other people in the thread.

DanteMasamune

popularity matters a bunch in these metrics since champions that are almost never seen are seen ten times more in other regions and the reverse happens too

Agreed actually. I think the biggest critique me saying "players find X champion less frustrating in this region" is that playrate likely plays a huge role.

You won't be as frustrated with a champion if you never see it, and you're likely going to be more frustrated by a champion if you constantly have to deal with it every other game.

Hey friends,

Goal of this was just to highlight some interesting disparities in player sentiment around champion frustration. I wasn't trying to suggest that because China doesn't think Yuumi is frustrating, that NA finding her more frustrating is invalid. There are other champs that China, or Korea, or Brazil find frustrating that NA doesn't as much.

For example, Leona (??) scored way higher for frustration in China than anywhere else. Idk maybe they meme Leona over there? But I do find it really interesting if memes or community perception could drive these differences to this degree (I think playrate and some other things factor in as well).

If y'all are curious about the data source, it's our Mass Champion survey. I responded to a comment in the thread and gave more details on this specific question's wording, sample size, and some other stuff.

I don't just make up these numbers and I'm not trying to cherry-pick to make arguments. If anything I just want to highlight that it's a global game, and we often find really big and interesting regional differences that I figure are cool to discuss.

MasterOfBinary

Thank you.

also made a comment in the main thread. wasn't my intent to be antagonistic

Cherubi420

Zilean would be in every top 10 if he was more popular

as someone who spams Zilean to get to diamond at the end of the season, I agree

m0bilize

He's clearly saying NA Yuumis > China Yuumis

idk why people keep missing the real takeaway. good catch u/m0bilize

_Aki_

In conclusion: pretty cool Pantheon rework. All Riot did was estrange old Pantheon mains while introducing him to the Top "frustrating to play against" list.

(1) he was only higher in Korea relative to other regions

(2) This was October 2019, around worlds. He was overtuned and 100% banned. I attribute hist higher frustration rating there to Korea being more sensitive to pro play.

(3) frustration isn't the only thing that speaks to the sucesses/misses of a rework.

GroundbreakingAlps2

It's quite dishonest to say that this is a mass survey and draw comparisons between regions when n=600 per champion. This data is worthless, it's just random noise (at least quite close to it).

Only 600 people got a chance to rate how frustrating X champion is to play against per region, and then you compare.

Run the same experiment again in NA and you would get absurdly different result. Something that might happen is that the 24th most frustrating champion to play against might become the 78th most frustrating champion to play against when you run the experiment again (and hence ask different people).

Another thing to note that you're surveying completely random people (or I guess it's not completely random, its from people that played the game in october 2019). Sure, that can be good but it can also be bad. It can be bad in the sense that you randomly survey a bunch of people that barely play the game, you survey a bunch of aram only players (who's opinion doesn't matter). When n=600, it might be the case that a lot of people that were asked about champion X have barely even played against it (since you're asking players that don't necessarily play the game a lot).

You would never take a champion winrate with 600 sample size seriously, why are you taking this ranking "how frustrating is champion X from 1 to 5" seriosly? with just n=600, the average for champion X might be 3.7, but the second time you run the experiment it's 2.7 (which is a massive difference). It's quite volatile with just n=600 per champion, just like champion winrates are.

A bunch of other people are writing in RE the statistics... I'll just say consider the business case.

I'm already surveying ~150,000 players. The full list of questions per each champion, plus a demographic block, plus we run some other blocks + questions in this survey means it's already long per person.

Now I can either survey more people, or I can make more people evaluate each champion. Either way I'm putting strain on players to be evaluating more champions or I'm expecting more people to take this survey (remember, we're running surveys on esports, on skins, on other cosmetic content, on seasonal events, etc. etc.).

Also we're not using this data like winrate data. I don't have the confidence to say that if a champion ranks 13th in frustrating with a likert score of 4.25 that the champion at rank 14 with a likert of 4.23 is less frustrating. But compared to the champion at rank 35 with a likert of 3.9? Now I'm more confident. The champion at 125 with a likert of 2.2? probably less frustrating.

We often split up the data from the mass champion survey into quintiles or quartiles. So if you're a top 30 champion for visual appeal, we just consider you as being pretty appealing regardless of if you're 26 or 23. The kinds of decisions we use this to inform are not as granular or as specific as nerfing champions RE winrate/banrate.