TheStarkingtonPost.com

Were National Qualifiers a Failure

by Chris McNutt | posted at 2010-06-08 03:11:00
tagged: MTG, Magic, , Magic the Gathering, Chris McNutt, Regionals, National Qualifiers

Doing the Math: Looking back on the Success’s of the 2010 National Qualifiers

Hi there cats and kittens! In this edition of “Doing the Math” we’re going to examine the numerical aftermath and consequences of the changes to our beloved yet posthumous Regionals and speculate on the causation and future of our beloved National Qualifiers. I’m anticipating that the canyon walls will be echoing with “I told you so,” for some time on this topic but as is my habit I’ll take a few more steps back than the immediate evidence and try to rationalize the bigger picture. I would also like to say that I am looking forward to seeing everyone at the upcoming Cascade Games 2.5k here in Seattle on June 26th (http://cascadegames.com/events/848) and that I would have been available for autographs if not for my acute fear of writing implements.

Pre-article Notes

As with my last article I’m going to be more formulaic about the data and reasoning while attempting to leave less to be independently concluded. This process should evolve over time, but I’ll try to keep the process familiar.

Background Research

The primary sources for the data in this edition were the Wizards web sites. I started by compiling the attendance of every National Qualifier and Regionals even for the last five years for all of the US and Canada (and a few others) from the DCI Event Online Database. These numbers were summed and compared against each year in total as well as by event. I also correlated these against the number of registered DCI players in each state as collected for my previous work and gathered some basic event information from the Wizards web site. The Wizards site was also scoured for comparative data about the event structure in other countries. Some geographical information was gathered from Wolfram Alpha. Additionally we have some less formal data in the form of answers to the poll that was run on TheStarkingtonPost.com main web page shortly after the completion of our National Qualifier Event and though not terribly scientific, it’s still interesting and tells a partial tale of some of our readers. The links have been included at the end of this article.

Assumptions and Disclaimers

The following are some of the assumptions I had to make with the data as well as a few disclaimers about the numbers.

1. Players crossing state lines to attend tournaments cannot be economically calculated so most of the numbers factor only for players attending events in their home state.

2. As with before, there is no way for me to calculate active DCI players so it’s assumed that the same relative % of active players is present in all regions, and that every player plays only within the state they originally registered in.

New Revelations

How do we measure the success of the new event format? That depends on the goal, the reason for the change, and a timeline. Was the goal to increase the total attendance? Was the goal to allow more players the opportunity to attend the qualifier? Was the goal to bring a more diverse player base (and possibly metagame) to the Nationals main event? Was the goal to change something that was either perceived to be broken or unfair?

Let’s look at the results of the changes and the possible goals highlighted above more closely.

Possible Goal #1: To increase overall attendance at the National Qualifier Events.

The year 2010 brought in 4,581 Nationals hopefuls which was about 18% less than the 5,588 from 2009. Over the last few years we’ve seen a slight but not significant decline in attendance in the US and some increases that oddly mirror the inverse percentages of the Canadian increases as shown below.

We can say for certain that attendance did not go up either in total or on average. However any success, partial, complete, or vacant, depends a lot on the timeline established for that achievement. For example, though the attendance was down this year we could consider it to be a transition period. The true success of this metric might be best looked at in a few years if the current structure is kept largely intact. As someone from the PTQ in Seattle on May 22nd commented to me, Regionals was long built up as a brand. It was an expected and anticipated tournament. The change to National Qualifiers was little more than mentioned and not particularly advertised so a decrease in attendance was probably expected.

0/0/1

Possible Goal #2: To make attending the qualifying events attainable for more players.

Certainly the average drive time for players was lowered and with the notable exceptions removed from the equations the events were easier to get to on the whole. Were the changes positively impactful? There isn’t an easy way to solidify an answer on this one. With more events and shorter drives we end up with events that are lower in attendance but more available to attend. Factoring in the decrease in overall attendance there is no way to tell how many newbies we had.

0/0/2

Possible Goal #3: To diversify the attendance at the Nationals main event.

Again we can’t really answer this one with the data available. We might get an indication of player diversity if we get a complete break down of accents at Nationals (I’m very much looking forward to hearing Rich Hagon’s dialect impressions on the coverage podcasts). Perhaps in the next few seasons and years we’ll have some new rising star who quotes owing his or her career to the 2010 National Qualifiers… we can hope. We might be out of luck on player diversity though since there were so many tournaments and players concentrated in the itty bitty Northeastern states.

0/0/3

Possible Goal #4: To hold more events.

Victory!

1/0/3

Possible Goal #5: To homogenize the US events with those in other countries.

During much of the time I spent poring over the numbers and looking into the situation in the U.S. I had assumed that the structures were mostly the same in all other regions with small allowances made here and there because of different local laws. In researching this article I found out how very wrong I was. I couldn’t find much info for the structure and timing of the events in most of the rest of the world, but those I did find had some shocking differences. I’ll be picking on France a lot here but they have it coming since Wizards published such complete event details…

First off, and possibly most glaring, France is allocated 210 total invites (100 by rank, 110 by qualifier) for a population of 15,965 total registered players versus the U.S. with 286 total invites fought over by 85,051 players. You might think that they’ll get a lot of players from other countries, but proof of citizenship and online preregistration is required there. Incidentally they had 2,514 players attend their qualifying event in 2010 which is 15.7% of all French duelists versus 5.3% of all U.S. mages, but since our events are closer on average that’s really just laziness on our part. For those keeping score at home this means that France has 1 invite for every 145.1 players in the country versus 1 invite per 457.3 in the U.S. and 1 invite for every 22.8 players that attended the French Regionals (yes they still call them Regionals) versus 1 per 24.6 in the U.S. In fact, the French events are allocated 1 invite per 20 people attending the events! (This is only about 12% off from the U.S. allocation for 2010 by player count, but would have been a much larger variance if attendance had been better). If you include the invites by rank (France, the U.S., Canada, Italy, and Spain each invite the top 100 players automatically) France gets 1 invite for every 76 players and the U.S. gets 1 invite per 297.9. Closely linked in my mind to France (because of Montreal and the movie "Canadian Bacon"), Canada had 170 invites on a population of 8,989 players. With the 70 invites allocated to the Canadian qualifying events and 968 players attending the 11 tournaments they average 1 qualifier invitation per aboot 13.8 attending players and 1 for every 128.4 players in the nation.

These numbers and those of the other countries I was able to find are in the Google Docs master spreadsheet. I would have gone to more effort to fill in the blank spots in the data but it seems to be impossible to find for nearly all the countries' invite totals. Personally, when I decide to turn pro I’m moving to New Zealand; 1 invite for every 4.2 players attending the four qualifying events whose roll call was between 19 and 60 people (with 8 invites per event). South Korea might turn out to be even more favorable after their attendance numbers are in since they have an average of 1 invite for every 5.6 invited players registered in the entire nation. It seems that our whole country is getting the Northern California treatment in regards to Nationals.

1/1/3

Misc Data and Information

We have a great deal of anecdotal data such as the poll results which showed that 26% of attendees reported incorrect tournament sizes (I got that one wrong from memory as well) and that 15% of the players who responded attended events in states other than their own (I suspect the fellow who claimed to have travelled from Maine to California in less than 30 minutes to play in a 32 round event with over 600 players while piloting Elfdrazi might actually be a delusional Yugioh player). We also have some interesting tidbits like that 21% of the respondents piloted Jund with a 68% win average. Curiously, but still possible, the other decks with over a 60% win average were Planeswalkers, Conscription, UW Control, Polymorph, RDW, Grixis, and several underplayed decks such as Runeflare, Koros, Bant Control, Mono-Black Discard and the list goes on. I’m also very distressed to report that only 77% of those polled believe that birds appear every time that I’m near.

More relevant to this article's topic we see that 58% of the players traveled 1 hour or less to the event with 31% under 30 minutes. If this data is a consistent representation of the whole country then we would have less than 5% of the players actually travelling more than 2 hours to an event. It could be said of course that most of those over 2 hours away simply didn’t go. Of our respondents, 7% were from California. Of those 6% travelled to another state, and 29% had travel times large enough to either be from Northern California or to have been leery of taking the freeway for fear that their 10 speed might get pulled over. Given the following data, I would say that nearly 30% of the players in the California tournament were certainly not from the Bay area though.

The California tournament was attended by 260 people. Not a bad number, but it exactly matches the attendance of the LA event in 2009. The tournaments in 2006 (292 players) and 2007 (310 players) were larger by a fair amount while 2008 (198 players) was lower. In total the 260 person 2010 California event was preceded by a statewide attendance of 517 players in 2009, 489 in 2008, 650 in 2007, and 719 in 2006. We can see that there has been an overall decline in attendance for California over the last few years and perhaps that was a contributing factor to the demise of the multiple-events-per-state structure. Outside of the Los Angeles event the average attendance for the last 5 years in California has been 146 players. That’s a pretty good size, but not huge. Out of the 4 states that had a reduced number of qualifiers this year, the average size of the events slashed from the schedule was 144 players. Is 145 some kind of a magic number, a threshold of concern, or just a coincidence? (I would guess it's probably coincidence.) Few socially functioning people are as obsessed with numbers as myself, but I’m sure there is some magic number out there that WotC wants the events to get up to. If nothing else, tournament attendance must be a litmus test for them as to the health of the sport. In this case the results may be a bit tainted because of the break from the known Regionals branding, but I’m sure they're watching.

How about another chart?

Here we are looking only at the States that suffered from a decrease in events. Due to the low attendance this year, we can see that at most of the events, your chances for qualifying were better than in years past, but it also reveals some data that I would find alarming. The events in Pennsylvania used to be quite a bit larger than the 2010 turn out. In fact, every Regionals in the previous four years in PA has been larger by at least 50% then the one in 2010. In Texas and Florida we see that this year’s qualifier was smaller than the larger of the events in the past five years. I’m not sure where to cast the blame on this one. If we could go back and hold two and three events per state would the historical per-event attendance be in line? What about if they were called Regionals? We can make guesses, but during this time of MTG, when its popularity is on the rise, these are not the results you would expect from a popular yearly tradition.

Obviously the following information can be tempered with the knowledge that many players could have gone to alternate venues but when I see that Georgia, who has averaged 312 players per event (the on-average largest in the U.S.) from 2006-2009 but has a showing of only 132 players in 2010, it should be time for some serious reevaluations. The only non-multi event states we saw with an increase in attendance from previous years were Oregon and Hawaii with 15% and 12% gains respectively. We can probably attribute the sharp declines in MA, MD, and NJ shown below to the prevalence of nearby events but what about Nebraska and North Carolina? And if y’all did go to closer events before, how come the overall turnout was so much lower?

I also have to wonder if the decreased attendance was a byproduct of the increased recent success of the game. Stick with me on this one for a second. MTG is going well; we have more players than ever, and therefore have more events than ever. If you're surround by back-to-back big events, PTQs, SCG Opens, 5ks, 2.5ks, special events, Prereleases, Releases, etc. and have to choose which few events you’ll go to, do you pick the National Qualifiers? I did (and did horribly) but that meant that I can’t go to the “Standard for a Mox,” event this weekend, and had to do a bit of fast talking to get to go to the PTQ on May 22nd (I went 6-2 and was 21st out of 209! This is really good for me as I’m an analyst, not a pro player…). If you’re the average player out there with a limited amount of time and money to devote to the big events, maybe you skip the National Qualifiers? I don’t know, but it could be a factor.

I hear you all crying “Tell us what this all means oh mighty mathematical gumshoe,” and so I will (ok, so that was actually my son crying after whacking his knee on the table, and no I don’t know why he always yells that when in pain)… It means that any success of the current program's format will only be known after a few years of seeing it in action. The question is can they afford to keep this structure intact when the short term panned out so poorly? Or maybe this was a "Farscape" move? Let’s say you have a successful yet expensive TV show yearly tournament that for whatever reason you want to kill. Do you move it to the 10 p.m. time slot to lower the viewers enough to make it not cost effective to renew and thereby give you the $10 million you needed in order to make the miniseries "Taken" by Steven Spielberg? In this case, probably not. I don’t think these events are all that costly to Wizards directly, and Nationals feeds into Worlds so unless you wanted to get rid of Worlds and maybe the Pro Tours you wouldn’t want to… wait a minute… This insane theory is a topic for another time, but I think for the health of the community, they would take these little details and make some changes for next year…

Program Change Recommendations from an Unqualified Amateur

I like having the events in each state. Outside of the rivalry of the Yankees and the Confederates I think it’s o.k. to structure it like it is and it makes it easier for a huge amount of people to attend. Although I personally would have probably chosen a title such as “Regionals (National Qualifiers)” for 2010-2012 before just calling them National Qualifiers, I think that for 2011 we stick with the new branding but maybe do more to familiarize players with the event before it comes out. Include mentioning it in FNM and Gateway documentation, maybe have more time devoted to it on DailyMTG.com, that kind of thing. The invite allocation is probably o.k. as is as well. However, in California, Texas, Montana, Pennsylvania, and probably Florida I think we need more events. The amendment policy to the one-per-state approach should be to add events to any location that would be more than a three hour drive from a significant population of Magic players. WotC knows (or could look up) where those locations are and encourage a T.O. to hold an event there. If the event is only solving a geographical problem than they could assign 1 or 2 invites to the event, or in the case of California, assign 4 to the Bay area and 8 to L.A.

There’s no reason why the allocation or distribution needs to be equal in each state. The player base and geographies certainly aren’t equal and the qualifiers feeding into an event that represents our nation as a unified country to compete against other countries should be reasonably convenient to our competitive player populations to attend. Plus we need all our best players to make sure we beat France.

Conclusion

Were the changes to the Regionals events a success? With a final score of 1/1/3 I would say no they weren’t. Whether this is because of the name change, because of advertising, because of the new geographic hardships, because of other events, because they insisted on running their Jund deck with none of the popular Jund cards in it like Thrinax, Blightning, or Bloodbraid (and he called it “Johnd”), or because of general player malaise it’s hard to find the diamond in the lump of coal we’ve been given.

Doing the Math,

Chris McNutt, Magic Analyst

fatecreatr@gmail.com

@fatecreatr on twitter

Links

Cascade Games

http://cascadegames.com

Wizards of the Coast Events Online Database

https://webapp.wizards.com/tournaments.aspx

National Qualifier Survey

http://www.thestarkingtonpost.com/articles/-/National_Qualifiers_or_Regionals_Survey

Wizards DCI Players online database