A Brief History of Certification - The First 10 Years

From Judge Wiki (English)
Jump to: navigation, search

Find other articles in the Judge Article Index.

AuthorDavid Hibbs
Date Published2012-02-08
Original SiteJudge Wiki
LinkThis article is only available on the wiki; see below
LanguageEnglish
TagsTesting
Recommended for Levelall levels
Notes
AbstractDavid presents the first 10 years of Judge Certification



If you are reading this article, there’s a good chance that you at some point wondered what it meant to be a judge and how to become one. You probably also realize that the process has not always been the same; after all, there have been changes announced even this year about requirements for different levels! The DCI and the judge program are over 15 years old. I’ve personally been a judge for over 10 of those years, and I’ve been talking to a number of other long-time judges. What information is included here is the best I have, and I’m sure it’s not 100% complete or accurate—but it should prove interesting nonetheless! So, without further ado, let’s dive in at the beginning!

As you know, Magic was released in 1993 and it wasn’t until 3 years later that Wizards of the Coast introduced the Pro Tour. Of course, you can’t have a high level event without officials or staff! So the question is this: how do you build an international organization from scratch? In order to get the ball rolling, much of the work done during this time was done person-to-person and on paper: paper that was not well-tracked during Magic’s meteoric rise in popularity, through mergers and acquisitions of TSR and Hasbro, and through the distraction of other products such as Pokemon.

Naturally, some of the earliest examinations were nominal. If you directly knew one of the WotC staff, they would bring you on board. I’ve heard reports from a few store owners and organizers that were either called directly by Wizards staff or met them at conventions like GenCon simply asking them if they’d like to be a certified official. No quiz, no policy… just an invitation! This was of course the fastest way to start, but it didn’t exactly engender consistency from region to region, or even store to store. Judges had wide discretion in those days, and more than a few events had questionable results.

As a means of promoting some degree of consistency at least in knowledge, the earliest paper exams were prototyped in 1996. A few judges were certified directly by Elaine Ferrao and Tara McDermott at the early Pro Tours. I have yet to find a copy of these, but from what I’ve learned they were 15 or so questions and had a high degree of focus on puzzle problem solving.

From this point, Wizards began to expand organized play by introducing the Arena League. Logically, they needed more officials and an Arena League test evolved. The Arena League exam consisted of 10 questions and so far is the only path to certification mailed directly to Wizards and taken on the honor system. It wasn’t widely publicized and again, records are mostly lost, so it is unclear how many judges/referees were actually brought in under this program but it seems clear this was not very successful.

Things continued along these lines—direct contact, conventions, phone calls, and a sparse few exams—until 1998 or 1999. There was even a bus tour where various Wizards employees would travel across the country and do demos and gunslinging at various events from comic book conventions to mall tours to advertise the game! Sometimes those on the tour would recruit locals to participate and these individuals would be certified based on their participation and performance. Some of them, like James Do Hung Lee, were eventually offered jobs in Renton.

As James Do Hung Lee wrote to me, “changes were being developed and implemented almost every Pro Tour. The growth of the game and the needs that entailed in those years between 1994 and 1998 were such that nearly everyone involved had never had any experience with massive, global program implementation and people were scrambling to produce results even as they were discovering mistakes or updating best practices.”

In approximately 1998—it’s not clear exactly when—a new series of exams was developed. These exams began to take the form we are more familiar with today; a question was posed and card text was provided where necessary. Unlike current exams, however, the test included not only multiple choice questions but also a number of True/False questions. These exams were paper and were given the designation of A1/A2/A3 for L1 and L2 or B1/B2/B3 for L3. Printed on paper, these exams and a separate answer sheet were used and reused for any number of candidates. Additionally, in order to become certified, your tester had to submit an evaluation form analyzing your strengths, weaknesses, and other qualities. This form was not unlike a current judge review—in fact, it has a great many similarities in areas to be observed.

At this time, L3 judges were allowed to test new judges, and were allowed to charge a fee for you to take the exam! Needless to say, with this being a paper-based process many certifications were lost or delayed. Even Scott Marshall, now L5, was not immune—his results and his fee disappeared the first time!

Often, testing was even more expensive due to availability of testers. If you were interested in becoming a judge, you first had to find an L3. At the time, L3s were not very common, so you would have to travel to even reach and talk to that L3—a sure disappointment if they deemed you “less than worthy,” which they could do for many reasons. For L2 or higher, you probably had to travel to a convention. L3 exams became a grueling ordeal: you first had to pass the written exam, and then you had to go through several hours of elaborate role play in how to handle difficult situations.

So what do you do if you know you have an expensive and difficult test to take? You prepare, of course, but how? Fortunately for you, a fellow named Michael Kastberg formed and headed what was to be known as The Delphi Group. This group included Laurie Cheers, Russell Bulmer, Paul Barclay, John Carter and Ingo Warnke and spent most of 1999 developing The Delphi Test #1, an exam consisting of 65 questions to test your knowledge and skill. They then developed The Delphi Test #2 and released it in 2000. At the time, these were the penultimate tests of your rules knowledge—though they did not in themselves confer any certification. It was simply the best way to prove to yourself that you had what it took to pass the “real” exam.

During this same time frame (1999-2000), a bit of a turning point was reached as L3 and L4 certifications were expanded. L4 judges were rare individuals either internal to Wizards or trusted by people on the inside as being super special. They included men such as Mike Guptil, Mike Donais, and Colin Jackson as well as a few large scale organizers. Well known L2s like Michael Kastberg were promoted to L3.

Coming back to the exams themselves, the tests at this point were paper-based and extremely static. This meant a number of challenges. First, what to do with someone who did not pass the exam? Well, you give them a different exam form. But what happens when you run out of forms? Are they out of the running? For some, this was undoubtedly the case. Second, due to the evolving nature of rules and policy, these exams were often outdated. I distinctly remember my own L2 exam in 2002 carrying a question about “Which of these cards are legal in Standard?” One of the answers was Erhnam Djinn and the answer key said I was wrong in selecting this big guy, but he had just been reprinted in Judgment! So I successfully argued for my point on the exam. Which raises a third point: the exam was only as good as the tester. If a rule changed such as happened often for Humility, the answer key was outdated. Was your tester aware of the change? Were you? It was often hard to know how hard to push for an answer. On the other hand, this process meant that L3s had a great deal of flexibility in certifications. Jumps from no official certification level (the paper-based process was extremely slow, after all) to L2 were even possible. As an example of this, Steven Zwanger’s L1 paperwork was held up or lost and in 1998 he was able to jump to L2!

With the growing presence of L3 and L4 membership and the resulting growth of L1 and L2 judges, these difficulties became much more evident. Collin Jackson headed up development of a new way to test and evaluate judges. The existing test questions were converted, and the doors opened for other judges to submit questions and reviews. In October of 2005, the Judge Center opened its doors. The first reviews were entered by Jaap Brouwer in this time frame, but it would be some time yet before L3 exams were done using the online systems. Sometime in 2006—10 years after the start of the program—this system went into widespread use.

As you can see, it was a lot of work on the part of many people to build the program to what it is today—a truly unique international organization. As of this writing, the first 10 years of the program have been compressed into a few short pages. Obviously, this no more than scratches the surface! Keep an eye and ear open when you have the opportunity to hear a long-time judge talk; they all have interesting stories about the earliest days of the program! When you hear one, or if you have one of your own, share it with me so I can add it to the judge wiki. If you’re interested, keep an eye on that page as it continues to develop and we collect stories from the trenches.

2012, David Hibbs