Computer Adaptive Testing and the GMAT

Back around Christmas, I had a few weeks free and decided to prepare for the GMAT exam, a "computer-adaptive" standardized test required (or at least accepted) by business schools around the world. With university application deadlines looming, most of the testing sessions were already fully booked, but I managed to find a mid-January test date just a few hours' drive away.

I'll start by saying that I finished university eight years ago, so—ignoring a few intense weeks of German classes—it's been a while since I "studied". And it was about half my lifetime ago that I last wrote a standardized test: you know, with those sealed envelopes, bar-coded stickers, big machine-readable answer papers, and detailed instruction books reminding you to use a #2 pencil and "chose the BEST answer". If, like me, you haven’t written one of these in a while, you may be surprised by how much has changed.

The GMAT, like a number of other admissions tests, is now administered exclusively by computer and the test centres even have palm and iris scanners, which are used any time you enter or leave the room. Unlike most computer-based tests, though, which stray little from the well-worn paths of their pencil-and-paper siblings, the GMAT uses a computer-adaptive process undoubtedly conceived by a singularly sinister set of scholars and statisticians. This process is complex and has a number of interesting implications but basically it works like this: when you get an answer wrong, the questions get easier; when you get an answer right, they get harder. The theory is that by adjusting the test to your ability, the computer is able to rate you more precisely against people of about the same level.

The material on the test is not really that hard. It's no walk in the park either, but it mostly limits itself to stuff you learned (then unfortunately forgot) in high school. Everything else about the GMAT, though, seems designed to maximize stress:

• The test is long. Nearly four hours long. There are three sections 60-75 minutes long, with a short break between each.

• The test is timed. Ok, what test doesn’t have a time limit? But this one has a clock counting down in the corner of your screen, taunting you to pick up the pace. Worse than that: you'll probably actually need all the time because the questions keep getting harder as you get them right, remember? The challenge on this test is not usually solving the problems, but rather solving them in time.

• The breaks are timed. Again, not surprising, I guess. But your break is only eight minutes long and, if you’re not back, the next section simply starts running without you! Inevitably you spend the entire break worrying about how many minutes you have left. Since you need to scan your palm and iris at the beginning and end of each break, your trip to the bathroom is not going to be leisurely.

• Erasable notepads. No pens or paper are allowed, presumably so you can't smuggle out questions. Try working out math problems quickly on a laminated card with a dry-erase marker.

• You can't skip a question. Remember that the questions get harder as you get them right and easier as you get them wrong. This means that the next question you see is largely determined by how you answer the current one. The computer needs an answer to the question, so you can’t skip it.

• You can't go back. Similarly, since your current position is determined by your earlier answers, you can't go back and change them. So if you're used to finishing early and then checking over your work, you'd better start unlearning that habit.

• You don't know the difficulty level of the questions. Is the test feeling easy because you really know your stuff or are you simply earning yourself easier questions by choosing a lot of wrong answers? The only saving grace here is that you're so busy madly answering questions that you don't have many brain cycles left to worry about this.

• Some of the questions are "experimental". About 25% of the test is made up of new questions being tested on you to determine their difficulty level, but of course you don’t know which. That's right: that really hard question you just spent 5 minutes working on because you were sure you could solve it... doesn't count!

• You are heavily penalized for not finishing. Right, ok, so you have a fixed time, you can't skip or come back, and you can’t predict the difficulty of the remaining questions. But if you want a decent score, you still need to pace yourself to answer all of them. Remember that countdown clock? You have about two minutes per question–so keep an eye on that average time! Oh, and the clock counts down but of course the question numbers go up, so you’d better get real quick at subtracting your time from 75 (you’ll be working out your average question time every few questions).

• Data Sufficiency questions. These nasty little buggers are, I think, unique to the GMAT. Given a math problem and two statements, you are asked whether the problem can be solved using either statement, both statements, or both statements together. You don't need to work out the answer to the problem but you need partially solve it several times with different information and keep each attempt separate in your mind. Don't think that sounds tricky? Try searching for "sample gmat data sufficiency questions" and try a few. I think I got only about a quarter of these right on my first practice test.

You have to admire the brilliantly evil minds that came up with this thing. The experience for the test-taker is four hours of pure, non-stop stress. At least that was my experience: my brain literally didn’t stop whirring. The adaptive process pushes everyone to their limit, challenging them to keep their feet under them and ensuring that they're sweating right until the end.

The test designers have really optimized the experience around their own needs: the test is easy for them to grade, minimizes cheating, allows new questions evaluated automatically, and measures something in a pretty precise, consistent way. I’m not entirely certain what it measures, but I’m pretty confidant that people who are generally smarter, better organized, faster to learn and adapt, and better at dealing with stress will obtain a better result.

As a company selling a product, it might seem odd that GMAC (the company that runs the test) can get away with optimizing the test for their own needs. But, although it may appear that the test is the product you’re buying, I think what you’re really buying is the report that is sent to the universities. The cost of this report just happens to be $250 + study time + four hours of stress. If GMAC had competitors, they might be forced to optimize for the test-taker but, as a virtual monopoly, the motivation just isn’t there.

The challenge with the GMAT, I think, is really learning an entirely new test-taking strategy. I used a couple of books (Cracking the GMAT, by Princeton Review and The Official Guide for GMAT Review) to first understand the test and the differences in approach that were required and then to practice as many questions as possible of the specific types that appear. Doing computer-based practice exams is, of course, also essential given that what you’re learning is the test-taking strategy more than the material.

I emerged from the exam feeling absolutely drained but energized by the rush of tackling something so intense and coming out on top. In some ways it was fun but I have no intention of rewriting it any time soon. :)