Evaluating Your Practice Tests, Part 2 of 2

by on October 1st, 2009

This week’s article is the second installment of this two-part series.  You can review the first installment of the series here.

Practice tests are an invaluable component of any test-taker’s study plan, but the most valuable thing is actually not the act of taking the practice test. Just taking a test doesn’t help us to improve all that much. While taking a test, we are concentrating on doing (using everything we’ve learned up to that point); as a result, we’re not really learning much.

The most valuable thing is actually the data that you can extract when you’re done with the test; that’s how you learn to get better and know what to study before you take another practice test. There are two main components to that data:

  1. Statistics and metrics based on timing, difficulty level, percentage correct, question category, and so on
  2. A thorough, analytical review of the specific questions that you saw on the test

Last week, we discussed #1; this analysis is necessary to do first so that we know how to review the specific questions, depending upon which of the 5 categories (discussed last week) a specific question represents.

Category 1: I got it right roughly within the expected timeframe (otherwise known as strengths)

Yes, these are strengths, but no, you don’t get to skip over them. There are still lots of things to learn here. First, did you get the question right for the right reasons? Or did you get a little bit (or a lot!) lucky? If you got lucky, then you just as easily could have gotten this question wrong, which means you need to move this question to category 2.

Okay, so you got it right and you knew what you were doing. Now you can move on, right? Not so fast. Did you do the problem in the best way that it could be done? Best = efficiency + effectiveness. Basically, did you do the problem as efficiently as you could while not sacrificing accuracy? Even when you get a problem right, the answer to this question is not always, “Yes!” Examine other ways to do the problem and figure out which way is the best for you.

Further, how are you going to recognize a different future problem that tests the same thing, so that you can immediately replicate your “best way” approach? You need to figure that out as well; your overall goal is to recognize future problems (as opposed to having to figure everything out from scratch).

Finally, if you had had to make a guess on this problem, how would you have done so? I know you didn’t need to — you knew what you were doing. But maybe you’ll have a harder problem of this same type in future, so learn how to make an educated guess now, on a problem that you actually did understand.

Category 2: I got it wrong roughly within the expected timeframe

(Possible weaknesses in content area, methodology, etc, BUT check the difficulty levels; maybe this question just happened to be really hard on this test!)

First, you need to figure out which weak areas here are actual weaknesses and which areas were merely consequences of other things happening on the test (eg, the question was highly rated). Why did you get this question wrong? If this one was 700+, you got another lower-ranked question of the same type right, and you were fine with these on your last test, then your fundamentals may be good, and it may be time to lift yourself into the toughest areas for this particular question type or content area. If you’re using ManhattanGMAT books, for instance, now would be a good time to take a look at the material in the Advanced chapters. You also want to explore the best way to make an educated guess on a problem of this type and difficulty level.

Alternatively, maybe you think you do know the material but you’re making a lot of careless mistakes. Start an error log, noting identifying info about the problem (source, number, etc.) and articulating (a) what mistake(s) you made, (b) why you made it, and (c) what habits you will need to make or break in order to avoid repeating that kind of mistake. On verbal questions, include why you thought the wrong answer was right (and make a note that this reason is not a good reason to use to pick an answer) and why you thought the right answer was wrong (and make another note that this reason is not a good reason to use to eliminate an answer). Use the error log every time you find you’ve made a careless mistake!

Finally, something in this category may indicate a fundamental weakness. Is the material common – something you already studied or something you should know? Or is the material rare? Prioritize your effort to learn this material based upon your answer and, as needed, return to the relevant sections of your books. In addition, explore how to make an educated guess – perhaps the material is so rare and the problem so hard that the most appropriate action is to learn to make an educated guess and move on.

Category 3: I got it wrong way too quickly (more than 30 seconds faster than it should be)

Are these really weaknesses or was the student just going too fast (and, naturally, making more careless mistakes)? Why was the student going too fast on these?

Again, for each problem, you need to figure out why you were going too fast. The only acceptable reason to get a problem wrong too quickly: you decided this problem was way too hard for you, so you made an educated guess and moved on. If you sped up because you thought it was easy, then made a careless mistake, your first instinct in future should be to take your time. (Also, add that problem to your error log!) Don’t sacrifice a correct answer just to save 30 seconds.

Alternatively, if you sped up because you thought or knew that you were behind on time, then you need to fix your timing problems elsewhere in the section. If this is the case, try to decide whether this problem is something you should be able to do in the expected timeframe or whether you still need some review and practice in this area. Check other problems of the same type on this test or previous tests to make this assessment.

Category 4: I got it right way too slowly (more than 30 seconds slower than it should be)

These are still weaknesses even though you got it right! These questions are costing you points elsewhere on the test – maybe more points than you’re gaining on this problem. Figure out why the timing is higher and how you can do these more efficiently. If the timing is just a little bit too high, that may be okay — perhaps the problem is extra hard and long. If you’re consistently going long, however, then perhaps you don’t know the best way to solve the problem, in which case (a) figure out the best solution, or (b) the best way to recognize that this problem requires a certain set of steps, or (c) both. Alternatively, perhaps you’re struggling to execute the necessary steps to finish the problem on time (in which case, practice the steps but also consider just making an educated guess, particularly if the problem type or content is fairly rare).

As with category 1, don’t forget to make sure that you really did know what you were doing on the ones you got right; if not, then move questions from this category to category 5.

Category 5: I got it wrong way too slowly (more than 30 seconds slower than it should be)

These are the biggest weaknesses, obviously. Get them wrong faster. Seriously – you’re getting them wrong anyway, so start by just taking less time to get them wrong! That will improve your performance on all those other ones on which you’re currently rushing and making careless mistakes!

What is slowing you down? Figure that out and it will tell you what to do next. You may need to review the material from your books, or do more practice with problems of this type, or find more efficient ways to solve, or learn better how to recognize questions of this type, or be more quick to make an educated guess… whatever that is, do what you need to do to get better. (At the same time, evaluate the frequency with which the particular material in question is tested; set higher priorities on the things that are more frequently tested.) Don’t forget to break out the error log again, of course.

Make sure to spend time figuring out how to make educated guesses in these categories as well. You may first have to go to similar but easier questions in order to learn how to make educated guesses on problems of a certain type, and then you’ll have to apply those lessons to the harder problems that are giving you trouble. Sometimes, the best solution to a problem in this category is, “I’m going to make an educated guess within the expected timeframe or faster.”

In the comments section below, let us know any additional tips you have for reviewing your work!

Read other articles in this series:

16 comments

  • Hi Stacey

    Thanks a ton for the article. BTW it will be great if u could provide us with the split up of how u would classify a question to be difficult, avg. or easy depending on the score.

    Thanks
    Chadaram

  • Great question! A very difficult question is one you can't do, a difficult question is one that you struggle with (and you may or may not be able to do correctly), an average question is one you can do with some work, and an easy question is one that you can do quickly.

    That wasn't quite the answer you were expecting was it? Yet, that's the technical definition - easy, average, and difficult are subjective categorizations based upon that student's strengths and weaknesses - and that varies from person to person. And that's also how you should classify questions, because that's what matters to you as you try to improve.

    If you do have an opportunity to see a general rating for a question (for example, our test reports show you the questions in four general buckets: 300-500, 500-600, 600-700, and 700-800), then your goal is to answer correctly the questions that are at or below your desired goal score. If you want a 650, for example, then you should (hopefully!) answer sub-600 questions correctly and around half of the 600-700 questions (you have to guess on that one because the reports don't show you anything more precise than "600-700"). You can get most of the 700+ questions wrong. If you want a 700, then you need to get most of the sub-700 questions right but you can also get many of the 700+ questions wrong.

    So, when reviewing, take a look at what you got wrong and ask yourself why: was this a tough one for you or did you make a careless mistake? If the former, how hard was it? Can you get better or should you have just made an educated guess because it was way too hard? If the latter, figure out why you made the careless mistake and what you can do to prevent the same kind of mistake in future.

  • Hi

    What would be the best timings per question type in Verbal in order to score 700+ .
    I hear from people that for SC - 1 min ; CR - 1.5 min ; RC - min.

    If this indeed is the case, then how can I improve on the timings. My timings for the correct answers in OG and gmat prep are
    SC - 1.5 mins
    CR - 2 mins
    RC - 2 mins.

    Any suggestions how to get down the timings of SC & CR .

  • Hi Stacey,

    I love your posts, especially the idea about quality over quantity, but my biggest problem is remembering what I have studied previously and applying it in the real exam. I think you framed it perfectly when you talked about a CR problem, but I am sure this can be applied to Quant:

    3. How do you analyze problems? Here are some questions you'd want to answer on that CR problem:
    - Was I able to CATEGORIZE this question by topic and subtopic? By process / technique? (eg, CR, draw a conclusion, look for something that must be true according to the argument)

    - Did I make a CONNECTION to previous experience? Or did I have to do it all from scratch? (eg, the question wording is a little funny, but I remember seeing this wording before for a draw a conclusion question, so it's probably the same type this time)

    - Did I COMPREHEND the symbols, text, questions, statements, and answer choices? (eg, I didn't quite understand what answer choice C was saying)

    - Did I choose the best APPROACH? (eg, I narrowed it to A and C but then I read those two choices about 4 times each and wasted a lot of time trying to decide between them)

    - Am I comfortable with OTHER STRATEGIES that would have worked, at least partially? (eg, how should I have made an educated guess on this problem?)

    - Do I understand every TRAP & TRICK that the writer built into the question, including wrong answers? (eg, specifically why are the 4 wrong answers wrong? which is the most tempting wrong answer and why - literally, how do they write an answer to be tempting even though it's wrong? why would someone be tempted to eliminate the right answer - again, how do they make an answer seem wrong even though it's right?)

    - If I got it wrong, WHY did I get it wrong? (eg, I overlooked an important word in the argument and chose an answer that is actually wrong if I remember that important word; or, I chose something that sounds true / good in the real world... but that isn't what the argument literally said)

    - Have I MASTERED this problem? Could I explain every aspect, fully, to someone else? (if you can, get together with others to study; explain a problem from start to finish - every single piece. If you don't have study partners, talk out loud and pretend someone's listening to you - if you haven't mastered a problem, it's really hard to do this effectively, even if no one is really listening to you!)

    - How will I RECOGNIZE similar problems in the future? (how will I know, immediately, when I see this type of problem again? How about this particular argument structure or this particular type of wrong answer choice?)

    - During the test I can’t seem to make a connection to my previous experience, or if I do I only have a vague memory, therefore on test day I seem to be doing everything from scratch!

    - During the review of my question, I can only make a basic link to the question topic. I will recognise questions that come from similar topics, but questions are never really that similar. Also I am nervous of trying to memorise questions as I guess I will never see questions I have already seen on test day.

    - I don’t know how to ‘recognize similar problems in the future’. In an effort to ‘remember and use lessons learned from this problem the next time I see a new problem’ I tried to create an error list or flash cards. If I got a ratio question wrong, my first approach was to copy that question out. But it took a long time to copy them out and secondly when I came to reviewing them I wasn’t recalling the method to solve it and it seemed just as hard as before therefore I was just doing it again! Hardly the best way to read a flash card on the way to work! My next approach was to learn the underlying concept. However, when I reviewed the concept a few days later it I was just reviewing a concept and I wouldn’t be able to recall how to apply it? I then tried to read pre-created flashcards, but I feel these aren’t really addressing my specific weakness?

    Therefore after following your excellent approach, how do you recommend putting these methods into your long term memory so you can apply them to new questions on test day?

    RJ7 - Trying to stay positive!!

  • I'd say repetition RJ7
    maybe you could skim the material you've read during the day before you go to bed

    • Hi Slavi,

      Thanks for the email. Good advice. I have given up on the GMAT, because I kept doing so badly...but I haven't given up on the MBA!

      Good luck.

      R

  • ruwan,

    can I give you a piece of advice ?

    I have taken the GMAT last month and scored 380 (no error).
    the funny thing is that I got the same score on GMATPrep software.
    option 1) was to give up the other is 2) to try again while my knowledge is fresh in my head.
    I chose option 2. My GMAT exam is at the end of sept 2010

    I'd advise you not to give up on the GMAT because if you keep procrastinating it you may become a lot more scared (emotions tend to grow if you continue to feed them) in the future because your brain will associate the unpleasant feelings with GMAT.

    I'd say accept the fear and focus on learning specific part of the test .e.g verbal: sentence correction, reading comp etc.

    I've read/heard this quote:
    "Confusion precedes clarity."

    Slavi

  • Hey Stacey,

    First of all, i loved your post. I would request your advice on how to analyze GMATPrep test. I know the other Prep Tests such as Kaplan, MGMT and Princeton give detailed analysis of the test, but GMATPrep doesnt quite do it. And marking questions in categories isn't possible while taking the test itself due to time constraints.

    I took a GMATprep yesterday and am going to analyze it today and tomorrow. How should i do it to as to extract the maximum out of it? One thing i know for sure is that i need massive improvement in timing department because i couldnt even finish the quant section within 75 min and guessed a lot of questions in verbal so as to avoid running out of time.

    Please help....

    Taran

    • It's a lot tougher to analyze GMATPrep. first, if you haven't taken a test yet (or the next time you do), time yourself per question for all of the multiple choice questions. Then, at least you'll have the timing data that a test prep score report would give to you.

      Next, classify the questions into broad categories according to whatever you're studying. Eg, first, you'd just classify into the 5 major question types, and then, if you were using MGMAT, you'd classify according to how we split things up. For example: These 3 questions are all geometry, one's a PS triangle question, one's a DS circle question, and one's a PS coordinate plane question. Build a spreadsheet that organizes roughly according to the MGMAT score reports you've already been using.

      This can allow you to figure out rough strengths and weaknesses according to timing, question type, and question sub-type. You'll have to remember that one major category of data is missing though: difficulty level. There's no way to get that data, so you'll just have to go without it.

      Now, you've got the timing data and you've organized the question types / content areas also by the same categories that you normally use without whatever test prep practice tests you take, so analyze (as much as you can) in the same way. eg, if you're using MGMAT tests, then organize as much as you can that way, and then use the above article to analyze your GMATPrep test as well. Again, you won't have 100% of the same data, but you have quite a bit of useful stuff anyway. :)

  • Hi Stacey, great article. As always, very informative.
    I took a manhattan cat test today and this article will help me evaluate and improve.
    One question, during the practice test, it was very hard for me too keep my focus after half way through in verbal section. I was not able to follow or understand what I was reading. So, I took a 30 minute break and continued. But, I cannot do this in real test. Any suggestions on this? Thanks.

    • Yes, my first suggestion is never to do that again on a practice test. :)

      I put a smiley, but I'm being serious. One of the ways that you build stamina is by pushing yourself through the hard parts - continuing anyway - so you missed an opportunity to build your stamina.

      And more bad news: you cheated yourself, in a sense - you inflated your score by taking a break, and now you don't know what your true scoring level really is. Your scoring level is not just what you know in a vacuum - it's everything, timing, technique, stamina, etc. But you have to assume that, whatever score you got, it's higher than what you're really capable of doing right now. How much? I don't know - a 30 minute break would help some people a ton and others not as much.

      So. What next? Next time you take a practice test, take it under 100% official conditions, including the essays. If you start feeling mental fatigue, have some ways to get your focus back. Take 30 seconds to close your eyes, focus on the ceiling, meditate, breathe deeply, roll your shoulders and feet around, whatever works for you. (These are things you can try while studying in order to figure out what does work for you.) If it's really bad, take 60 to 90 seconds, skip the question (guess randomly) and try to (a) clear your mind, (b) think about some really happy memory that you already have thought about before, (c) review a mental checklist of things you need to know about certain question types, (d) ? (Again, you're going to have to try things and see what works for you.)

      When studying (not taking a practice test), set up your entire study session before you begin - know exactly what you're going to do for the entire 90 minutes to 2 hours. Then start and DO NOT STOP until it's time to take a break (halfway, for 10 minutes). Then start again EXACTLY at the end of the breaktime, and GO again until your time is up. Don't check your email. Don't wonder what you're going to do next and start leafing through some books. GO full out (with one break) for 90-120 minutes.

  • Hi Stacey, iam gorging on your wonderful articles..feel very comfortable with the style of writing as iam used to the manhattan books (duh ur part of manhattan)

    But point being, i faced a similar problem just as PB did. I was just NOT able to comprehend questions at all! for instance, i read a DS question which already said If X > 2 as IS X > 2..this happened for many questions..

    but your right..mental conditioning is key..but i replicate the test conditions by taking the test in an internet cafe..with the actual gmat eraser board etc. But i have not started taking the awa part yet..that scares me..you mentioned elsewhere that NOT taking the AWA part actually inflates the score..so il be taking the awa part frm cat 4 onwards..

    thanks for the useful insights again!

    • I'm glad you like the article. :)

      Yes, one of the hardest things about this test is just understanding what the question is saying in the first place!

  • Hi Stacey,

    Wonderful article. :)

    I've been on and off with my GMAT Prep and am looking at cracking it by mid December. Reading your articles (the error log review as well), I realize that I haven't been doing my analysis pretty well. I've just been madly solving questions after questions without realizing the root causes of my errors much. My scores improved somehow, but now thanks to you, I know how to improve it much more. :)

    Is there any new advise or best methods for reviewing error logs and mocks? Would love to incorporate them and get cracking.

    Keep writing! :)

    • Stacey,

      I've an additional question and I'd like to seek your inputs.

      I have an error log. I review the questions I went wrong (to some extent - not as detailed as you have advised) and search forums and understand my mistake.

      Most of the time when I look at that question again, I'm however unable to solve it properly - because I somehow remember that the answer is option A for that question. This happens more often than not and I feel that my review of the error log is not that productive. What should I do?

Ask a Question or Leave a Reply

The author Stacey Koprince gets email notifications for all questions or replies to this post.

Some HTML allowed. Keep your comments above the belt or risk having them deleted. Signup for a Gravatar to have your pictures show up by your comment.