4 Steps to Get the Most out of Your CATs - Part 2
Yesterday we talked about the first two elements of getting the most out of your CATs:
#1: How NOT to use your practice CATs
#2: How to analyze your Strengths and Weaknesses with respect to timing
This week, were going to dive even further into Strengths and Weaknesses using the Assessment Reports.
#3: Run The Reports
Note: this article series is based on the metrics that are given in ManhattanGMAT tests, but you can extrapolate to other tests that give you similar performance data.
In the ManhattanGMAT system, click on the link Generate Assessment Reports. The first time, run the report based solely on the most recent test that you just did; later, well aggregate data from your last two or three tests.
The first report produced is the Assessment Summary; this report provides the percentages correct for the five main Q and V question types, as well as average timing and difficulty levels. Heres an example; see whether you can spot any problem areas. (Click on the image to see a larger version.)
Most people will immediately say Oh, this student is better at PS than DS. Why? Because the percentage correct is higher for PS.
Actually, thats the wrong takeaway for this particular set of results. Its crucial to compare three data points at once: the percentage correct, the time spent, and the difficulty levels.
Its true that the PS percentage correct is significantly higher. But look at that timing: the student is spending a lot longer on PS. DS is averaging well below the normal time of 2 minutes. Further, check out those difficulty levels; the correct answers are at the same level of difficulty!
Its likely that the students DS percentage correct is artificially depressed because hes just not spending enough time. Id tell this student to start looking for careless errorseverywhere, but specifically on DS problems. He needs to learn to cut himself off on the harder PS problems (that hes getting wrong anyway!) and reallocate that time to DS.
These kinds of results should catch your eye:
- Percentages correct below approximately 50%, especially when coupled with lower average difficulty levels and higher average times. Note that I dont consider DS a straight weakness for the above student, even though the percentage correct is below 50%. The other two data points indicate that the real culprit is the timing.
- Average timing that is 30 seconds (or more) higher or lower than the expected average. The above student needs to take a look at timing on Word Problems (too slow) and Reading Comprehension (too fast).
- A big discrepancy (more than 30 seconds) in average time for correct vs. incorrect questions of the same type; its normal to spend a little extra time on incorrect questions (because those are probably the harder ones!), but not a tonthat just means youre being stubborn.
The second and third reports sort the questions by Question Format and Difficulty. Heres an example of the Verbal report.(Click on the image to see a larger version.)
Spot anything?
The student has done a phenomenal job on SC. I am wondering, though, why he was rushing so much on CR and RC. He should check these for careless mistakes. Or maybe mental fatigue was a factor? It might be worth checking out the Problem List again to see if he was speeding up as the test went on (and getting more wrong); thats a classic sign of mental fatigue.
In these two reports, look for:
- average timing that is 30 seconds (or more) higher or lower than the expected average, and whether that is happening on correct or incorrect questions (or both)
- lower percentages correct on lower-level questions than on higher-level questions
In particular, these two things might appear together. If that happens, you might be spending too much time on incorrect higher-level questions and not enough time on lower-level questions (that you then get wrong because youre rushing).
Note: the timing averages for Reading Comprehension can be misleading because the first question for each passage includes the time to read the passage itself. For Reading Comp, you need to dive back into the problem list to look at each problem individually in order to get a true picture of what happened.
Finally, we're going to take a look at the fourth and fifth reports (Quant by Content Area and Topic, Verbal by Verbal Type and Topic). First, though, I suggest that you run the reports based on your last 2 or 3 tests rather than just your last test. Were diving deep into the details with these final two reports, so there will be lots of categories with only one or two questions unless we add more data to the report.
If your last test was more than six weeks ago, though, then the data might be too old. You may also decide to look at these last two assessment reports based just on the last test first, and then run the reports again using your last two testsyour choice. If you use only one test, be aware that your analysis may need to be flexible for those sub-categories with only 1 question. If you get 0% of 1 question right, that doesnt mean that area is a big weakness!
The fourth and fifth reports show all of the questions broken out by question type and sub-type or sub-topic. Youre going to use these reports, coupled with everything youve learned so far, to complete the 4th and final step of your analysis.
#4: Fill Your Buckets
You will have 5 buckets into which you are going to place all of these problems. Some of these buckets are good; others indicate where you need to concentrate your study efforts. Note that the guidelines I give are approximate. If something is only slightly higher or lower than it should be, and you feel comfortable with it, then you can still count that in a good bucket.
Before you dive in, you may want to run the assessment reports again. Often, certain categories will contain only 1 or 2 problems, so you will be able to do better analysis with more data. If youve taken other tests within the past month, include them at this stage. Dont include more than 3 tests (you shouldnt be taking that many within a month anyway!).
Okay, run the program again and go to the fourth and fifth reports.
Bucket 1. I get these right roughly within the expected timeframe (>50% right and neither way too slow nor way too fast).
These are your strengths. Make sure that you actually knew what you were doing for each problem and didnt just get lucky! Going forward, theyre not high on your priority list, but there may still be things you can learn:
- faster ways to do the problem
- ways to make educated guesses (so that you can use the thought process on harder problems of the same type)
- how to quickly recognize future problems of the same type.
You may want to move on to more advanced material in these areas.
Bucket 2. I get these wrong roughly within the expected timeframe (<50% right and neither way too slow nor way too fast).
These indicate a possible weakness in content or methodology, but check the difficulty levelsperhaps you just happened to get a couple of really hard ones in the same category.
First, figure out why you got each question wrong. If it was a high difficulty level, you got another lower-ranked question of the same type right, and you were fine with these on your last test, then your fundamentals may be good, and it may be time to lift your study into tougher areas for this particular question type or content area.
Alternatively, maybe you did know the material but you made careless mistakes. Figure out what mistakes you made and how you can minimize the chances of repeating that type of error.
Finally, something in this category may indicate a fundamental weakness. Is the material something you already studied? Return to it. Have you not studied it yet? Thats okaybut maybe its time to start. Is the material commonly or rarely tested? Prioritize the commonly tested material first. As needed, return to the relevant sections of your books.
Bucket 3: I get these wrong way too quickly (more than 30 seconds faster than expected)
Why were you going too fast on these?
If you chose to rush because you knew you didnt know what to do (in other words, you made a guess and moved on), thats a great outcome. Decide now whether you want to study this area further or continue to get these wrong fast. Everyone should have a few my goal is to get these wrong fast areas (mine are Combinatorics and 3D geometry).
If you chose to rush because you thought it was easy and then you made a careless mistake, add these to your error log, figure out what mistake you made, and remind yourself not to sacrifice a correct answer just to save 30 seconds!
Alternatively, if you were forced to speed up because you were low on time, then you need to fix your timing problems elsewhere in the section.
Bucket 4: I get these right way too slowly (more than 30 seconds slower than expected)
These are still weaknesses; it doesnt matter that youre getting them right! Theyre costing you points elsewhere in the sectionpossibly more points than you earned by getting the too-slow ones right.
First, did you actually know what you were doing or did you get lucky? If you got lucky, move the problem to Bucket 5.
Figure out why the timing is higher and whether and how you can do these more efficiently. If the timing is just a little bit too high on one problem of that type, that may be okayperhaps the problem is extra hard and long.
If you're consistently going long, however, then figure out or research (a) more efficient solutions, or (b) the best way to recognize that this problem requires a certain set of steps, or (c) both. Dont forget that, sometimes, the solution is actually to guess faster and move on. It's usually better to get something wrong in 2 minutes than right in 4 minutes (because of the eventual consequences).
Bucket 5: I get these wrong way too slowly (more than 30 seconds slower than expected)
These are the biggest weaknesses, obviously. Get them wrong faster. Im entirely serious. You're getting them wrong anyway, so take less time to get them wrong! Re-allocate that time to questions from one of the other buckets, where additional time is more likely to make a difference.
What is slowing you down? The short answer is that youre being stubborn: you likely knew you couldnt do the problem but you didnt want to let go. Go back to the first half of this article and read (or re-read!) the time management articles.
You may eventually be able to learn how to do some of these correctly and within normal time, but set these aside for now and concentrate on Buckets 2, 3 and 4. Dont forget to have a few My goal is to get problems like this wrong fast categories.
One more thing: frequency
For all of the above, dont forget to think about the frequency with which the material is tested. You might be terrible at 3D geometry (as I am), but that category is so rare that its not even worth studying. If, on the other hand, youre also not so great at exponents and roots, you need to start studying; those topics are common. (If youre not sure what is more or less frequently tested, ask your teacher or ask on the forums.)
The 4 Steps to Getting the Most out of your CATs
(1) Stop Taking So Many CATs. Get a minimum of 2 weeks worth of study out of each one.
(2) Learn your strengths and weaknesses. Start with an analysis of your timing, since bad timing will kill your score no matter how good you are with the actual material.
(3) Run the reports to dive into the content and question types. Its critically important to evaluate your performance across all three main axes at oncepercentage correct, timing, and difficulty.
(4) Fill Your Buckets. Concentrate on Buckets 2, 3 and 4. Every time you take a test, youll have new Buckets; youll be able to see your progress and adjust accordingly.