Researching National Merit Cutoffs

Below is an in-depth exploration of the research we have done to understand the relationship between the Commended Student cutoff and the NMSF cutoffs. To go straight to the estimates, scroll to the bottom of this page or visit the regularly updated National Merit Semifinalists cutoffs post. An earlier post explains why we predicted an increase in the cutoffs this year and explores some of the reasons behind the rise. As of April 2017, we know that the Commended Student cutoff for National Merit has risen from a Selection Index of 209 to an SI of 211. The 2-point rise  in the Commended level means that a majority of states will see increases in their Semifinalist cutoffs. Will NMSF  cutoff increases  be clustered around the 2-point mark, or will they be skewed higher or lower?  How one interprets the historical data  makes a significant difference in what sort of distribution to expect. The Historical Record National Merit Semifinalist cutoffs for the old PSAT cannot be directly compared to those for the Class of 2017 and later. The test changed, the scoring changed, and the scale changed.   We can use the historical NMSF cutoffs in one of two ways: 1) We can look at the old PSAT data in isolation and try to understand how Commended level and state cutoff changes were correlated. 2) We can try to convert older cutoffs onto the new scale and then look at the data across all class years. Both paths have limitations. The first option doesn’t allow us to easily translate the findings into how scores look on the new, compressed range of Semifinalist cutoffs. The second option introduces the bias of the conversion factor. We based our conversion methodology on how Class of 2016 scores translated to Class of 2017 scores. This puts a lot of weight on scores (those from October 2015) that were clearly still a work in progress. Pathway 1: Old PSAT Cutoffs Below are the Commended cutoffs for the classes of 2009 to 2016. Even on the old exam, scores could rise and fall in unexpected ways because of scoring inconsistencies (see, in particular, the 2012-2014 period). Below is a summary of the state (and DC) Semifinalist cutoff changes versus the Commended changes. The only one-year period in which the Commended level remained unchanged was 2009-2010. In the 0 column, each number for a row corresponds to how many states saw a particular NMSF cutoff change. For example, 17  states saw no change, 11 states went up 1 point, 11 states went down 1 point, and the rest saw larger increases or decreases. The 2009-2010 period reflects how evenly things are distributed when there is no change in the Commended level. It also illustrates the danger in expecting that a single estimate is going to be correct. Despite the fact that there was no Commended change, and despite the fact that there was a clear mode (most common value), only 1/3 of the time (17/51) was there no change in cutoff. A prediction that a state cutoff would not change would have been wrong in 2/3rds of the cases. Understanding Distribution of Changes Things grow still murkier when there is a change in the Commended level. In both the 2010-2011 and 2015-2016 periods, the Commended cutoff went up 1 point. Of the 102 state and DC Semifinalist cutoff changes, only 22 were exactly 1 point. That means that 78% of cutoffs changed more or less than did the Commended figure. In fact, more states saw no change (24) than saw 1-point increases (22). Pathway 2: Restated PSAT Cutoffs One of the difficulties in drawing conclusions from how scores shifted on the old PSAT is that the range of â€Å"usable† Selection Indexes is far narrower on the new SAT. A 3-point change on the old PSAT should not automatically be equated to a 3-point change on the new scale. In the chart below, the 51 state and DC cutoffs for the Class of 2016 and the Class of 2017 are sorted from low to high. [Also included are the estimates for 2018.] The range from bottom to top (209-222) on the new PSAT is about half that seen on the old test (202-225). The change was also not uniform. The lowest cutoffs went up 7 points, whereas the highest cutoffs declined by 3 points. A mapping of the two sets of figures was used to restate historical figures onto the new scale. Below are the restated Commended levels. In theory, this restatement of scores should allow us to view prior year changes in the context of the 2017-2018 period. The data table below extends the year-over-year analysis to include comparisons across two and three years. For example, it’s obvious that there was a 1-point change between 2010 and 2011 and between 2015 and 2016, but the same could be said when comparing 2009 and 2011 or 2015 and 2017 (among others). The reference periods were limited to 3 years in an attempt to avoid measuring long-term trends. Below is the summary of Semifinalist cutoff changes versus the Commended changes when restated as new PSAT Selection Indexes: Changes in the Commended cutoff, in this model, show less of an impact on Semifinalist cutoffs. While the 0 column  shows a symmetric distribution, a 2-point increase shows most cutoffs going up 1 point or less. One way of visually summarizing the relationship between a Commended change and the ensuing state cutoff changes is to plot the Commended change against the average cutoff change. The  graph of average scores for the old PSAT and for the restated PSAT scores shows greater regression to the mean on the latter. Given the 2-point change this year, a muted impact on state cutoffs is exactly what most students are hoping for. It’s unclear whether this regression is a legitimate difference caused by scale compression on the new PSAT or whether this is an unintended artifact of remapping old PSAT scores based on a single class results. I’ve tried to present as much explanation as possible so as not to pretend that there is a magical â€Å"best score† predictor. Cutoffs at the Commended Level While no state’s cutoff should be considered a sure bet, there is a narrower set of possibilities at the low and high ends of the range. States with cutoffs at 209 for the Class of 2017 (North Dakota, South Dakota, West Virginia, and Wyoming) are most likely to move to 211 (they cannot go any lower). However, each of those states has, at some point in the last decade, had a cutoff above the Commended level. It would not be surprising to see at least one of those states move to a higher cutoff or to see another state join them at 211. The  estimated ranges reflect this uncertainty. Cutoffs at the Highest End of the Range The scoring of the new PSAT makes the highest Selection Indexes extremely difficult to achieve. Top marks require not just great skills; they require a lucky combination of scores. For example, there are only two combinations of scores that produce a Selection index of 225 — 760 EBRW / 730 Math and 750/750. In fact, there are only 12 combinations of scores that determine Selection Indexes from 223 to 228. The air gets thin quickly. Last year’s numbers seemed to establish a practical limit of 222-223 for a Semifinalist cutoff. While an increase to 224 for MA/NJ/DC seems unlikely, it is premature — based on a single year — to rule out such a change. Most Likely Scores I am expecting the highest frequency of changes to be +1 and +2. Choosing between +1 and +2 to determine a â€Å"most likely† score is often arbitrary (think ranges!). Among the considerations: When, and at what level, did a state achieve its previous high? What is its average cutoff in recent years? How have its cutoffs differed from the Commended level in the same year? How has it ranked against other states? Have its cutoffs bounced around or been stable? An argument can be made for any score across the estimated range, and each year usually brings at least 1 or 2 surprises. The table below is the same one found on National Merit Semifinalist Cutoffs, but that page is more likely to have updated notes when new information becomes available. You can find all of our National Merit posts here. Please feel free to contribute to the discussion. Class of 2017 Semifinalist Cutoffs  and Estimated Range for Class of 2018 StateClass of 2017 ActualClass of 2018 Est. RangeClass of 2018 Most Likely Alabama215214-218216 Alaska213212-216215 Arizona219218-221220 Arkansas213212-215214 California221220-223222 Colorado218217-221219 Connecticut220220-222221 Delaware218217-221219 District of Columbia222222-224223 Florida217216-220218 Georgia219218-221220 Hawaii217216-220219 Idaho214213-217216 Illinois219218-221220 Indiana217216-220219 Iowa215214-217216 Kansas217216-220219 Kentucky215214-218216 Louisiana214213-217216 Maine214214-218216 Maryland221220-223222 Massachusetts222221-224223 Michigan216215-219217 Minnesota219218-221220 Mississippi212211-215214 Missouri216215-219217 Montana210211-214212 Nebraska215214-218216 Nevada214213-217216 New Hampshire216215-219218 New Jersey222222-224223 New Mexico213212-216215 New York219219-222220 North Carolina218217-221219 North Dakota209211-213211 Ohio217216-220219 Oklahoma213212-216215 Oregon219218-221220 Pennsylvania218217-221219 Rhode Island217216-220219 South Carolina215214-218216 South Dakota209211-213211 Tennessee218216-220219 Texas220219-222221 Utah215214-217216 Vermont215215-219217 Virginia221220-223222 Washington220220-222221 West Virginia209211-213211 Wisconsin215214-218216 Wyoming209211-213211 ​U.S. Citizens Studying Abroad222222-224223 ​U.S. Territories209N/A211