Oregon School Data Explorer

Proficiency threshold sensitivity

Ho-style read of Oregon 2024-25 achievement levels. Analysis regenerated: 2026-09-08.

Back to statistical findings Download TXT View source script
Question

Andrew Ho argues that proficiency percentages can be technically correct yet misleading because they summarize a score distribution at one cut score. This report asks what that concern means for Oregon's 2024-25 school achievement analyses, where the main summary outcome is usually Percent Proficient.

Scope
  • Data: Oregon 2024-25 school achievement rows from the processed ELA, math, and science files.
  • Rows: authoritative ODE Total Population All Grades school-subject rows.
  • Filters: charter, virtual, and curated special-enrollment schools excluded.
  • Achievement population: the official combined row's Level 1-4 counts and Scored Performance Denominator.
  • Inclusion threshold: at least 30 students in the official scored population.
  • Attendance correlations require a numeric tested-grade rollup; blank or censored-unavailable attendance is omitted without midpoint or whole-school substitution.
  • ODE inequality disclosures such as <5% use the interval midpoint for computation, with raw tokens and legal bounds retained in the processed data.
Main read

Ho's warning applies, but it does not overturn the broad SES findings. Percent Proficient is Oregon's Level 3 plus Level 4 share, so it hides whether students are concentrated in Level 1 versus Level 2 or Level 3 versus Level 4. When the same data are summarized with a simple four-level index, the neighborhood adult BA+ rate, student poverty, and attendance remain strongly associated with achievement.

Stacked bars showing Oregon 2024-25 achievement levels by subject
Among eligible ordinary-school school-subjects with at least 30 scored Level 1-4 results, Oregon's proficiency rate is Level 3 + Level 4, but the four achievement-level mix differs by subject. Charter, virtual, and curated special-enrollment schools are excluded.
What the four-level index adds

The index assigns Level 1 = 1, Level 2 = 2, Level 3 = 3, and Level 4 = 4, then averages those values using student counts. It is not an official scale score. It is a sensitivity check that asks whether the same SES relationships appear when all four achievement buckets are used instead of only the Level 2/3 proficiency cut. All percentages and analysis weights use students with reported Level 1-4 results.

Scatterplot comparing percent proficient to a four-level achievement index
Most schools follow the same broad pattern, but the vertical spread shows information lost when Levels 1-4 collapse to one cut score.
Context correlations
  • ELA: Percent Proficient correlations are BA+ +0.64, poverty -0.75, attendance +0.56; the four-level index is +0.64, -0.75, +0.58.
  • Math: Percent Proficient correlations are BA+ +0.66, poverty -0.66, attendance +0.69; the four-level index is +0.65, -0.65, +0.70.
  • Science: Percent Proficient correlations are BA+ +0.58, poverty -0.67, attendance +0.43; the four-level index is +0.55, -0.63, +0.52.
Heatmaps showing context correlations for percent proficient, the four-level index, Level 4, Level 1, and Level 2 plus 3 share
The broad SES story survives the index, but Level 4, Level 1, and near-cut concentration emphasize different parts of the distribution.
Stress test

Because the equal-spacing assumption is unproven, the analysis tested alternate scoring rules: extra Level 4 premiums, stronger Level 1 penalties, a compressed middle, and a near-pass-friendly Level 2 score. Across those core alternatives, the SES correlations stayed in the same range: BA+ remained positive, poverty remained negative, and attendance remained positive. The exact index value is not sacred, but the main SES association is not driven by the equal-spacing assumption.

Matched school examples with nearly identical percent proficient but different achievement-level mixes
These diagnostic examples show why individual school comparisons should not rely on Percent Proficient alone.
Practical interpretation
  • Percent Proficient remains useful as a descriptive snapshot, but it should not carry the whole evidentiary load.
  • Statewide SES findings look robust to the four-level index and to alternate level-spacing rules.
  • Individual school comparisons need more caution: pair proficiency with Level 1, Level 4, or stacked Level 1-4 distributions.
  • Trend claims remain the hardest case because school-level public data do not provide scale-score means or percentile points.
Reference

Ho, A. D. (2008). The problem with "proficiency": Limitations of statistics and policy under No Child Left Behind. Educational Researcher, 37(6), 351-360. https://doi.org/10.3102/0013189X08323842