Abstract: I wrote this brief in response to the ridiculous amount of reading I had to do about data and high stakes testing analysis. I short, I argue that data and test analysis is important, but it has become an obsession, and American school reform is reaping diminishing returns in regards, "...the amount of analysis put in verses what the analysis will
actually give us, which I would argue is more of the same."
-------------------------------
In Murnane and Levy’s piece
entitled Teaching the New Basic Skills,
they point out a technique that companies, such as Honda and Mitsubishi, use to
promote certain applicants to the next level of interviews. Murnane and Levy write, “…once reading
and math scores…are above a certain threshold, the soft skills – teamwork and
communications skills – are the best predictors of performance.” In other words, Murnane and Levy point
out an important principle for vocational productivity: one must reach a
certain threshold of performance in a skill set in order to perform, but
anything above and beyond that threshold is not an indicator of
performance. Moreover, the more a
person performs over the threshold in any expendable skill will lead to diminishing
returns in regards to the amount invested in that expendable skill versus that
lack of product the expendable skill will produce.
Malcolm Gladwell, in his book
Outliers, writes about this same idea.
He tells the story of a researcher that followed a group of ‘geniuses’
from adolescence to adulthood, and what this researcher found illustrates Murnane
and Levy’s point. Although the
group of geniuses had an IQ far greater than average man or woman, their
accomplishments were marginal.
Some geniuses did succeed vocationally and became lawyers, doctors, and
statesmen; however, other geniuses became janitors and bouncers. Thus, this researcher determined that having
a high IQ was not the golden ticket to vocational success that he
anticipated. Similar to Murnane
and Levy’s argument, one only has to have a certain level of intelligence (ie.
a threshold level) in order to succeed, and anything above that threshold has
the potential to reap diminishing returns.
It
is here that I make my first critical judgment of the current American
educational obsession with data. Data
and high stakes testing are important.
Anne Lewis, in her Mid-Continent Research For Education and Learning policy
brief, writes, “Even the
severest critics of high-stakes testing acknowledge that assessments are
necessary for a variety of purposes.”
This is a fact of life.
However, I would argue that we have pushed passed the threshold, and our
obsession with high-stakes testing and data has – like an auto technician with
a 145 IQ – resulted in diminishing returns. In Boudett, City, and Murnane’s book Data Wise, they evidence the aforementioned idea clearly when they
write 27 pages of assessment literacy that outlines the different forms of high
stakes testing that currently exists – including norm-referenced tests,
criterion-referenced tests, and standards-referenced tests among others. Familiarity with these types of tests,
their pros and cons is useful; nonetheless, I perceive an obsession, since
there exists a superfluous cornucopia of various types of data collection. Last year in April on the two Friday
afternoons preceding North Carolina’s high-stakes test, the charter school
where I previously worked dedicated 4 hours of PD to analyze the variability of
individual students in regards to standards referenced tests. Although this is only one isolated
experience, it suggests something of the whole; namely that the American
educational system is beyond the threshold of the utility for data and high
stakes testing analysis. Test and
data analysis is only one body of evidence that can be used to support the
coherence between the instructional core (ie. the relationship between the teacher,
student, and content); yet it is the one element of evidence that overshadows
all others in the current state of reform here in America. In doing so, it reaps diminishing
returns in terms of the amount of analysis put in verses what the analysis will
actually give us, which I would argue is more of the same.
Boudett,
K., City, E., and Murnane, R.
(2005) Data Wise: A
Step-byStep Guide for Using Assessment Results to Improve Learning. Cambridge, MA: Harvard Education Press.
No comments:
Post a Comment