“Econometrics, Teacher Quality Research, and Policy Levers: Implications for Music Teacher Evaluation”

Ryan D. Shaw, Capital University

Teacher evaluation systems are being revamped at a furious pace. In addition to evaluations becoming more frequent for more teachers, more than 35 states have mandated that student growth/achievement data must be a significant (or the most significant) factor in final teacher ratings (National Council on Teacher Quality, 2013). These data are most often derived through value-added measures (VAMs) based on standardized test scores, or from alternate measures like student-learning objectives (SLOs) for the estimated 70 % of teachers who do not teach in tested subjects/grades (Gill, Bruch, & Booker, 2013; Marion, DePascale, Domaleski, Gong, & Diaz-Bilello, 2012). These changes have come amidst erosion of teacher tenure protections that make assigning teacher effectiveness ratings an especially high stakes endeavor.

While it can seem like these reforms have arrived “out of the blue,” I argue that they were birthed out of a chance “collision” of research and policy. One can trace the development of the teacher effectiveness/quality literature, starting with the Coleman Report (Coleman et al., 1966), which set the stage for debates about the relative importance of student background, school effects, and teacher effects. This literature continued with seminal teacher quality studies that brought variations in teacher effectiveness to the fore (Hanushek, 1971; Murnane, 1975), and more recent econometric research on teacher quality (Clotfelter, Ladd, & Vigdor, 2010; Kane, Rockoff, & Staiger, 2008) and the lasting impacts of teacher effectiveness (Chetty, Friedman, & Rockoff, 2014a, 2014b). This research trajectory can be linked directly to many of the current changes in the way teachers are evaluated.

Because they teach in a “non-tested” subject, music educators have existed on the margins of many recent teacher evaluation conversations. This marginalization has not, however, protected music teachers from the deleterious effects of policy changes. Music teachers increasingly have been evaluated on the basis of building- level VAMs derived from math and reading test scores (Postal, 2012). They have also been asked to design student-learning objectives, which are goal-setting frameworks for subsets of students (Reform Support Network, 2011), even though experts doubt the validity and reliability of SLOs in differentiating teacher quality (Gill et al., 2013).

I will attempt in this presentation to accomplish three tasks. First, I use the Multiple Streams Framework (Kingdon, 1995) to frame a discussion of how teacher effectiveness/quality research influenced current teacher evaluation policy. Second, I discuss changes to teacher evaluation in terms of policy levers and political capital. Finally, I situate music teachers within the contemporary evaluation discourse and offer reflections on the unsettled questions regarding music teacher evaluation.

References

Chetty, R, Friedman, J. N., & Rockoff, J. E. (2014a). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 2593-2632.

Chetty, R, Friedman, J. N., & Rockoff, J. E. (2014b). Measuring the impacts of teachers II: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), 2633-2679.

Clotfelter, C.T., Ladd, H.F., & Vigdor, J. L. (2010). Teacher credentials and student achievement in high school: A cross-subject analysis with student fixed effects. Journal of Human Resources, 45(3), 655-681.

Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., & York, R. L. (1966). Equality of educational opportunity. Washington, D.C.: U.S. Government Printing Office.

Gill, B., Bruch, J., & Booker, K. (2013, September). Using alternative student growth measures for evaluating teacher performance: What the literature says. Washington, D.C.: United States Department of Education.

Hanushek, E. (1971). Teacher characteristics and gains in student achievement: Estimation using micro data. American Economic Review, 61(2), 280–288.

Kane, T. J., Rockoff, J. E., & Staiger, D. O. (2008). What does certification tell us about teacher effectiveness? Evidence from New York City. Economics of Education Review, 27(6), 615-631.

Kingdon, J. W. (1995). Agendas, alternatives, and public policies. New York: Longman.

Marion, S., DePascale, C., Domaleski, C., Gong, B., & Diaz-Bilello, E. (2012). Considerations for analyzing educators’ contributions to student learning in non-tested subjects and grades with a focus on student learning objectives. Dover, NH: National Center for the Improvement of Educational Assessment.

Murnane, R. J. (1975). The impact of school resources on learning of inner city children. Cambridge, MA: Ballinger Publishing.

National Council on Teacher Quality. (2013). State of the states 2013 connect the dots: Using evaluations of teacher effectiveness to inform policy and practice. Retrieved from http://www.nctq.org/dmsStage/State_of_the_States_2013_Using_Teacher_Evaluations_NCTQ_Report.

Postal, L. (2012, November 22). Confused, discouraged teachers file complaints over new reviews. Orlando Sentinel. Retrieved from http://articles.orlandosentinel.com/2012-11-22/features/os-teacher-evaluations-test-confusion-20121122_1_value-added-data-test-score-data-orange-teachers

Reform Support Network. (2011). Targeting growth: Using student learning objectives as a measure of educator effectiveness. Reform Support Network.