First Impressions Matter: The Performance of Music Teacher Candidates on Prescreening Interview Instruments
Ryan D. Shaw, Capital University
Amidst the considerable research attention devoted to the various dimensions of teacher quality over the last 50 years, scholars have debated raising entrance requirements and/or exiting ineffective teachers based on value-added measures (Hanushek, 2009). However, the viability of such suggestions rests on who would fill vacant teaching positions. Scholars have differed on whether the “teacher pipeline” is robust enough to ensure that less effective teachers would be replaced with average or highly effective teachers (Dee & Wyckoff, 2015; Rockoff, Jacob, Kane, & Staiger, 2011; Staiger & Rockoff, 2010).
The teacher hiring process is a site of scrutiny for those studying the teacher pipeline. While much teacher hiring is localized and focused on semi-structured interviews, many larger districts have made use of commercial prescreening interview instruments such as the Star Teacher Pre-Screener (Haberman, 1995), Ventures for Excellence (now HumanEx Ventures) interview and Gallup’s Teacher Perceiver Interview (TPI). While vendors claim to have validated the instruments using a measure of predictive validity (often based on observed characteristics of a group of excellent teachers), peer-reviewed evidence is almost impossible to find or suggests a lack of predictive validity (Ebmeier & Ng, 2005; Metzger & Wu, 2008). Following scholarship that has investigated the validity of various measures of teacher quality/effectiveness for music educators (see Elpus, 2015; Parkes & Powell, 2015; Maranzano, 2000; Taebel, 1990), a next logical step would be to examine the validity of prescreening instruments for music educators, since these are a pervasive measure controlling the music teacher pipeline. The purpose of this study was to investigate the performance of music teacher job candidates on prescreening interview instruments in five Midwestern school districts. Research questions were:
• How do music teacher job candidates perform in relation to educators from other licensure areas on several common prescreener interview instruments?
• What are human resources (HR) professionals’ perceptions of the areas of prescreening interview instruments that may positively or negatively affect music teacher candidates’ performance?
This study employed a multiple case study design (Stake, 2006), investigating five Midwestern school districts that used different combinations of commercial prescreeners, and data sources included interviews with human resources (HR) professionals involved in hiring certified personnel for each of the district, as well as numerical examples of performance on the screening instruments.
Findings suggested that prescreener instruments revealed differences between primary and secondary licensure areas, with participants noting that primary candidates scored higher in general, especially on areas addressing a teacher’s empathy and compassion. Music candidates were felt to mirror secondary candidates in performance, and participants suggested that prospective music educators operated with a “coach” mentality. Participants felt that music teachers had difficulty discussing individual student development and instead focused on policies and directing a large program. Music teacher candidates were thought to excel on components addressing problem-solving, creativity, and mastery. In addition, participants discussed ways in which they personalize or “flex” prescreener instruments depending on licensure type and applicant pool. The study has important policy implications for those studying music teacher preparation, quality, and personnel issues.
Boyd, D., Goldhaber, D. D., Lankford, H., & Wyckoff, J. H. (2007). The effect of certification and preparation on teacher quality. The Future of Children, 17(1), 45-68.
Brinkman, D., & Mallett, C. (2000). Questions asked in the job interview of music teachers. Journal of Music Teacher Education, 10(1), 16–23.
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014a). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104, 2593-2632. http://dx.doi.org/10.1257/aer.104.9.2593
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014b). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104, 2633-2679. http://dx.doi.org/10.1257/aer.104.9.2633
Clemons, B. L. (2010). A correlational study of the ventures for excellence interview-rating system and first year teacher evaluations (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI: 3389740)
Danielson, C. (2013). 2013 Framework for Teaching evaluation instrument. Retrieved from https://www.danielsongroup.org/framework/
Dee, T. S., & Wyckoff, J. (2015). Incentives, selection, and teacher performance: Evidence from IMPACT. Journal of Policy Analysis and Management, 34, 267-297. doi:10.1002/pam.21818
Ebmeier, H., & Ng, J. (2005). Development and field test of an employment selection instrument for teachers in urban school districts. Journal of Personnel Evaluation in Education, 18, 201-218. doi:10.1007/s11092-006-9021-4
Edgar, S. (2012). Communication of expectations between principals and entry-year instrumental music teachers: Implications for music teacher assessment. Arts Education Policy Review, 113, 136–146. doi: 10.1080/10632913.2012.719426
Elpus, K. (2015). Music teacher licensure candidates in the United States: A demographic profile and analysis of licensure examination scores. Journal of Research in Music Education, 63, 314-335. doi:10.1177/0022429415602470
Gallup Organization Education Division. (n.d.). TeacherInsight assessment. Retrieved from https://gx.gallup.com/teacherinsight
Goldhaber, D. (2007). Everyone's doing it, but what does teacher testing tell us about teacher effectiveness? Journal of Human Resources, 42, 765-794.
Goldhaber, D., & Anthony, E. (2007). Can teacher quality be effectively assessed? National Board Certification as a signal of effective teaching. The Review of Economics and Statistics, 89, 134-150. doi:10.1162/rest.89.1.134
Goldhaber, D., Crout, C., & Huntington-Klein, N. (2016). Screen twice, cut once: Assessing the predictive validity of applicant selection tools. Education Finance and Policy, 12, 197-223. doi:10.1162/EDFP_a_00208
Haberman, M. (1995). Star teachers of children in poverty. Milwaukee: Kappa Delta Pi Publishers.
Hanushek, E. (2009). Teacher deselection. In D. Goldhaber & J. Hannaway (Eds.), Creating a new teaching profession (pp. 165-180). Washington, DC: Urban Institute Press.
Hanushek, E., & Rivkin, S. G. (2006). Teacher quality. In E. Hanushek & F. Welch (Eds.), Handbook of the economics of education (pp. 1051-1078). Amsterdam: North Holland.
Harris, D. N. (2009). Would accountability based on teacher value-added be smart policy? An examination of the statistical properties and policy alternatives. Education Finance and Policy, 4, 319-350. doi:10.1162/edfp.2009.4.4.319
Harris, D. N., Ingle, W. K., & Rutledge, S. A. (2014). How teacher evaluation methods matter for accountability: A comparative analysis of teacher effectiveness ratings by principals and teacher value-added measures. American Educational Research Journal, 51, 73–112. doi:10.3102/0002831213517130
Juchniewicz, J. (2015). An examination of music teacher job interview questions. Journal of Music Teacher Education, 26(1), 56-68. doi:10.1177/1057083715613590
Kane, T. J., Rockoff, J. E., & Staiger, D. O. (2008). What does certification tell us about teacher effectiveness? Evidence from New York City. Economics of Education Review, 27, 615-631. doi:10.1016/j.econedurev.2007.05.005
Kirchner, J., Evans, S. C., & Norman, A. D. (2010). Examining the relationship between two assessments of teacher effectiveness. Action in Teacher Education, 32, 73–81. doi:10.1080/01626620.2010.10463544
Maranzano, C. (2000). Music teacher performance evaluation: A call for more inclusive models. Journal of Personnel Evaluation in Education, 14, 267–274. doi:10.1023/A:1008152400957
Metzger, S. A., & Wu, M. J. (2008). Commercial teacher selection instruments: The validity of selecting teachers through beliefs, attitudes, and values. Review of Educational Research, 78, 921–940. doi:10.3102/0034654308323035
National Association for Music Education. (2015a). Workbook for building and evaluating effective music education in general music. Reston, VA: Author.
National Association for Music Education. (2015b). Workbook for building and evaluating effective music education in the school ensemble. Reston, VA: Author.
Nelson, B. (2013). Exploring the relationship between the Ventures for Excellence Teacher StyleProfile Data and teacher performance (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No: 3605257)
Novotny, M. T. (2009). The relationship between TeacherInsight(TM) scores and professional development and appraisal system domain scores (Doctoral dissertation). Retrieved from ProQuest Dissertation and Theses database. (Accession No: ED532419)
Oestreich, T. A. (2016). A study of the effectiveness of the AppliTrack TeacherFit hiring selection tool when compared with a teacher’s summative evaluation (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database.
Parkes, K. A., & Powell, S. R. (2015). Is the edTPA the right choice for evaluating teacher readiness?. Arts Education Policy Review, 116, 103-113. doi:10/1080/10632913.2014.944964
Raiber, M., & Teachout, D. (2014). The journey from music student to teacher: A professional approach. New York: Routledge.
Rockoff, J. E., Jacob, B. A., Kane, T. J., & Staiger, D. O. (2011). Can you recognize an effective teacher when you recruit one? Education Finance and Policy, 6, 43-74. doi:10.1162/EDFP_a_00022
Rutledge, S. A., Harris, D. N., & Ingle, W. K. (2010). How principals “bridge and buffer” the new demands of teacher quality and accountability: A mixed‐methods analysis of teacher hiring. American Journal of Education, 116, 211–242. doi:10.1086/649492
Schmidt, M. (2013). Transition from student to teacher: Preservice teachers’ beliefs and practices. Journal of Music Teacher Education, 23(1), 27-49. doi:10.1177/1057083712469111
Staiger, D. O., & Rockoff, J. E. (2010). Searching for effective teachers with imperfect information. Journal of Economic Perspectives, 24(3), 97-117. doi:10.1257/jep.24.3.97
Stake, R. E. (2006). Multiple case study analysis. New York, NY: Guilford Press.
Taebel, D. K. (1990). An assessment of the classroom performance of music teachers. Journal of Research in Music Education, 38, 5–23. doi:10.2307/3344826
Winters, M. A., & Cowen, J. M. (2013). Would a value-added system of retention improve the distribution of teacher quality? A simulation of alternative policies. Journal of Policy Analysis and Management, 32, 634-654. doi:10.1002/pam.21705
Young, I. P., & Delli, D. A. (2002). The validity of the teacher perceiver interview for predicting performance of classroom teachers. Educational Administration Quarterly, 38, 586– 612. doi:10.1177/0013161X02239640