Annotated Bibliography for Standards Implementation Matrix and Scaling Up Education Reforms

A Primer on Coburn’s Four Dimensions of Scale as it Applies to the Standards Implementation Matrix

Guide to topical descriptors:

  • Reform theory — An entry tagged with reform theory postulates the logic or underlying principles of an educational reform, and/or describes why reforms appear to thrive or fail.
  • Reform impact — Describes how a particular reform has unfolded in a specific context; includes case study examinations of a reform-in-progress, or broad looks at reform progress to date.
  • Scaling reform — Specifically addresses implications of scale and the conditions associated with it, empirically or theoretically.
  • Teacher learning — Addresses how teachers learn to change their practice in the context of educational reform, or how that learning can (or should) be supported to achieve reform goals.
  • Strategies for reform — Articulates a vision or a framework for implementing a desired educational change.

Anson, R. J. (Ed.). (1994). Systemic reform: Perspectives on personalizing education. Washington, DC: U.S. Office of Education, Office of Educational Research and Improvement.

In this collection of eight papers, the authors tackle the topic of how change strategies in educational reform can aspire to pay more attention to the individuals responsible for moving change forward within a complex schooling system. Among the most relevant in this volume are Fullan’s "Coordinating Top-Down and Bottom-Up Strategies for Educational Reform," on the need to blend features of centralized and decentralized reform efforts; Stiegelbauer’s "Change Has Changed: Implications for Implementation of Assessments from the Organizational Change Literature," on the importance of engaging individual actors in school reform, who must be able to articulate their role within it; Wohlstetter, Smyler, and Mohrman’s "New Boundaries for School-Based Management: The High Involvement Model," on developing shared systems of power within schools that encourage intrinsic investment in the proposed change; and Little’s "Teachers’ Professional Development in a Climate of Educational Reform," on the mismatch between reform aspirations and the professional learning models provided to teachers.

Reform theory Strategies for reform Teacher learning


Bryk, et al. (2015). Learning to improve: How America’s schools can get better at getting better. Cambridge, MA: Harvard Education Press.

Abstract: As a field, education has largely failed to learn from experience. Time after time, promising education reforms fall short of their goals and are abandoned as other promising ideas take their place. In Learning to Improve, the authors argue for a new approach. Rather than "implementing fast and learning slow," they believe educators should adopt a more rigorous approach to improvement that allows the field to "learn fast to implement well." Using ideas borrowed from improvement science, the authors show how a process of disciplined inquiry can be combined with the use of networks to identify, adapt, and successfully scale up promising interventions in education. Organized around six core principles, the book shows how "networked improvement communities" can bring together researchers and practitioners to accelerate learning in key areas of education. Examples include efforts to address the high rates of failure among students in  community college remedial math courses and strategies for improving feedback to novice teachers.

Strategies for reform


Calvert, L. (2016). Moving from compliance to agency: What teachers need to make professional learning work. Oxford, OH: Learning Forward and NCTAF.

According to Calvert, most teacher professional development fails due to insufficient focus on cultivating teachers’ agency in their own learning—in other words, "the capacity of teachers to act purposefully and constructively to direct their professional growth and contribute to the growth of their colleagues." Teachers must be encouraged to set their own learning goals, plan the learning activities that will help them achieve those goals, and turn to each other for solutions. This brief asserts that teachers by and large hunger for mastery and they desire to improve their practice, but the conditions must be in place for them to direct their own learning in pursuit of self-identified goals, or else professional development devolves into an exercise in compliance.

Strategies for reform; Teacher learning


Cannata, M., Rutledge, S., Redding, C., Smith, T., & Rubin, M. (2017, April).  Using continuous improvement approaches to achieve scale: Implications for depth, spread, ownership, and sustainability.  Paper presented at the annual meeting of the American Educational Research Association, San Antonio, TX.

Abstract: Recent history of school reform has numerous examples of promising programs that are developed and shown to have a positive impact on student outcomes, but encounter significant challenges regarding scale and sustainability. These challenges include building teacher buy-in, attending to the organizational context in which innovations are enacted, and aligning new practices with the broader array of programs that exist in schools. This paper is situated within the broader context of identifying how to more effectively scale and sustain effective practices. By both enacting a new approach to scale and studying how key principles of the approach contribute to scale, this paper informs how improvement science and design-based implementation research can lead to meaningful changes in schools. We find that an improvement model that emphasizes co-construction of practices in with practitioners, continuous improvement, and an authentic partnership between researchers, developers, and practitioners contributed to building internal ownership of the innovations, but encountered dilemmas over depth, spread, and sustainability.

Strategies for reform Scaling reform


Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3–12.

Coburn lays out a vision for conceptualizing "scale" in educational reform that goes beyond pure numerical spread of a policy or program. In her view, measures of scale that look only at how many sites or locales adopted a promising reform overlooks other qualitative elements that may be important to assessing whether or not the intended changes have been allowed to flourish. Specifically, she articulates four "dimensions" of scale: depth, sustainability, spread, and shift in reform ownership. The last of these, ownership, represents an essential goal to lasting improvement in teaching and learning, referring to educators’ ability to internalize reform principles over time. This is a particularly useful lens for examining today’s reform efforts, many of which depend upon teacher buy-in, yet often fail to gain traction if teachers perceive changes to be pushed on them from the outside.

Scaling reform


Coburn, C. E., Hill, H. C., & Spillane, J. P. (2016, May). Alignment and accountability in policy design and implementation: The Common Core State Standards and implementation research. Educational Researcher, 45(4), 243–251.

Abstract: Both the Common Core State Standards (CCSS) and recent efforts to hold schools and teachers accountable have been hotly debated among practitioners, policymakers, and the public at large. Much of the debate centers on the merits and demerits of these initiatives and the general approach they represent to reforming teaching and learning. In this article, we focus on a different issue, that is, the opportunity to advance research on policy implementation afforded by the intertwined nature of CCSS and accountability efforts. Arguing that it is essential for stakeholders, regardless of their stance on either reform, to understand whether and how both influence classroom teaching and learning, we outline elements of a research agenda to generate knowledge important to the design of future instructional policies. For this to happen, we argue that an implementation research agenda needs to build on (rather than reinvent) lessons learned from the past quarter century of implementation scholarship on instructional policy. To that end, we review theoretical and empirical insights from implementation research on standards-based reform and outline specific avenues for potential theory testing research on educational policy implementation.

Reform theory


Cohen, D. K., & Ball, D. L. (2001, September). Making change: Instruction and its improvement. Phi Delta Kappan.

Cohen and Ball point out that many school improvement efforts, such as the adoption of a new textbook or curriculum, do not adequately focus on the dynamic aspects of instruction that are necessary for coordinating and implementing the desired change. For this reason, reforms are quickly swept aside to make room for the next intervention, or they result in such variable adoption that it is impossible to gauge the impact of the reform. Policy and program changes should pay attention to how the intervention influences the instructional interactions between teachers and students, including how teachers’ own learning about the intervention is made accessible to them.

Reform theory Teacher learning


Cohen, D. K., & Barnes, C. A. (1993). Pedagogy and policy. In D. K. Cohen, M. W. McLaughlin, & J. E. Talbert (Eds.), Teaching for understanding: Challenges for policy and practice (Chapter 7, pp. 207–239). San Francisco, CA: Jossey-Bass Publishers.

The history of reform is filled with seemingly well-meaning policies deployed en masse throughout the nation’s schools with little regard for how the chief enactors of those policies—teachers—should learn about them. Cohen and Barnes call this the problem of "the pedagogy of education policy." While teachers tend to make their own learning out of new policy paradigms, policymakers have tended to overestimate the potential of their proposed reforms to transform teaching and learning, and underestimate the need to educate teachers to carry out those reforms. Exuberant faith in the "right" curriculum materials, in the equalization of resources, and in the implicit ability of students to learn has often resulted in overlooking the learning needs of teachers themselves as a prerequisite to any ambitious reform agenda.

Teacher learning


Cohen, D. K., & Hill, H. C. (2000, February). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record, 102(2), 294–343.

Cohen and Hill assert that teachers’ opportunity to learn about instructional policy is a crucial piece in the logic of reform which posits that changes in policy can influence student performance. Their findings lead them to suggest that such learning should be grounded in the curriculum that students study, but also touch other elements of instruction such as assessment, and learning opportunities should be spread over time. Though seemingly intuitive, professional development has rarely paid attention to the need for teachers to learn deeply about the academic content of new instructional policies, such that teacher learning mirrors what students are expected to learn as well.

Teacher learning


Elmore, R. F. (1996). Getting to scale with good educational practice. Harvard Educational Review, 66(1), 1–26.

This seminal piece by Elmore describes the central challenge involved in scaling educational reform: namely, that isolated changes in teaching and learning practices rarely attempt to affect the core relationship of knowledge-transfer between teachers and students, and when they do, the cultural norms and incentive structures in schools by and large render them incapable of adopting and extending new ideas. For these reasons, despite pockets of great success throughout the U.S. system, it has proven virtually impossible to replicate promising practices elsewhere. Elmore concludes that large-scale attempts to reform teaching and learning will never fulfill their goals of lasting, effective change unless the prevailing organizational logic of schools is restructured to promote the acquisition of new ideas and the appropriate incentives to support them.

Reform theory Scaling reform


Fullan, M. G. (1994). Coordinating top-down and bottom-up strategies for educational reform. In R. J. Anson (Ed.), Systemic reform: Perspectives on personalizing education (pp. 7–22). Washington, DC: U.S. Office of Education, Office of Educational Research and Improvement.

Fullan, citing the failure of both top-down and bottom-up approaches to meaningfully effect educational change, argues for a middle-of-the-road approach that allows local systems to pursue a common framework established by a centralized authority, but to do so by flexibly managing their own change processes. Centralized and decentralized strategies must be coordinated, according to Fullan, so as to capitalize on the strengths of each: perspective and resource support, for example, from a central entity (like a district), and creativity and responsiveness, for example, from local entities (schools). Over time, a new culture will emerge as a result of patterns in local change-making under a coordinated framework, putting pressure on less effective entities to conform to the newer, more successful characteristics of the system. Change, then, comes about as a result of numerous individual efforts that accumulate into a shared culture.

Reform theory


Fuhrman, S. H. (Ed.). (2001). From the capitol to the classroom: Standards-based reform in the states. One Hundredth Yearbook of the National Society for the Study of Education, Part II. Chicago: University of Chicago Press.

This volume describes the known effects of standards-based reforms some ten years on from when standards policies swept the nation and become commonplace across the states. The various essays collected here include discussions of the theory of accountability in standards reform, the effects of implementation in classrooms and on student achievement, the district response to reforms, how schools and teachers have responded to accountability policies, and progress to date. For example, in Wilson and Floden’s "Hedging Bets: Standards-based Reform in Classrooms," the authors describe how new standards-based expectations neither managed to fully transform the culture of teaching and learning at schools in their sample, nor fail completely in registering some effect on teacher behavior. Rather, teachers tended to "hedge their bets," layering what they perceived to be the appropriate response to new expectations on top of existing, more comfortable practices.

Reform impact


Hargreaves, A., & Ainscow, M. (2015, October). The top and bottom of leadership and change. Phi Delta Kappan.

Abstract: All versions of top-down reform have an Achilles heel: Their focus on delivering the details of two or three measurable priorities is suitable only for systems pursuing traditional and comparatively narrow achievement goals. A digital age of complex skills, cultural diversity, and high-speed change calls for more challenging educational goals and more sophisticated and flexible change strategies. The authors assess and discuss large-scale reform efforts in England and in Ontario, Canada as examples of how a leading from the middle approach can be effective and superior to other approaches.

Reform theory Reform impact


Hord, S. M. (1987). Taking charge of change. Alexandria, VA: Association for Supervision and Curriculum Development.

Abstract: This book provides diagnostic techniques for assessing the needs of school personnel involved in implementing new innovations for school improvement. Using the Concerns-Based Adoption Model (CBAM) as a framework, it discusses the roles and personal needs of the people involved in the change process and provides strategies for the total management of an innovation. The first strategy offers ways to introduce the change or innovation and to monitor the variety and diversity of implementation. A component checklist for determining the range of operational patterns found in classrooms is described. In the second strategy, seven stages of concern experienced by teachers involved in the change process are identified, and suggestions are given on how to deliver interventions that will respond to each stage of concern. The concept of the innovation’s levels of use provides the third strategy, which identifies the degree to which teachers are using the new practices. It is noted that this tool is useful for assisting teachers to move to higher levels of use as well as for evaluating the progress of the change implementation effort.

Strategies for reform


Klingner, J. K., Boardman, A. G., & McMaster, K. L. (2013). What does it take to scale up and sustain evidence-based practices? Exceptional Children, 79(2), 195–211.

Abstract: This article discusses the strategic scaling up of evidence-based practices. The authors draw from the scholarly work of fellow special education researchers and from the field of learning sciences. The article defines scaling up as the process by which researchers or educators initially implement interventions on a small scale, validate them, and then implement them more widely in real-world conditions. Examples of scale-up research are included. The authors discuss challenges to scaling up and sustaining evidence-based practices, followed by factors that can potentially support scaling up, including professional development and district leadership. A case example describes how these issues can play out by highlighting experiences with a Collaborative Strategic Reading (CSR) scale-up research project in a large urban school district. The article concludes by offering recommendations for research, policy, and practice.

Scaling reform Reform impact


Little, J. W. (1994). Teachers’ professional development in a climate of educational reform. In R. J. Anson (Ed.), Systemic reform: Perspectives on personalizing education (pp. 105–135). Washington, DC: U.S. Office of Education, Office of Educational Research and Improvement.

Teacher professional development, argues Little, has not evolved to match the complex range of skills and expectations demanded of teachers by contemporary reform agendas. At the time of writing, Little identified five "streams of reforms" that had significantly shaped the national climate of teaching and learning: reforms in subject matter teaching, equity, assessment, school organization, and the professionalization of teaching. Together, these reforms represented a more challenging (and diverse) set of goals for which teachers were increasingly responsible, yet existing professional development still tended to favor discrete skill development among teachers rather than encouraging alternative models such as teacher collaboratives, subject matter associations, and similar professional networks as vehicles for teacher growth and exchange. Little advocates for professional development to change to better fit the aspirations of reform agendas, shifting the focus to how teachers can integrate multiple, systemic reform demands into the idiosyncratic context of their own classrooms.

Teacher learning Strategies for reform


McLaughlin, M. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9(2), 171–178.

Abstract: The first generation of implementation analysts discovered the problem of policy implementation—the uncertain relationship between policies and implemented programs—and sketched its broad parameters. The second generation began to unpack implementation processes and to zero in on relations between policy and practice. Together, these examinations generate a number of important lessons for policy, practice, and analysis; for example: policy cannot always mandate what matters to outcomes at the local level; individual incentives and beliefs are central to local responses; effective implementation requires a strategic balance of pressure and support; policy-directed change ultimately is a problem of the smallest unit. These lessons frame the conceptual and instrumental challenge for a third generation of implementation analysts—integrating the macro world of policymakers with the micro world of individual implementors.

Reform impact


O’Day, J. A., & Smith, M. S. (2016, September). Equality and quality in U.S. education: Systemic problems, systemic solutions. Washington, DC: Education Policy Center, American Institutes for Research.

This brief presents an overview of current and past educational reform efforts through the lens of America’s historical struggle to balance the twin goals of equality and quality. O’Day and Smith advance a vision for a more equitable school system, beginning with a strong educational foundation for all students, and followed by strategies for connecting schools with community-based services. The recent passage of the Every Student Succeeds Act, the authors argue, presents an opportunity to rethink how this vision can be achieved at scale, namely by delineating what each of the federal, state, and district levels of government can do to support a comprehensive improvement and equity agenda. O’Day and Smith show that a framework of support and professional accountability, departing from the coerce-and-punish approach of more recent top-down reforms, holds the greatest promise for closing achievement gaps and elevating school performance to meet the demands of a new era of reform.

Strategies for reform Scaling reform


Park, S. et al. (2013). Continuous improvement in education. Stanford: Carnegie Foundation.

Park et al. define continuous improvement as "the act of integrating quality improvement into the daily work of individuals in the system" (5). Importantly, continuous improvement efforts focus not just on measuring system outcomes, but equally on identifying and monitoring the processes that produce them. Organizations that engage in continuous improvement seek ways to learn about their own learning: they engage in ongoing testing of the validity of their own knowledge, constantly asking how underlying processes can evolve to better respond to observed outcomes. The authors analyze 11 different educational organizations that use continuous improvement methodology in school-, system-, or community-level work, documenting their chief characteristics and the cross-cutting themes that surface among them.

Strategies for reform


Perry, R., Reade, F., Heredia, A., & Finkelstein, N. (2017). Three structures in the Garden Grove Unified School District that support implementation of the Common Core State Standards in mathematics. San Francisco, CA: WestEd.

This report is the eighth in a series of publications from WestEd’s Math in Common evaluation, part of a year-over-year analysis of Math in Common district schools’ performance on the California Assessment of Student Performance and Progress (CAASPP). Examining the case of Garden Grove Unified, which saw over half of its schools outperform their predicted 2016 CAASPP scores by a margin greater than 5 percent, Perry et al. identify three structures that appear to support an environment of continuous learning in the context of new standards implementation. They include a layered, organized curriculum and pacing guide, dedicated investment in and support of teachers on special assignment (TOSAs), and a structured, routine professional learning system. The many detailed features of how these structures operate are revealed in the report through the testimony of 18 district officials and a former superintendent.

Reform impact


Peurach, D. J., & Glazer, J. L. (2012). Reconsidering replication: New perspectives on large-scale school improvement. Journal of Educational Change, 13, 155–190.

Organizational replication in education contexts can be thought of as a knowledge-producing enterprise, in which original ("hub") and franchise ("outlet") sites collaborate to learn from adaptations in the organizational model. Using this framework, Peurach and Glazer analyze the case of the comprehensive school reform program Success for All. By exploring possible program improvements through its own research agenda, as well as exploiting new knowledge that emerged during expansion into new sites, a constantly evolving understanding of effective replication flourished within the organization. The authors credit the organization’s success to this flexible, coordinated approach to organizational learning, privileging "adaptive, locally-responsive use" over traditional ideals of "fidelity of implementation" (176).

Reform theory Reform impact


RAND Education. (2004). Expanding the reach of education reforms: What have we learned about scaling up educational interventions? (Research Brief). Santa Monica, CA: Rand Corporation. https://www.rand.org/pubs/research_briefs/RB9078.html

Abstract: The process of developing and scaling up education reforms is iterative and complex, requiring cooperative interactions among program developers, policymakers, and school authorities. Successful scale-up efforts have four properties: widespread implementation, deep changes in classroom practices, sustainability, and a sense of ownership of new practices and policies among teachers and school leaders. Reform efforts must take into account a set of eight core tasks: developing and providing support for implementation, ensuring high-quality implementation at each school site, evaluating and improving the intervention, obtaining financial support, building organizational capacity, marketing, adapting to local contexts, and sustaining the reform over time.

Scaling reform


Rincon-Gallardo, S., & Fleisch, B. (Eds.). (2017). Bringing effective instructional practice to scale [Special issue]. Journal of Educational Change, 17(4).

Sample relevant article: Zavadsky, H. (2016). Bringing effective instructional practice to scale in American schools: Lessons from the Long Beach Unified School District. Journal of Educational Change, 17(4), 505–527.

Abstract: Workforce and societal needs have changed significantly over the past few decades while educational approaches have remained largely the same over the past 50 years. Walk into any random classroom in the United States and you will likely see instruction being delivered to students in straight rows by teachers through lecture style. It is possible to find classrooms that utilize technology and cross-disciplinary projects to explore real-world problems, however, those are scarce and in high demand. If we are serious about transforming instruction to better prepare our students for a global society, we need to do it beyond one classroom, school, or zip code. Long Beach Unified school district provides an uncommon example of an education system that has successfully improved instruction across schools by improving the overall district system. This chapter details how the district created a strong coherent system by connecting the critical elements necessary to scale and sustain high quality instruction for all students, and ultimately, better prepare them for the changing demands of our country’s global economy.

Reform impact


Stosich, E. L. (2016, December). Joint inquiry: Teachers’ collective learning about the Common Core in high-poverty urban schools. American Educational Research Journal, 53(6), 1698–1731.

Abstract: Recent research on the relationship between standards and teachers’ practice suggests that teachers are unlikely to make changes to practice without extensive opportunities for learning about standards with colleagues. This article extends this line of research, using a comparative case study of three high-poverty urban schools to examine the nature of teachers’ collaborative work around the Common Core State Standards and the conditions that support this work. It argues that collaborative practices that encourage joint examination of instruction and student learning against standards support teachers in noticing and attending to differences between their current practice and standards. In addition, it examines the role of teachers’ instructional knowledge and principals’ leadership in supporting teachers’ collaboration around standards.

Teacher learning Strategies for reform