SUMMARY TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL SEE MANUAL Manual and guidance to assess the quality of Teacher’s Guides in primary grades 1-4 COACH TOOLS AND RESOURCES Coach is the World Bank’s program focused on accelerating student learning by improving in-service teacher professional development around the world. OVERVIEW The Teacher’s Guide Diagnostic Tool is an instrument to assess the quality of teacher’s guides1 in any subject in primary grades 1-4. The Tool assesses teacher’s guides based on two sets of criteria: Guide Criteria which looksat overall organization, structure, and level of scripting of the entire teacher’s guide; and Lesson Criteria which looks at composition of the individual lessons themselves in terms oflayout, structure, and general pedagogical practices. The Tool assigns a numerical score for the evaluated teacher’s guide and will produce a set of recommendations that suggest specific and prioritized components for inclusion into future teacher’s guide revisions. Note: 1. A teacher’s guide refers to a set of lesson plans for teachers. They can range from full-scripted (i.e., the teacher is meant to read directly from the guide to students) to highly-structured (i.e., the guide includes general guidance of activities within a lesson but does not script out instruction word-for-word). COACH 1 WHY USE A TEACHER’S GUIDE? Teacher’s guides that have tightly structured and effective pedagogy greatly help children learn (World Bank 2019). Studies have shown that the use of scripted and highly-structured lessons can: • Increase student time on task (Rieth and Evertson 1988); • Increases the rate of correct student responses within class time (Gunter, Venn, and Hummel 2004); and • Improve student learning outcomes (Brunette and others 2017; Gove, Pool, and Piper 2017; Piper and others 2018). In some cases, teacher’s guides take the form of a literal script, while in others, the guide outlines a narrowly prescribed path for instruction, but does not fully script actions. Research has found that structured teachers’ guides improve learning outcomes, but that overly-scripted guides are somewhat less effective than structured guides that give specific and descriptive guidance but are not written word for word for each lesson in the guide. Teacher’s Guides are not a permanent solution. They should be used only to provide additional supportto teachers in contexts in which pre- and in-service teacher training does not yet provide teachers with sufficient expertise to implement high-quality instruction. The long-term ambition of education systemsshould be gradually to reduce the use of Teachers’ Guides as teachers’ expertise grows. TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 2 WHAT CAN THIS TOOL HELP INFORM? The Teacher’s Guide Diagnostic Tool can be used to guide revisions in a pre-existing set of teacher’s guides or to accompany the development of a new set of guides. In systems where no In systems where Teacher’s Guides Teacher’s Guides exist: have already been developed: Teacher’s Guide Diagnostic Tool can be used The Tool provides a purposefully evaluative to provide a framework and guiding principles set of criteria that assesses the quality of for consideration as TG’s are developed. a teacher’s guide and provides strategic recommendations on how it can be improved. The Tool has been designed for use by technical as well as non-technical users. Potential users can include Ministries of Education, technical firms supporting implementation efforts in a particular country publishers, NGOs, and international organizations, among others. COACH 3 WHAT DOES THE TEACHER’S GUIDE DIAGNOSTIC TOOL MEASURE? The Diagnostic Tool measures ease of use (i.e., how the guide supports teachers to deliver content) and quality of the pedagogical practices prescribed within each lesson. To do so, the Diagnostic Tool evaluates the overall teacher’s guide and looks at specific, individual lesson plans. • Guide Criteria looks at overall organization and structure of the entire teacher’s guide. • Lesson Criteria looks at composition of the individual lessons themselves. Lesson Criteria are comprised of three components; 1. Layout (e.g., does the teacher’s guide have a picture of the corresponding page in the student textbook); 2. Structure (e.g., is there an appropriate number of distinct activities in a lesson); and 3. General pedagogical practices (e.g., do all lessons start with a clear objective). The Diagnostic Tool also divides criteria into Essential and Bonus. • Essential Criteria are those that are required for a minimally effective teacher’s guide. • Bonus Criteria refer to the additional criteria that make a quality teacher’s guide. TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 4 MATERIALS Required materials The required materials necessary to successfully score a set of teacher’s guides are: COACH 5 HOW DO YOU SCORE contd. on USING THE TOOL? next slide Ex. Overall Guide Criteria Checklist Each set of criteria has a checklist for scoring. The various criteria checklists make up the Tool. There are a total of 3 checklists: 1. Overall Guide Criteria 2. Essential Lesson Criteria 3. Bonus Lesson Criteria Click here to access the Teacher’s Guide Diagnostic Tool Scoresheet. TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 6 HOW DO YOU SCORE USING THE TOOL? (contd.) A full set of scores includes one completed Overall Guide Criteria Checklist, six completed Essential Lesson Criteria Checklists, and six completed Bonus Lesson Criteria Checklists (one for each individual lesson sampled). All scores must be inputted into the Scoresheet. The Scoresheet is programmed to apply weights to the checklist results (your raw score) and will convert these into a standard score. COACH 7 HOW SHOULD THE SCORES BE INTERPRETED? Designed in a checklist form, the Diagnostic Tool aims to make it easy to identify the presence or absence of essential and good-to-have components within teacher’s guides. In making revisions, it is recommended to prioritize essential components over good-to-have components. In addition, the Scoresheet has been programmed to generate a standard score as well as a color indicator (Red, Yellow, Green). This score and color indicator is purposefully evaluative and draws from implementation and research evidence to​ make recommendations for improvement.​ TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 8 ADDITIONAL EXAMPLES AND SUPPORTING LITERATURE This Tool has three annexes: ANNEX 1 ANNEX 2 ANNEX 3 CRITERIA EXAMPLES SCORING EXAMPLES SUPPORTING LITERATURE Provides examples taken Provides an example Provides the from existing public-domain Tool and Screenshot that supporting literature teacher’s guides that has been filled out for a and research demonstrate how criteria lesson evidence on the under the Tool can be effectiveness of each satisfied criterion within the Diagnostic Tool COACH 9 ANNEX 1: CRITERIA EXAMPLES Annex 1 of the Tool provides examples taken from existing teacher’s guides that exemplify presence each criterion in the Diagnostic Tool. In addition, it includes frequently seen practices that do not satisfy criteria within the Tool. TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 10 ANNEX 2: SCORESHEET EXAMPLE Annex 2 of the Tool provides an example of how an individual lesson should be scored. It includes the lesson, a completed checklist, and a filled Scoresheet. COACH 11 ANNEX 3: SUPPORTING LITERATURE Annex 3 of the Tool provides the research and references to support each of the evaluative criteria set forth in the Tool. TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 12 Additional Reading This summary is based on the accompanying Teacher’s Guide Diagnostic Tool Manual. The manual contains the Diagnostic Tool to assess the quality of teacher’s guides for primary grades 1 -4, drawing on implementation experience and a literature review. Citation: Ding, Elaine. 2021. "Teacher’s Guide Diagnostic Tool Manual." Coach Series, World Bank, Washington, DC. License: Creative Commons Attribution CC BY 4.0 IGO. COACH 13 REFERENCES Azigwe, J.B., L. Kyriakides, A. Panayiotou, and B.P.M. Creemers. 2016. “The Impact of Effective Teaching Characteristics in Promoting Student Achievement in Ghana.” International Journal of Educational Development 51 (November): 51–61. https://doi.org/10.1016/j.ijedudev.2016.07.004. Brophy, Jere. 1999. “Toward a Model of the Value Aspects of Motivation in Education: Developing Appreciation for Particular Learning Domains and Activities.” Educational Psychologist 34 (2): 75–85. https://doi.org/10.1207/s15326985ep3402_1. Brunette, Tracy, Benjamin Piper, Rachel Jordan, Simon King, and Rehemah Nabacwa. 2019. “The Impact of Mother Tongue Reading Instruction in Twelve Ugandan Languages and the Role of Language Complexity, Socioeconomic Factors, and Program Implementation.” Comparative Education Review 63 (4): 591–612. https://doi.org/10.1086/705426. Department of Basic Education, South Africa. 2018. Early Grade Reading Study: English First Additional Language (EGRS II) Grade 2 Lesson Plans. Department of Education, Kenya. 2016. Tusome Early Grade Reading Activity: Grade 1 English Teacher’s Guide. Prepared by RTI International and produced with support from USAID and licensed under the Creative Commons Attribution 4.0 International License. Evans, Miriam, and Alyssia R. Boucher. 2015. “Optimizing the Power of Choice: Supporting Student Autonomy to Foster Motivation and Engagement in Learning.” Mind, Brain, and Education 9 (2) (June): 87–91. https://doi.org/10.1111/mbe.12073. Evertson, Carolyn M., Charles W. Anderson, Linda M. Anderson, and Jere E. Brophy. 1980. “Relationships between Classroom Behaviors and Student Outcomes in Junior High Mathematics and English Classes.” American Educational Research Journal 17 (1): 43–60. https://doi.org/10.3102/00028312017001043. Good, Thomas L., and Douglas A. Grouws. 1977. “Teaching Effects: A Process-Product Study in Fourth-Grade Mathematics Classrooms.” Journal of Teacher Education 28 (3): 49–54. https://doi.org/10.1177/002248717702800310. TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 14 Gove, Amber, Medina Korda Poole, and Benjamin Piper. 2017. “Designing for Scale: Reflections on Rolling Out Reading Improvement in Kenya and Liberia.” New Directions for Child and Adolescent Development 155: 77–95. https://doi.org/10.1002/cad.20195. Gunter, Philip, Martha Venn, and John Hummel. 2004. “Teacher-Made Scripted Lessons.” In Evidence-Based Educational Methods, 95– 108. San Diego, CA: Elsevier Academic Press. https://doi.org/10.1016/B978- 012506041-7/50008-5. Lemov, Doug. 2010. “Teach like a Champion: 49 Techniques That Put Students on the Path to College.” San Francisco, CA: Jossey- Bass. Ministry of Education. Kenya. 2018. Primary Education Development Project (PRIEDE). Mathematics Teacher’s Guide: Grades 1 and 2. NERDC (Nigerian Educational Research and Development Council) and USAID (United States Agency for International Development). 2019. Nigeria Northern Education Initiative Plus: Teacher’s Guide. P2/P3. Developed under the USAID-funded Northern Education Initiative Plus program and licensed under the Creative Commons Attribution 4.0 International License. Piper, Benjamin, Yasmin Sitabkhan, Jessica Mejía, and Kellie Betts. 2018. “Effectiveness of Teachers’ Guides in the Global South: Scripting, Learning Outcomes, and Classroom Utilization.” RTI Press Publication OP- 0053-1805. Research Triangle Park, NC: RTI Press. https://doi.org/10.3768/rtipress.2018.op.0053.1805. Pittinsky, Todd L., and R. Matthew Montoya. 2016. “Empathic Joy in Positive Intergroup Relations: Empathic Joy.” Journal of Social Iss ues 72 (3): 511–23. https://doi.org/10.1111/josi.12179. Reeve, Johnmarshall, and Jang Hyungshim. 2006. “What Teachers Say and Do to Support Students’ Autonomy during a Learning Activity.” Journal of Educational Psychology 98 (1): 209–18. https://doi.org/10.1037/0022-0663.98.1.209. Rieth, Herbert, and Carolyn Evertson. 1988. “Variables Related to the Effective Instruction of Difficult-to-Teach Children.” Focus on Exceptional Children 20 (5): 1–8. https://doi.org/10.17161/foec.v20i5.7503. Rosenshine, Barak. 2010. “Principles of Instruction.” Educational Practices Series 1–21. International Bureau of Education, International Academy of Education. http://www.ibe.unesco.org/filead- min/user_upload/Publications/Educational_Practices/EdPractices_21.pdf. Rosenshine, Barak. 2012. “Principles of Instruction: Research-Based Strategies That All Teachers Should Know.” American Educator Spring Issue. https://www.aft.org/sites/default/files/periodicals/Rosenshine.pdf. COACH 15 Tyler, John H., Eric S. Taylor, Thomas J. Kane, and Amy L. Wooten. 2010. “Using Student Performance Data to Identify Effective Classroom Practices.” American Economic Review 100 (2): 256–60. https://doi.org/10.1257/aer.100.2.256. Uganda National Curriculum Development Center. 2018. School Health and Reading Program: Teacher’s Guides. Primary 4. Prepared by RTI International. Reproduced on the basis of an original work developed under the USAID/Uganda School Health and Reading Program and licensed under the Creative Commons Attribution 4.0 International License. USAID (United States Agency for International Development) and Liberia MoE (Ministry of Education). 2019. Accelerated Quality Education Teacher Guides and Learner Workbooks Series, Literacy, Level 1, Semester 1. Monrovia: USAID and MoE License: Creative Commons Attribution CC BY 4.0 IGO. USAID and The Philippines Department of Education (DoE). 2015. Basa Pilipinas: Teacher’s Guides. Grade 3 English. Examples from this tool are adapted from Basa Pilipinas Teacher’s Guide Grade 3 English Quarter 1, published March 2015, developed with the generous support of the American people through the United States Agency for International Development (USAID) under the Basa Pilipinas Project and the Department of Education. World Bank. 2019. “Ending Learning Poverty: What Will It Take?” World Bank, Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/32553 License: CC BY 3.0 IGO. World Bank. 2020. “Cost-Effective Approaches to Improve Global Learning: What does recent evidence tell us are “Smart Buys” for improving learning in low- and middle-income countries? Recommendations of the Global Education Evidence Advisory Panel.” Washington, DC. World Bank. https://documents1.worldbank.org/curated/en/719211603835247448.pdf TEACHER’S GUIDE DIAGNOSTIC TOOL MANUAL 16 Contact us at coach@worldbank.org and visit us at www.worldbank.org/coach