Measure 1. Impact on P-12 Learning and Development (Component 4.1)
The provider documents multiple growth measures that completers contribute to student learning.
The University of North Georgia’s Educator Preparation Provider (EPP) uses several measures to demonstrate “the impact of its completers on P-12 student learning and development, classroom instruction, and schools, and the satisfaction of its completers with the relevance and effectiveness of their preparation” (CAEP Standard 4). In terms of 4.1, the goals are to document our completers’ impact on student learning and development.
As an initial piece of evidence provided here to meet these aspects of the standard, we have data from surveys developed by the Georgia Professional Standards Commission (GaPSC) provided to inductee or first-year teachers and the employers of these educators (see “Induction Teacher and Leader Surveys and Employer Surveys” in the supporting evidence below). The GaPSC sends a survey to each EPP’s alumni and their employers one year after they have completed their initial certification and have been employed in a public P-12 school in Georgia (E45). These surveys are based on InTASC standards, and they measure perceived comfort/skill regarding meeting these standards in the classroom (the questions have been provided in the evidence along with the data). These surveys are an indirect measure of completers’ impact on learning and development, classroom instruction, and overall effectiveness in that direct student data is not included in these surveys. They are a subjective measure of how completers measure, assess, and reflect on their own skills and their perceived impact on their classrooms and students and how their employers view the completers’ impact on and overall effectiveness in regard to classrooms and students.
The results from these surveys, though, do provide us a means to triangulate data when we see trends and patterns across surveys and then via additional measures. This affords us a way to utilize multiple measures to determine where trends exist. While analysis of these surveys has been provided in the supporting evidence below, patterns that emerged from the surveys indicated that these first-year educators need additional training in the differentiation of instruction and classroom management. Moreover, the need for an increased focus on literacy was noted as a pattern. These three main areas were seen in employer and completer surveys, and they were emphasized via other data measures, which will be discussed below. Overall, the results indicated that inductees were confident in their skills and approaches to teaching, instruction, and assessment, along with their use of technology and their understandings of diversity. They also noted that they felt confident in terms of their interactions and professional behaviors as educators.
As another and perhaps more significant measure, we received Teacher Keys Effectiveness System (TKES) data from the Georgia Department of Education (GaDOE) on completers’ Teacher Effectiveness Measure (TEM) Scores, Teacher Assessment Performance System (TAPS) scores, Professional Growth ratings, and student growth measures, including Student Growth Percentile (SGP) scores, Student Growth Ratings (SGR), and non-Student Growth Ratings (non-SGR) (see supporting evidence below). To explain this evaluation system, in the state of Georgia, the Teacher Keys Effectiveness System (TKES), which is used as a performance system for all public-school educators, “is comprised of three components which contribute to an overall Teacher Effectiveness Measure (TEM): Teacher Assessment on Performance Standards (TAPS), Professional Growth, and Student Growth” (GaDOE, Georgia’s Teacher Keys Effectiveness System: Implementation Handbook, p. 5). As for student growth, this is based on Student Learning Objectives (SLOs), which are “the measure of student growth for non-state-tested subjects. The aggregate measure of SLO performance for all non-tested courses taught by a teacher will be used in calculating his or her TEM” (GaDOE, TEM Scoring Guide and Methodology). Additionally, these scores include Student Growth Percentiles (SGPs), which are “the measure of student growth for core state-tested subjects” (GaDOE, TEM Scoring Guide and Methodology).
The GaDOE’s supplemental descriptions for determining these scores are included as evidence here as well. In terms of the TAPS data, we received a separate breakdown of first-year teachers’ scores by each of the ten standards on which they are evaluated, including the following: (1) Professional Knowledge, (2) Instructional Planning, (3) Instructional Strategies, (4) Differentiated Instruction, (5) Assessment Strategies, (6) Assessment Uses, (7) Positive Learning Environment, (8) Academically Challenging Environment, (9) Professionalism, and (10) Communication. Supervisors evaluate teachers each year on all ten tap standards, and this evaluation includes walk-throughs, formal observations, and documentation/artifacts provided to supervisors over the course of the year. For our evidence here, we have included the TKES data in several forms including the following: (1) a three-year aggregate for the Educator Preparation Provider’s (EPP) initial programs as a whole, (2) a breakdown by year for three cycles for the EPP’s initial programs as a whole, and (3) a breakdown by year for three cycles by the program (E66).
This data, which is included as an evidence piece for Standard 4, demonstrates that the overwhelming majority of our completers for the past three years have scored at a level 3 (proficient) out of 4 on all aspects, which is where we hope they will be as induction-level, or new, teachers. For the overall TEM scores, the majority scored at a level 3, and for the SGPs, the majority scored at a level 3, with 11.71 % scoring at a level 2. With student growth ratings (SGR), the majority scored at a level 3, with 8.40 scoring at a level 2, and with non-Student Growth Percentiles, the majority was at a level 3, with 6.34 scoring at a level 2. In terms of TAPS data, again, the majority scored at a level 3, with a small percentage scoring at a level 2 for Academically Challenging Environment, Positive Learning Environment, Instructional Strategies, and Differentiated Instruction. While these percentages scoring at a level 2 were small (5.23, 3.03, 3.14, and 3.77, respectively), we will keep these areas in mind as we continue to revise courses, key assessments, and field and clinical placement requirements to ensure that our completers are as prepared to have the highest possible positive impact on the P-12 students with whom they work. With regard to TAPS, while the majority scored at a level 3, as noted above, 20.92 % scored a level 4 on Academically Challenging Environments, and 19.35 scored a level 4 on Professionalism. For induction-level teachers, they surpassed expectations by achieving the highest possible scores in these areas. When looking across programs, the Elementary and Special Education program showed the most variation in scoring, with more induction teachers scoring below a 3; however, the majority still scored at a level 3. This variation is not surprising considering that this program produces the most alumni, and thus there is room for more variation amongst inductee teachers. The TAPS data are an excellent indicator of both impact on students, as they directly address planning, teaching, and assessment, and overall effectiveness, as they cover a wide range of overall responsibilities in regard to the role of a teacher, from student impact to professionalism.
We chose to supplement and triangulate this data with a case study to demonstrate impact and effectiveness, for which we interviewed completers across all programs (37 inductee teachers/completers, 14 advanced completers), their employers (37 administrators), and veteran teachers who worked with inductees as mentors in their first or induction-level year (9 mentor teachers). The entire case study, including methodology, data, analysis, and interview protocols/questions, has been included here as evidence, but we will address some highlights here (see supporting evidence below). There were multiple codes that emerged from the interviews, including the following: (1) relationship building, (2) academic growth, (3) classroom management, (4) collaboration, (5) communication, (6) content knowledge, (7) data and assessment, (8) differentiation, (9) diversity, (10) field placement experience, (11) pedagogy, (12) professional dispositions, (13) social growth of P-12 students, and (14) technology. For each of these areas, the case study highlights sub-codes from inductee teachers, administrators, and veteran teachers acting as mentors. For each sub-code, the case study covers strengths, areas for improvement, suggestions, and overall implications/summaries. In all areas, the feedback was overwhelmingly positive regarding inductees’ skills in terms of pedagogy and their overall impact and effectiveness on their P-12 students. As mirrored in the employer and inductee surveys, classroom management, differentiation, and literacy skills emerged as areas in which inductees could use additional professional development and/or could have used additional or enhanced preparation while in their initial programs. Administrators noted, however, that these are all areas in which they see a need for improvement with veteran teachers as well, and they were not specific to completers from our EPP. In fact, literacy has become a state initiative, as this is a recognized need across the board. As one administrator noted, “I mean, I feel like there’s not one university out there that actually teaches reading the way it’s supposed to be taught. I mean, just from my own personal experience.” With this in mind, we are developing a literacy pilot project as part of a state initiative to integrate additional reading content and/or courses earlier within our programs. Also, we have had one speaker provide professional development as related to dyslexia and techniques for early literacy and reading comprehension, and we are currently working on embedding content related to reading, comprehension, and dyslexia into our programs. Additionally, based on these interviews, we plan to add additional information to our programs regarding the development of Individualized Education Plans (IEPs) and behavior management plans. While all students receive this information, these were other areas in which the need for increased emphasis was noted. Our graduates were praised for their ability to build relationships with students, parents, community members, and colleagues, and their abilities to engage students in the classroom were highlighted throughout the interviews. As one administrator noted in regard to student engagement, “I think North Georgia is giving them the skills to do that and not line up 25 desks in a row and sit and teach in the front. ... They realize you’ve got to keep the kids engaged.” Interviews also highlighted completers’ use of technology as a teaching tool, as a data tool, and as a tool for inciting critical thinking. As one administrator stated of an inductee teacher, “I remember going in there, going, ‘Wow. It’s amazing!’ She uses technology not just as tool to make sure students are proficient with skills, but she is guiding them to create their own stuff. She has them doing electronic portfolios to share. It was just amazing to watch.” In looking at from inductees across programs, there were no major differences in terms of skills across the areas listed above, but again, the majority of comments for all areas were positive. In fact, most of the areas for improvement seemed to be geared to one particular inductee rather than emerging as patterns.
We are also including Georgia's Teacher Preparation Program Effectiveness Measures (PPEMs) in support of component 4.1. Recently, the Georgia Professional Standards Commission (GaPSC) began publishing scores for Educator Preparation Providers (EPPs) across the state. The EPP as a whole is scored, as is each program (programs are combined for the aggregate score). The PPEM scores are based on in-program measures (50 percent), including the edTPA scores (30 percent) and the Georgia Assessments for the Certification of Educators (GACE) content scores (20 percent), and the other 50 percent is based on outcome measures that come into play once completers are in their first year of teaching. These outcome measures stem from the EPP completers' TAPS scores (30%), the GaPSC employer survey (10%), and the GaPSC inductee/first-year teacher survey (10%), all of which have been mentioned previously in this narrative. These five measures "generate one of four possible overall ratings for a provider or program. Level 3 is the expected standard of performance for programs and providers in the state, with about three in four providers earning this rating in prior years for which the calculation was modeled" (Georgia's TPPEM_Academic Year 2018-2019, GaPSC). The scores are calculated utilizing an aggregate of three years of data for all programs (Georgia's TPPEM_Academic Year 2018-2019, GaPSC). More information on how scores are calculated and the consequences of these scores have been included as evidence for Standard 4. As of the 2018-2019 academic year, these scores became consequential, meaning they have an impact on program approval. For the 2018-2019 year, our EPP scored a 4 overall, which is the highest possible score. This is the first and only year these scores have been published, although we should receive a rating for the 2019-2020 academic year this fall. Data have been included for the EPP as a whole and for each program (see supporting evidence below).
As part of this data, both in-program and outcome measures have been included, although the outcome measures are the significant measures for Standard 4, as they relate solely to completers. As part of this data, reviewers will see, as well, the employment context for both the EPP and programs. This illustrates employment percentages in Georgia Public Schools and Employment by Area in our state. In terms of program scores, the following programs received a score of 4: Elementary and Special Education, Middle Grades, English Education, History Education, Science Education, Kinesiology with Teacher Certification (Health and Physical Education), and Post Baccalaureate/MAT. The Mathematics Education program scored a 3, and the following programs did not have enough data for an overall score due to having a low N: Art Education and Music Education. As noted in the data from the GaDOE described above, these scores corroborate that our graduates are performing well in the classroom. Again, their TAPS data, which covers the state's ten standards for teaching, has consistently been strong during the induction years. We hope that this is a result of the fact that their course objectives and content, assessments, and field and clinical experience assignments are aligned to state requirements, InTASC standards, and TAPS requirements. In fact, the Candidate Assessment on Performance Standards (CAPS) EPP-wide assessment that faculty and contract supervisors utilize as a summative field and clinical placement assessment is based on the TAPS. By the time graduates leave our program, they are very familiar with the state TAPS rubric, as they are evaluated using this same rubric by both their mentor teachers and university supervisors. Additionally, they self-evaluate throughout the program, and these standards become a part of their daily lives starting in the junior year.
As seen here, we have multiple measures to demonstrate that our completers are contributing "to an expected level of student-learning growth," and, in many cases, they are surpassing these expected levels. Their performance as first-year teachers is strong, consistently scoring at a level 3 and sometimes 4 across all data measures. These measures illustrate both their impact on P-12 student learning and development and they are indicators of their teaching effectiveness.
Supporting Evidence:
