Reading Reform Foundation of New York (RRF) is a not-for-profit literacy organization that offers multisensory, systematic approaches to the teaching of reading, writing and spelling.  The In-School Teacher Training Program (ISTTP) is an intensive course of instruction that takes place right in the teacher's classroom.  An RRF consultant meets with the teacher twice per week for an entire school yera, during which the consultant models lessons in the classroom and provides support in instructional planning.  An independent consultint firm, Strategic Research, completed an evaluation of the ISTTP.  The study included 1) an assessment of two years of student test scores; 2) written surveys of teachers who had participated in the ISTTP during the past two years; and 3) on-site interviews with school personnel who had two years of program implementation experience.  The results provide strong evidence that the ISTTP produces positive outcomes for teachers and their students.

Student Test Scores

RRF routinely administers the Gates-MacGinitie Vocabulary subtest on a pretest-posttest basis to children in grades 2-8 whose teachers participate in the ISTTP.  For the '96-'97 and '97-'98 years, average pretest-posttest scores were analyzed to determine whether the gains were statistically significant and educationally meaningful.¹  Score gains were also compared to those of the test's national norm group.  Because the norm group's gains represent average student growth that occurs over a school year, larger gains would suggest the presence of an effective intervention.

The test results from both years illustrate the powerful impact of the ISTTP.  For 1996-97, all 17 classes (100%) showed statistically significant and educationally meaningful gains in reading skill.  Moreover, each of the classes (100%) manifested score gains that exceeded those made by the test's national norm group.  For 1997-98, 32 of the 33 classes (97%) increased their scores to a statistically significant and educationally meaningful degree.  The magnitude of these gains showed that RRF students outperformed the national norm group.  For both years, these gains were present for all grade levels and types of students (i.e., regular education, ESL/bilingual, and at-risk/low functioning).

Results of Teacher Surveys

All teachers who participated in the ISTTP during '96-'97 and '97-'98 were surveyed during the fall 1998.  Teachers reported that the training program had a beneficial impact on both their teaching and on students' reading skills.  Specifically, all (100%) teachers who completed the survey (N=36) said that the ISTTP improved their effectiveness in teaching reading, and 94% said that it had improved the reading skills of their students.  When asked to rate the degree to which the quality of their reading instruction had changed, two-thirds (67%) responded that it has improved substantially.

Teachers also reported that the ISTTP had benefitted their professional sharing with other teachers, with over four-fifths (83%) indicating that it had increased.  Their average rating of the overall quality of the training program was excellent.  Teachers who were trained more recently (i.e., '97-'98) were asked how effective the RRF graduate level course was in facilitating their learning and skill acquisition in the ISTTP.  Nine out of ten (90%) reported that the course had facilitated their learning to a moderate or substantial degree.

When asked about the frequency of use, teachers reported that they had implemented the RRF approach in their classrooms on an average of 4 9'96-'97) or 5 ('97-'98) days per week.  The average duration of the typical lesson for those trained in '96-'97 was 31-35 minutes, while the average for those trained in '97-'98 was 41-45 minutes.  This high use of the instructional method strengthens the casual link with the observed student test score gains and substantiates teacher claims that the method improves student reading skills.

On-Site Interviews

The independent evaluator visited 4 schools during the fall of 1998 and interviewed 8 teachers (2 per school) and 6 administrators.  Teachers who had been trained during 1996-97 were selected for interviews because they had one full year of experience implementing the instructional method with their students following completion of their training.  It was expected that these teachers could offer a more experiential assessment of the RRF approach than could newly trained teachers.

Results of the interview corroborated the survey findings.  All (100%) eight teachers were satisfied with the RRF instructional method and would recommend it to other teachers.  All four principals also stated that they would recommend the ISTTP to other schools, and some already had done so.  With regard to the quality or effectiveness of their teaching of reading, all of the teachers said that it had improved.  They reported that the instructional method promoted greater student learning than previously used methods.

Two teachers explained that the method provides students with a tool to attack words they don't know, which is especially helpful to bilingual children.  Another teacher said that students were learning better and faster than they were before.  All but one of the teachers stated that students were now more enthusiastic about reading and spelling.

When asked to identify the most valuable aspect of their RRF training, 7 of the 8 teachers cited the in-class modeling and on-going support provided by their RRF mentor.  Also mentioned as valuable were the RRF course and the hands-on materials that were provided.

¹ Student's t Test (p<.05);

Educational meaningfulness = (Mean _posttest - Mean _pretest)/SD _gain