Research Evaluation Summary of the ENVoY Program

Christoph Arnold Research Leave a Comment

by Dr. Emily Garfield

 

It is evident that ENVoY training provides, expands, and/or reinforces good teaching skills. We see this reflected in the research findings, we hear this from the teachers, and we observe this in student behavior. Our challenge is how to persuade the educational community that it is important to incorporate non-verbal techniques routinely into the classroom.

Participants continue to rate the trainings and the trainers as “very good” to “excellent.”

Looking for evidence of effectiveness has been a major part of our program efforts from the beginning. The September 1996 Report and the June 1997 Addendum cover the evaluation finds for grant years one and two.

This section will concentrate on the results from year three. Given, therefore, are the facts that participants continue to rate the trainings and the trainers as “very good” to “excellent” and that there is a substantial informational gain when comparing pre and post tests on the content courses.

The Coaching Component

We contacted all 78 certified coaches by mail in March-April 1998 with follow-up phone calls in early May asking them to tell us about their ENVoY implementation activities from October 1, 1996 through January 31, 1998. Thus we have data from 58 of them, representing 75% of the sample. All of them (100%) stated that ENVoY strategies “have added to my personal teaching skills,” 98% “have shared the skills with colleagues,” 57% have presented to staff at my school,” 34% “have presented to staff at other schools,” 71% “have coached at my site,” and 24% “have coached at other sites.” We left room on the reporting form for comments. We have included a few of these below.

The Comparison Studies

Three regression analyses were conducted comparing trained to untrained teachers while controlling for class size. These analyses examined three sets of outcome data hypothesized to result from the acquisition of ENVoY skills by classroom teachers.

The sets were:

  1. Time required to get class attention;
  2. Off task student observations; and
  3. Number of discipline referrals.
Time required to get class attention

Twenty-four teachers were involved in this sub-study, 12 trained and 12 untrained, for whom we had 106 observations. The dependent variable was time measured in seconds. While larger class sizes did make a difference, it was apparent that trained teachers were consistently able to gain class attention in less time than the untrained. This was significant at the .01 level.

Off task student observations

Twenty minute observations were conducted in twelve classrooms on three occasions. We had, therefore, 36 observations: 18 from the classrooms of six trained teachers and 18 from the classrooms of six untrained teachers. Data were gathered on the number of students off task and the number of times off task within the observed time period. For example, during one observation we documented that in one untrained teacher’s classroom 3 students were off task on 4 occasions, 6 students were off task on 2 occasions, and 1 student was off task the entire twenty minutes. The dependent variable, therefore, was “off taskness.” The formula used looked at number of students off task times the number of times off task divided by class size. Students in the trained teachers’ classrooms were less likely to be off task both in number of students and on number of occasions. This finding was significant at the .0006 level.

“It is indisputable how the use of ENVoY effectively improves on task behaviors, briefer transitions, and better student-teacher relations in the classroom. Too bad parents aren’t trained to use it at home. Thank you.”

Number of discipline referrals: Data were gathered on the number of discipline referrals from twenty teachers in a four month time span ten teachers were ENVoY trained and ten untrained. The dependent variable was number of referrals. Again, controlling for class size, it was apparent that trained teachers made fewer referrals than the untrained. This finding was significant t at the .001 level.

In summary, it is evident that ENVoY training provides, expands, and/or reinforces good teaching skills. We see this reflected in the research findings, we hear this from the teachers, and we observe this in student behavior. Our challenge is how to persuade the educational community that it is important to incorporate non-verbal techniques routinely into the classroom.

  • “We have also established a professional library with many of the materials and books mentioned during ENVoY training and plan to start a book club to meet and discuss material.”
  • “We currently have coached ten staff members. Folks find it effective and useful during teaching day. We are featuring ‘Skill of the Week’ as a staff to support, train, and establish ‘standardized’ practices in our school. Thanks.”
  • “I am encouraged as I coach in other ‘ENVoY’ classes to see many of the skills being used successfully during my visit. I certainly appreciated being given the opportunity to participate in ENVoY. Michael Grinder is truly a ‘gem’.”
  • “Thank you – this has been a wonderful program.”

Dr. Emily Garfield, Professor Emeritus, Stanford University, May 1998.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.