Student Ratings of Teaching Effectiveness: Online SRTEs

This posting is prompted by a recent comment from a faculty member expressing concern about the decreased response rates seen with Online SRTEs.  A number of faculty over the past year have also communicated that they've heard that the Schreyer Institute says the response rates for SRTEs have not decreased.  This is incorrect.

Instead, what we have communicated is that
a) the response rate decrease was expected, but
b) that average ratings have held steady. 

Our research has shown that the response rates have decreased between 20-25%.  We fully expected a decrease because the assessment has moved out of the classroom, so students were no longer a "captive audience."  In fact, the decrease we have experienced at Penn State is smaller than we expected and much smaller than other institutions have experienced.  The average response rates by college and campus are posted online (see http://www.srte.psu.edu/OnlineReports/) and all are above 50%. 

The reports above also show that the average scores have held steady.  At the request of the University Faculty Senate's Committee on Faculty Affairs and the Vice Provost for Academic Affairs, we have been monitoring the SRTE results. Other than overall decrease in response rates, we have found no patterns that can be attributed to the online SRTEs.  In other words, we see no evidence that faculty have been 'harmed' by the decreased response rates.  Vice Provost Blannie Bowen has indicated that unless we see a significant negative impact on faculty, we will continue with the online administration of the SRTEs. 

We have requested that campus and college administrators communicate to faculty review committees and academic unit heads that response rate decreases not be over-interepreted or attributed to the actions of any individual faculty members.  

If a faculty member believes that their results are the exception to the trend, i.e. that their average scores have decreased solely because of the online administration, that should be communicated to her/his academic unit head.  The University Faculty Senate's Statement of Practices for the Evaluation of Teaching Effectiveness for Promotion and Tenure states that "If there is some reason to explain the results or the absence of results in a particular case, the appropriate academic administrator shall make a note to that effect in the dossier" (see p. 3, section I.A.11.a.2).

Please remember that the paper SRTEs had more than 20 years to become embedded in student culture.  We need to give the online SRTEs some time too.  We are still in the transition phase where many of our current students experienced paper SRTEs and now are having to do them online.  It will be a few years before all students have only submitted SRTEs online.  Institutions that have been administering student ratings online for far longer than we have indicate that the response rates do rebound.

The Schreyer Institute is currently gathering information from faculty who have response rates of 70% or higher (see http://www.srte.psu.edu/ResponseRate/) and we will continue to add to this site.  The faculty included in this project come from a wide variety of colleges, campuses, and disciplines. The faculty teach courses with enrollments from 30-493.  We deliberately excluded graduate and upper-level majors courses in which it might be easier to get higher response rates. 

What this project indicates is what we've always known--that faculty are the most important determinant of students' participation.  In short, students make the effort to submit ratings when they feel that "someone is listening."  The faculty with the highest response rates communicate that they value students' views, that they take suggestions for improvement seriously, and they tell their students changes made as a result of students' feedback.  Some faculty do this by talking about the SRTEs, others never mention SRTEs, but instead gather student feedback throughout the semester, which creates a culture of feedback among the students.  None of these faculty discuss the fact that SRTEs are required for P&T and reappointment.