We are getting more requests to move course evaluations from paper-based to online. In most cases, the opportunity to explore greater alignment with outcomes assessment is subordinate to technology efficiencies. The issue of response rates emerges, as always, and we have resources and rationale for thinking about response rates differently. The note below reflects a response to the issue, the first of mine to more directly, in light of recent research, recast the issue of response rates more directly to response bias.
On the migration issue, what we encounter are lots of concerns about response rates, which typically runon averageat about 50% (course based paper evaluations run at roughly 70%, which is a bit low, considering). We have read lots of and done more than a little research on this and conclude that the online aspect of the concern is largely misplaced. Different instruments evince different response rates. The rate varies dramatically from class to class suggesting that something else is going on. The expected biasthat only malcontents and the deliriously happy respondis just something we cannot document. (We have come to wonder the extent to which coercion in the classroom introduces bias, and there is research now suggesting that is so.) Extra credit adds no more than a 5% boost and who knows what kind of attendant bias.
Response rates rise when faculty, their colleagues, and their leadership demonstrate that student opinions matter to them.
And of course all of this is, in part, contingent upon what faculty and programs do with the results.
I look forward to conversations about this and anything else in the not too distant future
It would be great if we could get the evaluations for the courses online for this semester, and it would also be great to pick Gary’s brain about other things to do with the evaluation form itself and/or the reporting process in the long run to make the process more efficient and useful for all.
That said, I think we are really looking (for now) at simply migrating our existing evaluation form to the online platform. The Curriculum Committee hasn’t discussed any changes to the evaluation form, and we have a reporting system in place that the Skylight data will just get fed into. There’s a fair amount of skepticism about whether moving to online evaluations is workable for us, so the idea here (at least my idea – the Assoc. Dean may have something else in mind) is to take this in steps to either show that it can work or figure out what might need to be done if it doesn’t work so well.