Moving evaluations online


We are getting more requests to move course evaluations from paper-based to online.  In most cases, the opportunity to explore greater alignment with outcomes assessment is subordinate to technology efficiencies.  The issue of response rates emerges, as always, and we have resources and rationale for thinking about response rates differently.   The note below reflects a response to the issue, the first of mine to more directly, in light of recent research, recast the issue of response rates more directly to response bias.

On the migration issue, what we encounter are lots of concerns about response rates, which typically run—on average—at about 50% (course based paper evaluations run at roughly 70%, which is a bit low, considering).  We have read lots of and done more than a little research on this and conclude that the online aspect of the concern is largely misplaced. Different instruments evince different response rates. The rate varies dramatically from class to class suggesting that something else is going on. The expected bias—that only malcontents and the deliriously happy respond—is just something we cannot document. (We have come to wonder the extent to which coercion in the classroom introduces bias, and there is research now suggesting that is so.)  Extra credit adds no more than a 5% boost and who knows what kind of attendant bias.

Response rates rise when faculty, their colleagues, and their leadership demonstrate that student opinions matter to them.

And of course all of this is, in part, contingent upon what faculty and programs do with the results.

I look forward to conversations about this and anything else in the not too distant future…

Gary

From:

All,

It would be great if we could get the evaluations for the courses online for this semester, and it would also be great to pick Gary’s brain about other things to do with the evaluation form itself and/or the reporting process in the long run to make the process more efficient and useful for all.

That said, I think we are really looking (for now) at simply migrating our existing evaluation form to the online platform.  The Curriculum Committee hasn’t discussed any changes to the evaluation form, and we have a reporting system in place that the Skylight data will just get fed into.  There’s a fair amount of skepticism about whether moving to online evaluations is workable for us, so the idea here (at least my idea – the Assoc. Dean may have something else in mind) is to take this in steps to either show that it can work or figure out what might need to be done if it doesn’t work so well.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: