Sanity Checking the May 17 scores


We have done a process to review language and scores on individual responses.  A couple weeks ago, Ashley led an activity to compare (in each dimension) across 4 programs. That found a couple anomalies.

This is an experiment to compare across the 22 programs that provided a report May 17. (Actually a few are not finished in the scoring process)

The activity uses a spreadsheet of the OAI contact, the Reviewer, the program and its scores.  The activity is for each OAI Contact to filter the list on themselves as contact, and then sort the programs by score on one dimension at a time — and see if the programs are in a reasonable ordinal order.

The activity continues for each reviewer, to filter on the programs they reviewed and then sort the programs by score on one dimension at a time —  and see if the programs are in a reasonable ordinal order.

The result is a variety of transects across the data, with the opportunity to spot something that is out of order

Different ideas were offered about what to look for:

  1. programs who’s scores were more than 1/2 point out of wack
  2. programs that were not in the right “bin” relative to others in that bin

The spreadsheet looks like this

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: