Key features for implementing a Harvesting Gradebook

In working on our DML competition entry, I found myself enumerating the features we’ve found important to our Harvesting Gradebook work. The Harvesting Gradebook consists of a web-based survey that can be embedded in, or linked from, link to, or embed within itself a piece of work that is to be evaluated by a reviewer using the survey in conjunction with various forms of data visualization such as radar chart and tag clouds.
The first proofs of concept were in the summer of 2008. We first used the tool with students in Fall 2008. That work, thru April 2009 can be found here. Our explorations have branched in several directions since then, including elaborating the idea to university-wide program level learning outcomes assessment. There is some overlap among these 3 categories:

These implementations have used several tools: paper&pencil, Google Docs Forms/Spreadsheet, Microsoft SharePoint survey, and most recently, Diigo and Google Sidewiki. Production implementations have been done with WSU’s Skylight Matrix Survey System.
We keep gravitating back to Skylight because of features that it has that make it particularly well suited for implementing the Harvesting Gradebook. They are:

  1. multiple “Respondent Pools,”  which allows multiple surveys to use a common data store and shared reporting mechanism;
  2. Respondent Pool Metadata” to store additional data elements that describe a Respondent Pool with that pool’s data;
  3. a method for embedding the metadata in both an email and a survey when delivered to a respondent;
  4. the “Rubric Question Type,” rubrics are a commonly accepted assessment tool;
  5. the “Dashboard” for automatic, real-time web-based reporting of data from one or more Respondent Pools to one or more authorized viewers;
  6. tabular reports of data, available for both the survey overall or for an individual Respondent Pool or selected groups of Respondent Pools;
  7. a mechanism for securing of report data without a password, including sharing the results when a rater finishes a rating session;
  8. a mechanism to get the tabular data from the Skylight Matrix Survey System to a third party system via a uniform resource locator (URL);
  9. a mechanism to download all data and metadata associated with a survey  and/or Respondent Pool, including URLs for taking the survey, in a format readily used by spreadsheets.

A Leader Asks, What is the Rain King Chronicle

It’s our blog of the story, or history of this assessment initiative we’ve launched.  We summarize and don’t name names (except our own).  Bits of notes go in this chronicle, for instance the liaison who told us “it is inappropriate for us to share our rubrics.”

It’s not just about keeping a history, which we are wishing we had done on a variety of projects in the past, but, we are also in this work reminded of the Bellow novel.  In the story, the lead character by happenstance becomes the Rain King of a remote African tribe only to learn that the tribal elders are plotting to have him beheaded….