In working on our DML competition entry, I found myself enumerating the features we’ve found important to our Harvesting Gradebook work. The Harvesting Gradebook consists of a web-based survey that can be embedded in, or linked from, link to, or embed within itself a piece of work that is to be evaluated by a reviewer using the survey in conjunction with various forms of data visualization such as radar chart and tag clouds.
The first proofs of concept were in the summer of 2008. We first used the tool with students in Fall 2008. That work, thru April 2009 can be found here. Our explorations have branched in several directions since then, including elaborating the idea to university-wide program level learning outcomes assessment. There is some overlap among these 3 categories:
These implementations have used several tools: paper&pencil, Google Docs Forms/Spreadsheet, Microsoft SharePoint survey, and most recently, Diigo and Google Sidewiki. Production implementations have been done with WSU’s Skylight Matrix Survey System.
We keep gravitating back to Skylight because of features that it has that make it particularly well suited for implementing the Harvesting Gradebook. They are:
- multiple “Respondent Pools,” which allows multiple surveys to use a common data store and shared reporting mechanism;
- “Respondent Pool Metadata” to store additional data elements that describe a Respondent Pool with that pool’s data;
- a method for embedding the metadata in both an email and a survey when delivered to a respondent;
- the “Rubric Question Type,” rubrics are a commonly accepted assessment tool;
- the “Dashboard” for automatic, real-time web-based reporting of data from one or more Respondent Pools to one or more authorized viewers;
- tabular reports of data, available for both the survey overall or for an individual Respondent Pool or selected groups of Respondent Pools;
- a mechanism for securing of report data without a password, including sharing the results when a rater finishes a rating session;
- a mechanism to get the tabular data from the Skylight Matrix Survey System to a third party system via a uniform resource locator (URL);
- a mechanism to download all data and metadata associated with a survey and/or Respondent Pool, including URLs for taking the survey, in a format readily used by spreadsheets.