Last summer we took Gary’s Harvesting Gradebook idea from concept to implementation and during the school year we conducted two pilots in classes. Figure 1 is a diagram of the concept, showing how data can serve the needs of student, instructor and academic program. Here is a live demo you can try. One of our goals of the demo is to illustrate that reviewers can give rubric-based feedback in 5-10 minutes – a level of time commitment we think is reasonable to ask of outside reviewers.
This summer our goal is to take the concept to production and integrate it with Washington State University’s systems. Figure 2 is our whiteboard analysis of the life cycle of the process. The key element is to be able to produce a spreadsheet with two columns, the student identifier and a letter grade summarized from the harvesting process.
Step 1. Create surveys for students to embed within their work
The life cycle of the process begins with the Registrar, where students select their classes. The data are extracted from there, either directly by faculty as they do to create their class lists, or from a shadow system (CTLT’s Enrollment Web). After massaging in Excel, the data are uploaded into Skylight Matrix Survey System to create the Respondent Pools that are the individual student surveys. Metadata about the student (importantly WSU ID number) can be included in the upload process so that it will be available in the reporting process to link students with their grades.
Step 2. URLs for linking the survey to the student work.
Student are assumed to be working in a variety of media and locations on the Internet. The ideal situation is to embed a survey in situ, less ideal (but more practical) is to place a URL to the survey with the work. There are two mechanisms for getting the URLs (created in step 1) distributed to students.
In Step 1, students can be authorized to the Skylight Dashboard (this tool was created for faculty for course evaluations, but it works the same for any person who is the subject of a survey). In the FAQ of the page above is information about getting the URL to the survey.
Alternately, an instructor can download an Excel report for the whole survey and obtain the URLs from it.
Step 3. Students access to the Harvested Feedback
Students can see summary data in their survey using the Skylight Dashboard. They can also download the raw data and/or can make a customized report (or use a template provided to them to get a customized report). Customized reports have great potential to reformat and visualize the data. For example, in the live demo this Custom report was created in Google Docs, and the resulting graphs were displayed in a SharePoint space (which could be a student’s portfolio).
Step 4. Converting data for use in traditional gradebook
The instructor can use a Standard Report from Skylight to get all the data for the class and then in another Excel sheet have it automatically processed into letter grades using formulas chosen by the instructor. Then a final Excel sheet is created to merge student identity information (downloaded from Skylight, e.g., name, WSU ID) and letter grade. This latter sheet is a key part of the following strategy.
Step 5. Reporting letter grades to students
While students can see their feedback and rubric scores in the Skylight Dashboard, they do not have access to the resulting letter grade (which may be a function of several instructor-set parameters). Using the Excel sheet discussed at the end of Step 4, the instructor can upload the grade information as a column in a gradebook within a course management system. Different CMS systems have slightly different requirements for this transaction, but all support it generally. Within the CMS gradebook, the grade from the harvested activity can be included with scores from other sources (e.g., quiz). The instructor can calculate a cumulative grade for the course from these multiple sources.
Step 6. Reporting the final grade to the Registrar
The instructor can download the scores, and final grade, from the CMS and with suitable adjustment of columns, be prepared to upload the final results to the Registrar at the end of the term. This activity, while an important productivity enhancement for the instructor is a general one that may be beyond the scope of this work.