Norming and workflow for May 17 self study reviews This is another post linked to the notes for OAI Team in this note https://communitylearning.wordpress.com/2010/06/02/process-for-creating-may-17-responses-to-programs/
There are three images
Results of norming on the Sociology Self-Study from Dec using the May 17 rubric. This chart compares OAI results with the results Gary & Nils brought back from the Assessment, Teaching and Learning conference the end of April. In that setting about 40 conference attendees rated Sociology, with about 10 people rating each dimension. No person rated all 4 dimensions. This session was done without norming, as part of sharing the WSU system.
A key agreement in the norming process was how to treat the self-studies.
1) the whole document is being read to inform each dimension, even if the particular section of the report does not describe the features sought in the rubric.
2) no “benefit of the doubt” or “inferring;” if the program did not describe a feature of its process, then it is to be treated as missing — an attempt to simulate the practice of outside accreditors. It was hoped that the feedback would be used by raters to communicate to programs in constructive ways when the score did not reflect what was known to exist.
Work flow for rating reports walks through the raters and OAI Contact roles and the flow of the documents and data. This process is somewhat changed from the December process, with more focus on how the raters reconcile and provide feedback to the OAI contact– with more emphasis on how to provide feedback to the program.
Plan for first week is a sketch of the timeline and scheduling to help OAI think about how to coordinate among various raters and reviewers.