Agenda June 30, 2010 Review of Recent Feedback to Programs Part II
Goal 1: Assess accuracy and alignment between a program’s score and feedback.
· We have made progress on this goal through last week’s review of Anthropology, Music, Geology and Communication feedback reports.
· Each OAI contact should carry out a similar exercise with each of the contact’s program feedback reports to ensure alignment. When there is misalignment, a discussion with the reviewers and reconciliation of scores and discrepant language needs to occur. This should currently be happening with Geology, Music and Communication.
· A next important exercise is to ensure consistency of look and overall message across feedback reports.
1) Standardized Language: each feedback report must have the following standard language at the top of the report (let’s read it now and offer feedback so we can make any necessary revisions asap):
Please find below the feedback to your program’s 5/17/09 assessment report, the second in the WSU Learning Outcomes Assessment System cycle. As in the December round, your program’s report was read independently by two reviewers from the Office of Assessment and Innovation (OAI). Your OAI contact was not one of the reviewers (simulating the accreditation review process we anticipate in fall 2010). Additional comments and suggestions by your OAI contact are located at the end of the document.
Each OAI feedback report includes observations, commendations and recommendations, using language from the WSU Guide to Effective Program Assessment Rubric, which are intended to help programs shape their next steps for assessment as well as future reporting to OAI and accreditors. Recommendations are in italics. This feedback is intended to guide programs in their next steps for assessment as well as future reporting to OAI and accreditors. Your OAI contact is happy to discuss any of the comments, questions or recommendations. We encourage you to proceed with action plans that will make your assessment efforts more fruitful; the OAI is available to help in those efforts.
OAI hopes that it has effectively conveyed the goal that the report you have shared is to be understood as a living document that will be used to respond to those interested accreditors and stakeholders in WSU’s mission– The Northwest Commission for Colleges and Universities (NWCC&U), The Washington state Higher Education Coordinating (HEC) Board, the Office of Fiscal Management (OFM), WSU’s Annual Program Review (APR) process as well as others as needed.
The next program assessment report is due Friday, AUGUST 20, 2010. This should include revisions to address OAI feedback. The report submitted in August will be used as evidence for WSU’s assessment report to NWCC&U, which will be submitted October 15, 2010. Please download a fresh template from the OAI website at http://www.oai.wsu.edu This site also provides a number of resources that you may find helpful during the revision and drafting process.
2) Any personalized comments must be put into an email and not be included in the official feedback report. A copy of this email must be placed somewhere for OAI future reference (Location TBA). There is also standard language that must be included in the email to program points. It is similar to the feedback report language and can be accessed at this link:https://assessment.wsu.edu/sites/OAITeam/2009-2010/Documents/Forms/AllItems.aspx
3) All radar charts must be de-wonkified. Currently they are all looking different. We need to decide whether or not the ratings themselves should be noted on the chart and/or on the headers for each section of the feedback report.
4) Reviewers’ comments must seem as ‘one voice’ and be edited by the OAI contact for consistency/alignment/clarity. No Reviewer 1, Reviewer 2. If there is concern regarding over-editing, the OAI contact should discuss with the reviewers. There has been a suggestion to organize the feedback by dimension category – what do you think about that?
5) Be sure to review the feedback given to the program from round one prior to sending the feedback back to the program point. In the case that the programs are receiving lower scores, then that will need to be part of the conversation with the program point. We are currently working on crafting some language regarding changes made to the rubric that could be used to explain lower scores. (Nils will show his draft and talk about it).
6) There has been a suggestion to add a scoping/contextualizing statement at the top of the August Template. (Kimberly will show her draft and talk about it).
7) We need to do another round of looking across programs to ensure consistency of scores/categories in order to move forward on Goal 2: Assess consistency of feedback and scores across programs — does the same score/category represent similar levels of quality?
The 14 programs who’s data is in process actions today are tabulated below.