OAI Working Meeting agenda and whiteboards

OAI Working Meeting agenda and whiteboards

On Jan 28, 2010 OAI staff had a working meeting to revisit the evolution of the organization from its CTLT roots of a year prior.

Questions of: what is our mission? to whom are we responsible? and what is my role? have been in tremendous flux and the university and unit transitioned.

The meeting today was aimed at reflecting on what we know now, from the vantage of nearly completing the Dec 18 review cycle. Another goal was to identify some useful and scalable resources that could be assembled to provide to programs. A final goal was to spend some time with current literature related to the work.

The attached agenda is a near final draft, updated just prior to the session with the current OAI mission & goals. Photos are of the whiteboard created during the session.

OAI staff working meeting agenda 1.28

A program’s response to next steps after receiving feedback

A program’s response to next steps after receiving feedback On Jan 28, 2010 the program representative replied

Nils,

After our meeting on Tuesday, I began recruiting reviewers for the norming session.  Everyone is busy and no one volunteered.  So, I brought this up with the chair who is unwilling to commit any resources, including faculty time, to reviewing the theses.   Therefore, the rating of the theses is not an option at this time.

However, I am willing to continue to work on preparing the self study to be submitted by May 17th.  Once I receive your propose changes, I will review them and distribute to the faculty for their comments.

Thanks,

On Tue, Jan 26, 2010 at 9:52 AM, Nils Peterson wrote:

Thanks for meeting this AM. In the future, I’m happy to reciprocate and go to your office, just share that with Judy.

Here are my notes summarizing what I believe we agreed. Please feel free to clarify or correct anything below.

Regarding the rating of thesis documents:
You have the 6 thesis documents, each about 20 pages. You are working on identifying raters, at present yourself and 2-3 others are likely.

I am going to organize OAI resources for a norming session to take place in about 10 days — perhaps the week of Feb 8. I will need your help arranging a 90 minutes that will work for the raters you are identifying. Please confirm if that week or the week following are preferable.

The process will use paper copies of the thesis, with names removed and code numbers applied. I will get you a guide sheet for doing this document preparation.

I will create an online survey using the rubric in your program’s self study document, and make you an owner of that survey so you can take over the process moving forward.

Following the norming session, raters will have until the end of spring break, March 21, to complete their rating tasks.

OAI will help you get reports of the data and think about next steps after the rating.

Regarding the December draft self-study
You have been sharing the OAI feedback with the chair and are continuing to think about the recommendations and which are politically and practically feasible

I will use Track Changes to propose a new draft of your self-study by the week of Feb 8.

You will accept/modify/reject my suggestions based on the context and we will discuss further revisions.

We did not discuss, but I would hope we have a revised version to send through the OAI rating process by March 8.

Regarding the May 17 self-study
Beginning after March 21, I will help you think about the results of ratings and the feedback on the revised shelf study in the process of creating the self-study due May 17

Thanks again for your attention to all this

Nils

Working towards 100% participation by faculty

I chatted with Gary after someone in a program asked me what actual threat existed that he could use to coerce his (more senior) colleagues into participating in program assessment.

I mentioned the carrot approach — all the potential benefits — but I figure that this question will continue to come up, and I sought out Gary for a more official answer regarding the Stick.

This is what I came up with after that conversation with Gary, which afterwards got his ok.

Regarding broad participation of faculty in assessment:

One thing the accreditors cited WSU for last year (2009) is the low level of participation in assessment found in many programs … a few people doing most or all of the work of assessment.

Getting more folks to participate:

Carrots:
·       What does your faculty want to know about its program, about student learning?  Program assessment can and should be designed to answer a question you all care about.

·       A significant number of faculty must be involved in assessment, including senior faculty, for these efforts to systematically impact a program.

·       Quality assessment supports ongoing program improvement, contributing to your program’s reputation.

The stick:
Programs whose assessment doesn’t have broad faculty involvement —  i.e., not close to full participation — may  jeopardize WSU’s institutional re-accreditation, which is risky particularly  in the current era of funding cuts at the state level and of increased competition (including online education) in general.

Status meeting 1.21.10

Status meeting 1.21.10 I’m going to try to be more systematic about recording something from our project status meetings, at least an image of the agenda/timeline.  Today we did a review of the process/SharePoint tool implemented from last week’s meeting. Various communication and procedural bumps still to iron out, and a few upgrades to the tool now that we are using it. Generally seems to be working; need to get all the data in so we know where we are and can drive for a 1/29 wrapup of returning reports to all programs that have turned one in (Dec 18 deadline + stragglers)

Assessment, Accountability, and Improvement: Revisiting the Tension

Assessment, Accountability, and Improvement: Revisiting the Tension
A key piece from Peter Ewell and NCHEMS, and a good resource for those who wonder why we do what we do the way we do.  This assessment of assessment is both a rationale and a blueprint:

Ewell says:

“Institutional accrediting organizations remain membership associations, however, so they cannot stray too far toward establishing common standards and applying them through aggressive review.”

“They also remain extremely limited in their ability to influence the majority of institutions not at risk of losing accreditation.”

“The future effectiveness of institutional accreditation in both promoting good practice and in reinforcing the academy’s assumption of consistent and transparent standards of student academic achievement lies entirely in the hands of the academy and its leadership.”

peterewell 2009

Implementing a workflow for Dec 18 Program Review

Below are Joshua’s notes for folks interacting with his new SharePoint mechanisms for implementing a workflow for our Dec 18 review process.

The attached image is from Thurs 14 (last week) where we diagrammed out the steps. Joshua implemented a trial mechanism to capture the remaining Dec 18 work as a test run for the May 17 work we are anticipating. Ashley has started populating that structure in Assessment.wsu.edu

In the figure, there is an agenda associated with the meeting and to the right of it, in blue and red, in Nils’ hand, a set of steps flowing down and to the right and an associated numbered list of steps in red.

—— Forwarded Message
From: Joshua Yeidel
Date: Tue, 19 Jan 2010 14:17:04 -0800

—————–DRAFT—————————–
Here are some tips for making successful use of the internal workspace for the 2009 December Planning document reviews:

There are two components:  a “Process Documents” library for non-public documents related to program reviews, and a “Process Actions” list which identifies what the next action is for each program’s review, and to whom the action is assigned.  Both components can be accessed via links in the left-hand navigation bar.

When a new review comes in, if it doesn’t already have one, create a “New Folder” with the program’s name in the “Process Documents” library.  This folder will be used for any internal (non-public) documents related to the review (e.g., draft responses).  Public documents (such as the original submission) go in the “2009-2010” site on UniversityPortfolio

ALSO create a new tracking item with the program name in the “Process Actions” list.  Set the “Next Action” and “Assigned To” fields appropriately. This will cause an email to be sent to the new assignee.  To identify the “Assigned To” person, Enter the person’s WSU network ID or “lastname,firstname”, then click the checkmark icon on the right end of the text box to “Check Names”.  The server will replace the id or name with an underlined name, meaning that a match for that entry has been found.  If no match is found (usually due to a typo), and you are using Internet Explorer, you can right-click on the unmatched entry for further search options.

Each time an action is completed, update the Process Actions item via the “Edit Item” button or drop-down menu item.  Change the “Next Action”, “Assigned To” and “Instructions” fields as needed.   This will cause an email to the assignee in which those fields are listed.

If you don’t know what the next action for a review should be, you can select “Figure out next action” at the bottom of the Next Actions pick-list, and assign it to a director.

You may assign a review to more than one person, but “Next Action” can have only one value at a time. (For example in cases where raters need to reconcile divergent ratings.)

Each “Process Action” item includes a “Log Notes” field which is cumulative.  When you make an entry, it is displayed at the top of a list of previous “Log Notes” entries for that item.  Use this for any information that is important to track (e.g., “rating cannot be completed until Joshua returns on 1/25/2010”.  There is no need for log notes of routine information.

Please let me know if you have successes or if you experience any difficulty.  This is a pilot of a tracking method we may use for the May cycle, so please surface issues so they can (hopefully) be addressed.

— Joshua

—— End of Forwarded Message

Notes From Design Circle Meeting

FW: Notes From Design Circle Meeting

Today we did some post-reviews reflection. Ashley led an activity that had the leadership and Designers doing a jigsaw read of a selection of the responses we have drafted to the Dec 18 planning reports.  The goal of the activity was to surface kinds of resources that would be helpful to have for programs — things that stand alone or work with mediation, to help programs understand and respond to the feedback.  The discussion produced a whiteboard of possible materials: things written by others with long shelf life, things that OAI might develop (and evolve), WSU-centric annotated samples, and other thinking, captured in these documents.

Also worth noting is this thinking about how an online community might be created that would expand the interest in, and ability to create, the documents outlined today.

attached document Notes from Design Circle Whiteboard