Draft note to Deans/Chairs to Come From Provost

Draft note to Deans/Chairs

Larry,

Here’s our first draft.  With your modifications, I hope we can entice a few hold outs to join in the fun.

Deans and Chairs

WSU is preparing its response to NWCCU’s Final Report. As part of that effort, the Office of Assessment and Innovation is working with all undergraduate programs to collect self-studies on program-level assessment activities. OAI has provided feedback to programs who submitted draft self-studies in December, and my office also used that preliminary information as part of a report to the HEC Board in March.

I am reminding you of OAI’s deadline of May 17 for a revised self-study, including the Action Plan and Evidence section which was not required last December. I am asking you to ensure that your program provides a self-study in May, using guidelines found on the OAI.wsu.edu website, or that your program is in contact with OAI to make other arrangements.

Timely delivery of self-studies will help OAI by allowing the time necessary to review and compile data from all programs for the report that WSU must provide to NWCCU in the Fall of 2010. My office is also working with OAI to develop a mechanism where the information you provide in these self-studies can be used to also meet many of the requirements of the HEC Board’s Annual Program Review. OAI, Institutional Research and my office are working to streamline as much of this reporting as possible.

College Liaisons, Program Points and OAI Contacts can all be found here https://universityportfolio.wsu.edu/2009-2010/Lists/Programs/All.aspx

NWCCU’s Final Report can be found on the Accreditation.wsu.edu website.

Thanks

Why Public Reporting

One faculty member responds to OAI assessment plan feedback:

“Oh -kiss my #$$, I have been here long enough to know that THAT thing
will be flushed down the toilet.”

[SACS] Student Learning Outcomes – Business

On the question, does every major have to report outcomes:

From: SACS Commission on Colleges Discussion Forum [mailto:SACS-L@LISTSERV.UHD.EDU]
Sent: Wednesday, March 31, 2010 2:57 PM
To: SACS-L@LISTSERV.UHD.EDU
Subject: Re: [SACS] Student Learning Outcomes – Business

I can guarantee it is every major within a degree. Also you must show that every program has been assesed and that improvement strategies have been implemented; not plans to improve. Also at your reaffirmation every program must be finished with this process not just a portion.

Comment to the original post

SACS is not NWCCU

Edit
I believe they are somewhat farther down the assessment trail. 

Can we ask this same question directly of our accreditation agency?

Yeidel, JoshuaNo presence information at 3/31/2010 5:39 PM

Faculty Senate

At Faculty Senate, Thursday, February 18th, concerns were expressed about the assessment initiative.  The concerns largely reflected, we are informed, questions about who was being asked to do this.

On Monday, February 22nd, we have been invited to provide a report.  I have proposed we share the HEC Board report we are preparing this week.

Status Update: From Harvesting Gradebook to Learning Outcomes Assessment

The Cal State University system has been holding an internal series of webinars on ways to integrate and assess general education, including the use of ePortfolios, the VALUE rubrics, and themes like “sustainability.” By invitation we just presented a summary of the harvesting gradebook and beyond

Thursday, February 11, 2010
10:00-11:00 a.m.

Draft abstract

WSU does not have a centrally endorsed ePortfolio tool, and has been moving to view the web itself as an ePortfolio. You can read a summary of our (somewhat radical) thinking here: https://communitylearning.wordpress.com/2009/09/27/not-your-fathers-portfolio/

One of the challenges inherent in the strategy is how to manage assessment when student work is scattered across the Internet. To meet that challenge, we have been developing a tool called the “Harvesting Gradebook” that allows multiple reviewers, both inside and outside the institution, to give rubric-based feedback to learners wherever the learners’ work resides.

This “embedded assessment” approach has the advantage that it can be rolled up from program to college to university level for meaningful program review. WSU is piloting such a system for use in their NWCCU accreditation reporting. Their concept piece can be found here: https://communitylearning.wordpress.com/2009/09/21/from-student-feedback-to-university-accreditation/

In the effort to balance the tension between accountability and assessment, WSU is currently refining a rubric ( https://universityportfolio.wsu.edu/2009-2010/December%20Packet/Guide%20to%20Assessment%20(Expanded).pdf ) to provide formative feedback to academic programs about their assessment work.

WSU’s work is attempting to articulate, coordinate and be accountable for student learning outcomes across many scales within a diverse university setting.

Institutional Self-Assessment Rubric

This post is in support of a TLT webinar, the series is titled “Power of Rubrics
[Archive of session]
Gary Brown, Theron DesRosier, Jayme Jacobson & Nils Peterson, Washington State University

Introduction and Background on the Problem

Washington State University is in the process of responding to changes in by its accrediting body, NWCCU. The response includes the transformation of the former Center for Teaching Learning and Technology (CTLT) to the Office of Assessment and Innovation (OAI).

The University is in the process of developing its response to NWCCU’s changed standards and OAI is helping move the institutional thinking toward one that embeds assessment in ways that help faculty to think about student learning outcomes, and about the processes that programs are using to assess their work on improving outcomes.

This work builds on work of the former CTLT known to as “Harvesting Gradebook.” Previous reports will provide context on using the Harvesting Gradebook with students: AAC&U report Jan 2009 Update Fall 2009. This report links to a webinar archive that paints a picture of how to roll harvesting up, From Student Work to University Accreditation.
Using Harvesting Feedback with Academic Programs

In the previous webinar (From Student Work to University Accreditation) we described a vision for how harvesting could be used to move data from the level of an individual piece of student work up through levels of assessment and reflection to a university-level accreditation report. Presently OAI is engaged in deploying a middle level piece of this vision, the assessment of program-level self studies with an “Assessment of Assessment” rubric. The most current version of the rubric and other materials for the process are linked from grey portion of OAI website banner.

Figure 1. The process involves the academic program collecting evidence, writing a self study, and having the self study assessed with the University’s rubric (Called Guide to Assessment on the OAI website, formerly called Assessment of Assessment rubric). This image shows the process from data sources (upper left, to self study, to rubric-based assessment, to radar graph of results. This diagram represents work on the level of an academic program, a “middle tier” in the vision presented in From Student Work to University Accreditation.

Readers interested in trying the process are invited to do so at the WSU University Portfolio site for 2009-10. The Department of Rocket Science (on the site as of 12/7/09) was created as a sample. Other programs appearing on the site (beginning in January 2010) are actual WSU programs seeking formative feedback. (strikeout March 2010) Contact us if you want.

A Prezi visual of the WSU assessment calendar, provides an interactive picture of the assessment cycle and calendar and will serve as a “Dashboard” for monitoring progress.

Guide to Assessment – Rubric
Because of the wide diversity of programs in the WSU 4-campus system, a one size approach to learning outcomes assessment will not fit all. Consequently, WSU is developing a rubric to assess the self-study plans (short form and long form). Like the AAC&U VALUE project, the WSU rubric assumes that “to achieve a high-quality education for all students, valid assessment data are needed to guide planning, teaching, and improvement.”

The Guide to Assessment is the tool OAI is creating to help programs assess the quality of their student learning outcomes assessment activities. Using the Harvesting mechanism, programs will be able to gather evidence from stakeholders outside the university — a requirement of the accreditor — as well as gathering self-, peer- and OAI reviews.

Short form of the Rubric

Suggested Script for Cold Calls to Program Points

DRAFT DRAFT DRAFT
Cold Calls to WSU Programs

Who you are
·        This is a beginning of the process of actively contacting all programs to solicit their participation in the Dec 18 self-study reporting. Prior to this point, contact was indirect, via the College-level program liaisons.

My name is _______________ with the Office of Assessment and Innovation.  As your College Assessment Liaison probably told you, accreditation and accountability expectations for higher education have been changing, and the accreditation process for us will now require updates every two years with a report on learning outcomes progress due next fall, 2010.

What your role is
·       I am the designated OAI contact selected to work with you and your program.  As you may know, the Office of A and I was established to meet new assessment requirements and to help do so in ways that keep the focus on improving the student learning experience.  Our charge is to help WSU programs develop, if they are not already in place, assessment strategies that meet new assessment requirements in ways that leverage faculty expertise and that are meaningful and useful for WSU faculty.

What you understand their role to be
·       As you may know, in order to meet the fall report deadline, some level of assessment needs to be conducted next spring so that the results can be used to guide improvements.  It is using evidence to ‘close the loop’ that is the common denominator of assessment requirements coming from all of our stakeholders, including the NWCC&U, professional accreditors, OFM, the legislature, or the HEC Board (all now pressing us for accountability).  I’m calling because I understand you are the point person for your program and I want to make sure you are aware that there is a December 18th deadline for sharing your plans.

The Process Overview
·       After you have shared your planning document in the template provided to you by your college assessment liaison, available from me, or accessible online at (https://universityportfolio.wsu.edu/2009-2010/Pages/default.aspx ), your plan will be reviewed and feedback provided. That review and feedback will be based on criteria available in the ‘Guide to Assessing Assessments [A of A].’  The Guide is being developed to help clarify the principles of assessment that our accreditors expect, and of course it might be a useful resource as you prepare your plan.
·       The same systematic process, our charge, will be conducted as WSU’s and your program’s assessment continues, and ideally you and your team will join us in refining the criteria and process to help us make the activity as useful as we can for improving the WSU student experience.
·       To be clear, this process at this point is intended to be formative assessment that we will use so that we can do more than comply with accreditation–we want to help shape it while the opportunity for us and for our accreditors is still possible.  We hope you may also help us identify colleagues in your field who might find this exercise useful and to ensure that the review of your program is conducted by experts you know and trust.
What we can do to help
·       It may be that what this activity entails is overwhelming or confusing right now.  The OAI was established to help you navigate this challenge and to do it in a way that is effectively integrated into your everyday teaching practice.  We have developed a number of strategies over the last several years that you may find useful, and we have developed and identified a number of resources that can help.  Let me know if you want to meet and talk about this, and of course I welcome the opportunity to meet with any of your assessment team who are available as well.

Comments added to original post

questions for an initial mtg, F2F or phone or whatever

Here’s what I try to run through to start the discussion of prog assessment:

Initial conversation about program assessment (phone or F2F)

Tell me about your program.
I’m not very familiar with XXXXXXXXX; could you tell me a bit about your program, students, faculty, and capstone to help us plan an appropriate pilot assessment.

What do you think is working well in your program?  What’s an issue that you all struggle with?  (See if they can identify a question.)

Does the program already have student learning goals?

What direct measures to target

Capstone:
•Does your program have a capstone project?
(Basic info re mode, size, how many students / projects each semester/year,  All seniors?  Mostly seniors?  Timing  )
•Which student work / class is appropriate for capstone assessment?  Individual or group work?
(If it’s a huge project, could students write a short piece about the project,  3-4 pages providing an overview of their thinking about the project, and reflection – like something you might include in a portfolio or prep for a job interview —  and the project is an “appendix” to that, for our assessment??)
•Logistical issues to address?  Format of project or other.  Can we collect digital copies? Need clean, unmarked.
•Assignment prompt and course syllabus
•Logistics of collecting student work this semester.

Lower division core class
•Does your program have a lower division core class or classes?
(Basic info re mode, size, how many students / projects each semester/year,  All first year students?  Mostly?  Timing  )
•Which student work / class is appropriate for assessment?  Individual or group work?
•Logistical issues to address?  Format of project or other.  Can we collect digital copies? Need clean, unmarked.
•Assignment prompt and course syllabus
•Logistics of collecting student work this semester.

Other stuff
•Do a program inventory.
•Who will participate in assessment?  Faculty, instructors, TAs, others
•Look at the timeline for getting started. General timeline this semester and next (attach our sample timeline)
•Student learning goals – into a rubric
•How do your faculty share ideas about teaching, or learn new teaching practices?  What kind of teaching resources (articles, workshops, conferences, brownbags, etc) are used by instructors?  How are they shared, informally or formally?

Green, Kimberly at 12/18/2009 12:33 PM