Status Update: From Harvesting Gradebook to Learning Outcomes Assessment

The Cal State University system has been holding an internal series of webinars on ways to integrate and assess general education, including the use of ePortfolios, the VALUE rubrics, and themes like “sustainability.” By invitation we just presented a summary of the harvesting gradebook and beyond

Thursday, February 11, 2010
10:00-11:00 a.m.

Draft abstract

WSU does not have a centrally endorsed ePortfolio tool, and has been moving to view the web itself as an ePortfolio. You can read a summary of our (somewhat radical) thinking here:

One of the challenges inherent in the strategy is how to manage assessment when student work is scattered across the Internet. To meet that challenge, we have been developing a tool called the “Harvesting Gradebook” that allows multiple reviewers, both inside and outside the institution, to give rubric-based feedback to learners wherever the learners’ work resides.

This “embedded assessment” approach has the advantage that it can be rolled up from program to college to university level for meaningful program review. WSU is piloting such a system for use in their NWCCU accreditation reporting. Their concept piece can be found here:

In the effort to balance the tension between accountability and assessment, WSU is currently refining a rubric ( ) to provide formative feedback to academic programs about their assessment work.

WSU’s work is attempting to articulate, coordinate and be accountable for student learning outcomes across many scales within a diverse university setting.

2 Responses

  1. This response from Terry Underwood (with his permission), a professor at the Sacramento State campus. Like Ken O’Donnell, he’s involved in the national GE reform effort led by the AAC&U, called Give Student a Compass. At the end of the email he considers a connection to “deliverology,” which is the CSU’s current push to improve graduation rates, particularly among the historically under-served.

    This was an incredibly useful presentation. I learned so much — I know so little about how to use technology in all of this. Are there any technical manuals or articles or other documents that might help someone like me understand a little about how to at least talk with our tech people about our needs?

    I saw one huge potential for connecting Community Colleges with the CSUs, and taking it a step further — including high school teachers. Using these online systems to invite and incentivize “externals” to assess not just student work, but assignments and rubrics, affords great potential for collaborative work on instructional strategies and assignments involving all of these stakeholders. Instructors in high school could upload assignments etc. for feedback from college instructors and give feedback on assignments uploaded by college instructors (CC or CSU).

    This would not have to be elaborate to get started. Written communication or critical thinking could be fruitful starting points, and the idea would not be to immediately reform instruction, but to build a platform for more informed instruction across the segments.

    As I watched the presentation, I was reminded of the “shopping cart” that emerged during the development of the PACT assessment (Performance Assessment for California Teachers). You may or may not know that PACT is an alternative teacher certification assessment that meets the Commission’s standards. Anyway, the original notion in PACT was to create two channels of data—a formal one with a structured portfolio, common tasks, reliability checks, etc. and an informal one made up of “signature assignments.” These assignments were to have maximum flexibility yet still plug into one or more dimensions of the formal rubric.

    The idea was to capitalize on local strengths by way of sustaining good practices and then to examine data from the signature assignments in relation to the formal, externally scored teaching event portfolio as a well to see into instruction and serve as a reliability check on the system.

    To develop and share these, Bob Calfee proposed a “shopping cart,” an online place where faculty could “shop” for assignments, rubrics, examples of student work, etc., to use for a variety of purposes, including direct application to teaching a course. As I recall, the only limitation on a contribution was the requirement that it connect reasonably to one or more of the guiding questions in the formal rubric (we have a parallel set of circumstances with the VALUE rubrics as the hub).

    Unfortunately, the daunting challenge of the portfolio itself, I think, short-circuited the full development of channel one, the signature assignments, and I don’t know where PACT is at this point. But I think—given the absence of a legal requirement for strict, psychometrically defensible measures in undergraduate education—the validity and practicality of the shopping cart could be a great thing in this context.

    Because instructors at all levels of the system could benefit enormously with just simple glimpses into the ratings of colleagues at different levels within the system, setting up a “workshop and shopping cart” where educators can help build and make use of assignments and working toward a culture that fosters such collaboration could be a very, very good thing for deliverology.

    Thanks. This was so very helpful today!

  2. This was a very engaging, thought-provoking presentation. I’m the organizer of the CSU webinar series Nils names in his post, and this one was particularly well received.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: