An invitation to shape my presentation at P3

I am going to be participating in a one day event Peer-to-Peer Pedagogies (P3) Workshop September 10th, 2010 sponsored by HASTAC on the Duke campus.  As part of that, I’ve been assigned to work with Kim Singletary on a bio/ speaker intro piece.

The event was great fun. Thanks to all. Here is the SlideShare from my talk. Coming back on the plane I remembered this piece by Downes on Open Assessment that seems as timely as ever.

This post is an invitation to collaborate with the two of us as we co-develop the materials that Kim will ultimately push into final form. At the same time, I see my dialog with Kim as sharpening my focus for the event. Please weigh in using this blog’s comments. I will continue to edit the body of the post based on your remarks, and any backchannel with Kim, or other readers.

After reviewing some of our blog (written with colleagues of mine at Washington State University) Kim sent a list of questions to get the discussion going. Here is the first of those questions.

Kim: What do you hope will be accomplished at P3? What is your goal for your appearance at the conference?

Nils: I feel like the open learning, peer pedagogy landscape is loosely defined and rapidly changing and I hope to come away with some ideas about where the clusters of agreement lie in this terrain. Brian Lamb and Jim Groom just wrote a piece in Educause Review “Never Mind the Edupunks; or, The Great Web 2.0 Swindle” July/Aug 2010 where they are suggesting that the open learning wave may have crested. They provide an interesting perspective on the shoals that may have dashed the vision.

In the same issue Dave Cormier and George Siemens had a piece on their experiments with open online courses, “Through the Open Door: Open Courses as Research, Learning, and Engagement”   See also some response in Dave’s blog.

My colleagues and I at Washington State University Office of Assessment and Innovation have been approaching this conversation with an eye to two contexts:

  1. The need to expand higher learning opportunities in the developing world is so large that it challenges historic thinking about brick and mortar institutions. John Seely Brown, Don Tappscott and others are talking about this challenge. Some of our analysis of this context is here.
  2. The other context (given our assessment/accountability role at WSU) is the increasing press for accountability and the rising volume in the discussion at national levels about the failures of US higher education to be accountable for its students’ learning outcomes. See this stream of bookmarks on the topic maintained by Gary Brown.

In that context, we’ve been looking at how Personal Learning Environments, or student-owned ePortfolios, can facilitate learning among communities (including peers) and how the learning in those contexts can be credentialed.  A condensation of some of that thinking appears in our discussion of our emerging understanding of an ePortfolio.

We have been developing those ideas, in part, by watching the peer learning and collaborative problem solving of one of WSU’s doctoral students, Margo Tamez.
She blogs:

I would advise anyone who is seriously committed to the larger structural uses of E-portfolios by contemporary students– who by the way do not view the college classroom as their most significant ‘center’ of knowledge and tool acquisition, (providing one is inclined to view the technology and control over the production as the domain of liberatory education)–,to examine their assumptions about knowledge and power and the political will of the Indigenous peoples.

So, one of the things I hope to get from the P3 conversation is more personal clarity on the ideas in this “4 models” figure that we created to think about the changing roles of the university as it moves from traditional brick and mortar, with traditional accountability, to a problem-based curriculum in peer-to-peer university with community-based accountability and credentialing. Along with the four models figure, we created a self-assessment to help clarify the conversation about where each of us in on an institution- vs community-based learning spectrum.

As Margo’s portfolio illustrates, open peer-to-peer learning is necessarily scattered widely across the Internet. Pulling that evidence together for assessment and credentialing is a challenge. Gary Brown coined the term “Harvesting Gradebook” to describe our approach to the problem of allowing learners to gather structured feedback on artifacts of their learning and to route evidence of that feedback back to a ‘home’ institution or to present that feedback as part of a claim within a community for a credential. An early description of the Harvesting Gradebook and how it might work in/out/with a university program appear here (skim down for “Variations in the Model” section).

In my P3 session I hope to introduce some of our Harvesting Gradebook thinking in a large scale real-time experiment using a backchannel where the audience can provide assessment feedback to my presentation.

16 Responses

  1. […] For me the discussion was a return to our previous conversation about the OAI’s web presence/strategy and my thinking about how to develop a professional learning community as part of that web strategy.  As I’m writing this, I’m also connecting to the nascent thinking I’m doing to get ready for the HASTAC P3 event in Sept. […]

  2. […] a course assignmentUpdate on Harvesting GradebookFrom Student Feedback to University AccreditationAn invitation to shape my presentation at P3Earning credentials in a learning communityHarvesting Gradebook in the […]

  3. […] is the page of feedback from my P3 presentation. Use this form ( ) to give more […]

  4. […] Posts An invitation to shape my presentation at P3Update on Harvesting GradebookCritical Thinking Skills Upgrade and PreziGoogle Wave unifies […]

  5. Hi all – I’m Kim; I am the one forcing Nils to answer all sorts of questions before the P3 conference. I emailed him a follow up question in relation to this post and he suggested I post it here.

    I’d like a little more information on who these potential students are. When you envision them, what kind of background do they have? Are they new users of technology or more advanced for their country or community? I’m right now, trying to wrap my head around the two points of interest, accountability and education in the developing world.

    I am thinking, specifically, of the comment from Miss Tamez, who argues that contemporary students don’t see the classroom as their center of information. I’m worried, slightly, that that statement may paint a broad brush, especially when we are talking about first gen, immigrant, or non-traditional college students. In many cases, the college classroom is a significant center for many students (although many may be hard-pressed to admit it).

    I’d like to know more about how you and your colleagues “see” these potential students. Where are those students located? What differences might there be between students who are from wealthier nations like the US, where teachers have to contend with students being bullied for being/acting too smart and those from places where attending school is a cherished privilege?

    I think this would better help me (and possibly others unfamiliar with the idea of e-Portfolios) better wrap our heads around how to address the two contexts you and your colleagues have outlined.

    • I recently have been thinking about how the 10 flatteners (Friedman) might be game changers for education (especially the preparation of new teachers). One of the first ideas in that list is when the Berlin Wall came down, it signaled an opening through which millions of people are now free – to learn and pursue happiness – with Internet resources as an ally, source, and personal assistant. These potential students, as Friedman suggests are indeed hungrier for knowledge and harder working than the U.S. norm.

      I see this in the high school kids who join up to do the Global Challenge Award ( The most persistent kids tend in larger numbers to be from outside the U.S. (There are certainly a few U.S. kids who are also persistent, but there are larger numbers from outside). We form teams that mix U.S. and non-U.S. students, who then work mostly online. We found, also in line with Friedman’s observations, that the U.S. kids excel in field-based learning, while kids from other cultures tend to stand around and wonder what to do next when there is no teacher present and the situation is more open-ended, but the non-U.S. kids excel at answering the test questions (this has been observed in the International Earth Science Olympiad ).

      In terms of techno-savvy and its relationship to e-portfolios (I have an eFolio that has been in production for about a dozen years – used in teacher and nurse licensing programs), I’ve found that I only have to teach tech-challenged adults how to use it. Everyone else pokes around and figures it out. Nowadays, with Linked In, Facebook and other social networking metaphors becoming ubiquitous, I think that Nil’s approach to thinking about the e-portfolio as a harvesting application across cloud (and crowd)-sourced spaces is an important way to go in the future (possibly integrated with an institutional-based e-portfolio approach) – and that if the user interface is on a mobile device, then the whole world will be comfortable using it long before educational institutions decide to adopt the practices.

      I agree with Tamez that people see the Web and their own media preferences as the sources of info for all of life. The classroom, I suspect, they see as a necessary requirement for advancement in work life.

      The real challenge for both informal and formal learning systems is (I think) whether learning is going to be guided by a helpful “other person” or not and whether that person (or capability) will interact with the learner to help shape ongoing thinking and work production; and then play a role in the validation of learning. Most classroom-based teaching is still a power relationship between the teacher and student (teacher knows, makes requests or demands and grades – students respond and hope for a good grade), but the game changer flatteners and participatory culture now emerging is evolving an alternative.

    • Kim,
      David gave one answer that I agree with, learning technology is not the problem, for example see Mitra’s TED talk on “Hole in the Wall.”

      And I prefer to talk about ‘learners’ rather than ‘students’ to recognize John Bransford’s perspective (How People Learn) that we are in school a short portion of our learning lives, and even when we are in school, our learning day is longer than the school day.

      I think Tapscott and Seely Brown, are suggesting that the learners in question are all learners with access to the Internet — a place where they can build learning communities. Seely Brown’s piece in EDUCAUSE Review is a good read

      Tapscott and Seely Brown are talking about both the developing and the developed world because World Internet Usage Stats show that the developing world is rapidly gaining access to the Internet.

      Dennis Haarsager at National Public Radio has spent the last decade thinking about the implications of the Internet on broadcasting. His focus is on the shift from information scarcity (making broadcasters important) to information abundance (making filtering, amplifying, connecting important).

      So when David answers above that learning technology is not the problem, I agree, but in the same breath I believe that the skills to navigate and learn in an information abundant environment need to be developed. And I think this context is what leads to Cathy Davidson’s thinking about ‘collaboration by difference.’

  6. David, Thanks for these interesting thoughts. I think we have some really interesting threads here for a discussion at P3.

    Nancy/Cathy. Looking at what is developing here, do you want to pull some of it into the HASTAC P3 wiki and invite a larger community in?

    Other interesting reading that someone new to this might enjoy is the HASTAC Scholar Forum from last fall Grading 2.0 Evaluation in the Digital Age ( ) with particular note of Cathy Davidson’s post Crowdsourcing Authority in the Classroom, here:

    This discussion has been very helpful to us at WSU thinking about pieces to prepare for P3.

  7. David replied again by email, adding Helen Barrett to the recipient list:

    Yes Nils, feel free to repost anything I send along.

    I think you are right that you are breaking through the bottleneck that existed when we wrote that piece. I’ve added Helen into this email because she might have some additional insights. I saw her recently (too briefly!) and I remarked that if Wave gets social tagging of the blips, then the e-portfolio tool development process will have reached a current zenith and we might not need any other tools for the time being (haha I’ve been quite wrong on this many times in life!).

    At the time Helen and I wrote about the CS versus GT way of thinking about portfolios, the idea of meta-tagging was in an era of a private, controlled activity of proprietary databases and community-controlled tags, so the open tagging and open harvesting idea was a bit distant. The generic tools of the day were not generally RSS-savvy (many were not even on the “web” in a social media context only a few short years ago).

    If we assume that open harvesting is possible, in a variety of formats and with tools that facilitate recapturing, reorganizing, re-contextualizing the contents – appropriating as Jenkins et al might say – then we will still have some interesting issues to face, that tie to the rubic-cube model. That model (in the “Decisions and Dilemmas” article in the Jafari book), in summary, has 3 dimensions of e-portfolios: Audience, Purpose and Artifact.

    We developed 3 subdivision ideas for each of these main dimensions, to make the point that there are a variety of questions that arise when thinking about an artifact, its history and the process that created it. The divisions are not exclusive (many can co-apply and they might have a combination of interfering and facilitating relationships). As we engage with each aspect, we draw in a set of related questions and issues. This made me think about the coherence and correspondence models of knowledge and truth – because the potentially conflicting or “complexifying” forces have to be reconciled or lived with in some fashion.

    Trusted others
    The wider public



    For example, if I undertake a process as an experiment for myself (such as composing a new piece of music) I might have my self as the audience as I experiment and mess around with ideas. I might have a “sonnet-like” purpose of expressing something abstractly for the joy of seeing how it will come out and how witty I can be. (I’m not thinking about how this artifact might be used later by someone else to judge whether I can compose music or not). I might choose a highly mathematical approach to manipulating the media I want to use. (this mathematical intention might not be readily apparent when you listen to the music). Suppose I do all this on the web and it can be harvested. Suppose I like the result and then decide to share it with the world.

    Now there is an artifact with my name on it, which may or may not be clearly connected with the contexts of its creation, the intentions of its creator, and so forth. The question for “assessment systems” is what can we know from interacting with this artifact? Do we learn something about the creator, ourselves, our culture and time, or more? The assessment system itself (be that another person, or an institution, employer, or even myself at a later time in life) is a rubrics-cube of multi-coherent purposes, audiences and artifacts; so how does it interact with this particular piece of music?

    I’ve attached the set of 27 questions that were evident at the time, that stemmed from the rubic-cube model.

    • Table 1

      Table 1.
      Questions for Decision-Making about E-Portfolios



      Trusted Others


      Focus of the


      Does the structure of the portfolio support its personal
      focus or reveal something about the learner?

      Does this collection provide an image that makes trusted
      others proud of their investment in the learner?

      Does the portfolio provide an image of a well-prepared


      Is there a primary message about or image of the learner?

      Does the portfolio represent the trusted group’s skill in
      guiding and advising me?

      Does the e-portfolio do a good job of representing the
      learner as a professional?


      Does the portfolio show how its parts are connected
      together and how they came to be or are still in stages of development? (e.g.
      "Working Portfolio")

      Does the portfolio contain a complete and sufficient
      linking to standards for graduation?
      Have the “Rubrics and Rules” (Carney, date) been followed
      and used? (e.g. "Working Portfolio" or "Graduation

      Does the collection include proof of readiness,
      certification, validity as a professional? Is it suitable for gaining
      employment? (e.g. "Employment Portfolio")

      Table 1. cont’d


      Ownership of the Artifact


      Are the statements created by the learner and for the
      learner’s benefit?

      Do the trusted others see clear connections in the
      portfolio to their professional culture and to the person who had an impact
      on his or her advisors and teachers?

      Does an appropriate public recognize the portfolio’s
      structure? Do they “own” the validity of the structure as a professional
      context for a representation of teaching?


      Does the learner have the power to create and present any
      kind of representation?

      Does the portfolio present an image that the trusted
      others would own or want to represent as their part of the history with this

      Does an appropriate public see a representation of an
      educator they would recognize as a peer? Is it an image they would own as


      Can the learner change the meanings of this collection at

      Does the portfolio document the advisors’ efforts to
      influence this person (or not) and why? Can the advisors trace their edits
      and influence?

      Does the portfolio show some of the important ways in
      which the pubic investment in this individual has or will pay off to others?

      Table 1. cont’d


      Affordances and Constraints of the Artifact’s Media


      Does the media enhance the content and structure of the

      Does the choice of media enhance the assessment and
      feedback on the portfolio?

      Does the choice of media influence the public perception
      of the portfolio?


      Does the choice of media enhance creativity and assist in
      the realization of the intentions of the learner?

      Does the quality of the use of media reflect the desired
      level of professional technology practice maintained by trusted others?

      Does the quality of the use of media reflect generally
      accepted professional practice?


      Do the affordances of the media (what the media allows)
      enhance the correspondence of the portfolio to reality?

      Do the affordances of the media (what the media allows)
      enhance institutional connections to standards or other external sources of

      Do the affordances of the media (what the media allows)
      enhance association with generally accepted levels of professional


  8. On Aug 13, 2010, at 3:51 PM, Nils Peterson replied to David by email:

    I skimmed the 2003 Gibson & Barrett article. One comment stood out “The ability to aggregate data for assessment is counted as a plus for CS and a minus for GT”

    We saw this issue also as an impediment for general tools (GT), and the Harvesting Gradebook idea is an attempt to break that bottleneck.
    [insert this image here: ]

  9. David replied by email (I’ll ask David if he can get the document he attached to this email online where you can access it. The attachment is different from the URL in his post below.):

    Hi everyone. I just landed from Australia and started getting back into focus today.

    Wow, fantastic blog posts Nils et al. I’m excited awed and a bit intimidated by your future-oriented thinking! I’d like to keep the tweet-track going as I play into the events (and I’ll add ideas to the blogs soon). Here is what’s on my mind right now and I’d like to check it with you all and also invited feedback and shaping from you.

    Helen Barrett and I wrote an article a while back (2003)

    Gibson, D. & Barrett, H. (2003). Directions in electronic portfolio development. Contemporary Issues in
    Technology and Teacher Education,(2)4. Available:

    in which we developed a rubric for a spectrum of approaches to e-portfolios (the “cloud” was not a common concept at the time). This led to further thinking that got put into the attached article, that contains a “rubic’s-cube” of issues that might still pertain to the discussions in today’s environment.

    a couple of others avenues that interest me are:

    1. How automated assessment might look stemming from games and simulations but crossing over into the cloud-based life-tracks that we are now leaving.
    2. How assessment theory is changing as a result of #1.
    3. The challenge (and need) for a new way to conduct educational research (not qual, not quant, not mixed methods but “modeling & visualization” methods) in order to begin to get a handle on the dynamical systems aspects of learning and educational environments.
    4. What global flatteners (a la Friedman) and complex systems thinking might mean for P3.

    I’d love to get some ideas back on what you think would be of interest to participants and would be the best fit with what you have in mind.

    For any of the above, I would probably try to do 10 mins of showing things with twitter coming in and 5 mins of people talking with each other about anything from either the talk or twitter feed (for each topic?) OR I’m open to any other approach as well.

  10. It’s a cool idea to figure out where crowd-sourcing fits in. I’m not sure if formal feedback is “overcome” by an informal crowd-sourced tagging operation but it might be in certain circumstances (where expertise is considered to be reliably spread among a population?).

    If I want to become a better piano player and I give a crowd a video of my performance, will I get helpful feedback? With some constraints on the crowd, it would probably work. If for example, if most of the crowd knew how to teach play and teach piano technique.

    I’ve re-read Paul Thagard’s little book on “Coherence in Thought and Action” recently, in which he outlines a convincing case for a connectionist network representation of a multi-coherence theory of knowledge. I think this might connect to what you are thinking about assessment in the cloud. It’s interesting that while the case for knowledge is very strong, he actually does not recommend a coherence-based approach to truth, since in the abstract, such a system does not necessarily have to be connected to reality. (Knowledge does not have to be connected to reality, but truth does). For truth, we have to add the “correspondence” theory – to tie the representation to an independent underlying reality (even if it cannot be known with any certainty).

  11. […] working on my HASTAC/P3 presentation. I want a back channel where the audience can provide feedback/ assessment of the session. The idea […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: