Editor’s Note: Dr. Juwah presents powerful documentation on effective implementation of major tenets of constructivism within the Distancce Learning arena. It would be an interesting challenge for our readers involved in distance learning practice to implement and evaluate this approach.

 

Using Peer Assessment to Develop Skills and Capabilities

Charles Juwah

 

Abstract

This paper presents the use of a seven–stage peer assessment process and peer learning in an online context to develop desired skills and capabilities. The contextualized and authentic assessments included case studies, projects, critique and portfolio of evidence. This approach helped ensure effectiveness and sustainability of the assessment method and practice in meeting intended learning outcomes. The ideas discussed in this paper are based on evidence drawn from research and practice in facilitating the development of online tutoring skills.

Introduction

Assessment must perform double duty; not only does it assess content it must also prepare learners for future learning (Boud, 2002 – research seminar presentation).

For assessment to be effective, it should perform the following functions:

·        act as a motivator for learning (Boud et al, 1999; Cowan, 1998);

·        promote deep learning – in which the learner engages with (a) the learning materials and resources, (b) other learners and (c) tutor/facilitator (Marton & Saljo, 1984);

·        contribute to the development of skills (Boud et al, 1999; 2002; Gibbs;1992; Ramsden, 1992);

·        be cost effective and sustainable (Boud, 2002).

In higher education, most of the assessment is based on traditional assessment practices of essay and problem type examination. These traditional assessment practices, it has been found cannot adequately test for critical thinking, creativity, reflection etc. (Lewis and Johnson, 2002, p. 7). However, alternative and diverse assessment methods, for example, peer assessment, portfolio, reflective journaling, etc. have been shown to be constructive, provide authentic and contextualized assessment that promotes deep learning and skills development (Boud et al, 1999; Cowan, 1998, Gipps, 1999; Race, 1998).

Peer assessment is an interactive and dynamic process that involves learners in assessing, critiquing and making value judgment on the quality and standard of work of other learners, and providing feedback to peers to enable them enhance performance. Topping (1998) in his paper on peer assessment between students in colleges and universities provides a detailed typology of peer assessment methods. Evidence from research findings abound supporting the benefits of peer assessment to learners. These include peer learning in a non-threatening environment, the removal of power domination by the teacher over the student, the involvement of the student in the assessment process (Topping, 1998). However, as learners are central in this assessment process, concerns are of raised about their expertise in the knowledge content of the subject matter and their assessment skills to ensure reliability, validity and fairness.

This paper discusses some of the issues of peer assessment and reports on the alignment of teaching and assessment method (peer assessment) in an online learning environment to foster the development of a range of desired skills set and capabilities - critical thinking, ability to analyze and synthesize information, problem solve, assessing and giving feedback, make value judgment and reflection. In addition, it details how this method of assessment can be employed to meet the requirements of reliability, validity and fairness of formal assessment but more importantly reduce assessment load for both the learners and tutor/facilitator.

Need for the Alignment of Assessment

Cowan (1999b, RGU Year of Assessment Workshop) states, “assessment is the powerhouse of learning. It is the engine that drives learning”. However, in many online courses, assessment is based on quizzes, multiple choice questions and/or essays. These assessment methods are very limited in effectively testing and developing higher order skills, for example, critical thinking, creativity and reflection. Therefore, to provide effective and meaningful learning, it is imperative that curriculum/courses are constructively aligned to meet the intended learning outcomes of developing the learners’ knowledge, skills and desired capabilities.

Biggs (1999) defines constructive alignment as

A good teaching system aligns teaching method and assessment to the learning activities stated in the objectives, so that all aspects of this system are in accord in supporting appropriate student learning. This system of constructive alignment is based on the twin principles of constructivism in learning and alignment in teaching. (p 11).

In focusing on assessment in a system of constructive alignment, assessment must be integral in the design of the curriculum for it to contribute towards meeting the intended learning outcomes. Thus, assessment must not be a ‘tag on’ to the curriculum or become a burden or obstacle in the learning process.

Drawing on the twin principles of constructivism in learning and alignment in teaching (Biggs, 1999) and Boud’s sustainable assessment models as frames of reference, an attempt was made to connect teaching and assessment methods and practice in an Online Tutoring (e-Tutoring) Skills course at the Centre for the Enhancement of Learning and Teaching (CELT), The Robert Gordon University, Aberdeen, UK. The Online Tutoring course was designed primarily for developing faculty to enable them to effectively teach and support students’ learning online. The course is fifteen-weeks in duration and can be delivered entirely online via the FirstClassÒ learning environment or via a blended learning approach (a mix of face-to-face and online delivery). The course has as its outcomes:

·        To familiarise themselves with the use of online learning environments in teaching, learning and assessment;

·        To appraise the pedagogy underpinning online teaching and learning;

·        To design fit-for-purpose online activities and/or course(s);

·        To embed ICT in the curriculum and to use innovative technology in teaching and supporting learning;

·        To develop effective facilitator roles of moderating, reviewing, summarising, assessing and giving feedback on individual and group performance, and

·        To reflect on own personal and professional practice and development.

Developing the complex range of knowledge, skills and capabilities that faculty required for facilitating online education needed a well thought through learning and assessment strategies. The constructivist and collaborative nature of online education necessitated an aligned teaching/learning and assessment methods. Based on research evidence and informed knowledge, peer learning and peer assessment methods were adopted.

Rationale for Peer Assessment

In constructivist education, learners construct knowledge and make meaning through social dialogue and interaction with the environment (Vygostky, 1978). This form of collaborative and cooperative learning, Dolittle & Camp (1999) posit, is underpinned by a set of theoretical principles. These are:

·        Learning should take place in authentic and real-world environments;

·        Learning should involve social negotiation and mediation;

·        Content and skills should be made relevant to the learner;

·        Content and skills should be understood within the framework of the learner’s prior knowledge;

·        Students should be assessed formatively, serving to inform future learning experiences;

·        Students should be encouraged to become self-regulatory, self-mediated, and self-aware;

·        Teachers serve primarily as guides and facilitators of learning, not instructors;

·        Teachers should provide for and encourage multiple perspectives and representations of content (paragraph 29).

In line with Dolittle & Camp’s theoretical principles and taking into account the nature of the online tutoring course characterized by 1) asynchronous collaborative learning, and 2) negotiation and construction of meaning through discourse (talk) in the form of postings to the discussion forum, peer learning and peer assessment were the most appropriate learning and assessment methods. In this context, the tutor/facilitator acts more as a guide on the side (Race, 1998; Topping, 1998).

The Peer Assessment Process

The adoption of peer assessment in assess learning in the online tutoring course necessitated the development of a framework to ensure that the assessment method is constructively aligned to teaching and learning methods. The framework developed for the peer assessment involved a seven-stage process (see figure 1).

Figure 1. Peer Assessment: The seven-stage process

1.  Explicit rationale.

The participants were provided with detailed appropriate information in the course handbook on the rationale of the assessment method. In addition, participants were also provided with a Guide on Peer Assessment containing exemplars on how to devise assessment criteria, develop an assessment grid/rubric and a brief on how to give and receive feedback after assessment.

2.  Engage learners in an authentic learning context

In developing the desired skills and capabilities of facilitating, moderating, reviewing, summarizing, assessing and giving feedback on individual and group performance, as well as reflecting one's own personal and professional practice and development, the course was designed and delivered in a way that the learning activities and assessment tasks involved each participant in taking turns to fulfill each of the above listed roles (see Table 1). This ensured that learning was authentic and contextualized, as well as provided the opportunity for the participants to learn by doing.

 

Table 1. Knowledge, Skills and Capabilities Development
in Online Tutoring Course

3.  Involve students in setting assessment criteria

As part of their learning and acquiring the desired tutoring/facilitating repertoires, participants through tasks were involved in devising assessment criteria and developing assessment rubric based on a staged learning process. This staged learning involved learning by examples. Participants were given a step-by-step guide to peer assessment including devising criteria, allocating weightings for grades/marks (Baume, 2001b). Next, the participants were involved in devising relevant assessment criteria to a given task/activity and/or completing partially supplied assessment rubrics (see exemplar below)

Lastly, participants were asked to devise and design from scratch assessment criteria and rubric for given tasks.

4.  Assess learning and give feedback

Participants were required to be involved in formative and summative assessment of learning and giving feedback to peers on their performance and development through set activities.

5. Coach for effective performance.

Participants were coached to promote the development and acquisition of desired skills and capabilities, as well as to ensure best practice. Coaching involves demonstrating/modeling by example, prompting, questioning, supporting and providing re-assurance and encouragement (Murphy, 2001).

6. Reflect on learning

Participants were encouraged to reflect on their learning, performance and practice. These were done via dialogue with peers and tutor/facilitator, responses to online tutorial questions and reflective journaling. Frameworks for reflection and/or trigger questions were provided as guides to help participants reflect effectively.

7. Tutor check to assure quality

Tutor checked and monitored the assessment process to verify that the stated criteria for the course were fully met and to assure quality and standard of learning.

Method

This study was carried out using an action research approach and relied on a variety of sources of data. This included participants’ contributions to online discussions, coursework, assessment tasks, portfolio of evidence (Baume, 2001a; Cowan, 1999b) and course evaluation questionnaire.

Discussion

Learning and assessment in the online tutoring course was based on critique of others’ work, dialogic iterations, analysis, synthesis and construction of new knowledge and making of meaning. This hermeneutic approach to learning underpinned and enhanced participants’ development of content knowledge and pedagogy of online teaching and assessment. The dialogic iterations and reiterations were crucial in enabling the participants to engage in deep learning (Marton & Saljo, 1984) and to acquire the language of discourse of online learning, thus enhancing participants’ confidence to effectively perform in the online environment. The open discourse as one participant stated:

….I believe that moving to online learning and facilitating not only engenders learner empowerment, but also engenders teacher empowerment! I think it engenders teacher empowerment because as a teacher I feel greater ownership of the course as a result of the ‘issue’ identification, reflection, implementation, and evaluation process. At the same time, I think it engenders learner empowerment because you tend to incorporate the same action learning process into the course design for your students’ learning with tasks that ask them to make choices, implement them and then reflect on, and learn from, the perceived outcomes of those choices.
 

Action learning (learning by doing) was critical in developing the various skills and repertoires required for facilitating and supporting learning online. Turn taking in performing the various roles of facilitator, summarizer and peer assessor ensured that participants were not only aware of but also achieved competency through performance and practice in an authentic and contextualized learning (Dolittle & Camp, 1999; Vygostky, 1978).

The training of participants in devising assessment criteria, developing assessment rubric/grade criteria as well as their involvement in assessing peers’ work contributed in developing participants’ ability to assess work, make value judgment, give and receive feedback. Evidence from the participants’ coursework showed the following:

·        that staged training in peer assessment and the interactive nature of the assessment, elicited improved quality and standard of participants’ performance as facilitator, summariser and assessor;

·        the dynamic and balanced assessment process equally ensured that through giving and receiving feedback participants were made aware of aspects of their knowledge and skills needing further development (Cowan, 1998; Marton & Saljo, 1976; Race, 1998).

From this study, which is still ongoing, it was quite apparent that clear guidelines and specific instructions in undertaking peer assessment were essential tools that must be provided for the learners. As we experienced, where instructions were unclear, learners were unsure what was required of them and felt disempowered and frustrated.

Coaching through modeling, prompting, questioning and encouragement provided the relevant scaffolding that enabled the participants to develop and achieve competency. This structured learning through interaction with and guidance of the tutor/facilitator and peers enhances the learners’ development through the zone of proximal development to actual development (Vygotsky, 1978).

The participants’ development through the zone of proximal development was very evident in the improvement of assessing and feedback giving skills from the start through to the latter stages of the course. The quality of feedback was much more focused, constructive and given with some degree of confidence at the latter stages of the course compared to the limited feedback, often un-critical and un-focused given at the start of the course.

Experience from the pilot course necessitated the introduction of a justification box as part of the assessment performance. The justification box was added to help limit or eliminate bias in assessment, for example, to prevent favoritism amongst participants. It requires the participants to justify with reasons their grading of the assessed candidate’s performance. (See Table 2 - Assessment Proforma).

 

Table 2. Assessment Proforma

Name of course participant: __________________________________________________

Topic:_________________________________ Date:________________________________

Peer assessment and peer learning enabled the participants to model cognitive behaviors, thus, contributing to learners gaining mastery of knowledge. In addition, meta-cognitive discussions enhance participants’ awareness of alternative perspectives on issues, as well as promote self-assessment and reflection (Cowan, 1999a; Moon 1999). An entry in the learning journal of one course participant read:

My feelings towards the learning portfolio were generally positive; it pushed me to think about what I had learnt and I valued the opportunity to reflect. It's useful to keep recording what we have learnt so that we can evaluate our learning at the end of the course.

This course has clarified some of the potential pitfalls and made me think about what might work in an online environment and how it might be designed to maximise the opportunities the new technologies create. The design of any online materials has to be carefully considered to match the pedagogical needs of the students to allow them to fulfil their potential. Personalities and the needs of the individual must be taken into consideration as much as they would in a traditional teaching environment.

An emergent outcome from the study was the reduction in tutor/facilitator assessment load. The reduction in time spent by the facilitator in commenting on individual posts as well as assessing individual work meant that this assessment approach was cost effective and sustainable (Boud, 2002). The adoption of this approach to peer assessment for very large groups of course participants has huge economical benefits for training providers.

Conclusion

Evidence from this study showed that peer assessment and peer learning were very effective and efficient in ensuring the development of the desired knowledge, skills and capabilities that faculty required for facilitating and supporting online learning.

In addition, peer assessment was contributory in promoting self-assessment and self-regulation in participants with the consequent effect of the enhancement of quality and standard of practice and performance.

For peer assessment to be effective, rigorous and appropriate training must be provided to enable the participants familiarize themselves with the process of devising assessment criteria, developing an assessment rubric, assessing work and giving and receiving feedback. In addition, participants require encouragement and support from the tutor/facilitator to help build their confidence in engaging with and using the assessment method to its maximum potential.

Assessment methods used in assessing learning must be fit-for-purpose in meeting the learner’s needs, and in addition, the implementation of the assessment should comply with good assessment practice (AAHE, 1996). In keeping with these two tenets, a Quality Spiral (see Figure 2) was used to enhance as well as assure both the quality of learning and the assessment processes.

 

Source: Juwah (2000)

Figure 2. Quality Spiral for Assessment

References

American Association for Higher Education. (1996). Nine principles of good practice for assessing student learning [Online]. Available: http://www.aahe.org/assessment/principl.htm

Baume, D. (2001a). Assessment Series No.6. A Briefing on Assessment of Portfolios http://www.ltsn.ac.uk/genericcentre/projects/assessment/assess_series/06Portfolios.rtf

Baume, D. (2001b). Assessment Series No.7. A Briefing on Key Concepts Formative and summative, criterion & norm-referenced assessment http://www.ltsn.ac.uk/genericcentre/projects/assessment/assess_series/07KeyConcepts.rtf

Biggs J (1999). Teaching for Quality Learning at University. Buckingham: Society for Research into Higher Education, RHE & Open University Press.

Boud, D., Cohen, R. & Sampson, J. (1999). Peer Learning and Assessment. Assessment and Evaluation in Higher Education, 24, (4), 413-426.

Boud, D. (2002). Seminar given on Assessment at the launch of the University of Glasgow’s Centre for Research in Higher Education. 10 February 2002.

Cowan, J (1998). On Becoming an Innovative University Teacher. Buckingham: Society for Research into Higher Education, RHE & Open University Press.

Cowan, J. (1999a). In A handbook of techniques for formative evaluation. George, J. & Cowan, J. London: Kogan Page.

Cowan, J. (1999b). Assessing Reflection. Workshop presented at the Year of Assessment, The Robert Gordon University, Aberdeen.

Doolittle, P. E. & Camp, W. G. (1999). Constructivism: The Career and Technical Education Perspective. Journal of Vocational and Technical Education 16, (1), 23-46. Available: http://scholar.lib.vt.edu/ejournals/JVTE/v16n1/doolittle.html

Elton, L. & Johnston, B. (2002). Assessment in Universities: A critical Review Assessment Research. LTSN Generic Centre, York, UK. Available:
http://www.ltsn.ac.uk/application.asp?app=resources.asp&process=full_record&section=generic&id=13

Gibbs, G. (1992). Improving the Quality of Student Learning. Bristol: Technical Education Services.

Gipps, C. (1999). Socio-cultural perspective on assessment. In A. Iran-Nejad, & P. D.
Pearson (Eds.), Review of Research in Education. Vol. 24, 355-392. Washington: American Educational Research Association.

Juwah, C. I. (2000). The Quality Spiral for Assessment. The Robert Gordon University (RGU) Year of Assessment.

Marton, F. & Saljo, R. (1976) On Qualitative Differences in Learning –1: Outcome and Process. British Journal of Educational Technology, 46, 4-11.

Marton, F. & Saljo, R. (1984) Approaches to Learning. In: The Experience of Learning. Marton,F., Hounsell, D. and Entwistle, N. (Eds). Edinburgh: Scottish Academic Press.

Moon, J. (1999). Reflection in Learning and Professional Development. London: Kogan Page.

Race, P. (1998). Practical Pointers on Peer-Assessment SEDA Paper 102, Birmingham: SEDA Publications.

Ramsden, P. (1992). Learning to Teach in Higher Education. London: Routledge.

Roger Murphy (2001) - Assessment Series No.5. A Briefing on Key Skills in Higher Education http://www.ltsn.ac.uk/genericcentre/projects/assessment/assess_series/05KeySkills.rtf

Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68, 249-276.

Vygotsky, L.S. (1978). Mind in society. Cambridge, MA: Harvard University Press.

About the Author

Charles Juwah PhD, MBA, ILTM is an educational developer and Deputy Head, Centre for the Enhancement of Learning and Teaching, The Robert Gordon University, Aberdeen, Scotland, UK. Dr Juwah is course leader for the new lecturers’ accreditation programme. His research interests include online (e-) learning, assessment issues, postgraduate supervision and personal development planning.

He is a member of the UK Institute for Learning and Teaching in Higher Education. He may be contacted via email: CharlesJuwah@aol.com