Web Content Display (Global)
Web Content Display
Social moderation as a means for improvement
Author: Michael Cimarosti
As Michael and the R-3 school community found, moving from a general discussion to a coordinated social moderation approach helps to develop teacher capacity and confidence in assessment practices, leading to improved student outcomes.
This report is based on the setting of a co-educational Catholic R–12 college in South Australia with enrolments of close to 900 students. The school is organised into different communities: Years R–3, 4–6, 7–9, and 10–12. This report will specifically look at the Reception–Year Three community, with some reference to the college as a whole. In the R–3 community, there are over 200 students and 10 teaching staff, most who have been at the school for over six years. There are multiple classes in each year level. The staff plan together and share the common planning document so that it is used by all classes. I have chosen this group as the focus group because I am currently the leader of Learning and Accountability for R-3 and also a member of the teaching team in this area. The staff are committed to working collaboratively and most of them have undertaken the modules through the Institute of Educational Assessors (IEA) Assessment for Educators course as a part of our school-based professional learning. The socio-economic backgrounds of the families in our community vary, as do the learning challenges and strengths of our students. Currently there are no school-based assessment documents that guide assessment practices and so the school uses the policy from Catholic Education South Australia as a guide.
We determined an area of focus for this case study after completing our work with the IEA and looking at current assessment practices. Currently teachers are required to plan collaboratively and engage in creating assessments and tasks that align to the curriculum and standards. Teachers in the R–3 community have previously used the Australian Curriculum as a guide to create tasks and provide opportunities for all students to demonstrate learning and achievement. An area that identified itself as one for improvement was in moderation — or, more specifically, social moderation as a means for improvement. According to Gipps (1994, p 117), social moderation occurs when groups of teachers come together to discuss pieces of work or individual performances. Moderation involves human judgments rather than comparing statistical scores (Maxwell 2002, p 14). Although staff currently engage in the moderation of student work, this often occurs casually in conversation or when report comments are due to be written. The process of moderation needs to be more embedded in the assessment practice of staff to ensure that they make comparable assessment decisions. Gipps (1994, p 117) identifies the importance of supporting this process and making it routine if it is to have any real impact. Social moderation in this setting would involve both the calibration and conferencing models of moderation. This would mean that assessors can use graded work to reach consensus and a common understanding of the standards, as well as select and discuss different levels of achievement or performance (Hipkins & Robertson 2011, p 9).
Comparability in assessment decisions relies on being able to make valid, consistent, and equitable judgments of student achievement. Solving the issue of how to increase the reliability of teacher judgments of student work can occur through moderation (Lenore 2008, p 4). Maxwell (2002, p 1) shows that the process of moderation can serve two purposes: by providing accountability and as a means for improvement.
According to Klenowski and Wyatt-Smith (2010, pp 114–115), moderation involves teachers who share their interpretations of assessment criteria and standards. It also provides the necessary checks and quality assurance to show that evidence-based judgments of student achievement are comparable. ‘It has been shown to build teacher assessment capacity, as well as teacher confidence in judgments’ (Klenowski & Wyatt-Smith 2010, p 115). The view that moderation plays a pivotal role in assessment and student achievement is also supported by Hipkins and Robertson (2011, pp 7–8), who note that moderation allows teachers to come together to share ideas and support each other. It is a professional learning opportunity that is bound to have an impact on student outcomes and improve teaching and learning because of its focus on student work. According to Klenowski and Wyatt-Smith: ‘it is through moderation that teachers develop a shared understanding of the meaning of standards and how to apply them’ (2010, pp 114–115). Both Maxwell (2002, p 11) and Hipkins and Robertson (2011, p 7) identify the opportunity that moderation provides to improve student outcomes and the quality of the educational system. Supported by these views about the importance of moderation and the impact it can have on student development and achievement, the following action plan was devised to assist staff to improve this area of their practice.
The follow four strategies were devised to assist staff in developing their capacity in assessment practices and social moderation. The aim was to improve student achievement and outcomes, while assisting teachers to deepen their own knowledge and use of moderation.
1. As a first step, identify common assessment tasks in planning that are valid, reliable, fit-for-purpose and can be used for moderation
Teachers work collaboratively to develop and write teaching and learning programs to use collectively across year levels. They are required to develop common tasks to help with consistency across classes and to have tasks that can then be used for moderation. Teachers were asked to think about the links to the curriculum and plan for these first. Hayward et al. notes ‘that by foregrounding moderation conversations in the planning process, there were signiﬁcant positive impacts on teachers’ understanding of success criteria and assessment standards and students’ understanding of expectations’ (2014, p 467).
2. Continue to develop understanding of Australian Curriculum achievement standards, content descriptors, and learning continuums
According to Armstrong et al. (2008, p 33), it is important that individuals who are not involved in setting the standards have the opportunity to discuss or unpack terms to help them understand the standard(s). Staff were required to explore the Australian Curriculum when planning learning programs and discuss their plans with other staff so that they understood what was required of the student to meet the standard. They also looked more closely at the content descriptors, learning continuums, and general capabilities to help broaden their understanding.
3. Use common assessments to collaboratively discuss and justify interpretations of achievement against the standards
Listed below, Luke, Weir, and Woods (2008, pp 151–152) identify the benefits for teachers who consistently engage in moderation:
- Assess student performance more consistently, effectively, confidently and fairly;
- Build common knowledge about curriculum expectations and levels of achievement;
- Identify strengths and areas for growth based on evidence of student learning;
- Adjust and acquire new learning by comparing one’s thinking to that of another student or teachers;
- Share effective practices to meet the needs of all students, monitor progress, and celebrate growth.
The aim of this action plan and strategies is for staff to be able to build on these outcomes through the process of moderation. Staff were asked as a team to moderate assessment tasks that would best support their teaching and show a range of levels of achievement. These exemplars help to clarify the meaning of particular levels (Masters 2013, p 44).
4. Use information or data gathered from moderation to inform subsequent planning for student growth and for accountability purposes
Luke, Weir, and Woods (2008, p 150) highlight that ‘moderation is one such system that serves both accountability and improvement purposes’. This view is also supported by Maxwell (2002, p 1) who confirms that any moderation process can be a combination of these two orientations. Throughout the moderation process, teachers need to use the information and data they have collected to inform their practice. This will enable them to refine teaching programs that assist student learning, while collecting evidence to justify decisions.
As a result of the action plan and the strategies employed, there were positive changes to moderation in the R-3 community. Many of these were ones identified as benefits by Luke, Weir, and Woods (2008, pp 151–152) as a result of moderation. The staff were enthusiastic about adopting changes to the planning documents and keen to apply their knowledge in creating assessments tasks that they could use with one another for moderation. By bringing the focus to assessment first and linking them to the content descriptors, we adopted a backwards planning style. Staff felt that this allowed them to better ‘get their head’ around what they were looking for in student achievement.
For an example of the change to the assessments in our planning documents, see below:
Staff noted that they felt comfortable with some areas of the Australian Curriculum (mainly English, Mathematics, and Science) but wanted to explore other subject areas in more detail. Individually, staff felt that they were stronger in some areas and less confident in others, and benefited from the opportunity to learn from each other. Hipkins and Robertson (2011, p 28) identify that this is a positive outcome of moderation in that teachers can identify areas for their own professional growth.
Time played an important factor in what tasks teachers chose to moderate. Generally they chose either a literacy or numeracy task because those were seen to have greater ‘value’. Staff did note that they would like to moderate tasks in other areas if given the time.
Teachers felt in a better position to allocate grades with confidence after they had an opportunity to moderate and discuss interpretations of the standards with each other. Building relationships helped them to feel comfortable in discussing their opinions about achievements and be more open to new interpretations.
Social moderation ideally involves belonging and participating in a community whose members feel safe to support and challenge one another. Their aim as they work together is to reach a consensus about the meaning of student work in relation to the specified standard(s) and this will sometimes require at least some of the community to reshape their current beliefs. (Hipkins & Robertson 2011, p 21)
Staff felt that the opportunity to moderate in their team gave them individually areas in which they could refine their teaching and learning programs. Again, time affected whether they could do this in a more collaborative way.
From these findings, the following additional recommendations are suggested as a means of embedding moderation in our assessment practice.
1. Development of a school-based assessment policy to guide staff and develop assessment practices
The development of a school-based policy in assessment is important. Within this document, information and expectations about moderation should be included. Having clear indicators of what is expected from staff in terms of moderation will help to make it part of the school culture. Having a document to check or reference against provides staff with confidence if they feel unsure about any assessment decisions. It also assists new staff in gaining a clear picture of what the school values in terms of assessment.
2. Formalise moderation in staff practice
Linked to the above recommendation, having a more formalised approach to the expectations of moderation will ultimately improve assessment practice. Gibbs identifies ‘the aim of moderation is … to achieve consistency in assessment in order to enhance quality’ (1994, p 72). A more formal approach to moderation that is listed in the school’s policy will ultimately enhance student achievement.
3. Opportunities for staff to engage in professional learning to deepen understanding of curriculum and standards (particularly in areas in which they feel less capable)
Staff have indicated that they feel more confident working with some areas of the curriculum than others. Moderation gives staff the chance to deepen their knowledge about standards and curriculum. As mentioned previously, Hipkins and Robertson (2011, p 28) identify that this is a positive outcome of moderation as teachers are able to identify areas for their own professional growth.
4. Develop wider opportunities through local area; Catholic Education to link with other educators to continue to develop knowledge and understanding
In our setting, we are fortunate to have a variety of teaching experiences and multiple teachers instructing the same year levels. However, we have the opportunity to develop practice across our region by involving other schools in our area to engage in moderation and help to develop a ‘community of practice’. Lenore (2008, p 8) highlights that this opportunity can occur through the changes in technology:
Online moderation meetings involve a gathering of teachers who are involved in judging student work according to the same set of stated standards. The shared experiences of these teachers are grounded in their work with the standards and their assessment practices using a common assessment task. It is anticipated that over time the teachers will develop common understandings of the quality of work required to meet a stated standard, which may be termed an assessment ‘community of practice’. (Lenore 2008, p 8)
5. Use feedback from moderation to refine task designs/common assessment tasks through our collaboration of teaching and learning plans
From moderation, staff will be provided with data and information that they need to use at the student level. They will also be able to use the data at a personal level to improve their programs and at a school level to identify areas of success or challenges. Using moderation as a way of improving these areas will ultimately increase student achievement and teacher confidence.
6. Explore and plan for the next stage in the continuous improvement of assessment practices
As part of any change, the evaluation of initiatives and continuous improvement play an important factor in delivering the highest quality program. This final recommendation is to carefully map out a plan for improvement of other areas of assessment while refining the moderation process. By devising a well-designed plan, we can implement improvements to our practice with the aim of student development and achievement. ‘All this moderation activity still requires intensive ongoing maintenance and fine tuning’ (Hipkins & Robertson 2011, p 6).
Through this case study and research, it has become clear that assessment is a vital part of quality teaching and learning and is equally important to the actual teaching or learning tasks. Staff responses reveal that moderation plays an important role in our assessment practice and that taking part in moderation benefits both students and teachers. One group of staff is also using moderation in their professional learning with numeracy as the focus to allow for further exploration. Our aim as educators is to impart knowledge and skills to our learners as effectively as possible. By engaging in moderation practices, teachers are able to refine and plan opportunities for effective student learning.
Armstrong, S, Chan, S, Malfroy, J & Thomson, R 2008 , Assessment guide: implementing criteria and standards-based assessment, University of Western Sydney, Sydney
Gipps, CV 1994, Beyond testing: towards a theory of educational assessment, The Falmer Press, London
Hayward, L, Higgins, S, Livingston, K, Wyse, D & Spencer, E 2014, ‘Special issue on assessment for learning’, The Curriculum Journal, vol 25, no 4, pp 465–469
Hipkins, R & Robertson, S 2011, Moderation and teacher learning. What can research tell us about their interrelationships?, New Zealand Council for Educational Research, Wellington
Klenowski, V & Wyatt-Smith, C 2010, ‘Standards, teacher judgement and moderation in contexts of national curriculum and assessment reform’, Assessment Matters: 2014, pp 107–131
Lenore, A 2008, Changing assessment practices: the case for online moderation, Australian Association for Research in Education, Canberra
Luke, A, Weir, K & Woods, A 2008, Development of a set of principles to guide a P-12 syllabus framework, Queensland Studies Authority, Brisbane
Masters, G N 2013, ‘Reforming educational assessment: imperatives, principles and challenges’, Australian Educational Review no.57, Australian Council for Educational Research (ACER)
Maxwell, G S 2002, Moderation of teacher judgements in student assessment, Queensland Curriculum Council, Brisbane