IEA | Research | Assessment Insider | Case studies | Marianne Farrugia

Understanding the changing nature of assessment

Author: Marianne Farrugia

Engaging with a professional learning project focussed on educational assessment has enabled a number of schools within the focus group to critically analyse the nature of their assessments and the impacts on student outcomes. The schools involved noticed significant changes in their teacher's assessment practices, which is changing the nature of assessment for their students.

Background

My role is to support the 103 CESA schools with the implementation of the Australian Curriculum. Over the previous number of years, the focus of my work has been to support school to implement the curriculum within the timelines recommended by CESA. Primarily, this has consisted of the facilitation of professional learning and networking opportunities for school and curriculum leaders.

During this time it has become apparent that the change for R-10 schools from an outcomes based curriculum to a standards reference curriculum has been significant. This is particularly the case as schools have been using both the SACSA Framework and the Australian Curriculum whilst awaiting the full suite of subjects. In these circumstances, it has been difficult for schools to develop clear and consistent understandings and practices around standards referenced design, assessment and reporting and thus use the curriculum in the way it was intended.

Whilst we provided a number of opportunities for leaders to develop further understanding of standards referenced design, assessment and reporting, for many schools this has not translated into practice.

In conversation with principals and school leaders the following common concerns were identified around teacher practice and understanding:

  • Little recognition of the importance of achievement standards in the whole of the curriculum design process - developing learning intentions, planning, teaching and learning, assessing and reporting.
  • Assessment practices do not have a direct line of site to the learning intentions / achievement standards
  • Assessment practices do not enable students to demonstrate achievement above the standard
  • Moderation practices do not exist
  • Staffs are too small to moderate effectively
  • Where moderation practices do exist they focus on single assessment piece rather than a portfolio of learning

Back to top


Action plan

In late 2015, I met with some principals with whom I had already been having conversations about their assessment practices and needs of their schools. I presented the draft proposal and was met with a grateful willingness to be a part of the pilot project. Principals then had to nominate and approach partner school/s and this resulted in seven schools undertaking the project in Semester 1 2016.

The pilot project consisted of five professional learning workshops, each approximately one and a half hours in duration and the project culminated with an interschool moderation session. As we were able to provide some funding to the pilot schools, I also facilitated some professional conversations with teams outside of the workshop sessions. The sessions focused on the following in order to explore some basic principles and practices of assessment and moderation:

Session 1

Assessment definition and purpose

Principles for assessment

Assessment and learning

Session 2

Standards referenced assessment

Achievement standards

Session 3

Valid and reliable assessment design

Differentiation in assessment

Session 4

Rubrics

Effective feedback for learning

Session 5

Moderation processes and protocols

Each of the groups of schools undertook the professional learning aspects of the project in a combination of whole or partial staff development days and staff meetings. In (almost) each instance, staff from the schools in the groups came together to participate in the workshops. The workshops themselves were predominately collaborative with much opportunity for discussion, engagement and the development of practical skills. Staff worked in year level teams across the schools. The first session required staff to undertake a survey on their assessment attitudes and practices.

Key successes in the pilot project

  • Teachers collaborating with colleagues from other schools

This was by far the most appreciated aspect of the project. In the first instance I deliberately targeted small primary schools where collaboration within a year level team was impossible within their own school. Many teachers commented as to the benefits of working with colleagues throughout the project. The structure of the project provided a good deal of opportunity for professional conversation and the opportunity to moderate with colleagues who taught the same year level and were working with the same standard was considered most beneficial. 76% of respondents to an evaluative survey identified that the opportunity to work with colleagues has contributed to improvement in their practice.

  • Reflection on practice in terms of how learning opportunities and assessment practices enabled high achievement

Many teachers recognised that their teaching and learning plans were often designed to enable students to hit the standard, but not always designed for students to exceed the standard. Therefore, many of their students were not achieving above a C level not because they were unable but because they hadn't been given opportunities to learn nor demonstrate learning at an A or B level. The re-introduction of CESA's Performance Indicators and Expectation tools was very useful in helping this realisation. Most groups were moderating Mathematics and therefore these tools gave some very clear indications of what learning at an A and B level looks like.

  • Improving the line of sight between the learning design, assessment task and the achievement standard

Though we explored achievement standards early in the project, the key moment of realisation seemed to occur in Session 4 where teachers were asked to examine the work samples on line with careful consideration of the annotations and then to annotate their own work samples. This proved a pivotal moment in terms of teacher recognition that the task needs to enable learning that is at or above the standard and practices like common assessment tasks were actually not necessary in regards to identifying evidence in relation the standard.

  • Change of practice of one school in relation to assessment cycles and requirements

One larger school who partnered two small schools in the region had practices in place that required teachers to A-E grade most learning evidence and upload this on a regular basis to the parent portal. This was a new requirement and had caused much angst amongst staff in relation to the work load implications. As the project progressed, teachers were invited to explore the research around the implication of grades on learning. Soon, both the teachers and the curriculum leaders were questioning their practice. I was able to provide further support through their process of critique, and assist as they embarked on a process of change to their practice. This school is now exploring feedback as a process of learning and use grades only for summative tasks.

Key challenges

  • Confusion as to what counts as evidence

In a number of instances teachers did not feel confident about the collection of evidence in the portfolio. A small number of teachers did not bring a portfolio to moderation because they didn't collect enough evidence. Given teachers need to be collecting evidence at all times in order to make assessment decisions and summative judgements, I wasn't asking them to do anything that could have been considered above normal practice. In one instance a teacher included a very limiting pre-test in their portfolio which I suggested did not represent best or summative work and therefore didn't belong in the portfolio. Despite my best advice this was still presented as a part of the portfolio of evidence.

  • Teachers whose practice differed greatly from those in their partner school

Despite teachers appreciating the opportunity to collaborate, there were some occasions where teachers struggled to do so with colleagues from another school site. The project was developed in order that the groups of schools all participated in the same professional learning and conversations with the idea that we develop a common assessment literacy. I believed this was a pre-requisite for successful moderation.

The main point of difference that occurred was not so much in the assessment literacy however, but in pedagogical practice and curriculum design. The vision of teaching and learning differed between schools and teachers struggled at times with those differences. This was not something that I had foreseen given that school leaders identified and approached their own partner school. However it clearly demonstrated the strong entwine between vision for teaching and learning, curriculum design, pedagogy, assessment and moderation.

Ways forward

  • On the ground leadership

In order for the changes created by the project to be sustainable, they need to continue to be led at school level. Practices need to be continually challenged in order that teachers don't revert to old ways. This is a school responsibility. Following the intensity of the professional learning in the project at a staff level, my role could now be to provide ongoing support to school leaders to sustain the improvements.

Further opportunities for interschool moderation

Most teachers said they would like further opportunities to moderate with colleagues from different schools. A firm commitment from schools to do this is required in order that the project is successful long term.

  • Extension of the project to include further schools into the future

There are now 9 primary schools in our system who are have undertaken the project. If further schools were given the same opportunity, this would provide greater scope for schools to moderate with other and different schools. Moderation is professional learning at its best and this can only enhance learning outcomes for our students.

Back to top


Evaluation

All principals reported improvement to teacher practice as a result of their teachers being a part of the project. All teachers who provided feedback via the feedback mechanism self-reported significant improvement in areas such as line of sight to achievement standards, feedback and validity of assessment tasks.

For a short-term project, this is most heartening. I do believe that a longer term project could only have had better outcomes. I would like the opportunity in future to work with schools for a full year instead of one semester. Overall, the project felt rushed - there was never enough time in the sessions to explore the principles, concepts and research. I found myself continually saying 'we were only scratching the surface' and this was definitely true. I also believe that we needed an opportunity after the moderation session to evaluate the process as well as to evaluate the portfolios. I believe many teachers still have a way to go in compiling a portfolio that truly evidences what a student understands and can do in relation to the achievement standard. There is also ongoing work to be done in improving learning design in order that students have the opportunity for deep learning which can result in more sophisticated understanding and skill and ultimately achievement at an A or B level.

Back to top


References for Project Development

  • I.E.A SA, CEA course work

Back to top


References

  • Black, P. & Wiliam, D. (1998) Inside The Black Box. Department of Education and Professional Studies. Kings College, London.
  • Brown, G. T. L.(2004). Teachers' conceptions of assessment: implications for policy and professional development in Assessment in Education mm Carfax Publishing Vol. 11, No. 3, Novembe.r
  • Hattie, J. (2012). Visible Learning for Teachers. Routledge.
  • Hattie, J and Timperley, H. (2007). The Power of Feedback. Review of Educational Research. Vol 77
  • Klenowski & Wyatt-Smith. (2014). Assessment for Education: Standards, Judgements and Moderation. SAGE.
  • Sadler, D. Royce(2009) Grade integrity and the representation of academic achievement, Studies in Higher Education, 34: 7, 807 - 826.
  • Wiliam, D. The Classroom Experiment. (2010) BBC Television.
  • Websites
  • ACARA Australian Curriculum Version 8.2 http://www.australiancurriculum.edu.au/
  • DECD Leading Learning http://www.acleadersresource.sa.edu.au/index.php?page=strategic_intent
  • Tomlinson, C.A. http://differentiationcentral.com/
  • Wormelli, R. Gradebooks https://www.youtube.com/watch?v=NC7ZI8zr_Mk

Back to top