Navigation Menu

Web Content Display (Global)

Stage 2 Moderation

Web Content Display (Global)

Introduction

Purposes of moderation

At Stage 2, moderation seeks to confirm schools’ assessment decisions about student achievement. The purposes of Stage 2 final moderation are to ensure:

  • that the performance standards in a subject have been applied consistently and accurately to school assessment
  • that school-assessed results awarded to students across schools are comparable and fair
  • valid school assessment results.

Operating principles

The following operating principles underpin the moderation procedures for SACE subjects at Stage 2:

Achievement order

Moderation decisions maintain the order of student achievement in assessment types for each assessment group (i.e., the range A+ to E–).

Consistency

Moderation decisions that confirm or recommend adjustments to school assessment results apply to all students in an assessment group with the same result. Moderation decisions seek to ensure that the interpretation and application of the performance standards in a subject are consistent across all schools.

Fairness

A student is neither advantaged nor disadvantaged by having their work included in the sample submitted for moderation.

Interdependence and responsibility

The procedures for assuring the integrity of SACE assessments are based on the interconnected and complementary responsibilities of the student, the teacher, school leaders, and the SACE Board in the assessment process.

Sampling

Moderation is based on evidence provided in a sample of student work from an assessment group. The evidence is representative of the range of grade levels assigned to students in the assessment group.

Transparency

The procedures for assuring the integrity of SACE assessments are explicit and open to scrutiny.

Validity

Moderation seeks to confirm the school’s assessment decisions about student achievement, based on the assumption that the learning requirements, assessment design criteria, and the specifications of the subject outline have been followed, and that the performance standards have been interpreted consistently.

Web Content Display (Global)

Definitions

The following terms are used in this document:

  • assessment group — the group to which students belong for assessment in a subject, as determined by the school. The assessment group can include
    • all the students in the school undertaking the subject and taught by a particular teacher
    • all the students in the school undertaking the subject taught by multiple teachers
    • the students in more than one school taught by one or more teachers
  • assessment types — subject outlines specify different types of assessments that students are to undertake. An assessment type comprises one or more tasks
  • benchmarking — the process used to maximise accuracy among moderators before moderation begins by ensuring that moderators’ assessment decisions on student folios materials are consistent with statewide performance standards
  • grade band — the five bands of achievement described in the performance standards:
    • A, B, C, D, E.
    • Each grade band contains three grade levels: +, mid-grade, and –
  • grade level — the levels of achievement that schools report to the SACE Board, and the SACE Board reports to students: A+, A, A–, B+, B, B–, C+, C, C–, D+, D, D–, E+, E, E–
  • inspection points — the initial grade levels reviewed by moderators Is this relevant online?
  • moderation sample — student folios materials selected by schools to represent their decisions about the assessment type. Teachers also provide a copy of an approved learning and assessment plan, a complete set of task sheets, and a Variations — moderation materials form (if applicable)
  • moderation task — the assessment group in online moderation. See assessment group.
  • school assessment (component) — assessments designed, administered, and marked by the school in accordance with the subject outline and SACE Board policies and procedures
  • school assessment grade — the combined result for all assessment types in the school assessment component of a subject
  • student folio — an entire set of marked student work for an assessment type as specified in the approved learning and assessment plan
  • supervisor — subject expert selected from the Lead Practitioner panel and appointed by the SACE Board who independently checks that the recommendations provided by moderators are consistent with statewide performance standards.
     

Web Content Display (Global)

Assumptions

When moderating school assessment results, moderators assume that:

  • students have been taught and assessed according to the specifications of the relevant subject outline
  • assessment has been conducted according to SACE Board policies and procedures
  • the performance standards for the subject have been applied consistently and accurately across an assessment group
  • teachers have determined grade levels (–, mid-grade, +) by considering whether or not evidence of learning demonstrates specific features that are predominantly from:
    • one particular grade (mid-grade)
    • one grade but some of which are from a higher grade band(s) (+)
    • one grade but some of which are from a lower grade band(s) (–)
  • the achievement order determined by the teacher is accurate
  • school assessment results sheets submitted by the school to the SACE Board are correct.
Moderation involves Moderation does not involve
  • looking for evidence in a sample to support the results provided by the teacher
  • considering students’ evidence of learning and the match between this evidence and the performance standards in the subject outline
  • confirming or adjusting a result when
  • supported by evidence of learning
  • providing schools with feedback about the outcomes of moderation.
  • marking or initiating a result
  • checking that the teacher has marked errors in the student work correctly
  • looking for disagreement with the teacher
  • giving advice about improvements or commenting on the work of individual students.

Web Content Display (Global)

Roles and responsibilities

The SACE Board:

  • appoints a SACE officer to lead and support the moderation process 
  • appoints an assessment panel for final moderation of each subject, consisting of
    • Lead Practitioner(s)
    • subject supervisors (the number depends on the number of moderators appointed)
    • moderators (the number depends on the size of the student cohort)
  • lists all schools, by school number, and allocates schools to moderators. Moderators are not allocated to
    • their own school
    • a school with which they have a conflict of interest
  • trains moderators and supervisors in the moderation procedures via PLATO courses and meetings, where applicable
  • requires all assessment panel members to
    • maintain confidentiality throughout and after the moderation process in accordance with the SACE Board’s Code of Conduct 
    • declare any potential conflicts of interest
    • ensure the security of school and student results, information, and materials.
  • advises moderators about work health and safety
  • advises moderators about the Independent Commissioner Against Corruption (ICAC).

Supervisors:

  • Submit completed Code of Conduct and Conflict of Interest forms
  • support the SACE Officer in selecting and preparing calibration materials, where applicable
  • together with the SACE Officer and Lead Practitioner(s), oversee the outcomes of the moderation process by:
    • independently reviewing adjustments recommended by moderators, to confirm or adjust results submitted by the school
    • identifying evidence in the sample that reasonably explains the basis of the adjustment at the inspection points and A+ grade levels (if awarded by the school)

Moderators:

  • Submit completed Code of Conduct and Conflict of Interest forms
  • participate in calibration activities prior to moderation in PLATO courses and meetings
  • moderate according to the moderation procedures and the instructions of the SACE Officer
  • identify potential breaches of rules

Web Content Display (Global)

Confidentiality and security

Conflict of interest

Lead Practitioners, supervisors, and moderators are not to:

  • moderate the work of students from any school at which they teach or with which they have a conflict of interest
  • contribute to discussion about the moderation of any school at which they teach or with which they have a conflict of interest
  • access information about the outcome of the final moderation process for any school at which they teach or with which they have a conflict of interest.

Confidentiality

Lead Practitioners, supervisors, and moderators maintain the confidentiality of information about students’ results
and schools before, during, and after the moderation process.

Security

Moderators ensure the security of school and student results, information, and materials by, for example:

  • ensuring their passwords and access to Schools Online are secure
  • deleting any files related to the moderation process
  • not undertaking moderation in the presence of others, or in any other way breaches the confidentiality of the student work.

Web Content Display (Global)

Moderation procedures

The moderation process is based on a sample of students’ work from each assessment group. The sample is selected by the school according to the moderation sample selection parameters.

Moderation is conducted on the basis of the grade levels reported by the school for each assessment type.

The SACE Board gives feedback to school about the outcomes of moderation for all assessment groups.

Moderation is conducted in Schools Online according to the following procedures. These procedures are also reflected in the Stage 2 final moderation workflows for moderators and supervisors.

Step-by-step process

1.    Moderators use Schools Online to work individually to complete all moderation tasks within each assessment group. Once completed these should be submitted.
2.    Each assessment group should include:

  • for each assessment type all work for the selected students on the school assessment results sheet
  • a complete set of task sheets (not applicable to the Research Project)
  • the approved learning and assessment plan (LAP) and addendum, if applicable (not applicable to the Research Project)
  • a Variations — Moderation Materials (VMM) form, if applicable.

If any student material is missing (not mentioned on the VMM), the moderator informs the SACE Officer — Curriculum
and Assessment, by using the issues button on the moderation dashboard, who will determine how to proceed.

3.    For each assessment type, the moderator initially reviews student materials at the inspection points as follows:

  • the lowest available grade level in the A grade band (e.g. A- if available)
  • the lowest available grade level in the B grade band
  • the lowest available grade level in the C grade band
  • the highest available grade level in the D grade band
  • the lowest available grade level in the E grade band
  • half of the A+.

4.    The moderator reviews the selected student materials for each assessment type according to the review procedure below.

Web Content Display (Global)

Moderator review guidelines

The Moderator Review Guidelines will help moderators in identifying some key features of the moderation sample in reviewing assigned grade levels. Moderators work individually in the online moderation space.

The following guidelines will help moderators to identify key evidence within the moderation sample and to review assigned grade levels:

  • Begin with the Performance Standard Record (PSR) Note the school result and profile of achievement recorded by the school
  • Note if variations in Moderation Materials (VMM) is provided before viewing student materials
  • Scan the assessment type as a whole, and then focus on evidence in specific tasks. It is not necessary for moderators to review every task or every detail in a task. A more detailed review is only necessary when doubt is raised about the grade level(s) assigned.
  • Actively look for evidence that is consistent with the assessment design criteria and grade level(s) assigned by the school.
  • Recognise unexpected evidence of the performance standards (e.g., any evidence of the assessment design criteria can be used to inform the moderator’s decision to confirm the school results).
  • Ignore distractors and evidence that is not relevant to the performance standards (e.g., effort, untidy work, teacher comments).
  • Avoid focusing on what is not there (e.g., one incomplete section of a task does not mean that the student has failed to demonstrate evidence of the assessment design criteria elsewhere in the assessment type).
  • When it is difficult to find evidence that is consistent with the grade level(s) assigned by the school, identify the assessment design criteria (or specific features for Stage 2 Research Project) that are inconsistent with the assigned grade level(s). Consider whether there is obvious and substantial evidence of the identified assessment design criteria (or specific features for Stage 2 Research Project) at a different grade level:
    • obvious evidence is important, significant, and easily recognisable. Obvious evidence is not nuanced or subtle
    • substantial evidence is defined as evidence that is of considerable size, frequency, or worth.
  • Moderator recommendations must be aligned to the performance standards and calibration (standards clarification).

Moderation review procedure

A to E inspection points

  • When evidence in the student materials supports the results awarded by the school at the inspection points, the moderator confirms all results for the assessment type.
  • When a moderator is unable to find evidence in the student materials that supports one or more of the results awarded by the school at the inspection points, the moderator recommends an adjustment.
  • Once an adjustment has been recommended, the moderator continues to review student samples at the grade levels closest to the grade level to be adjusted, to determine whether or not a recommended adjustment should also extend to these grade levels.
  • If the adjustment does extend to the closest grade level(s), the moderator then reviews another student folio at the next-closest grade level.

Closest grade levels are defined as the grade levels closest to (above and below) an Inspection point. Closest grade
levels may or may not be immediately adjacent to an inspection point

  1. When evidence in half of the total number of A+ student materials support the A+ result, the moderator confirms all A+ results. If the moderator is unable to find evidence in more than half of the A+ student materials, the moderator recommends an adjustment to the A+ results.
  2. The moderator repeats the review procedure for the remaining assessment type(s).
  3. The moderator completes the materials submission section as required.
  4. On completion of all steps in the process moderators should click the submit button.

Web Content Display (Global)

Supervisor review guidelines

The following guidelines are provided to support supervisors to review moderators’ recommendations to adjust
results:

  • Supervisors independently review recommendations provided by moderators. Supervisors do not seek further clarification from moderators.
  • Supervisors begin with the Performance Standards Record
  • Supervisors review the:
    • result submitted by the school
    • recommended adjustment by the moderator
    • assessment design criteria (or specific features for Stage 2 Research Project) indicated as the reason for the recommended adjustment by the moderator.
  • Supervisors review the student evidence and focus on the assessment design criteria (or specific features for Stage 2 Research Project) that the moderator has noted on the Performance Standards Record (PSR) and seek evidence to confirm the school result.
  • Supervisors determine if the student evidence (or specific features for Stage 2 Research Project) highlighted by the moderator on the PSR is obvious and substantial:
    • obvious evidence is important, significant, and easily recognisable. Obvious evidence is not nuanced or subtle
    • substantial evidence is of considerable importance, size, or worth at the assessment type level.
  • When making an adjustment, supervisors write comments about the adjustment, citing obvious and substantial evidence that is referenced to the performance standards, and comparable to calibration activities (standards clarification).
  • Supervisors determine adjustments independently, without influence from, or discussion with, the moderator.
  • Supervisors do not advise moderators about their decisions.

Supervisor review procedure

  1. The supervisor independently reviews the student folio(s) at the initial inspection points recommended for an adjustment by the moderator, focusing on the criteria (or specific features for Stage 2 Research Project) that the moderator has highlighted on the PSR. The supervisor seeks evidence to confirm the school result.
  2. When evidence in the student materials supports the results awarded by the school, the adjustment(s) recommended by the moderator is overturned. The supervisor confirms the result(s) recommended by the school, and any adjustments recommended for additional grade levels are disregarded (where appropriate).
  3. When a supervisor is unable to find evidence in the student materials that supports one or more of the results awarded by the school, the supervisor makes an adjustment. Any adjustments made by the supervisor must be based on obvious and substantial evidence in the student materials, the performance standards, and the specifications of the relevant subject outline. Adjustments reflect the moderator’s recommendation or an adjustment that the supervisor determines more appropriately reflects the performance standards and calibration activity (standards clarification).
  4. The supervisor writes comments referencing obvious and substantial evidence in the student materials that explain the adjustment(s), citing performance standards and, if required, comparable calibration activities (standards clarification)
    a.    Obvious evidence is important, significant, and easily recognisable. Obvious evidence is not nuanced or subtle.
    b.    Substantial evidence is of considerable importance, size, or worth at the assessment type level.
  5. The supervisor finalises the adjustment(s) on the school assessment — moderation results sheet and clicks the submit button.

Web Content Display (Global)

Guidelines for confirming and adjusting results

Confirming results

A result is confirmed when the student materials in the moderation sample demonstrates evidence at the grade level reported by the school.

Adjusting results

Adjustments:

  • from one grade band to another grade band (e.g. from a C to a D) are made when evidence in the student materials consistently demonstrates the assessment design criteria and the associated specific features of a different grade band
  • from the mid-grade level in a grade band to plus (+) grade level in that grade band (e.g., from a B to a B+) are made when, taken as a whole, the evidence has elements of the assessment design criteria of a higher grade band(s) (or individual specific features applicable to assessment types that assess single assessment design criteria)
  • from the mid-grade level in a grade band to minus (–) grade level in that grade band (e.g., from a B to a B–) are made when, taken as a whole, the evidence has elements of the assessment design criteria of a lower grade band(s) (or individual specific features applicable to assessment types that assess single assessment design criteria)
  • from plus (+) or minus (–) grade levels to mid-grade levels (e.g., from a C– to a C) are made when, taken as a whole, the evidence of the assessment design criteria and associated specific features most appropriately aligns with a particular grade band
  • from the A+ grade level to A are made when, taken as a whole, evidence in the A+ student folio(s) does not demonstrate sustained achievement at the upper level of the A grade band (i.e. the assessment design criteria and associated specific features are at the mid-grade of the A grade band)
  • from the A grade level to A+ are made when, taken as a whole, evidence in the A student folio(s) demonstrates sustained achievement at the upper level of the A grade band; note that adjustments from A to A+
    • may be made to individual student results when a breach of rules has been identified and confirmed by the school
    • are not usually made on the basis of a single inconsistently assigned grade.

Web Content Display (Global)

Quality assurance

After moderation

Checking moderation outcomes and feedback

  • All final moderation feedback is quality assured.
  • The SACE Board provides feedback in Schools Online, to confirm the outcomes of moderation.
  • Schools note the feedback resulting from final moderation for each subject and incorporate this in their improving and planning processes.
  • Subject Assessment Advice: The SACE Officer involved in the moderation process contribute information and data from the school assessment and external assessment processes for use in the Subject Assessment Advice.

Web Content Display (Global)

Web Content Display (Global)