Breaking out of the box Understanding rela5onships between - - PowerPoint PPT Presentation

Dr Nick Saville

Breaking out

• f the box

Understanding rela5onships between learning and assessment

Breaking

• ut of the

box

• a metaphor for

thinking about rela5onships and interpreta5ons

breaking out breaking out

• f the box
• f the box

breaking out breaking out

• f the box
• f the box

What is the meaning of this picture for you?

My personal journey

• “Doing” tes5ng as a novice teacher

– Italy

• “Doing” educa5onal reform projects

– Japan, Thailand …..

• Does assessment help or hinder learning?

– Washback models – Posi5ve impact by design – Stakeholders and systems

6

False dichotomies

forma5ve assessment summa5ve assessment

classroom large scale

Tradi5onally …

Forma5ve assessment Classroom assessment Assessment for learning Summa5ve assessment Standardised assessment Assessment of learning

Dichotomies and opposi5ons

A spurious dis5nc5on

“The supposed dis5nc5on between forma5ve and summa5ve is not grounded in the use to which assessment judgements are put. Why? Simply because there is no meaningful dis5nc5on to be

• drawn. The rhetoric appears to dis5nguish between two

conceptually dis5nct types of use to which results can be put; in fact, it simply foregrounds one par5cular type, the forma5ve use.” Newton 2007

“… for all assessments whether predominantly summa5ve or forma5ve in func5on a key aim is to promote produc5ve student learning.”

Carless 2009

classroom assessment large scale assessment

Learning

classroom assessment large scale assessment

Learning

Assessment for learning

• Assessment Reform Group 1999
• A reac5on against standardised tes5ng and tes5ng for

accountability

• Assessment that improves learning, flexibly integrated into

day-to-day classroom prac5ce

• Crucial features:
• 1. The provision of effec5ve feedback
• 2. The ac5ve involvement of pupils in their own learning
• 3. Adjus5ng teaching to take account of the results of assessment
• 4. A recogni5on of the influence of assessment on mo5va5on and self-

esteem

• 5. The need for pupils to be able to assess themselves and to understand

how to improve.

assessment as learning

assessment for learning

assessment of learning

Language Assessment Literacy?

What sort of assessment literacy levels might be needed by

1. Classroom teacher? 2. University administrator? 3. Teacher, tester, administrator?

How might their profiles differ?

Scales of Assessment Literacy 5 levels

• Level 0 - Illiteracy
• Level 1 - Nominal literacy
• Level 2 - Func5onal literacy
• Level 3 - Procedural and conceptual literacy
• Level 4 – Mul5-dimensional literacy

1 2 3 4 Knowledge of theory Technical skills Principles and concepts Language pedagogy Socio-cultural values Local practices Personal beliefs/ attitudes Scores and decisionmaking

8 aspects – 5 levels

Lynda Taylor, 2009, 2013

1 2 3 4 Knowledge of theory Technical skills Principles and concepts Language pedagogy Socio-cultural values Local practices Personal beliefs/ attitudes Scores and decisionmaking

1 2 3 4 Knowledge of theory Technical skills Principles and concepts Language pedagogy Socio-cultural values Local practices Personal beliefs/ attitudes Scores and decisionmaking

hand in hand hand in hand

A learning-oriented approach

“…involves the collec5on and interpreta5on of

evidence about performance so that

judgements can be made about further language development” …to promote learning.

Purpura 2004

Learning Oriented Assessment

LOA is an ecological approach that ensures that all aspects of the system work in harmony to serve the most important goals. In this systemic view, different stakeholders are enabled to work to a common purpose. LOA identifies four worlds to be brought together: · the world of the individual learner · the social world · the world of education · the world of assessment

pick up your free sample chapter

Teachers … In an ideal world

Set clear learning objec5ves

Teachers … In an ideal world

Set clear learning objec5ves Iden5fy suitable tasks & deliver them effec5vely

Teachers … In an ideal world

Set clear learning objec5ves Assess how the students are doing and encourage them to self assess Iden5fy suitable tasks & deliver them effec5vely

Teachers … In an ideal world

Set clear learning objec5ves Assess how the students are doing and encourage them to self assess Provide feedback on performance and encourage self and peer feedback Iden5fy suitable tasks & deliver them effec5vely

Teachers … In an ideal world

Set clear learning objec5ves Assess how the students are doing and encourage them to self assess Provide feedback on performance and encourage self and peer feedback Adjust future learning outcomes and lessons in response to this Iden5fy suitable tasks & deliver them effec5vely

learning

• bjectives

course

Task

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Interpretation

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Interpretation

Record

Informal record

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Interpretation

Record

Informal record Teacher decision-making

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Interpretation

Record

Informal record Teacher decision-making Feedback & modify learning objectives

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Interpretation

Record

Informal record Teacher decision-making Feedback & modify learning objectives

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Interpretation

Record

Informal record Teacher decision-making Feedback & modify learning objectives Structured record Record of achievement

Learning Oriented Assessment

learning

• bjectives

course

Task Language activity Teacher

• bservation

Interpretation

Record

Informal record Teacher decision-making Feedback & modify learning objectives Structured record Record of achievement Interpretation Monitoring of performance External exam

Learning Oriented Assessment

Professional development with a focus

• n assessment literacy is a key aspect of

a learning-oriented model Teachers who are trained in the fundamental principles of LOA can be empowered to:

• Set both higher-order learning outcomes and

specific curricula objec5ves

• acknowledging their complementary aims
• aligning both in terms of goals, constructs and frames of

reference

• Iden5fy the different types of assessment and

their purposes

• Use them to elicit complementary evidence

for and of learning

• classroom interac5on providing feedback on curricula
• bjec5ves
• large-scale assessment which links performance to

higher-order learning goals

• Collect this evidence in a systema5c way and

make appropriate interpreta4ons

• Turn evidence into individualised learning
• by providing feedback and necessary scaffolding to help

learners operate within their zone of proximal development

• Build on the learners’ ability to learn -

producing be9er learning and be9er outcomes

See Jones and Saville, 2016: 120

Summary of LOA concepts

• A “Learning Ladder” to map progression

– Linked to interna5onal standards – Sensi5ve to local contexts

• A model of language use and language learning

– Constructs – Tasks

• Use of digital technology
• Changing role of the teacher

Digital technology and assessment literacy

48

a Fourth Industrial Revolu5on is building on the Third, the digital revolu5on that has been occurring since the middle

• f the last century.

It is characterized by a fusion of technologies that is blurring the lines between the physical, digital, and biological spheres.

Klaus Schwab, Founder and Execu5ve Chairman January 2016

Here to stay ….

Billions of people connected by mobile devices, with unprecedented processing power, storage capacity, and access to knowledge Mul5plied by emerging technology breakthroughs in ar5ficial intelligence, robo5cs, etc .....

The internet of things

Learning experiences are being driven by the small screen

• Extend learning beyond the physical classroom
• Scaffold learners doing tasks in authen5c

contexts

• Enable new forms of learning interac5on
• Remove administra5ve burdens in collec5ng and

processing informa5on

• Capture new forms of evidence for learning

For assessment, digital technology can:

52

• The changing role of the teacher

Teacher Development

• New knowledge, skills and

behaviours

• Higher levels of assessment

literacy

Teacher Development

Teacher Development

Learning Oriented Assessment

• Sehng clear learner objec5ves
• Tracking progress regularly
• Using a frame of reference
• Assessing inside and outside the classroom
• Adjus5ng teaching cycle
• Suppor5ng autonomous learners
• … using

Digital tools

Cloud-based Learning - the Learner at the centre

Face-to-face classroom eTutors Phone-Tablet-PC

(device agnos5c)

LMS-based Practice MyPortfolio

(User generated content & Social Media community)

Blended Course via LMS

VOIP

FaceTime

The Learner

Language activity

Before Class

In Class A[er Class

Ac5vi5es:

• Prac5ce - with

automated feedback

• Comprehension

ques5ons

• Online workbook
• Prac5se vocab with

Apps

• Forma5ve feedback

from peers

Ac5vi5es:

• Reading & Listening

ac5vi5es

• Study text
• Learn vocab online
• Grammar in Use

ac5vity with Apps

• Self-assessment

Ac5vi5es:

• Speaking ac5vi5es
• Pairwork
• Concept ques5ons
• Communica5on

ac5vi5es, games storytelling

• Mentoring
• Forma5ve feedback

from teacher

• Quizzes

To sum up …

Evidence for learning Record keeping Ownership and mo5va5on Flipped classroom Feedback

LOA and Technology

thinking out of the box (1)

Using automated assessment to support learning

Demo: Write & Improve

www.writeandimprove.com

thinking out of the box (2)

The poten5al of Virtual Reality – VR

Demo – Cambridge 360

hnps://www.youtube.com/watch?v=1g9y3heXJO4&list=PLpmCHL8PnXq- yAx7by3q_H72GW4n8fs-m hnps://www.youtube.com/watch?v=IArd6ZadFJ4&list=PLpmCHL8PnXq- yAx7by3q_H72GW4n8fs-m#t=54.159341

Quiz: thinking out of the box

• 1. In what way might VR help support Learning Oriented

Assessment?

• 2. What is one important challenge in making use of this

technology?

• 3. What needs to be added to our Frameworks for teacher

PD and language assessment literacy? Answers to me by lunch5me tomorrow 10 prizes – Google Cardboard

thank you