ET-805 Modeling Learners Emotions Ramkumar.Rajendran@iitb.ac.in - - PowerPoint PPT Presentation

ET-805 Modeling Learner’s Emotions

Ramkumar.Rajendran@iitb.ac.in

Activity - TPS

An ITS adapts the learning content/feedback based on learners’ performance and preferences. Think individually and write down two most important preferences we should consider for adaptation (2 mins) Share (5 mins)

2

Activity - Class Response

• Motivation - Interest
• Skill on a concept or topic - mental state of the student
• Based on student level of response
• Language - Formal or informal
• Affective State - mood
• 3

Activity - TPS

Think How humans teachers detect emotions in a traditional classroom scenario write one such approach (3 mins) Pair select one approach that the teachers used to predict the students' emotion and discuss how to use that approach in ITS (i.e., automate the approach) (5 mins) Share (3 mins)

4

Activity - Class Response

• Self reporting - Emotion panel, Ask students directly
• Body Posture and gesture
• Camera
• Engagement - asking student a simple reporting

5

Affective States

Basic Emotions: Sad, Fear, Happy, Anger, Surprise, Disgust and Contempt Learner-centered emotion: frustration, boredom, confusion, curiosity, delight, engagement, surprise, and anxiety are more applicable to computer learning environments

6

Rafael A. Calvo, and Sidney D’Mello, Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications, 2010

Voice and Text

• Paralinguistic Features of Speech (Voice)
• Text: Natural Language Processing
• Low cost
• Non-intrusive
• Scalable

But ?

• Very Few ITS developed for Voice and Text input

7

Body Language and Posture

• Non-Intrusive
• Body Pressure Measurement

System

• Machine learning methods by

analyzing data has shown up-to 83% detection accuracy

8

Physiology Sensors

Electromyogram (EMG)

9

Electroencephalography (EEG)

Physiology Sensors

Electrodermal Activity (EDA)

10

Activity - Discussion

List the drawbacks of detection emotions using sensors

• Cost - non-scaleable
• Sensitive to small moves
• Lab studies not for real classroom
• Computationally costly
• Privacy issues

11

Emotion Detection using Facial Expressions

• Manual Observation
• Trained coders can observe student’s emotion like in the

traditional classroom

• Can be done in real-time
• Cost ineffective, not scalable
• Coarse Grain data

12

Activity - Discussion

List the drawbacks of detecting emotion using human observers

• Can’t be used to personalize
• Observers might have bias
• Non scalable

13

Facial Expressions

Automatic emotion detection using facial expressions

• Learner’s facial expression is captured using web camera
• 24 frames of pictures/Second video
• Machine learning and deep learning networks are used to

identify the units (AU) in each frame

• Dataset labeled using human observers are used for training

the machine learning classifiers

14

What is Action Units (AUs)!

• FACS – Facial Action Coding System

15

AU Number FACS Name Neutral face 1 Inner Brow Raiser 2 Outer Brow Raiser 4 Brow Lowerer 5 Upper Lid Raiser 6 Cheek Raiser

Research Studies using Facial Expression

16

Activity - Discussion

List the drawbacks of automatic facial emotion detection software

• Too much of data - finer grained
• Basic emotions
• Posture and gesture in not considered
• Can’t detect student facial expression all the time

17

Modeling Emotions using Log data

• Mining log data and developing classifiers using labelled data
• Labelling done by human observers
• Features in data mining approaches are created by experts
• Example features: response to questions, response to last

3,5,…, n questions, time taken to answer the question, etc

• Machine Learning classifiers are used to detect emotions
• Detection Accuracy > 80%

18

Activity - Discussion

List the drawbacks of data mining approaches to detect emotions

19

Modeling Emotions using Log data

• Learner’s emotions are detected by applying theory
• Features for the classifiers are constructed by applying

theoretical definition of emotions

• Captures both when and why
• Accuracy is less compared to data mining approaches

20

Theory-Driven Approach to Detect Frustration

21

Definition of Frustration

The following factors of frustration are considered to model the student's frustration.

• Frustration is the blocking of a behavior directed towards a

goal

• The distance to the goal is a factor that influences frustration
• Frustration is cumulative in nature
• Time spent to achieve the goal is a factor that influences

frustration

22

Performance of Theory-Driven Approach Using Linear Regression

23

Performance of Theory-Driven Approach Using Different Classifiers

24

Activity - Discussion

List the drawbacks of theory-driven approach to detect emotions

25

Last Activity - Muddy Points

List down

• two important and
• two least clear

(muddy) points from today’s class

• https://tinyurl.com/et8

05mp

26