Fine-Grained Similarity Measurement of Educational Videos and - - PowerPoint PPT Presentation

▶

Nov 22, 2022 141 likes •318 views

Fine-Grained Similarity Measurement of Educational Videos and Exercises Xin Wang 1 , Wei Huang 1 , Qi Liu 1 *, Yu Yin 1 , Zhenya Huang 1 , Le Wu 2 , Jianhui Ma 1 , Xue Wang 3 1 University of Science and Technology of China 2 Hefei University of

SLIDE 1

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Fine-Grained Similarity Measurement of Educational Videos and Exercises

Xin Wang1, Wei Huang1, Qi Liu1*, Yu Yin1, Zhenya Huang1, Le Wu2, Jianhui Ma1, Xue Wang3

1University of Science and Technology of China 2Hefei University of Technology 3Nankai University

SLIDE 2

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

01 Introduction

SLIDE 3

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Introduction

ØRelated Content Recommendation

Related content recommendation on Khan Academy

SLIDE 4

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Introduction

ØPartial Similar

An example from Khan Academy

Q1: Are they similar? Q2: Which segments are similar to this exercise? (fine-grained)

SLIDE 5

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Introduction

ØFine-Grained Similarity Measurement

Input Model Output

SLIDE 6

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

02 Research Contents

SLIDE 7

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 1

Ø How to model the multimodal segment?

Captions
Keyframes

Ø Spatial and Temporal Information Segment Representation Network

SLIDE 8

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 2

Ø Semantic Associations among Video Segments

SLIDE 9

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 2

Ø How to model the semantic associations between adjacent video segments? Multiscale Perceptual Fusion

SLIDE 10

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 3

!" similar exercise !#" dissimilar exercise $

margin distance

regularization hyperparameter

Ø How to learn the fine-grained similarity by just exploiting the video-level labeled data?

The segment-level labeled data is scarce and costly.
The video-level labeled data is much easier to obtain.

SLIDE 11

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

03 Experiments

SLIDE 12

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Dataset

All the data were crawled from the Khan Academy’s math domain (https://www.khanacademy.org/math)

SLIDE 13

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Results

SLIDE 14

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Ablation Experiments

Ø Visual Information is helpful Ø Textual Information is more important Ø All the key modules are eddective

SLIDE 15

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Case Study

SLIDE 16

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Thanks for Listening!

SLIDE 17

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

ACM MM 2020 QA Session

Fine-Grained Similarity Measurement of Educational Videos and Exercises Any Questions? Just be free to let me know!