Fine-Grained Similarity Measurement of Educational Videos and - - PowerPoint PPT Presentation

fine grained similarity measurement of educational videos
SMART_READER_LITE
LIVE PREVIEW

Fine-Grained Similarity Measurement of Educational Videos and - - PowerPoint PPT Presentation

Fine-Grained Similarity Measurement of Educational Videos and Exercises Xin Wang 1 , Wei Huang 1 , Qi Liu 1 *, Yu Yin 1 , Zhenya Huang 1 , Le Wu 2 , Jianhui Ma 1 , Xue Wang 3 1 University of Science and Technology of China 2 Hefei University of


slide-1
SLIDE 1

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Fine-Grained Similarity Measurement of Educational Videos and Exercises

Xin Wang1, Wei Huang1, Qi Liu1*, Yu Yin1, Zhenya Huang1, Le Wu2, Jianhui Ma1, Xue Wang3

1University of Science and Technology of China 2Hefei University of Technology 3Nankai University

slide-2
SLIDE 2

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

01 Introduction

slide-3
SLIDE 3

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Introduction

ØRelated Content Recommendation

Related content recommendation on Khan Academy

slide-4
SLIDE 4

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Introduction

ØPartial Similar

An example from Khan Academy

Q1: Are they similar? Q2: Which segments are similar to this exercise? (fine-grained)

slide-5
SLIDE 5

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Introduction

ØFine-Grained Similarity Measurement

Input Model Output

slide-6
SLIDE 6

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

02 Research Contents

slide-7
SLIDE 7

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 1

Ø How to model the multimodal segment?

  • Captions
  • Keyframes

Ø Spatial and Temporal Information Segment Representation Network

slide-8
SLIDE 8

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 2

Ø Semantic Associations among Video Segments

slide-9
SLIDE 9

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 2

Ø How to model the semantic associations between adjacent video segments? Multiscale Perceptual Fusion

slide-10
SLIDE 10

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Challenge 3

!" similar exercise !#" dissimilar exercise $

margin distance

%

regularization hyperparameter

Ø How to learn the fine-grained similarity by just exploiting the video-level labeled data?

  • The segment-level labeled data is scarce and costly.
  • The video-level labeled data is much easier to obtain.
slide-11
SLIDE 11

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

03 Experiments

slide-12
SLIDE 12

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Dataset

All the data were crawled from the Khan Academy’s math domain (https://www.khanacademy.org/math)

slide-13
SLIDE 13

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Results

slide-14
SLIDE 14

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Ablation Experiments

Ø Visual Information is helpful Ø Textual Information is more important Ø All the key modules are eddective

slide-15
SLIDE 15

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Case Study

slide-16
SLIDE 16

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

Thanks for Listening!

slide-17
SLIDE 17

Anhui Province Key Lab. of Big Data Analysis and Application, University of S&T of China BDAA, USTC

ACM MM 2020 QA Session

Fine-Grained Similarity Measurement of Educational Videos and Exercises Any Questions? Just be free to let me know!