Affect- and Personality-based Recommender Systems Hands-on: - - PowerPoint PPT Presentation

▶

Sep 12, 2023 290 likes •495 views

Affect- and Personality-based Recommender Systems Hands-on: Unobtrusive Acquisition of Emotions Marko Tkali, Free University of Bozen-Bolzano ACM Summer School on Recommender Systems 2017 Marko Tkali, RecSys2017SummerSchool-HandsOn 1/18

SLIDE 1

Affect- and Personality-based Recommender Systems

Hands-on: Unobtrusive Acquisition of Emotions

Marko Tkalčič, Free University of Bozen-Bolzano

ACM Summer School on Recommender Systems 2017 Marko Tkalčič, RecSys2017SummerSchool-HandsOn 1/18

SLIDE 2

Goal

Problem statement: lack of recsys datasets with emotional feedback
Solution: To set-up a web app for acquiring emotions during RecSys sessions,

hence enabling large-scale acquisition of recsys interactions.

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 2/18

SLIDE 3

Goal

Problem statement: lack of recsys datasets with emotional feedback
Solution: To set-up a web app for acquiring emotions during RecSys sessions,

hence enabling large-scale acquisition of recsys interactions.

How? Using a public API for emotion detection.

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 2/18

SLIDE 4

Pre-requisites

In order to be able to implement the solution you should have familiarity with

programming. The tutorial will be on web client-side (HTML/DOM/Javascript),

although you may use another solution (Android, iOS/macOS, Linux, Unity, WIndows). Before the session starts please make sure that you:

have a laptop with an integrated camera
installed an HTTP server on your laptop
apache
MAMP/XAMPP or some other integrated solution
know where your htdocs folder is

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 3/18

SLIDE 5

Scenario

Affectiva SDK : https://developer.affectiva.com/
we will build an HTML/Javascript page
the full code is available at

http://www.inf.unibz.it/~tkalcic/recsys_summerschool_slides/demo.html.txt

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 4/18

SLIDE 6

Step 1

run the HTTP server
create a subdirectory in the htdocs directory (e.g. .../htdocs/RecSys_demo)

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 5/18

SLIDE 7

Step 2

in a text editor create the blank HTML page (e.g.

.../htdocs/RecSys_demo/demo.html)

<!DOCTYPE html> <html> <head> <title>ACM RecSys Summer School 2017 Demo</title> </head> <body> </body> </html> Marko Tkalčič, RecSys2017SummerSchool-HandsOn 6/18

SLIDE 8

Step 3

in your web browser open the demo.html page

http://localhost/RecSys_demo/demo.html Marko Tkalčič, RecSys2017SummerSchool-HandsOn 7/18

SLIDE 9

Step 4

create three buttons
create the three visualization boxes

<!DOCTYPE html> <html> <head> </head> <body> <button id="start" onclick="onStart()">Start</button> <button id="stop" onclick="onStop()">Stop</button> <button id="reset" onclick="onReset()">Reset</button> <hr> <div id="content" style="border: 1px solid; width: 400px; height: 620px; float: left; margin: 10px; padding: 0;"> CONTENT </div> <div id="preview" style="border: 1px solid; width: 400px; height: 300px; float: left; margin: 10px; padding: 0;"> </div> <div id="results" style="border: 1px solid; width: 400px; height: 700px; float: left; margin: 10px; padding: 0;"> </div> </body> </html> Marko Tkalčič, RecSys2017SummerSchool-HandsOn 8/18

SLIDE 10

Step 5

Check your HTML code in the web browser
after each modification to the HTML file go back to the browser, refresh the page and
bserve the effects of the change

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 9/18

SLIDE 11

Step 6

Add the needed SDKs to the head of the HTML

<head> <title>ACM RecSys Summer School 2017 Demo</title> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <script type="text/javascript" src="//code.jquery.com/jquery-3.1.0.js"/></script> <script type="text/javascript" src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"/></script> <script type="text/javascript" src="https://download.affectiva.com/js/3.1/affdex.js"/></script> </head>

For reference, see https://knowledge.affectiva.com/docs/getting-started-with-the- emotion-sdk-for-javascript

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 10/18

SLIDE 12

Step 7

start a <script> element in the bottom of the HTML

... <div id="results" style="border: 1px solid; width: 400px; height: 700px; float: left; margin: 10px; padding: 0;"> </div> <script type="text/javascript"></script> ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 11/18

SLIDE 13

Step 8

set up the detector

... // Link a div for previewing the camera feed var divRoot = $("#preview")[0]; // Configuration var width = 400; var height = 300; var faceMode = affdex.FaceDetectorMode.LARGE_FACES; //Construct a CameraDetector var detector = new affdex.CameraDetector(divRoot, width, height, faceMode); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 12/18

SLIDE 14

Step 9

Choose the classifiers
The next step is to turn on the detection of the metrics needed.
list of classifiers: http://developer.affectiva.com/metrics/

... detector.detectAllExpressions(); detector.detectAllEmotions(); detector.detectAllEmojis(); detector.detectAllAppearance(); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 13/18

SLIDE 15

Step 10

add the callback functions for the three buttons

... //function executes when Start button is pushed. function onStart() { if (detector && !detector.isRunning) { detector.start(); } } //function executes when the Stop button is pushed. function onStop() { if (detector && detector.isRunning) { detector.removeEventListener(); detector.stop(); } }; //function executes when the Reset button is pushed. function onReset() { if (detector && detector.isRunning) { detector.reset(); $('#results').html(""); } }; ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 14/18

SLIDE 16

Step 11

Configure the callback functions
The Detectors use callbacks to communicate events and results.
For each action there are two callbacks.
A success callback is called when an action successfully completes, and
a failure callback is called in case of an action failure.
The functions addEventListener and removeEventListener are used to register or

deregister a callback.

... //Add a callback to notify when the detector is initialized and ready for runing. detector.addEventListener("onInitializeSuccess", function() { //The following two objects are oart of the Affectiva SDK //Display canvas instead of video feed because we want to draw the feature points on it $("#face_video_canvas").css("display", "block"); $("#face_video").css("display", "none"); }); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 15/18

SLIDE 17

Step 12

... //Add a callback to receive the results from processing an image. //The faces object contains the list of the faces detected in an image. //Faces object contains probabilities for all the different expressions, emotions and appearance metrics detector.addEventListener("onImageResultsSuccess", function(faces, image, timestamp) { if (faces.length > 0) { var resultsHtml = ""; resultsHtml += "Number of faces found: " + faces.length; resultsHtml += "<br>Appearances: " + JSON.stringify(faces[0].appearance); resultsHtml += "<br>Emotions: " + JSON.stringify(faces[0].emotions); resultsHtml += "<br>Expressions: " + JSON.stringify(faces[0].expressions); resultsHtml += "<br>Emoji: " + faces[0].emojis.dominantEmoji; var htmlToDisplay = resultsHtml.replace(/,/g,",<br>"); // make output nicer $('#results').html(htmlToDisplay); drawFeaturePoints(image, faces[0].featurePoints); } }); ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 16/18

SLIDE 18

Step 13

add the drawFeaturePoints method

... //Draw the detected facial feature points on the image function drawFeaturePoints(img, featurePoints) { var contxt = $('#face_video_canvas')[0].getContext('2d'); var hRatio = contxt.canvas.width / img.width; var vRatio = contxt.canvas.height / img.height; var ratio = Math.min(hRatio, vRatio); contxt.strokeStyle = "#FFFFFF"; for (var id in featurePoints) { contxt.beginPath(); contxt.arc(featurePoints[id].x, featurePoints[id].y, 2, 0, 2 * Math.PI); contxt.stroke(); } } ... Marko Tkalčič, RecSys2017SummerSchool-HandsOn 17/18

SLIDE 19

Check the result in the browser

Marko Tkalčič, RecSys2017SummerSchool-HandsOn 18/18