Alexa, My College Buddy Interim Report Dan Yu, Yanqi Liu, John - PowerPoint PPT Presentation

Alexa, My College Buddy Interim Report Dan Yu, Yanqi Liu, John Kusner, Lauren Bright Advised by Professor Chuah

Today’s Presentation Project Developed Overview Challenges Features Future Q&A Demos Goals

Overview

Project Overview ● Improve the lives of college students ● Develop Alexa/Google Home skills to help them... ○ quit smoking ○ cope with academic anxiety ● Create an interactive counselor that is available 24/7

Project Overview (cont) ● Develop questionaires to gather info from students struggling with (i) anxieties over academic performance, (ii) smoking, and provide tips to help them ● Questionaires accomplied via structured conversations. ● Develop Group Chat feature to provide peer support ● In future, ● Research and develop unstructured conversation ● Possibly add in virtual reality app.

Project Challenges

Ethical Issues Issues ● Confidentiality ● Medical advice issues ● IRB Adjustments ● Focus on academic realm ● Keep as much/little personal information as the user is comfortable with

Alexa Skill Challenges ● Alexa skills are highly restricted ○ No way to capture full utterance or audio ○ Fallback intents don’t include input ○ Skills exit after no response from user ○ Develop skills using Jovo so that can be easily ported to Google Home.

Group Chat Challenges ● Real time communication is complicated ● Peer-to-peer connections costly with large groups ● External signaling server is required ● Choosing an implementation that minimizes usage of time and resources

Progress

Major Accomplishments ● Developed an extensible Alexa questionnaire framework ● Created 6 Alexa skills ● Built group voice chat app with React Native

Extensible Questionnaire Framework ● Abuses loophole in Alexa Dialog fulfillment ● Easy conversation structuring with branching based on response ● Added NLU support with Wit.ai API connection John

Quit Smoking - Alexa Skills ● Smoking Health Updates ● Smoking Questionnaire ● Based skills on provided smoking call info protocol ● Responses stored in a database Dan and John

Mental Health Progress ● Researched on mental health issues among college students and professionals ○ Prof. DuPaul ○ Dr. Ian Birky ○ Counseling sessions ○ Active Minds club ○ Reddit Yanqi and Lauren

Mental Health - Alexa Skills ● Flash Briefings ○ Mental Health Updates ● Questionnaires ○ Mental Health Questionnaire ○ Academic Dialogue Yanqi and Lauren

Mental Health Progress ● Integrated Wit.ai to achieve entity extraction/NLU ○ Keyword based entity matching ● Finalized structured conversation ○ Time management ○ Test anxiety ○ Understanding course materials Yanqi and Lauren

Group Chat App Progress ● Decided to use fully connected P2P chat ● Set up AWS cloud machine for signaling server ● Forked open source P2P WebRTC implementation Dan and John

Group Chat App Progress ● Created demo app for web, in Cordova, and React Native ● Added username support ● Partially implemented Firebase Cloud Messaging Dan and John

Future Goals

Next Semester We Will ● Perform real world trials of our Alexa skills ● Make changes based on user feedback ● Make our skills work on Google Homes

Next Semester We Will ● Final product projection ○ Intelligent counselors to help students ○ Counselor handles what sort of issues students are dealing with and responds accordingly ○ Add text based group chat to Android app ○ Explore virtual reality options

Demos

Alexa Skills

Group Chat App

Alexa Skills

Summary ● Project Goal: ○ A tool set consisting of Alexa skills and other apps to improve the lives of college students ● Progress so far: ○ 6 Alexa skills, group chat app ● Next semester: ○ Trial run and get feedback ○ Implement unstructured conversation ○ Explore Virtual reality