Machine Learning APIs Comm mmon n appli pplications cations - - PowerPoint PPT Presentation

▶
machine learning apis comm mmon n appli pplications
SMART_READER_LITE
LIVE PREVIEW

Machine Learning APIs Comm mmon n appli pplications cations - - PowerPoint PPT Presentation

Jan 23, 2023 287 likes •626 views

Machine Learning APIs Comm mmon n appli pplications cations Autonomous vehicles Optical character recognition Automatic image tagging Language translation Speech to text Video recommendations Portland State University CS 430P/530

slide-1
SLIDE 1

Machine Learning APIs

slide-2
SLIDE 2

Optical character recognition Language translation Automatic image tagging Autonomous vehicles Video recommendations Speech to text

Comm mmon n appli pplications cations

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-3
SLIDE 3

AI/ML: /ML: an en enormou rmous s sp space ce

 Many different models and approaches

 Expert systems / decision trees / knowledge engineering (medical

diagnosis)

 Supervised learning (Bayesian filters for spam detection)  Unsupervised learning (Clustering algorithms for Google News)  Combinatorial search (Chess)  Reinforcement learning (NPC in games)  Evolutionary/genetic algorithms (Smart fuzzing)

 Often combined with each other  But recently…neural networks with supervised learning

 Convolutional neural networks (CNNs)

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-4
SLIDE 4

Neu eural ral netw etwor

  • rks

ks app pproach

  • ach

 Model the way the brain works via large collections of simulated

neurons

 Formation and pruning of selected connections encodes information

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-5
SLIDE 5

Simu mulat lated ed in comput puter ers

 Selected weights within a simulated network of neurons encodes

information

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-6
SLIDE 6

Everything old…

 Neural Networks circa 1950s-1960s (perceptrons)  Multi-layer neural networks 1970s-1980s

 Ran on original Macintosh!

 Why the renaissance?

 Video cards with massive numbers of processing units (thanks to

gaming)

 Massive storage capacities for data  Crowd-sourced platforms to provide labeling (to learn by example)

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-7
SLIDE 7

Ima mageNe geNet t (2009) 09)

 Pioneered "Deep Learning"

 Use convolutional neural networks with massive sets of labeled data to

solve the computer vision problem

 Approach

 Ignore academic skeptics and take risk on ancient algorithm  Collect image data from the Internet  Hire humans to label it via Mechanical Turk  Send through enormous neural network  Profit?

 Fei Fei Li https://youtu.be/40riCqvRoMs?t=2m46s

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-8
SLIDE 8

Now…“The of x”

 But, not all sunshine and rainbows…

Portland State University CS 430P/530 Internet, Web & Cloud Systems

SpaceNet

DigitalGlobe, CosmiQ Works, NVIDIA

ShapeNet

A.Chang et al, 2015

MusicNet

  • J. Thickstun et al, 2017

EventNet

  • G. Ye et al, 2015

Medical ImageNet

Stanford Radiology, 2017

ActivityNet

  • F. Heilbron et al, 2015
slide-9
SLIDE 9

Corpus rpus can n have e iss ssues ues

 US vs. Russian tanks in 1980s with early NN

 US images crisp marketing shots on sunny days  Russian images grainy and on cloudy days  ML trains on wrong feature (crisp/sunny vs. grainy/cloudy)

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-10
SLIDE 10

 SE Asian workers used to label ideal vacation photos

 Label conference reception photos in hotels as the ideal vacation!

 Perhaps the beach is hard work if they fish for a living?  Labeling is a relative task!

 What happens when one uses ImageNet photos taken by humans on

high-quality cameras for drone applications with poor imaging hardware and no human?

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-11
SLIDE 11

Bi Bias as in labeli eling ng

 Institutionalize bias behind facade of

an "objective" algorithm

 Racial, socio-economic bias in data

being labeled and used

 TED talk  GCP podcast #114 (2/2018)  FAT* conference

 Example: ML for domestic violence

 Most common way is via neighbor

complaint

 ML incorrectly learns that only those

who live in row-homes and apartments commit domestic violence!

 Hard problem

 "It’s not always so obvious ahead of time

what the bad outcomes might be"

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-12
SLIDE 12

 MM's NY Times article (11/2018) and book…

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-13
SLIDE 13

Rare e pr problematic blematic cases ses

 Example: Self-driving cars

 Kangaroos, white trucks against a white sky  Need millions of miles trained to get enough anomalous conditions  Must also train in winter/spring, in snow/rain/clear conditions

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-14
SLIDE 14

Res esistance stance to adver ersaries saries

 Both in training and in

inference

 Small changes fool

classifier

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-15
SLIDE 15

2010

35 29 81 123 157 172

2011 2012 2013 2014 2015 2016

Number of Entries Classification Errors (top-5)

0.28 0.03 0.23 0.66

Average Precision For Object Detection

Requi uires res la large-scale scale particip icipat ation/da ion/data ta

 In thousands of entries

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-16
SLIDE 16

“Datasets—not algorithms—might be the key limiting factor to development of human-level artificial intelligence.”

A L E X A N D E R W I S S N E R - G R O S S Edge.org, 2016

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-17
SLIDE 17

Towar ards ds crowd wd-sour sourced ced ML

Portland State University CS 430P/530 Internet, Web & Cloud Systems

We’re passing the baton to Kaggle: a community of more than 1M data scientists. Why? democratizing data is vital to democratizing AI. image-net.org remains live at Stanford.

slide-18
SLIDE 18

An Ex Expl plosion

  • sion of Da

Datase tasets ts

Portland State University CS 430P/530 Internet, Web & Cloud Systems

1627

Hosted Datasets

276

Commercial Competitions

1MM

Data Scientists

4MM

ML Models Submitted

1919

Student Competitions

slide-19
SLIDE 19

Qu Ques estio tion

 Who on this planet has the largest, most interesting, data-sets?

 Monetize data via ML models

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-20
SLIDE 20

Softw tware's are's new w ap applic lication ation bu buildin lding bl blocks cks

 Don't build your own when you can use pre-trained models done

by experts on massive datasets

 Part of an emerging API development platform  Accessed via REST APIs with results sent back in JSON  Abstraction raised, “Hello world” will never be the same

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-21
SLIDE 21

Cloud ud Visi sion

  • n API

 Image recognition

 Image labeling on thousands of labels  Face detection  Sentiment analysis (Emotobooth)  Text detection (optical character recognition)  SafeSearch content identification (adult/violent content)  Logo identification  Landmark identification

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-22
SLIDE 22

Cloud ud Spe peech ech APIs Is

 Speech-to-Text

 Word recognition  Context-aware transcription  Automated punctuation  Offensive content detection

 Text-to-Speech (speech synthesis)  Both in 120 languages

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-23
SLIDE 23

Cloud ud Translation anslation API

 Language detection and translation

 100+ languages  Python, Java, Ruby, Objective-C bindings via Google API client

libraries

Portland State University CS 430P/530 Internet, Web & Cloud Systems

Cloud ud Natural tural Lang nguage uage API

 Language analysis  Syntax analysis  Semantic analysis, entity recognition  Sentiment analysis  Common e-mail responses

slide-24
SLIDE 24

Cloud ud Video eo Intelligence elligence API

 Summary and information extraction from video

 Autotag objects in video to enable searching  Scene detection for thumbnail generation  Automated highlights (RedZone!)  Trained on massive set of labeled

YouTube videos

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-25
SLIDE 25

Cloud ud Video eo Intelligence elligence API

 Demo code similar to image blurring lab

 Flow chart triggered when new video placed in bucket

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-26
SLIDE 26

ML as s a se service ice

 Use ML to generate custom ML models so regular users can build

their own ML models

 AmazonML

 http://cloudacademy.com/blog/aws-machine-learning/

 AzureML

 http://cloudacademy.com/blog/azure-machine-learning/

 Watson Analytics

 http://www.ibm.com/analytics/watson-analytics/

 Google AutoML

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-27
SLIDE 27

Cloud ud Aut utoML ML for Visi sion

  • n (2018)

8)

 ML services trained on general datasets, but custom

images and domain labels often needed

 AutoML

 Apply transfer learning to re-use trained models to quickly

learn new domains

 Custom models via

 Labeled data (done by user)  Unlabeled data (done by humans at Google)

 Evaluate many different ML models and pick best one (> 13)

 Example: Custom models to recognize machine parts  Democratizes ML for the masses

 All that is required is for you to upload your data with labels

  • r exemplars for Google to label

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-28
SLIDE 28

Aut utoML ML Visi sion

  • n exa

xample ple

 https://www.blog.google/topics/machine-learning/noodle-

machine-learning-can-identify-ramen-shop/

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-29
SLIDE 29

Aut utoML ML Natural tural Lang anguage uage Proce

  • cessing

ssing exa xample ple

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-30
SLIDE 30

Put uttin ting g APIs Is toget gether er (Un Univision ivision)

 Univision live video broadcasts

 “YouTube infrastructure” as a service  YouTube code for video capture, encoding, transcoding, close-

captioning, ad insertion, and re-distribution

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-31
SLIDE 31

Real-time close-captioning (Speech API for transcription, Translate API for multiple languages) Transcode to different resolutions (Compute Engine VMs) Insert ads based on video content and user (Video Intelligence and Vision for object detection, Natural Language for video context, AdSense) Distribution to global users (CDN)

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-32
SLIDE 32

Digitize in full res Upload to Cloud Storage Trigger Cloud Pub/Sub Transcode via GKE & Compute Engine Analyze via Cloud Vision Store metadata in CloudSQL Store images in Cloud Storage

Put uttin ting g APIs Is toget gether er (NY Times mes 11/2 /2018) 8)

Portland State University CS 430P/530 Internet, Web & Cloud Systems

slide-33
SLIDE 33

ML API Labs