Crowdsourcing of Weather Data on Mobile App and Deep Learning Lior - - PowerPoint PPT Presentation

Crowdsourcing of Weather Data

• n Mobile App

and Deep Learning

Lior Perez 99th AMS annual meeting

Crowdsourcing on Meteo-France mobile app

■

Context:

―

fewer resources devoted to human observation

■

Crowdsourcing can help:

―

To get a high density of human observations

―

To get information on impacts of weather events

■

Dedicated observation app: NO

―

Too difficult to get a large audience

■

Add a crowdsourcing module in our general public app

―

Benefit from a 1M visitors per day audience

Keep it simple

■

We wanted maximum participation rate In first version:

■

Only immediate observation

■

Only for geolocalized users

■

No quantitative observation

■

Very few details in each observation

Keep it simple!

A challenge for

• ur culture of weather experts...

Feedback and gamification to increase user engagement

Success in quantity and quality

■

10k to 40k observations every day

■

Approx 1000 obs / h disseminated on all the French territory

■

Good quality, very few outliers

■

Large increase of observations rate in severe weather (13 observations of a tornado at 1am)

Outliers filtering

■

Methods investigated:

―

Obvious outliers removal

►

For instance: Hail + Fog + Sun + Strong wind

―

Anomaly detection using the multivariate gaussian distribution, to detect unreliable users

■

Conclusions:

―

Most unreliable users don’t come back

―

Returning users are generally reliable

―

Fake observations < 1%

What are we doing with the data?

Internal visualization interface for forecasters

Subjective product validation

■

Product : distinction of hydrometeors

No precipitation / No snow lying on the ground No precipitation / Snow lying on the ground No precipitation / Ground invisible (night or clouds) Drizzle Rain Drizzle over frozen ground Rain over frozen ground Freezing drizzle Freezing rain Rain and snow mixed Slushy snow Wet snow Dry snow Slushy snow lying on the ground Wet snow lying on the ground Dry snow lying on the ground Ice pellets Small hail Medium hail Large hail

Validation of the distinction between snow and rain

New feature: Observation with picture

Enabling users to post pictures

■

New feature: observation with picture

Issue We need real time moderation

OK Not OK

Image classification: a problem solved by Deep Learning

■

Dogs vs. cats

Dog Cat

Image classification: a problem solved by Deep Learning

■

ImageNet: Database of 14 million hand-annotated images

■

ImageNet Challenge: Image classification models, better than human performance

Transfer Learning

1) Use an image classification model that has been trained on 1.2 million of images from ImageNet

―

Inception v3 2) Re-train it: specialize it on our two classes

―

Class 1: OK, it’s related to weather

―

Class 2: Not OK, it’s not related to weather It’s an easy an quick process!

Prepare a training dataset: images in two folders (OK / Not OK)

OK Not OK

Pictures on the app in production since November

■

No incident, the automatic moderation system has worked

Accepted Rejected

Crowdsourcing of weather data: conclusion

■

Excellent user feedback

―

Already used

►

By forecasters

►

For subjective validation

• f products

―

Good public participation level

■

Perspectives

―

Use of Deep Learning image classification to identify the type

• f weather on pictures

―

Enable advanced users to make more detailed observations

Lior.perez@meteo.fr