rdwd - an R package to select, download and read climate data from - - PowerPoint PPT Presentation

▶
rdwd an r package to select download and read climate
SMART_READER_LITE
LIVE PREVIEW

rdwd - an R package to select, download and read climate data from - - PowerPoint PPT Presentation

Aug 24, 2022 351 likes •738 views

Motivation Usage Applications Community rdwd - an R package to select, download and read climate data from the German Weather Service Berry Boessenkool, uni-potsdam.de, Feb 2017 berry-b@gmx.de github.com/brry/rdwd 1 / 18 Motivation Usage

slide-1
SLIDE 1

Motivation Usage Applications Community

rdwd - an R package to select, download and read climate data from the German Weather Service

Berry Boessenkool, uni-potsdam.de, Feb 2017 berry-b@gmx.de github.com/brry/rdwd

1 / 18

slide-2
SLIDE 2

Motivation Usage Applications Community

The German Weather Service (DWD) provides over 25’000 climate datasets

2 / 18

slide-3
SLIDE 3

Motivation Usage Applications Community

The German Weather Service (DWD) provides over 25’000 climate datasets

◮ Too much for manual inspection ◮ Somewhat difficult to search ◮ File format inconsistent (e.g. column widths)

2 / 18

slide-4
SLIDE 4

Motivation Usage Applications Community

The German Weather Service (DWD) provides over 25’000 climate datasets

◮ Too much for manual inspection ◮ Somewhat difficult to search ◮ File format inconsistent (e.g. column widths)

Screenshot of FTP server:

2 / 18

slide-5
SLIDE 5

Motivation Usage Applications Community

R saves the day R package rdwd − > easy usage of the datasets

3 / 18

slide-6
SLIDE 6

Motivation Usage Applications Community

Overview

◮ Motivation ◮ Usage ◮ Applications ◮ Community

4 / 18

slide-7
SLIDE 7

Motivation Usage Applications Community

Usage

  • get URL
  • download
  • read
  • plot
  • map

5 / 18

slide-8
SLIDE 8

Motivation Usage Applications Community

U1/5: Get dataset URL with selectDWD

6 / 18

slide-9
SLIDE 9

Motivation Usage Applications Community

U1/5: Get dataset URL with selectDWD library("rdwd")

6 / 18

slide-10
SLIDE 10

Motivation Usage Applications Community

U1/5: Get dataset URL with selectDWD library("rdwd") link <- selectDWD("Potsdam", res="daily", var="kl", per="recent")

6 / 18

slide-11
SLIDE 11

Motivation Usage Applications Community

U1/5: Get dataset URL with selectDWD library("rdwd") link <- selectDWD("Potsdam", res="daily", var="kl", per="recent") ## ftp://ftp-cdc.dwd.de/pub/CDC/observations_germany/ ## /climate/daily/kl/recent/tageswerte_KL_03987_akt.zip

6 / 18

slide-12
SLIDE 12

Motivation Usage Applications Community

U2/5: Download dataset with dataDWD

7 / 18

slide-13
SLIDE 13

Motivation Usage Applications Community

U2/5: Download dataset with dataDWD file <- dataDWD(link, read=FALSE)

## dataDWD -> dirDWD: creating directory ’C:/Users/boessenkool/Dropbox/Public/rdwd/presentation/DWDdata’ ## dataDWD -> fileDWD: creating 1 file: ’daily kl recent tageswerte KL 03987 akt.zip’ 7 / 18

slide-14
SLIDE 14

Motivation Usage Applications Community

U2/5: Download dataset with dataDWD file <- dataDWD(link, read=FALSE)

## dataDWD -> dirDWD: creating directory ’C:/Users/boessenkool/Dropbox/Public/rdwd/presentation/DWDdata’ ## dataDWD -> fileDWD: creating 1 file: ’daily kl recent tageswerte KL 03987 akt.zip’

file ## [1] "daily_kl_recent_tageswerte_KL_03987_akt.zip"

7 / 18

slide-15
SLIDE 15

Motivation Usage Applications Community

U3/5: Unzip file and read + convert data with readDWD

8 / 18

slide-16
SLIDE 16

Motivation Usage Applications Community

U3/5: Unzip file and read + convert data with readDWD clim <- readDWD(file)

8 / 18

slide-17
SLIDE 17

Motivation Usage Applications Community

U3/5: Unzip file and read + convert data with readDWD clim <- readDWD(file)

str(clim) ## 'data.frame': 550 obs. of 18 variables: ## $ STATIONS_ID : int 3987 3987 3987 3987 3987 3987 3987 3987 3987 3987 ... ## $ MESS_DATUM : POSIXct, format: "2015-08-02" "2015-08-03" ... ## $ QUALITAETS_NIVEAU : int 3 3 3 3 3 3 3 3 3 3 ... ## $ LUFTTEMPERATUR : num 22.4 23.8 25.8 20.6 25.2 27.8 24.5 22.5 25.3 25.8 ... ## $ DAMPFDRUCK : num 11.7 13.3 15.7 15.4 15.8 17.4 18.6 15.3 17.7 19.1 ... ## $ BEDECKUNGSGRAD : num 4.5 2 2.9 4.9 3.6 3.4 4.4 2.3 3.3 4 ... ## $ LUFTDRUCK_STATIONSHOEHE : num 1007 1006 1002 1007 1004 ... ## $ REL_FEUCHTE : num 46.8 49.4 52.4 66.2 54.1 ... ## $ WINDGESCHWINDIGKEIT : num 3 3 5 3.4 3.4 4 4.3 3.5 3.8 3.8 ... ## $ LUFTTEMPERATUR_MAXIMUM : num 30 32.3 35.3 26.3 34.6 37.6 33.3 29.5 33.5 33.1 ... ## $ LUFTTEMPERATUR_MINIMUM : num 15.1 14 18.4 16.4 16.1 21.2 19.2 16.6 17.1 19.4 ... ## $ LUFTTEMP_AM_ERDB_MINIMUM: num 11.6 11.7 16.1 14.9 13.5 18.1 17.8 15.5 16.1 18.7 ... ## $ WINDSPITZE_MAXIMUM : num 8.1 9.2 17.3 9.1 9.6 9.1 12.5 8.2 8.4 11.7 ... ## $ NIEDERSCHLAGSHOEHE : num 0 0 4.1 0 0 0 0.1 0 0 0 ... ## $ NIEDERSCHLAGSHOEHE_IND : int 0 0 6 0 0 0 6 0 0 0 ... ## $ SONNENSCHEINDAUER : num 13.4 14.4 11.6 10.7 13.3 ... ## $ SCHNEEHOEHE : int 0 0 0 0 0 0 0 0 0 0 ... ## $ eor : Factor w/ 1 level "eor": 1 1 1 1 1 1 1 1 1 1 ... 8 / 18

slide-18
SLIDE 18

Motivation Usage Applications Community

U4/5: Data can be plotted with regular R code

9 / 18

slide-19
SLIDE 19

Motivation Usage Applications Community

U4/5: Data can be plotted with regular R code plot(clim[,c(2,4)], type="l", xaxt="n", las=1) berryFunctions::monthAxis(ym=TRUE) ; abline(h=0)

−10 10 20 MESS_DATUM LUFTTEMPERATUR

J A S O N D J F M A M J J A S O N D J F

2016 Daily temperature Potsdam Source: Deutscher Wetterdienst

9 / 18

slide-20
SLIDE 20

Motivation Usage Applications Community

U5/5: Interactive map (local) data(mapDWD) ; library(leaflet) ; mapDWD

10 / 18

slide-21
SLIDE 21

Motivation Usage Applications Community

U5/5: Interactive map (CRAN)

11 / 18

slide-22
SLIDE 22

Motivation Usage Applications Community

Applications

  • climate graph
  • event analysis
  • rainfall extremes

12 / 18

slide-23
SLIDE 23

Motivation Usage Applications Community

A1/3: Long term climate graph (Potsdam 1893:2015) clim <- dataDWD(selectDWD("Potsdam", res="monthly", var="kl", per="h")) clim$month <- substr(clim$MESS_DATUM_BEGINN,5,6) temp <- tapply(clim$LUFTTEMPERATUR, clim$month, mean) prec <- tapply(clim$NIEDERSCHLAGSHOEHE, clim$month, mean) berryFunctions::climateGraph(temp, prec, main="")

13 / 18

slide-24
SLIDE 24

Motivation Usage Applications Community

A1/3: Long term climate graph (Potsdam 1893:2015)

m −−−− J F M A M J J A S O N D T °C −−−− −0.5 0.3 3.8 8.4 13.4 16.5 18.3 17.5 13.9 9.0 4.1 0.8 P mm −−−− 45 36 36 40 53 63 70 64 46 42 44 48

Ø 8.8 °C

∑ 587.5 mm

10 20 30 20 40 60 J F M A M J J A S O N D

14 / 18

slide-25
SLIDE 25

Motivation Usage Applications Community

A2/3: Flashflood event rainfall analysis (Taskforce report)

15 / 18

slide-26
SLIDE 26

Motivation Usage Applications Community

A2/3: Flashflood event rainfall analysis (Taskforce report)

DWD−Stationen bei Braunsbach

10 km 1. 82.2 2. 94.0 3. 72.0 4. 76.6 5. 105.0 6. 82.5 8. 45.8 9. 69.6 10. 62.7 11. 78.8 7. 75.3 6 km²

10 20 30 40

Niederschlagsverlauf

Niederschlag [mm/h] 16:00 20:00 00:00 04:00 So, 2016−05−29 Mo, 2016−05−30

  • DWD Station
  • 1. Vellberg−Kleinaltdorf
  • 2. Kirchberg/Jagst−Herboldshausen
  • 3. Kupferzell−Rechbach
  • 4. Ingelfingen−Stachenhausen
  • 5. Langenburg−Atzenrod
  • 6. Mulfingen/Jagst
  • 7. Niederstetten
  • 8. Großerlach−Mannenweiler
  • 9. Waldenburg−Obersteinbach
  • 10. Wüstenrot−Oberheimbach
  • 11. Öhringen

15 / 18

slide-27
SLIDE 27

Motivation Usage Applications Community

A3/3: Extreme rainfall over temperature (github.com/brry/prectemp)

5 10 15 20 Dewpoint temperature (mean of preceding 5 hours) [ °C] Precipitation 99.9% quantile [mm/h] 5 10 20 50 100 Empirical 5 10 15 20 5 10 20 50 100 Parametric

16 / 18

slide-28
SLIDE 28

Motivation Usage Applications Community

A3/3: Extreme rainfall over temperature (github.com/brry/prectemp)

5 10 15 20 Dewpoint temperature (mean of preceding 5 hours) [ °C] Precipitation 99.9% quantile [mm/h] 5 10 20 50 100 Empirical 5 10 15 20 5 10 20 50 100 Parametric

16 / 18

slide-29
SLIDE 29

Motivation Usage Applications Community

The FOSS community role

17 / 18

slide-30
SLIDE 30

Motivation Usage Applications Community

The FOSS community role

◮ Stackoverflow for programming help

17 / 18

slide-31
SLIDE 31

Motivation Usage Applications Community

The FOSS community role

◮ Stackoverflow for programming help ◮ Lobbying DWD into publishing tax-paid data

17 / 18

slide-32
SLIDE 32

Motivation Usage Applications Community

The FOSS community role

◮ Stackoverflow for programming help ◮ Lobbying DWD into publishing tax-paid data ◮ Package distribution infrastructure (CRAN)

17 / 18

slide-33
SLIDE 33

Motivation Usage Applications Community

The FOSS community role

◮ Stackoverflow for programming help ◮ Lobbying DWD into publishing tax-paid data ◮ Package distribution infrastructure (CRAN) ◮ leaflet interactive map really easy to create

17 / 18

slide-34
SLIDE 34

Motivation Usage Applications Community

Conclusion

18 / 18

slide-35
SLIDE 35

Motivation Usage Applications Community

Conclusion

◮ FOSS is awesome

18 / 18

slide-36
SLIDE 36

Motivation Usage Applications Community

Conclusion

◮ FOSS is awesome ◮ DWD is awesome

18 / 18

slide-37
SLIDE 37

Motivation Usage Applications Community

Conclusion

◮ FOSS is awesome ◮ DWD is awesome ◮ Usage of the data is easy with rdwd

18 / 18