Pawe Lebioda, Tomasz Kapela Agenda Introduction: Tutorial: - - PowerPoint PPT Presentation

▶
pawe lebioda tomasz kapela agenda
SMART_READER_LITE
LIVE PREVIEW

Pawe Lebioda, Tomasz Kapela Agenda Introduction: Tutorial: - - PowerPoint PPT Presentation

Sep 10, 2022 212 likes •1.14k views

Pawe Lebioda, Tomasz Kapela Agenda Introduction: Tutorial: Persistent Memory Kernel support NVML NVML overview PMEMCHECK libpmem libpmemblk libpmemobj C API libpmemobj C++ API PMEMCHECK

slide-1
SLIDE 1

Paweł Lebioda, Tomasz Kapela

slide-2
SLIDE 2

2

Agenda

Introduction:

  • Persistent Memory
  • NVML
  • PMEMCHECK

Tutorial:

  • Kernel support
  • NVML overview
  • libpmem
  • libpmemblk
  • libpmemobj C API
  • libpmemobj C++ API
  • PMEMCHECK
slide-3
SLIDE 3

Just a little bit of theory...

slide-4
SLIDE 4

What is it ?

slide-5
SLIDE 5

5

Persistent Memory

What is Persistent Memory ?

  • Non-volatile
  • Retains data without power
  • Byte addressable
  • Load/Store access
  • Memory-like performance
  • DMA-able
  • NVDIMM
slide-6
SLIDE 6

6

Persistent Memory use cases

  • Block storage
  • Volatile Memory
  • Persistent Memory aware filesystem
  • Persistent Memory aware applications
slide-7
SLIDE 7

7

Persistent Memory characteristics

  • Stores are not durable
  • The data must be flushed
  • 8-byte atomicity
  • Virtual address may change
slide-8
SLIDE 8

8

Persistent Memory kernel support

  • NVDIMM subsystem
  • DAX (Direct Access)
  • ndctl – github.com/pmem/ndctl
  • DAX support in ext2, ext4, xfs
slide-9
SLIDE 9

Non-Volatile Memory Libraries https://github.com/pmem/nvml

slide-10
SLIDE 10

10

Persistent Memory Programming Model

slide-11
SLIDE 11

11

NVML introduction

  • Set of user-space libraries
  • PMEM-aware applications
  • Source code on github - github.com/pmem/nvml
  • BSD license
  • Web page - pmem.io
  • Distros (Fedora, openSUSE)
slide-12
SLIDE 12

12

NVML libraries

Library Description libvmem Volatile memory allocator libvmmalloc malloc() replacement using libvmem libpmem Low-level support for persistence libpmemlog Append only log structure libpmemblk Atomic access to memory blocks libpmemobj Transactional object store libpmempool Managing library

slide-13
SLIDE 13

„In theory, theory and practice are the same. In practice, they are not”

Albert Einstein

slide-14
SLIDE 14

How to configure my Linux OS for persistent memory ?

slide-15
SLIDE 15

15

Kernel support

  • Kernel config: NVDIMM, DAX
  • Persistent memory emulation:

– http://pmem.io/2016/02/22/pm-emulation.html – X86_PMEM_LEGACY – Memmap=<size>!<offset>

  • Create filesystem
  • Mount with DAX
slide-16
SLIDE 16

16

Persistent Memory configuration

slide-17
SLIDE 17

Quick insight into source tree

slide-18
SLIDE 18

18

NVML overview

  • Source tree organization
  • Building
  • Running tests
  • Examples
  • Packages
slide-19
SLIDE 19

19

NVML packages

slide-20
SLIDE 20

Low level persistent memory support

slide-21
SLIDE 21

21

libpmem examples

  • How to check if memory is Persistent Memory ?
  • How to access Persistent Memory ?
  • Debug library and log messages
  • How to make memory persistent ?
  • Running examples under Valgrind pmemcheck
slide-22
SLIDE 22

22

How to check if memory is Persistent Memory ?

slide-23
SLIDE 23

23

How to access Persistent Memory ?

slide-24
SLIDE 24

24

Running pmem_is_pmem example

slide-25
SLIDE 25

25

libpmem: turning on log messages

slide-26
SLIDE 26

26

libpmem: pmem_persist - 1

slide-27
SLIDE 27

27

libpmem: pmem_persist - 2

slide-28
SLIDE 28

Persistent Memory aware tool for Valgrind github.com/pmem/valgrind

slide-29
SLIDE 29

29

PMEMCHECK

  • Valgrind tool
  • Need to build nvml with special flags
  • Checks correctness of persistent memory usage
  • Checking data persistence
  • Support for transactions (libpmemobj)
  • NVML: make EXTRA_CFLAGS=-DUSE_VALGRIND
slide-30
SLIDE 30

30

libpmem: valgrind support

slide-31
SLIDE 31

31

libpmem: pmem_persist + valgrind pmemcheck

slide-32
SLIDE 32

32

libpmem: pmem_persist - 2

slide-33
SLIDE 33

Persistent Memory pools utils

slide-34
SLIDE 34

34

pmempool: available commands

slide-35
SLIDE 35

Persistent Memory resident array of blocks

slide-36
SLIDE 36

36

Libpmemblk: examples

  • Pmempool introduction
  • Creating pmemblk pool with pmempool
  • Opening pmemblk pool
  • Reading pool parameters
  • Reading and writing blocks
slide-37
SLIDE 37

37

pmempool: create command

slide-38
SLIDE 38

38

Opening pmemblk pool

slide-39
SLIDE 39

39

Reading and writing block

slide-40
SLIDE 40

40

libpmempool: running example

slide-41
SLIDE 41

Persistent Memory Transactional Object Store

slide-42
SLIDE 42

42

libpmemobj

  • Creating pmemobj pool with pmempool
  • Opening pmemobj pool
  • PMEMoid – persistent memory pointer
  • Root object
  • Allocations
slide-43
SLIDE 43

43

Libpmemobj: simple example

  • Application which stores list of people
  • Stores first and last name
  • Constant size array
slide-44
SLIDE 44

44

Data structures

slide-45
SLIDE 45

45

Main function

slide-46
SLIDE 46

46

List of people

slide-47
SLIDE 47

47

Insert function - 1

slide-48
SLIDE 48

48

Insert function - 2

slide-49
SLIDE 49

49

Object’s constructor

slide-50
SLIDE 50

50

Clear function

slide-51
SLIDE 51

51

Recovery

slide-52
SLIDE 52

52

Running example

slide-53
SLIDE 53

53

Missing persist

slide-54
SLIDE 54

54

Valgrind pmemcheck

slide-55
SLIDE 55

55

libpmemobj: type safety macros

  • PMEMoid as void *
  • Type safety macros for convenience
  • No need to track persistent and volatile pointers
slide-56
SLIDE 56

56

Type safety macros: data structures

slide-57
SLIDE 57

57

Type safety macros: insert function

slide-58
SLIDE 58

58

Type safety macros: clear function

slide-59
SLIDE 59

59

libpmemobj: foreach macros

  • API for traversing through all objects
  • Foreach object per type
  • Useful for recovery actions
  • Avoids persistent memory leaks
slide-60
SLIDE 60

60

Foreach type macro

slide-61
SLIDE 61

61

libpmemobj: atomic lists API

  • Circular doubly linked list
  • Atomic operations
  • sys/queue – like API
  • Useful for hybrid persistent-volatile data structures
slide-62
SLIDE 62

62

atomic lists API: data structures

slide-63
SLIDE 63

63

Atomic lists API: insert function

slide-64
SLIDE 64

64

Atomic lists: clear function

slide-65
SLIDE 65

65

Atomic lists API: listing people

slide-66
SLIDE 66

66

Atomic lists API: recovery

slide-67
SLIDE 67

67

libpmemobj: transactions

  • Most generic API
  • No need for recovery
  • Convenient macros
  • Operations:

– Alloc – Free – Modify

slide-68
SLIDE 68

68

Transactions: data structures

slide-69
SLIDE 69

69

Transactions: insert function

slide-70
SLIDE 70

70

Transactions: entry initialization

slide-71
SLIDE 71

71

Transactions: inserting elements

slide-72
SLIDE 72

72

Transactions: clear function

slide-73
SLIDE 73

73

Transactions: listing people

slide-74
SLIDE 74

74

Transactions: valgrind pmemcheck

slide-75
SLIDE 75

75

Transactions: valgrind pmemcheck

slide-76
SLIDE 76

76

C++ API

  • Using language features
  • Overloading operators
  • Hiding required operations
  • Scope transactions
  • Lambda
slide-77
SLIDE 77

77

C++ API: main function

slide-78
SLIDE 78

78

C++ API: list of people class

slide-79
SLIDE 79

79

C++ API: listing people

slide-80
SLIDE 80

80

C++ API: insert function

slide-81
SLIDE 81

81

C++ API: clear function

slide-82
SLIDE 82

82

C++ API: list entry class

slide-83
SLIDE 83

83

C++ API: persistent string

slide-84
SLIDE 84

84

C++ API: valgrind pmemcheck

slide-85
SLIDE 85

85

C++ API: valgrind pmemcheck - 1

slide-86
SLIDE 86

86

C++ API: valgrind pmemcheck - 2

slide-87
SLIDE 87

87

C++ STL

  • STL allocator concept for libpmemobj
  • Using STL containers
  • Need some changes in std libraries
slide-88
SLIDE 88

88

C++ STL: list of people class

slide-89
SLIDE 89

89

C++ STL: list entry class

slide-90
SLIDE 90
slide-91
SLIDE 91