Cross-Platform OpenCL Application Development Tyler Sorensen (led - - PowerPoint PPT Presentation

▶
cross platform opencl application development
SMART_READER_LITE
LIVE PREVIEW

Cross-Platform OpenCL Application Development Tyler Sorensen (led - - PowerPoint PPT Presentation

May 12, 2023 138 likes •923 views

The Hitchhikers Guide to Cross-Platform OpenCL Application Development Tyler Sorensen (led the work and made the slidese) Alastair F. Donaldson (delivered this version of the talk) Imperial College London, UK UKMAC May 2016 1 OpenCL

slide-1
SLIDE 1

The Hitchhiker’s Guide to Cross-Platform OpenCL Application Development

Tyler Sorensen (led the work and made the slidese) Alastair F. Donaldson (delivered this version of the talk) Imperial College London, UK UKMAC May 2016

1

slide-2
SLIDE 2

“OpenCL supports a wide range of applications… through a low-level, high-performance, portable abstraction.” Page 11: OpenCL 2.1 specification

2

slide-3
SLIDE 3

“OpenCL supports a wide range of applications… through a low-level, high-performance, portable abstraction.” Page 11: OpenCL 2.1 specification

3

slide-4
SLIDE 4

“OpenCL supports a wide range of applications… through a low-level, high-performance, portable abstraction.” Page 11: OpenCL 2.1 specification

We consider functional portability rather than performance portability

4

slide-5
SLIDE 5

Example

  • single source shortest path application

Quadro K5200 (Nvidia) Intel HD5500

5

slide-6
SLIDE 6

Example

  • single source shortest path application

Quadro K5200 (Nvidia) Intel HD5500

6

slide-7
SLIDE 7

Example

  • single source shortest path application

Quadro K5200 (Nvidia) Intel HD5500

7

slide-8
SLIDE 8

An experience report on OpenCL portability

  • How well is portability evaluated?
  • Our experience running applications on 8 GPUs spanning 4 vendors
  • Recommendations going forward

8

slide-9
SLIDE 9

An experience report on OpenCL portability

  • How well is portability evaluated?
  • Our experience running applications on 8 GPUs spanning 4 vendors
  • Recommendations going forward

9

slide-10
SLIDE 10

Portability in research literature

  • Reviewed the 50 most recent OpenCL papers on:

http://hgpu.org/

  • Only considered papers including GPU targets
  • Only considered papers with some type of experimental evaluation
  • How many different vendors did the study experiment with?

10

slide-11
SLIDE 11

Portability in research literature

58% (29)

1

Results (number of evaluated vendors)

11

slide-12
SLIDE 12

Portability in research literature

58% (29) 36% (18)

1 2

Results (number of evaluated vendors)

12

slide-13
SLIDE 13

Portability in research literature

58% (29) 36% (18) 6% (3)

1 2 3

Results (number of evaluated vendors)

13

slide-14
SLIDE 14

Portability in research literature

Results (which vendor)

39 23 8 3 1 Nvidia AMD Intel ARM Imagination

14

slide-15
SLIDE 15

Portability in research literature

Results (which vendor)

39 23 8 3 1 Nvidia AMD Intel ARM Imagination

Portability is not well tested in research literature!

15

slide-16
SLIDE 16

An experience report on OpenCL portability

  • How well is portability evaluated?
  • Our experience running applications on 8 GPUs spanning 4 vendors
  • Recommendations going forward

16

slide-17
SLIDE 17

Applications

  • Part of a larger study on GPU irregular parallelism

https://github.com/pannotia/pannotia

17

slide-18
SLIDE 18

Applications

Pannotia

  • Target AMD Radeon HD 7000
  • Written in OpenCL 1.x
  • 4 graph algorithms applications
  • Our aim: run these benchmarks on

OpenCL platorms from several vendors

https://github.com/pannotia/pannotia

18

slide-19
SLIDE 19

Applications

Pannotia

https://github.com/pannotia/pannotia GPU_linear_algebra_routine1; GPU_linear_algebra_routine2; GPU_linear_algebra_routine3; Loop until a fixed point is reached.

19

  • Target AMD Radeon HD 7000
  • Written in OpenCL 1.x
  • 4 graph algorithms applications
  • Our aim: run these benchmarks on

OpenCL platorms from several vendors

slide-20
SLIDE 20

Applications

LonestarGPU

  • Target Nvidia Kepler and Fermi
  • Written in CUDA
  • 4 graph algorithms applications
  • Our aim: port these benchmarks to

OpenCL to run across a range of platforms

http://iss.ices.utexas.edu/?p=projects/galois/lonestargpu

20

slide-21
SLIDE 21

Applications

LonestarGPU

http://iss.ices.utexas.edu/?p=projects/galois/lonestargpu shared worklist wg0 wg1 wg2 wg3

21

  • Target Nvidia Kepler and Fermi
  • Written in CUDA
  • 4 graph algorithms applications
  • Our aim: port these benchmarks to

OpenCL to run across a range of platforms

slide-22
SLIDE 22

Chip Vendor Compute Units OpenCL Version Type GTX 980 Nvidia 16 1.1 Discrete Quadro K500 Nvidia 12 1.1 Discrete Iris 6100 Intel 47 2.0 Integrated HD 5500 Intel 24 2.0 Integrated Radeon R9 AMD 28 2.0 Discrete Radeon R7 AMD 8 2.0 Integrated Mali-T628 ARM 4 1.2 Integrated Mali-T628 ARM 2 1.2 integrated

GPUs

22

slide-23
SLIDE 23

Portability Issues

12 issues encountered, grouped into categories

  • 3 Framework bugs
  • 6 Specification limitations
  • 3 Programming bugs

23

slide-24
SLIDE 24

Portability Issues

12 issues encountered, grouped into categories

  • 3 Framework bugs
  • 6 Specification limitations
  • 3 Programming bugs

24

slide-25
SLIDE 25

Framework bugs

#1 Compiler crash

Platforms: Intel

25

slide-26
SLIDE 26

Framework bugs

#1 Compiler crash

Platforms: Intel

26

slide-27
SLIDE 27

Framework bugs

#1 Compiler crash

Platforms: Intel compiling several large kernels occasionally crashes compiler Workaround: reduce the number of kernels in file

27

slide-28
SLIDE 28

Framework bugs

#2 Non-terminating loops

Platforms: Nvidia and AMD

28

slide-29
SLIDE 29

Framework bugs

#2 Non-terminating loops

Platforms: Nvidia and AMD

while(true) { more_work = false; .. // Do computation, .. // if more work, set more_work if (!more_work) break; } This looping idiom used in kernel code

29

slide-30
SLIDE 30

Framework bugs

#2 Non-terminating loops

Platforms: Nvidia and AMD

while(true) { more_work = false; .. // Do computation, .. // if more work, set more_work if (!more_work) break; } Does not terminate on Nvidia and AMD platforms!! This looping idiom used in kernel code

30

slide-31
SLIDE 31

Framework bugs

#2 Non-terminating loops

Platforms: Nvidia and AMD

Change while loop to for loop This looping idiom used in kernel code

while(true) { for (int i = 0; i < INT_MAX; i++) { more_work = false; .. // Do computation, .. // if more work, set more_work if (!more_work) break; }

End value of i is consistent across platforms

31

slide-32
SLIDE 32

Framework bugs

#3 AMD defunct processes

Platforms: AMD on Linux Long running kernels become defunct and un-killable requiring a reboot. Workaround: Switch to Windows OS

32

slide-33
SLIDE 33

Portability Issues

12 issues encountered, grouped into categories

  • 3 Framework bugs
  • 6 Specification limitations
  • 3 Programming bugs

33

slide-34
SLIDE 34

Specification limitations

#1 GPU watchdogs

Platforms and operating systems handle watchdogs differently.

GPU GPU GPU

Windows Linux (Ubuntu) Chrome OS

34

slide-35
SLIDE 35

Specification limitations

#1 GPU watchdogs

Platforms and operating systems handle watchdogs differently.

GPU GPU GPU

Windows Linux (Ubuntu) Chrome OS Controlled with registry Watchdog kills entire OpenCL process

35

slide-36
SLIDE 36

Specification limitations

#1 GPU watchdogs

Platforms and operating systems handle watchdogs differently.

GPU GPU GPU

Windows Linux (Ubuntu) Controlled with registry Watchdog kills entire OpenCL process Controlled in X server settings Watchdog only kills kernel Chrome OS

36

slide-37
SLIDE 37

Specification limitations

#1 GPU watchdogs

Platforms and operating systems handle watchdogs differently.

GPU GPU GPU

Windows Linux (Ubuntu) Controlled with registry Watchdog kills entire OpenCL process Controlled in X server settings Watchdog only kills kernel Cannot control at all without recompiling the driver Chrome OS

37

slide-38
SLIDE 38

Specification limitations

#2 Occupancy vs compute units

An OpenCL device has one or more compute units. A workgroup executes on a single compute unit.

Intel OpenCL Optimisation Guide

38

slide-39
SLIDE 39

Specification limitations

#2 Occupancy vs compute units

Persistent thread model (Gupta et al. PIPC’12): once scheduled, a workgroup is guaranteed to make progress

An OpenCL device has one or more compute units. A workgroup executes on a single compute unit.

Intel OpenCL Optimisation Guide

39

slide-40
SLIDE 40

Specification limitations

#2 Occupancy vs compute units

Persistent thread model (Gupta et al. PIPC’12): once scheduled, a workgroup is guaranteed to make progress LonestarGPU applications depend on this

An OpenCL device has one or more compute units. A workgroup executes on a single compute unit.

Intel OpenCL Optimisation Guide

40

slide-41
SLIDE 41

#2 Occupancy vs compute units

Specification limitations

chip compute units PT occupancy GTX 980 16 Quadro K500 12 Iris 6100 47 HD 5500 24 Radeon R9 28 Radeon R7 8 Mali-T628 4 Mali-T628 2

41

slide-42
SLIDE 42

#2 Occupancy vs compute units

Specification limitations

chip compute units PT occupancy GTX 980 16 Quadro K500 12 12 Iris 6100 47 HD 5500 24 Radeon R9 28 Radeon R7 8 Mali-T628 4 4 Mali-T628 2 2

Compute units are safe and optimal

42

slide-43
SLIDE 43

#2 Occupancy vs compute units

Specification limitations

chip compute units PT occupancy GTX 980 16 32 Quadro K500 12 12 Iris 6100 47 HD 5500 24 Radeon R9 28 48 Radeon R7 8 16 Mali-T628 4 4 Mali-T628 2 2

Compute units are safe and optimal Compute units are safe but not optimal

43

slide-44
SLIDE 44

#2 Occupancy vs compute units

Specification limitations

chip compute units PT occupancy GTX 980 16 32 Quadro K500 12 12 Iris 6100 47 6 HD 5500 24 3 Radeon R9 28 48 Radeon R7 8 16 Mali-T628 4 4 Mali-T628 2 2

Compute units are safe and optimal Compute units are safe but not optimal Compute units are not safe

44

slide-45
SLIDE 45

Portability Issues

12 issues encountered, grouped into categories

  • 3 Framework bugs
  • 6 Specification limitations
  • 3 Programming bugs

45

slide-46
SLIDE 46

Programming bugs

#1 Data-races

Application: LonestarGPU bfs and sssp Fix: Add additional synchronisation barriers

Quadro K5200 (Nvidia) Intel HD5500

46

slide-47
SLIDE 47

Programming bugs

#1 Data-races

Application: LonestarGPU bfs and sssp Fix: Add additional synchronisation barriers

Quadro K5200 (Nvidia) Intel HD5500

!

Bug was dormant on Nvidia but caused crashes on Intel

47

slide-48
SLIDE 48

Programming bugs

#2 Struct kernel arguments

How to represent a graph:

48

slide-49
SLIDE 49

Programming bugs

#2 Struct kernel arguments

  • adjacency matrix
  • array of edge weights
  • number of nodes
  • number of edges

How to represent a graph:

49

slide-50
SLIDE 50

Programming bugs

#2 Struct kernel arguments

  • adjacency matrix
  • array of edge weights
  • number of nodes
  • number of edges

struct Graph

How to represent a graph: Graphs are large and globally shared so they go into global memory. Some struct members are global memory pointers

50

slide-51
SLIDE 51

Programming bugs

#2 Struct kernel arguments

Chip GTX 980 Quadro K500 Iris 6100 HD 5500 Radeon R9 Radeon R7 Mali-T628 Mali-T628

clSetKernelArg (bfs_kernel, 0, sizeof(Graph), &graph1); // Execute bfs kernel

51

slide-52
SLIDE 52

Programming bugs

#2 Struct kernel arguments

Chip GTX 980 Quadro K500 Iris 6100 HD 5500 Radeon R9 Radeon R7 Mali-T628 Mali-T628

clSetKernelArg (bfs_kernel, 0, sizeof(Graph), &graph1); // Execute bfs kernel

52

slide-53
SLIDE 53

Programming bugs

#2 Struct kernel arguments

“Arguments to kernel functions that are declared to be a struct or union do not allow OpenCL objects to be passed as elements of the struct or union”

Page 176: OpenCL 2.0 specification

53

slide-54
SLIDE 54

An experience report on OpenCL portability

  • How well is portability evaluated?
  • Our experience running applications on 8 GPUs spanning 4 vendors
  • Recommendations going forward

54

slide-55
SLIDE 55

Going forward

  • Conformance tests
  • Compiler Fuzzing
  • “Many-Core Compiler Fuzzing” PLDI’16, Lidbury et al.
  • Memory consistency
  • “GPU Concurrency: Weak Behaviours and Programming Assumptions” ASPLOS’15,

Alglave et al.

55

slide-56
SLIDE 56

Going forward

  • Conformance tests
  • Compiler Fuzzing
  • “Many-Core Compiler Fuzzing” PLDI’16, Lidbury et al.
  • Memory consistency
  • “GPU Concurrency: Weak Behaviours and Programming Assumptions” ASPLOS’15,

Alglave et al.

unofficial open source tests?

56

slide-57
SLIDE 57

Going forward

  • Specification clarifications
  • Inter-workgroup execution model
  • “A Study of Persistent Threads Style GPU Programming for GPGPU Workloads”, PIPC’12

Gupta et al.

  • GPU watchdog

57

slide-58
SLIDE 58

Going forward

  • Programming tools
  • Data-race checkers
  • GPUVerify “The Design and Implementation of a Verification Technique for GPU Kernels”,

TOPLAS’15, Betts et al.

  • Dynamic analysis tools
  • OCLGrind “Oclgrind: an extensible OpenCL device simulator”, IWOCL’15, Price and

McIntosh-Smith

58

slide-59
SLIDE 59

Conclusions

  • Most applications were able to run cross-platform!
  • Many portability challenges
  • We believe that as a community we can overcome these challenges

for a more portable OpenCL world!

59

slide-60
SLIDE 60

We are hir iring

  • Postdoctoral researcher in Reliable Many-Core Programming
  • Two fully-funded UK/EU PhD studentships on reliability and efficiency
  • f concurrent and parallel software
  • Talk to me, or email (afd@imperial.ac.uk) if you are interested
  • About our group: http://multicore.doc.ic.ac.uk

60

slide-61
SLIDE 61

Thank You

Tyler Sorensen http://www.doc.ic.ac.uk/~tsorensen/ Alastair Donaldson http://multicore.doc.ic.ac.uk/

  • Assessed the OpenCL portability evaluation in research
  • Surveyed 50 most recent OpenCL papers
  • Found portability issues across 8 GPUs (4 Vendors)
  • 3 framework bugs, 6 specification limitations, 3 Programming Bugs
  • Suggested ways to improve OpenCL portability
  • Conformance tests, specification clarifications, testing/verification tools

61

slide-62
SLIDE 62

Specification limitations

#4 Floating point accuracy

Application: LonestarGPU DMR 32 bit floating point application successful on Intel

62

slide-63
SLIDE 63

Specification limitations

#4 Floating point accuracy

Application: LonestarGPU DMR 32 bit floating point application successful on Intel 32 bit floating point application fails on Nvidia

63

slide-64
SLIDE 64

Specification limitations

#5 OS portability

Chip Windows Linux Radeon R9 Radeon R7 Mali-T628 Mali-T628

64

slide-65
SLIDE 65

Specification limitations

#5 OS portability

Chip Windows Linux Radeon R9 Radeon R7 Mali-T628 Mali-T628

Defunct process bug

65

slide-66
SLIDE 66

Specification limitations

#5 OS portability

Chip Windows Linux Radeon R9 Radeon R7 Mali-T628 Mali-T628

Thus entire OpenCL application (device and host) must be cross platform

Defunct process bug

66

slide-67
SLIDE 67

Specification limitations

#1 Memory allocation failures

Platforms: Intel Host memory allocations can cause device memory allocations to fail Due to fragmentation

67

slide-68
SLIDE 68

Specification limitations

#3 Memory consistency

OpenCL 2.0 atomics allow synchronisation idioms

68

slide-69
SLIDE 69

Specification limitations

#3 Memory consistency

OpenCL 2.0 atomics allow synchronisation idioms

Chip OpenCL Version GTX 980 1.1 Quadro K500 1.1 Mali-T628 1.2 Mali-T628 1.2

No support for OpenCL 2.0!

69

slide-70
SLIDE 70

Specification limitations

#3 Memory consistency

Implement our own atomic operations

typedef int atomic_int; void atomic_store(atomic_int *addr, int val) { mem_fence() *addr = val; mem_fence() }

70

slide-71
SLIDE 71

Specification limitations

#3 Memory consistency

These chips passed our memory consistency unit tests

Chip OpenCL Version GTX 980 1.1 Quadro K500 1.1 Mali-T628 1.2 Mali-T628 1.2

71

slide-72
SLIDE 72

Specification limitations

#3 Memory consistency

Several other (older) chips did not

Chip Vendor OpenCL Version GTX 480 Nvidia 1.1 Tesla C2075 Nvidia 1.1 HD 4400 Intel 1.2 Radeon HD 7970 AMD 1.2 Radeon HD 6570 AMD 1.2

72

slide-73
SLIDE 73

Specification limitations

#3 Memory consistency

Several other (older) chips did not

Chip Vendor OpenCL Version GTX 480 Nvidia 1.1 Tesla C2075 Nvidia 1.1 HD 4400 Intel 1.2 Radeon HD 7970 AMD 1.2 Radeon HD 6570 AMD 1.2

We did not consider these chips further

73

slide-74
SLIDE 74

Programming bugs

#2 Stability

Application: LonestarGPU DMR

DRM() Quadro K5200 (Nvidia) execute application repeatedly

74

slide-75
SLIDE 75

Programming bugs

#2 Stability

Application: LonestarGPU DMR

  • ccasional failures

(known by developer and deemed acceptable) Due to floating point precision DRM() execute application repeatedly Quadro K5200 (Nvidia)

75

slide-76
SLIDE 76

Programming bugs

#2 Stability

Application: LonestarGPU DMR

DRM() execute application repeatedly Radeon R9 (AMD)

76

slide-77
SLIDE 77

Programming bugs

#2 Stability

Application: LonestarGPU DMR

Fails nearly every iteration

  • n AMD chips

DRM() execute application repeatedly Radeon R9 (AMD)

77