in OpenMP Paolo Burgio paolo.burgio@unimore.it Outline Expressing - - PowerPoint PPT Presentation
Barriers in OpenMP Paolo Burgio paolo.burgio@unimore.it Outline Expressing parallelism Understanding parallel threads Memory Data management Data clauses Synchronization Barriers, locks, critical sections Work
› Expressing parallelism
– Understanding parallel threads
› Memory Data management
– Data clauses
› Synchronization
– Barriers, locks, critical sections
› Work partitioning
– Loops, sections, single work, tasks…
› Execution devices
– Target
2
– barrier – flush – master – critical – atomic – taskwait – taskgroup – ordered – ..and OpenMP locks
3
– Master thread spawns a team of Slave threads – They all perform computation in parallel – At the end of the parallel region, implicit barrier
4
int main() { /* Sequential code */ #pragma omp parallel num_threads(4) { /* Parallel code */ } // Parreg end: (implicit) barrier /* (More) sequential code */ }
– Master thread spawns a team of Slave threads – They all perform computation in parallel – At the end of the parallel region, implicit barrier
4
int main() { /* Sequential code */ #pragma omp parallel num_threads(4) { /* Parallel code */ } // Parreg end: (implicit) barrier /* (More) sequential code */ }
– Master thread spawns a team of Slave threads – They all perform computation in parallel – At the end of the parallel region, implicit barrier
4
int main() { /* Sequential code */ #pragma omp parallel num_threads(4) { /* Parallel code */ } // Parreg end: (implicit) barrier /* (More) sequential code */ }
– Master thread spawns a team of Slave threads – They all perform computation in parallel – At the end of the parallel region, implicit barrier
4
int main() { /* Sequential code */ #pragma omp parallel num_threads(4) { /* Parallel code */ } // Parreg end: (implicit) barrier /* (More) sequential code */ }
› All threads in a team must wait for all the other threads before going on
– "Each barrier region must be encountered by all threads in a team or by none at all" – "The sequence of barrier regions encountered must be the same for every thread in a team" – Why?
› Binding set is the team of threads from the innermost enclosing parreg
– "It applies to"
› Also, it enforces a consistent view of the shared memory
– We'll see this..
5
#pragma omp barrier new-line (a standalone directive)
– Printing "Hello World" – Put a #pragma omp barrier – Reprint "Hello World" after
– Now, remove the barrier construct
– E.g., if(omp_get_thread_num() == 0) { ... } – What do you see? – Error!!!!
6
Let's code!
– You cannot trust any (potentially modified) shared vars before a barrier – Of course, there are no problems with private vars
7
– Threads have access to "a place to store and to retrieve variables, called the memory" – Threads can have a temporary view of the memory › Caches, registers, scratchpads… › Can still be accessed by other threads
8
Process
Shared
T T T
VAR
Temp Priv.
VAR VAR
Temp Priv. Temp Priv. first/ private(a)
shared(a)
????? ??
– ..and to the main memory – Few KB of data
– Used to store a copy mostly accessed data – To speedup execution even by 10-20 times – Istruction caches/Data caches
– And transparently – Poor or no control at all at application level – Extremely dangerous in multi- and many-cores
10
11
A cache is a hardware or software component that stores data so future requests for that data can be served faster; the data stored in a cache might be the result
an earlier computation,
the duplicate
data stored elsewhere.
eng.wikipedia.org
CPU
D$
Main memory, or L3 cache Offchip memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
I$ I$ I$ I$ Level-2 $
– Some embedded architectures do not have D$
– E.g., BOSCH removed I$s
– Flush command (typically, all cache) – Color cache (assign to threads) – Prefetch (move data before it's actually needed)
12
13
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; b = a; // ... c = a;
a
13
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 b = a; // ... c = a;
a
13
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a; // ... c = a;
a
13
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a; // ... c = a;
a
14
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; b = a; // ... dcache_flush(); c = a;
a
14
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 b = a; // ... dcache_flush(); c = a;
a
14
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a; // ... dcache_flush(); c = a;
a
14
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 5 b = a; // ... dcache_flush(); c = a;
a
14
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 5 b = a; // ... dcache_flush(); c = a; 5
a
15
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 b = a;
a
15
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a; 5
a
15
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a; 5
a
15
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a; 5 5
a
16
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; b = a;
a
16
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 5 b = a;
a
16
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a;
a
16
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; 11 5 b = a; 5
a
17
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; dcache_flush(); 11 b = a;
a
17
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; dcache_flush(); 11 5 b = a;
a
17
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; dcache_flush(); 11 b = a; 5
a
17
CPU
D$
Main memory
CPU 1
D$
CPU 2
D$
CPU 3
D$
11 a = 5; dcache_flush(); 11 b = a; 5 5
a
– More "relaxed"
– Makes its temporary view of the shared memory consistent with other threads – "Calls to dcache_flush()"
18
#pragma omp flush [(list)] new-line
#pragma omp barrier › Joins the threads of a team › Applies to all threads of a team › Forces consistency of threads' temporary view of the shared memory #pragma omp flush › Applies to one thread › Forces consistency of its temporary view of the shared memory › Much lighter!
19
– Engineered for portability
20
User code Operating System Hardware OpenMP runtime
a = 5; #pragma omp flush void GOMP_flush() { dcache_flush(); } D$ void dcache_flush() { asm("mov r15, #1"); }
– Engineered for portability
20
User code Operating System Hardware OpenMP runtime
a = 5; #pragma omp flush void GOMP_flush() { dcache_flush(); } D$ void dcache_flush() { asm("mov r15, #1"); } D$
21
Let's code!
– http://hipert.unimore.it/people/paolob/pub/PhD/index.html
– paolo.burgio@unimore.it – http://hipert.mat.unimore.it/people/paolob/
– http://www.google.com – http://www.openmp.org – https://gcc.gnu.org/
– http://www.google.com
22