CPL 2016, week 3 Thread management: execution and shutdown Oleg - - PowerPoint PPT Presentation

CPL 2016, week 3

Thread management: execution and shutdown Oleg Batrashev

Institute of Computer Science, Tartu, Estonia

February 22, 2016

Outline

Thread management Thread pros and cons Executor framework Cancellation and shutdown Thread usage anti-patterns

Sequential web server

Example from [3] section 6.1.1

class SingleThreadWebServer { public static void main(String [] args) throws IOException { ServerSocket socket = new ServerSocket (80); while (true) { Socket connection = socket.accept (); handleRequest (connection ); } } } ◮ only one client is served at a time ◮ other clients must wait ◮ possible bad CPU/memory/IO utilization

Win of threads

◮ faster execution

◮ get weather forecast in reasonable time

◮ better throughput

◮ serve more web clients

◮ better responsiveness, for example in GUI

◮ delegate long-running task to other threads ◮ hide IO latency by delegation of blocking operations

◮ better resource utilization

◮ why quad-core if using always one core?

◮ better structured apps (next lecture)

Cost of threads

◮ creating new thread is not free (see lab1)

◮ mode switch (from user to kernel mode) ◮ ask OS for the new thread context and memory

◮ switching threads is not free:

◮ CPUs/cores must be shared between threads – (context

switch)

◮ synchronizing threads is not free

◮ inhibit optimizations due to visibility constraints (lecture 1) ◮ contended synchronization – many threads request the same

resource and fail

◮ race conditions and deadlocks in your application!

Threading strategies

◮ have only single thread – no wins ◮ new thread for every new task

◮ pay cost of creating many threads

◮ reuse old threads – thread pools

◮ avoid costs of thread creation ◮ no resource exhaustion (too many threads)

possible strategies for a server request handling:

◮ thread per client request ◮ thread per connection ◮ N connections to M threads

Thread per task web-server

Example from [3] section 6.1.2

class ThreadPerTaskWebServer { public static void main(String [] args) throws IOException { ServerSocket socket = new ServerSocket (80); while (true) { final Socket connection = socket.accept (); Runnable task = new Runnable () { public void run () { handleRequest (connection ); } }; new Thread(task ). start (); } } } ◮ many clients are served ◮ cost of creating threads, danger of resource exhaustion

Outline

Thread management Thread pros and cons Executor framework Cancellation and shutdown Thread usage anti-patterns

Executor interface

◮ Java5 introduced new java.util.concurrent framework public interface Executor { void execute(Runnable command ); } ◮ executor with its own strategy implements the interface ◮ Runnable command represents the task to be executed ◮ to execute a command invoke executor.execute(mycommand); ◮ separate tasks (code to run) from execution (how to run)

◮ flexibility to replace the stategy ◮ same socket server with 3 different executors (next slides)

◮ later will see task cancellation and executor shutdown

Executor: fixed size thread pool

Example from [3] section 6.2.1

private static final Executor exec = Executors. newFixedThreadPool (NTHREADS ); .. while (true) { final Socket connection = socket.accept (); Runnable task = new Runnable () { public void run () { handleRequest (connection ); } }; exec.execute(task ); } ◮ requests are handled in threads concurrently up to NTHREADS

at a time

Executor: thread per task and no threads

Thread per task executor:

public class ThreadPerTaskExecutor implements Executor { public void execute(Runnable r) { new Thread(r). start (); }; }

No thread executor:

public class WithinThreadExecutor implements Executor { public void execute(Runnable r) { r.run (); }; }

Standard executors

Executors class factory methods:

◮ newFixedThreadPool – executor with fixed number of threads ◮ newCachedThreadPool – creates threads on demand

◮ old threads will be reused if available ◮ destroys thread on 60 seconds of inactivity

◮ newSingleThreadExecutor – one thread executes tasks

◮ processed sequentially in FIFO, LIFO, or priority order ◮ like newFixedThreadPool(1) but not reconfigurable to several

threads

◮ newScheduledThreadPool – fixed size thread pool that

supports delayed and periodic task execution

◮ similar to Timer but handles unchecked exceptions ◮ DelayedQueue allowes to schedule delayed messages

Result bearing tasks

public interface Callable <V> { V call () throws Exception; } public interface Future <V> { boolean cancel(boolean mayInterruptIfRunning ); boolean isCancelled (); boolean isDone (); V get () throws InterruptedException , ExecutionException , CancellationException ; V get(long timeout , TimeUnit unit) throws InterruptedException , ExecutionException , CancellationException , TimeoutException ; } ◮ Callable is the task with a result V

◮ use it instead of Runnable if it returns a result

◮ Future is the interface to control your task execution

◮ provided by Executor but may be tuned if needed

Using Futures

Create task, submit, and use the future to get results:

Callable <Data > task = new Callable <Data >() { ... } Future <Data > future = executor.submit(task ); // task is executed in separate thread // do something else in a while Data data = future.get () // blocks until data is available ◮ should cancel it if not needed anymore ◮ could use get() with timeout to poll the result ◮ it throws exceptions depending on what happened

since Java 6 ExecutorService implementations can override newTaskFor in AbstractExecutorService:

◮ default is just new FutureTask<T>(callable) ◮ own implementation allows to modify cancel procedure, e.g.

close sockets or other resources on task cancellation

Completion service

Imagine you have several futures that may block:

Data d1 = future1.get (); process(d1); Data d2 = future2.get (); process(d2); ◮ it is not defined in which order future1 and future2 are

available Executor meets blocking queue ExecutorCompletionService:

◮ finished results are put into the queue as Futures ◮ implementation is quite straightforward

◮ override FutureTask done method

protected void done () { completionQueue .add(this ); }

◮ delegate future.get() to this queue

Outline

Thread management Thread pros and cons Executor framework Cancellation and shutdown Thread usage anti-patterns

Cancellation reasons

A task may be abstained from execution or stopped:

◮ a user requests that

◮ “cancel” button in GUI ◮ user selects another object to request and show

◮ time-limited activity

◮ timeout on server connect or some calculation

◮ errors in execution

◮ may trigger cancellation of related tasks

◮ shutdown – application is being closed

Thread interruption

Forcefully stopping a thread is not a good idea:

◮ it may need some finalizing operations ◮ Java Thread.stop() method is deprecated

Instead, Thread.interrupt() “asks” thread to stop

◮ thread interrupt flag is set (interrupted state) ◮ some blocking operations (sleep,wait) throw

InterruptedException

◮ interrupt flag is cleared when exception is thrown! ◮ usually must re-set after catch (see next slide why)

◮ there are no other consequences, i.e. thread continue running

as if nothing happened!

◮ a thread must check for the interrupt flag regularly ◮ it may/should use its own volatile boolean isRunning

◮ socket.connect, stream.read, synchronized are not

interruptible

Example: consuming interrupt state

public void run () { try { while (! Thread. currentThread (). isInterrupted ()) { Element elem = queue.take (); System.out.println(elem ); } } catch ( InterruptedException consumed) { /* Allow thread to exit but consume interrupted

• flag. */

} } public void cancel () { interrupt (); } ◮ may be ok if you own the thread ◮ otherwise, it may be interrupt request to shutdown the

executor, not only cancel the task

◮ executor code may suspend in blocking operation, because you

consumed the flag

◮ use Thread.currentThread.interrupt() in

catch(InterruptedException) to restore the flag

Non-interruptible blocking

Some blocking operations are non-interruptible:

◮ socket in java.io – stream read and write methods

◮ closing the socket throws SocketException

◮ channels and selectors in java.nio are mostly interruptible ◮ implicit lock – not interruptible

◮ some locks in java.util.concurrent are interruptible, ◮ otherwise, there is nothing you can do about it!

Example, lets stop a service by

◮ calling service’s synchronized stop() which

◮ clears service state and calls thread.interrupt()

◮ that service thread just decided to take synchronized(this)

◮ after lock is acquired it tries to use cleaned service state ◮ remember interrupt() just “asks” to stop

Non-standard cancellation

◮ Socket should be closed when task is cancelled

Could define special termination for a Callable

public interface CallableWithInterrupt <V> extends Callable <V> { void interrupt (); }

Define executor with modified task cancel routine

protected <T> RunnableFuture <T> newTaskFor(Callable <T> callable) { final CallableWithInterrupt <T> callableMy = ( CallableWithInterrupt <T>) callable; return new FutureTask <T>( callableMy ) { public boolean cancel(boolean mayInterruptIfRunning ) { // let the task specify its own cancel routine callableMy .interrupt (); return super.cancel( mayInterruptIfRunning ); } }; } ◮ use socket.close() in callable.interrupt()

Executor shutdown

ExecutorService provides two methods to stop:

• 1. shutdown() - graceful shutdown

◮ no new tasks are accepted ◮ existing tasks (including the queue) are allowed to complete ◮ after that is shuts down the threads

• 2. shutdownNow() - abrupt shutdown

◮ cancels all tasks that are running ◮ returns all tasks in from the queue

The decision on what to use is very much service dependent:

◮ let critical tasks to finish, or ◮ allow unimportant tasks to be forgotten.

Outline

Thread management Thread pros and cons Executor framework Cancellation and shutdown Thread usage anti-patterns

AWT thread

AWT thread = Event dispatch thread = GUI thread

◮ Some Swing component methods are labelled "thread safe" in

the API specification; these can be safely invoked from any

• thread. All other Swing component methods must be invoked

from the event dispatch thread. Programs that ignore this rule may function correctly most of the time, but are subject to unpredictable errors that are difficult to reproduce.

◮ Other tasks can be scheduled by application code, using

invokeLater or invokeAndWait. https://docs.oracle.com/javase/tutorial/uiswing/ concurrency/dispatch.html

Runnable updateListModel = new Runnable () { @Override public void run () { travelerListModel .push(traveler ); } }; SwingUtilities . invokeLater ( updateListModel );

Update list/tree model from non-AWT thread

TravelerListModel contains travelers to be shown in the listbox

private List <Traveler > travelers = new ArrayList < >(); ◮ JList uses two to iterate and show the elements: public synchronized int getSize () public synchronized Traveler getElementAt (int index) ◮ Connection thread uses: public synchronized void push(Traveler traveler) public synchronized void delete(Traveler traveler) ◮ Although all methods are synchronized, may still get Exception in thread "AWT -EventQueue -1" java.lang. IndexOutOfBoundsException : Index: 44, Size: 44 ◮ Swing does not lock model during iteration! ◮ Use SwingUtilities.invokeLater()

Wild threads and thread knots

◮ Should not create threads without necessity

◮ susceptible to race conditions and deadlocks

◮ manage your threads directly or through executors

◮ think about cancellation policy!

◮ knot of threads appears if there are no restrictions where a

thread may enter

GUI Middle Net layer

◮ red rectangles are locks ◮ deadlock be solved with lock ordering: lets say GUI lock is first ◮ decide while in the network layer whether you will need GUI

lock – happy code designing

◮ say “yes” just in case – GUI lock becomes global lock ◮ otherwise danger of deadlock!

Conclusions

◮ manage your threads wisely

◮ do not overuse – think if you need them

◮ use Executor framework thread pools ◮ think about task cancellation and service shutdown

◮ create your own cancellation routines

◮ do not update Swing and its data from non-AWT thread ◮ thread knots – disaster of multi-threaded programs

◮ there are nice solutions (next 2 lectures)