Document Number: N2276=07-0136
Anthony Williams <anthony@justsoftwaresolutions.co.uk>
Just Software Solutions Ltd
2007-05-07

N2276 — Thread Pools and Futures

1 Introduction

At Oxford, the combined EWG and LWG voted to proceed with work on Thread Pools and Futures for C++0x, even though this work had previously been destined for TR2. This paper is provided to further discussions in this area. It draws heavily on N2094 and N2185.

2 Overview

2.1 Building Blocks

The building blocks of the system are three class templates: future, packaged_task, and promise.

2.1.1 future<R>

A future<R> encapsulates a potential value of type R, or an exception. When the future is ready, you can retrieve the encapsulated value or exception either by using the explicit get() member function, or by making use of the implicit conversion to R. If the future is not ready, then the access functions will block until the future becomes ready. If a ready future encapsulates a value, then the access functions will return that value. If a ready future encapsulates an exception, then the access functions will throw a copy of the encapsulated exception.

2.1.2 packaged_task<R>

This is a wrapper for a callable object with a return type of R. A packaged_task is itself a callable type — invoking it's function call operator will invoke the encapsulated callable object. This still doesn't get you the value returned by the encapsulated object, though — to do that you need a future<R> which you can obtain using the get_future() member function. Once the function call operator has been invoked, all futures associated with a particular packaged_task will become ready, and will store either the value returned by the invocation of the encapsulated object, or the exception thrown by said invocation. packaged_task is intended to be used in cases where the user needs to build a thread pool or similar structure because the standard facilities do not have the desired characteristics.

2.1.3 promise<R>

This is an alternative means of creating futures without a callable object — the value of type R to be returned by the associated futures is specified directly by invoking the set_value() member function. Alternatively, the exception to be returned can be specified by invoking the set_exception() member function. This allows the result to be set from a callback invoked from code that has no knowledge of the promise or its associated futures.

2.2 High Level Support

As well as the low level building blocks, this proposal also includes high level support in the form of the thread_pool class, and the launch_in_thread and launch_in_pool function templates.

2.2.1 launch_in_pool

launch_in_pool submits a task to a system-supplied thread pool to be run. It takes a single parameter: the function or callable object to run, and returns a future which encapsulate the result of the function call. The intent is that the library implementor can provide a thread pool that provides what they believe to be optimal performance characteristics for their customers, taking advantage of system-specific information. It is intended that the vast majority of applications that need thread pool functionality will use this function to allow the system to define an appropriate thread pool.

2.2.2 launch_in_thread

launch_in_thread is similar to launch_in_pool, but it creates a separate thread dedicated to the task being run, which terminates when the task completes. It takes a single parameter: the function or callable object to run, and returns a future which encapsulate the result of the function call. This is an enhancement over std::thread for tasks that need a dedicated thread, but where the caller wishes to capture the returned value or any unhandled exception.

2.2.3 thread_pool

A thread_pool is just that: a pool of threads. These threads will run for the lifetime of the thread_pool object. The user can submit a task to the thread_pool, which will then be added to a queue of tasks. When one of the threads in the pool becomes idle, it will take a task from the queue and execute it. Once the task is complete, the thread will try and obtain a new task, and so on. A task can be submitted to the thread_pool by passing a callable object to the submit() member function. In return, the caller is given a future to encapsulate the result of the task.

3 Proposed Wording

3.1 Header <threadpool> synopsis

    namespace std
    {
        template <typename R> class future;
        template <typename R> class future<R&&>;
        template <> class future<void>;

        template <typename R> class packaged_task;

        template <typename F>
        packaged_task<typename result_of<F()>::type> package_task(F const& f);

        template <typename F>
        packaged_task<typename result_of<F()>::type> package_task(F&& f);

        template <typename R> class promise;
        template <typename R> class promise<R&>;
        template <> class promise<void>;

        template <typename F>
        future<typename result_of<F()>::type> launch_in_pool(F const& f);

        template <typename F>
        future<typename result_of<F()>::type> launch_in_pool(F&& f);

        template <typename F>
        future<typename result_of<F()>::type> launch_in_thread(F const& f);

        template <typename F>
        future<typename result_of<F()>::type> launch_in_thread(F&& f);

        class thread_pool;

        class future_canceled;

        class future_result_moved;

        class promise_result_already_set;
    }

3.2 Class template future

    namespace std
    {
        template<typename R>
        class future
        {
        public:
            // copying
            future(const future& other);
            future& operator=(future const& other);

            // functions to retrieve the stored value
            operator R() const;
            R get() const;
            R move();
            bool result_moved() const;

            // functions to check ready state, and wait for ready
            bool ready() const;
            bool has_exception() const;
            bool has_value() const;
            void wait() const;
            bool timed_wait(target_time const& wait_until) const;

            // cancel the task generating the result
            void cancel();
        };
    }

3.2.1 Copying futures

    future(const future& other);

Effects: The only public constructor exposed by future is the copy constructor. The copy constructor will yield a second future which is waiting for the same result as *this. Both the *this and the copy will become ready when the result is available, and both will return the same stored value, or throw the same exception when the result is accessed.

    future& operator=(future const& other);

Effects: Change *this to wait for the same result as other. *this and other will both become ready when the result is available, and both will return the same stored value, or throw the same exception when the result is accessed.

Returns: *this

3.2.2 Functions to retrieve the stored result

    operator R() const;

Returns: get()

    R get() const;

Effects: Blocks until *this is ready, and retrieves the stored result. This function is a cancellation point if ready() would return false on entry. If result_moved would return true on entry, throws an exception of type future_result_moved.

Returns: A copy of the stored value, if any.

Throws: An exception of type future_result_moved as described above, or if there is no stored value, throws a copy of the stored exception. If the copy constructor of the stored value throws an exception, this is propagated to the caller.

    R move();

Effects: Blocks until *this is ready, and retrieves the stored result. This function is a cancellation point if ready() would return false on entry. If result_moved would return true on entry, throws an exception of type future_result_moved.

Returns: A copy of the stored value x, if any, as-if by return std::move(x).

Throws: An exception of type future_result_moved as described above, or if there is no stored value, throws a copy of the stored exception.

    bool result_moved() const;

Returns: true if move has already been called on any of the futures associated with the same state as *this, false otherwise.

3.2.3 Functions to check ready state, and wait for ready

    bool ready() const;

Returns: true if *this has a stored result (whether a value or an exception), false otherwise.

    bool has_exception() const;

Returns: true if *this is ready with a stored exception, false otherwise.

    bool has_value() const;

Returns: true if *this is ready with a stored value, false otherwise.

    void wait() const;

Effects: Blocks until *this is ready. This function is a cancellation point if ready() would return false on entry.

    bool timed_wait(target_time const& wait_until) const;

Effects: Blocks until *this is ready or the specified time is reached. This function is a cancellation point if ready() would return false on entry.

Returns: true if *this is ready, false if the operation timed out.

3.2.4 Cancelling the task generating the result

    void cancel();

Effects: None, if *this is ready on entry to this function. Otherwise attempts to cancel the task generating the result asynchronously. If the cancellation attempt is successful, then *this shall become ready, with a stored exception of type future_canceled. This function shall return immediately, and not wait for *this to become ready.

3.2.5 The specialization future<void>

    namespace std
    {
        template<>
        class future<void>
        {
        public:
            // copying
            future(const future& other);
            future& operator=(future const& other);

            // functions to retrieve the stored value
            void get() const;
            bool result_moved() const;

            // functions to check ready state, and wait for ready
            bool ready() const;
            bool has_exception() const;
            bool has_value() const;
            void wait() const;
            bool timed_wait(target_time const& wait_until) const;

            // cancel the task generating the result
            void cancel();
        };
    }

This specialization is identical to the primary template, except that there is no implicit conversion operator or move function, and get, has_value and result_moved behave as described below.

    void get() const;

Effects: Blocks until *this is ready. This function is a cancellation point if ready() would return false on entry.

Returns: Nothing.

Throws: A copy of the stored exception, if any.

    bool has_value() const;

Returns: true if *this is ready with no stored exception, false otherwise.

3.2.6 The specialization future<R&&>

    namespace std
    {
        template<typename R>
        class future<R&&>
        {
        public:
            // copying
            future(const future& other);
            future& operator=(future const& other);

            // functions to retrieve the stored value
            operator R();
            R get();
            R move();
            bool result_moved() const;

            // functions to check ready state, and wait for ready
            bool ready() const;
            bool has_exception() const;
            bool has_value() const;
            void wait() const;
            bool timed_wait(target_time const& wait_until) const;

            // cancel the task generating the result
            void cancel();
        };
    }

This specialization is identical to the primary template, except that get behaves as described below.

    operator R();
    R get();

Returns: move().

3.3 Class template packaged_task

    namespace std
    {
        template<typename R>
        class packaged_task
        {
        private:
            // packaged_task is not copyable
            packaged_task(packaged_task&); // for exposition only
            packaged_task& operator=(packaged_task&); // for exposition only
        public:
            // construction and destruction
            template<typename F>
            explicit packaged_task(F const& f);
            template<typename F>
            explicit packaged_task(F&& f);
            packaged_task(packaged_task&& other);
            ~packaged_task();

            // assignment
            packaged_task& operator=(packaged_task&& other);
            void swap(packaged_task& other);

            // result retrieval
            future<R> get_future();

            // execution
            void operator()();

            // cancellation
            void cancel();
        };
    }

3.3.1 Construction and destruction of a packaged_task

    template<typename F>
    packaged_task(F const& f);

    template<typename F>
    packaged_task(F&& f);

Effects: Construct a new packaged_task that encapsulates a copy of the callable object f.

    packaged_task(packaged_task&& other);

Effects: Construct a new packaged_task that encapsulates the callable object previously encapsulated in other. All futures associated with other become associated with the newly constructed packaged_task. other no longer has an associated callable object or any associated futures.

    ~packaged_task();

Effects: Destroy the packaged_task and its encapsulated callable object, if any. If the encapsulated callable object has not yet been invoked, all futures associated with *this become ready, with a stored exception of type future_canceled.

3.3.2 Assignment

    packaged_task& operator=(packaged_task&& other);

Effects: As if:

            packaged_task temp(other);
            temp.swap(*this);

Returns: *this

    void swap(packaged_task& other);

Effects: *this encapsulates the callable object previously encapsulated by other; other encapsulates the callable object previously encapsulated by *this. Any futures previously associated with other become associated with *this; any futures previously associated with *this become associated with other.

3.3.3 Retrieving the result from a packaged_task

    future<R> get_future();

Returns: A new future associated with *this. If the encapsulated callable object has already been invoked, then the future is ready, with the stored result the same as-if the future had been constructed prior to the invocation.

3.3.4 Executing the packaged_task

    void operator()();

Effects: Nothing, if *this has no encapsulated callable object, or the encapsulated callable object has already been invoked. Otherwise, invokes the encapsulated callable object. If the invocation of the encapsulated callable object returns normally, then all futures associated with *this become ready with a stored value that is a copy of the value returned. If the invocation of the encapsulated callable object throws an exception, then all futures associated with *this become ready with a stored exception that is a copy of the exception thrown. If the invocation of the encapsulated callable object is cancelled, then the futures associated with *this will become ready, with a stored exception of type future_canceled.

3.3.5 Cancellation of a packaged_task

    void cancel();

Effects: Nothing, if *this has no encapsulated callable object, or the invocation of the encapsulated callable object has already completed. If the invocation of the encapsulated callable object has started on another thread, but not completed, then the task is cancelled as-if by calling cancel() on the std::thread object for the thread in which the invocation of the encapsulated callable object is running. Otherwise, destroys the encapsulated callable object without invoking it. All futures associated with *this become ready, with a stored exception of type future_canceled.

3.4 Function template package_task

    template<typename F>
    packaged_task<typename result_of<F()>::type> package_task(F const& f);

    template<typename F>
    packaged_task<typename result_of<F()>::type> package_task(F&& f);

Effects: Constructs a new packaged_task encapsulating a copy of the supplied callable object f. The template parameter R for the packaged_task is the return type of f().

Returns: The newly constructed packaged_task.

3.5 Class template promise

    namespace std
    {
        template<typename R>
        class promise
        {
        private:
            // promise is not copyable
            promise(promise&); // for exposition only
            promise& operator=(promise&); // for exposition only

        public:
            // Construction and Destruction
            promise();
            promise(promise&& other);
            ~promise();

            // Assignment
            promise& operator=(promise&& other);
            void swap(promise& other);

            // Result retrieval
            future<R> get_future();

            // Updating the state
            void cancel();
            void set_value(R const& r);
            void set_value(R&& r);
            void set_exception(exception_ptr e);        
        };
    }

3.5.1 Construction and Destruction

    promise();

Effects: Create a new promise with no stored value, no stored exception, and no associated futures.

    promise(promise&& other);

Effects: Transfer the state currently associated with other to *this. All futures previously associated with other become associated with *this.

    ~promise();

Effects: If *this currently has neither a stored value, nor a stored exception, all futures associated with *this become ready, with a stored exception of type future_canceled.

3.5.2 Assignment

    promise& operator=(promise&& x);

Effects: As if:

            promise temp(other);
            temp.swap(*this);

Returns: *this

    void swap(promise& other);

Effects: *this encapsulates the stored value or exception (if any) previously encapsulated by other; other encapsulates the stored value or exception (if any) previously encapsulated by *this. Any futures previously associated with other become associated with *this; any futures previously associated with *this become associated with other.

3.5.3 Retrieving the result from a promise

    future<R> get_future();

Returns: A new future associated with *this. If *this already has a stored value or exception, then the future is ready, with the stored value or exception, respectively. Otherwise, the future is not ready.

3.5.4 Updating the state of a promise

    void cancel();

Effects: Nothing, if *this already has a stored value or exception. Otherwise, all futures associated with *this become ready, with a stored exception of type future_canceled.

    void set_value(R const& r);
    void set_value(R&& r);

Effects: If *this already has a stored value or exception, throws an exception of type promise_result_already_set. Otherwise, all futures associated with *this become ready, with a stored value that is a copy of r. If the copy constructor of r throws an exception, the associated futures become ready with a stored exception that is a copy of that exception, as if set by set_exception(current_exception()).

    void set_exception(exception_ptr e);        

Effects: If *this already has a stored value or exception, throws an exception of type promise_result_already_set. Otherwise, all futures associated with *this become ready, with a stored exception that is a copy of the exception referred to by e.

3.5.5 The specialization promise<void>

    namespace std
    {
        template<>
        class promise<void>
        {
        private:
            // promise is not copyable
            promise(promise&); // for exposition only
            promise& operator=(promise&); // for exposition only

        public:
            // Construction and Destruction
            promise();
            promise(promise&& other);
            ~promise();

            // Assignment
            promise& operator=(promise&& other);
            void swap(promise& other);

            // Result retrieval
            future<void> get_future();

            // Updating the state
            void cancel();
            void set_value();
            void set_exception(exception_ptr e);        
        };
    }

This specialization behaves identically to the primary template, except that set_value behaves as described below.

    void set_value();

Effects: If *this already has a stored exception, or set_value has already been called, throws an exception of type promise_result_already_set. Otherwise, all futures associated with *this become ready, with no stored exception.

3.5.6 The specialization promise<R&>

    namespace std
    {
        template<typename R>
        class promise<R&>
        {
        private:
            // promise is not copyable
            promise(promise&); // for exposition only
            promise& operator=(promise&); // for exposition only

        public:
            // Construction and Destruction
            promise();
            promise(promise&& other);
            ~promise();

            // Assignment
            promise& operator=(promise&& other);
            void swap(promise& other);

            // Result retrieval
            future<R&> get_future();

            // Updating the state
            void cancel();
            void set_value(R& r);
            void set_exception(exception_ptr e);        
        };
    }

This specialization behaves identically to the primary template, except that set_value behaves as described below.

    void set_value();

Effects: If *this already has a stored value or exception, throws an exception of type promise_result_already_set. Otherwise, all futures associated with *this become ready, with a stored value that is a reference to r.

3.6 Function template launch_in_pool

    template<typename F>
    future<typename result_of<F()>::type> launch_in_pool(F const& f);

    template<typename F>
    future<typename result_of<F()>::type> launch_in_pool(F&& f);

Effects: Submits a copy of f to the system-supplied thread pool for execution. Constructs a new future associated with the result of invoking the submitted copy of f. When the system thread pool invokes the stored copy of f, the future will become ready, with a copy of the value returned or the execution thrown by the invocation. The template parameter R for the future is the return type of f(). A task running in the system-supplied thread pool can submit a task with launch_in_pool and wait on the future returned without causing deadlock.

Returns: The newly constructed future.

3.7 Function template launch_in_thread

    template<typename F>
    future<typename result_of<F()>::type> launch_in_thread(F const& f);

    template<typename F>
    future<typename result_of<F()>::type> launch_in_thread(F&& f);

Effects: Start a new thread which will execute a copy of f. Constructs a new future associated with the result of invoking the copy of f. When the newly created thread has invoked the stored copy of f, the future will become ready, with a copy of the value returned or the execution thrown by the invocation. The template parameter R for the future is the return type of f().

Returns: The newly constructed future.

3.8 Class template thread_pool

    namespace std
    {
        class thread_pool
        {
        private:
            // thread_pool is not copyable
            thread_pool(thread_pool&); // for exposition only
            thread_pool& operator=(thread_pool&); // for exposition only

        public:
            // Construction and Destruction
            explicit thread_pool(unsigned thread_count);
            ~thread_pool();

            // Submitting tasks for execution
            template<typename F>
            future<typename result_of<F()>::type> submit_task(F const& f);

            template<typename F>
            future<typename result_of<F()>::type> submit_task(F&& f);
        };
    }

3.8.1 Construction and Destruction of a thread_pool

    thread_pool(unsigned thread_count);

Effects: Create a new thread_pool with thread_count threads, and an empty task queue.

    ~thread_pool();

Effects: Cancel all threads in the thread pool. Any tasks on the queue that have not yet been scheduled for execution on a thread in the pool are cancelled: all futures associated with such tasks become ready, with a stored exception of type future_canceled. Blocks until all threads in the pool have finished execution.

3.8.2 Submitting tasks to a thread_pool for execution

    template<typename F>
    future<typename result_of<F()>::type> submit_task(F const& f);

    template<typename F>
    future<typename result_of<F()>::type> submit_task(F&& f);

Effects: Adds a copy of f to the task queue associated with *this for execution. Constructs a new future associated with the result of invoking the submitted copy of f. When the stored copy of f is invoked by one of the threads in the pool, the future will become ready, with a copy of the value returned or the execution thrown by the invocation. The template parameter R for the future is the return type of f().

Returns: The newly constructed future.

Acknowledgements

Thanks to Peter Dimov and Howard Hinnant for their earlier papers, and feedback given on the thoughts presented here.

Thanks to Clark Nelson for allowing me to submit this revised draft after the deadline.