From 9e0ae5398097e56490bcc720fe7ca56d50aec175 Mon Sep 17 00:00:00 2001 From: Serhiy Storchaka Date: Sat, 24 Aug 2013 00:23:38 +0300 Subject: [PATCH] Issue #18757: Improved cross-references in the concurrent package. --- Doc/library/concurrent.futures.rst | 2 +- Doc/library/multiprocessing.rst | 93 +++++++++++++++++------------- Doc/library/queue.rst | 10 ++-- Doc/library/select.rst | 32 +++++----- Doc/library/subprocess.rst | 18 +++--- Doc/library/threading.rst | 13 +++-- 6 files changed, 91 insertions(+), 77 deletions(-) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst index a12316c0e53..c2f92b305aa 100644 --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -40,7 +40,7 @@ Executor Objects .. method:: map(func, *iterables, timeout=None) - Equivalent to ``map(func, *iterables)`` except *func* is executed + Equivalent to :func:`map(func, *iterables) ` except *func* is executed asynchronously and several calls to *func* may be made concurrently. The returned iterator raises a :exc:`TimeoutError` if :meth:`~iterator.__next__` is called and the result isn't available diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 03be9284cf4..3c9c9aae785 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -386,7 +386,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the Unix daemons or services, they are normal processes that will be terminated (and not joined) if non-daemonic processes have exited. - In addition to the :class:`Threading.Thread` API, :class:`Process` objects + In addition to the :class:`threading.Thread` API, :class:`Process` objects also support the following attributes and methods: .. attribute:: pid @@ -405,7 +405,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the The process's authentication key (a byte string). When :mod:`multiprocessing` is initialized the main process is assigned a - random string using :func:`os.random`. + random string using :func:`os.urandom`. When a :class:`Process` object is created, it will inherit the authentication key of its parent process, although this may be changed by @@ -530,7 +530,7 @@ Note that one can also create a shared queue by using a manager object -- see (1) After putting an object on an empty queue there may be an infinitesimal delay before the queue's :meth:`~Queue.empty` method returns :const:`False` and :meth:`~Queue.get_nowait` can - return without raising :exc:`Queue.Empty`. + return without raising :exc:`queue.Empty`. (2) If multiple processes are enqueuing objects, it is possible for the objects to be received at the other end out-of-order. @@ -547,7 +547,8 @@ Note that one can also create a shared queue by using a manager object -- see .. warning:: As mentioned above, if a child process has put items on a queue (and it has - not used :meth:`JoinableQueue.cancel_join_thread`), then that process will + not used :meth:`JoinableQueue.cancel_join_thread + `), then that process will not terminate until all buffered items have been flushed to the pipe. This means that if you try joining that process you may get a deadlock unless @@ -580,7 +581,7 @@ For an example of the usage of queues for interprocess communication see thread is started which transfers objects from a buffer into the pipe. The usual :exc:`queue.Empty` and :exc:`queue.Full` exceptions from the - standard library's :mod:`Queue` module are raised to signal timeouts. + standard library's :mod:`queue` module are raised to signal timeouts. :class:`Queue` implements all the methods of :class:`queue.Queue` except for :meth:`~queue.Queue.task_done` and :meth:`~queue.Queue.join`. @@ -696,7 +697,7 @@ For an example of the usage of queues for interprocess communication see call to :meth:`task_done` tells the queue that the processing on the task is complete. - If a :meth:`~Queue.join` is currently blocking, it will resume when all + If a :meth:`~queue.Queue.join` is currently blocking, it will resume when all items have been processed (meaning that a :meth:`task_done` call was received for every item that had been :meth:`~Queue.put` into the queue). @@ -712,7 +713,7 @@ For an example of the usage of queues for interprocess communication see queue. The count goes down whenever a consumer calls :meth:`task_done` to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, - :meth:`~Queue.join` unblocks. + :meth:`~queue.Queue.join` unblocks. Miscellaneous @@ -874,8 +875,8 @@ Connection objects are usually created using :func:`Pipe` -- see also .. versionadded:: 3.3 Connection objects now support the context manager protocol -- see - :ref:`typecontextmanager`. :meth:`__enter__` returns the - connection object, and :meth:`__exit__` calls :meth:`close`. + :ref:`typecontextmanager`. :meth:`~contextmanager.__enter__` returns the + connection object, and :meth:`~contextmanager.__exit__` calls :meth:`close`. For example: @@ -948,7 +949,7 @@ object -- see :ref:`multiprocessing-managers`. object from :mod:`multiprocessing`. .. versionchanged:: 3.3 - The :meth:`wait_for` method was added. + The :meth:`~threading.Condition.wait_for` method was added. .. class:: Event() @@ -1095,8 +1096,9 @@ processes. array. If *lock* is ``True`` (the default) then a new lock object is created to - synchronize access to the value. If *lock* is a :class:`Lock` or - :class:`RLock` object then that will be used to synchronize access to the + synchronize access to the value. If *lock* is a + :class:`~multiprocessing.Lock` or :class:`~multiprocessing.RLock` object + then that will be used to synchronize access to the value. If *lock* is ``False`` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe". @@ -1110,8 +1112,8 @@ processes. object. If *lock* is ``True`` (the default) then a new lock object is created to - synchronize access to the value. If *lock* is a :class:`Lock` or - :class:`RLock` object then that will be used to synchronize access to the + synchronize access to the value. If *lock* is a :class:`~multiprocessing.Lock` or + :class:`~multiprocessing.RLock` object then that will be used to synchronize access to the value. If *lock* is ``False`` then access to the returned object will not be automatically protected by a lock, so it will not necessarily be "process-safe". @@ -1296,8 +1298,8 @@ their parent process exits. The manager classes are defined in the :attr:`proxytype._exposed_` is used instead if it exists.) In the case where no exposed list is specified, all "public methods" of the shared object will be accessible. (Here a "public method" means any attribute - which has a :meth:`__call__` method and whose name does not begin with - ``'_'``.) + which has a :meth:`~object.__call__` method and whose name does not begin + with ``'_'``.) *method_to_typeid* is a mapping used to specify the return type of those exposed methods which should return a proxy. It maps method names to @@ -1318,11 +1320,11 @@ their parent process exits. The manager classes are defined in the .. versionchanged:: 3.3 Manager objects support the context manager protocol -- see - :ref:`typecontextmanager`. :meth:`__enter__` starts the server - process (if it has not already started) and then returns the - manager object. :meth:`__exit__` calls :meth:`shutdown`. + :ref:`typecontextmanager`. :meth:`~contextmanager.__enter__` starts the + server process (if it has not already started) and then returns the + manager object. :meth:`~contextmanager.__exit__` calls :meth:`shutdown`. - In previous versions :meth:`__enter__` did not start the + In previous versions :meth:`~contextmanager.__enter__` did not start the manager's server process if it was not already started. .. class:: SyncManager @@ -1354,7 +1356,7 @@ their parent process exits. The manager classes are defined in the :class:`threading.Lock` or :class:`threading.RLock` object. .. versionchanged:: 3.3 - The :meth:`wait_for` method was added. + The :meth:`~threading.Condition.wait_for` method was added. .. method:: Event() @@ -1798,8 +1800,8 @@ with the :class:`Pool` class. .. versionadded:: 3.3 Pool objects now support the context manager protocol -- see - :ref:`typecontextmanager`. :meth:`__enter__` returns the pool - object, and :meth:`__exit__` calls :meth:`terminate`. + :ref:`typecontextmanager`. :meth:`~contextmanager.__enter__` returns the + pool object, and :meth:~contextmanager.`__exit__` calls :meth:`terminate`. .. class:: AsyncResult @@ -1860,7 +1862,8 @@ Listeners and Clients :synopsis: API for dealing with sockets. Usually message passing between processes is done using queues or by using -:class:`Connection` objects returned by :func:`Pipe`. +:class:`~multiprocessing.Connection` objects returned by +:func:`~multiprocessing.Pipe`. However, the :mod:`multiprocessing.connection` module allows some extra flexibility. It basically gives a high level message oriented API for dealing @@ -1928,7 +1931,8 @@ multiple connections at the same time. private temporary directory created using :func:`tempfile.mkstemp`. If the listener object uses a socket then *backlog* (1 by default) is passed - to the :meth:`listen` method of the socket once it has been bound. + to the :meth:`~socket.socket.listen` method of the socket once it has been + bound. If *authenticate* is ``True`` (``False`` by default) or *authkey* is not ``None`` then digest authentication is used. @@ -1946,8 +1950,8 @@ multiple connections at the same time. .. method:: accept() Accept a connection on the bound socket or named pipe of the listener - object and return a :class:`Connection` object. If authentication is - attempted and fails, then + object and return a :class:`~multiprocessing.Connection` object. If + authentication is attempted and fails, then :exc:`~multiprocessing.AuthenticationError` is raised. .. method:: close() @@ -1969,8 +1973,8 @@ multiple connections at the same time. .. versionadded:: 3.3 Listener objects now support the context manager protocol -- see - :ref:`typecontextmanager`. :meth:`__enter__` returns the - listener object, and :meth:`__exit__` calls :meth:`close`. + :ref:`typecontextmanager`. :meth:`~contextmanager.__enter__` returns the + listener object, and :meth:~contextmanager.`__exit__` calls :meth:`close`. .. function:: wait(object_list, timeout=None) @@ -2106,7 +2110,8 @@ an ``'AF_PIPE'`` address rather than an ``'AF_UNIX'`` address. Authentication keys ~~~~~~~~~~~~~~~~~~~ -When one uses :meth:`Connection.recv`, the data received is automatically +When one uses :meth:`Connection.recv `, the +data received is automatically unpickled. Unfortunately unpickling data from an untrusted source is a security risk. Therefore :class:`Listener` and :func:`Client` use the :mod:`hmac` module to provide digest authentication. @@ -2256,9 +2261,10 @@ Joining zombie processes On Unix when a process finishes but has not been joined it becomes a zombie. There should never be very many because each time a new process starts (or - :func:`active_children` is called) all completed processes which have not - yet been joined will be joined. Also calling a finished process's - :meth:`Process.is_alive` will join the process. Even so it is probably good + :func:`~multiprocessing.active_children` is called) all completed processes + which have not yet been joined will be joined. Also calling a finished + process's :meth:`Process.is_alive ` will + join the process. Even so it is probably good practice to explicitly join all the processes that you start. Better to inherit than pickle/unpickle @@ -2271,20 +2277,23 @@ Better to inherit than pickle/unpickle Avoid terminating processes - Using the :meth:`Process.terminate` method to stop a process is liable to + Using the :meth:`Process.terminate ` + method to stop a process is liable to cause any shared resources (such as locks, semaphores, pipes and queues) currently being used by the process to become broken or unavailable to other processes. Therefore it is probably best to only consider using - :meth:`Process.terminate` on processes which never use any shared resources. + :meth:`Process.terminate ` on processes + which never use any shared resources. Joining processes that use queues Bear in mind that a process that has put items in a queue will wait before terminating until all the buffered items are fed by the "feeder" thread to the underlying pipe. (The child process can call the - :meth:`Queue.cancel_join_thread` method of the queue to avoid this behaviour.) + :meth:`Queue.cancel_join_thread ` + method of the queue to avoid this behaviour.) This means that whenever you use a queue you need to make sure that all items which have been put on the queue will eventually be removed before the @@ -2361,7 +2370,7 @@ Beware of replacing :data:`sys.stdin` with a "file like object" resulting in a bad file descriptor error, but introduces a potential danger to applications which replace :func:`sys.stdin` with a "file-like object" with output buffering. This danger is that if multiple processes call - :func:`close()` on this file-like object, it could result in the same + :meth:`~io.IOBase.close()` on this file-like object, it could result in the same data being flushed to the object multiple times, resulting in corruption. If you write a file-like object and implement your own caching, you can @@ -2390,14 +2399,16 @@ More picklability as the ``target`` argument on Windows --- just define a function and use that instead. - Also, if you subclass :class:`Process` then make sure that instances will be - picklable when the :meth:`Process.start` method is called. + Also, if you subclass :class:`~multiprocessing.Process` then make sure that + instances will be picklable when the :meth:`Process.start + ` method is called. Global variables Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value - in the parent process at the time that :meth:`Process.start` was called. + in the parent process at the time that :meth:`Process.start + ` was called. However, global variables which are just module level constants cause no problems. @@ -2453,7 +2464,7 @@ Demonstration of how to create and use customized managers and proxies: :language: python3 -Using :class:`Pool`: +Using :class:`~multiprocessing.pool.Pool`: .. literalinclude:: ../includes/mp_pool.py :language: python3 diff --git a/Doc/library/queue.rst b/Doc/library/queue.rst index bc35dfe40e1..680d6900529 100644 --- a/Doc/library/queue.rst +++ b/Doc/library/queue.rst @@ -54,13 +54,15 @@ The :mod:`queue` module defines the following classes and exceptions: .. exception:: Empty - Exception raised when non-blocking :meth:`get` (or :meth:`get_nowait`) is called + Exception raised when non-blocking :meth:`~Queue.get` (or + :meth:`~Queue.get_nowait`) is called on a :class:`Queue` object which is empty. .. exception:: Full - Exception raised when non-blocking :meth:`put` (or :meth:`put_nowait`) is called + Exception raised when non-blocking :meth:`~Queue.put` (or + :meth:`~Queue.put_nowait`) is called on a :class:`Queue` object which is full. @@ -181,6 +183,6 @@ Example of how to wait for enqueued tasks to be completed:: context. :class:`collections.deque` is an alternative implementation of unbounded - queues with fast atomic :func:`append` and :func:`popleft` operations that - do not require locking. + queues with fast atomic :meth:`~collections.deque.append` and + :meth:`~collections.deque.popleft` operations that do not require locking. diff --git a/Doc/library/select.rst b/Doc/library/select.rst index 4e60f4ad883..b1fd3a83df7 100644 --- a/Doc/library/select.rst +++ b/Doc/library/select.rst @@ -78,7 +78,7 @@ The module defines the following: This is a straightforward interface to the Unix :c:func:`select` system call. The first three arguments are sequences of 'waitable objects': either integers representing file descriptors or objects with a parameterless method - named :meth:`fileno` returning such an integer: + named :meth:`~io.IOBase.fileno` returning such an integer: * *rlist*: wait until ready for reading * *wlist*: wait until ready for writing @@ -104,8 +104,8 @@ The module defines the following: objects ` (e.g. ``sys.stdin``, or objects returned by :func:`open` or :func:`os.popen`), socket objects returned by :func:`socket.socket`. You may also define a :dfn:`wrapper` class yourself, - as long as it has an appropriate :meth:`fileno` method (that really returns - a file descriptor, not just a random integer). + as long as it has an appropriate :meth:`~io.IOBase.fileno` method (that + really returns a file descriptor, not just a random integer). .. note:: @@ -119,7 +119,7 @@ The module defines the following: .. attribute:: PIPE_BUF The minimum number of bytes which can be written without blocking to a pipe - when the pipe has been reported as ready for writing by :func:`select`, + when the pipe has been reported as ready for writing by :func:`~select.select`, :func:`poll` or another interface in this module. This doesn't apply to other kind of file-like objects such as sockets. @@ -147,10 +147,10 @@ object. .. method:: devpoll.register(fd[, eventmask]) Register a file descriptor with the polling object. Future calls to the - :meth:`poll` method will then check whether the file descriptor has any pending - I/O events. *fd* can be either an integer, or an object with a :meth:`fileno` - method that returns an integer. File objects implement :meth:`fileno`, so they - can also be used as the argument. + :meth:`poll` method will then check whether the file descriptor has any + pending I/O events. *fd* can be either an integer, or an object with a + :meth:`~io.IOBase.fileno` method that returns an integer. File objects + implement :meth:`!fileno`, so they can also be used as the argument. *eventmask* is an optional bitmask describing the type of events you want to check for. The constants are the same that with :c:func:`poll` @@ -176,7 +176,7 @@ object. Remove a file descriptor being tracked by a polling object. Just like the :meth:`register` method, *fd* can be an integer or an object with a - :meth:`fileno` method that returns an integer. + :meth:`~io.IOBase.fileno` method that returns an integer. Attempting to remove a file descriptor that was never registered is safely ignored. @@ -288,10 +288,10 @@ linearly scanned again. :c:func:`select` is O(highest file descriptor), while .. method:: poll.register(fd[, eventmask]) Register a file descriptor with the polling object. Future calls to the - :meth:`poll` method will then check whether the file descriptor has any pending - I/O events. *fd* can be either an integer, or an object with a :meth:`fileno` - method that returns an integer. File objects implement :meth:`fileno`, so they - can also be used as the argument. + :meth:`poll` method will then check whether the file descriptor has any + pending I/O events. *fd* can be either an integer, or an object with a + :meth:`~io.IOBase.fileno` method that returns an integer. File objects + implement :meth:`!fileno`, so they can also be used as the argument. *eventmask* is an optional bitmask describing the type of events you want to check for, and can be a combination of the constants :const:`POLLIN`, @@ -330,7 +330,7 @@ linearly scanned again. :c:func:`select` is O(highest file descriptor), while Remove a file descriptor being tracked by a polling object. Just like the :meth:`register` method, *fd* can be an integer or an object with a - :meth:`fileno` method that returns an integer. + :meth:`~io.IOBase.fileno` method that returns an integer. Attempting to remove a file descriptor that was never registered causes a :exc:`KeyError` exception to be raised. @@ -390,8 +390,8 @@ http://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2 Value used to identify the event. The interpretation depends on the filter but it's usually the file descriptor. In the constructor ident can either - be an int or an object with a fileno() function. kevent stores the integer - internally. + be an int or an object with a :meth:`~io.IOBase.fileno` method. kevent + stores the integer internally. .. attribute:: kevent.filter diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 3a30821aa8e..f115634daf0 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -76,7 +76,7 @@ use cases, the underlying :class:`Popen` interface can be used directly. Run command with arguments. Wait for command to complete. If the return code was zero then return, otherwise raise :exc:`CalledProcessError`. The :exc:`CalledProcessError` object will have the return code in the - :attr:`returncode` attribute. + :attr:`~CalledProcessError.returncode` attribute. The arguments shown above are merely the most common ones, described below in :ref:`frequently-used-arguments` (hence the use of keyword-only notation @@ -122,8 +122,8 @@ use cases, the underlying :class:`Popen` interface can be used directly. If the return code was non-zero it raises a :exc:`CalledProcessError`. The :exc:`CalledProcessError` object will have the return code in the - :attr:`returncode` attribute and any output in the :attr:`output` - attribute. + :attr:`~CalledProcessError.returncode` attribute and any output in the + :attr:`~CalledProcessError.output` attribute. The arguments shown above are merely the most common ones, described below in :ref:`frequently-used-arguments` (hence the use of keyword-only notation @@ -607,14 +607,14 @@ Instances of the :class:`Popen` class have the following methods: .. method:: Popen.poll() - Check if child process has terminated. Set and return :attr:`returncode` - attribute. + Check if child process has terminated. Set and return + :attr:`~Popen.returncode` attribute. .. method:: Popen.wait(timeout=None) - Wait for child process to terminate. Set and return :attr:`returncode` - attribute. + Wait for child process to terminate. Set and return + :attr:`~Popen.returncode` attribute. If the process does not terminate after *timeout* seconds, raise a :exc:`TimeoutExpired` exception. It is safe to catch this exception and @@ -860,8 +860,8 @@ In this section, "a becomes b" means that b can be used as a replacement for a. In addition, the replacements using :func:`check_output` will fail with a :exc:`CalledProcessError` if the requested operation produces a non-zero - return code. The output is still available as the ``output`` attribute of - the raised exception. + return code. The output is still available as the + :attr:`~CalledProcessError.output` attribute of the raised exception. In the following examples, we assume that the relevant functions have already been imported from the :mod:`subprocess` module. diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index 1bcf0a2ea51..5eb00694ad4 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -63,7 +63,7 @@ This module defines the following functions: Set a trace function for all threads started from the :mod:`threading` module. The *func* will be passed to :func:`sys.settrace` for each thread, before its - :meth:`run` method is called. + :meth:`~Thread.run` method is called. .. function:: setprofile(func) @@ -72,7 +72,7 @@ This module defines the following functions: Set a profile function for all threads started from the :mod:`threading` module. The *func* will be passed to :func:`sys.setprofile` for each thread, before its - :meth:`run` method is called. + :meth:`~Thread.run` method is called. .. function:: stack_size([size]) @@ -825,10 +825,11 @@ This class represents an action that should be run only after a certain amount of time has passed --- a timer. :class:`Timer` is a subclass of :class:`Thread` and as such also functions as an example of creating custom threads. -Timers are started, as with threads, by calling their :meth:`start` method. The -timer can be stopped (before its action has begun) by calling the :meth:`cancel` -method. The interval the timer will wait before executing its action may not be -exactly the same as the interval specified by the user. +Timers are started, as with threads, by calling their :meth:`~Timer.start` +method. The timer can be stopped (before its action has begun) by calling the +:meth:`~Timer.cancel` method. The interval the timer will wait before +executing its action may not be exactly the same as the interval specified by +the user. For example::