mirrors/cpython
0
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-01-26 19:13:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

bpo-33649: First asyncio docs improvement pass (GH-9142)

Browse Source 
Rewritten/updated sections:

* Event Loop APIs
* Transports & Protocols
* Streams
* Exceptions
* Policies
* Queues
* Subprocesses
* Platforms
This commit is contained in:
Yury Selivanov 2018-09-11 09:54:40 -07:00 committed by GitHub
parent 735171e334
commit 7c7605ff11
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 2370 additions and 1804 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
Doc/library/asyncio-dev.rst
View File

@ -23,7 +23,7 @@ To enable all debug checks for an application:
* Enable the asyncio debug mode globally by setting the environment variable
:envvar:`PYTHONASYNCIODEBUG` to ``1``, using ``-X dev`` command line option
(see the :option:`-X` option), or by calling
:meth:`AbstractEventLoop.set_debug`.
:meth:`loop.set_debug`.
* Set the log level of the :ref:`asyncio logger <asyncio-logger>` to
:py:data:`logging.DEBUG`. For example, call
``logging.basicConfig(level=logging.DEBUG)`` at startup.
@ -35,11 +35,11 @@ Examples debug checks:
* Log :ref:`coroutines defined but never "yielded from"
<asyncio-coroutine-not-scheduled>`
* :meth:`~AbstractEventLoop.call_soon` and :meth:`~AbstractEventLoop.call_at` methods
* :meth:`loop.call_soon` and :meth:`loop.call_at` methods
raise an exception if they are called from the wrong thread.
* Log the execution time of the selector
* Log callbacks taking more than 100 ms to be executed. The
:attr:`AbstractEventLoop.slow_callback_duration` attribute is the minimum
:attr:`loop.slow_callback_duration` attribute is the minimum
duration in seconds of "slow" callbacks.
* :exc:`ResourceWarning` warnings are emitted when transports and event loops
are :ref:`not closed explicitly <asyncio-close-transports>`.
@ -51,7 +51,7 @@ Examples debug checks:
.. seealso::
The :meth:`AbstractEventLoop.set_debug` method and the :ref:`asyncio logger
The :meth:`loop.set_debug` method and the :ref:`asyncio logger
<asyncio-logger>`.
@ -75,7 +75,7 @@ For example, write::
Don't schedule directly a call to the :meth:`~Future.set_result` or the
:meth:`~Future.set_exception` method of a future with
:meth:`AbstractEventLoop.call_soon`: the future can be cancelled before its method
:meth:`loop.call_soon`: the future can be cancelled before its method
is called.
If you wait for a future, you should check early if the future was cancelled to
@ -96,13 +96,14 @@ The :func:`shield` function can also be used to ignore cancellation.
Concurrency and multithreading
------------------------------
An event loop runs in a thread and executes all callbacks and tasks in the same
thread. While a task is running in the event loop, no other task is running in
the same thread. But when the task uses ``await``, the task is suspended
and the event loop executes the next task.
An event loop runs in a thread (typically the main thread) and executes
all callbacks and tasks in its thread. While a task is running in the
event loop, no other task is running in the same thread. When a task
executes an ``await`` expression, the task gets suspended and the
event loop executes the next task.
To schedule a callback from a different thread, the
:meth:`AbstractEventLoop.call_soon_threadsafe` method should be used. Example::
:meth:`loop.call_soon_threadsafe` method should be used. Example::
loop.call_soon_threadsafe(callback, *args)
@ -122,7 +123,7 @@ To schedule a coroutine object from a different thread, the
future = asyncio.run_coroutine_threadsafe(coro_func(), loop)
result = future.result(timeout) # Wait for the result with a timeout
The :meth:`AbstractEventLoop.run_in_executor` method can be used with a thread pool
The :meth:`loop.run_in_executor` method can be used with a thread pool
executor to execute a callback in different thread to not block the thread of
the event loop.
@ -151,7 +152,7 @@ APIs like :ref:`protocols <asyncio-protocol>`.
An executor can be used to run a task in a different thread or even in a
different process, to not block the thread of the event loop. See the
:meth:`AbstractEventLoop.run_in_executor` method.
:meth:`loop.run_in_executor` method.
.. seealso::
@ -182,7 +183,7 @@ Detect coroutine objects never scheduled
----------------------------------------
When a coroutine function is called and its result is not passed to
:func:`ensure_future` or to the :meth:`AbstractEventLoop.create_task` method,
:func:`ensure_future` or to the :meth:`loop.create_task` method,
the execution of the coroutine object will never be scheduled which is
probably a bug. :ref:`Enable the debug mode of asyncio <asyncio-debug-mode>`
to :ref:`log a warning <asyncio-logger>` to detect it.
@ -204,7 +205,7 @@ Output in debug mode::
test()
The fix is to call the :func:`ensure_future` function or the
:meth:`AbstractEventLoop.create_task` method with the coroutine object.
:meth:`loop.create_task` method with the coroutine object.
.. seealso::
@ -279,7 +280,7 @@ coroutine in another coroutine and use classic try/except::
loop.run_forever()
loop.close()
Another option is to use the :meth:`AbstractEventLoop.run_until_complete`
Another option is to use the :meth:`loop.run_until_complete`
function::
task = asyncio.ensure_future(bug())

1210
Doc/library/asyncio-eventloop.rst
View File

File diff suppressed because it is too large Load Diff

244
Doc/library/asyncio-eventloops.rst
View File

@ -1,244 +0,0 @@
.. currentmodule:: asyncio
Event loops
===========
**Source code:** :source:`Lib/asyncio/events.py`
Event loop functions
--------------------
The following functions are convenient shortcuts to accessing the methods of the
global policy. Note that this provides access to the default policy, unless an
alternative policy was set by calling :func:`set_event_loop_policy` earlier in
the execution of the process.
.. function:: get_event_loop()
Equivalent to calling ``get_event_loop_policy().get_event_loop()``.
.. function:: set_event_loop(loop)
Equivalent to calling ``get_event_loop_policy().set_event_loop(loop)``.
.. function:: new_event_loop()
Equivalent to calling ``get_event_loop_policy().new_event_loop()``.
.. function:: get_running_loop()
Return the running event loop in the current OS thread. If there
is no running event loop a :exc:`RuntimeError` is raised.
.. versionadded:: 3.7
.. _asyncio-event-loops:
Available event loops
---------------------
asyncio currently provides two implementations of event loops:
:class:`SelectorEventLoop` and :class:`ProactorEventLoop`.
.. class:: SelectorEventLoop
Event loop based on the :mod:`selectors` module. Subclass of
:class:`AbstractEventLoop`.
Use the most efficient selector available on the platform.
On Windows, only sockets are supported (ex: pipes are not supported):
see the `MSDN documentation of select
<https://msdn.microsoft.com/en-us/library/windows/desktop/ms740141%28v=vs.85%29.aspx>`_.
.. class:: ProactorEventLoop
Proactor event loop for Windows using "I/O Completion Ports" aka IOCP.
Subclass of :class:`AbstractEventLoop`.
Availability: Windows.
.. seealso::
`MSDN documentation on I/O Completion Ports
<https://msdn.microsoft.com/en-us/library/windows/desktop/aa365198%28v=vs.85%29.aspx>`_.
Example to use a :class:`ProactorEventLoop` on Windows::
import asyncio, sys
if sys.platform == 'win32':
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
.. _asyncio-platform-support:
Platform support
----------------
The :mod:`asyncio` module has been designed to be portable, but each platform
still has subtle differences and may not support all :mod:`asyncio` features.
Windows
^^^^^^^
Common limits of Windows event loops:
- :meth:`~AbstractEventLoop.create_unix_connection` and
:meth:`~AbstractEventLoop.create_unix_server` are not supported: the socket
family :data:`socket.AF_UNIX` is specific to UNIX
- :meth:`~AbstractEventLoop.add_signal_handler` and
:meth:`~AbstractEventLoop.remove_signal_handler` are not supported
- :meth:`EventLoopPolicy.set_child_watcher` is not supported.
:class:`ProactorEventLoop` supports subprocesses. It has only one
implementation to watch child processes, there is no need to configure it.
:class:`SelectorEventLoop` specific limits:
- :class:`~selectors.SelectSelector` is used which only supports sockets
and is limited to 512 sockets.
- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` only
accept file descriptors of sockets
- Pipes are not supported
(ex: :meth:`~AbstractEventLoop.connect_read_pipe`,
:meth:`~AbstractEventLoop.connect_write_pipe`)
- :ref:`Subprocesses <asyncio-subprocess>` are not supported
(ex: :meth:`~AbstractEventLoop.subprocess_exec`,
:meth:`~AbstractEventLoop.subprocess_shell`)
:class:`ProactorEventLoop` specific limits:
- :meth:`~AbstractEventLoop.create_datagram_endpoint` (UDP) is not supported
- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` are
not supported
The resolution of the monotonic clock on Windows is usually around 15.6 msec.
The best resolution is 0.5 msec. The resolution depends on the hardware
(availability of `HPET
<https://en.wikipedia.org/wiki/High_Precision_Event_Timer>`_) and on the Windows
configuration. See :ref:`asyncio delayed calls <asyncio-delayed-calls>`.
.. versionchanged:: 3.5
:class:`ProactorEventLoop` now supports SSL.
Mac OS X
^^^^^^^^
Character devices like PTY are only well supported since Mavericks (Mac OS
10.9). They are not supported at all on Mac OS 10.5 and older.
On Mac OS 10.6, 10.7 and 10.8, the default event loop is
:class:`SelectorEventLoop` which uses :class:`selectors.KqueueSelector`.
:class:`selectors.KqueueSelector` does not support character devices on these
versions. The :class:`SelectorEventLoop` can be used with
:class:`~selectors.SelectSelector` or :class:`~selectors.PollSelector` to
support character devices on these versions of Mac OS X. Example::
import asyncio
import selectors
selector = selectors.SelectSelector()
loop = asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop(loop)
Event loop policies and the default policy
------------------------------------------
Event loop management is abstracted with a *policy* pattern, to provide maximal
flexibility for custom platforms and frameworks. Throughout the execution of a
process, a single global policy object manages the event loops available to the
process based on the calling context. A policy is an object implementing the
:class:`AbstractEventLoopPolicy` interface.
For most users of :mod:`asyncio`, policies never have to be dealt with
explicitly, since the default global policy is sufficient (see below).
The module-level functions
:func:`get_event_loop` and :func:`set_event_loop` provide convenient access to
event loops managed by the default policy.
Event loop policy interface
---------------------------
An event loop policy must implement the following interface:
.. class:: AbstractEventLoopPolicy
Event loop policy.
.. method:: get_event_loop()
Get the event loop for the current context.
Returns an event loop object implementing the :class:`AbstractEventLoop`
interface. In case called from coroutine, it returns the currently
running event loop.
Raises an exception in case no event loop has been set for the current
context and the current policy does not specify to create one. It must
never return ``None``.
.. versionchanged:: 3.6
.. method:: set_event_loop(loop)
Set the event loop for the current context to *loop*.
.. method:: new_event_loop()
Create and return a new event loop object according to this policy's
rules.
If there's need to set this loop as the event loop for the current
context, :meth:`set_event_loop` must be called explicitly.
The default policy defines context as the current thread, and manages an event
loop per thread that interacts with :mod:`asyncio`. An exception to this rule
happens when :meth:`~AbstractEventLoopPolicy.get_event_loop` is called from a
running future/coroutine, in which case it will return the current loop
running that future/coroutine.
If the current thread doesn't already have an event loop associated with it,
the default policy's :meth:`~AbstractEventLoopPolicy.get_event_loop` method
creates one when called from the main thread, but raises :exc:`RuntimeError`
otherwise.
Access to the global loop policy
--------------------------------
.. function:: get_event_loop_policy()
Get the current event loop policy.
.. function:: set_event_loop_policy(policy)
Set the current event loop policy. If *policy* is ``None``, the default
policy is restored.
Customizing the event loop policy
---------------------------------
To implement a new event loop policy, it is recommended you subclass the
concrete default event loop policy :class:`DefaultEventLoopPolicy`
and override the methods for which you want to change behavior, for example::
class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy):
def get_event_loop(self):
"""Get the event loop.
This may be None or an instance of EventLoop.
"""
loop = super().get_event_loop()
# Do something with loop ...
return loop
asyncio.set_event_loop_policy(MyEventLoopPolicy())

88
Doc/library/asyncio-exceptions.rst Normal file
View File

@ -0,0 +1,88 @@
.. currentmodule:: asyncio
==========
Exceptions
==========
.. exception:: TimeoutError
The operation has exceeded the given deadline.
.. note::
This exception is different from the builtin :exc:`TimeoutError`
exception.
.. exception:: CancelledError
The operation has been cancelled.
This exception can be caught to perform custom operations on
when asyncio Tasks are cancelled. In almost all situations the
exception must always be re-raised.
.. note::
This exception is a subclass of :exc:`Exception`, so it can be
accidentally suppressed by ``try..except`` block::
try:
await operation
except Exception:
# The cancellation is broken because the *except* block
# suppresses the CancelledError exception.
log.log('an error has occurred')
Instead, the following pattern should be used::
try:
await operation
except asyncio.CancelledError:
raise
except Exception:
log.log('an error has occurred')
.. exception:: InvalidStateError
Invalid internal state of :class:`Task` or :class:`Future`.
Can be raised in situations like setting a result value for a
*Future* object that already has a result value set.
.. exception:: SendfileNotAvailableError
The "sendfile" syscall for is not available for the given
socket or file type.
A subclass of :exc:`RuntimeError`.
.. exception:: IncompleteReadError
Incomplete read error.
Raised by :ref:`asyncio streams <asyncio-streams>` APIs.
This exception is a subclass of :exc:`EOFError`.
.. attribute:: expected
Total number (:class:`int`) of expected bytes.
.. attribute:: partial
Read :class:`bytes` string before the end of stream was reached.
.. exception:: LimitOverrunError
Reached the buffer limit while looking for a separator.
Raised by :ref:`asyncio streams <asyncio-streams>` APIs.
.. attribute:: consumed
Total number of to be consumed bytes.

105
Doc/library/asyncio-platforms.rst Normal file
View File

@ -0,0 +1,105 @@
.. currentmodule:: asyncio
.. _asyncio-platform-support:
=================
Platforms Support
=================
The :mod:`asyncio` module has been designed to be portable,
but some platforms have subtle differences and limitations.
All Platforms
=============
* :meth:`loop.add_reader` and :meth:`loop.add_writer`
cannot be used to monitor file IO.
Windows
=======
All event loops on Windows do not support the following methods:
* :meth:`loop.create_unix_connection` and
:meth:`loop.create_unix_server` are not supported.
The :data:`socket.AF_UNIX` socket family is specific to UNIX/
* :meth:`loop.add_signal_handler` and
:meth:`loop.remove_signal_handler` are not supported.
:class:`SelectorEventLoop` has the following limitations:
* :class:`~selectors.SelectSelector` is used to wait on socket events:
it supports sockets and is limited to 512 sockets.
* :meth:`loop.add_reader` and :meth:`loop.add_writer` only accept
socket handles (e.g. pipe file descriptors are not supported).
* Pipes are not supported, so the :meth:`loop.connect_read_pipe`
and :meth:`loop.connect_write_pipe` methods are not implemented.
* :ref:`Subprocesses <asyncio-subprocess>` are not supported, i.e.
:meth:`loop.subprocess_exec` and :meth:`loop.subprocess_shell`
methods are not implemented.
:class:`ProactorEventLoop` has the following limitations:
* The :meth:`loop.create_datagram_endpoint` method
is not supported.
* The :meth:`loop.add_reader` and :meth:`loop.add_writer`
methods are not supported.
The resolution of the monotonic clock on Windows is usually around 15.6
msec. The best resolution is 0.5 msec. The resolution depends on the
hardware (availability of `HPET
<https://en.wikipedia.org/wiki/High_Precision_Event_Timer>`_) and on the
Windows configuration.
.. _asyncio-windows-subprocess:
Subprocess Support on Windows
-----------------------------
:class:`SelectorEventLoop` on Windows does not support subproceses,
so :class:`ProactorEventLoop` should be used instead::
import asyncio
asyncio.set_event_loop_policy(
asyncio.WindowsProactorEventLoopPolicy())
asyncio.run(your_code())
The :meth:`policy.set_child_watcher()
<AbstractEventLoopPolicy.set_child_watcher>` function is also
not supported, as :class:`ProactorEventLoop` has a different mechanism
to watch child processes.
macOS
=====
Modern macOS versions are fully supported.
.. rubric:: macOS <= 10.8
On macOS 10.6, 10.7 and 10.8, the default event loop
uses :class:`selectors.KqueueSelector`, which does not support
character devices on these versions. The :class:`SelectorEventLoop`
can be manually configured to use :class:`~selectors.SelectSelector`
or :class:`~selectors.PollSelector` to support character devices on
these older versions of macOS. Example::
import asyncio
import selectors
selector = selectors.SelectSelector()
loop = asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop(loop)

222
Doc/library/asyncio-policy.rst Normal file
View File

@ -0,0 +1,222 @@
.. currentmodule:: asyncio
.. _asyncio-policies:
========
Policies
========
An event loop policy, a global per-process object, controls
management of the event loop. Each event loop has a default
policy, which can be changed and customized using the API.
A policy defines the notion of context and manages a
separate event loop per context. The default policy
defines context to be the current thread.
By using a custom event loop policy, the behavior of
:func:`get_event_loop`, :func:`set_event_loop`, and
:func:`new_event_loop` functions can be customized.
Policy objects should implement the APIs defined
in the abstract base class :class:`AbstractEventLoopPolicy`.
Access the Policy
=================
The following functions can be used to get and set the policy
for the current process:
.. function:: get_event_loop_policy()
Return the current process-wide policy.
.. function:: set_event_loop_policy(policy)
Set the current process-wide policy to *policy*.
If *policy* is set to ``None``, the default policy is restored.
Policy Objects
==============
The abstract event loop policy base class is defined as follows:
.. class:: AbstractEventLoopPolicy
An abstract base class for asyncio policies.
.. method:: get_event_loop()
Get the event loop for the current context.
Return an event loop object implementing the
:class:`AbstractEventLoop` interface.
This method should never return ``None``.
.. versionchanged:: 3.6
.. method:: set_event_loop(loop)
Set the event loop for the current context to *loop*.
.. method:: new_event_loop()
Create and return a new event loop object.
This method should never return ``None``.
.. method:: get_child_watcher()
Get a child process watcher object.
Return a watcher object implementing the
:class:`AbstractChildWatcher` interface.
This function is Unix specific.
.. method:: set_child_watcher(watcher)
Get the current child process watcher to *watcher*.
This function is Unix specific.
asyncio ships with the following built-in policies:
.. class:: DefaultEventLoopPolicy
The default asyncio policy. Uses :class:`SelectorEventLoop`
on both Unix and Windows platforms.
There is no need to install the default policy manually; asyncio
is configured to use it automatically.
.. class:: WindowsProactorEventLoopPolicy
An alternative event loop policy that uses the
:class:`ProactorEventLoop` event loop implementation.
Availability: Windows.
Process Watchers
================
A process watcher allows customization of how an event loop monitors
child processes on Unix. Specifically, the event loop needs to know
when a child process has finished its execution.
In asyncio, child processes are created with
:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec`
functions.
asyncio defines an abstract base class :class:`AbstractChildWatcher`
that child watchers should implement, and has two different
implementations: :class:`SafeChildWatcher` (configured to be used
by default) and :class:`FastChildWatcher`.
See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>`
section.
The following two functions can be used to customize the watcher
implementation used by the asyncio event loop:
.. function:: get_child_watcher()
Return the current child watcher for the current policy.
.. function:: set_child_watcher(watcher)
Set the current child watcher to *watcher* for the current
policy. *watcher* must implement methods defined in the
:class:`AbstractChildWatcher` base class.
.. note::
Third-party event loops implementations might not support
custom child watchers. For such event loops, using
:func:`set_child_watcher` might have no effect or even can
be prohibited.
.. class:: AbstractChildWatcher
.. method:: add_child_handler(pid, callback, \*args)
Register a new child handler.
Arrange for ``callback(pid, returncode, *args)`` to be called
when a process with PID equal to *pid* terminates. Specifying
another callback for the same process replaces the previous
handler.
*callback* callable must be thread-safe.
.. method:: remove_child_handler(pid)
Removes the handler for process with PID equal to *pid*.
The function returns ``True`` if the handler was successfully
removed, ``False`` if there was nothing to remove.
.. method:: attach_loop(loop)
Attach the watcher to an event loop.
If the watcher was previously attached to an event loop, then
it is first detached before attaching to the new loop.
Note: loop may be ``None``.
.. method:: close()
Close the watcher.
This method has to be called to ensure that underlying
resources are cleaned-up.
.. class:: SafeChildWatcher
This implementation avoids disrupting other code spawning processes
by polling every process explicitly on a :py:data:`SIGCHLD` signal.
This is a safe solution but it has a significant overhead when
handling a big number of processes (*O(n)* each time a
:py:data:`SIGCHLD` is received).
asyncio uses this implementation by default.
.. class:: FastChildWatcher
This implementation reaps every terminated processes by calling
``os.waitpid(-1)`` directly, possibly breaking other code spawning
processes and waiting for their termination.
There is no noticeable overhead when handling a big number of
children (*O(1)* each time a child terminates).
Custom Policies
===============
To implement a new event loop policy, it is recommended to subclass
:class:`DefaultEventLoopPolicy` and override the methods for which
custom behavior is wanted, e.g.::
class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy):
def get_event_loop(self):
"""Get the event loop.
This may be None or an instance of EventLoop.
"""
loop = super().get_event_loop()
# Do something with loop ...
return loop
asyncio.set_event_loop_policy(MyEventLoopPolicy())

1170
Doc/library/asyncio-protocol.rst
View File

File diff suppressed because it is too large Load Diff

197
Doc/library/asyncio-queue.rst
View File

@ -1,41 +1,41 @@
.. currentmodule:: asyncio
======
Queues
======
**Source code:** :source:`Lib/asyncio/queues.py`
asyncio queues are designed to be similar to classes of the
:mod:`queue` module. Although asyncio queues are not thread-safe,
they are designed to be used specifically in async/await code.
Queues:
Note that methods on asyncio queues don't have a *timeout* parameter;
use :func:`asyncio.wait_for` function to do queue operations with a
timeout.
* :class:`Queue`
* :class:`PriorityQueue`
* :class:`LifoQueue`
asyncio queue API was designed to be close to classes of the :mod:`queue`
module (:class:`~queue.Queue`, :class:`~queue.PriorityQueue`,
:class:`~queue.LifoQueue`), but it has no *timeout* parameter. The
:func:`asyncio.wait_for` function can be used to cancel a task after a timeout.
See also the `Examples`_ section below.
Queue
-----
=====
.. class:: Queue(maxsize=0, \*, loop=None)
A queue, useful for coordinating producer and consumer coroutines.
A first in, first out (FIFO) queue.
If *maxsize* is less than or equal to zero, the queue size is infinite. If
it is an integer greater than ``0``, then ``await put()`` will block
when the queue reaches *maxsize*, until an item is removed by :meth:`get`.
If *maxsize* is less than or equal to zero, the queue size is
infinite. If it is an integer greater than ``0``, then
``await put()`` blocks when the queue reaches *maxsize*
until an item is removed by :meth:`get`.
Unlike the standard library :mod:`queue`, you can reliably know this Queue's
size with :meth:`qsize`, since your single-threaded asyncio application won't
be interrupted between calling :meth:`qsize` and doing an operation on the
Queue.
Unlike the standard library threading :mod:`queue`, the size of
the queue is always known and can be returned by calling the
:meth:`qsize` method.
This class is :ref:`not thread safe <asyncio-multithreading>`.
.. versionchanged:: 3.4.4
New :meth:`join` and :meth:`task_done` methods.
.. attribute:: maxsize
Number of items allowed in the queue.
.. method:: empty()
@ -45,26 +45,16 @@ Queue
Return ``True`` if there are :attr:`maxsize` items in the queue.
.. note::
If the Queue was initialized with ``maxsize=0`` (the default), then
:meth:`full()` is never ``True``.
If the queue was initialized with ``maxsize=0`` (the default),
then :meth:`full()` never returns ``True``.
.. coroutinemethod:: get()
Remove and return an item from the queue. If queue is empty, wait until
an item is available.
This method is a :ref:`coroutine <coroutine>`.
.. seealso::
The :meth:`empty` method.
Remove and return an item from the queue. If queue is empty,
wait until an item is available.
.. method:: get_nowait()
Remove and return an item from the queue.
Return an item if one is immediately available, else raise
:exc:`QueueEmpty`.
@ -72,26 +62,16 @@ Queue
Block until all items in the queue have been gotten and processed.
The count of unfinished tasks goes up whenever an item is added to the
queue. The count goes down whenever a consumer thread calls
:meth:`task_done` to indicate that the item was retrieved and all work on
it is complete. When the count of unfinished tasks drops to zero,
:meth:`join` unblocks.
This method is a :ref:`coroutine <coroutine>`.
.. versionadded:: 3.4.4
The count of unfinished tasks goes up whenever an item is added
to the queue. The count goes down whenever a consumer thread calls
:meth:`task_done` to indicate that the item was retrieved and all
work on it is complete. When the count of unfinished tasks drops
to zero, :meth:`join` unblocks.
.. coroutinemethod:: put(item)
Put an item into the queue. If the queue is full, wait until a free slot
is available before adding item.
This method is a :ref:`coroutine <coroutine>`.
.. seealso::
The :meth:`full` method.
Put an item into the queue. If the queue is full, wait until a
free slot is available before adding item.
.. method:: put_nowait(item)
@ -107,54 +87,111 @@ Queue
Indicate that a formerly enqueued task is complete.
Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a
subsequent call to :meth:`task_done` tells the queue that the processing
on the task is complete.
Used by queue consumers. For each :meth:`~Queue.get` used to
fetch a task, a subsequent call to :meth:`task_done` tells the
queue that the processing on the task is complete.
If a :meth:`join` is currently blocking, it will resume when all items
have been processed (meaning that a :meth:`task_done` call was received
for every item that had been :meth:`~Queue.put` into the queue).
If a :meth:`join` is currently blocking, it will resume when all
items have been processed (meaning that a :meth:`task_done`
call was received for every item that had been :meth:`~Queue.put`
into the queue).
Raises :exc:`ValueError` if called more times than there were items
placed in the queue.
.. versionadded:: 3.4.4
.. attribute:: maxsize
Number of items allowed in the queue.
Raises :exc:`ValueError` if called more times than there were
items placed in the queue.
PriorityQueue
-------------
Priority Queue
==============
.. class:: PriorityQueue
A subclass of :class:`Queue`; retrieves entries in priority order (lowest
first).
A variant of :class:`Queue`; retrieves entries in priority order
(lowest first).
Entries are typically tuples of the form: (priority number, data).
Entries are typically tuples of the form
``(priority_number, data)``.
LifoQueue
---------
LIFO Queue
==========
.. class:: LifoQueue
A subclass of :class:`Queue` that retrieves most recently added entries
first.
A variant of :class:`Queue` that retrieves most recently added
entries first (last in, first out).
Exceptions
^^^^^^^^^^
==========
.. exception:: QueueEmpty
Exception raised when the :meth:`~Queue.get_nowait` method is called on a
:class:`Queue` object which is empty.
This exception is raised when the :meth:`~Queue.get_nowait` method
is called on an empty queue.
.. exception:: QueueFull
Exception raised when the :meth:`~Queue.put_nowait` method is called on a
:class:`Queue` object which is full.
Exception raised when the :meth:`~Queue.put_nowait` method is called
on a queue that has reached its *maxsize*.
Examples
========
Queues can be used to distribute workload between several
concurrent tasks::
import asyncio
import random
import time
async def worker(name, queue):
while True:
# Get a "work item" out of the queue.
sleep_for = await queue.get()
# Sleep for the "sleep_for" seconds.
await asyncio.sleep(sleep_for)
# Notify the queue that the "work item" has been processed.
queue.task_done()
print(f'{name} has slept for {sleep_for:.2f} seconds')
async def main():
# Create a queue that we will use to store our "workload".
queue = asyncio.Queue()
# Generate random timings and put them into the queue.
total_sleep_time = 0
for _ in range(20):
sleep_for = random.uniform(0.05, 1.0)
total_sleep_time += sleep_for
queue.put_nowait(sleep_for)
# Create three worker tasks to process the queue concurrently.
tasks = []
for i in range(3):
task = asyncio.create_task(worker(f'worker-{i}', queue))
tasks.append(task)
# Wait until the queue is fully processed.
started_at = time.monotonic()
await queue.join()
total_slept_for = time.monotonic() - started_at
# Cancel our worker tasks.
for task in tasks:
task.cancel()
# Wait until all worker tasks are cancelled.
await asyncio.gather(*tasks, return_exceptions=True)
print('====')
print(f'3 workers slept in parallel for {total_slept_for:.2f} seconds')
print(f'total expected sleep time: {total_sleep_time:.2f} seconds')
asyncio.run(main())

294
Doc/library/asyncio-stream.rst
View File

@ -2,83 +2,107 @@
.. _asyncio-streams:
+++++++++++++++++++++++++++++
Streams (coroutine based API)
+++++++++++++++++++++++++++++
=======
Streams
=======
**Source code:** :source:`Lib/asyncio/streams.py`
Streams are high-level async/await-ready primitives to work with
network connections. Streams allow send and receive data without
using callbacks or low-level protocols and transports.
Stream functions
================
Here's an example of a TCP echo client written using asyncio
streams::
.. note::
import asyncio
The top-level functions in this module are meant as convenience wrappers
only; there's really nothing special there, and if they don't do
exactly what you want, feel free to copy their code.
async def tcp_echo_client(message):
reader, writer = await asyncio.open_connection(
'127.0.0.1', 8888)
print(f'Send: {message!r}')
writer.write(message.encode())
data = await reader.read(100)
print(f'Received: {data.decode()!r}')
print('Close the connection')
writer.close()
asyncio.run(tcp_echo_client('Hello World!'))
.. coroutinefunction:: open_connection(host=None, port=None, \*, loop=None, limit=None, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None)
.. rubric:: Stream Functions
A wrapper for :meth:`~AbstractEventLoop.create_connection()` returning a (reader,
writer) pair.
The following top-level asyncio functions can be used to create
and work with streams:
The reader returned is a :class:`StreamReader` instance; the writer is
a :class:`StreamWriter` instance.
When specified, the *loop* argument determines which event loop to use,
and the *limit* argument determines the buffer size limit used by the
.. coroutinefunction:: open_connection(host=None, port=None, \*, \
loop=None, limit=None, ssl=None, family=0, \
proto=0, flags=0, sock=None, local_addr=None, \
server_hostname=None, ssl_handshake_timeout=None)
Establish a network connection and return a pair of
``(reader, writer)``.
The returned *reader* and *writer* objects are instances of
:class:`StreamReader` and :class:`StreamWriter` classes.
The *loop* argument is optional and can always be determined
automatically when this method is awaited from a coroutine.
*limit* determines the buffer size limit used by the
returned :class:`StreamReader` instance.
The rest of the arguments are passed directly to
:meth:`AbstractEventLoop.create_connection`.
This function is a :ref:`coroutine <coroutine>`.
:meth:`loop.create_connection`.
.. versionadded:: 3.7
The *ssl_handshake_timeout* parameter.
.. coroutinefunction:: start_server(client_connected_cb, host=None, port=None, \*, loop=None, limit=None, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, start_serving=True)
.. coroutinefunction:: start_server(client_connected_cb, host=None, \
port=None, \*, loop=None, limit=None, \
family=socket.AF_UNSPEC, \
flags=socket.AI_PASSIVE, sock=None, \
backlog=100, ssl=None, reuse_address=None, \
reuse_port=None, ssl_handshake_timeout=None, \
start_serving=True)
Start a socket server, with a callback for each client connected. The return
value is the same as :meth:`~AbstractEventLoop.create_server()`.
Start a socket server.
The *client_connected_cb* callback is called whenever a new client
connection is established. It receives a reader/writer pair as two
arguments, the first is a :class:`StreamReader` instance,
and the second is a :class:`StreamWriter` instance.
connection is established. It receives a ``(reader, writer)`` pair
as two arguments, instances of the :class:`StreamReader` and
:class:`StreamWriter` classes.
*client_connected_cb* accepts a plain callable or a
*client_connected_cb* can be a plain callable or a
:ref:`coroutine function <coroutine>`; if it is a coroutine function,
it will be automatically converted into a :class:`Task`.
it will be automatically wrapped into a :class:`Task`.
When specified, the *loop* argument determines which event loop to use,
and the *limit* argument determines the buffer size limit used by the
:class:`StreamReader` instance passed to *client_connected_cb*.
The *loop* argument is optional and can always be determined
automatically when this method is awaited from a coroutine.
*limit* determines the buffer size limit used by the
returned :class:`StreamReader` instance.
The rest of the arguments are passed directly to
:meth:`~AbstractEventLoop.create_server()`.
This function is a :ref:`coroutine <coroutine>`.
:meth:`loop.create_server`.
.. versionadded:: 3.7
The *ssl_handshake_timeout* and *start_serving* parameters.
.. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, limit=None, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None)
.. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, \
limit=None, ssl=None, sock=None, \
server_hostname=None, ssl_handshake_timeout=None)
A wrapper for :meth:`~AbstractEventLoop.create_unix_connection()` returning
a (reader, writer) pair.
Establish a UNIX socket connection and return a pair of
``(reader, writer)``.
When specified, the *loop* argument determines which event loop to use,
and the *limit* argument determines the buffer size limit used by the
returned :class:`StreamReader` instance.
Similar to :func:`open_connection` but operates on UNIX sockets.
The rest of the arguments are passed directly to
:meth:`~AbstractEventLoop.create_unix_connection()`.
This function is a :ref:`coroutine <coroutine>`.
See also the documentation of :meth:`loop.create_unix_connection`.
Availability: UNIX.
@ -90,27 +114,16 @@ Stream functions
The *path* parameter can now be a :term:`path-like object`
.. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \*, loop=None, limit=None, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, start_serving=True)
.. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \
\*, loop=None, limit=None, sock=None, \
backlog=100, ssl=None, ssl_handshake_timeout=None, \
start_serving=True)
Start a UNIX Domain Socket server, with a callback for each client connected.
Start a UNIX socket server.
The *client_connected_cb* callback is called whenever a new client
connection is established. It receives a reader/writer pair as two
arguments, the first is a :class:`StreamReader` instance,
and the second is a :class:`StreamWriter` instance.
Similar to :func:`start_server` but operates on UNIX sockets.
*client_connected_cb* accepts a plain callable or a
:ref:`coroutine function <coroutine>`; if it is a coroutine function,
it will be automatically converted into a :class:`Task`.
When specified, the *loop* argument determines which event loop to use,
and the *limit* argument determines the buffer size limit used by the
:class:`StreamReader` instance passed to *client_connected_cb*.
The rest of the arguments are passed directly to
:meth:`~AbstractEventLoop.create_unix_server()`.
This function is a :ref:`coroutine <coroutine>`.
See also the documentation of :meth:`loop.create_unix_server`.
Availability: UNIX.
@ -123,6 +136,13 @@ Stream functions
The *path* parameter can now be a :term:`path-like object`.
.. rubric:: Contents
* `StreamReader`_ and `StreamWriter`_
* `StreamReaderProtocol`_
* `Examples`_
StreamReader
============
@ -159,8 +179,6 @@ StreamReader
If the EOF was received and the internal buffer is empty,
return an empty ``bytes`` object.
This method is a :ref:`coroutine <coroutine>`.
.. coroutinemethod:: readline()
Read one line, where "line" is a sequence of bytes ending with ``\n``.
@ -171,8 +189,6 @@ StreamReader
If the EOF was received and the internal buffer is empty,
return an empty ``bytes`` object.
This method is a :ref:`coroutine <coroutine>`.
.. coroutinemethod:: readexactly(n)
Read exactly *n* bytes. Raise an :exc:`IncompleteReadError` if the end of
@ -180,8 +196,6 @@ StreamReader
:attr:`IncompleteReadError.partial` attribute of the exception contains
the partial read bytes.
This method is a :ref:`coroutine <coroutine>`.
.. coroutinemethod:: readuntil(separator=b'\\n')
Read data from the stream until ``separator`` is found.
@ -208,7 +222,8 @@ StreamReader
.. method:: at_eof()
Return ``True`` if the buffer is empty and :meth:`feed_eof` was called.
Return ``True`` if the buffer is empty and :meth:`feed_eof`
was called.
StreamWriter
@ -299,7 +314,8 @@ StreamWriter
StreamReaderProtocol
====================
.. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, loop=None)
.. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, \
loop=None)
Trivial helper class to adapt between :class:`Protocol` and
:class:`StreamReader`. Subclass of :class:`Protocol`.
@ -314,36 +330,8 @@ StreamReaderProtocol
accidentally calling inappropriate methods of the protocol.)
IncompleteReadError
===================
.. exception:: IncompleteReadError
Incomplete read error, subclass of :exc:`EOFError`.
.. attribute:: expected
Total number of expected bytes (:class:`int`).
.. attribute:: partial
Read bytes string before the end of stream was reached (:class:`bytes`).
LimitOverrunError
=================
.. exception:: LimitOverrunError
Reached the buffer limit while looking for a separator.
.. attribute:: consumed
Total number of to be consumed bytes.
Stream examples
===============
Examples
========
.. _asyncio-tcp-echo-client-streams:
@ -354,28 +342,26 @@ TCP echo client using the :func:`asyncio.open_connection` function::
import asyncio
async def tcp_echo_client(message, loop):
reader, writer = await asyncio.open_connection('127.0.0.1', 8888,
loop=loop)
async def tcp_echo_client(message):
reader, writer = await asyncio.open_connection(
'127.0.0.1', 8888)
print('Send: %r' % message)
print(f'Send: {message!r}')
writer.write(message.encode())
data = await reader.read(100)
print('Received: %r' % data.decode())
print(f'Received: {data.decode()!r}')
print('Close the socket')
print('Close the connection')
writer.close()
message = 'Hello World!'
loop = asyncio.get_event_loop()
loop.run_until_complete(tcp_echo_client(message, loop))
loop.close()
asyncio.run(tcp_echo_client('Hello World!'))
.. seealso::
The :ref:`TCP echo client protocol <asyncio-tcp-echo-client-protocol>`
example uses the :meth:`AbstractEventLoop.create_connection` method.
example uses the low-level :meth:`loop.create_connection` method.
.. _asyncio-tcp-echo-server-streams:
@ -391,35 +377,33 @@ TCP echo server using the :func:`asyncio.start_server` function::
data = await reader.read(100)
message = data.decode()
addr = writer.get_extra_info('peername')
print("Received %r from %r" % (message, addr))
print("Send: %r" % message)
print(f"Received {message!r} from {addr!r}")
print(f"Send: {message!r}")
writer.write(data)
await writer.drain()
print("Close the client socket")
print("Close the connection")
writer.close()
loop = asyncio.get_event_loop()
coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop)
server = loop.run_until_complete(coro)
async def main():
server = await asyncio.start_server(
handle_echo, '127.0.0.1', 8888)
# Serve requests until Ctrl+C is pressed
print('Serving on {}'.format(server.sockets[0].getsockname()))
try:
loop.run_forever()
except KeyboardInterrupt:
pass
addr = server.sockets[0].getsockname()
print(f'Serving on {addr}')
async with server:
await server.serve_forever()
asyncio.run(main())
# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
.. seealso::
The :ref:`TCP echo server protocol <asyncio-tcp-echo-server-protocol>`
example uses the :meth:`AbstractEventLoop.create_server` method.
example uses the :meth:`loop.create_server` method.
Get HTTP headers
@ -434,30 +418,34 @@ Simple example querying HTTP headers of the URL passed on the command line::
async def print_http_headers(url):
url = urllib.parse.urlsplit(url)
if url.scheme == 'https':
connect = asyncio.open_connection(url.hostname, 443, ssl=True)
reader, writer = await asyncio.open_connection(
url.hostname, 443, ssl=True)
else:
connect = asyncio.open_connection(url.hostname, 80)
reader, writer = await connect
query = ('HEAD {path} HTTP/1.0\r\n'
'Host: {hostname}\r\n'
'\r\n').format(path=url.path or '/', hostname=url.hostname)
reader, writer = await asyncio.open_connection(
url.hostname, 80)
query = (
f"HEAD {url.path or '/'} HTTP/1.0\r\n"
f"Host: {url.hostname}\r\n"
f"\r\n"
)
writer.write(query.encode('latin-1'))
while True:
line = await reader.readline()
if not line:
break
line = line.decode('latin1').rstrip()
if line:
print('HTTP header> %s' % line)
print(f'HTTP header> {line}')
# Ignore the body, close the socket
writer.close()
url = sys.argv[1]
loop = asyncio.get_event_loop()
task = asyncio.ensure_future(print_http_headers(url))
loop.run_until_complete(task)
loop.close()
asyncio.run(print_http_headers(url))
Usage::
@ -467,6 +455,7 @@ or with HTTPS::
python example.py https://example.com/path/page.html
.. _asyncio-register-socket-streams:
Register an open socket to wait for data using streams
@ -476,14 +465,18 @@ Coroutine waiting until a socket receives data using the
:func:`open_connection` function::
import asyncio
from socket import socketpair
import socket
async def wait_for_data(loop):
# Create a pair of connected sockets
rsock, wsock = socketpair()
async def wait_for_data():
# Get a reference to the current event loop because
# we want to access low-level APIs.
loop = asyncio.get_running_loop()
# Register the open socket to wait for data
reader, writer = await asyncio.open_connection(sock=rsock, loop=loop)
# Create a pair of connected sockets.
rsock, wsock = socket.socketpair()
# Register the open socket to wait for data.
reader, writer = await asyncio.open_connection(sock=rsock)
# Simulate the reception of data from the network
loop.call_soon(wsock.send, 'abc'.encode())
@ -498,17 +491,14 @@ Coroutine waiting until a socket receives data using the
# Close the second socket
wsock.close()
loop = asyncio.get_event_loop()
loop.run_until_complete(wait_for_data(loop))
loop.close()
asyncio.run(wait_for_data())
.. seealso::
The :ref:`register an open socket to wait for data using a protocol
<asyncio-register-socket>` example uses a low-level protocol created by the
:meth:`AbstractEventLoop.create_connection` method.
<asyncio-register-socket>` example uses a low-level protocol and
the :meth:`loop.create_connection` method.
The :ref:`watch a file descriptor for read events
<asyncio-watch-read-event>` example uses the low-level
:meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a
socket.
:meth:`loop.add_reader` method to watch a file descriptor.

473
Doc/library/asyncio-subprocess.rst
View File

@ -2,137 +2,91 @@
.. _asyncio-subprocess:
Subprocess
==========
============
Subprocesses
============
**Source code:** :source:`Lib/asyncio/subprocess.py`
This section describes high-level async/await asyncio APIs to
create and manage subprocesses.
Windows event loop
------------------
Here's an example of how asyncio can run a shell command and
communicate its result back::
On Windows, the default event loop is :class:`SelectorEventLoop` which does not
support subprocesses. :class:`ProactorEventLoop` should be used instead.
Example to use it on Windows::
import asyncio
import asyncio, sys
async def run(cmd):
proc = await asyncio.create_subprocess_shell(
cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE)
if sys.platform == 'win32':
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
stdout, stderr = await proc.communicate()
.. seealso::
print(f'[{cmd!r} exited with {proc.returncode}]')
if stdout:
print(f'[stdout]\n{stdout.decode()}')
if stderr:
print(f'[stderr]\n{stderr.decode()}')
:ref:`Available event loops <asyncio-event-loops>` and :ref:`Platform
support <asyncio-platform-support>`.
asyncio.run(run('ls /zzz'))
will print::
['ls /zzz' exited with 1]
[stderr]
ls: /zzz: No such file or directory
Because all asyncio subprocess functions are asynchronous and asyncio
provides many tools to work with such functions, it is easy to execute
and monitor multiple subprocesses in parallel. It is indeed trivial
to modify the above example to run a few commands at once::
async def main():
await asyncio.gather(
run('ls /zzz'),
run('sleep 1; echo "hello"'))
asyncio.run(main())
See also the `Examples`_ subsection.
Create a subprocess: high-level API using Process
-------------------------------------------------
Creating Subprocesses
=====================
.. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds)
.. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, \
stdout=None, stderr=None, loop=None, \
limit=None, \*\*kwds)
Create a subprocess.
The *limit* parameter sets the buffer limit passed to the
:class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_exec` for other
parameters.
The *limit* argument sets the buffer limit for :class:`StreamReader`
wrappers for :attr:`Process.stdout` and :attr:`Process.stderr`
(if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr*
arguments).
Return a :class:`~asyncio.subprocess.Process` instance.
This function is a :ref:`coroutine <coroutine>`.
See the documentation of :meth:`loop.subprocess_exec` for other
parameters.
.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds)
.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \
stdout=None, stderr=None, loop=None, \
limit=None, \*\*kwds)
Run the shell command *cmd*.
The *limit* parameter sets the buffer limit passed to the
:class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_shell` for other
parameters.
The *limit* argument sets the buffer limit for :class:`StreamReader`
wrappers for :attr:`Process.stdout` and :attr:`Process.stderr`
(if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr*
arguments).
Return a :class:`~asyncio.subprocess.Process` instance.
It is the application's responsibility to ensure that all whitespace and
metacharacters are quoted appropriately to avoid `shell injection
<https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_
vulnerabilities. The :func:`shlex.quote` function can be used to properly
escape whitespace and shell metacharacters in strings that are going to be
used to construct shell commands.
See the documentation of :meth:`loop.subprocess_shell` for other
parameters.
This function is a :ref:`coroutine <coroutine>`.
Use the :meth:`AbstractEventLoop.connect_read_pipe` and
:meth:`AbstractEventLoop.connect_write_pipe` methods to connect pipes.
Create a subprocess: low-level API using subprocess.Popen
---------------------------------------------------------
Run subprocesses asynchronously using the :mod:`subprocess` module.
.. coroutinemethod:: AbstractEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs)
Create a subprocess from one or more string arguments (character strings or
bytes strings encoded to the :ref:`filesystem encoding
<filesystem-encoding>`), where the first string
specifies the program to execute, and the remaining strings specify the
program's arguments. (Thus, together the string arguments form the
``sys.argv`` value of the program, assuming it is a Python script.) This is
similar to the standard library :class:`subprocess.Popen` class called with
shell=False and the list of strings passed as the first argument;
however, where :class:`~subprocess.Popen` takes a single argument which is
list of strings, :func:`subprocess_exec` takes multiple string arguments.
The *protocol_factory* must instantiate a subclass of the
:class:`asyncio.SubprocessProtocol` class.
Other parameters:
* *stdin*: Either a file-like object representing the pipe to be connected
to the subprocess's standard input stream using
:meth:`~AbstractEventLoop.connect_write_pipe`, or the constant
:const:`subprocess.PIPE` (the default). By default a new pipe will be
created and connected.
* *stdout*: Either a file-like object representing the pipe to be connected
to the subprocess's standard output stream using
:meth:`~AbstractEventLoop.connect_read_pipe`, or the constant
:const:`subprocess.PIPE` (the default). By default a new pipe will be
created and connected.
* *stderr*: Either a file-like object representing the pipe to be connected
to the subprocess's standard error stream using
:meth:`~AbstractEventLoop.connect_read_pipe`, or one of the constants
:const:`subprocess.PIPE` (the default) or :const:`subprocess.STDOUT`.
By default a new pipe will be created and connected. When
:const:`subprocess.STDOUT` is specified, the subprocess's standard error
stream will be connected to the same pipe as the standard output stream.
* All other keyword arguments are passed to :class:`subprocess.Popen`
without interpretation, except for *bufsize*, *universal_newlines* and
*shell*, which should not be specified at all.
Returns a pair of ``(transport, protocol)``, where *transport* is an
instance of :class:`BaseSubprocessTransport`.
This method is a :ref:`coroutine <coroutine>`.
See the constructor of the :class:`subprocess.Popen` class for parameters.
.. coroutinemethod:: AbstractEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs)
Create a subprocess from *cmd*, which is a character string or a bytes
string encoded to the :ref:`filesystem encoding <filesystem-encoding>`,
using the platform's "shell" syntax. This is similar to the standard library
:class:`subprocess.Popen` class called with ``shell=True``.
The *protocol_factory* must instantiate a subclass of the
:class:`asyncio.SubprocessProtocol` class.
See :meth:`~AbstractEventLoop.subprocess_exec` for more details about
the remaining arguments.
Returns a pair of ``(transport, protocol)``, where *transport* is an
instance of :class:`BaseSubprocessTransport`.
.. note::
It is the application's responsibility to ensure that all whitespace and
metacharacters are quoted appropriately to avoid `shell injection
@ -141,106 +95,125 @@ Run subprocesses asynchronously using the :mod:`subprocess` module.
escape whitespace and shell metacharacters in strings that are going to be
used to construct shell commands.
This method is a :ref:`coroutine <coroutine>`.
.. note::
The default event loop that asyncio is pre-configured
to use on **Windows** does not support subprocesses.
See :ref:`Subprocess Support on Windows <asyncio-windows-subprocess>`
for details.
.. seealso::
The :meth:`AbstractEventLoop.connect_read_pipe` and
:meth:`AbstractEventLoop.connect_write_pipe` methods.
asyncio has also *low-level* APIs to work with subprocesses:
:meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`,
:meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`,
as well as the :ref:`Subprocess Transports <asyncio-subprocess-transports>`
and :ref:`Subprocess Protocols <asyncio-subprocess-protocols>`.
Constants
---------
=========
.. data:: asyncio.subprocess.PIPE
Special value that can be used as the *stdin*, *stdout* or *stderr* argument
to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and
indicates that a pipe to the standard stream should be opened.
Can be passed to the *stdin*, *stdout* or *stderr* parameters.
If *PIPE* is passed to *stdin* argument, the
:attr:`Process.stdin <asyncio.subprocess.Process.stdin>` attribute
will point to a :class:`StreamWriter` instance.
If *PIPE* is passed to *stdout* or *stderr* arguments, the
:attr:`Process.stdout <asyncio.subprocess.Process.stdout>` and
:attr:`Process.stderr <asyncio.subprocess.Process.stderr>`
attributes will point to :class:`StreamReader` instances.
.. data:: asyncio.subprocess.STDOUT
Special value that can be used as the *stderr* argument to
:func:`create_subprocess_shell` and :func:`create_subprocess_exec` and
indicates that standard error should go into the same handle as standard
output.
Can be passed to the *stderr* parameter to redirect process'
*stderr* to *stdout*.
.. data:: asyncio.subprocess.DEVNULL
Special value that can be used as the *stdin*, *stdout* or *stderr* argument
to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and
indicates that the special file :data:`os.devnull` will be used.
Can be passed as the *stdin*, *stdout* or *stderr* parameters
to redirect the corresponding subprocess' IO to :data:`os.devnull`.
Process
-------
Interacting with Subprocesses
=============================
Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell`
functions return instances of the *Process* class. It is a high-level
wrapper that allows to watch for subprocesses completion and
communicate with them.
.. class:: asyncio.subprocess.Process
A subprocess created by the :func:`create_subprocess_exec` or the
:func:`create_subprocess_shell` function.
An object that wraps OS processes created by the
:func:`create_subprocess_exec` and :func:`create_subprocess_shell`
functions.
The API of the :class:`~asyncio.subprocess.Process` class was designed to be
close to the API of the :class:`subprocess.Popen` class, but there are some
differences:
This class is designed to have a similar API to the
:class:`subprocess.Popen` class, but there are some
notable differences:
* There is no explicit :meth:`~subprocess.Popen.poll` method
* The :meth:`~subprocess.Popen.communicate` and
:meth:`~subprocess.Popen.wait` methods don't take a *timeout* parameter:
use the :func:`wait_for` function
* The *universal_newlines* parameter is not supported (only bytes strings
are supported)
* The :meth:`~asyncio.subprocess.Process.wait` method of
the :class:`~asyncio.subprocess.Process` class is asynchronous whereas the
:meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen`
class is implemented as a busy loop.
* unlike Popen, Process instances do not have an equivalent to
the :meth:`~subprocess.Popen.poll` method;
This class is :ref:`not thread safe <asyncio-multithreading>`. See also the
:ref:`Subprocess and threads <asyncio-subprocess-threads>` section.
* the :meth:`~asyncio.subprocess.Process.communicate` and
:meth:`~asyncio.subprocess.Process.wait` methods don't take a
*timeout* parameter: use the :func:`wait_for` function;
* the :meth:`Process.wait() <asyncio.subprocess.Process.wait>` method
is asynchronous, whereas :meth:`subprocess.Popen.wait` method
is implemented as a blocking busy loop;
* the *universal_newlines* parameter is not supported.
This class is :ref:`not thread safe <asyncio-multithreading>`.
See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>`
section.
.. coroutinemethod:: wait()
Wait for child process to terminate. Set and return :attr:`returncode`
attribute.
Wait for child process to terminate.
This method is a :ref:`coroutine <coroutine>`.
Set and return the :attr:`returncode` attribute.
.. note::
This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` and
the child process generates enough output to a pipe such that it
blocks waiting for the OS pipe buffer to accept more data. Use the
:meth:`communicate` method when using pipes to avoid that.
This method can deadlock when using ``stdout=PIPE`` or
``stderr=PIPE`` and the child process generates so much output
that it blocks waiting for the OS pipe buffer to accept
more data. Use the :meth:`communicate` method when using pipes
to avoid this condition.
.. coroutinemethod:: communicate(input=None)
Interact with process: Send data to stdin. Read data from stdout and
stderr, until end-of-file is reached. Wait for process to terminate.
The optional *input* argument should be data to be sent to the child
process, or ``None``, if no data should be sent to the child. The type
of *input* must be bytes.
Interact with process:
:meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``.
1. send data to *stdin* (if *input* is not ``None``);
2. read data from *stdout* and *stderr*, until EOF is reached;
3. wait for process to terminate.
If a :exc:`BrokenPipeError` or :exc:`ConnectionResetError` exception is
raised when writing *input* into stdin, the exception is ignored. It
occurs when the process exits before all data are written into stdin.
The optional *input* argument is the data (:class:`bytes` object)
that will be sent to the child process.
Note that if you want to send data to the process's stdin, you need to
create the Process object with ``stdin=PIPE``. Similarly, to get anything
other than ``None`` in the result tuple, you need to give ``stdout=PIPE``
and/or ``stderr=PIPE`` too.
Return a tuple ``(stdout_data, stderr_data)``.
This method is a :ref:`coroutine <coroutine>`.
If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError`
exception is raised when writing *input* into *stdin*, the
exception is ignored. This condition occurs when the process
exits before all data are written into *stdin*.
.. note::
If its desired to send data to the process' *stdin*,
the process needs to be created with ``stdin=PIPE``. Similarly,
to get anything other than ``None`` in the result tuple, the
process has to be created with ``stdout=PIPE`` and/or
``stderr=PIPE`` arguments.
The data read is buffered in memory, so do not use this method if the
data size is large or unlimited.
.. versionchanged:: 3.4.2
The method now ignores :exc:`BrokenPipeError` and
:exc:`ConnectionResetError`.
Note, that the data read is buffered in memory, so do not use
this method if the data size is large or unlimited.
.. method:: send_signal(signal)
@ -255,67 +228,81 @@ Process
.. method:: terminate()
Stop the child. On Posix OSs the method sends :py:data:`signal.SIGTERM`
to the child. On Windows the Win32 API function
:c:func:`TerminateProcess` is called to stop the child.
Stop the child.
On Posix OSs the method sends :py:data:`signal.SIGTERM` to the
child process.
On Windows the Win32 API function :c:func:`TerminateProcess` is
called to stop the child process.
.. method:: kill()
Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to
the child. On Windows :meth:`kill` is an alias for :meth:`terminate`.
Kill the child.
On Posix OSs the function sends :py:data:`SIGKILL` to the child
process.
On Windows this method is an alias for :meth:`terminate`.
.. attribute:: stdin
Standard input stream (:class:`StreamWriter`), ``None`` if the process
was created with ``stdin=None``.
Standard input stream (:class:`StreamWriter`) or ``None``
if the process was created with ``stdin=None``.
.. attribute:: stdout
Standard output stream (:class:`StreamReader`), ``None`` if the process
was created with ``stdout=None``.
Standard output stream (:class:`StreamReader`) or ``None``
if the process was created with ``stdout=None``.
.. attribute:: stderr
Standard error stream (:class:`StreamReader`), ``None`` if the process
was created with ``stderr=None``.
Standard error stream (:class:`StreamReader`) or ``None``
if the process was created with ``stderr=None``.
.. warning::
Use the :meth:`communicate` method rather than :attr:`.stdin.write
<stdin>`, :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>`
to avoid deadlocks due to streams pausing reading or writing and blocking
the child process.
Use the :meth:`communicate` method rather than
:attr:`process.stdin.write() <stdin>`,
:attr:`await process.stdout.read() <stdout>` or
:attr:`await process.stderr.read <stderr>`
to avoid deadlocks due to streams pausing reading or writing
and blocking the child process.
.. attribute:: pid
The identifier of the process.
Process identification number (PID).
Note that for processes created by the :func:`create_subprocess_shell`
function, this attribute is the process identifier of the spawned shell.
function, this attribute is the PID of the spawned shell.
.. attribute:: returncode
Return code of the process when it exited. A ``None`` value indicates
that the process has not terminated yet.
Return code of the process when it exits.
A negative value ``-N`` indicates that the child was terminated by signal
``N`` (Unix only).
A ``None`` value indicates that the process has not terminated yet.
A negative value ``-N`` indicates that the child was terminated
by signal ``N`` (Unix only).
.. _asyncio-subprocess-threads:
Subprocess and threads
Subprocess and Threads
----------------------
asyncio supports running subprocesses from different threads, but there
are limits:
asyncio built-in event loops support running subprocesses from
different threads, but there are the following limitations:
* An event loop must run in the main thread
* The child watcher must be instantiated in the main thread, before executing
subprocesses from other threads. Call the :func:`get_child_watcher`
function in the main thread to instantiate the child watcher.
* An event loop must run in the main thread.
The :class:`asyncio.subprocess.Process` class is not thread safe.
* The child watcher must be instantiated in the main thread,
before executing subprocesses from other threads. Call the
:func:`get_child_watcher` function in the main thread to instantiate
the child watcher.
Note, that alternative event loop implementations might not share
the above limitations; please refer to their documentation.
.. seealso::
@ -323,97 +310,43 @@ The :class:`asyncio.subprocess.Process` class is not thread safe.
<asyncio-multithreading>` section.
Subprocess examples
-------------------
Examples
--------
Subprocess using transport and protocol
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
An example using the :class:`~asyncio.subprocess.Process` class to
control a subprocess and the :class:`StreamReader` class to read from
the *stdout*.
Example of a subprocess protocol using to get the output of a subprocess and to
wait for the subprocess exit. The subprocess is created by the
:meth:`AbstractEventLoop.subprocess_exec` method::
import asyncio
import sys
class DateProtocol(asyncio.SubprocessProtocol):
def __init__(self, exit_future):
self.exit_future = exit_future
self.output = bytearray()
def pipe_data_received(self, fd, data):
self.output.extend(data)
def process_exited(self):
self.exit_future.set_result(True)
async def get_date(loop):
code = 'import datetime; print(datetime.datetime.now())'
exit_future = asyncio.Future(loop=loop)
# Create the subprocess controlled by the protocol DateProtocol,
# redirect the standard output into a pipe
transport, protocol = await loop.subprocess_exec(
lambda: DateProtocol(exit_future),
sys.executable, '-c', code,
stdin=None, stderr=None)
# Wait for the subprocess exit using the process_exited() method
# of the protocol
await exit_future
# Close the stdout pipe
transport.close()
# Read the output which was collected by the pipe_data_received()
# method of the protocol
data = bytes(protocol.output)
return data.decode('ascii').rstrip()
if sys.platform == "win32":
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
else:
loop = asyncio.get_event_loop()
date = loop.run_until_complete(get_date(loop))
print("Current date: %s" % date)
loop.close()
Subprocess using streams
^^^^^^^^^^^^^^^^^^^^^^^^
Example using the :class:`~asyncio.subprocess.Process` class to control the
subprocess and the :class:`StreamReader` class to read from the standard
output. The subprocess is created by the :func:`create_subprocess_exec`
The subprocess is created by the :func:`create_subprocess_exec`
function::
import asyncio.subprocess
import asyncio
import sys
async def get_date():
code = 'import datetime; print(datetime.datetime.now())'
# Create the subprocess, redirect the standard output into a pipe
# Create the subprocess; redirect the standard output
# into a pipe.
proc = await asyncio.create_subprocess_exec(
sys.executable, '-c', code,
stdout=asyncio.subprocess.PIPE)
# Read one line of output
# Read one line of output.
data = await proc.stdout.readline()
line = data.decode('ascii').rstrip()
# Wait for the subprocess exit
# Wait for the subprocess exit.
await proc.wait()
return line
if sys.platform == "win32":
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
else:
loop = asyncio.get_event_loop()
asyncio.set_event_loop_policy(
asyncio.WindowsProactorEventLoopPolicy())
date = loop.run_until_complete(get_date())
print("Current date: %s" % date)
loop.close()
date = asyncio.run(get_date())
print(f"Current date: {date}")
See also the :ref:`same example <asyncio-subprocess-proto-example>`
written using low-level APIs.

42
Doc/library/asyncio-task.rst
View File

@ -63,7 +63,7 @@ the coroutine object returned by the call doesn't do anything until you
schedule its execution. There are two basic ways to start it running:
call ``await coroutine`` or ``yield from coroutine`` from another coroutine
(assuming the other coroutine is already running!), or schedule its execution
using the :func:`ensure_future` function or the :meth:`AbstractEventLoop.create_task`
using the :func:`ensure_future` function or the :meth:`loop.create_task`
method.
@ -129,7 +129,7 @@ Example of coroutine displaying ``"Hello World"``::
.. seealso::
The :ref:`Hello World with call_soon() <asyncio-hello-world-callback>`
example uses the :meth:`AbstractEventLoop.call_soon` method to schedule a
example uses the :meth:`loop.call_soon` method to schedule a
callback.
@ -159,7 +159,7 @@ using the :meth:`sleep` function::
The :ref:`display the current date with call_later()
<asyncio-date-callback>` example uses a callback with the
:meth:`AbstractEventLoop.call_later` method.
:meth:`loop.call_later` method.
Example: Chain coroutines
@ -190,32 +190,12 @@ Sequence diagram of the example:
.. image:: tulip_coro.png
:align: center
The "Task" is created by the :meth:`AbstractEventLoop.run_until_complete` method
The "Task" is created by the :meth:`loop.run_until_complete` method
when it gets a coroutine object instead of a task.
The diagram shows the control flow, it does not describe exactly how things
work internally. For example, the sleep coroutine creates an internal future
which uses :meth:`AbstractEventLoop.call_later` to wake up the task in 1 second.
InvalidStateError
-----------------
.. exception:: InvalidStateError
The operation is not allowed in this state.
TimeoutError
------------
.. exception:: TimeoutError
The operation exceeded the given deadline.
.. note::
This exception is different from the builtin :exc:`TimeoutError` exception!
which uses :meth:`loop.call_later` to wake up the task in 1 second.
Future
@ -231,7 +211,7 @@ Future
raise an exception when the future isn't done yet.
- Callbacks registered with :meth:`add_done_callback` are always called
via the event loop's :meth:`~AbstractEventLoop.call_soon`.
via the event loop's :meth:`loop.call_soon`.
- This class is not compatible with the :func:`~concurrent.futures.wait` and
:func:`~concurrent.futures.as_completed` functions in the
@ -281,7 +261,7 @@ Future
The *callback* is called with a single argument - the future object. If the
future is already done when this is called, the callback is scheduled
with :meth:`~AbstractEventLoop.call_soon`.
with :meth:`loop.call_soon`.
An optional keyword-only *context* argument allows specifying a custom
:class:`contextvars.Context` for the *callback* to run in. The current
@ -344,11 +324,11 @@ Example combining a :class:`Future` and a :ref:`coroutine function
The coroutine function is responsible for the computation (which takes 1 second)
and it stores the result into the future. The
:meth:`~AbstractEventLoop.run_until_complete` method waits for the completion of
:meth:`loop.run_until_complete` method waits for the completion of
the future.
.. note::
The :meth:`~AbstractEventLoop.run_until_complete` method uses internally the
The :meth:`loop.run_until_complete` method uses internally the
:meth:`~Future.add_done_callback` method to be notified when the future is
done.
@ -433,7 +413,7 @@ Task
logged: see :ref:`Pending task destroyed <asyncio-pending-task-destroyed>`.
Don't directly create :class:`Task` instances: use the :func:`create_task`
function or the :meth:`AbstractEventLoop.create_task` method.
function or the :meth:`loop.create_task` method.
Tasks support the :mod:`contextvars` module. When a Task
is created it copies the current context and later runs its coroutine
@ -644,7 +624,7 @@ Task functions
.. seealso::
The :func:`create_task` function and
:meth:`AbstractEventLoop.create_task` method.
:meth:`loop.create_task` method.
.. function:: wrap_future(future, \*, loop=None)

4
Doc/library/asyncio.rst
View File

@ -50,7 +50,8 @@ Table of contents:
:maxdepth: 3
asyncio-eventloop.rst
asyncio-eventloops.rst
asyncio-policy.rst
asyncio-platforms.rst
asyncio-task.rst
asyncio-protocol.rst
asyncio-stream.rst
@ -58,6 +59,7 @@ Table of contents:
asyncio-sync.rst
asyncio-queue.rst
asyncio-dev.rst
asyncio-exceptions.rst
.. seealso::

2
Doc/library/ipc.rst
View File

@ -16,11 +16,11 @@ The list of modules described in this chapter is:
.. toctree::
asyncio.rst
socket.rst
ssl.rst
select.rst
selectors.rst
asyncio.rst
asyncore.rst
asynchat.rst
signal.rst

28
Doc/whatsnew/3.5.rst
View File

@ -777,18 +777,18 @@ all changes introduced in Python 3.5 have also been backported to Python 3.4.x.
Notable changes in the :mod:`asyncio` module since Python 3.4.0:
* New debugging APIs: :meth:`loop.set_debug() <asyncio.BaseEventLoop.set_debug>`
and :meth:`loop.get_debug() <asyncio.BaseEventLoop.get_debug>` methods.
* New debugging APIs: :meth:`loop.set_debug() <asyncio.loop.set_debug>`
and :meth:`loop.get_debug() <asyncio.loop.get_debug>` methods.
(Contributed by Victor Stinner.)
* The proactor event loop now supports SSL.
(Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.)
* A new :meth:`loop.is_closed() <asyncio.BaseEventLoop.is_closed>` method to
* A new :meth:`loop.is_closed() <asyncio.loop.is_closed>` method to
check if the event loop is closed.
(Contributed by Victor Stinner in :issue:`21326`.)
* A new :meth:`loop.create_task() <asyncio.BaseEventLoop.create_task>`
* A new :meth:`loop.create_task() <asyncio.loop.create_task>`
to conveniently create and schedule a new :class:`~asyncio.Task`
for a coroutine. The ``create_task`` method is also used by all
asyncio functions that wrap coroutines into tasks, such as
@ -805,10 +805,10 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0:
(Contributed by Yury Selivanov.)
* New :meth:`loop.set_task_factory()
<asyncio.AbstractEventLoop.set_task_factory>` and
:meth:`loop.get_task_factory() <asyncio.AbstractEventLoop.get_task_factory>`
<asyncio.loop.set_task_factory>` and
:meth:`loop.get_task_factory() <asyncio.loop.get_task_factory>`
methods to customize the task factory that :meth:`loop.create_task()
<asyncio.BaseEventLoop.create_task>` method uses. (Contributed by Yury
<asyncio.loop.create_task>` method uses. (Contributed by Yury
Selivanov.)
* New :meth:`Queue.join() <asyncio.Queue.join>` and
@ -822,7 +822,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0:
Updates in 3.5.1:
* The :func:`~asyncio.ensure_future` function and all functions that
use it, such as :meth:`loop.run_until_complete() <asyncio.BaseEventLoop.run_until_complete>`,
use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,
now accept all kinds of :term:`awaitable objects <awaitable>`.
(Contributed by Yury Selivanov.)
@ -834,20 +834,20 @@ Updates in 3.5.1:
method to check if the transport is closing or closed.
(Contributed by Yury Selivanov.)
* The :meth:`loop.create_server() <asyncio.BaseEventLoop.create_server>`
* The :meth:`loop.create_server() <asyncio.loop.create_server>`
method can now accept a list of hosts.
(Contributed by Yann Sionneau.)
Updates in 3.5.2:
* New :meth:`loop.create_future() <asyncio.BaseEventLoop.create_future>`
* New :meth:`loop.create_future() <asyncio.loop.create_future>`
method to create Future objects. This allows alternative event
loop implementations, such as
`uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
:class:`asyncio.Future` implementation.
(Contributed by Yury Selivanov.)
* New :meth:`loop.get_exception_handler() <asyncio.BaseEventLoop.get_exception_handler>`
* New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`
method to get the current exception handler.
(Contributed by Yury Selivanov.)
@ -856,13 +856,13 @@ Updates in 3.5.2:
sequence appears.
(Contributed by Mark Korenberg.)
* The :meth:`loop.create_connection() <asyncio.BaseEventLoop.create_connection>`
and :meth:`loop.create_server() <asyncio.BaseEventLoop.create_server>`
* The :meth:`loop.create_connection() <asyncio.loop.create_connection>`
and :meth:`loop.create_server() <asyncio.loop.create_server>`
methods are optimized to avoid calling the system ``getaddrinfo``
function if the address is already resolved.
(Contributed by A. Jesse Jiryu Davis.)
* The :meth:`loop.sock_connect(sock, address) <asyncio.BaseEventLoop.sock_connect>`
* The :meth:`loop.sock_connect(sock, address) <asyncio.loop.sock_connect>`
no longer requires the *address* to be resolved prior to the call.
(Contributed by A. Jesse Jiryu Davis.)

16
Doc/whatsnew/3.6.rst
View File

@ -824,7 +824,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
(Contributed by Yury Selivanov in :issue:`28613`.)
* The :func:`~asyncio.ensure_future` function and all functions that
use it, such as :meth:`loop.run_until_complete() <asyncio.BaseEventLoop.run_until_complete>`,
use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,
now accept all kinds of :term:`awaitable objects <awaitable>`.
(Contributed by Yury Selivanov.)
@ -836,18 +836,18 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
method to check if the transport is closing or closed.
(Contributed by Yury Selivanov.)
* The :meth:`loop.create_server() <asyncio.BaseEventLoop.create_server>`
* The :meth:`loop.create_server() <asyncio.loop.create_server>`
method can now accept a list of hosts.
(Contributed by Yann Sionneau.)
* New :meth:`loop.create_future() <asyncio.BaseEventLoop.create_future>`
* New :meth:`loop.create_future() <asyncio.loop.create_future>`
method to create Future objects. This allows alternative event
loop implementations, such as
`uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
:class:`asyncio.Future` implementation.
(Contributed by Yury Selivanov in :issue:`27041`.)
* New :meth:`loop.get_exception_handler() <asyncio.BaseEventLoop.get_exception_handler>`
* New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`
method to get the current exception handler.
(Contributed by Yury Selivanov in :issue:`27040`.)
@ -860,12 +860,12 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
has been improved.
(Contributed by Mark Korenberg in :issue:`28370`.)
* The :meth:`loop.getaddrinfo() <asyncio.BaseEventLoop.getaddrinfo>`
* The :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`
method is optimized to avoid calling the system ``getaddrinfo``
function if the address is already resolved.
(Contributed by A. Jesse Jiryu Davis.)
* The :meth:`loop.stop() <asyncio.BaseEventLoop.stop>`
* The :meth:`loop.stop() <asyncio.loop.stop>`
method has been changed to stop the loop immediately after
the current iteration. Any new callbacks scheduled as a result
of the last iteration will be discarded.
@ -876,7 +876,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
the :exc:`StopIteration` exception.
(Contributed by Chris Angelico in :issue:`26221`.)
* New :meth:`loop.connect_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>`
* New :meth:`loop.connect_accepted_socket() <asyncio.loop.connect_accepted_socket>`
method to be used by servers that accept connections outside of asyncio,
but that use asyncio to handle them.
(Contributed by Jim Fulton in :issue:`27392`.)
@ -884,7 +884,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
* ``TCP_NODELAY`` flag is now set for all TCP transports by default.
(Contributed by Yury Selivanov in :issue:`27456`.)
* New :meth:`loop.shutdown_asyncgens() <asyncio.AbstractEventLoop.shutdown_asyncgens>`
* New :meth:`loop.shutdown_asyncgens() <asyncio.loop.shutdown_asyncgens>`
to properly close pending asynchronous generators before closing the
loop.
(Contributed by Yury Selivanov in :issue:`28003`.)

44
Doc/whatsnew/3.7.rst
View File

@ -637,10 +637,10 @@ include:
(Contributed by Yury Selivanov in :issue:`32314`.)
* asyncio gained support for :mod:`contextvars`.
:meth:`loop.call_soon() <asyncio.AbstractEventLoop.call_soon>`,
:meth:`loop.call_soon_threadsafe() <asyncio.AbstractEventLoop.call_soon_threadsafe>`,
:meth:`loop.call_later() <asyncio.AbstractEventLoop.call_later>`,
:meth:`loop.call_at() <asyncio.AbstractEventLoop.call_at>`, and
:meth:`loop.call_soon() <asyncio.loop.call_soon>`,
:meth:`loop.call_soon_threadsafe() <asyncio.loop.call_soon_threadsafe>`,
:meth:`loop.call_later() <asyncio.loop.call_later>`,
:meth:`loop.call_at() <asyncio.loop.call_at>`, and
:meth:`Future.add_done_callback() <asyncio.Future.add_done_callback>`
have a new optional keyword-only *context* parameter.
:class:`Tasks <asyncio.Task>` now track their context automatically.
@ -651,11 +651,11 @@ include:
to ``asyncio.get_event_loop().create_task()``.
(Contributed by Andrew Svetlov in :issue:`32311`.)
* The new :meth:`loop.start_tls() <asyncio.AbstractEventLoop.start_tls>`
* The new :meth:`loop.start_tls() <asyncio.loop.start_tls>`
method can be used to upgrade an existing connection to TLS.
(Contributed by Yury Selivanov in :issue:`23749`.)
* The new :meth:`loop.sock_recv_into() <asyncio.AbstractEventLoop.sock_recv_into>`
* The new :meth:`loop.sock_recv_into() <asyncio.loop.sock_recv_into>`
method allows reading data from a socket directly into a provided buffer making
it possible to reduce data copies.
(Contributed by Antoine Pitrou in :issue:`31819`.)
@ -683,13 +683,13 @@ include:
can be used to determine if the writer is closing.
(Contributed by Andrew Svetlov in :issue:`32391`.)
* The new :meth:`loop.sock_sendfile() <asyncio.AbstractEventLoop.sock_sendfile>`
* The new :meth:`loop.sock_sendfile() <asyncio.loop.sock_sendfile>`
coroutine method allows sending files using :mod:`os.sendfile` when possible.
(Contributed by Andrew Svetlov in :issue:`32410`.)
* The new :meth:`Task.get_loop() <asyncio.Task.get_loop>` and
:meth:`Future.get_loop() <asyncio.Future.get_loop>` methods
return the instance of the loop on which a task or a future were created.
* The new :meth:`Future.get_loop() <asyncio.Future.get_loop>` and
``Task.get_loop()`` methods return the instance of the loop on which a task or
a future were created.
:meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for
:class:`asyncio.Server` objects.
(Contributed by Yury Selivanov in :issue:`32415` and
@ -698,8 +698,8 @@ include:
* It is now possible to control how instances of :class:`asyncio.Server` begin
serving. Previously, the server would start serving immediately when created.
The new *start_serving* keyword argument to
:meth:`loop.create_server() <asyncio.AbstractEventLoop.create_server>` and
:meth:`loop.create_unix_server() <asyncio.AbstractEventLoop.create_unix_server>`,
:meth:`loop.create_server() <asyncio.loop.create_server>` and
:meth:`loop.create_unix_server() <asyncio.loop.create_unix_server>`,
as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and
:meth:`Server.serve_forever() <asyncio.Server.serve_forever>`
can be used to decouple server instantiation and serving. The new
@ -717,20 +717,20 @@ include:
(Contributed by Yury Selivanov in :issue:`32662`.)
* Callback objects returned by
:func:`loop.call_later() <asyncio.AbstractEventLoop.call_later>`
:func:`loop.call_later() <asyncio.loop.call_later>`
gained the new :meth:`when() <asyncio.TimerHandle.when>` method which
returns an absolute scheduled callback timestamp.
(Contributed by Andrew Svetlov in :issue:`32741`.)
* The :meth:`loop.create_datagram_endpoint() \
<asyncio.AbstractEventLoop.create_datagram_endpoint>` method
<asyncio.loop.create_datagram_endpoint>` method
gained support for Unix sockets.
(Contributed by Quentin Dawans in :issue:`31245`.)
* The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions,
:meth:`loop.create_connection() <asyncio.AbstractEventLoop.create_connection>`,
:meth:`loop.create_server() <asyncio.AbstractEventLoop.create_server>`,
:meth:`loop.create_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>`
:meth:`loop.create_connection() <asyncio.loop.create_connection>`,
:meth:`loop.create_server() <asyncio.loop.create_server>`,
:meth:`loop.create_accepted_socket() <asyncio.loop.connect_accepted_socket>`
methods and their corresponding UNIX socket variants now accept the
*ssl_handshake_timeout* keyword argument.
(Contributed by Neil Aspinall in :issue:`29970`.)
@ -2360,11 +2360,11 @@ Changes in the Python API
(Contributed by Brett Cannon in :issue:`33169`.)
* In :mod:`asyncio`,
:meth:`loop.sock_recv() <asyncio.AbstractEventLoop.sock_recv>`,
:meth:`loop.sock_sendall() <asyncio.AbstractEventLoop.sock_sendall>`,
:meth:`loop.sock_accept() <asyncio.AbstractEventLoop.sock_accept>`,
:meth:`loop.getaddrinfo() <asyncio.AbstractEventLoop.getaddrinfo>`,
:meth:`loop.getnameinfo() <asyncio.AbstractEventLoop.getnameinfo>`
:meth:`loop.sock_recv() <asyncio.loop.sock_recv>`,
:meth:`loop.sock_sendall() <asyncio.loop.sock_sendall>`,
:meth:`loop.sock_accept() <asyncio.loop.sock_accept>`,
:meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`,
:meth:`loop.getnameinfo() <asyncio.loop.getnameinfo>`
have been changed to be proper coroutine methods to match their
documentation. Previously, these methods returned :class:`asyncio.Future`
instances.

4
Doc/whatsnew/3.8.rst
View File

@ -170,7 +170,7 @@ Deprecated
* Passing an object that is not an instance of
:class:`concurrent.futures.ThreadPoolExecutor` to
:meth:`asyncio.AbstractEventLoop.set_default_executor()` is
:meth:`asyncio.loop.set_default_executor()` is
deprecated and will be prohibited in Python 3.9.
(Contributed by Elvis Pranskevichus in :issue:`34075`.)
@ -264,7 +264,7 @@ Changes in the Python API
* Asyncio tasks can now be named, either by passing the ``name`` keyword
argument to :func:`asyncio.create_task` or
the :meth:`~asyncio.AbstractEventLoop.create_task` event loop method, or by
the :meth:`~asyncio.loop.create_task` event loop method, or by
calling the :meth:`~asyncio.Task.set_name` method on the task object. The
task name is visible in the ``repr()`` output of :class:`asyncio.Task` and
can also be retrieved using the :meth:`~asyncio.Task.get_name` method.