Skip to main content

1711. v0.12.4 asyncio full port

Why this spec exists

Spec 1710's sub-system blocker table parks two rows on T6 with the note "port the asyncio package (event loop, transports, protocols, futures, tasks, streams, subprocess, queues, locks) as its own spec". That spec is this one.

The immediate need is small: unittest.mock does import asyncio at module load and then reaches into asyncio.coroutines._is_coroutine, which is a single sentinel object. stdlib/unittest/async_case.py is heavier; it constructs an asyncio.Runner and runs coroutines. Both consumers sit on the path to test_tabnanny.py (via unittest.mock) and to anything that uses IsolatedAsyncioTestCase.

A stub that defines asyncio.coroutines._is_coroutine only would satisfy unittest.mock. The standing project rule ("port whole subsystem, never partial slices") rules that out: when a gate hits a subsystem, every function in that subsystem gets a faithful port. So this spec covers the full asyncio surface — Python + C — even though only a sliver is exercised on day one.

Checklist

Status legend: DONE = landed and verified, WIP = in progress, TODO = not started, BLOCKED = waiting on a larger sub-system spec.

Sources to fully port (CPython 3.14)

Python layer (Lib/asyncio/):

FileLinesgopy destinationStatusCommit
__init__.py74stdlib/asyncio/__init__.pyTODO
base_events.py2082stdlib/asyncio/base_events.pyTODO
base_futures.py67stdlib/asyncio/base_futures.pyTODO
base_subprocess.py319stdlib/asyncio/base_subprocess.pyTODO
base_tasks.py94stdlib/asyncio/base_tasks.pyTODO
constants.py41stdlib/asyncio/constants.pyTODO
coroutines.py118stdlib/asyncio/coroutines.pyTODO
events.py878stdlib/asyncio/events.pyTODO
exceptions.py62stdlib/asyncio/exceptions.pyTODO
format_helpers.py84stdlib/asyncio/format_helpers.pyTODO
futures.py481stdlib/asyncio/futures.pyTODO
graph.py276stdlib/asyncio/graph.pyTODO
locks.py617stdlib/asyncio/locks.pyTODO
log.py7stdlib/asyncio/log.pyTODO
mixins.py21stdlib/asyncio/mixins.pyTODO
proactor_events.py896stdlib/asyncio/proactor_events.pyTODO
protocols.py216stdlib/asyncio/protocols.pyTODO
queues.py307stdlib/asyncio/queues.pyTODO
runners.py225stdlib/asyncio/runners.pyTODO
selector_events.py1326stdlib/asyncio/selector_events.pyTODO
sslproto.py929stdlib/asyncio/sslproto.pyTODO
staggered.py179stdlib/asyncio/staggered.pyTODO
streams.py787stdlib/asyncio/streams.pyTODO
subprocess.py229stdlib/asyncio/subprocess.pyTODO
taskgroups.py286stdlib/asyncio/taskgroups.pyTODO
tasks.py1141stdlib/asyncio/tasks.pyTODO
threads.py24stdlib/asyncio/threads.pyTODO
timeouts.py174stdlib/asyncio/timeouts.pyTODO
tools.py291stdlib/asyncio/tools.pyTODO
transports.py337stdlib/asyncio/transports.pyTODO
trsock.py98stdlib/asyncio/trsock.pyTODO
unix_events.py972stdlib/asyncio/unix_events.pyTODO
windows_events.py903stdlib/asyncio/windows_events.pyTODO
windows_utils.py181stdlib/asyncio/windows_utils.pyTODO
__main__.py245stdlib/asyncio/__main__.pyTODO

C layer (Modules/):

FileLinesgopy destinationStatusCommit
_asynciomodule.c4429module/_asyncio/module.goTODO

Gate tests

TestLinesStatusCommitNotes
test_tabnanny.py354TODOUnblocked once import asyncio succeeds; the unittest.mockasyncio.coroutines._is_coroutine path is the only asyncio surface this test touches.
test_asyncio/ (41 files)n/aOUT OF SCOPEFull asyncio test corpus is a separate panel.

Downstream consumers unlocked

ConsumerSurfaceStatus
stdlib/unittest/mock.py:284mock._is_coroutine = asyncio.coroutines._is_coroutineTODO
stdlib/unittest/async_case.py:137asyncio.Runner(debug=True, loop_factory=self.loop_factory)TODO

Goal

Replace the missing asyncio package with a one-to-one port of every CPython 3.14 source file in the subsystem. The Python layer ships as byte-equal vendoring under stdlib/asyncio/; the C layer ships as a faithful Go port under module/_asyncio/. After this spec lands, import asyncio succeeds, asyncio.coroutines._is_coroutine is the documented sentinel object, and asyncio.Runner(...).run(coro) runs a coroutine to completion using gopy's selector-loop port.

The success criterion is test_tabnanny.py running green under test/regrtest. The wider test_asyncio/ corpus (41 files) is a separate panel and not gated here.

Workflow

The spec follows the durable port-not-patch / full-subsystem rule. Phases below are ordered smallest-fix-first so the very first commit flips the immediate gate (test_tabnanny.py) and subsequent phases fill out the surface the rest of asyncio needs. Every phase carries its own tracking table; tick rows off as work lands.

Phase 1: leaf files (zero-runtime vendor)

Byte-equal vendor of the leaf files that have no runtime dependencies and define the package's constants and exception types. None execute non-trivial code at import time.

#TaskFileSurfaceStatusCommit
1P1.1log.pylogger = logging.getLogger("asyncio") (vendored; lazy-bound — logging still blocked on threading/itertools)WIP12621ea
2P1.2constants.pynumeric/string constants used across the packageDONE12621ea
3P1.3exceptions.pyCancelledError, InvalidStateError, TimeoutError, IncompleteReadError, LimitOverrunError, SendfileNotAvailableError, BrokenBarrierErrorDONE12621ea
4P1.4mixins.py_LoopBoundMixin (vendored; lazy-bound — pulls threading)WIP12621ea
5P1.5format_helpers.pyrepr helpersDONE12621ea
6P1.6base_futures.pyshared future state constants + _format_callbacksDONE12621ea
7P1.7base_tasks.pyshared task helpers (vendored; lazy-bound — pulls coroutines which Phase 2 ships)WIP12621ea
8P1.8__init__.py (stub)placeholder package marker; eagerly binds the leaves that import without runtime depsDONE12621ea
9P1.9_thread (3.14 lifecycle handle)_thread.start_joinable_thread, _thread._ThreadHandle, _thread._make_thread_handle, _thread.daemon_threads_allowed, _thread._shutdown, _thread._get_main_thread_ident, _thread._is_main_interpreter, _thread.LockType attribute. Required because asyncio.log imports logging imports threading.DONE12621ea

Phase 2: coroutines + _is_coroutine sentinel

Vendor coroutines.py byte-equal. The single observable surface unittest.mock requires is the module-level sentinel _is_coroutine = object(). Audit that gopy's inspect.iscoroutinefunction (vendored in spec 1710 T5.4) works against gopy's CoroutineType so _iscoroutinefunction returns the expected answer.

This is the milestone that turns test_tabnanny.py green once __init__.py (Phase 9) lands: import asyncio reaches the end of the package init without raising, unittest.mock import succeeds, and the test loads.

#TaskSub-systemSurfaceStatusCommit
1P2.1stdlib vendorstdlib/asyncio/coroutines.py (byte-equal)DONE8c6d539
2P2.2inspectverify inspect.iscoroutinefunction returns True for async def over gopy's CoroutineType (relies on spec 1710 T5.4); fixed CO_COROUTINE bit value in compile/codegen.goDONE8c6d539
3P2.3itertoolsgopy itertools.count exposes __next__; required transitively by threading.py:853 (_counter = _count(1).__next__). Bundle: _thread._ThreadHandle() + lock() TpNew, sys.excepthook, os.path.normcase.DONEd77a030

Phase 3: futures + events + protocols + transports

Vendor futures.py, events.py, protocols.py, transports.py, trsock.py. These define the abstract interfaces (AbstractEventLoop, BaseProtocol, BaseTransport) and Future semantics that the rest of the package builds on. Pure Python, no OS calls at import time.

#TaskSub-systemSurfaceStatusCommit
1P3.1stdlib vendorstdlib/asyncio/futures.py (byte-equal) — staged; not bound in __init__ until concurrent.futures ships real executorsWIP576b021
2P3.2stdlib vendorstdlib/asyncio/events.py (byte-equal) — staged; blocked on subprocess import (needs os.waitstatus_to_exitcode)WIP576b021
3P3.3stdlib vendorstdlib/asyncio/protocols.py (byte-equal)DONE576b021
4P3.4stdlib vendorstdlib/asyncio/transports.py (byte-equal)DONE576b021
5P3.5stdlib vendorstdlib/asyncio/trsock.py (byte-equal)DONE576b021
6P3.6contextvarsContext.run(callable, *args, **kw) returns the callable's return value, swapping the active context for the durationTODO
7P3.7weakrefWeakSet, WeakValueDictionary resolvable via weakref (audit, not new code)TODO

Phase 4: locks + queues + streams

Vendor locks.py, queues.py, streams.py, staggered.py. These build on the futures/events surface and don't reach into selectors or sockets. streams.py references socket.socket indirectly through connect/open_connection, but that path only fires when the loop is used; import is safe.

#TaskSub-systemSurfaceStatusCommit
1P4.1stdlib vendorstdlib/asyncio/locks.py (byte-equal)TODO
2P4.2stdlib vendorstdlib/asyncio/queues.py (byte-equal)TODO
3P4.3stdlib vendorstdlib/asyncio/streams.py (byte-equal)TODO
4P4.4stdlib vendorstdlib/asyncio/staggered.py (byte-equal)TODO
5P4.5collectionscollections.deque.appendleft / popleft present (audit)TODO

Phase 5: tasks + taskgroups + timeouts + runners

Vendor tasks.py, taskgroups.py, timeouts.py, runners.py, threads.py. runners.py exposes asyncio.Runner (the surface stdlib/unittest/async_case.py uses). At this point asyncio.Runner(...).run(coro) should work for trivial coroutines that do not touch I/O — drive scheduling via the gopy port of tasks._step plus the base_events ready-queue (Phase 7).

#TaskSub-systemSurfaceStatusCommit
1P5.1stdlib vendorstdlib/asyncio/tasks.py (byte-equal)TODO
2P5.2stdlib vendorstdlib/asyncio/taskgroups.py (byte-equal)TODO
3P5.3stdlib vendorstdlib/asyncio/timeouts.py (byte-equal)TODO
4P5.4stdlib vendorstdlib/asyncio/runners.py (byte-equal)TODO
5P5.5stdlib vendorstdlib/asyncio/threads.py (byte-equal)TODO
6P5.6VMcoroutine .send() / .throw() / .close() slot wiring through to genobject.c semanticsTODO

Phase 6: graph + tools + subprocess

Vendor graph.py, tools.py, subprocess.py, base_subprocess.py. These layer on top of tasks and have no fresh OS surface beyond subprocess (which fails fast and clean on gopy until the subprocess port lands separately).

#TaskSub-systemSurfaceStatusCommit
1P6.1stdlib vendorstdlib/asyncio/graph.py (byte-equal)TODO
2P6.2stdlib vendorstdlib/asyncio/tools.py (byte-equal)TODO
3P6.3stdlib vendorstdlib/asyncio/subprocess.py (byte-equal)TODO
4P6.4stdlib vendorstdlib/asyncio/base_subprocess.py (byte-equal)TODO

Phase 7: base_events (event loop core)

Vendor base_events.py. This is the largest file (2082 lines) and defines BaseEventLoop, the canonical scheduler. The port is verbatim Python; the runtime surface it depends on is large.

#TaskSub-systemSurfaceStatusCommit
1P7.1stdlib vendorstdlib/asyncio/base_events.py (byte-equal)TODO
2P7.2selectorsselectors.DefaultSelector resolves on darwin/linux/windowsTODO
3P7.3signalsignal.set_wakeup_fd, signal.valid_signals on module/_signalTODO
4P7.4socketsocket.socket resolvable; module/_socket covers the surface base_events usesTODO
5P7.5threadingthreading.Event wired (audit)TODO

Phase 8: selector_events + proactor_events + sslproto

Vendor the concrete loop implementations and the SSL handshake state machine. Imports execute cleanly; the actual loop.run_forever() path is what exercises them and is tested in test_asyncio/ (out of scope for 1711).

#TaskSub-systemSurfaceStatusCommit
1P8.1stdlib vendorstdlib/asyncio/selector_events.py (byte-equal)TODO
2P8.2stdlib vendorstdlib/asyncio/proactor_events.py (byte-equal)TODO
3P8.3stdlib vendorstdlib/asyncio/sslproto.py (byte-equal)TODO
4P8.4sslssl.MemoryBIO exists or sslproto import is guarded; verify on gopy _sslTODO

Phase 9: platform loops + full __init__.py

Vendor unix_events.py, windows_events.py, windows_utils.py, and swap the Phase 1 stub __init__.py for the byte-equal CPython copy that imports every submodule. __init__.py:44 does if sys.platform == 'win32': from .windows_events import * else: from .unix_events import *, so one of them executes on every import. Verify both build cleanly even on the opposite platform (unimported branches must still parse).

#TaskSub-systemSurfaceStatusCommit
1P9.1stdlib vendorstdlib/asyncio/unix_events.py (byte-equal)TODO
2P9.2stdlib vendorstdlib/asyncio/windows_events.py (byte-equal)TODO
3P9.3stdlib vendorstdlib/asyncio/windows_utils.py (byte-equal)TODO
4P9.4stdlib vendorreplace Phase 1 stub __init__.py with the byte-equal CPython copyTODO
5P9.5stdlib vendorstdlib/asyncio/__main__.py (byte-equal)TODO

Phase 10: C-level _asyncio module

Port Modules/_asynciomodule.c (4429 lines) into module/_asyncio/module.go. The Python layer falls back to pure-Python implementations when _asyncio is missing (see e.g. futures.py's try: import _asyncio blocks), so the full port can be staged after Phase 9. Until this phase lands the Python fallback paths are exercised; that is the correct behaviour, not a divergence.

#TaskSub-systemSurfaceStatusCommit
1P10.1module portFuture C type (fast path for Future)TODO
2P10.2module portTask C type and _PyAsyncioState task-context plumbingTODO
3P10.3module port_swap_current_task, _register_task, _unregister_task, _enter_task, _leave_taskTODO
4P10.4module port_get_event_loop C-side fast path, _set_event_loop / set_running_loop accessorsTODO
5P10.5module portmodule-level constants exposed back to PythonTODO

Phase 11: inittab + path-finder wiring

module/_asyncio/module.go calls imp.AppendInittab("_asyncio", buildModule) in its init(), and the blank import lands in stdlibinit/registry.go next to _opcode. The Python package at stdlib/asyncio/ is picked up automatically by the existing PathFinder walk; no extra wiring is needed beyond a smoke test.

#TaskSub-systemSurfaceStatusCommit
1P11.1inittabbind _asyncio in stdlibinit/registry.goTODO
2P11.2smokeimport asyncio; asyncio.Runner resolves; import _asyncio; _asyncio.Future resolvesTODO

Phase 12: land the test_tabnanny.py gate

Run test/regrtest test_tabnanny.py. Fix any divergence in the vendored Python files or the gopy runtime; never edit the test. test/cpython/ is the canonical gate location.

#TaskSub-systemSurfaceStatusCommit
1P12.1gate testtest/cpython/test_tabnanny.py green under test/regrtestTODO
2P12.2spec 1710flip 1710 T6 rows in both test_tabnanny.py and test_tokenize.py chains to DONE with this spec's tip commitTODO

DFS execution order: Phase 1 → Phase 2 (closes the immediate unittest.mock import gap) → short-circuit to Phase 12 to flip the gate as soon as it goes green → Phases 3–11 fill the rest of the surface. Each row in each phase ships as its own commit; large rows (P7.1, P8.1, P8.2, P10.1, P10.2) may split further as runtime gaps surface.

Out of scope

  • Lib/test/test_asyncio/ (41 files) — the full asyncio test corpus is a separate v0.12.4 panel; 1711 only gates test_tabnanny.py.
  • IocpProactor / _overlapped / Windows-only IO primitives — vendored for byte equivalence but not exercised on darwin/linux.
  • asyncio integration with the gopy GIL / scheduler — the port preserves CPython's single-threaded event-loop model; any future goroutine-backed loop is a new spec.
  • The Modules/_overlapped.c Windows extension and the Modules/socketmodule.c proactor surface — both belong to their own full-port specs (socket / Windows IO) and only matter when the proactor loop is actually run.

Cross-references

  • Spec 1710 (1710_v0124_lexer_tokenizer_full_port.md) T6 rows in the test_tabnanny.py and test_tokenize.py chains, both BLOCKED on this spec.
  • Spec 1700 (v0.12.4 panel) task #484 — flips to DONE only after both 1710 and 1711 are green.
  • Standing rules: port-not-patch, full-subsystem, CPython citations with line numbers, specs live under website/docs/specs/<group>/.