Skip to main content

1711. v0.12.4 asyncio full port

Why this spec exists​

Spec 1710's sub-system blocker table parks two rows on T6 with the note "port the asyncio package (event loop, transports, protocols, futures, tasks, streams, subprocess, queues, locks) as its own spec". That spec is this one.

The immediate need is small: unittest.mock does import asyncio at module load and then reaches into asyncio.coroutines._is_coroutine, which is a single sentinel object. stdlib/unittest/async_case.py is heavier; it constructs an asyncio.Runner and runs coroutines. Both consumers sit on the path to test_tabnanny.py (via unittest.mock) and to anything that uses IsolatedAsyncioTestCase.

A stub that defines asyncio.coroutines._is_coroutine only would satisfy unittest.mock. The standing project rule ("port whole subsystem, never partial slices") rules that out: when a gate hits a subsystem, every function in that subsystem gets a faithful port. So this spec covers the full asyncio surface — Python + C — even though only a sliver is exercised on day one.

Checklist​

Status legend: DONE = landed and verified, WIP = in progress, TODO = not started, BLOCKED = waiting on a larger sub-system spec.

Sources to fully port (CPython 3.14)​

Python layer (Lib/asyncio/):

FileLinesgopy destinationStatusCommit
__init__.py74stdlib/asyncio/__init__.pyTODO—
base_events.py2082stdlib/asyncio/base_events.pyTODO—
base_futures.py67stdlib/asyncio/base_futures.pyTODO—
base_subprocess.py319stdlib/asyncio/base_subprocess.pyTODO—
base_tasks.py94stdlib/asyncio/base_tasks.pyTODO—
constants.py41stdlib/asyncio/constants.pyTODO—
coroutines.py118stdlib/asyncio/coroutines.pyTODO—
events.py878stdlib/asyncio/events.pyTODO—
exceptions.py62stdlib/asyncio/exceptions.pyTODO—
format_helpers.py84stdlib/asyncio/format_helpers.pyTODO—
futures.py481stdlib/asyncio/futures.pyTODO—
graph.py276stdlib/asyncio/graph.pyTODO—
locks.py617stdlib/asyncio/locks.pyTODO—
log.py7stdlib/asyncio/log.pyTODO—
mixins.py21stdlib/asyncio/mixins.pyTODO—
proactor_events.py896stdlib/asyncio/proactor_events.pyTODO—
protocols.py216stdlib/asyncio/protocols.pyTODO—
queues.py307stdlib/asyncio/queues.pyTODO—
runners.py225stdlib/asyncio/runners.pyTODO—
selector_events.py1326stdlib/asyncio/selector_events.pyTODO—
sslproto.py929stdlib/asyncio/sslproto.pyTODO—
staggered.py179stdlib/asyncio/staggered.pyTODO—
streams.py787stdlib/asyncio/streams.pyTODO—
subprocess.py229stdlib/asyncio/subprocess.pyTODO—
taskgroups.py286stdlib/asyncio/taskgroups.pyTODO—
tasks.py1141stdlib/asyncio/tasks.pyTODO—
threads.py24stdlib/asyncio/threads.pyTODO—
timeouts.py174stdlib/asyncio/timeouts.pyTODO—
tools.py291stdlib/asyncio/tools.pyTODO—
transports.py337stdlib/asyncio/transports.pyTODO—
trsock.py98stdlib/asyncio/trsock.pyTODO—
unix_events.py972stdlib/asyncio/unix_events.pyTODO—
windows_events.py903stdlib/asyncio/windows_events.pyTODO—
windows_utils.py181stdlib/asyncio/windows_utils.pyTODO—
__main__.py245stdlib/asyncio/__main__.pyTODO—

C layer (Modules/):

FileLinesgopy destinationStatusCommit
_asynciomodule.c4429module/_asyncio/module.goTODO—

Gate tests​

TestLinesStatusCommitNotes
test_tabnanny.py354TODO—Unblocked once import asyncio succeeds; the unittest.mock → asyncio.coroutines._is_coroutine path is the only asyncio surface this test touches.
test_asyncio/ (41 files)n/aOUT OF SCOPE—Full asyncio test corpus is a separate panel.

Downstream consumers unlocked​

ConsumerSurfaceStatus
stdlib/unittest/mock.py:284mock._is_coroutine = asyncio.coroutines._is_coroutineTODO
stdlib/unittest/async_case.py:137asyncio.Runner(debug=True, loop_factory=self.loop_factory)TODO

Goal​

Replace the missing asyncio package with a one-to-one port of every CPython 3.14 source file in the subsystem. The Python layer ships as byte-equal vendoring under stdlib/asyncio/; the C layer ships as a faithful Go port under module/_asyncio/. After this spec lands, import asyncio succeeds, asyncio.coroutines._is_coroutine is the documented sentinel object, and asyncio.Runner(...).run(coro) runs a coroutine to completion using gopy's selector-loop port.

The success criterion is test_tabnanny.py running green under test/regrtest. The wider test_asyncio/ corpus (41 files) is a separate panel and not gated here.

Workflow​

The spec follows the durable port-not-patch / full-subsystem rule. Phases below are ordered smallest-fix-first so the very first commit flips the immediate gate (test_tabnanny.py) and subsequent phases fill out the surface the rest of asyncio needs. Every phase carries its own tracking table; tick rows off as work lands.

Phase 1: leaf files (zero-runtime vendor)​

Byte-equal vendor of the leaf files that have no runtime dependencies and define the package's constants and exception types. None execute non-trivial code at import time.

#TaskFileSurfaceStatusCommit
1P1.1log.pylogger = logging.getLogger("asyncio") (vendored; lazy-bound — logging still blocked on threading/itertools)WIP12621ea
2P1.2constants.pynumeric/string constants used across the packageDONE12621ea
3P1.3exceptions.pyCancelledError, InvalidStateError, TimeoutError, IncompleteReadError, LimitOverrunError, SendfileNotAvailableError, BrokenBarrierErrorDONE12621ea
4P1.4mixins.py_LoopBoundMixin (vendored; lazy-bound — pulls threading)WIP12621ea
5P1.5format_helpers.pyrepr helpersDONE12621ea
6P1.6base_futures.pyshared future state constants + _format_callbacksDONE12621ea
7P1.7base_tasks.pyshared task helpers (vendored; lazy-bound — pulls coroutines which Phase 2 ships)WIP12621ea
8P1.8__init__.py (stub)placeholder package marker; eagerly binds the leaves that import without runtime depsDONE12621ea
9P1.9_thread (3.14 lifecycle handle)_thread.start_joinable_thread, _thread._ThreadHandle, _thread._make_thread_handle, _thread.daemon_threads_allowed, _thread._shutdown, _thread._get_main_thread_ident, _thread._is_main_interpreter, _thread.LockType attribute. Required because asyncio.log imports logging imports threading.DONE12621ea

Phase 2: coroutines + _is_coroutine sentinel​

Vendor coroutines.py byte-equal. The single observable surface unittest.mock requires is the module-level sentinel _is_coroutine = object(). Audit that gopy's inspect.iscoroutinefunction (vendored in spec 1710 T5.4) works against gopy's CoroutineType so _iscoroutinefunction returns the expected answer.

This is the milestone that turns test_tabnanny.py green once __init__.py (Phase 9) lands: import asyncio reaches the end of the package init without raising, unittest.mock import succeeds, and the test loads.

#TaskSub-systemSurfaceStatusCommit
1P2.1stdlib vendorstdlib/asyncio/coroutines.py (byte-equal)DONE8c6d539
2P2.2inspectverify inspect.iscoroutinefunction returns True for async def over gopy's CoroutineType (relies on spec 1710 T5.4); fixed CO_COROUTINE bit value in compile/codegen.goDONE8c6d539
3P2.3itertoolsgopy itertools.count exposes __next__; required transitively by threading.py:853 (_counter = _count(1).__next__). Bundle: _thread._ThreadHandle() + lock() TpNew, sys.excepthook, os.path.normcase.DONEd77a030

Phase 3: futures + events + protocols + transports​

Vendor futures.py, events.py, protocols.py, transports.py, trsock.py. These define the abstract interfaces (AbstractEventLoop, BaseProtocol, BaseTransport) and Future semantics that the rest of the package builds on. Pure Python, no OS calls at import time.

#TaskSub-systemSurfaceStatusCommit
1P3.1stdlib vendorstdlib/asyncio/futures.py (byte-equal) — staged; not bound in __init__ until concurrent.futures ships real executorsWIP576b021
2P3.2stdlib vendorstdlib/asyncio/events.py (byte-equal) — staged; blocked on subprocess import (needs os.waitstatus_to_exitcode)WIP576b021
3P3.3stdlib vendorstdlib/asyncio/protocols.py (byte-equal)DONE576b021
4P3.4stdlib vendorstdlib/asyncio/transports.py (byte-equal)DONE576b021
5P3.5stdlib vendorstdlib/asyncio/trsock.py (byte-equal)DONE576b021
6P3.6contextvarsContext.run(callable, *args, **kw) returns the callable's return value, swapping the active context for the durationTODO—
7P3.7weakrefWeakSet, WeakValueDictionary resolvable via weakref (audit, not new code)TODO—

Phase 4: locks + queues + streams​

Vendor locks.py, queues.py, streams.py, staggered.py. These build on the futures/events surface and don't reach into selectors or sockets. streams.py references socket.socket indirectly through connect/open_connection, but that path only fires when the loop is used; import is safe.

#TaskSub-systemSurfaceStatusCommit
1P4.1stdlib vendorstdlib/asyncio/locks.py (byte-equal)TODO—
2P4.2stdlib vendorstdlib/asyncio/queues.py (byte-equal)TODO—
3P4.3stdlib vendorstdlib/asyncio/streams.py (byte-equal)TODO—
4P4.4stdlib vendorstdlib/asyncio/staggered.py (byte-equal)TODO—
5P4.5collectionscollections.deque.appendleft / popleft present (audit)TODO—

Phase 5: tasks + taskgroups + timeouts + runners​

Vendor tasks.py, taskgroups.py, timeouts.py, runners.py, threads.py. runners.py exposes asyncio.Runner (the surface stdlib/unittest/async_case.py uses). At this point asyncio.Runner(...).run(coro) should work for trivial coroutines that do not touch I/O — drive scheduling via the gopy port of tasks._step plus the base_events ready-queue (Phase 7).

#TaskSub-systemSurfaceStatusCommit
1P5.1stdlib vendorstdlib/asyncio/tasks.py (byte-equal)TODO—
2P5.2stdlib vendorstdlib/asyncio/taskgroups.py (byte-equal)TODO—
3P5.3stdlib vendorstdlib/asyncio/timeouts.py (byte-equal)TODO—
4P5.4stdlib vendorstdlib/asyncio/runners.py (byte-equal)TODO—
5P5.5stdlib vendorstdlib/asyncio/threads.py (byte-equal)TODO—
6P5.6VMcoroutine .send() / .throw() / .close() slot wiring through to genobject.c semanticsTODO—

Phase 6: graph + tools + subprocess​

Vendor graph.py, tools.py, subprocess.py, base_subprocess.py. These layer on top of tasks and have no fresh OS surface beyond subprocess (which fails fast and clean on gopy until the subprocess port lands separately).

#TaskSub-systemSurfaceStatusCommit
1P6.1stdlib vendorstdlib/asyncio/graph.py (byte-equal)TODO—
2P6.2stdlib vendorstdlib/asyncio/tools.py (byte-equal)TODO—
3P6.3stdlib vendorstdlib/asyncio/subprocess.py (byte-equal)TODO—
4P6.4stdlib vendorstdlib/asyncio/base_subprocess.py (byte-equal)TODO—

Phase 7: base_events (event loop core)​

Vendor base_events.py. This is the largest file (2082 lines) and defines BaseEventLoop, the canonical scheduler. The port is verbatim Python; the runtime surface it depends on is large.

#TaskSub-systemSurfaceStatusCommit
1P7.1stdlib vendorstdlib/asyncio/base_events.py (byte-equal)TODO—
2P7.2selectorsselectors.DefaultSelector resolves on darwin/linux/windowsTODO—
3P7.3signalsignal.set_wakeup_fd, signal.valid_signals on module/_signalTODO—
4P7.4socketsocket.socket resolvable; module/_socket covers the surface base_events usesTODO—
5P7.5threadingthreading.Event wired (audit)TODO—

Phase 8: selector_events + proactor_events + sslproto​

Vendor the concrete loop implementations and the SSL handshake state machine. Imports execute cleanly; the actual loop.run_forever() path is what exercises them and is tested in test_asyncio/ (out of scope for 1711).

#TaskSub-systemSurfaceStatusCommit
1P8.1stdlib vendorstdlib/asyncio/selector_events.py (byte-equal)TODO—
2P8.2stdlib vendorstdlib/asyncio/proactor_events.py (byte-equal)TODO—
3P8.3stdlib vendorstdlib/asyncio/sslproto.py (byte-equal)TODO—
4P8.4sslssl.MemoryBIO exists or sslproto import is guarded; verify on gopy _sslTODO—

Phase 9: platform loops + full __init__.py​

Vendor unix_events.py, windows_events.py, windows_utils.py, and swap the Phase 1 stub __init__.py for the byte-equal CPython copy that imports every submodule. __init__.py:44 does if sys.platform == 'win32': from .windows_events import * else: from .unix_events import *, so one of them executes on every import. Verify both build cleanly even on the opposite platform (unimported branches must still parse).

#TaskSub-systemSurfaceStatusCommit
1P9.1stdlib vendorstdlib/asyncio/unix_events.py (byte-equal)TODO—
2P9.2stdlib vendorstdlib/asyncio/windows_events.py (byte-equal)TODO—
3P9.3stdlib vendorstdlib/asyncio/windows_utils.py (byte-equal)TODO—
4P9.4stdlib vendorreplace Phase 1 stub __init__.py with the byte-equal CPython copyTODO—
5P9.5stdlib vendorstdlib/asyncio/__main__.py (byte-equal)TODO—

Phase 10: C-level _asyncio module​

Port Modules/_asynciomodule.c (4429 lines) into module/_asyncio/module.go. The Python layer falls back to pure-Python implementations when _asyncio is missing (see e.g. futures.py's try: import _asyncio blocks), so the full port can be staged after Phase 9. Until this phase lands the Python fallback paths are exercised; that is the correct behaviour, not a divergence.

#TaskSub-systemSurfaceStatusCommit
1P10.1module portFuture C type (fast path for Future)TODO—
2P10.2module portTask C type and _PyAsyncioState task-context plumbingTODO—
3P10.3module port_swap_current_task, _register_task, _unregister_task, _enter_task, _leave_taskTODO—
4P10.4module port_get_event_loop C-side fast path, _set_event_loop / set_running_loop accessorsTODO—
5P10.5module portmodule-level constants exposed back to PythonTODO—

Phase 11: inittab + path-finder wiring​

module/_asyncio/module.go calls imp.AppendInittab("_asyncio", buildModule) in its init(), and the blank import lands in stdlibinit/registry.go next to _opcode. The Python package at stdlib/asyncio/ is picked up automatically by the existing PathFinder walk; no extra wiring is needed beyond a smoke test.

#TaskSub-systemSurfaceStatusCommit
1P11.1inittabbind _asyncio in stdlibinit/registry.goTODO—
2P11.2smokeimport asyncio; asyncio.Runner resolves; import _asyncio; _asyncio.Future resolvesTODO—

Phase 12: land the test_tabnanny.py gate​

Run test/regrtest test_tabnanny.py. Fix any divergence in the vendored Python files or the gopy runtime; never edit the test. test/cpython/ is the canonical gate location.

#TaskSub-systemSurfaceStatusCommit
1P12.1gate testtest/cpython/test_tabnanny.py green under test/regrtestTODO—
2P12.2spec 1710flip 1710 T6 rows in both test_tabnanny.py and test_tokenize.py chains to DONE with this spec's tip commitTODO—

DFS execution order: Phase 1 → Phase 2 (closes the immediate unittest.mock import gap) → short-circuit to Phase 12 to flip the gate as soon as it goes green → Phases 3–11 fill the rest of the surface. Each row in each phase ships as its own commit; large rows (P7.1, P8.1, P8.2, P10.1, P10.2) may split further as runtime gaps surface.

Out of scope​

  • Lib/test/test_asyncio/ (41 files) — the full asyncio test corpus is a separate v0.12.4 panel; 1711 only gates test_tabnanny.py.
  • IocpProactor / _overlapped / Windows-only IO primitives — vendored for byte equivalence but not exercised on darwin/linux.
  • asyncio integration with the gopy GIL / scheduler — the port preserves CPython's single-threaded event-loop model; any future goroutine-backed loop is a new spec.
  • The Modules/_overlapped.c Windows extension and the Modules/socketmodule.c proactor surface — both belong to their own full-port specs (socket / Windows IO) and only matter when the proactor loop is actually run.

Cross-references​

  • Spec 1710 (1710_v0124_lexer_tokenizer_full_port.md) T6 rows in the test_tabnanny.py and test_tokenize.py chains, both BLOCKED on this spec.
  • Spec 1700 (v0.12.4 panel) task #484 — flips to DONE only after both 1710 and 1711 are green.
  • Standing rules: port-not-patch, full-subsystem, CPython citations with line numbers, specs live under website/docs/specs/<group>/.