Python-ast.c — generated AST node types

Python/Python-ast.c is machine-generated by running Parser/asdl_c.py against Parser/Python.asdl. It defines every AST node type as a Python PyTypeObject, wires up field names and attributes, and exports the ast module's public namespace. Do not edit this file by hand; edit the ASDL grammar or the code generator instead.

Map

Lines	Symbol	Role
1–80	header comment, includes	Generation timestamp, _ast.h include
81–300	ast_type_reduce, ast_type_fields	Pickling support for all node types
301–700	ast_new_impl, ast_init_impl	ast.AST.__new__ and __init__
701–1500	Module, Interactive, Expression	Top-level mod node types
1501–4500	stmt subtypes: FunctionDef through Expr	Statement node registrations
4501–6500	expr subtypes: BoolOp through Constant	Expression node registrations
6501–7500	pattern, excepthandler, arg, etc.	Remaining grammar categories
7501–8000	_PyAST_Init, PyInit__ast	Module init and type finalisation

Reading

ast.AST base class and __reduce__

Every node inherits from a single C base type. The ast_type_reduce function lets nodes survive pickle.dumps by reconstructing them from their field values.

// CPython: Python/Python-ast.c:112 ast_type_reduce
static PyObject *
ast_type_reduce(PyObject *self, PyObject *unused)
{
    PyObject *dict;
    /* __dict__ may hold extra attributes set by the user. */
    if (_PyObject_GetMethod(self, &_Py_ID(__dict__), &dict) == 0) {
        return PyTuple_Pack(3,
            Py_TYPE(self),
            _PyTuple_FromArray(NULL, 0),
            dict);
    }
    Py_CLEAR(dict);
    return PyTuple_Pack(2,
        Py_TYPE(self),
        _PyTuple_FromArray(NULL, 0));
}

The three-element tuple form (type, args, state) is the standard __reduce__ protocol. state here is the node's __dict__, which carries any annotator-added attributes such as lineno overrides or type-checker metadata.

Field registration for a statement node

Each concrete node type calls a shared helper to register its field names. Here is the pattern for FunctionDef, whose ASDL definition is:

FunctionDef(identifier name, arguments args, stmt* body,
            expr* decorator_list, expr? returns, string? type_comment,
            type_param* type_params)

The generated C code builds a static PyMemberDef-style field table:

// CPython: Python/Python-ast.c:1620 FunctionDef_fields
static const char * const FunctionDef_fields[] = {
    "name", "args", "body", "decorator_list",
    "returns", "type_comment", "type_params",
};

static int
FunctionDef_init(PyObject *self, PyObject *args, PyObject *kw)
{
    return ast_type_init(self, args, kw,
        FunctionDef_fields,
        Py_ARRAY_LENGTH(FunctionDef_fields));
}

ast_type_init stores each keyword argument into self.__dict__ under the matching field name, which is how ast.parse() populates nodes after the C-level parser fills in the ASDL structs.

Attribute vs field distinction

Every stmt and expr node also carries four positional attributes (lineno, col_offset, end_lineno, end_col_offset) that are not in _fields_ but in _attributes_. The generated code registers them separately so that tools like ast.fix_missing_locations can iterate attributes independently.

// CPython: Python/Python-ast.c:203 ast_attributes
static const char * const stmt_attributes[] = {
    "lineno", "col_offset", "end_lineno", "end_col_offset",
};

This separation is why ast.walk only descends into _fields_ values and never recurses into lineno.

Module init and type finalisation

_PyAST_Init is called once during interpreter startup. It finalises every PyTypeObject in dependency order (base before subtype) and installs them all into the _ast module dict.

// CPython: Python/Python-ast.c:7820 _PyAST_Init
int
_PyAST_Init(PyInterpreterState *interp)
{
    struct ast_state *state = &interp->ast;
    if (state->initialized) return 0;

    if (init_types(state) < 0) return -1;
    state->initialized = 1;
    return 0;
}

init_types calls PyType_Ready for each of the roughly 120 node types defined in the ASDL grammar. The resulting type objects are singletons for the lifetime of the interpreter.

gopy notes

gopy does not need to port Python-ast.c in full; the Go compiler pipeline builds its own IR. However, two patterns here are directly relevant.

First, the ast_type_reduce design matches how gopy should implement pickling support for objects that carry a __dict__: pack (type, (), __dict__) and let __setstate__ restore from the third element.

Second, the field-vs-attribute split (_fields_ / _attributes_) is worth adopting for any gopy type that mixes structural children with metadata. Keeping metadata out of _fields_ prevents ast.walk-style traversals from accidentally descending into non-node values.

CPython 3.14 changes

• type_params was added to FunctionDef, AsyncFunctionDef, and ClassDef in 3.12 (PEP 695 type parameter syntax). The field is stable in 3.14.
• Python-ast.c is now regenerated as part of the standard make regen-ast target; the file is checked into the repository for platforms that cannot run the generator.
• In 3.13 the per-interpreter ast_state struct replaced the old process-global statics, enabling sub-interpreter isolation for AST type objects.
• 3.14 adds PEP 701 f-string node changes: JoinedStr children can now be FormattedValue or plain string Constant nodes interleaved, matching the new tokeniser output.