PEP: 306
Title: How to Change Python's Grammar
Version: $Revision$
Last-Modified: $Date$
Author: Michael Hudson <[email protected]>, Jack Diederich <[email protected]>, Alyssa Coghlan <[email protected]>, Benjamin Peterson <[email protected]>
Status: Withdrawn
Type: Informational
Content-Type: text/x-rst
Created: 29-Jan-2003
Post-History: 30-Jan-2003


Note
====

This PEP has been moved to the Python dev guide [1]_.


Abstract
========

There's more to changing Python's grammar than editing
``Grammar/Grammar`` and ``Python/compile.c``.  This PEP aims to be a
checklist of places that must also be fixed.

It is probably incomplete.  If you see omissions, just add them if
you can -- you are not going to offend the author's sense of
ownership.  Otherwise submit a bug or patch and assign it to mwh.

This PEP is not intended to be an instruction manual on Python
grammar hacking, for several reasons.


Rationale
=========

People are getting this wrong all the time; it took well over a
year before someone noticed [2]_ that adding the floor division
operator (``//``) broke the ``parser`` module.


Checklist
=========

- ``Grammar/Grammar``: OK, you'd probably worked this one out :)

- ``Parser/Python.asdl`` may need changes to match the ``Grammar``.  Run
 ``make`` to regenerate ``Include/Python-ast.h`` and
 ``Python/Python-ast.c``.

- ``Python/ast.c`` will need changes to create the AST objects
 involved with the ``Grammar`` change.  ``Lib/compiler/ast.py`` will
 need matching changes to the pure-python AST objects.

- ``Parser/pgen`` needs to be rerun to regenerate ``Include/graminit.h``
 and ``Python/graminit.c``. (make should handle this for you.)

- ``Python/symbtable.c``: This handles the symbol collection pass
 that happens immediately before the compilation pass.

- ``Python/compile.c``: You will need to create or modify the
 ``compiler_*`` functions to generate opcodes for your productions.

- You may need to regenerate ``Lib/symbol.py`` and/or ``Lib/token.py``
 and/or ``Lib/keyword.py``.

- The ``parser`` module.  Add some of your new syntax to ``test_parser``,
 bang on ``Modules/parsermodule.c`` until it passes.

- Add some usage of your new syntax to ``test_grammar.py``.

- The ``compiler`` package.  A good test is to compile the standard
 library and test suite with the ``compiler`` package and then check
 it runs.  Note that this only needs to be done in Python 2.x.

- If you've gone so far as to change the token structure of
 Python, then the ``Lib/tokenizer.py`` library module will need to
 be changed.

- Certain changes may require tweaks to the library module
 ``pyclbr``.

- Documentation must be written!

- After everything's been checked in, you're likely to see a new
 change to ``Python/Python-ast.c``.  This is because this
 (generated) file contains the SVN version of the source from
 which it was generated.  There's no way to avoid this; you just
 have to submit this file separately.


References
==========

. [1] CPython Developer's Guide: Changing CPython's Grammar
      https://devguide.python.org/grammar/

. [2] SF Bug #676521, parser module validation failure
      https://bugs.python.org/issue676521


Copyright
=========

This document has been placed in the public domain.



.
 Local Variables:
 mode: indented-text
 indent-tabs-mode: nil
 sentence-end-double-space: t
 fill-column: 70
 End:

You are viewing proxied material from gopher.fnord.one. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.