PEP: 562
Title: Module __getattr__ and __dir__
Author: Ivan Levkivskyi <[email protected]>
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017
Resolution: https://mail.python.org/pipermail/python-dev/2017-December/151033.html


. canonical-doc:: `Customizing Module Attribute Access <https://docs.python.org/3/reference/datamodel.html#customizing-module-attribute-access>`_


Abstract
========

It is proposed to support ``__getattr__`` and ``__dir__`` function defined
on modules to provide basic customization of module attribute access.



Rationale
=========

It is sometimes convenient to customize or otherwise have control over
access to module attributes. A typical example is managing deprecation
warnings. Typical workarounds are assigning ``__class__`` of a module object
to a custom subclass of ``types.ModuleType`` or replacing the ``sys.modules``
item with a custom wrapper instance. It would be convenient to simplify this
procedure by recognizing ``__getattr__`` defined directly in a module that
would act like a normal ``__getattr__`` method, except that it will be defined
on module *instances*. For example::

 # lib.py

 from warnings import warn

 deprecated_names = ["old_function", ...]

 def _deprecated_old_function(arg, other):
     ...

 def __getattr__(name):
     if name in deprecated_names:
         warn(f"{name} is deprecated", DeprecationWarning)
         return globals()[f"_deprecated_{name}"]
     raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

 # main.py

 from lib import old_function  # Works, but emits the warning

Another widespread use case for ``__getattr__`` would be lazy submodule
imports. Consider a simple example::

 # lib/__init__.py

 import importlib

 __all__ = ['submod', ...]

 def __getattr__(name):
     if name in __all__:
         return importlib.import_module("." + name, __name__)
     raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

 # lib/submod.py

 print("Submodule loaded")
 class HeavyClass:
     ...

 # main.py

 import lib
 lib.submod.HeavyClass  # prints "Submodule loaded"

There is a related proposal :pep:`549` that proposes to support instance
properties for a similar functionality. The difference is this PEP proposes
a faster and simpler mechanism, but provides more basic customization.
An additional motivation for this proposal is that :pep:`484` already defines
the use of module ``__getattr__`` for this purpose in Python stub files,
see :pep:`484#stub-files`.

In addition, to allow modifying result of a ``dir()`` call on a module
to show deprecated and other dynamically generated attributes, it is
proposed to support module level ``__dir__`` function. For example::

 # lib.py

 deprecated_names = ["old_function", ...]
 __all__ = ["new_function_one", "new_function_two", ...]

 def new_function_one(arg, other):
    ...
 def new_function_two(arg, other):
     ...

 def __dir__():
     return sorted(__all__ + deprecated_names)

 # main.py

 import lib

 dir(lib)  # prints ["new_function_one", "new_function_two", "old_function", ...]


Specification
=============

The ``__getattr__`` function at the module level should accept one argument
which is the name of an attribute and return the computed value or raise
an ``AttributeError``::

 def __getattr__(name: str) -> Any: ...

If an attribute is not found on a module object through the normal lookup
(i.e. ``object.__getattribute__``), then ``__getattr__`` is searched in
the module ``__dict__`` before raising an ``AttributeError``. If found, it is
called with the attribute name and the result is returned. Looking up a name
as a module global will bypass module ``__getattr__``. This is intentional,
otherwise calling ``__getattr__`` for builtins will significantly harm
performance.

The ``__dir__`` function should accept no arguments, and return
a list of strings that represents the names accessible on module::

 def __dir__() -> List[str]: ...

If present, this function overrides the standard ``dir()`` search on
a module.

The reference implementation for this PEP can be found in [2]_.


Backwards compatibility and impact on performance
=================================================

This PEP may break code that uses module level (global) names ``__getattr__``
and ``__dir__``.  (But the language reference explicitly reserves *all*
undocumented dunder names, and allows "breakage without warning"; see [3]_.)
The performance implications of this PEP are minimal, since ``__getattr__``
is called only for missing attributes.

Some tools that perform module attributes discovery might not expect
``__getattr__``. This problem is not new however, since it is already possible
to replace a module with a module subclass with overridden ``__getattr__`` and
``__dir__``, but with this PEP such problems can occur more often.


Discussion
==========

Note that the use of module ``__getattr__`` requires care to keep the referred
objects pickleable. For example, the ``__name__`` attribute of a function
should correspond to the name with which it is accessible via
``__getattr__``::

 def keep_pickleable(func):
     func.__name__ = func.__name__.replace('_deprecated_', '')
     func.__qualname__ = func.__qualname__.replace('_deprecated_', '')
     return func

 @keep_pickleable
 def _deprecated_old_function(arg, other):
     ...

One should be also careful to avoid recursion as one would do with
a class level ``__getattr__``.

To use a module global with triggering ``__getattr__`` (for example if one
wants to use a lazy loaded submodule) one can access it as::

   sys.modules[__name__].some_global

or as::

   from . import some_global

Note that the latter sets the module attribute, thus ``__getattr__`` will be
called only once.


References
==========

. [2] The reference implementation
  (https://github.com/ilevkivskyi/cpython/pull/3/files)

. [3] Reserved classes of identifiers
  (https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers)


Copyright
=========

This document has been placed in the public domain.



.
  Local Variables:
  mode: indented-text
  indent-tabs-mode: nil
  sentence-end-double-space: t
  fill-column: 70
  coding: utf-8
  End:

You are viewing proxied material from gopher.fnord.one. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.