Introspection in Python

Discussion

• What built-in functions facilitate object introspection in Python?

  The following built-in functions are useful:

  • callable(): Returns True if object is callable. Eg. callable(mycar). Objects can be made callable by defining __call__ method in their class.
  • dir(): Obtain all attributes (data and methods) of an object. Eg. dir(mycar).
  • hasattr(): Check if object has an attribute. Eg. hasattr(mycar, 'num_gears'). This is useful since attributes can be dynamically added/removed at runtime. Note that if 'num_gears' in dir(mycar) is equivalent to hasattr(mycar, 'num_gears').
  • help(): Obtain help on an object's interface based on docstrings.
  • id(): Obtain the unique ID of the object. Eg. id(mycar).
  • issubclass(): Check if a class is a subclass of a specified class. Eg. issubclass(Honda, Car).
  • isinstance(): Check if an object is of a specified class. Eg. isinstance(mycar, Honda) will be True when mycar = Honda(). If Honda is a subclass of Car, isinstance(mycar, Car) is also True. Note that type(mycar) is Car is False.
  • sys.getsizeof(): Part of the sys module, this obtains the size of the object in bytes.
  • type(): Obtain the object's type. Eg. type(2) will give <class 'int'>.
  • vars(): Obtain all instance variables (name and value) as a dictionary. Eg. vars(mycar). This is equivalent to mycar.__dict__.
• What object attributes help with introspection in Python?

  There are some attributes that give useful information about objects. For example, given an instance mycar, mycar.__class__ retrieves its class. Attributes of the class give more information: __name__, __qualname__, __str__, __repr__, __doc__, and __self__. We briefly describe these:

  • __name__: Original name of the class, function or method.
  • __qualname__: Qualified name of the class function or method. This is often useful where class/function/method definitions are nested.
  • __doc__: Documentation string, which can also be retrieved by calling built-in function help().
  • __self__: Instance to which a method is bound.
• Could you describe Python's inspect module?

  Python module inspect provides many functions to enable introspection. The module documentation notes that it provides four main kinds of services: "type checking, getting source code, inspecting classes and functions, and examining the interpreter stack."

  Inspect documentation has full details. We note some functions:

  • Checking types: ismodule(), isclass(), ismethod(), isfunction(), isgenerator(), isabstract(), isbuiltin(), isawaitable(), and many more.
  • Inspecting objects: getmembers() is quite useful. For example, inspect.getmembers(mycar, inspect.ismethod) will return only methods of the instance mycar. Function getfullargspec() returns names and default values of a function. These can be nicely formatted using formatargspec(). Callables can be inspected using signature(). Class definitions relevant here are Signature, Parameter and BoundArguments.
  • Retrieving source code: getdoc(), getcomments(), getfile(), getmodule(), getsourcefile(), getsourcelines(), getsource(), and cleandoc().
  • Examining the stack: getframeinfo(), getouterframes(), getinnerframes(), currentframe(), stack() and trace().
• How can I obtain the current runtime context in Python?

  

  Illustrating the Python call stack. Source: Bagheri 2020.

  The Python interpreter maintains a stack of frames. The bottom-most frame is called the global frame or module frame. When a function is called, a new frame is created and pushed to the stack. When that function returns, the frame is popped off the stack.

  Python's inspect module provides many functions to inspect the interpreter stack. Function currentframe() returns a frame object representing the current frame. More information about the frame object can be obtained by calling getframeinfo(), which returns a named tuple Traceback(filename, lineno, function, code_context, index).

  To obtain the entire stack, call stack() instead. To inspect the call graph, we can use getouterframes(). When an exception is raised and later handled in an outer frame, we can use trace() or getinnerframes(). These can be useful for logging and debugging.

  Apart from inspecting the stack, two built-in functions help us inspect variable scope. These are globals() and locals(). At the module level, both functions return the same dictionary.

• How can I inspect class hierarchy in Python?

  

  Inspecting the class hierarchy of an instance using __class__ instance attribute and __bases__ class attribute. Source: Adapted from java2s 2021.

  Class hierarchy can easily be inspected by accessing the class attribute __bases__ and recursively printing the class names up the hierarchy. If an instance is being inspected, we can first get to its class via the __class__ attribute and then process the __bases__ attribute. In the figure, we show an example of how this can be done.

  The method inspect.getclasstree() from inspect module is an alternative. Given a list of classes, it returns the class hierarchy. For every class, its base classes are returned as a tuple.

  Another useful method is inspect.getmro() to which a class name must be passed as an argument. This is equivalent to calling the built-in class method mro(). Either way, these don't return the inheritance hierarchy but they do tell us the order in which methods are looked up the hierarchy.

• What are some specialized modules for Python introspection?

  

  Function functools.wraps() helps with inspecting decorated functions. Source: New Relic Docs 2021.

  Python processes source code in three steps: (i) parsing, where it tokenizes the source code and produces an Abstract Syntax Tree (AST); (ii) compiling, where it transforms the AST into Python bytecode; (iii) interpreting, where the Python Virtual Machine (VM) interprets and executes the bytecode. In this context, there are two Python modules that help with introspection: ast to inspect the AST, and dis to inspect the bytecode.

  While inspect module has functions to inspect the call stack, sys module also has functions that can help with debugging recursive calls: _getframe(), getrecursionlimit() and setrecursionlimit().

  Support we decorate the function foo(). Accessing foo.__name__ will return the name of the function returned by the decorator. This is probably not what we want. We can solve this with the wraps() function from the functools module. The figure shows an example how to use this. An alternative is to use inspect.signature() function.

• Could you share some tips on using Python's introspection features?

  While type() and isinstance() could be useful, code that rely on them are hard to maintain. We might be checking for specific types but when another type must be handled, we need to update the code. Instead, Python programmers prefer the approach of duck typing.

  With duck typing, we don't care about an object's type. Instead, we focus on the interfaces the object exposes. In this approach, hasattr() can be useful though this method can cause problems in Python 2 when called on a property attribute.

  An alternative to hasattr() is to simply call a method or access a data attribute without doing any checks. If it doesn't exist, we should handle the exception. Another alternative is to use an Abstract Base Class (ABC) from the collections.abc module.

Milestones

References

1. Bagheri, Reza. 2020. "Python Stack Frames and Tail-Call Optimization." Towards Data Science, on Medium, April 25. Accessed 2021-12-15.
2. Bullock, Jamie. 2020. "Python Reflection and Introspection." Better Programming, on Medium, January 16. Accessed 2021-11-21.
3. Carattino, Aquiles. 2019. "Duck Typing, or how to check variable types." Blog, Python for the Lab, June 11. Accessed 2021-12-15.
4. Cui, Yong. 2020. "The 5 Most Useful Introspection Functions in Python." Better Programming, on Medium, February 8. Accessed 2021-12-15.
5. Goyal, Rohit. 2018. "A Guide To Object Introspection in Python." Blog, Zeolearn, September 28. Accessed 2021-12-15.
6. Hellmann, Doug. 2018. "inspect – Inspect Live Objects." PyMOTW-3, March 18. Accessed 2021-12-15.
7. Kettler, Rafe. 2015. "A Guide to Python's Magic Methods." Version 1.17, September 4. Accessed 2021-12-15.
8. Kuchling, A.M. 2001. "What’s New in Python 2.1." Release 2.1, April 17. Updated 2021-12-15. Accessed 2021-12-15.
9. New Relic Docs. 2021. "Python tips and tricks." Documentation, APM, New Relic, October 21. Accessed 2021-12-15.
10. Pitrou, Antoine. 2011. "PEP 3155 -- Qualified name for classes and functions." Python.org, October 29. Updated 2021-02-09. Accessed 2021-11-21.
11. Pranskevichus, Elvis, and Yury Selivanov. 2015. "What’s New in Python 3.5." Release 3.5, September 13. Updated 2021-12-15. Accessed 2021-11-23.
12. PyPI. 2016. "funcsigs." 1.0.2, PyPI, April 26. Accessed 2021-12-15.
13. Python Docs. 2012. "What’s New in Python 3.3." Release 3.3, September 29. Updated 2021-11-23. Accessed 2021-11-23.
14. Python Docs. 2021a. "inspect — Inspect live objects." The Python Standard Library, v3.10.1, December 13. Accessed 2021-12-15.
15. Python Docs. 2021b. "Built-in Functions." The Python Standard Library, v3.10.1, December 13. Accessed 2021-12-15.
16. Python Docs. 2021c. "Glossary." Documentation, v3.10.1, December 13. Accessed 2021-12-15.
17. Schlawack, Hynek. 2016. "hasattr() – A Dangerous Misnomer." Blog, January 13. Updated 2016-02-10. Accessed 2021-12-15.
18. Shaun. 2020. "Python objects speak for themselves." Blog, Anvil Works, November 19. Accessed 2021-12-15.
19. Sitaker, Kragen Javier. 2002. "isinstance() considered harmful." Blog, March 1. Updated 2016-11-30. Accessed 2021-12-15.
20. java2s. 2021. "Print out class tree : Class Inheritance « Class « Python." java2s. Accessed 2021-12-15.
21. van Rossum, Guido. 2009. "A Brief Timeline of Python." Blog, The History of Python, January 20. Accessed 2021-11-23.

Further Reading

1. Cui, Yong. 2020. "The 5 Most Useful Introspection Functions in Python." Better Programming, on Medium, February 8. Accessed 2021-12-15.
2. Shaun. 2020. "Python objects speak for themselves." Blog, Anvil Works, November 19. Accessed 2021-12-15.
3. Wikibooks. 2020. "Python Programming/Reflection." Wikibooks, April 16. Accessed 2021-11-21.
4. Python Docs. 2021a. "inspect — Inspect live objects." The Python Standard Library, v3.10.1, December 13. Accessed 2021-12-15.
5. Python Docs. 2021b. "Built-in Functions." The Python Standard Library, v3.10.1, December 13. Accessed 2021-12-15.

Article Stats

Author-wise Stats for Article Edits

Author

No. of Edits

No. of Chats

DevCoins

arvindpdmn

3

0

1584

DevCoins due to articles, chats, their likes and article hits are included.

Cite As