Advanced Lesson
Metaprogramming & Introspection
Master Python's introspection tools, inspect module, dynamic code generation, and metaclasses
What Is Metaprogramming?
Metaprogramming means writing code that treats functions, classes, and modules as data and manipulates them.
- Automatically registering classes
- Inspecting function signatures
- Modifying functions with decorators
- Generating methods dynamically
- Building plugin systems
- Auto-validating data
Python makes this easy because functions, classes, and modules are all objects that can be inspected and modified.
Introspection Basics: dir(), type(), vars()
Python provides built-in functions for basic introspection.
Basic Introspection
Use dir(), type(), vars(), and attribute functions
# Basic introspection tools
class User:
def __init__(self, name, age):
self.name = name
self.age = age
def greet(self):
return f"Hello, I'm {self.name}"
user = User("Alice", 30)
# dir() - shows all attributes and methods
print("Attributes and methods:")
print([attr for attr in dir(user) if not attr.startswith('_')])
# type() - returns the class
print(f"\nType: {type(user)}")
print(f"Class name: {type(user).__name__}")
# vars() - returns object's __dict__
...The inspect Module
The inspect module provides powerful tools for analyzing Python objects deeply.
Inspect Module
Get function signatures, source code, and docstrings
import inspect
def greet(name: str, greeting: str = "Hello") -> str:
"""Greet someone with a custom message."""
return f"{greeting}, {name}!"
# Get function signature
sig = inspect.signature(greet)
print("Signature:", sig)
# Get parameters
print("\nParameters:")
for param_name, param in sig.parameters.items():
print(f" {param_name}:")
print(f" default: {param.default}")
print(f" annotation: {param.annotation}")
# Check return annotation
print(f"\nReturn type: {sig.
...Inspecting Function Signatures
Extract detailed information about function parameters, defaults, and type hints.
Function Signatures
Analyze function parameters and their kinds
import inspect
def complex_function(
required: str,
optional: int = 10,
*args: str,
keyword_only: bool,
with_default: float = 3.14,
**kwargs: dict
) -> list:
"""Complex function with various parameter types."""
pass
sig = inspect.signature(complex_function)
print("Detailed parameter analysis:\n")
for name, param in sig.parameters.items():
print(f"Parameter: {name}")
print(f" Kind: {param.kind}")
print(f" Default: {param.default}")
print(f" A
...Detecting Types of Objects
Inspect can identify exactly what kind of object you're dealing with.
Type Detection
Identify functions, classes, coroutines, and generators
import inspect
def regular_function():
pass
async def async_function():
pass
def generator_function():
yield 1
class MyClass:
def method(self):
pass
obj = MyClass()
# Type detection
checks = [
("regular_function", regular_function),
("async_function", async_function),
("generator_function", generator_function),
("MyClass", MyClass),
("obj", obj),
]
print("Object type detection:\n")
for name, item in checks:
print(f"{name}:")
print(f" i
...Getting the Call Stack
Inspect where your code is being called from - essential for debugging and logging.
Call Stack Inspection
Examine the call stack and frame information
import inspect
def show_caller_info():
"""Show information about who called this function."""
# stack[0] is this function
# stack[1] is the caller
frame_info = inspect.stack()[1]
print(f"Called from function: {frame_info.function}")
print(f" File: {frame_info.filename}")
print(f" Line: {frame_info.lineno}")
print(f" Code: {frame_info.code_context[0].strip()}")
def level_three():
print("Level 3:")
show_caller_info()
def level_two():
print("\n
...Dynamic Function Generation
Create functions at runtime based on configuration or input.
Dynamic Functions
Create functions dynamically using factory patterns
import inspect
# Factory pattern - create functions dynamically
def create_validator(field_name, field_type):
"""Generate a validation function for a field."""
def validator(value):
if not isinstance(value, field_type):
raise TypeError(
f"{field_name} must be {field_type.__name__}, "
f"got {type(value).__name__}"
)
return value
# Set a helpful name
validator.__name__ = f"validate_{field_name}"
retur
...Intercepting Attribute Access
Use __getattr__, __setattr__ to intercept and control attribute access.
Attribute Interception
Control attribute access with __getattr__ and __setattr__
# Lazy loading with __getattr__
class LazyConfig:
"""Config that generates values on first access."""
def __getattr__(self, name):
if name.startswith('db_'):
# Simulate loading from database
value = f"<loaded {name} from DB>"
# Cache it
setattr(self, name, value)
return value
raise AttributeError(f"'{name}' not found")
config = LazyConfig()
print("First access (loads from DB):", config.db_host)
print("Secon
...Function Metadata & Annotations
Functions carry metadata that frameworks use for validation and documentation.
Function Metadata
Access function annotations, docstrings, and defaults
def api_endpoint(
user_id: int,
include_posts: bool = False,
page: int = 1
) -> dict:
"""
Fetch user data from the API.
Args:
user_id: The ID of the user
include_posts: Whether to include user's posts
page: Page number for pagination
Returns:
User data dictionary
"""
return {"id": user_id, "posts": include_posts, "page": page}
# Access function metadata
print("Function metadata:\n")
print(f"Name: {api_endpoint.__name_
...Inspecting Closures
Examine captured variables in closures - how decorators store state.
Closure Inspection
Examine captured variables in closures
import inspect
import time
def create_counter(start=0, step=1):
"""Create a counter function with captured state."""
count = start
def counter():
nonlocal count
count += step
return count
return counter
# Create counters with different captured values
counter1 = create_counter(0, 1)
counter2 = create_counter(100, 10)
print("Counting:")
print(counter1()) # 1
print(counter1()) # 2
print(counter2()) # 110
print(counter2()) # 120
# Inspect th
...Building a Plugin System
Use introspection to automatically discover and register plugins.
Plugin System
Auto-discover and register plugins
import inspect
# Base class for plugins
class Plugin:
"""Base plugin class."""
name = "base"
def execute(self):
raise NotImplementedError
# Plugin implementations
class EmailPlugin(Plugin):
name = "email"
def execute(self):
return "Sending email..."
class SMSPlugin(Plugin):
name = "sms"
def execute(self):
return "Sending SMS..."
class SlackPlugin(Plugin):
name = "slack"
def execute(self):
return "Posting
...Metaclasses — Advanced Class Creation
Metaclasses intercept class creation itself, enabling automatic registration and validation.
Metaclasses
Control class creation with metaclasses
# Metaclass for auto-registration
class RegistryMeta(type):
"""Metaclass that auto-registers all classes."""
registry = {}
def __new__(mcls, name, bases, namespace):
# Create the class
cls = super().__new__(mcls, name, bases, namespace)
# Register it (skip base class)
if bases: # Has parent, so it's a real class
RegistryMeta.registry[name] = cls
print(f"Registered: {name}")
return cls
class Base(
...Descriptors — Custom Attribute Behavior
Descriptors control how attributes are accessed, enabling typed fields and validation.
Descriptors
Create type-checked fields with descriptors
# Type-checked descriptor
class TypedField:
"""Descriptor that enforces type checking."""
def __init__(self, field_type, default=None):
self.field_type = field_type
self.default = default
self.name = None
def __set_name__(self, owner, name):
"""Called when descriptor is assigned to a class attribute."""
self.name = name
def __get__(self, instance, owner):
"""Called when attribute is accessed."""
if instance is
...Auto-Generating Documentation
Use introspection to automatically generate documentation from code.
Auto Documentation
Generate documentation from code introspection
import inspect
class DataProcessor:
"""Main data processing class."""
def load(self, path: str) -> list:
"""
Load data from file.
Args:
path: File path to load from
Returns:
List of data records
"""
return []
def transform(self, data: list, operation: str = "clean") -> list:
"""
Transform data with specified operation.
Args:
data: Input dat
...Building a Dynamic API Router
Use introspection to auto-discover and route API endpoints.
Dynamic API Router
Auto-discover and route API endpoints
import inspect
# Decorator to mark functions as routes
def route(path):
"""Decorator that marks a function as an API route."""
def decorator(func):
func.route_path = path
func.route_method = "GET"
return func
return decorator
# Define some API endpoints
@route("/users")
def list_users():
"""List all users."""
return {"users": ["Alice", "Bob"]}
@route("/users/{user_id}")
def get_user(user_id: int):
"""Get a specific user."""
return {"id": use
...Summary
You've mastered Python's metaprogramming and introspection capabilities:
- Basic introspection with dir(), type(), vars()
- The inspect module for deep code analysis
- Inspecting function signatures and parameters
- Detecting object types (functions, classes, coroutines)
- Examining call stacks and frames
- Dynamic function generation and factories
- Intercepting attribute access with __getattr__ and __setattr__
- Inspecting closures and captured variables
- Building plugin systems with auto-discovery
- Metaclasses for controlling class creation
- Descriptors for custom attribute behavior
- Auto-generating documentation from code
- Building dynamic API routers
These techniques power major frameworks like Django, FastAPI, SQLAlchemy, pytest, and Pydantic. Metaprogramming enables you to build flexible, self-documenting systems that adapt to code structure automatically.
📋 Quick Reference — Metaprogramming
| Tool / Syntax | What it does |
|---|---|
| type(name, bases, attrs) | Create a class dynamically |
| class Meta(type): | Define a metaclass |
| getattr(obj, 'name') | Get attribute by name string |
| hasattr(obj, 'name') | Check if attribute exists |
| inspect.signature(fn) | Inspect function parameters at runtime |
🎉 Great work! You've completed this lesson.
You can now write code that introspects and modifies other code at runtime — the power behind pytest, FastAPI, Django ORM, and Pydantic.
Up next: Decorator Libraries — build reusable decorator toolkits for production code.
Sign up for free to track which lessons you've completed and get learning reminders.