Lesson 28 • Advanced
Magic Methods & the Python Data Model
Master Python's powerful magic methods (dunder methods) and understand how the Python data model works under the hood. Learn to create custom classes that behave like built-in types, implement operator overloading, and build professional-grade objects.
Note: To run Python code on your computer, you'll need Python installed. Download Python here. Keep the website open and test our code examples with Python running on your computer.
🔥 1. What Are Magic Methods?
Magic methods are methods Python calls automatically when certain operations happen.
Examples:
| Operation | Magic Method |
|---|---|
len(obj) | __len__ |
obj + other | __add__ |
obj[i] | __getitem__ |
for x in obj | __iter__ |
str(obj) | __str__ |
with obj: | __enter__, __exit__ |
obj() | __call__ |
They let you define how your objects behave in every situation.
⚙️ 2. Object Construction (Creation & Initialization)
__new__ — create object
__init__ — initialize object
Object Construction
__new__ and __init__ methods
Python
class User:
def __new__(cls, *args, **kwargs):
print("Allocating memory...")
return super().__new__(cls)
def __init__(self, name):
print("Initializing...")
self.name = name
# Create a user
user = User("Alice")
print(f"User name: {user.name}")Use cases:
- ✔ custom immutable types
- ✔ singleton patterns
- ✔ building objects from cached pools
🧱 3. Representation Methods
These control how an object looks when printed or displayed.
__repr__ — official representation (for developers)
__repr__ Method
Developer-friendly representation
Python
class User:
def __init__(self, name):
self.name = name
def __repr__(self):
return f"User(name={self.name!r})"
user = User("Alice")
print(repr(user)) # User(name='Alice')__str__ — user-friendly representation
__str__ Method
User-friendly representation
Python
class User:
def __init__(self, name):
self.name = name
def __str__(self):
return self.name
user = User("Alice")
print(str(user)) # Alice
print(user) # Alice__format__ — formatting rules
Used for f-strings, formatting currencies, dates, metrics.
🔢 4. Numeric Magic Methods (Act Like Numbers!)
Implementing these turns objects into custom numeric types:
Addition
Numeric Operations
Vector addition example
Python
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
return Vector(self.x + other.x, self.y + other.y)
def __repr__(self):
return f"Vector({self.x}, {self.y})"
v1 = Vector(1, 2)
v2 = Vector(3, 4)
v3 = v1 + v2
print(v3) # Vector(4, 6)Also available: Subtraction, Multiplication, Division, Modulo, Power, Negation
Used in:
- ✔ game engines
- ✔ simulation math
- ✔ ML tensor wrappers (PyTorch does this heavily)
🧠 5. Comparisons (Sorting, Ordering, Equality)
__eq__— equals__ne__— not equal__lt__— less than__gt__— greater than__le__— <=__ge__— >=
Example:
Comparison Methods
Sorting with __lt__
Python
class Player:
def __init__(self, name, score):
self.name = name
self.score = score
def __lt__(self, other):
return self.score < other.score
def __repr__(self):
return f"{self.name}: {self.score}"
players = [Player("Alice", 100), Player("Bob", 150), Player("Charlie", 75)]
sorted_players = sorted(players)
for p in sorted_players:
print(p)Used for:
- ✔ leaderboard systems
- ✔ sorting objects
- ✔ priority queues
- ✔ ranking algorithms
📦 6. Container Protocol (Behaving Like Lists & Dicts)
__len__
Controls what len(obj) returns.
__getitem__
Container Protocol
List-like behavior
Python
class MyList:
def __init__(self, data):
self.data = data
def __len__(self):
return len(self.data)
def __getitem__(self, index):
return self.data[index]
my_list = MyList([1, 2, 3, 4, 5])
print(f"Length: {len(my_list)}")
print(f"First item: {my_list[0]}")
print(f"Slice: {my_list[1:3]}")Enables:
- ✔ slicing
- ✔ indexing
- ✔ iterating
Also: __setitem__, __delitem__
🔄 7. Iterable Protocol (for...in Loops)
To make an object iterable:
__iter__— return an iterator__next__— steps through values
Iterable Protocol
Custom iterator
Python
class Countdown:
def __init__(self, start):
self.start = start
def __iter__(self):
self._current = self.start
return self
def __next__(self):
if self._current <= 0:
raise StopIteration
self._current -= 1
return self._current + 1
for num in Countdown(5):
print(num)Used in:
- ✔ custom data streams
- ✔ tokenizers
- ✔ ML data loaders
- ✔ scrapers
- ✔ generators
🧩 8. Callable Objects (Pretend to Be Functions)
__call__
Callable Objects
__call__ method
Python
class Adder:
def __init__(self, n):
self.n = n
def __call__(self, x):
return x + self.n
# Now:
add5 = Adder(5)
print(add5(10)) # 15
print(add5(20)) # 25Used in:
- ✔ ML models (PyTorch layers implement __call__)
- ✔ function wrappers
- ✔ on-the-fly factories
- ✔ middleware
🔐 9. Attribute Access Control
__getattr__— fallback for missing attributes__getattribute__— intercept ALL attribute access__setattr__— intercept setting attributes__delattr__— intercept deletion
Example:
Attribute Access
__getattr__ fallback
Python
class FlexibleObject:
def __init__(self):
self.data = {}
def __getattr__(self, name):
return f"{name} is not defined"
obj = FlexibleObject()
print(obj.name) # name is not defined
print(obj.anything) # anything is not definedUsed for:
- ✔ lazy loading
- ✔ proxies (database lazy models in Django)
- ✔ dynamic API clients
- ✔ configuration wrappers
🧱 10. Boolean Value
__bool__
Boolean Value
__bool__ method
Python
class Container:
def __init__(self, items):
self.items = items
def __bool__(self):
return len(self.items) > 0
empty = Container([])
full = Container([1, 2, 3])
if full:
print("Container has items")
if not empty:
print("Container is empty")Now objects behave logically in:
- if statements
- while loops
- conditional checks
🧊 11. Context Managers
__enter__, __exit__
Context Manager
__enter__ and __exit__
Python
class Timer:
def __enter__(self):
import time
self.start = time.time()
print("Starting timer...")
return self
def __exit__(self, exc_type, exc, tb):
import time
elapsed = time.time() - self.start
print(f"Elapsed: {elapsed:.4f} seconds")
with Timer():
# Some operation
total = sum(range(1000000))
print(f"Sum: {total}")Used in:
- ✔ file handling
- ✔ database sessions
- ✔ locking systems
- ✔ resource guards
- ✔ timers
🧱 12. Copy & Serialization Hooks
__copy____deepcopy____getstate____setstate__
Control:
- how objects are serialized
- custom caching behavior
- saving/loading ML models
- multiprocessing transfers
🔥 13. Operator Overloading (Make Objects Behave Like Built-ins)
Python lets you define how your objects react to operators.
Arithmetic operators
__add__(self, other)— +__sub__(self, other)— -__mul__(self, other)— *__truediv__(self, other)— /__floordiv__(self, other)— //__mod__(self, other)— %__pow__(self, other)— **
Example: Vector math
Operator Overloading
Vector math example
Python
class Vector:
def __init__(self, x, y):
self.x, self.y = x, y
def __add__(self, other):
return Vector(self.x + other.x, self.y + other.y)
def __sub__(self, other):
return Vector(self.x - other.x, self.y - other.y)
def __mul__(self, scalar):
return Vector(self.x * scalar, self.y * scalar)
def __repr__(self):
return f"Vector({self.x}, {self.y})"
v1 = Vector(3, 4)
v2 = Vector(1, 2)
print(f"v1 + v2 = {v1 + v2}")
print(f"v1 - v2
...Operator overloading is used everywhere:
- ✔ game engines
- ✔ physics simulations
- ✔ ML tensors (PyTorch / TensorFlow)
- ✔ financial modeling
- ✔ vector graphics engines
🧱 14. Rich Comparisons — Smarter Sorting & Ranking
If you must support ordering, implement the following:
__lt__(self, other)— <__le__(self, other)— <=__gt__(self, other)— >__ge__(self, other)— >=__eq__(self, other)— ==__ne__(self, other)— !=
Example: sortable players by score
Rich Comparisons
Sorting and ranking
Python
class Player:
def __init__(self, name, score):
self.name = name
self.score = score
def __lt__(self, other):
return self.score < other.score
def __eq__(self, other):
return self.score == other.score
def __repr__(self):
return f"{self.name}({self.score})"
p1 = Player("Alice", 100)
p2 = Player("Bob", 150)
print(f"{p1} < {p2}: {p1 < p2}")
print(f"{p1} == {p2}: {p1 == p2}")🧩 15. Sequence Protocol (Full Custom List Behavior)
To behave like a real sequence, implement:
Required
__len__(self)__getitem__(self, index)
Optional (for mutability)
__setitem__(self, index, value)__delitem__(self, index)__contains__(self, item)
Example: Read-only sequence
Sequence Protocol
Read-only sequence
Python
class ReadOnlySeq:
def __init__(self, data):
self._data = tuple(data)
def __len__(self):
return len(self._data)
def __getitem__(self, i):
return self._data[i]
def __contains__(self, item):
return item in self._data
seq = ReadOnlySeq([1, 2, 3, 4, 5])
print(f"Length: {len(seq)}")
print(f"First: {seq[0]}")
print(f"3 in seq: {3 in seq}")🧠 16. Iterable vs Iterator (Clear Distinction)
Iterable
Has __iter__(), returns an iterator.
Iterator
Has __iter__() and __next__().
Why this matters:
- for loops depend on it
- generator pipelines rely on it
- async systems use async iterators
- ML dataloaders use custom iterables
🧱 17. Custom Iterators (Full Control of Data Streams)
Custom Iterator
Countdown iterator
Python
class Countdown:
def __init__(self, n):
self.n = n
def __iter__(self):
return self
def __next__(self):
if self.n <= 0:
raise StopIteration
self.n -= 1
return self.n + 1
for num in Countdown(5):
print(num)Used for:
- ✔ streaming input
- ✔ chunked database queries
- ✔ on-the-fly data generation
- ✔ scraper crawls
- ✔ batching ML data
🔐 18. Attribute Access Magic
Python gives complete control over attribute access.
__getattr__(self, name)— Occurs when attribute is missing__getattribute__(self, name)— Intercepts every attribute lookup__setattr__(self, name, value)— Intercepts setting attributes__delattr__(self, name)— Intercepts deletion
Attribute Access Magic
Dynamic attribute handling
Python
class DynamicObject:
def __getattr__(self, name):
return f"{name} does not exist"
obj = DynamicObject()
print(obj.foo) # foo does not exist
print(obj.bar) # bar does not exist🧩 19. Emulating Functions With __call__
Anything can behave like a function.
Callable Objects
Emulating functions
Python
class Multiply:
def __init__(self, factor):
self.factor = factor
def __call__(self, x):
return x * self.factor
# Usage:
double = Multiply(2)
triple = Multiply(3)
print(double(10)) # 20
print(triple(10)) # 30Used by:
- neural network layers
- preprocessing pipelines
- middleware wrappers
- job schedulers
- configurable callbacks
🧱 20. Context Manager Magic
Implement:
__enter__(self)__exit__(self, exc_type, exc, tb)
Context Manager
Database connection example
Python
class DatabaseConnection:
def __enter__(self):
print("Opening connection...")
return self
def __exit__(self, exc, val, tb):
print("Closing connection...")
def query(self, sql):
print(f"Executing: {sql}")
with DatabaseConnection() as db:
db.query("SELECT * FROM users")🔥 21. Descriptors — The Hidden Power Behind @property
Descriptors define how attributes behave.
A descriptor is any object defining one or more of:
__get__(self, instance, owner)__set__(self, instance, value)__delete__(self, instance)
Descriptors
Custom attribute behavior
Python
class Logged:
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
if instance is None:
return self
print(f"Accessed {self.name}")
return instance.__dict__.get(self.name)
def __set__(self, instance, value):
print(f"Setting {self.name} to {value}")
instance.__dict__[self.name] = value
class User:
name = Logged()
u = User()
u.name = "Alice"
print(u.name)🧬 22. Properties Built on Top of Descriptors
property is just a wrapper around descriptors.
Properties
Using @property decorator
Python
class User:
def __init__(self, name):
self._name = name
@property
def name(self):
return self._name
@name.setter
def name(self, value):
if not value:
raise ValueError("Name cannot be empty")
self._name = value
u = User("Alice")
print(u.name)
u.name = "Bob"
print(u.name)⚡ 23. Slots — Memory-Efficient Objects
Use __slots__ to avoid dynamic dictionaries:
Slots
Memory-efficient objects
Python
class Point:
__slots__ = ("x", "y")
def __init__(self, x, y):
self.x = x
self.y = y
p = Point(1, 2)
print(f"Point: ({p.x}, {p.y})")
# This would raise AttributeError:
# p.z = 3 # Can't add new attributes!Benefits:
- ✔ lower memory
- ✔ faster attribute access
- ✔ prevents accidental attributes
⚡ 24. Understanding the Python Object Lifecycle
Every Python object goes through:
- Allocation →
__new__ - Initialization →
__init__ - Destruction →
__del__
Singleton Pattern
Object lifecycle control
Python
class Singleton:
_instance = None
def __new__(cls):
if not cls._instance:
print("Creating new instance")
cls._instance = super().__new__(cls)
else:
print("Returning existing instance")
return cls._instance
s1 = Singleton()
s2 = Singleton()
print(f"Same object: {s1 is s2}")🔥 25. Metaclasses — The Most Advanced Python Feature
A metaclass is the class of a class.
Classes create objects. Metaclasses create classes.
Default metaclass: type
Metaclasses
Creating classes dynamically
Python
class Meta(type):
def __new__(mcls, name, bases, attrs):
print(f"Creating class {name}")
return super().__new__(mcls, name, bases, attrs)
# Usage:
class User(metaclass=Meta):
pass
class Admin(User):
passWhat metaclasses are used for IN REAL SYSTEMS
- ✔ Django ORM
- ✔ SQLAlchemy Models
- ✔ Pydantic / FastAPI Models
- ✔ TensorFlow Layers
- ✔ Enum internals
- ✔ Plugin systems
- ✔ Service registries
🧩 26. Class Decorators vs Metaclasses
Class decorators modify the class after it's created:
Class Decorators
Modifying classes after creation
Python
registry = []
def register(cls):
registry.append(cls)
return cls
@register
class User:
pass
@register
class Admin:
pass
print(f"Registered classes: {registry}")Metaclasses modify the class while being created.
When to use what:
- Class decorator → simple modification
- Metaclass → structural modification
🧠 27. Emulating Containers (Full Custom Collections)
Custom Collections
Bounded list example
Python
class BoundedList:
def __init__(self, limit):
self.data = []
self.limit = limit
def __len__(self):
return len(self.data)
def __getitem__(self, i):
return self.data[i]
def append(self, val):
if len(self.data) >= self.limit:
raise ValueError("List full")
self.data.append(val)
def __repr__(self):
return f"BoundedList({self.data})"
bl = BoundedList(3)
bl.append(1)
bl.append(2)
bl.append(3)
print(bl)
# bl.app
...🧱 28. Making Your Objects Hashable
For objects to be used as dictionary keys OR in sets:
Implement:
__hash____eq__
Hashable Objects
Dictionary keys and sets
Python
class Point:
__slots__ = ("x", "y")
def __init__(self, x, y):
self.x = x
self.y = y
def __hash__(self):
return hash((self.x, self.y))
def __eq__(self, other):
return self.x == other.x and self.y == other.y
def __repr__(self):
return f"Point({self.x}, {self.y})"
# Now can use as dict keys
points = {Point(0, 0): "origin", Point(1, 1): "diagonal"}
print(points[Point(0, 0)])🔍 29. Overriding Truthiness & Boolean Behavior
Boolean Behavior
Custom truthiness
Python
class Connection:
def __init__(self, status):
self.status = status
def __bool__(self):
return self.status == "ready"
conn1 = Connection("ready")
conn2 = Connection("disconnected")
if conn1:
print("Connection 1 is ready")
if not conn2:
print("Connection 2 is not ready")🧬 30. Controlling String Representations
String Representations
__repr__ vs __str__
Python
class User:
def __init__(self, id, name):
self.id = id
self.name = name
def __repr__(self):
return f"User(id={self.id})"
def __str__(self):
return self.name
u = User(1, "Alice")
print(repr(u)) # User(id=1)
print(str(u)) # Alice
print(u) # Alice (uses __str__)⚙️ 31-34. Advanced Topics
Additional advanced magic method patterns:
- Dynamic Attribute Computation - Using __getattr__ for lazy loading
- Proxy Objects - Forwarding magic methods
- Custom Number Types - Money, Temperature classes
- Full Domain Models - Complete custom types
Custom Money Type
Full domain model
Python
class Money:
def __init__(self, amount):
self.amount = amount
def __add__(self, other):
return Money(self.amount + other.amount)
def __mul__(self, rate):
return Money(self.amount * rate)
def __repr__(self):
return "Money(" + str(self.amount) + ")"
m1 = Money(100)
m2 = Money(50)
print("m1 + m2 =", m1 + m2)
print("m1 * 1.5 =", m1 * 1.5)🎓 35. Final Summary — You Now Understand the Entire Python Data Model
By mastering this lesson, you now understand:
- ✔ operator overloading
- ✔ full comparison system
- ✔ container emulation
- ✔ iterator/iterable protocols
- ✔ attribute access magic
- ✔ callable classes
- ✔ context manager internals
- ✔ descriptor protocol
- ✔ property internals
- ✔ slots memory optimization
- ✔ object lifecycle
- ✔ metaclass architecture
- ✔ custom domain-specific types
- ✔ proxy patterns
- ✔ advanced debugging behaviors
You now write Python the way framework authors, not beginners, write it.
This knowledge places you firmly at the top 1% of Python engineers.
📋 Quick Reference — Magic Methods
| Method | Triggered by |
|---|---|
| __init__(self) | Object creation: MyClass() |
| __repr__(self) | repr() and interactive shell display |
| __len__(self) | len(obj) |
| __getitem__(self, key) | obj[key] indexing |
| __enter__ / __exit__ | with obj: context manager |
🎉 Great work! You've completed this lesson.
You now control how your objects respond to built-in Python operations — the same protocol powering NumPy, pandas, and SQLAlchemy.
Up next: Operator Overloading — make your custom types support +, -, <, and other operators naturally.
Sign up for free to track which lessons you've completed and get learning reminders.