Lesson 1: Advanced Functions and Decorators#
Master advanced function concepts that separate intermediate from expert Python developers.
🎯 Learning Objectives#
By the end of this lesson, you will:
Understand first-class functions and functional programming in Python
Master closures and variable scope (LEGB rule)
Create and use decorators effectively
Build advanced decorator patterns (with arguments, chaining, classes)
Apply decorators to solve real-world problems
Understand performance implications and best practices
📚 Prerequisites#
Solid understanding of Python functions
Familiarity with
*argsand**kwargsBasic understanding of nested functions
🌟 Why This Matters#
Advanced function concepts are foundational to:
Web frameworks: Flask/Django use decorators for routing (
@app.route('/home'))Testing: pytest uses decorators for fixtures and markers
APIs: FastAPI uses decorators extensively for endpoints
Performance: Caching, memoization, profiling
Code organization: Separation of concerns, DRY principle
Part 1: First-Class Functions#
In Python, functions are first-class citizens - they can be:
Assigned to variables
Passed as arguments to other functions
Returned from functions
Stored in data structures
# Functions are objects
def greet(name):
return f"Hello, {name}!"
# Assign function to variable
say_hello = greet
print(say_hello("Alice")) # Hello, Alice!
# Store functions in data structures
operations = {
'greet': greet,
'upper': str.upper,
'lower': str.lower
}
print(operations['greet']("Bob")) # Hello, Bob!
print(operations['upper']("hello")) # HELLO
Higher-Order Functions#
Functions that:
Take other functions as arguments, OR
Return functions as results
def apply_operation(func, x, y):
"""
Higher-order function: takes a function as argument.
"""
return func(x, y)
# Pass different functions
def add(a, b):
return a + b
def multiply(a, b):
return a * b
print(apply_operation(add, 5, 3)) # 8
print(apply_operation(multiply, 5, 3)) # 15
# Using lambda functions (anonymous functions)
print(apply_operation(lambda a, b: a ** b, 2, 3)) # 8 (2^3)
Built-in Higher-Order Functions#
Python provides several powerful built-in higher-order functions.
from functools import reduce
numbers = [1, 2, 3, 4, 5]
# map(function, iterable) - apply function to each element
squared = list(map(lambda x: x**2, numbers))
print(f"Squared: {squared}")
# Output: [1, 4, 9, 16, 25]
# filter(function, iterable) - keep elements where function returns True
evens = list(filter(lambda x: x % 2 == 0, numbers))
print(f"Evens: {evens}")
# Output: [2, 4]
# reduce(function, iterable) - accumulate values
product = reduce(lambda x, y: x * y, numbers)
print(f"Product: {product}")
# Output: 120 (1*2*3*4*5)
# sorted with custom key function
words = ['apple', 'pie', 'zoo', 'a']
by_length = sorted(words, key=len)
print(f"Sorted by length: {by_length}")
# Output: ['a', 'pie', 'zoo', 'apple']
Practical Example: Strategy Pattern#
Use first-class functions to implement different algorithms.
def calculate_discount(price, discount_strategy):
"""
Calculate price with discount using a strategy function.
"""
return discount_strategy(price)
# Different discount strategies
def percentage_discount(percent):
return lambda price: price * (1 - percent/100)
def fixed_discount(amount):
return lambda price: max(0, price - amount)
def bulk_discount(min_qty, discount_percent):
def discount(price):
# Assumes price represents total for multiple items
return price * (1 - discount_percent/100)
return discount
# Usage
price = 100
print(f"Original: ${price}")
print(f"20% off: ${calculate_discount(price, percentage_discount(20))}")
print(f"$15 off: ${calculate_discount(price, fixed_discount(15))}")
print(f"Bulk 30% off: ${calculate_discount(price, bulk_discount(10, 30))}")
Part 2: Closures#
A closure is a function that:
Is defined inside another function (nested)
References variables from the enclosing function’s scope
Can be returned and used later, still remembering those variables
The LEGB Rule#
Python searches for variables in this order:
Local - inside current function
Enclosing - in enclosing functions (closures)
Global - module level
Built-in - Python’s built-in names
# Simple closure example
def make_multiplier(factor):
"""
Returns a function that multiplies by 'factor'.
The returned function 'closes over' the factor variable.
"""
def multiplier(x):
return x * factor # 'factor' from enclosing scope
return multiplier
# Create specialized functions
double = make_multiplier(2)
triple = make_multiplier(3)
times_ten = make_multiplier(10)
print(double(5)) # 10
print(triple(5)) # 15
print(times_ten(5)) # 50
# Each function remembers its own 'factor'
print(double.__closure__[0].cell_contents) # 2
print(triple.__closure__[0].cell_contents) # 3
Practical Closure: Private State#
Use closures to create functions with private state.
def make_counter(start=0):
"""
Create a counter with private state.
"""
count = start # Private variable
def increment():
nonlocal count # Modify enclosing scope variable
count += 1
return count
def decrement():
nonlocal count
count -= 1
return count
def get_count():
return count
# Return multiple functions
return increment, decrement, get_count
# Create counter
inc, dec, get = make_counter(10)
print(get()) # 10
print(inc()) # 11
print(inc()) # 12
print(dec()) # 11
print(get()) # 11
# Can't access 'count' directly - it's private!
# print(count) # NameError
Closure with Multiple Variables#
def make_accumulator():
"""
Create an accumulator that tracks sum and count.
"""
total = 0
count = 0
def add(value):
nonlocal total, count
total += value
count += 1
return total
def average():
return total / count if count > 0 else 0
def reset():
nonlocal total, count
total = 0
count = 0
return add, average, reset
# Usage
add_value, get_avg, reset_acc = make_accumulator()
add_value(10)
add_value(20)
add_value(30)
print(f"Average: {get_avg()}") # 20.0
reset_acc()
print(f"After reset: {get_avg()}") # 0
Part 3: Decorators Fundamentals#
A decorator is a function that:
Takes a function as input
Returns a new function (usually wrapping the original)
Is applied using
@decorator_namesyntax
Why Use Decorators?#
Add functionality without modifying original code
Keep code DRY (Don’t Repeat Yourself)
Separate concerns (logging, timing, authentication, etc.)
Make code more readable and maintainable
# Basic decorator
def simple_decorator(func):
"""
A decorator that prints before and after function execution.
"""
def wrapper(*args, **kwargs):
print(f"Before calling {func.__name__}")
result = func(*args, **kwargs)
print(f"After calling {func.__name__}")
return result
return wrapper
# Apply decorator using @ syntax
@simple_decorator
def say_hello(name):
print(f"Hello, {name}!")
return f"Greeted {name}"
# This is equivalent to:
# say_hello = simple_decorator(say_hello)
result = say_hello("Alice")
print(f"Returned: {result}")
Common Decorator Pattern: Timing#
import time
from functools import wraps
def timing_decorator(func):
"""
Measure and print function execution time.
"""
@wraps(func) # Preserves original function metadata
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f"{func.__name__} took {end - start:.6f} seconds")
return result
return wrapper
@timing_decorator
def slow_function():
time.sleep(0.1)
return "Done"
@timing_decorator
def calculate_sum(n):
return sum(range(n))
slow_function()
calculate_sum(1000000)
Why Use @wraps?#
Without @wraps, decorated functions lose their metadata.
# Without @wraps
def bad_decorator(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
# With @wraps
def good_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@bad_decorator
def function_a():
"""This is function A."""
pass
@good_decorator
def function_b():
"""This is function B."""
pass
print(f"function_a.__name__: {function_a.__name__}") # 'wrapper' ❌
print(f"function_b.__name__: {function_b.__name__}") # 'function_b' ✅
print(f"function_a.__doc__: {function_a.__doc__}") # None ❌
print(f"function_b.__doc__: {function_b.__doc__}") # 'This is function B.' ✅
Part 4: Practical Decorators#
Real-world decorator examples you’ll use in production code.
1. Logging Decorator#
import logging
from functools import wraps
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def log_calls(func):
"""
Log function calls with arguments and return values.
"""
@wraps(func)
def wrapper(*args, **kwargs):
args_repr = [repr(a) for a in args]
kwargs_repr = [f"{k}={v!r}" for k, v in kwargs.items()]
signature = ", ".join(args_repr + kwargs_repr)
logger.info(f"Calling {func.__name__}({signature})")
result = func(*args, **kwargs)
logger.info(f"{func.__name__} returned {result!r}")
return result
return wrapper
@log_calls
def add(a, b):
return a + b
@log_calls
def greet(name, greeting="Hello"):
return f"{greeting}, {name}!"
add(5, 3)
greet("Alice", greeting="Hi")
2. Validation Decorator#
def validate_positive(func):
"""
Ensure all numeric arguments are positive.
"""
@wraps(func)
def wrapper(*args, **kwargs):
# Check positional arguments
for arg in args:
if isinstance(arg, (int, float)) and arg < 0:
raise ValueError(f"Expected positive number, got {arg}")
# Check keyword arguments
for key, value in kwargs.items():
if isinstance(value, (int, float)) and value < 0:
raise ValueError(f"Expected positive {key}, got {value}")
return func(*args, **kwargs)
return wrapper
@validate_positive
def calculate_area(width, height):
return width * height
print(calculate_area(5, 10)) # 50
try:
calculate_area(-5, 10)
except ValueError as e:
print(f"Error: {e}")
3. Retry Decorator#
import time
from functools import wraps
def retry(max_attempts=3, delay=1):
"""
Retry function on failure.
"""
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
attempts = 0
while attempts < max_attempts:
try:
return func(*args, **kwargs)
except Exception as e:
attempts += 1
if attempts == max_attempts:
raise
print(f"Attempt {attempts} failed: {e}. Retrying in {delay}s...")
time.sleep(delay)
return wrapper
return decorator
# Simulate unreliable function
attempt_count = 0
@retry(max_attempts=3, delay=0.5)
def unreliable_api_call():
global attempt_count
attempt_count += 1
if attempt_count < 3:
raise ConnectionError("API unavailable")
return "Success!"
result = unreliable_api_call()
print(result)
4. Cache/Memoization Decorator#
from functools import wraps
import time
def memoize(func):
"""
Cache function results based on arguments.
"""
cache = {}
@wraps(func)
def wrapper(*args):
if args in cache:
print(f"Cache hit for {func.__name__}{args}")
return cache[args]
print(f"Computing {func.__name__}{args}")
result = func(*args)
cache[args] = result
return result
wrapper.cache = cache # Expose cache
return wrapper
@memoize
def fibonacci(n):
"""Calculate nth Fibonacci number (slow recursive version)."""
if n <= 1:
return n
return fibonacci(n-1) + fibonacci(n-2)
# First call - computes everything
start = time.time()
result = fibonacci(10)
print(f"Result: {result}, Time: {time.time() - start:.4f}s")
# Second call - uses cache
start = time.time()
result = fibonacci(10)
print(f"Result: {result}, Time: {time.time() - start:.4f}s")
Python’s Built-in @lru_cache#
from functools import lru_cache
@lru_cache(maxsize=128) # Cache up to 128 recent calls
def fibonacci_fast(n):
if n <= 1:
return n
return fibonacci_fast(n-1) + fibonacci_fast(n-2)
# Much faster!
start = time.time()
result = fibonacci_fast(100)
print(f"fib(100) = {result}")
print(f"Time: {time.time() - start:.6f}s")
# View cache stats
print(fibonacci_fast.cache_info())
Part 5: Advanced Decorator Patterns#
More sophisticated decorator techniques.
Decorators with Arguments#
def repeat(times):
"""
Decorator that repeats function execution.
This is a decorator factory - it returns a decorator.
"""
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
results = []
for _ in range(times):
result = func(*args, **kwargs)
results.append(result)
return results
return wrapper
return decorator
@repeat(3)
def greet(name):
return f"Hello, {name}!"
print(greet("Alice"))
# Output: ['Hello, Alice!', 'Hello, Alice!', 'Hello, Alice!']
Class-Based Decorators#
class CountCalls:
"""
Decorator class that counts function calls.
"""
def __init__(self, func):
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
print(f"Call {self.count} of {self.func.__name__}")
return self.func(*args, **kwargs)
@CountCalls
def say_hello():
print("Hello!")
say_hello()
say_hello()
say_hello()
print(f"Total calls: {say_hello.count}")
Stacking Multiple Decorators#
def bold(func):
@wraps(func)
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return f"<b>{result}</b>"
return wrapper
def italic(func):
@wraps(func)
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return f"<i>{result}</i>"
return wrapper
# Decorators are applied bottom-to-top
@bold
@italic
def greet(name):
return f"Hello, {name}!"
# Equivalent to: bold(italic(greet))
print(greet("Alice"))
# Output: <b><i>Hello, Alice!</i></b>
Decorator with Optional Arguments#
from functools import wraps
def optional_debug(func=None, *, prefix="DEBUG"):
"""
Decorator that can be used with or without arguments.
Usage:
@optional_debug
def func1(): ...
@optional_debug(prefix="INFO")
def func2(): ...
"""
def decorator(f):
@wraps(f)
def wrapper(*args, **kwargs):
print(f"[{prefix}] Calling {f.__name__}")
result = f(*args, **kwargs)
print(f"[{prefix}] {f.__name__} returned {result}")
return result
return wrapper
# If called without arguments
if func is not None:
return decorator(func)
# If called with arguments
return decorator
@optional_debug
def add(a, b):
return a + b
@optional_debug(prefix="INFO")
def multiply(a, b):
return a * b
add(2, 3)
multiply(2, 3)
Part 6: Property Decorators#
Use decorators to create managed attributes.
class Temperature:
"""Temperature class with validation."""
def __init__(self, celsius):
self._celsius = celsius
@property
def celsius(self):
"""Get temperature in Celsius."""
return self._celsius
@celsius.setter
def celsius(self, value):
"""Set temperature in Celsius with validation."""
if value < -273.15:
raise ValueError("Temperature below absolute zero!")
self._celsius = value
@property
def fahrenheit(self):
"""Get temperature in Fahrenheit."""
return (self._celsius * 9/5) + 32
@fahrenheit.setter
def fahrenheit(self, value):
"""Set temperature in Fahrenheit."""
self.celsius = (value - 32) * 5/9
# Usage
temp = Temperature(25)
print(f"{temp.celsius}°C = {temp.fahrenheit}°F")
temp.fahrenheit = 98.6
print(f"{temp.celsius:.1f}°C = {temp.fahrenheit}°F")
try:
temp.celsius = -300 # Below absolute zero
except ValueError as e:
print(f"Error: {e}")
Part 7: Real-World Decorator Examples#
Decorators you’ll see in popular frameworks.
Flask-Style Route Decorator#
# Simplified Flask-style routing
class SimpleApp:
def __init__(self):
self.routes = {}
def route(self, path):
"""Register a function for a URL path."""
def decorator(func):
self.routes[path] = func
return func
return decorator
def handle(self, path):
"""Handle a request to a path."""
handler = self.routes.get(path)
if handler:
return handler()
return "404 Not Found"
app = SimpleApp()
@app.route('/')
def home():
return "Welcome to the home page!"
@app.route('/about')
def about():
return "This is the about page."
# Simulate requests
print(app.handle('/')) # Welcome to the home page!
print(app.handle('/about')) # This is the about page.
print(app.handle('/missing')) # 404 Not Found
Authentication Decorator#
# Simulated authentication system
current_user = None
def require_auth(func):
"""
Require user to be authenticated.
"""
@wraps(func)
def wrapper(*args, **kwargs):
if current_user is None:
raise PermissionError("Authentication required")
return func(*args, **kwargs)
return wrapper
def require_admin(func):
"""
Require user to be an admin.
"""
@wraps(func)
def wrapper(*args, **kwargs):
if current_user is None or not current_user.get('is_admin'):
raise PermissionError("Admin access required")
return func(*args, **kwargs)
return wrapper
@require_auth
def view_profile():
return f"Profile for {current_user['username']}"
@require_admin
def delete_user(username):
return f"Deleted user {username}"
# Test without authentication
try:
view_profile()
except PermissionError as e:
print(f"Error: {e}")
# Login as regular user
current_user = {'username': 'alice', 'is_admin': False}
print(view_profile()) # Works
# Try admin action as regular user
try:
delete_user('bob')
except PermissionError as e:
print(f"Error: {e}")
# Login as admin
current_user = {'username': 'admin', 'is_admin': True}
print(delete_user('bob')) # Works
Rate Limiting Decorator#
import time
from collections import deque
def rate_limit(max_calls, period):
"""
Limit function to max_calls within period seconds.
"""
def decorator(func):
calls = deque()
@wraps(func)
def wrapper(*args, **kwargs):
now = time.time()
# Remove old calls outside the window
while calls and calls[0] < now - period:
calls.popleft()
if len(calls) >= max_calls:
raise Exception(f"Rate limit exceeded: {max_calls} calls per {period}s")
calls.append(now)
return func(*args, **kwargs)
return wrapper
return decorator
@rate_limit(max_calls=3, period=5)
def api_call(data):
return f"Processing: {data}"
# Make 3 calls - OK
print(api_call("request 1"))
print(api_call("request 2"))
print(api_call("request 3"))
# 4th call - rate limited
try:
print(api_call("request 4"))
except Exception as e:
print(f"Error: {e}")
📝 Exercises#
Test your understanding with these challenges.
Exercise 1: Custom Memoization#
Create a memoization decorator that:
Stores results in a dictionary
Uses function arguments as cache keys
Has a
cache_clear()method to reset the cachePrints cache hits and misses
Test with Fibonacci function
# Your code here
def memoize(func):
# TODO: Implement memoization decorator
pass
@memoize
def fibonacci(n):
if n <= 1:
return n
return fibonacci(n-1) + fibonacci(n-2)
# Test
# print(fibonacci(10))
# print(fibonacci(10)) # Should use cache
Exercise 2: Argument Validator#
Create a decorator that validates function arguments:
Check types of arguments
Check value ranges
Raise
TypeErrororValueErrorif validation fails
# Your code here
def validate_args(type_checks=None, value_checks=None):
"""
Decorator to validate function arguments.
type_checks: dict of {arg_name: expected_type}
value_checks: dict of {arg_name: validation_function}
"""
# TODO: Implement
pass
# Example usage:
# @validate_args(
# type_checks={'age': int, 'name': str},
# value_checks={'age': lambda x: x > 0}
# )
# def create_user(name, age):
# return f"User: {name}, Age: {age}"
Exercise 3: Execution Profiler#
Create a decorator that profiles function execution:
Track number of calls
Track total execution time
Track average execution time
Provide a
stats()method to view statistics
# Your code here
class ProfilerDecorator:
"""Class-based decorator for profiling."""
# TODO: Implement
pass
Exercise 4: Deprecated Decorator#
Create a @deprecated decorator that:
Warns when a deprecated function is called
Accepts an optional message
Suggests alternative function if provided
# Your code here
import warnings
def deprecated(message=None, alternative=None):
# TODO: Implement
pass
# Example usage:
# @deprecated(message="Use new_function instead", alternative="new_function")
# def old_function():
# return "This is old"
💡 Pro Tips#
Always use
@wraps: Preserves function metadataKeep decorators simple: Each decorator should do one thing well
Use
functools.lru_cache: Don’t reinvent memoizationConsider performance: Decorators add overhead
Document decorator behavior: Explain what the decorator does
Test decorated functions: Ensure decorator doesn’t break functionality
Use class decorators for state: When you need to maintain state across calls
Decorator factories for parameters: Use the
def decorator_factory(param): def decorator(func):pattern
⚠️ Common Mistakes#
Forgetting
@wraps: Loses function metadataNot handling
*args, **kwargs: Decorator won’t work with all functionsMutable default arguments in decorators: Can cause unexpected behavior
Not returning the result: Decorator swallows return value
Circular imports: Decorators in separate modules
Thread safety: Cache decorators may not be thread-safe
🎯 Key Takeaways#
Functions are first-class objects in Python
Closures allow functions to remember their enclosing scope
Decorators modify function behavior without changing source code
Use
@wrapsto preserve function metadataDecorator factories enable parameterized decorators
Class-based decorators are useful for maintaining state
Common decorator patterns: timing, logging, caching, validation, retry
Decorators are essential in modern Python frameworks
📚 Further Reading#
PEP 318 - Decorators for Functions and Methods
Python Cookbook by David Beazley
Fluent Python by Luciano Ramalho
Real Python decorator tutorials
🚀 Next Steps#
Practice writing decorators for your own projects
Explore
contextlibfor context manager decoratorsLearn about metaclasses (next level of metaprogramming)
Study decorator patterns in popular frameworks (Flask, Django, FastAPI)
Implement your own framework using decorators
Congratulations! You’ve mastered advanced functions and decorators in Python! 🎉