Posted on Jun 27, 2023

A few lesser-known but pretty useful Python concepts

A. Context Managers:

Context managers allow you to allocate and release resources when needed, such as opening and closing files, acquiring and releasing locks, etc. They can be implemented using the with statement and the contextlib module.

Examples

File Handling

with open('file.txt', 'r') as file: data = file.read() # Perform operations on the file # File automatically closed outside the 'with' block

Lock Acquisition and Release

import threading lock = threading.Lock() with lock: # Perform thread-safe operations # Lock automatically released outside the 'with' block

Database Connection

import sqlite3 with sqlite3.connect('database.db') as connection: cursor = connection.cursor() # Perform database operations # Connection automatically closed outside the 'with' block

Timing Execution

import time class Timer: def __enter__(self): self.start_time = time.time() return self def __exit__(self, exc_type, exc_val, exc_tb): elapsed_time = time.time() - self.start_time print(f"Execution time: {elapsed_time} seconds") with Timer(): # Code to measure execution time # Timer context manager prints the execution time

Creating a custom context manager

class CustomContextManager: def __enter__(self): # Code to run when entering the 'with' block  print("Entering the 'with' block") # You can optionally return an object or set up resources  def __exit__(self, exc_type, exc_val, exc_tb): # Code to run when exiting the 'with' block  print("Exiting the 'with' block") # Perform any cleanup actions or handle exceptions if necessary  # Usage example: with CustomContextManager(): # Code inside the 'with' block  print("Inside the 'with' block") # Output: # Entering the 'with' block # Inside the 'with' block # Exiting the 'with' block

B. Generators and Generator Expressions:

Generators are functions that generate values on the fly, allowing you to iterate over a sequence of values without creating the entire sequence in memory. Generator expressions are similar to list comprehensions but return a generator object instead of a list.

Generator Function

def countdown(n): while n > 0: yield n n -= 1 # Using the generator function for num in countdown(5): print(num) # Output: # 5 # 4 # 3 # 2 # 1

Generator Expression

# Using a generator expression to calculate the squares of numbers squares = (x ** 2 for x in range(1, 6)) # Accessing the values from the generator expression for square in squares: print(square) # Output: # 1 # 4 # 9 # 16 # 25

Infinite Generator

def infinite_sequence(): num = 0 while True: yield num num += 1 # Using the infinite generator for i in infinite_sequence(): if i > 10: break print(i) # Output: # 0 # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10

C. Function Decorators:

Decorators are a way to modify the behavior of functions or classes by wrapping them with another function. They are denoted by the @ symbol and can be used for tasks like logging, timing, and caching.

Logging Decorator: This decorator logs the name of the decorated function before and after its execution

def log_decorator(func): def wrapper(*args, **kwargs): print(f'Calling {func.__name__}...') result = func(*args, **kwargs) print(f'{func.__name__} called.') return result return wrapper @log_decorator def add_numbers(a, b): return a + b result = add_numbers(3, 5) print(result) # Output: # Calling add_numbers... # add_numbers called. # 8

Timing Decorator: This decorator measures the execution time of the decorated function

import time def time_decorator(func): def wrapper(*args, **kwargs): start_time = time.time() result = func(*args, **kwargs) end_time = time.time() execution_time = end_time - start_time print(f'{func.__name__} executed in {execution_time} seconds.') return result return wrapper @time_decorator def factorial(n): if n == 0 or n == 1: return 1 else: return n * factorial(n - 1) result = factorial(5) print(result) # Output: 120  # Output: # factorial executed in 9.5367431640625e-07 seconds. # factorial executed in 7.605552673339844e-05 seconds. # factorial executed in 9.322166442871094e-05 seconds. # factorial executed in 0.00010275840759277344 seconds. # factorial executed in 0.00011801719665527344 seconds. # 120

Authorization Decorator: This decorator checks if the user is authorized to access the decorated function

def is_user_authorized(): return False def authorize_decorator(func): def wrapper(*args, **kwargs): if is_user_authorized(): return func(*args, **kwargs) else: raise PermissionError('User is not authorized to access this function.') return wrapper @authorize_decorator def sensitive_operation(): return 'Sensitive data' result = sensitive_operation() print(result) # Output: # Traceback (most recent call last): # File "/Users/dev1/python/app.py", line 16, in <module> # result = sensitive_operation() # File "/Users/dev1/mygitlab/python/app.py", line 9, in wrapper # raise PermissionError('User is not authorized to access this function.') # PermissionError: User is not authorized to access this function.

D. Namedtuples:

Namedtuples are lightweight data structures that are similar to tuples but have named fields. They provide a convenient way to define simple classes without writing a custom class definition. They are commonly used in scenarios where lightweight data containers are required.

Creating a Namedtuple

from collections import namedtuple # Define a namedtuple called 'Point' with fields 'x' and 'y' Point = namedtuple('Point', ['x', 'y']) # Create an instance of Point p = Point(2, 3) # Access the fields using dot notation print(p.x) # Output: 2 print(p.y) # Output: 3

Namedtuple as a Return Type

from collections import namedtuple # Define a namedtuple called 'Person' with fields 'name', 'age', and 'city' Person = namedtuple('Person', ['name', 'age', 'city']) # Function that returns a Person namedtuple def get_person(): return Person('John', 25, 'New York') # Call the function and access the fields of the returned namedtuple person = get_person() print(person.name) # Output: John print(person.age) # Output: 25 print(person.city) # Output: New York

Unpacking a Namedtuple

from collections import namedtuple # Define a namedtuple called 'Color' with fields 'red', 'green', and 'blue' Color = namedtuple('Color', ['red', 'green', 'blue']) # Create an instance of Color color = Color(255, 128, 0) # Unpack the values into separate variables red, green, blue = color print(red) # Output: 255 print(green) # Output: 128 print(blue) # Output: 0

E. Coroutines:

Coroutines are functions that can pause their execution and yield control back to the caller while maintaining their state. They are used for asynchronous programming and are a powerful tool for managing concurrency and asynchronous programming, allowing for more efficient and flexible code execution.

Simple Coroutine

def coroutine_example(): while True: x = yield print('Received:', x) coroutine = coroutine_example() next(coroutine) # Initialize the coroutine coroutine.send(10) # Send a value to the coroutine coroutine.send('Hello') # Send another value to the coroutine coroutine.close() # close it  # Output: # Received: 10 # Received: Hello

Coroutine with Producer and Consumer

def producer(coroutine): for i in range(5): print('Producing:', i) coroutine.send(i) coroutine.close() def consumer(): while True: x = yield print('Consumed:', x) coroutine = consumer() next(coroutine) # Initialize the coroutine producer(coroutine) # Output: # Producing: 0 # Consumed: 0 # Producing: 1 # Consumed: 1 # Producing: 2 # Consumed: 2 # Producing: 3 # Consumed: 3 # Producing: 4 # Consumed: 4

Coroutine Chaining

def coroutine1(): while True: x = yield print('Coroutine 1:', x) def coroutine2(): while True: x = yield print('Coroutine 2:', x) coroutine = coroutine1() next(coroutine) # Initialize the first coroutine coroutine2() # Initialize the second coroutine coroutine.send(10) # Send a value to the first coroutine coroutine.send('Hello') # Send another value to the first coroutine  # Output: # Coroutine 1: 10 # Coroutine 1: Hello

F. Operator Overloading:

Python allows you to redefine the behavior of operators for your custom classes by implementing special methods such as __add__, __sub__, __mul__, etc. This concept is known as operator overloading. You can overload various operators such as arithmetic operators, comparison operators, and more. Operator overloading allows you to customize the behavior of objects to make your code more expressive and intuitive

Arithmetic Operators

class Vector: def __init__(self, x, y): self.x = x self.y = y def __add__(self, other): return Vector(self.x + other.x, self.y + other.y) def __sub__(self, other): return Vector(self.x - other.x, self.y - other.y) def __mul__(self, scalar): return Vector(self.x * scalar, self.y * scalar) def __str__(self): return f"x: {self.x}, y: {self.y}" # Usage: v1 = Vector(2, 3) v2 = Vector(4, 5) v3 = v1 + v2 # Addition v4 = v1 - v2 # Subtraction v5 = v1 * 2 # Scalar multiplication  print(v3) print(v4) print(v5) # Output: # x: 6, y: 8 # x: -2, y: -2 # x: 4, y: 6

Comparison Operators

class Point: def __init__(self, x, y): self.x = x self.y = y def __eq__(self, other): return self.x == other.x and self.y == other.y def __lt__(self, other): return self.x < other.x and self.y < other.y # Usage: p1 = Point(2, 3) p2 = Point(4, 5) print(p1 == p2) # Equality print(p1 < p2) # Less than  # Output: # False # True

String Representation

class Book: def __init__(self, title, author): self.title = title self.author = author def __str__(self): return f"{self.title} by {self.author}" # Usage: book = Book("Python Programming", "John Smith") print(book) # String representation  # Output: # Python Programming by John Smith

G. Monkey Patching:

Monkey patching refers to the ability to modify or extend the behavior of existing code at runtime. In Python, you can dynamically modify classes, objects, or modules by adding, replacing, or deleting attributes or methods.

Adding a Method to a Class

class MyClass: def __init__(self, name): self.name = name def say_hello(self): print(f"Hello, {self.name}!") # Monkey patching MyClass.say_hello = say_hello # Creating an instance and calling the patched method obj = MyClass("Alice") obj.say_hello() # Output: # Hello, Alice!

Modifying an Existing Method

class MyClass: def greeting(self): return "Hello!" # Monkey patching def modified_greeting(self): return "Hola!" MyClass.greeting = modified_greeting # Creating an instance and calling the modified method obj = MyClass() print(obj.greeting()) # Output: # Hola!

Adding a Function to a Module

def multiply(a, b): return a * b # Monkey patching import math math.multiply = multiply # Calling the patched function result = math.multiply(5, 6) print(result) # Output: # 30

H. Type Hints and Annotations:

Type hints are a way to statically declare the expected types of variables, arguments, and return values in Python code. They are not enforced at runtime but can be used by static analysis tools to catch potential type-related errors.

Variable type hints:

def add(a: int, b: int) -> int: return a + b

Class annotations:

class Person: def __init__(self, name: str, age: int): self.name = name self.age = age

Type hints for collections

from typing import List, Tuple def process(data: List[Tuple[str, str]]) -> List[str]: result: List[str] = [] for item in data: result.append(item[0] * item[1]) return result

I. Metaclasses

Metaclasses in Python provide a way to define the behavior and structure of classes themselves. They allow you to customize the creation and initialization of classes.

Metaclasses provide a powerful mechanism for customizing class creation and behavior, but they should be used sparingly and only when necessary, as they can make the code more complex and harder to understand.

Creating a Singleton Metaclass

class SingletonMeta(type): _instances = {} def __call__(cls, *args, **kwargs): if cls not in cls._instances: cls._instances[cls] = super().__call__(*args, **kwargs) return cls._instances[cls] class SingletonClass(metaclass=SingletonMeta): def __init__(self): print("Initializing SingletonClass") # Usage instance1 = SingletonClass() instance2 = SingletonClass() print(instance1 is instance2) # Output: # Initializing SingletonClass # True

Creating an Attribute Validation Metaclass

class ValidationMeta(type): def __new__(cls, name, bases, attrs): for name, value in attrs.items(): if name.startswith("_"): continue if not isinstance(value, (int, float)): raise TypeError(f"Attribute '{name}' must be numeric.") return super().__new__(cls, name, bases, attrs) class MyClass(metaclass=ValidationMeta): x = 10 y = "test" # Output: # Traceback (most recent call last): # File "/Users/dev1/python/learnings/app.py", line 13, in <module> # class MyClass(metaclass=ValidationMeta): # File "/Users/dev1/python/learnings/app.py", line 8, in __new__ # raise TypeError(f"Attribute '{name}' must be numeric.") # TypeError: Attribute 'y' must be numeric

Registering Subclasses with a Metaclass

class PluginMeta(type): def __init__(cls, name, bases, attrs): if not hasattr(cls, "plugins"): cls.plugins = [] else: cls.plugins.append(cls) class PluginBase(metaclass=PluginMeta): pass class Plugin1(PluginBase): pass class Plugin2(PluginBase): pass # Usage print(PluginBase.plugins) # Output # [<class '__main__.Plugin1'>, <class '__main__.Plugin2'>]

J. Data Classes

Python's data classes are a convenient way to define classes that primarily hold data, similar to structs in other programming languages. They provide several benefits, such as automatic generation of common methods like __init__, __repr__, and __eq__. Data classes provide additional features like type hints, default values, ordering, and more.

Basic Data Class

from dataclasses import dataclass @dataclass class Point: x: float y: float z: float p = Point(1.0, 2.0, 3.0) q = Point(1.0, 2.0, 3.0) print(p.x) print(p.y) print(p.z) print(p) print(p == q) # Output # 1.0 # 2.0 # 3.0 # Point(x=1.0, y=2.0, z=3.0) # True

Data Class with Default Values

from dataclasses import dataclass @dataclass class Person: name: str age: int = 0 profession: str = "Unknown" # Create an instance of the Person class person1 = Person("Alice", 25) person2 = Person("Bob", profession="Engineer") # Access the fields of the instances print(person1.name, person1.age, person1.profession) print(person2.name, person2.age, person2.profession) # Output: # Alice 25 Unknown # Bob 0 Engineer

Inheritance with Data Classes

from dataclasses import dataclass @dataclass class Person: name: str age: int @dataclass class Employee(Person): company: str # Create an instance of the Employee class employee = Employee("Alice", 30, "Acme Corporation") # Access the fields of the instance print(employee.name) print(employee.age) print(employee.company) # Output: # Alice # 30 # Acme Corporation

K. Context Variables

Introduced in Python 3.7, context variables allow you to create variables that retain their values within a context, even if the context is asynchronous or multi-threaded. They are useful for propagating values across functions without passing them explicitly as arguments.

Using contextvars module

import contextvars user_id = contextvars.ContextVar("user_id", default=None) def process_request(request): user_id.set(request.user_id) process_data() def process_data(): uid = user_id.get() print(f"processing for user: {uid}")

Using threading.local for thread-local context

import threading context = threading.local() def process_request(request): context.user_id = request.user_id process_data() def process_data(): uid = context.user_id print(f"processing for user: {uid}")

Using contextlib.ContextDecorator for context-based behavior

from contextlib import ContextDecorator import time class TimingContext(ContextDecorator): def __enter__(self): self.start_time = time.time() def __exit__(self, exc_type, exc_val, exc_tb): elapsed_time = time.time() - self.start_time print("Elapsed time:", elapsed_time) @TimingContext() def process_data(): # Perform some time-consuming operation  time.sleep(3) print("Data processing complete.") process_data() # Output: # Data processing complete. # Elapsed time: 3.0054383277893066

L. Callable Objects

In Python, any object that can be called as a function is considered callable. This includes functions, methods, classes (which create instances when called), and objects implementing the __call__ method. You can use callable objects to create more flexible and dynamic code.

Classes with __call__ method: Classes that define the __call__ method can be called like functions

class Multiply: def __call__(self, a, b): return a * b multiply = Multiply() result = multiply(2, 3) print(result) # Output: # 6

Built-in callable objects: Some built-in objects in Python are callable, such as int, str, list, dict, etc.

result = str(42) print(result) # Output # 42

M. Extended Iterable Unpacking

Extended iterable unpacking, introduced in Python 3, allows you to unpack an iterable into multiple variables, including capturing remaining items into another variable. It simplifies tasks such as splitting lists, assigning values from tuples, and processing variable-length iterables.

Unpacking a list into variables

my_list = [1, 2, 3, 4, 5] first, *middle, last = my_list print(first) print(middle) print(last) # Output: # 1 # [2, 3, 4] # 5

Unpacking a string into variables

my_string = "Hello" first, *rest = my_string print(first) print(rest) # Output: # H # ['e', 'l', 'l', 'o']

Unpacking a tuple of unknown length

my_tuple = (1, 2, 3, 4, 5) first, *middle, last = my_tuple print(first) print(middle) print(last) # Output: # 1 # [2, 3, 4] # 5

N. Multiple Inheritance and Method Resolution Order (MRO)

Python supports multiple inheritance, allowing a class to inherit from multiple base classes. Method Resolution Order (MRO) determines the order in which base classes are searched for a method. Understanding MRO helps you resolve method name conflicts and grasp the inheritance hierarchy.

class A: def say_hello(self): print("Hello from A") class B(A): def say_hello(self): print("Hello from B") class C(A): def say_hello(self): print("Hello from C") class D(B, C): pass d = D() d.say_hello() # Output: # Hello from B

When we create an instance of D and call the say_hello() method, Python follows the Method Resolution Order (MRO) to determine which implementation of the method should be executed.

The MRO is determined by the C3 linearization algorithm, which is a consistent and predictable method resolution order. In this case, the MRO for class D would be [D, B, C, A, object]. Thus, say_hello() of class B gets invoked.

DEV Community