Python

Context

If you want to learn multiple programming languages, first you need to master one. I first learned Java, but I’ve only worked with Python professionally, so I’m putting some effort to make sure I know it well.

The main reason I want to master Python is that it's the go to language for data science and machine learning, which are fields I’m very interested in.

On the other hand, since it’s a very popular language, I see a lot of people using it without really knowing it well, which can lead to bad practices and inefficient code.

So in this post, I’m going to cover some concepts which I think are important to understand and work well with Python. I won’t cover basic topics like syntax, data types, or control flow, but rather will focus on concepts that are more specific to Python and that will help me write better code.

Quick Reference

This guide covers:

• Language Characteristics: C-based implementation, dynamic typing, interpreted execution
• Type System: Type hints, docstrings for better code documentation
• Data Structures: Lists vs tuples, sets operations, defaultdict
• Comprehensions: List and dictionary comprehensions for cleaner code
• Functions: Lambda functions, variable-length arguments (*args/**kwargs), generators
• Advanced Patterns: Decorators, context managers
• Best Practices: Virtual environments, Pythonic built-in functions

Language Characteristics

Python is built on top of C, which means that many of its core functionalities are implemented in C for performance reasons, such as the garbage collector and many of its data types (e.g. lists, dictionaries, and so on). However, Python itself is not a compiled language like C or C++. Instead, it is an interpreted language, which means that Python code is executed by an interpreter at runtime, rather than being compiled into a standalone executable file like C or Go.

Is not strongly typed: you don't need to declare variable types explicitly, and types can change dynamically at runtime. You can use type hints to indicate the expected types of variables and function parameters, but they are not enforced by the interpreter.

TIP

Understanding these characteristics helps you write more efficient Python code. For example, using list comprehensions leverages C-level optimizations, and type hints improve code readability even though they’re not enforced at runtime.

Code Quality & Documentation

Type Hints

Python supports type hints, which allow you to indicate the expected types of variables and function parameters. They are optional, but very useful and improve code quality.

def greet(name: str) -> str:
    return f"Hello, {name}!"

I particularly like them because they make it clear what types of arguments a function expects and what type it returns, and also because IDEs can use them to provide better autocompletion and type checking.

Additionally, type hints can be used by static type checkers like mypy to catch potential type errors before running the code.

TIP

WHEN TO USE: Always use type hints for function signatures in production code. They serve as self-documenting code and enable better tooling support.

Docstrings

Python has a built-in way to document functions, classes, and modules using docstrings. Docstrings are multi-line strings that are placed immediately after the definition of a function, class, or module. They provide a convenient way to document the purpose and behavior of the code.

def add(a: int, b: int) -> int:
    """
    Adds two integers and returns the result.
 
    Parameters:
    a (int): The first integer to add.
    b (int): The second integer to add.
 
    Returns:
    int: The sum of the two integers.
    """
    return a + b

This way, you can hover over the function name in an IDE or use help() to see what the function does.

Data Structures & Comprehensions

List Comprehensions

List comprehensions are a concise way to create lists in Python:

# Traditional way
squares = []
for x in range(10):
    squares.append(x**2)
 
# List comprehension
squares = [x**2 for x in range(10)]

They can also be used with conditional statements to filter elements:

# List comprehension with condition
even_squares = [x**2 for x in range(10) if x % 2 == 0]

They’re not only more concise but also faster than traditional loops, as they leverage C-level optimizations under the hood, making them more efficient.

NOTE

This is called the “Pythonic” way of doing things, which emphasizes readability and efficiency. Some other examples of Pythonic practices include using built-in functions, avoiding unnecessary loops, and using generators for large data sets.

Dictionary Comprehensions

Are similar to list comprehensions, but for creating dictionaries:

# Traditional way
squares_dict = {}
for x in range(10):
    squares_dict[x] = x**2
 
# Dictionary comprehension
squares_dict = {x: x**2 for x in range(10)}

TIP

WHEN TO USE: Use comprehensions when transforming or filtering data. They’re more readable and performant than traditional loops for simple transformations.

Lists vs Tuples

Lists and tuples are both used to store collections of items in Python, but they have some key differences:

# List are mutable and ordered
my_list = [1, 2, 3]
 
# Tuples are immutable and ordered
my_tuple = (1, 2, 3)

Use tuples for fixed collections of items and lists for collections that may change.

Keep in mind that tuples are faster and more memory efficient than lists.

TIP

WHEN TO USE: Use tuples for data that shouldn’t change (like coordinates, RGB values, or function return values with multiple items). Use lists for collections that need to be modified.

Sets Operations

Sets are unordered collections of unique items in Python. They are useful when you want to store a collection of items without duplicates and perform operations like union, intersection, and difference.

# Creating sets
set_a = {1, 2, 3, 4}
set_b = {3, 4, 5, 6}
 
# Union
union_set = set_a | set_b  # {1, 2, 3, 4, 5, 6}
 
# Intersection
intersection_set = set_a & set_b  # {3, 4}
 
# Difference
difference_set = set_a - set_b  # {1, 2}
 
# Symmetric Difference
symmetric_difference_set = set_a ^ set_b  # {1, 2, 5, 6}

You can also check if a set is a subset or superset of another set using the issubset() and issuperset() methods:

# Subset
is_subset = set_a.issubset(union_set)  # True
 
# Or using the <= operator
is_subset = set_a <= union_set  # True
 
# Superset
is_superset = union_set.issuperset(set_b)  # True
 
# Or using the >= operator
is_superset = union_set >= set_b  # True

TIP

WHEN TO USE: Use sets when you need to eliminate duplicates or perform mathematical set operations. They’re also extremely fast for membership testing (item in my_set).

defaultdict

Is a subclass of the built-in dict class in Python. In simple terms, when you try to access a key that doesn’t exist in a defaultdict, it automatically creates that key with a default value, instead of raising a KeyError.

from collections import defaultdict
 
count_dict = defaultdict(int)
items = ['apple', 'banana', 'orange', 'apple', 'banana', 'apple']
for item in items:
    count_dict[item] += 1
print(count_dict)

This is better than using a regular dictionary, where you would have to check if the key exists before incrementing its value.

TIP

WHEN TO USE: Use defaultdict when building dictionaries where you’re accumulating values (counting, grouping, or collecting items). It eliminates the need for key existence checks.

Functions & Control Flow

Lambda Functions

Lambda functions are small, anonymous functions defined in a single line. They’re useful for passing simple functions as arguments to functions like map() or filter(), or when you need a short function to be used just once.

# Traditional function
def square(x):
    return x**2
 
# Lambda function
square = lambda x: x**2

NOTE

Lambda functions are conceptually similar to anonymous functions found in other programming languages, like JavaScript’s arrow functions (() => {}).

TIP

WHEN TO USE: Use lambda functions for simple, one-line operations passed as arguments (e.g., sorting with custom keys, mapping simple transformations). For anything more complex, define a regular function.

Variable-Length Arguments (*args and **kwargs)

In Python, *args and **kwargs are used in function definitions to allow for variable-length arguments. They stand for “arguments” and “keyword arguments”, respectively.

def my_function(*args, **kwargs):
    for arg in args:
        print(arg)
    for key, value in kwargs.items():
        print(f"{key}: {value}")

They are useful when you want to create functions that can accept a varying number of arguments, such as when wrapping other functions or creating decorators.

TIP

WHEN TO USE: Use *args for variable positional arguments and **kwargs for variable keyword arguments. Common use cases include wrapper functions, decorators, and APIs that need flexibility.

Generators

Generators are a way to create iterators in Python. They allow you to iterate over a sequence of values without having to store them all in memory at once. This is particularly useful when working with large data sets, as it can help reduce memory usage and improve performance.

Generators are defined using functions that use the yield keyword to produce a sequence of values:

def count_up_to(n):
    count = 1
    while count <= n:
        yield count
        count += 1

Understanding yield

The yield keyword is what makes generators special. Any function that contains yield automatically becomes a generator function, regardless of whether it has return statements or not.

Here’s the key difference in behavior:

# Regular function
def regular_function():
    return 1
 
result = regular_function()  # Executes immediately, returns 1
print(result)  # 1
 
# Generator function
def generator_function():
    yield 1
 
result = generator_function()  # Returns a generator object, doesn't execute yet!
print(result)  # <generator object generator_function at 0x...>
print(next(result))  # Now it executes and yields 1

How yield works:

• Unlike return, which exits a function completely, yield pauses the function and saves its state
• When the generator is called again (in the next iteration), it resumes right where it left off, with all variables intact
• Each time yield is encountered, it produces a value to the caller, then waits
• The function keeps its “memory” between calls - variables maintain their values

Think of it like a bookmark in a book: return closes the book and forgets your page, while yield bookmarks your spot so you can continue reading later.

NOTE

Key Difference: Calling a generator function doesn’t execute its code - it returns a generator object. The code only runs when you iterate over it (with for, next(), etc.).

You can then use the generator function in a loop to iterate over the values it produces:

for number in count_up_to(5):
    print(number)

This is basically how the range() function works in Python 3, which returns a generator instead of a list.

TIP

WHEN TO USE: Use generators when working with large datasets or infinite sequences. They’re memory-efficient because they generate values on-the-fly rather than storing everything in memory.

Advanced Patterns

Decorators

Decorators are a way to add extra functionality to your functions without changing the original function’s code. Think of them like a wrapper that adds something extra to a function, without modifying the function itself.

To implement a decorator, you define a function that takes another function as an argument, and returns a new function that adds the desired functionality.

def handle_errors(func):
    def wrapper():
        try:
            func()
        except Exception as e:
            print(f"Error occurred: {e}")
    return wrapper
 
@handle_errors
def divide():
    result = 10 / 0
    print(result)

I think this example implementation is particularly useful, I’ve used it before to handle errors in functions without having to add try-except blocks everywhere.

You can also go a bit deeper and use decorators with parameters, create class-based decorators for more complex use cases, use built-in decorators like @staticmethod and @classmethod, stack multiple decorators on a single function, or even use functools like functools.wraps to preserve the original function’s metadata when creating decorators.

TIP

WHEN TO USE: Use decorators for cross-cutting concerns like logging, timing, authentication, caching, or error handling. They help keep your core function logic clean and focused.

Context Managers

Context managers are a way to manage resources in Python, such as file handles or database connections. They ensure that resources are properly acquired and released, even if an error occurs during their use.

You can create a context manager using the with statement, which automatically handles the setup and teardown of the resource.

with open('file.txt', 'r') as file:
    content = file.read()

To implement a class that acts as a context manager, e.g. for managing a database connection, you need to define the __enter__ and __exit__ methods:

class DatabaseConnection:
    def __enter__(self):
        self.connection = create_database_connection()
        return self.connection
 
    def __exit__(self, exc_type, exc_value, traceback):
        self.connection.close()

Then you can use it like this:

with DatabaseConnection() as conn:
    # Use the database connection

TIP

WHEN TO USE: Use context managers whenever you need to ensure proper resource cleanup (files, database connections, locks, etc.). They guarantee cleanup happens even if exceptions occur.

Best Practices

Virtual Environments

In Python, packages and dependencies are installed globally by default, which can lead to conflicts between different projects that require different versions of the same package.

To avoid this, Python provides a way to create virtual environments, which are isolated environments that can have their own packages and dependencies, separate from the global Python installation.

For me, the best way to create virtual environments is through tools like uv or poetry, which simplify the process of creating and managing virtual environments, as well as handling dependencies and package management.

TIP

KEY TAKEAWAY: Always use virtual environments for Python projects. This prevents dependency conflicts and ensures reproducible builds across different machines.

Essential Built-in Functions

One of the things that makes a code Pythonic is to leverage built-in functions and libraries as much as possible.

Here are some of the most useful built-in functions in Python with examples:

enumerate

Adds a counter to an iterable:

for index, value in enumerate(['a', 'b', 'c']):
    print(f"{index}: {value}")  # 0: a, 1: b, 2: c

zip

Combines multiple iterables into tuples:

names = ['Alice', 'Bob']
scores = [85, 92]
for name, score in zip(names, scores):
    print(f"{name}: {score}")  # Alice: 85, Bob: 92

sorted

Returns a sorted list from an iterable:

sorted([3, 1, 4, 1, 5], reverse=True)  # [5, 4, 3, 1, 1]
sorted(['banana', 'apple'], key=len)  # ['apple', 'banana']

sum, min, max

Aggregate functions for iterables:

sum([1, 2, 3, 4, 5])  # 15
min([3, 1, 4, 1, 5])  # 1
max([3, 1, 4, 1, 5])  # 5

any and all

Boolean aggregation:

any([False, False, True])  # True (at least one is true)
all([True, True, True])    # True (all are true)

isinstance

Type checking:

isinstance(42, int)        # True
isinstance('hello', str)   # True

map

Apply a function to each item:

list(map(str.upper, ['hello', 'world']))  # ['HELLO', 'WORLD']

NOTE

Iterables are objects that can be iterated over, such as lists, tuples, and strings. You can use the iter() function to create an iterator from an iterable.

TIP

KEY TAKEAWAY: Mastering these built-in functions will make your code more concise, readable, and Pythonic. They’re optimized in C and often faster than manual implementations.