Stop Using Exceptions Like This in Python

Welcome to the exciting world of error handling in Python! Imagine building a blog that's running smoothly, with users signing up and creating posts. But wait a minute, what if something unexpected happens? Like a user trying to use an email that's already taken? That's where exceptions come in!

Exception handling is one of the most important parts of building an app. It helps us to keep things running smoothly and fix problems faster. And we get to decide what to do with them. Maybe, we show a friendly error message or write it down in our secret logging diary.

Raising exceptions in Python is super easy! Just use the raise keyword and specify the error type, like this:

raise ValueError("Argument s.jerrynsh.com/UcFDnviQ is an invalid URL.")

Let's dive into the world of error handling in Python where I will share some common bad practices to avoid.

TL;DR

• Never use bare except
• Stop raising generic Exception
• Avoid catching using except Exception
• Log, don't pass

1. Do not use bare except

The first rule of thumb is to absolutely avoid using bare except block.

The use of bare except catches all exceptions indiscriminately – even the ones that you didn't expect and can crash your program (e.g. SystemExit or KeyboardInterrupt. Check out the Python exception hierarchy). Furthermore, it doesn't give us any exceptions objects to inspect.

Here’s an example of using bare except, which is not recommended:

# Do NOT ever use bare except:

try:
    books = fetch_all_books()
    send_slack_notification(books)

except:  # !!!
    raise

So, what’s wrong with using bare except clause?

Catching every exception could cause our application to fail without us really knowing why. This is a horrible idea as it makes our code harder to maintain as bugs can be hidden behind an unspecific exception handler.

Let's say you're building a feature that lets users upload PDF files. You put a try-except block around that code and use bare except.

Below, we catch a generic exception saying the file upload failed no matter what the actual problem is:

# Do NOT ever use bare except:

def upload_proof_of_address_doc(file):
    try:
        status = upload_pdf_to_s3(file)
        send_notifications(status)

    except:
        print("File upload failed.")    
        raise

Now, no matter what the problem is (like the user not having the right permissions or having issues sending notifications), the user will just get a generic error message saying the file upload failed.

Imagine the user uploads a file, but it's an image file instead of a PDF. They keep trying and trying, but keep getting the same error message. They might start thinking, "This app is trash! The file is right there and it still won't upload!"

Solution

So, how do we fix this situation? Easy – introduce specific exception classes!

Instead of catching all exceptions with bare except, catch only the specific exceptions you expect to occur. Using our example above, we can introduce FileTypeError and UserPermissionError just to name a few.

This way, you can handle the problems you know about, and let all the other exceptions bubble up to a higher-level error-handling mechanism.

2. Stop using raise Exception

Secondly, we should avoid raising a generic Exception in Python because it tends to hide bugs.

Here's another example that you should avoid using:

# Do NOT raise a generic Exception:

def get_book_List():
    try:
        if not fetch_books():
            raise Exception("This exception will not be caught by specific catch")  # !!!

    except ValueError as e:
        print("This doesn't catch Exception")
        # and other error handling code...


get_book_List()
# Exception: general exceptions not caught by specific handling

While there are plenty of ways to write bad code, this is one of the worst anti-patterns known as error hiding.

In my experience, this pattern has stood out as being the greatest drain on developer productivity.

Solution

Again, use specific exception classes.

We should always plan and figure out what can break and what exceptions are expected to be thrown.

For instance, if we’re working with a database call to fetch a user profile with an email, we should expect that the email might not exist and handle it accordingly.

In this scenario, we can raise a custom UserDoesNotExist error and prompt our users to try again, allowing our app to positively recover from the exception.

Here’s a very basic example of how to raise a custom user-defined exception in Python:

class UserDoesNotExist(Exception):
    """Raised when user does not exist"""
    pass

Reuse existing exceptions appropriately

Before writing our custom user-defined exceptions, we should always check if the framework or library that we use has built-in exceptions that meet our use case.

In short, we should only catch errors that we are interested in with a specific exception that semantically fits our use case.

3. Avoid using except Exception

This closely ties back to the previous point.

As developers, we tend to wrap our function code with a try-except block on autopilot mode using Exception (or worse, bare except). We love doing this because we know that there is always a chance of exceptions being thrown.

Gotta catch 'em all, right? No.

# Do NOT catch with a generic Exception:

def fetch_all_books():
    try:
        if not fetch_all_books():
            raise DoesNotExistError("No books found!")

    except Exception as e:  # !!!
        print(e)

In the example above, it means that we will catch everything. Including exceptions that we cannot or perhaps should not recover from.

Finally, here’s a better example of how to properly handle an exception:

# Do:

def fetch_user_profile(id):
    try:
        user = get_user_profile(id)
        if not user:
            raise UserDoesNotExist("User ID does not exist.")  # Raise specific exception

    except UserDoesNotExist as e:  # Catch specific exception
        logger.exception(e)  # Log the exception
        raise  # Just raise
        # raise UserDoesNotExist # Don't do this or you'll lose the stack trace

But I don’t know what exceptions to use

Understandably, it’s very unlikely that we are always prepared for every possible exception.

In such cases, some people suggest that we should at least catch them with an Exception as it won't include things like SystemExit and KeyboardInterrupt, which will terminate our app.

Still, I’d argue that we should spend the time to figure out what the exceptions are. When catching generic exceptions becomes a habit, it becomes a slippery slope.

Here’s another good example of why we should not catch generic exceptions.

4. Refrain from passing in except blocks

When designing an app, there might be specific exceptions where we are completely fine without doing anything.

However, the worst possible thing a developer can do is the following:

# Example 1
# NOTE: do NOT ever pass a bare exception:
try:
    result = compute_combination_sum(value)	# error 1
    notify_slack(result)	# error 2 here will be obsecured

except:
    pass	# do not just pass!


# Example 2
try:
    compute_combination_sum(value)
    notify_slack(result)

except ValueError:
    pass    # do not just pass!

Using a bare except block to catch all exceptions (except Exception:) means that if there are multiple errors in the code, the second error will be obscured.

This is because the first error will always be caught and the program will exit the try block, preventing the second error from being detected.

If we’re just passing an except statement, it’s a good sign that we aren’t really prepared for the exception that we are catching. Perhaps it’s a good time to rethink and refactor.

Log, don’t pass!

Nevertheless, if we don’t have to do anything about the exception, we should at least use a more specific exception while also logging the exception.

import logging

logger = logging.getLogger(__name__)


try:
    parse_number(value)

except ValueError as e:
    logger.exception(e)

Besides considering including some recovery code, we can also add a comment to inform other developers about the use case.

The bottom line is, we should steer away from passing in except blocks unless explicitly desired. Again, this is usually a bad sign.

Ultimately, we should log the exception to a monitoring system so that we at least have a log of what actually went wrong.

Closing Thoughts

To summarize everything we went through in this article, we should:

• Never use bare except.
• Stop raising generic Exception.
• Stop catching generic Exception.
• Avoid just passing in except blocks.

In most situations, it’s often better for the app to fail at the point of an exception rather than having our app continue to behave in weird unexpected ways. Hence, it’s best to catch only the exceptions that we know and intend to handle.

# Do:

def fetch_user_profile(id):
    try:
        user = get_user_profile(id)
        if not user:
            raise UserDoesNotExist("User ID does not exist.")

    except UserDoesNotExist as e
        logger.exception(e)
        raise

That’s all! Happy coding!