flask

When Exceptions Aren’t the Right Thing: Reporting and Handling Errors at the Same Time in Python

When writing web application code (as opposed to, say, a packaged library) it’s common to have engineers notified when unhandled exceptions are thrown—these are nearly always unexpected conditions that should be investigated. We use Sentry (https://www.getsentry.com) for this, which has served us pretty well.

It brings up a question, though: how should your code (in our case, python / flask) handle unexpected conditions? Here’s a super simple illustration:

# member should always exist if we get into this code path
member.do_something()
# member should always exist if we get into this code path, but let's catch
# the error so that the user doesn't see an internal server error.
if member:
    member.do_something()
else:
    log("ERROR: member is not expected to be None here")

If, due to some error elsewhere in the code, member is None, the first piece of code has the advantage of throwing an exception, and therefore properly notifying engineers that the unexpected case is happening and should be investigated. But, is has the disadvantage of returning a 500 error to the user.

The second case fixes the 500, perhaps (depending on the situation) improving the user experience, but doesn’t do enough to alert engineering to investigate the underlying issue. Maybe you’re digging through your logs for errors like this, but it’s best & easiest to have it all in one place.

So better still is:

# member should always exist if we get into this code path, but let's catch
# the error so that the user doesn't see an internal server error.
if member:
    member.do_something()
else:
    report_error_to_sentry()

We’ve implemented a simple wrapper for this case called verify (the name comes from the old days of C—it was common practice for ASSERT to be a macro that would compile out in release builds, where VERIFY would not). We have:

# member should always exist if we get into this code path, but let's catch
# the error so that the user doesn't see an internal server error.
if verify(member):
    member.do_something()

I love this! It gives us the error reporting and error handling we want, with just about as little code overhead as you can imagine. Further, it’s one central place where all the error reporting is done, so you can report the errors in a consistent way, including any other reporting you might want to do other than Sentry.

verify has a super-simple implementation:

def verify(condition):
    if not condition:
        report_error_to_sentry()
    return condition

As for report_error_to_sentry()—it turns out getting a stack trace into Sentry and continuing execution is a bit tricky. The library we use to talk to Sentry is called raven, and I had to dig through some of that code to pick out the pieces I needed, including using fragments of both the traceback and inspect libraries. Here’s a simplified version of where I ended up:

import inspect
import traceback
from raven.utils import stacks

def report_error_to_sentry():
    # we always ignore the call to report_error_to_sentry() itself
    framesToIgnore = 1

    # much of this code is adapted from raven's exception capturing
    # implementation. apparently, inspect.stack() and
    # traceback.extract_stack() return their stacks in opposite order; I need
    # this reversed() to get the stack order correct on the sentry side.
    stackframes = reversed(inspect.stack()[framesToIgnore:])

    # this is a raven function that filters/formats the stack frames in
    # how they want it.
    sentryframes = stacks.iter_stack_frames(stackframes)

    # include the line of code from the most recent stack frame
    codeLine = traceback.extract_stack()[-framesToIgnore-1][3]
    message = 'Error: {}'.format(codeLine)

    # unfortunately, this call to capture the exception block on the request to
    # sentry's servers. For now the hope is that this will be rare enough that
    # this is not a big deal -- hopefully eventually raven will provide some
    # kind of way to do this asynchronously.
    #
    # TODO: besides including stack information, we could extend this to
    # include all the web request information that raven's flask implementation
    # includes (checking of course to make sure such information exists, since
    # this code can also be called in standalone apps).
    #
    # sentry_client is a raven.Client()
    sentry_client.captureMessage(message, stack=sentryframes)

Hope someone finds this helpful or interesting! It’s a simple idea, but it’s one of my favorite things.

Discussion

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s