Why not improve error handling in our Go projects?

As I was saying already, I love Go, I really do. And in particular I like the minimalistic approach to error handling: no exceptions, each caller should check the error explicitly and deal with it as it wants.

value, err := DoSomething()
if err != nil {
        // handle error
}

On the cost of some repetition here and there, it makes programs look very explicit, which helps readers to understand the code better, instead of wondering: can the exception be thrown here or not?

However, I believe we could do a lot better than the majority does. Quite often, the error is just returned further as it is:

value, err := DoSomething()
if err != nil {
        return err
}

The problem with it is that the errors end up being too specific and often confusing. For example, imagine we have an utility which parses some config file, and reads some kind of projects accordingly to the config. To make it less abstract, here's an example config file:

projects:
  foo:
    ~/projects/foo/file1
    ~/projects/foo/file2
    ~/projects/anotherfile
  bar:
    ~/projects/bar/file1
    ~/projects/bar/file2

The application is likely to have a few levels: at least, to open a particular config file, and to read files of a particular project.

And now imagine that the file ~/projects/anotherfile doesn't exist, and when we run the utility, we get this:

open /home/bob/projects/anotherfile: no such file or directory

It is very specific, and doesn't contain any context about why and when do we need this file in the first place. The user might have an assumption that some other config file was used, for example, and the error message doesn't help him. This might be really really annoying and confusing, both for the users and developers.

So, let me introduce juju/errors package to you: https://github.com/juju/errors. It's not new, it has been around for a few years already, but apparently very few people use it, so let me spread a word a bit more.

This package provides helpers to annotate errors, so that each level of our application could add the context which it knows about the error before returning it further, like this:

if err := DoSomething(); err != nil {
        return errors.Annotatef(err, "doing %s and %s", "this", "that")
}

Those annotations are prepended to the original message, adding a context which we have on each level. For example, this snippet:

err := errors.New("some specific error")
err = errors.Annotatef(err, "doing that")
err = errors.Annotatef(err, "doing this")
fmt.Println(err)

Would print this:

doing this: doing that: some specific error

So let's get back to our example with reading a non-existing project file. Armed with the annotation tools, on the level of handling a project we would do something like this:

// projName is a name of the project we're currently handling
 
if err := ReadProjectFile(curFilename); err != nil {
        return errors.Annotatef(err, "opening project %q", projName)
}

And on the level above, when we parse the config file:

// configFilename is a file we have read cfg from
 
if err := HandleConfig(cfg); err != nil {
        return errors.Annotatef(err, "parsing config file %s", configFilename)
}

And in the end, the error caused by the non-existing file will look like:

parsing config file /foo/bar/cfg.yml: opening project "foo": open /home/bob/projects/anotherfile: no such file or directory

The message is very descriptive and it's immediately clear what went wrong. Sometimes it is overly verbose, but in practice it's still quite readable and understandable, and it's arguably better to have error messages which are a bit too verbose, rather than not verbose enough.

This approach has proven to be immensely useful in projects I have worked on so far.

Actually, errors.Annotatef() not only prepends the error message with the new context, it also stores the filename and line number, which can be retrieved with errors.ErrorStack():

err := errors.New("some specific error")
err = errors.Annotatef(err, "doing that")
err = errors.Annotatef(err, "doing this")
fmt.Println(errors.ErrorStack(err))

Which provides the output like this:

/path/to/file.go:10: some specific error
/path/to/file.go:11: doing that
/path/to/file.go:12: doing this

Also, it's not always appropriate to use errors.Annotatef(): sometimes we don't have any context useful enough to be included in the error messages. In this case, just use errors.Trace(), which only adds filename and line number, and leaves the message intact.

Another example with Trace:

err := errors.New("some specific error")
err = errors.Trace(err)
err = errors.Annotatef(err, "doing this")
fmt.Println(err)
fmt.Println("---")
fmt.Println(errors.ErrorStack(err))

The output would be:

doing this: some specific error
---
/path/to/file.go:10: some specific error
/path/to/file.go:11:
/path/to/file.go:12: doing this

Coming up with a good annotation message could be burdensome, but errors.Trace() is basically free, so I tend to use it more often than Annotatef. Still, when something goes wrong and I can't quickly figure what exactly just by looking at the error message, this tracing provides me with the thread of Ariadne, so following it I recover the cause of the error. And having to do so usually means that I should finally replace a few Traces with proper annotations.

Needless to say, I (almost) never return a plain error which is received from a called function, because in that case this saving thread is lost. I said “almost” because there are a few gotchas, so let's talk about them now.

So the biggest problem with the juju/errors package is that it's not widely used, and let me explain why it's a problem. It's not just because errors returned from 3rd-party code are not descriptive enough, it's because we should write our code so that it's compatilbe with the outside world. In the end, it just boils down to being careful.

So, wrapped error values work great for us as long as we use them consistently in our own code, but right when we leak them to stdlib or 3rd-party code which knows nothing about juju/errors, we're in trouble. So we have to be careful not to.

The most often issue which I myself stepped into multiple times is the code like this:

if os.IsNotExist(err) {
        // Do something special for non-existing file
}

In the Go world, errors are often compared by value, to distinguish different kinds of errors and behave accordingly. But as we begin wrapping errors, these comparisons aren't valid anymore. So when we pass errors to any code which we don't own (like the os.IsNotExist function above), we have to pass the original error value. This is what errors.Cause is for: it returns the original, unwrapped value, no matter how many times we wrapped it already. So, the correct code in this case would be:

if os.IsNotExist(errors.Cause(err)) {
        // Do something special for non-existing file
}

And another form of the same issue is when we provide callbacks to the stdlib or 3rd-party code. Of course, these callbacks also should never return wrapped errors: e.g. the Serve() method of http.Server alters its behavior depending on the kind of error returned from the listener's Accept method.

When building, say, a backend service (a common use case for Go), we often want to distinguish external errors (e.g. caused by the invalid input received from a client) and internal errors, like the unability to execute SQL query or something else. In the former case, the client should be provided with the descriptive error message like handling bookmarks: saving bookmark "foo bar": URL can't be empty. In the latter case though, the client should only get internal error, and we want to log the exact (and descriptive) internal error for us to examine.

So I wrote a small package for that: interrors. Here's how it can be used.

In our project we can create an error value to represent internal server error:

internalServerError := errors.New("internal server error")

Now, let's imagine we have some SQL execution error:

origErr := errors.New("some SQL execution error")
 
err := origErr
err = errors.Annotatef(err, "some context")
err = errors.Annotatef(err, "some more context")

This error should not be reported to the client, so at the point we realize we've got an internal problem, we do this:

err = interrors.WrapInternalError(err, internalServerError)

So from now on, err can be further wrapped using errors.Annotatef() or friends, but it will behave like the cause was internalServerError, not origErr.

err = errors.Annotatef(err, "some public context")
 
fmt.Println(err)  // Prints "some public context: internal server error"
errors.Cause(err) // Returns internalServerError
errors.ErrorStack(err) // Returns error stack up to internalServerError, not further

And, of course, there are helpers to get the details of an internal error back, when we need to log them:

fmt.Println(interrors.InternalErr(err))   // Prints "some more context: some context: specific error"
fmt.Println(interrors.InternalCause(err)) // Prints "specific error"
fmt.Println(interrors.ErrorStack(err))    // Prints full error stack, from "some public context" to "specific error"

I use it in my beloved Geekmarks, and so far I'm quite happy with it.

Many people complain that error handling is not something Go did right. So I'm of two minds here. On the one hand, I like the explicitness of it, it really makes reading the code easier. On the other, I dislike how the majority, including the authors of Go, uses it.

Of course I would be glad if something like juju/errors becomes standard, but I assume it'll never happen, alas.

At least I can improve error handling in the projects I work on, and be careful to interact with the outside world properly.

You might as well be interested in:

• Here's why I love Go