Investigating the performance overhead of C++ exceptions

This article was first published in February 2020 and was updated in July 2024.

Runtime error handling is hugely important for many common operations we encounter in software development — from responding to user input, to dealing with a malformed network packet. An application shouldn’t crash because a user has loaded a PNG instead of a PDF, or if they disconnect the network cable while it’s fetching the latest version of Nutrient Android SDK. Users expect a program to gracefully handle errors, either quietly in the background, or with an actionable and user-friendly message.

Dealing with exceptions can be messy, complicated, and — crucially, for many C++ developers — slow. But as with so many things in software development, there are many roads that lead to Rome. This blog post will take a closer look at C++ exceptions, what their downsides are, how they might affect the performance of your application, and what alternatives we have at our disposal to lessen their use.

The purpose of this post is not to deter us from using exceptions entirely; exceptions have their use, and in many cases, they’re unavoidable. (Consider an error detected inside a constructor. How do we report that error? Throwing an exception is your best bet.) Rather, this post is focused on using exceptions for general flow control, and it provides us with an alternative to help develop robust and easy-to-follow applications.

Exceptional complexity

Exceptions are much more complicated than a simple break or return. Here are a few things that affect C++ exception performance:

• Stack unwinding — When an exception is thrown, the runtime must unwind the stack, which involves walking back through the call stack to find the exception handler. This process can be time-consuming, thus affecting C++ exception performance, because it may involve calling destructors for all objects that go out of scope as the stack unwinds.

• Side table lookups — The compiler generates a side table that maps points in the code that may throw an exception to a list of handlers. When an exception is thrown, this list is consulted to pick the correct handler. This can involve cache misses and fetching data from memory, which is slower compared to CPU operations.

• Runtime yype identification (RTTI) — Determining the correct exception handler involves RTTI, which requires fetching many RTTI descriptors scattered around memory and running complex operations, akin to a dynamic_cast test for each handler.

• Code bloat — Exception handling can increase the size of the binary due to additional code for handling exceptions, which can affect the overall performance due to larger memory footprint and potential cache misses.

A quick benchmark test

But exactly how much slower are exceptions in C++ compared to simpler flow control mechanisms? How much extra work does the processor have to do? And how long does it take? Let’s put it to the test with SCIENCE!

In the code below, we have a simple function that generates a random number and checks for a certain number before producing an error. The random number check ensures the computer has some work to do at runtime so that the compiler cannot simply optimize away our tests. Here are our test cases:

1. Exit by throwing an int. Although not especially practical, this is the simplest exception we can do in C++, and it’s useful for stripping out as much complication as possible for this particular test.

2. Exit by throwing an std::runtime_error, which can contain a string message. This is much more practical than the int example. We’ll see if there’s much overhead in addition to the added complexity.

3. Exit with a void return.

4. Exit with a C-style int error code.

The test is run using Google’s lightweight benchmark library, which runs many cycles of each test. Eager readers may want to skip straight to the results.

The code

Our super complex random number generator:

const int randomRange = 2;  // Give me a number between 0 and 2.
const int errorInt = 0;     // Stop every time the number is 0.
int getRandom() {
    return random() % randomRange;
}

And the test functions:

// 1.
void exitWithBasicException() {
    if (getRandom() == errorInt) {
        throw -2;
    }
}
// 2.
void exitWithMessageException() {
    if (getRandom() == errorInt) {
        throw std::runtime_error("Halt! Who goes there?");
    }
}
// 3.
void exitWithReturn() {
    if (getRandom() == errorInt) {
        return;
    }
}
// 4.
int exitWithErrorCode() {
    if (getRandom() == errorInt) {
        return -1;
    }
    return 0;
}

Finally, we can integrate our tests with the Google benchmark library:

// 1.
void BM_exitWithBasicException(benchmark::State& state) {
    for (auto _ : state) {
        try {
            exitWithBasicException();
        } catch (int ex) {
            // Caught! Carry on next iteration.
        }
    }
}
// 2.
void BM_exitWithMessageException(benchmark::State& state) {
    for (auto _ : state) {
        try {
            exitWithMessageException();
        } catch (const std::runtime_error &ex) {
            // Caught! Carry on next iteration.
        }
    }
}
// 3.
void BM_exitWithReturn(benchmark::State& state) {
    for (auto _ : state) {
        exitWithReturn();
    }
}
// 4.
void BM_exitWithErrorCode(benchmark::State& state) {
    for (auto _ : state) {
        auto err = exitWithErrorCode();
        if (err < 0) {
            // `handle_error()` ...
        }
    }
}

// Add the tests.
BENCHMARK(BM_exitWithBasicException);
BENCHMARK(BM_exitWithMessageException);
BENCHMARK(BM_exitWithReturn);
BENCHMARK(BM_exitWithErrorCode);

// Run the tests!
BENCHMARK_MAIN();

For readers who wish to try it out for themselves, the full test code can be found here.

The results

Below we have the output from the benchmark — first without any compiler optimizations, and then with -O2.

Debug -O0:

------------------------------------------------------------------
Benchmark                        Time             CPU   Iterations
------------------------------------------------------------------
BM_exitWithBasicException     1407 ns         1407 ns       491232
BM_exitWithMessageException   1605 ns         1605 ns       431393
BM_exitWithReturn              142 ns          142 ns      5172121
BM_exitWithErrorCode           144 ns          143 ns      5069378

Release -O2:

------------------------------------------------------------------
Benchmark                        Time             CPU   Iterations
------------------------------------------------------------------
BM_exitWithBasicException     1092 ns         1092 ns       630165
BM_exitWithMessageException   1261 ns         1261 ns       547761
BM_exitWithReturn             10.7 ns         10.7 ns     64519697
BM_exitWithErrorCode          11.5 ns         11.5 ns     62180216

(Run on 2015 MacBook Pro 2.5GHz i7)

The results are pretty staggering! We can see a huge gap in time taken to run the exception code vs. the basic return or error code approach. With compiler optimizations, there’s an even greater difference.

This is by no means a perfect test. The complier can probably do a lot of optimization with the code we have supplied in tests 3 and 4. Caveats aside, the differences are huge and the test demonstrates how much overhead we can expect to see when we throw and catch exceptions instead of error handling with function return codes.

Thanks to the zero-cost exception model used in most C++ implementations (see section 5.4 of TR18015), the code in a try block runs without any overhead. However, a catch block is orders of magnitude slower. In our simple example, we can see how slow throwing and catching an exception can be, even in a tiny call stack! Speed will decrease linearly with the depth of the call stack, which is why it’s always best to catch an exception as close to the throw point as possible.

So if exceptions are so slow, why do we use them?

Why use exceptions?

The benefits to exceptions are explained nice and succinctly in the Technical Report on C++ Performance (section 5.4):

The use of exceptions isolates the error handling code from the normal flow of program execution, and unlike the error code approach, it cannot be ignored or forgotten. Also, automatic destruction of stack objects when an exception is thrown renders a program less likely to leak memory or other resources. With exceptions, once a problem is identified, it cannot be ignored – failure to catch and handle an exception results in program termination.

The key takeaway here is that an exception cannot be ignored or forgotten; if we have an exception, it must be dealt with. This makes exceptions extremely powerful built-in tools in C++, and something that no simple C-style error code can replace. Exceptions are useful for situations that are out of the program’s control, e.g. the hard disk is full, or a mouse has chewed through your network cable. In such rare situations, an exception is an ideal and perfectly performant tool for the task.

But what about all those errors inside the program’s control? If a function can produce an error, we want a mechanism where it’s semantically obvious to the programmer that they need to check for an error, while also providing useful information about the error, if it occurs, in the form of a message or some other data (much like an exception).

Expected

We know that exceptions are really slow, and if you’re programming in C++, you generally don’t want slow — especially for general flow control error handling. Exceptions are also hopelessly serial, meaning they must be dealt with immediately, and they don’t allow for storing of an error to be handled at a later time.

With the upsides and downsides in mind, what alternatives do we have at our disposal? Here at Nutrient, we use a class called Expected<T> which is an idea originally proposed by Dr. Andrei Alexandrescu. His talk on Systematic Error Handling in C++ is excellent, and it’s well worth a watch to help you understand the full power of this strategy.

The following pseudocode gives an idea of how Expected<T> can look. Expected<T> allows for either a T to be created or the exception that prevents T to be created. Simply put, it’s a wrapper for a union of an expected return value and an error:

template <class T>
class Expected {
private:
    // Our union of value and error. Only one of these can exist in any `Expected` object created.
    union {
        T value;
        Exception exception;
    };

public:
    // Instantiate the `Expected` object with the successfully created value.
    Expected(const T& value) ...

    // Instantiate the `Expected` object with an exception.
    Expected(const Exception& ex) ...

    // Was there an error?
    bool hasError() ...

    // Access the value.
    T value() ...

    // Access the exception.
    Exception error() ...
};

The real-world implementation is a bit more complex than this; the talk mentioned above will fill in all the nitty-gritty implementation details.

So the basic idea behind Expected is that a function that can produce an error has an expected return value of a certain type (an int, a class, void, and so on). If the function call succeeds, the return value is stored in an Expected instance and retrieved with the value() accessor. If something goes wrong, the error is stored in the same instance of Expected and retrieved with the error() accessor. Once the function has been called, it’s easy enough to check whether we have the value or an error. If there’s an error, there’s no need for a slow and “must-be-handled-immediately” catch block; instead, we can check for hasError() and get the error message whenever it’s suitable.

Speed test!

Let’s plug our Expected class into the test functions described above and see how it can be used:

// 5. Expected! Testcase 5 function.
Expected<int> exitWithExpected() {
    if (getRandom() == errorInt) {
        return std::runtime_error("Halt! If you want...");  //  Return; don't throw!
    }
    return 0;
}

// Benchmark function with example usage.
void BM_exitWithExpected(benchmark::State& state) {
    for (auto _ : state) {
        auto expected = exitWithExpected();

        if (expected.hasError()){
            // Handle in our own time.
        }
        // Or we can use the value...
        // else {
        //     doSomethingInteresting(expected.value());
        // }
    }
}

// Add the test.
BENCHMARK(BM_exitWithExpected);

// Run the test...
BENCHMARK_MAIN();

Drumroll please…

Debug -O0:

------------------------------------------------------------------
Benchmark                        Time             CPU   Iterations
------------------------------------------------------------------
BM_exitWithExpected            147 ns          147 ns      4685942

Release -O2:

------------------------------------------------------------------
Benchmark                        Time             CPU   Iterations
------------------------------------------------------------------
BM_exitWithExpected           57.5 ns         57.5 ns     11873261

Not bad! We’ve managed to take our no-optimizations std::runtime_error result from 1605 ns to a mere 147 ns. The results are even more impressive with optimizations: 1261 ns to 57.5 ns. That’s more than 10 times faster with -O0 and well more than 20 times faster in with -O2!

So there we have it: Expected gives us a much faster and more versatile control flow error-handling mechanism than exceptions do. It’s also semantically clear, and we don’t need to compromise on error messages.

At Nutrient, code performance is a high priority for us. We use the Expected return technique to speed up our code while failing quickly and gracefully when required — all the while retaining the ability to show our users (and ourselves) descriptive, type-safe, and information-packed error messages from our APIs.

Conclusion

Exceptions aren’t all bad. In fact, they’re extremely performant at what they’re designed for: exceptional circumstances! We only start running into problems when we throw exceptions for general control flow, where much more efficient solutions are already available.

The benchmark tests, albeit rather crude, showed us the huge gains in performance we can achieve by avoiding throwing and catching exceptions when a return will suffice. Benchmarking is incredibly valuable for putting real data to ideas and essential for maintaining a performant document SDK used by millions around the world.

In this post, we also briefly got a glimpse of the Expected<T> class and how we can use this design to speed up our error handling. Expected allows us to be more flexible with when to handle errors, while also keeping the code flow easy to follow and retaining our descriptive messages for our users and programmers.

FAQ

Here are a few frequently asked questions about the performance overhead of C++ exceptions.

What is the performance impact of using C++ exceptions?

C++ exceptions can have a significant performance overhead due to stack unwinding, RTTI, and side table lookups during exception handling.

Are exceptions always a bad choice for error handling in C++?

No, exceptions are useful for handling unexpected, exceptional situations. However, for general program flow, alternatives like error codes or Expected<T> are recommended.

What alternatives to C++ exceptions can improve performance?

Alternatives like using error codes or classes like Expected<T> can significantly improve performance while keeping code readable and maintainable.

Why are exceptions slow in C++?

The slowness of exceptions comes from the overhead of stack unwinding, dynamic type checks, and additional memory operations that happen when an exception is thrown.

What is the benefit of using the Expected class over exceptions?

The Expected<T> class allows handling errors in a more flexible, efficient way without the immediate performance hit associated with exceptions.