<feed xmlns='http://www.w3.org/2005/Atom'>
<title>catch3/src/catch2/internal/catch_run_context.cpp, branch v3.10.0</title>
<subtitle>Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results.</subtitle>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/'/>
<entry>
<title>Make message macros (FAIL, WARN, INFO, etc) thread safe</title>
<updated>2025-08-23T09:08:18+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-08-22T22:15:53+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=582200a1f8ddd6f435d1264668c7074121c2636c'/>
<id>582200a1f8ddd6f435d1264668c7074121c2636c</id>
<content type='text'>
This builds on the existing work to make assertion thread safe,
by adding an extra synchronization point in the holder of
`ReusableStringStream`'s stream instances, as those are used to
build the messages, and finishing the move of message scope holders
to be thread-local.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This builds on the existing work to make assertion thread safe,
by adding an extra synchronization point in the holder of
`ReusableStringStream`'s stream instances, as those are used to
build the messages, and finishing the move of message scope holders
to be thread-local.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add configuration option to make assertions thread-safe</title>
<updated>2025-07-24T08:10:00+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-17T20:58:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=2a8a8a7210513fc2f6ae5e65986c0f2d730c1537'/>
<id>2a8a8a7210513fc2f6ae5e65986c0f2d730c1537</id>
<content type='text'>
All the previous refactoring to make the assertion fast paths
smaller and faster also allows us to implement the fast paths
just with thread-local and atomic variables, without full mutexes.

However, the performance overhead of thread-safe assertions is
still significant for single threaded usage:

|  slowdown |  Debug  | Release |
|-----------|--------:|--------:|
| fast path |   1.04x |   1.43x |
| slow path |   1.16x |   1.22x |

Thus, we don't make the assertions thread-safe by default, and instead
provide a build-time configuration option that the users can set to get
thread-safe assertions.

This commit is functional, but it still needs some follow-up work:
 * We do not need full seq_cst increments for the atomic counters,
   and using weaker ones can be faster.
 * We brute-force updating the reporter-friendly totals from internal
   atomic counters by doing it everywhere. We should properly trace
   where this is needed instead.
 * Message macros (`INFO`, `UNSCOPED_INFO`, `CAPTURE`, etc) are not
   made thread safe in this commit, but they can be made thread safe
   in the future, by building on top of this work.
 * Add more tests, including with thread-sanitizer, and compiled
   examples to the repository. Right now, these changes have been
   compiled with tsan manually, but these tests are not added to CI.

Closes #2948
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
All the previous refactoring to make the assertion fast paths
smaller and faster also allows us to implement the fast paths
just with thread-local and atomic variables, without full mutexes.

However, the performance overhead of thread-safe assertions is
still significant for single threaded usage:

|  slowdown |  Debug  | Release |
|-----------|--------:|--------:|
| fast path |   1.04x |   1.43x |
| slow path |   1.16x |   1.22x |

Thus, we don't make the assertions thread-safe by default, and instead
provide a build-time configuration option that the users can set to get
thread-safe assertions.

This commit is functional, but it still needs some follow-up work:
 * We do not need full seq_cst increments for the atomic counters,
   and using weaker ones can be faster.
 * We brute-force updating the reporter-friendly totals from internal
   atomic counters by doing it everywhere. We should properly trace
   where this is needed instead.
 * Message macros (`INFO`, `UNSCOPED_INFO`, `CAPTURE`, etc) are not
   made thread safe in this commit, but they can be made thread safe
   in the future, by building on top of this work.
 * Add more tests, including with thread-sanitizer, and compiled
   examples to the repository. Right now, these changes have been
   compiled with tsan manually, but these tests are not added to CI.

Closes #2948
</pre>
</div>
</content>
</entry>
<entry>
<title>Clear unscoped messages lazily</title>
<updated>2025-07-21T19:50:51+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-21T17:45:43+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=5aa8d113215bd9a97ecc1a2f3fc9506947a2fa57'/>
<id>5aa8d113215bd9a97ecc1a2f3fc9506947a2fa57</id>
<content type='text'>
This improves the fast path performance for successful assertions
by about 7%, at the cost of potentially keeping around the message
allocation longer.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This improves the fast path performance for successful assertions
by about 7%, at the cost of potentially keeping around the message
allocation longer.
</pre>
</div>
</content>
</entry>
<entry>
<title>Simplify removal of messages from RunContext</title>
<updated>2025-07-21T19:50:49+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-21T17:25:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=9c5c21df82372b9d266f6b6bc1cf0edd71337109'/>
<id>9c5c21df82372b9d266f6b6bc1cf0edd71337109</id>
<content type='text'>
Instead of doing range-based removals, we only erase specific message,
because there can never be more than 1 message with the specific ID.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Instead of doing range-based removals, we only erase specific message,
because there can never be more than 1 message with the specific ID.
</pre>
</div>
</content>
</entry>
<entry>
<title>Regularize scoped message lifetime to only consider the object's scope</title>
<updated>2025-07-21T19:50:47+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-21T15:47:59+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=10aef62f21e52e5cd73bf0dc6ac8be35fbc4a798'/>
<id>10aef62f21e52e5cd73bf0dc6ac8be35fbc4a798</id>
<content type='text'>
This means that:
1) Scoped messages are always removed at the end of their scope,
   even if the scope ended due to an exception.
2) Scoped messages outlive section end, if that section's scope is
   enclosed in their own.

Previously neither of these were true, which has led to a number
of surprising behaviour, where e.g. this:
```cpp
TEST_CASE() {
    try {
        INFO( "some info" );
        throw std::runtime_error( "ex" );
    } catch (std::exception const&amp;) {}

    REQUIRE( false );
}
```
would print "some info" as the message for the assertion, while this:
```cpp
TEST_CASE() {
    INFO("Hello");
    SECTION("dummy") {}
    REQUIRE(false);
}
```
would not print out "Hello" as the message for the assertion.

This had an underlying reason, in that it was trying to helpfully
keep the messages around in case of unexpected exceptions, so that
code like this:
```cpp
TEST_CASE() {
    auto [input, expected] = GENERATE(...);
    CAPTURE(input);
    auto result = transform(input); // throws
    REQUIRE(result == expected);
}
```
would report the value of `input` when `transform` throws. However,
it was surprising in practice and was causing various issues around
handling of messages in other cases.

Closes #1759
Closes #2019
Closes #2959
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This means that:
1) Scoped messages are always removed at the end of their scope,
   even if the scope ended due to an exception.
2) Scoped messages outlive section end, if that section's scope is
   enclosed in their own.

Previously neither of these were true, which has led to a number
of surprising behaviour, where e.g. this:
```cpp
TEST_CASE() {
    try {
        INFO( "some info" );
        throw std::runtime_error( "ex" );
    } catch (std::exception const&amp;) {}

    REQUIRE( false );
}
```
would print "some info" as the message for the assertion, while this:
```cpp
TEST_CASE() {
    INFO("Hello");
    SECTION("dummy") {}
    REQUIRE(false);
}
```
would not print out "Hello" as the message for the assertion.

This had an underlying reason, in that it was trying to helpfully
keep the messages around in case of unexpected exceptions, so that
code like this:
```cpp
TEST_CASE() {
    auto [input, expected] = GENERATE(...);
    CAPTURE(input);
    auto result = transform(input); // throws
    REQUIRE(result == expected);
}
```
would report the value of `input` when `transform` throws. However,
it was surprising in practice and was causing various issues around
handling of messages in other cases.

Closes #1759
Closes #2019
Closes #2959
</pre>
</div>
</content>
</entry>
<entry>
<title>Deprecate comparison operators on MessageInfo</title>
<updated>2025-07-21T19:50:45+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-21T09:53:00+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=98b4bbb35efc3937b24e1de5559c974bd2578a14'/>
<id>98b4bbb35efc3937b24e1de5559c974bd2578a14</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Use the fast path for passing assertions in RunContext::handleNonExpr</title>
<updated>2025-07-20T19:01:10+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-20T16:40:05+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=715558fd97a48533fd41cb7dddf23b05d333cec6'/>
<id>715558fd97a48533fd41cb7dddf23b05d333cec6</id>
<content type='text'>
`handleNonExpr` is responsible for handling assertions that do not
result in a decomposable expression, e.g. `REQUIRE_THROWS`, or
`REQUIRE_NOTHROW`.

Running benchmark on these two macros specifically, with
`REQUIRE_THROWS([](){ throw 1; }())`, and `REQUIRE_NOTHROW([](){}())`,
we get these speedups:

|         |  Debug | Release |
|---------|--------|---------|
| THROWS  |  3.69x |   2.10x |
| NOTHROW |  1.18x |   1.05x |

Obviously the actual performance improvement is dependent on
how expensive the expression under test is.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
`handleNonExpr` is responsible for handling assertions that do not
result in a decomposable expression, e.g. `REQUIRE_THROWS`, or
`REQUIRE_NOTHROW`.

Running benchmark on these two macros specifically, with
`REQUIRE_THROWS([](){ throw 1; }())`, and `REQUIRE_NOTHROW([](){}())`,
we get these speedups:

|         |  Debug | Release |
|---------|--------|---------|
| THROWS  |  3.69x |   2.10x |
| NOTHROW |  1.18x |   1.05x |

Obviously the actual performance improvement is dependent on
how expensive the expression under test is.
</pre>
</div>
</content>
</entry>
<entry>
<title>Update last seen line info in the assertionEnded fast path</title>
<updated>2025-07-20T16:34:21+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-18T22:18:45+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=55b14e1b3482e1233f0fbf41dee8dceebee8db1e'/>
<id>55b14e1b3482e1233f0fbf41dee8dceebee8db1e</id>
<content type='text'>
This costs us about 1% perf in Debug build and 3% in Release build,
but it is worth it for more precise information during unexpected
exceptions or fatal errors.

Given a simple test case like this
```cpp
TEST_CASE("Hard fail") {
    REQUIRE( 1 == 1 );
    REQUIRE( 2 == 2 );
    throw 1;
    REQUIRE( 3 == 3 );
}
```

Catch2 before this change would report the line info from the
`TEST_CASE` macro as the last seen expression before error. With
this change, it will correctly report the line info from the
`REQUIRE(2 == 2)` assertion as the last seen expression before error.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This costs us about 1% perf in Debug build and 3% in Release build,
but it is worth it for more precise information during unexpected
exceptions or fatal errors.

Given a simple test case like this
```cpp
TEST_CASE("Hard fail") {
    REQUIRE( 1 == 1 );
    REQUIRE( 2 == 2 );
    throw 1;
    REQUIRE( 3 == 3 );
}
```

Catch2 before this change would report the line info from the
`TEST_CASE` macro as the last seen expression before error. With
this change, it will correctly report the line info from the
`REQUIRE(2 == 2)` assertion as the last seen expression before error.
</pre>
</div>
</content>
</entry>
<entry>
<title>Let reporters opt into fast path in RunContext::assertionStarting</title>
<updated>2025-07-17T11:47:44+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-17T11:13:08+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=c22096846c98211e0bcd23dbbd155698fbd718b3'/>
<id>c22096846c98211e0bcd23dbbd155698fbd718b3</id>
<content type='text'>
The fast path allows `RunContext` to skip disabling output redirect,
and notifying the reporters, turning `RunContext::notifyAssertionStarted`
into a no-op. This improves the overall performance of assertion handling
significantly, and also prepares ground for future changes around
assertion handling and thread safety.

For simple 10M assertion run, this improves the running time by ~30%
in Debug build and ~40% in Release build.

For backwards-compatibility reasons, the fast path is disabled
by default. However, none of the first party reporters use the
`assertionStarting` event, so all first party reporters are opted-in.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The fast path allows `RunContext` to skip disabling output redirect,
and notifying the reporters, turning `RunContext::notifyAssertionStarted`
into a no-op. This improves the overall performance of assertion handling
significantly, and also prepares ground for future changes around
assertion handling and thread safety.

For simple 10M assertion run, this improves the running time by ~30%
in Debug build and ~40% in Release build.

For backwards-compatibility reasons, the fast path is disabled
by default. However, none of the first party reporters use the
`assertionStarting` event, so all first party reporters are opted-in.
</pre>
</div>
</content>
</entry>
<entry>
<title>Only track the last line info in RunContext</title>
<updated>2025-07-17T09:27:03+00:00</updated>
<author>
<name>Martin Hořeňovský</name>
</author>
<published>2025-07-16T18:43:34+00:00</published>
<link rel='alternate' type='text/html' href='https://git.progmasoft.com/catch3/commit/?id=0e8112a7625bc415ec20aac9a5a62243c7fbfb36'/>
<id>0e8112a7625bc415ec20aac9a5a62243c7fbfb36</id>
<content type='text'>
There were only two places where we used the full `AssertionInfo`
instance in `m_lastAssertionInfo`:
1) when reporting unexpected exception from running a test case
2) when reporting fatal error

because in those two places we do not have access to a real
instance of `AssertionInfo`, but we still need to send one to the
reporters. As a bonus, in both of these places we were already
constructing a fake-ish assertion info, by using the last encountered
source location, but dummying out the other information.

Instead, we only keep track of the last encountered source location,
and construct the dummy `AssertionInfo` on-demand.

This finishes the set of refactoring around `m_lastAssertionInfo`
in `RunContext` and improves the performance of running assertions
by ~5% in both Debug and Release mode.

--------------

Note that this change also causes small difference in output. It
could be avoided by having an invalidation flag and tracking where
the information would be invalidated before, but the difference
includes more precise line location for unexpected errors (both
exceptions and fatals), so I prefer the new output.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
There were only two places where we used the full `AssertionInfo`
instance in `m_lastAssertionInfo`:
1) when reporting unexpected exception from running a test case
2) when reporting fatal error

because in those two places we do not have access to a real
instance of `AssertionInfo`, but we still need to send one to the
reporters. As a bonus, in both of these places we were already
constructing a fake-ish assertion info, by using the last encountered
source location, but dummying out the other information.

Instead, we only keep track of the last encountered source location,
and construct the dummy `AssertionInfo` on-demand.

This finishes the set of refactoring around `m_lastAssertionInfo`
in `RunContext` and improves the performance of running assertions
by ~5% in both Debug and Release mode.

--------------

Note that this change also causes small difference in output. It
could be avoided by having an invalidation flag and tracking where
the information would be invalidated before, but the difference
includes more precise line location for unexpected errors (both
exceptions and fatals), so I prefer the new output.
</pre>
</div>
</content>
</entry>
</feed>
