diff options
| author | Martin Hořeňovský | 2025-08-23 00:15:53 +0200 |
|---|---|---|
| committer | Martin Hořeňovský | 2025-08-23 11:08:18 +0200 |
| commit | 582200a1f8ddd6f435d1264668c7074121c2636c (patch) | |
| tree | 06629812ca77ebbd77a35424162e197bb979d85e /docs | |
| parent | f4e05a67bb5a57ee6e051fc8c391b32cbf927f7d (diff) | |
| download | catch3-582200a1f8ddd6f435d1264668c7074121c2636c.tar.gz | |
Make message macros (FAIL, WARN, INFO, etc) thread safe
This builds on the existing work to make assertion thread safe,
by adding an extra synchronization point in the holder of
`ReusableStringStream`'s stream instances, as those are used to
build the messages, and finishing the move of message scope holders
to be thread-local.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/thread-safety.md | 131 |
1 files changed, 113 insertions, 18 deletions
diff --git a/docs/thread-safety.md b/docs/thread-safety.md index c3f610f0..3d48eb0b 100644 --- a/docs/thread-safety.md +++ b/docs/thread-safety.md @@ -2,7 +2,9 @@ # Thread safety in Catch2 **Contents**<br> -[Using assertion macros from multiple threads](#using-assertion-macros-from-multiple-threads)<br> +[Using assertion macros from spawned threads](#using-assertion-macros-from-spawned-threads)<br> +[Assertion-like message macros and spawned threads](#assertion-like-message-macros-and-spawned-threads)<br> +[Message macros and spawned threads](#message-macros-and-spawned-threads)<br> [examples](#examples)<br> [`STATIC_REQUIRE` and `STATIC_CHECK`](#static_require-and-static_check)<br> [Fatal errors and multiple threads](#fatal-errors-and-multiple-threads)<br> @@ -10,17 +12,18 @@ > Thread safe assertions were introduced in Catch2 3.9.0 -Thread safety in Catch2 is currently limited to all the assertion macros. -Interacting with benchmark macros, message macros (e.g. `INFO` or `CAPTURE`), -sections macros, generator macros, or test case macros is not thread-safe. -The message macros are likely to be made thread-safe in the future, but -the way sections define test runs is incompatible with user being able -to spawn threads arbitrarily, thus that limitation is here to stay. +Thread safety in Catch2 is currently limited to all the assertion macros, +and to message or message-adjacent macros (e.g. `INFO` or `WARN`). + +Interacting with benchmark macros, sections macros, generator macros, or +test case macros is not thread-safe. The way sections define paths through +the test is incompatible with user spawning threads arbitrarily, so this +limitation is here to stay. **Important: thread safety in Catch2 is [opt-in](configuration.md#experimental-thread-safety)** -## Using assertion macros from multiple threads +## Using assertion macros from spawned threads The full set of Catch2's runtime assertion macros is thread-safe. However, it is important to keep in mind that their semantics might not support @@ -30,7 +33,7 @@ Specifically, the `REQUIRE` family of assertion macros have semantics of stopping the test execution on failure. This is done by throwing an exception, but since the user-spawned thread will not have the test-level try-catch block ready to catch the test failure exception, failing a -`REQUIRE` assertion inside this thread will terminate the process. +`REQUIRE` assertion inside user-spawned thread will terminate the process. The `CHECK` family of assertions does not have this issue, because it does not try to stop the test execution. @@ -38,16 +41,32 @@ does not try to stop the test execution. Note that `CHECKED_IF` and `CHECKED_ELSE` are also thread safe (internally they are assertion macro + an if). -**`SKIP()`, `FAIL()`, `SUCCEED()` are not assertion macros, and are not -thread-safe.** + +## Assertion-like message macros and spawned threads + +Similarly to assertion macros, not all assertion-like message macros can +be used from spawned thread. + +`SKIP` and `FAIL` macros stop the test execution. Just like with `REQUIRE`, +this means that they cannot be used inside user-spawned threads. `SUCCEED`, +`FAIL_CHECK` and `WARN` do not attempt to stop the test execution and +thus can be used from any thread. + + +## Message macros and spawned threads + +Macros that add extra messages to following assertion, such as `INFO` +or `CAPTURE`, are all thread safe and can be used in any thread. Note +that these messages are per-thread, and thus `INFO` inside a user-spawned +thread will not be seen by the main thread, and vice versa. ## examples -### `REQUIRE` from main thread, `CHECK` from spawned threads +### `REQUIRE` from the main thread, `CHECK` from spawned threads ```cpp -TEST_CASE( "Failed REQUIRE in main thread is fine" ) { +TEST_CASE( "Failed REQUIRE in the main thread is fine" ) { std::vector<std::jthread> threads; for ( size_t t = 0; t < 16; ++t) { threads.emplace_back( []() { @@ -85,7 +104,7 @@ TEST_CASE( "Successful REQUIRE in spawned thread is fine" ) { This will also work as expected, because the `REQUIRE` is successful. ```cpp -TEST_CASE( "Failed REQUIRE in spawned thread is fine" ) { +TEST_CASE( "Failed REQUIRE in spawned thread kills the process" ) { std::vector<std::jthread> threads; for ( size_t t = 0; t < 16; ++t) { threads.emplace_back( []() { @@ -99,12 +118,88 @@ TEST_CASE( "Failed REQUIRE in spawned thread is fine" ) { This will fail catastrophically and terminate the process. +### INFO across threads + +```cpp +TEST_CASE( "messages don't cross threads" ) { + std::jthread t1( [&]() { + for ( size_t i = 0; i < 100; ++i ) { + INFO( "spawned thread #1" ); + CHECK( 1 == 1 ); + } + } ); + + std::thread t2( [&]() { + for (size_t i = 0; i < 100; ++i) { + UNSCOPED_INFO( "spawned thread #2" ); + } + } ); + + for (size_t i = 0; i < 100; ++i) { + CHECK( 1 == 2 ); + } +} +``` +None of the failed checks will show the "spawned thread #1" message, as +that message is for the `t1` thread. If the reporter shows passing +assertions (e.g. due to the tests being run with `-s`), you will see the +"spawned thread #1" message alongside the passing `CHECK( 1 == 1 )` assertion. + +The message "spawned thread #2" will never be shown, because there are no +assertions in `t2`. + + +### FAIL/SKIP from the main thread + +```cpp +TEST_CASE( "FAIL in the main thread is fine" ) { + std::vector<std::jthread> threads; + for ( size_t t = 0; t < 16; ++t) { + threads.emplace_back( []() { + for (size_t i = 0; i < 10; ++i) { + CHECK( true ); + CHECK( false ); + } + } ); + } + + FAIL(); +} +``` + +This will work as expected, that is, the process will finish running +normally, the test case will fail and there will be 321 total assertions, +160 passing and 161 failing (`FAIL` counts as failed assertion). + +However, when the main thread hits `FAIL`, it will wait for the other +threads to finish due to `std::jthread`'s destructor joining the spawned +thread. Due to this, using `SKIP` is not recommended once more threads +are spawned; while the main thread will bail from the test execution, +the spawned threads will keep running and may fail the test case. + + +### FAIL/SKIP from spawned threads + +```cpp +TEST_CASE( "FAIL/SKIP in spawned thread kills the process" ) { + std::vector<std::jthread> threads; + for ( size_t t = 0; t < 16; ++t) { + threads.emplace_back( []() { + for (size_t i = 0; i < 10'000; ++i) { + FAIL(); + } + } ); + } +} +``` +As with failing `REQUIRE`, both `FAIL` and `SKIP` in spawned threads +terminate the process. + + ## `STATIC_REQUIRE` and `STATIC_CHECK` -None of `STATIC_REQUIRE`, `STATIC_REQUIRE_FALSE`, `STATIC_CHECK`, and -`STATIC_CHECK_FALSE` are currently thread safe. This might be surprising -given that they are a compile-time checks, but they also rely on the -message macros to register the result with reporter at runtime. +All of `STATIC_REQUIRE`, `STATIC_REQUIRE_FALSE`, `STATIC_CHECK`, and +`STATIC_CHECK_FALSE` are thread safe in the delayed evaluation configuration. ## Fatal errors and multiple threads |