Read-only mirrors. Development, issues and pull requests are on GitHub.

summaryrefslogtreecommitdiffhomepage
path: root/docs
diff options
context:
space:
mode:
authorMartin Hořeňovský2025-08-23 00:15:53 +0200
committerMartin Hořeňovský2025-08-23 11:08:18 +0200
commit582200a1f8ddd6f435d1264668c7074121c2636c (patch)
tree06629812ca77ebbd77a35424162e197bb979d85e /docs
parentf4e05a67bb5a57ee6e051fc8c391b32cbf927f7d (diff)
downloadcatch3-582200a1f8ddd6f435d1264668c7074121c2636c.tar.gz
Make message macros (FAIL, WARN, INFO, etc) thread safe
This builds on the existing work to make assertion thread safe, by adding an extra synchronization point in the holder of `ReusableStringStream`'s stream instances, as those are used to build the messages, and finishing the move of message scope holders to be thread-local.
Diffstat (limited to 'docs')
-rw-r--r--docs/thread-safety.md131
1 files changed, 113 insertions, 18 deletions
diff --git a/docs/thread-safety.md b/docs/thread-safety.md
index c3f610f0..3d48eb0b 100644
--- a/docs/thread-safety.md
+++ b/docs/thread-safety.md
@@ -2,7 +2,9 @@
# Thread safety in Catch2
**Contents**<br>
-[Using assertion macros from multiple threads](#using-assertion-macros-from-multiple-threads)<br>
+[Using assertion macros from spawned threads](#using-assertion-macros-from-spawned-threads)<br>
+[Assertion-like message macros and spawned threads](#assertion-like-message-macros-and-spawned-threads)<br>
+[Message macros and spawned threads](#message-macros-and-spawned-threads)<br>
[examples](#examples)<br>
[`STATIC_REQUIRE` and `STATIC_CHECK`](#static_require-and-static_check)<br>
[Fatal errors and multiple threads](#fatal-errors-and-multiple-threads)<br>
@@ -10,17 +12,18 @@
> Thread safe assertions were introduced in Catch2 3.9.0
-Thread safety in Catch2 is currently limited to all the assertion macros.
-Interacting with benchmark macros, message macros (e.g. `INFO` or `CAPTURE`),
-sections macros, generator macros, or test case macros is not thread-safe.
-The message macros are likely to be made thread-safe in the future, but
-the way sections define test runs is incompatible with user being able
-to spawn threads arbitrarily, thus that limitation is here to stay.
+Thread safety in Catch2 is currently limited to all the assertion macros,
+and to message or message-adjacent macros (e.g. `INFO` or `WARN`).
+
+Interacting with benchmark macros, sections macros, generator macros, or
+test case macros is not thread-safe. The way sections define paths through
+the test is incompatible with user spawning threads arbitrarily, so this
+limitation is here to stay.
**Important: thread safety in Catch2 is [opt-in](configuration.md#experimental-thread-safety)**
-## Using assertion macros from multiple threads
+## Using assertion macros from spawned threads
The full set of Catch2's runtime assertion macros is thread-safe. However,
it is important to keep in mind that their semantics might not support
@@ -30,7 +33,7 @@ Specifically, the `REQUIRE` family of assertion macros have semantics
of stopping the test execution on failure. This is done by throwing
an exception, but since the user-spawned thread will not have the test-level
try-catch block ready to catch the test failure exception, failing a
-`REQUIRE` assertion inside this thread will terminate the process.
+`REQUIRE` assertion inside user-spawned thread will terminate the process.
The `CHECK` family of assertions does not have this issue, because it
does not try to stop the test execution.
@@ -38,16 +41,32 @@ does not try to stop the test execution.
Note that `CHECKED_IF` and `CHECKED_ELSE` are also thread safe (internally
they are assertion macro + an if).
-**`SKIP()`, `FAIL()`, `SUCCEED()` are not assertion macros, and are not
-thread-safe.**
+
+## Assertion-like message macros and spawned threads
+
+Similarly to assertion macros, not all assertion-like message macros can
+be used from spawned thread.
+
+`SKIP` and `FAIL` macros stop the test execution. Just like with `REQUIRE`,
+this means that they cannot be used inside user-spawned threads. `SUCCEED`,
+`FAIL_CHECK` and `WARN` do not attempt to stop the test execution and
+thus can be used from any thread.
+
+
+## Message macros and spawned threads
+
+Macros that add extra messages to following assertion, such as `INFO`
+or `CAPTURE`, are all thread safe and can be used in any thread. Note
+that these messages are per-thread, and thus `INFO` inside a user-spawned
+thread will not be seen by the main thread, and vice versa.
## examples
-### `REQUIRE` from main thread, `CHECK` from spawned threads
+### `REQUIRE` from the main thread, `CHECK` from spawned threads
```cpp
-TEST_CASE( "Failed REQUIRE in main thread is fine" ) {
+TEST_CASE( "Failed REQUIRE in the main thread is fine" ) {
std::vector<std::jthread> threads;
for ( size_t t = 0; t < 16; ++t) {
threads.emplace_back( []() {
@@ -85,7 +104,7 @@ TEST_CASE( "Successful REQUIRE in spawned thread is fine" ) {
This will also work as expected, because the `REQUIRE` is successful.
```cpp
-TEST_CASE( "Failed REQUIRE in spawned thread is fine" ) {
+TEST_CASE( "Failed REQUIRE in spawned thread kills the process" ) {
std::vector<std::jthread> threads;
for ( size_t t = 0; t < 16; ++t) {
threads.emplace_back( []() {
@@ -99,12 +118,88 @@ TEST_CASE( "Failed REQUIRE in spawned thread is fine" ) {
This will fail catastrophically and terminate the process.
+### INFO across threads
+
+```cpp
+TEST_CASE( "messages don't cross threads" ) {
+ std::jthread t1( [&]() {
+ for ( size_t i = 0; i < 100; ++i ) {
+ INFO( "spawned thread #1" );
+ CHECK( 1 == 1 );
+ }
+ } );
+
+ std::thread t2( [&]() {
+ for (size_t i = 0; i < 100; ++i) {
+ UNSCOPED_INFO( "spawned thread #2" );
+ }
+ } );
+
+ for (size_t i = 0; i < 100; ++i) {
+ CHECK( 1 == 2 );
+ }
+}
+```
+None of the failed checks will show the "spawned thread #1" message, as
+that message is for the `t1` thread. If the reporter shows passing
+assertions (e.g. due to the tests being run with `-s`), you will see the
+"spawned thread #1" message alongside the passing `CHECK( 1 == 1 )` assertion.
+
+The message "spawned thread #2" will never be shown, because there are no
+assertions in `t2`.
+
+
+### FAIL/SKIP from the main thread
+
+```cpp
+TEST_CASE( "FAIL in the main thread is fine" ) {
+ std::vector<std::jthread> threads;
+ for ( size_t t = 0; t < 16; ++t) {
+ threads.emplace_back( []() {
+ for (size_t i = 0; i < 10; ++i) {
+ CHECK( true );
+ CHECK( false );
+ }
+ } );
+ }
+
+ FAIL();
+}
+```
+
+This will work as expected, that is, the process will finish running
+normally, the test case will fail and there will be 321 total assertions,
+160 passing and 161 failing (`FAIL` counts as failed assertion).
+
+However, when the main thread hits `FAIL`, it will wait for the other
+threads to finish due to `std::jthread`'s destructor joining the spawned
+thread. Due to this, using `SKIP` is not recommended once more threads
+are spawned; while the main thread will bail from the test execution,
+the spawned threads will keep running and may fail the test case.
+
+
+### FAIL/SKIP from spawned threads
+
+```cpp
+TEST_CASE( "FAIL/SKIP in spawned thread kills the process" ) {
+ std::vector<std::jthread> threads;
+ for ( size_t t = 0; t < 16; ++t) {
+ threads.emplace_back( []() {
+ for (size_t i = 0; i < 10'000; ++i) {
+ FAIL();
+ }
+ } );
+ }
+}
+```
+As with failing `REQUIRE`, both `FAIL` and `SKIP` in spawned threads
+terminate the process.
+
+
## `STATIC_REQUIRE` and `STATIC_CHECK`
-None of `STATIC_REQUIRE`, `STATIC_REQUIRE_FALSE`, `STATIC_CHECK`, and
-`STATIC_CHECK_FALSE` are currently thread safe. This might be surprising
-given that they are a compile-time checks, but they also rely on the
-message macros to register the result with reporter at runtime.
+All of `STATIC_REQUIRE`, `STATIC_REQUIRE_FALSE`, `STATIC_CHECK`, and
+`STATIC_CHECK_FALSE` are thread safe in the delayed evaluation configuration.
## Fatal errors and multiple threads