catch3/tests/CMakeLists.txt, branch v3.3.1 Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results. Add new SKIP macro for skipping tests at runtime (#2360) 2023-01-12T14:01:47+00:00 Philip Salzmann 2023-01-12T14:01:47+00:00 d548be26e3c54abc7e0b076319057d334e3ac589 * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com> 
* Add new SKIP macro for skipping tests at runtime

This adds a new `SKIP` macro for dynamically skipping tests at runtime.
The "skipped" status of a test case is treated as a first-class citizen,
like "succeeded" or "failed", and is reported with a new color on the
console.

* Don't show "skipped assertions" in console/compact reporters

Also extend skip tests to cover a few more use cases.

* Return exit code 4 if all test cases are skipped

* Use LightGrey for the skip colour

This isn't great, but is better than the deep blue that was borderline
invisible on dark backgrounds. The fix is to redo the colouring
a bit, including introducing light-blue that is actually visible.

* Add support for explicit skips in all reporters

* --allow-running-no-tests also allows all tests to be skipped

* Add docs for SKIP macro, deprecate IEventListener::skipTest

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
 Update basic cost estimates for approvals/benchmarks 2022-11-06T20:23:03+00:00 Martin Hořeňovský 2022-11-05T23:35:50+00:00 d1ef4614711ec157f61274b68c4065f801bd128a 






Don't write ApprovalTests' temporary files into the source tree
2022-11-05T23:11:38+00:00

Martin Hořeňovský

2022-11-05T23:11:32+00:00

0c75caf77b5c288a4a9461c08943e263b365180a

This has two effects:
 1) This significantly improves the performance of ApprovalTests
    run in WSL, because running the massively multi-reporter
    test would cause lots of small and parallel writes across the
    WSL-Windows FS boundary, completely mudering performance.

 2) ApprovalTests can be run from multiple build dirs in parallel,
    as long as they do not fail. However, if they do fail,
    the multiple runs will still step on each other toes when
    writing the unapproved files for user.





This has two effects:
 1) This significantly improves the performance of ApprovalTests
    run in WSL, because running the massively multi-reporter
    test would cause lots of small and parallel writes across the
    WSL-Windows FS boundary, completely mudering performance.

 2) ApprovalTests can be run from multiple build dirs in parallel,
    as long as they do not fail. However, if they do fail,
    the multiple runs will still step on each other toes when
    writing the unapproved files for user.







Support decomposing types that only compare with literal 0
2022-11-04T18:24:44+00:00

Martin Hořeňovský

2022-11-01T14:01:43+00:00

ec59cd8736b7ee8fcd69c76c71c2a0b72293b00c

This is primarily done to support new `std::*_ordering` types,
but the refactoring also supports any other type with this
property.

The compilation overhead is surprisingly low. Testing it with
clang on a Linux machine, compiling our SelfTest project takes
only 2-3% longer with these changes than it takes otherwise.

Closes #2555





This is primarily done to support new `std::*_ordering` types,
but the refactoring also supports any other type with this
property.

The compilation overhead is surprisingly low. Testing it with
clang on a Linux machine, compiling our SelfTest project takes
only 2-3% longer with these changes than it takes otherwise.

Closes #2555







Add traits for checking whether types are comparable
2022-11-04T18:24:42+00:00

Martin Hořeňovský

2022-11-01T07:48:23+00:00

d7f8c36e4cb856ee632b4ca01f8d5847984aa276












Split out parseTestSpec into test-only helpers
2022-10-28T11:27:46+00:00

Martin Hořeňovský

2022-10-27T22:16:38+00:00

3b40cf13eb5f56f83c3c5df4c61227187e3fe15d












Change reporters to report filters in round-trippable format
2022-10-27T18:46:59+00:00

Martin Hořeňovský

2022-10-27T18:43:00+00:00

d41da10c549f13d5d0d0fe54300c5f1d9f2082d9












Add test for round-tripping serialization of unmatched test specs
2022-10-27T17:11:20+00:00

Martin Hořeňovský

2022-10-27T17:11:20+00:00

d2294ad9b67001a56640eb0babde3a9fd3092bde












Provide round-tripping serialization for TestSpec
2022-10-27T16:23:59+00:00

Martin Hořeňovský

2022-10-27T15:10:28+00:00

e19ed221bde6b8d8a1a62a71ed942c238ae345a7












Split tests for TestSpec into its own file
2022-10-26T19:03:56+00:00

Martin Hořeňovský

2022-10-26T19:03:49+00:00

c6dfeb5e7dc175ca4765e884414dab52736cde9e

They also got slapped with the `[approvals]` tag in the process,
because we have too many approval tests and want less of them,
and these particular tests don't bring much value.

Related to #2090





They also got slapped with the `[approvals]` tag in the process,
because we have too many approval tests and want less of them,
and these particular tests don't bring much value.

Related to #2090