diff options
| author | Martin Hořeňovský | 2026-07-26 13:15:10 +0200 |
|---|---|---|
| committer | Martin Hořeňovský | 2026-07-26 19:51:58 +0200 |
| commit | 1079da4c5f5fea2a31d795bba95eacaa2f622849 (patch) | |
| tree | 4f708b68ab9038d75a50e15467b3ea8182e939f1 /tests/TestScripts/DiscoverTests/register-tests.cpp | |
| parent | 60c8b878292ec4afc625a807cdc3902397a408a3 (diff) | |
| download | catch3-1079da4c5f5fea2a31d795bba95eacaa2f622849.tar.gz | |
Avoid quadratic JSON array parse behaviour in `catch_discover_tests`
Using CMake's `string(JSON` to parse JSON array leads to quadratic
running time in number of tests, see https://gitlab.kitware.com/cmake/cmake/-/work_items/27985
This leads to _terrible_ runtime for `catch_discover_tests` when called
on binaries with lot of tests (1k+). To get reasonable runtimes, we have
to avoid using `string(JSON` to parse out the individual test objects
from the array with all tests.
This commit replaces the sane approach of using real JSON parser with
a set of terrible hacks, where we use CMake's string APIs to split the
JSON array on what looks like object boundary (`}<ws>*,<ws>*{`), and then
checking whether the resulting thing can be parsed as JSON object. If not,
we append the next piece and check again. And again, and again, until we
get a proper JSON object.
This is all around a hilariously terrible idea, however:
1) It works in practice for all tested inputs.
2) It improves the time it takes to run `catch_discover_tests` on binary
with 1k tests from 4.2s to 1.1s and 2k tests from 16s to 3.9s.
Diffstat (limited to 'tests/TestScripts/DiscoverTests/register-tests.cpp')
| -rw-r--r-- | tests/TestScripts/DiscoverTests/register-tests.cpp | 21 |
1 files changed, 21 insertions, 0 deletions
diff --git a/tests/TestScripts/DiscoverTests/register-tests.cpp b/tests/TestScripts/DiscoverTests/register-tests.cpp index 5aec0f19..b833dba2 100644 --- a/tests/TestScripts/DiscoverTests/register-tests.cpp +++ b/tests/TestScripts/DiscoverTests/register-tests.cpp @@ -41,3 +41,24 @@ public: TEST_CASE_METHOD(TestCaseFixture, "A test case as method", "[tagstagstags]") {} TEST_CASE( "Newlines\nAnd\rOther\n\tWhitespace", "[whitespace-going-wild]" ) {} + +// Some JSON-like and JSON-adjacent characters and substrings in the test names/tags +// This serves to test that the parse-json-via-string-splitting hack in +// catch_discover_tests works properly. +TEST_CASE( "We split on variants of },{ in name" ) {} +TEST_CASE( "Then }\t, { we }\t,\t{ concatenate } , { them } ,{back},{" ) {} +TEST_CASE( "}},{{" ) {} +TEST_CASE( "Arrays [{},{}] wheee", "[also},{tags]" ) {} +TEST_CASE( "Let's add semicolon into the mix ;},{;},{};,{}" ) {} +TEST_CASE( "[", "[unmatched-square-bracket]" ) {} +TEST_CASE( "]", "[unmatched-square-bracket]" ) {} + +// Some CMake-like special strings ($ as dereference) strings in test names. +// This serves to test that the names of test cases are not evaluated +// inside the catch_discover_tests. +TEST_CASE( "Plain ${NOT_A_VAR} variable" ) {} +TEST_CASE( "Env variable access $ENV{HOME}" ) {} +TEST_CASE( "Cache check $CACHE{FOO}" ) {} +TEST_CASE( "Also some generator exprs $<CONFIG> in $<1:yes> name" ) {} +TEST_CASE( "Mess of bare $ $$ $$$ and unterminated $} ${ exprs" ) {} +TEST_CASE( "$ENV{X};[weird]{},{ mix $<0:no> ${VAR} };,{ }},{{" ) {} |