catch3/include/internal/catch_test_case_registry_impl.cpp, branch v2.x Progmasoft Catch3: an independently maintained C++17 test framework with a parallel runner, property checks, snapshots and structured results. Small cleanups for the reworked test case order hashing 2020-11-17T20:08:33+00:00 Martin Hořeňovský 2020-11-17T20:08:33+00:00 dc7e705672ba29a57dcff53ab1ad7be51e7b6e33 






Modified hash to make it more independent of the choice of test names.
2020-11-17T19:58:16+00:00

Sergio Losilla

2020-11-17T09:16:36+00:00

3ba745552bd26cbbb7c271720c0ec9eaaf1dc8dd












Workaround AppleClang bug by renaming TestHasher constructor argument
2020-10-02T21:22:49+00:00

Martin Hořeňovský

2020-10-02T21:22:49+00:00

7c75ecaef441e951a1b9240c5400e2947f190d48

As far as I understand the standard, if there is a function called
`rng` in the global namespace, and a function argument called `rng`,
then the argument should shadow the function. This then means that
uses of `rng` inside the function should refer to the argument.

This is not the case for AppleClang 12.0.0. Luckily the workaround
is simple enough; just rename the argument. Given that the function
is 3 lines and uncomplicated, the change of the name doesn't really
affect readability.

Still, WTF AppleClang?

Closes #2030





As far as I understand the standard, if there is a function called
`rng` in the global namespace, and a function argument called `rng`,
then the argument should shadow the function. This then means that
uses of `rng` inside the function should refer to the argument.

This is not the case for AppleClang 12.0.0. Luckily the workaround
is simple enough; just rename the argument. Given that the function
is 3 lines and uncomplicated, the change of the name doesn't really
affect readability.

Still, WTF AppleClang?

Closes #2030







Refactor implementation of hashed random order test sorting
2020-04-14T14:39:48+00:00

Martin Hořeňovský

2020-04-14T14:39:45+00:00

bad0fb51f836a8d155fb2034980721aa36917fa8












Change random test shuffling technique
2020-04-14T10:47:36+00:00

John Bytheway

2020-04-10T00:17:05+00:00

f696ab836b9c7d9ea2f80b64597541cdc67f6821

Previously a random test ordering was obtained by applying std::shuffle
to the tests in declaration order.  This has two problems:

- It depends on the declaration order, so the order in which the tests
  will be run will be platform-specific.
- When trying to debug accidental inter-test dependencies, it is helpful
  to be able to find a minimal subset of tests which exhibits the issue.
  However, any change to the set of tests being run will completely
  change the test ordering, making it difficult or impossible to reduce
  the set of tests being run in any reasonably efficient manner.

Therefore, change the randomization approach to resolve both these
issues.

Generate a random value based on the user-provided RNG seed.  Convert
every test case to an integer by hashing a combination of that value
with the test name.  Sort the test cases by this integer.

The test names and RNG are platform-independent, so this should be
consistent across platforms.  Also, removing one test does not change
the integer value associated with the remaining tests, so they remain in
the same order.

To hash, use the FNV-1a hash, except with the basis being our randomly
selected value rather than the fixed basis set in the algorithm.  Cannot
use std::hash, because it is important that the result be
platform-independent.





Previously a random test ordering was obtained by applying std::shuffle
to the tests in declaration order.  This has two problems:

- It depends on the declaration order, so the order in which the tests
  will be run will be platform-specific.
- When trying to debug accidental inter-test dependencies, it is helpful
  to be able to find a minimal subset of tests which exhibits the issue.
  However, any change to the set of tests being run will completely
  change the test ordering, making it difficult or impossible to reduce
  the set of tests being run in any reasonably efficient manner.

Therefore, change the randomization approach to resolve both these
issues.

Generate a random value based on the user-provided RNG seed.  Convert
every test case to an integer by hashing a combination of that value
with the test name.  Sort the test cases by this integer.

The test names and RNG are platform-independent, so this should be
consistent across platforms.  Also, removing one test does not change
the integer value associated with the remaining tests, so they remain in
the same order.

To hash, use the FNV-1a hash, except with the basis being our randomly
selected value rather than the fixed basis set in the algorithm.  Cannot
use std::hash, because it is important that the result be
platform-independent.







Introduce Catch's own RNG based on the PCG family of RNGs
2019-10-07T17:56:23+00:00

Martin Hořeňovský

2019-10-06T19:47:54+00:00

535da5c5132bcbba6f29c60ed6bb7ded29b9ec3e

In the future, we will also want to introduce our own
`uniform_int_distribution` and `uniform_real_distribution` to get
repeatable test runs across different platforms.





In the future, we will also want to introduce our own
`uniform_int_distribution` and `uniform_real_distribution` to get
repeatable test runs across different platforms.







Make StringRef's operator std::string explicit
2019-09-08T13:58:10+00:00

Martin Hořeňovský

2019-09-08T12:49:40+00:00

7b865daccc7bba2dcc25a4f0b9ea7348b1b4d61f

This way it is explicit when there is a `StringRef` -> `std::string`
conversion and makes it easier to look for allocations that could
be avoided.

Doing this has already removed one allocation per registered test
case, as there was a completely pointless `StringRef` -> `std::string`
conversion when parsing tags of a test case.





This way it is explicit when there is a `StringRef` -> `std::string`
conversion and makes it easier to look for allocations that could
be avoided.

Doing this has already removed one allocation per registered test
case, as there was a completely pointless `StringRef` -> `std::string`
conversion when parsing tags of a test case.







Improve reporting of unmatched filters (#1684)
2019-08-06T18:51:19+00:00

Steven Franzen

2019-08-06T18:51:19+00:00

6070745cab45a0f2284c56d5e2a1eed444447f88

This PR ultimately does 3 things
* Separately tracks matched tests per each filter part (that is, a set of filters separated by an OR (`,`)), which allows Catch2 to report each of the alternative filters that don't match any tests.
* Fixes `-w NoTests` to return non-zero in the process
* Adds tests for `-w NoTests`.





This PR ultimately does 3 things
* Separately tracks matched tests per each filter part (that is, a set of filters separated by an OR (`,`)), which allows Catch2 to report each of the alternative filters that don't match any tests.
* Fixes `-w NoTests` to return non-zero in the process
* Adds tests for `-w NoTests`.







Avoid adding a default test spec when none was provided
2019-04-09T09:50:59+00:00

Martin Hořeňovský

2019-04-09T09:50:59+00:00

e99f1efd2899441429da55ba717b8abd20c2b53b












Replaced use of std::rand with std::mt19937
2018-06-15T13:35:47+00:00

Phil Nash

2018-06-15T13:35:47+00:00

35a57b070fd4db56759dc3ecb8156f40841111a3

This uses a global instance of the RNG





This uses a global instance of the RNG