Each new file added to the testsuite may as well be adding 5 seconds onto the overall runtime. Logical groupings could be done to improve the overall CI speed. For example, all fail_compilation tests that check coverage of error messages in `dmd/dinterpret.d` could be put into a single module `fail_compilation/fail_dinterpret.d`
Not saying anything about the logical grouping, but this is a bit exaggerated imho: > Each new file added to the testsuite may as well be adding 5 seconds onto the overall runtime. _All_ 1283 fail_compilation tests compile in ~11s on my machine. The 774 compilable tests are slightly slower with ~25s ... and you don't want to know how long runnable takes :/ (of course the numbers are taken with full parallelism on an i7 4th Gen) Anyhow, if someone wants to make the testsuite faster, the runnable tests are the real bad boys and should be focussed on first.
Grouping test files doesn't achieve much in regard to the general test execution time. The real time sink are the permutations for DMD's optimizer/inliner (compare `./run.d` with `./run.d quick`)
Hi! I'm looking to contribute to D and have experience as an SDET so this issue looks perfect. However, from the comments and age of the issue, it's not clear if this is still required. Is it a genuine open issue?
(In reply to Eddy Schauman-Haigh from comment #3) > However, from the comments and age of the issue, it's > not clear if this is still required. Is it a genuine open issue? Yes, it's still an open issue, the test suite has only grown since. Contributions are appreciated! You can find the test suite here: https://github.com/dlang/dmd/tree/master/test
(In reply to Dennis from comment #4) > (In reply to Eddy Schauman-Haigh from comment #3) > > However, from the comments and age of the issue, it's > > not clear if this is still required. Is it a genuine open issue? > > Yes, it's still an open issue, the test suite has only grown since. > Contributions are appreciated! You can find the test suite here: > > https://github.com/dlang/dmd/tree/master/test Great! In which case I'll start having a look at this issue. Thanks for the swift reply Dennis!
Just be sure to benchmark, benchmark, and benchmark some more! Things that have changed since the original raising of this include: - Running the most expensive tests first had reduced the time it took to complete by about a minute when using parallelism. - Fail compilation tests no longer run with all permutations. - Neither do compilable tests.
THIS ISSUE HAS BEEN MOVED TO GITHUB https://github.com/dlang/dmd/issues/17832 DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB