Issue 24584 - [phobos] `make unittest` should not rerun tests unnecessarily
Summary: [phobos] `make unittest` should not rerun tests unnecessarily
Status: NEW
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D2
Hardware: x86_64 Linux
: P1 enhancement
Assignee: No Owner
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-03 16:09 UTC by Nick Treleaven
Modified: 2024-12-01 16:42 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Nick Treleaven 2024-06-03 16:09:58 UTC
Currently it seems to run every test, regardless of whether relevant Phobos dependencies have changed. This makes it a very slow process to find & fix all errors when introducing a dmd error.

Alternatively, having some way of running all tests despite any failing tests would be useful.
Comment 1 Nick Treleaven 2024-06-12 15:55:12 UTC
> running all tests despite any failing tests would be useful

Looks like Gnu make has a -k option for that:
https://www.gnu.org/software/make/manual/html_node/Options-Summary.html#index-_002d_002dkeep_002dgoing-2

Not tested. But regardless it would be useful if there was a way to run an individual module's unittests rather than have to rerun the ones that already succeeded.
Comment 2 kinke 2024-06-14 08:58:22 UTC
(In reply to Nick Treleaven from comment #1)
> But regardless it would be useful if there was a way to run an
> individual module's unittests rather than have to rerun the ones that
> already succeeded.

There are multiple existing options for running the unittests of a single module or package, all based on https://github.com/dlang/phobos/blob/ad9f87d1b816783bd8c13f461490e68f7fcae20a/Makefile#L431-L448:

```
$ make -j$(nproc) std/algorithm/mutation.test # single module
$ make -j$(nproc) std/algorithm.test # whole package
$ make -j$(nproc) unittest/std/algorithm/mutation.run # single module
```

That said, *running* (not building) all Phobos unittests on my laptop takes about 6 (release) / 7.5 (debug) seconds, using current master.
Comment 3 Nick Treleaven 2024-06-14 12:09:23 UTC
> There are multiple existing options for running the unittests of a single module or package

Oh great, thanks. I completely missed that but I see now that .test is mentioned at the top of the Makefile.

> *running* (not building) all Phobos unittests on my laptop takes about 6 (release) / 7.5 (debug) seconds

I have an old PC (I don't use -j as I only have 2 cores and 1 I keep for other things):

time make unittest
...
make[1]: Leaving directory '/home/nick/git/phobos'

real	16m57.602s
user	15m21.911s
sys	1m0.822s
Comment 4 Nick Treleaven 2024-06-14 12:30:37 UTC
(In reply to kinke from comment #2)
> ```
> $ make -j$(nproc) std/algorithm/mutation.test # single module
> $ make -j$(nproc) std/algorithm.test # whole package
> $ make -j$(nproc) unittest/std/algorithm/mutation.run # single module
> ```

Pull to update Makefile comment:
https://github.com/dlang/phobos/pull/9014
Comment 5 kinke 2024-06-14 13:27:58 UTC
(In reply to Nick Treleaven from comment #3)
> I have an old PC (I don't use -j as I only have 2 cores and 1 I keep for
> other things):
> 
> time make unittest
> ...
> make[1]: Leaving directory '/home/nick/git/phobos'
> 
> real	16m57.602s
> user	15m21.911s
> sys	1m0.822s

Oh wow. Then the issue most likely isn't about *running* a subset of unittests only, but improving incremental builds of the unittest runners. Currently, changing a single Phobos source module leads to a complete rebuild (recompiling all Phobos unittest object files): https://github.com/dlang/phobos/blob/ad9f87d1b816783bd8c13f461490e68f7fcae20a/Makefile#L400-L401

The Makefiles don't exploit the compiler `-makedeps` switch yet. That would allow only recompiling object files which are dirty (directly or indirectly importing a changed Phobos module).
Comment 6 dlangBugzillaToGithub 2024-12-01 16:42:42 UTC
THIS ISSUE HAS BEEN MOVED TO GITHUB

https://github.com/dlang/phobos/issues/10555

DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB