D issues are now tracked on GitHub. This Bugzilla instance remains as a read-only archive.
Issue 8156 - Very slow compilation with string-imported file ~100 MiB
Summary: Very slow compilation with string-imported file ~100 MiB
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: dmd (show other issues)
Version: D2
Hardware: All All
: P2 critical
Assignee: No Owner
URL:
Keywords: performance, pull
Depends on:
Blocks:
 
Reported: 2012-05-27 22:43 UTC by Denis Shelomovskii
Modified: 2020-11-05 04:15 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Denis Shelomovskii 2012-05-27 22:43:05 UTC
dmd takes 14 s with full load of one CPU core to include 40 MiB file
as a string import on my PC. During this dmd process slowly increases its memory using by 2-3 MiB/s.
Comment 1 Martin Nowak 2012-05-29 06:35:25 UTC
I run into longer compilation times at about 300MB but at that
point it's mainly due to paging in consecutive runs.

Are you processing/casting the string?
What OS are you on?
Are you seeing O(N^2) behavior?
Comment 2 Denis Shelomovskii 2012-05-29 08:26:35 UTC
(In reply to comment #1)
> I run into longer compilation times at about 300MB but at that
> point it's mainly due to paging in consecutive runs.
> 
> Are you processing/casting the string?
No CT processing, casting doesn't matter.

> What OS are you on?
Windows XP SP3

> Are you seeing O(N^2) behavior?
Yes. My timings in CSV format:
---
Size, MiB;Time, s
1;0
13.4;2
28.3;7
40.5;17
96.7;111
215;582
---

y = a * x^b approximation:
time(size) = 6.79E-00003 x^2.117 with R^2 = 0.9999
Comment 3 Boris Carvajal 2020-08-12 04:50:55 UTC
I've pushed two patches to resolve this old bug.

First part is here : https://github.com/dlang/dmd/pull/11554
I'll link the second to this issue.
Comment 4 Dlang Bot 2020-08-12 05:21:02 UTC
@BorisCarvajal created dlang/dmd pull request #11555 "Fix Issue 8156 - Optimize string comdats name generation (Windows)" fixing this issue:

- Fix Issue 8156 - Optimize string comdats name generation (Windows)

https://github.com/dlang/dmd/pull/11555
Comment 5 Dlang Bot 2020-11-05 04:15:39 UTC
dlang/dmd pull request #11555 "Fix Issue 8156 - Optimize string comdats name generation (Windows)" was merged into master:

- 3af440b76d36f2d6db561b184807f5132eeb929d by Boris Carvajal:
  Fix Issue 8156 - Optimize string comdats name generation (Windows)

https://github.com/dlang/dmd/pull/11555