D issues are now tracked on GitHub. This Bugzilla instance remains as a read-only archive.
Issue 4487 - 16 bytes long structs requires 32 bytes if allocated singularly on the heap
Summary: 16 bytes long structs requires 32 bytes if allocated singularly on the heap
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: dmd (show other issues)
Version: D2
Hardware: x86 Windows
: P2 enhancement
Assignee: No Owner
URL:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2010-07-19 12:46 UTC by bearophile_hugs
Modified: 2012-12-20 16:48 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description bearophile_hugs 2010-07-19 12:46:34 UTC
(This is my first bug report with 'major' severity, because this is a quite important bug.)

This comes after a report by Steven Schveighoffer. This program allocates a linked list of 10 million structs on the heap (this number is set to high just to improve the measurements).

The presence of GC.disable() doesn't change the total memory allocated, but decreases a lot the run time. On a 32 bit Windows at the end of the list allocation this program has allocated about 326 MB, it means:

326_200_000 bytes / 10_000_000 ~= 32.62 bytes each Foo

This can't be accepted in a serious "system language" (also because 16 bytes long structs are quite common in my 32 bit code).

import core.memory: GC;
struct Foo {
    Foo* next;
    ubyte[12] arr;
    this(Foo* ptr) { this.next = ptr; }
}
static assert(Foo.sizeof == 16);
void main() {
    GC.disable();
    enum n = 10_000_000;
    Foo* lh;
    foreach (i; 0 .. n)
        lh = new Foo(lh);
    GC.enable();
}


Maybe this bug can be fixed introducing a specific allocator function for single structs, that don't sees them as arrays of length 1 (that needs 1 byte of information padding for appends).
Comment 1 Steven Schveighoffer 2010-07-19 13:10:48 UTC
DMD is the main culprit here, not druntime.  And this is not a bug, it's an enhancement.  DMD functions exactly as designed.

DMD is the one generating the code to call the arrayNew function with length 1.  Druntime cannot tell between someone actually allocating an array of 1 or someone allocating a single struct.

With the "Appendable" bit I just added for druntime, this could be alleviated if the compiler would call a separate function for struct allocators.

As a workaround, you can pre-allocate a large block of nodes, which will only have one byte of pad per block allocated.
Comment 2 Andrej Mitrovic 2012-12-20 14:14:57 UTC
I can only see around 160MB used now, bear please verify and close if it's fixed, thanks.
Comment 3 bearophile_hugs 2012-12-20 16:48:22 UTC
(In reply to comment #2)
> I can only see around 160MB used now, bear please verify and close if it's
> fixed, thanks.

It was fixed time ago. Closed.