Issue 19946 - In betterC filling an array with a non-zero value fails for types of size > 1 due to missing _memset16/_memset32/etc.
Summary: In betterC filling an array with a non-zero value fails for types of size > 1...
Status: NEW
Alias: None
Product: D
Classification: Unclassified
Component: dmd (show other issues)
Version: D2
Hardware: All All
: P1 blocker
Assignee: No Owner
URL:
Keywords: backend, betterC, pull
: 17778 20689 21750 21879 (view as issue list)
Depends on: 21750
Blocks:
  Show dependency treegraph
 
Reported: 2019-06-07 08:59 UTC by Nathan S.
Modified: 2023-02-24 11:54 UTC (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Nathan S. 2019-06-07 08:59:48 UTC
Example of code that doesn't work:

```d
extern(C) void main()
{
    alias int3 = int[3];
    union U { ulong a = 1; uint[3] b; }

    // Can't set entire array slice to non-zero value unless type is size 1.
    { ushort[] a; a[] = 1; } //error: undefined reference to '_memset16'
    { int[] a; a[] = 1; } //error: undefined reference to '_memset32'
    { ulong[] a; a[] = 1; } //error: undefined reference to '_memset64'
    { float[] a; a[] = 1; } //error: undefined reference to '_memsetFloat'
    { double[] a; a[] = 1; } //error: undefined reference to '_memsetDouble'
    { cdouble[] a; a[] = cdouble.max; } //error: undefined reference to '_memset128'
    { creal[] a; a[] = creal.max; } //error: undefined reference to '_memset160'
    { int3[] a; int3 val = [1, 2, 3]; a[] = val; } //error: undefined reference to '_memsetn'
    { U[] a; a[] = U.init; } //error: undefined reference to '_memset128ii'
    
    // Can't give all members of static array non-zero initial value unless type is size 1.
    { ushort[4] a = 1; } //error: undefined reference to '_memset16'
    { int[4] a = 1; } //error: undefined reference to '_memset32'
    { ulong[4] a = 1; } //error: undefined reference to '_memset64'
    { float[4] a = 1; } //error: undefined reference to '_memsetFloat'
    { double[4] a = 1; } //error: undefined reference to '_memsetDouble'
    { cdouble[4] a = cdouble.max; } //error: undefined reference to '_memset128'
    { creal[4] a = creal.max; } //error: undefined reference to '_memset160'
    { U[4] a = U.init; } //error: undefined reference to '_memset128ii'
    
    // Static arrays of types with non-zero init also fail if not explicitly initialized.
    { wchar[4] a; } //error: undefined reference to '_memset16'
    { float[4] a; } //error: undefined reference to '_memsetFloat'
    { double[4] a; } //error: undefined reference to '_memsetDouble'
    { real[4] a; } //error: undefined reference to '_memset80'
    { cdouble[4] a; } //error: undefined reference to '_memset128'
    { creal[4] a; } //error: undefined reference to '_memset160'
    { U[4] a; } //error: undefined reference to '_memset128ii'

    // Static arrays of scalars wider than 8 bytes can't even be explicitly initialized as zero.
    { real[4] a = 0; } //error: undefined reference to '_memset80'
    { cdouble[4] a = 0+0i; } //error: undefined reference to '_memset128'
    { creal[4] a = 0+0i; } //error: undefined reference to '_memset160'
    { U[4] a = U(0); } //error: undefined reference to '_memset128ii'
    
    // Sometimes there is trouble even when individually specifying each element of the array.
    { int[4] a = [1, 1, 1, 1]; } //error: undefined reference to '_memset32'
    { int[4] a = [1, 2, 3, 4]; } //Works because compiler doesn't try to use memset-alike.
    
    // These work.
    { char[10] a; } //Works because ubyte.sizeof == 1 so it can use memset.
    { wchar[2] a = [0x1234, 0x5678]; a[] = 0; } //Works because the compiler realizes that it can use memset to zero the bits of a type regardless of its width.    
}
```
Comment 1 Paul Backus 2022-02-10 13:41:33 UTC
*** Issue 21879 has been marked as a duplicate of this issue. ***
Comment 2 Paul Backus 2022-02-10 13:42:20 UTC
*** Issue 21750 has been marked as a duplicate of this issue. ***
Comment 3 Dennis 2022-06-02 18:04:37 UTC
These functions are defined in druntime's rt/memset, which is not linked with betterC. I'm not sure what the best fix is. The functions could be emitted in every translation unit that relies on them. Or, the code needs to be always inlined (though since some memset functions use inline asm, dmd cannot inline it currently).
Comment 4 mhh 2022-06-02 19:01:13 UTC
The fix is easy. I wrote the code but never got around to pushing it.

It just needs to be a template in druntime. Hypothetically you could make it inline a loop but this might hurt LDC and GDC.

arrayOp actually supports memset but making the array lowering use arrayOp would make it end up using itself.

I also looked into using an arbitrary set memset in the backend but yuck.
Comment 5 RazvanN 2022-06-03 06:57:08 UTC
(In reply to mhh from comment #4)
> The fix is easy. I wrote the code but never got around to pushing it.
> 
> It just needs to be a template in druntime. Hypothetically you could make it
> inline a loop but this might hurt LDC and GDC.

Besides making it a template, you also need to move the call insertion from e2ir to the frontend so that semantic analysis is performed on the instantiation.

> 
> arrayOp actually supports memset but making the array lowering use arrayOp
> would make it end up using itself.
> 
> I also looked into using an arbitrary set memset in the backend but yuck.

I think that long-term the best solution would be to lower the code in the frontend and, indeed, lower the hooks to templates.
Comment 6 mhh 2022-06-03 11:58:18 UTC
I am aware.

I actually think we should have another dmd-only semantic pass just for this in dmd because the existing code is a complete mess.

The case for array assign operations for example is just a branch in a hundreds of line visitor member.
Comment 7 Walter Bright 2022-06-04 06:10:24 UTC
Razvan, your idea is the right approach. mhh, you're right, but this refactoring should be in a separate PR.
Comment 8 Dennis 2022-06-08 10:24:27 UTC
*** Issue 20689 has been marked as a duplicate of this issue. ***
Comment 9 Dennis 2022-06-08 10:25:47 UTC
*** Issue 17778 has been marked as a duplicate of this issue. ***
Comment 10 Dennis 2022-06-08 10:30:14 UTC
(In reply to RazvanN from comment #5)
> I think that long-term the best solution would be to lower the code in the
> frontend and, indeed, lower the hooks to templates.

I agree. My only reservation is that LDC and GDC seem to handle setting arrays in -betterC fine, but with a frontend lowering they might have a performance regression unless they start recognizing the new template hooks.
Comment 11 Dlang Bot 2022-06-08 17:02:03 UTC
@dkorpel created dlang/druntime pull request #3837 "Issue 19946 - Add memset template function" mentioning this issue:

- Issue 19946 - Add memset template function

https://github.com/dlang/druntime/pull/3837
Comment 12 Richard (Rikki) Andrew Cattermole 2022-11-16 03:54:26 UTC
Another example that can trigger this (in -betterC):

```d
extern(C) void main() {
    wchar[2] got;
    func(got);
}

void func(out wchar[2] v) {
}
```

Why do these hooks even exist? They could be a simple rewrite to a foreach loop in the compiler and not depend on the runtime at all, let alone functions in rt.
Comment 13 Iain Buclaw 2022-12-27 17:01:49 UTC
This works with GDC and LDC.  The problem is in the dmd backend.
Comment 14 Walter Bright 2022-12-30 07:42:01 UTC
Start towards fixing this:

 https://issues.dlang.org/show_bug.cgi?id=19946
Comment 15 Richard (Rikki) Andrew Cattermole 2022-12-30 09:02:54 UTC
(In reply to Walter Bright from comment #14)
> Start towards fixing this:

https://github.com/dlang/dmd/pull/14761
Comment 16 Dlang Bot 2022-12-30 19:32:36 UTC
@WalterBright created dlang/dmd pull request #14762 "fix Issue 19946 - In betterC filling an array with a non-zero value f…" fixing this issue:

- fix Issue 19946 - In betterC filling an array with a non-zero value fails for types of size > 1

https://github.com/dlang/dmd/pull/14762
Comment 17 Walter Bright 2022-12-30 19:38:10 UTC
This is not a regression, as it never worked.
Comment 18 Dlang Bot 2023-01-02 09:07:23 UTC
@WalterBright created dlang/dmd pull request #14770 "handle byte, short, and int cases for bug 19946" mentioning this issue:

- handle byte, short, and int cases for bug 19946

https://github.com/dlang/dmd/pull/14770
Comment 19 Dlang Bot 2023-01-03 10:44:17 UTC
dlang/dmd pull request #14770 "handle byte, short, and int cases for bug 19946" was merged into master:

- afb2a9bbc90edaf7063e073950adb1076ae56c7c by Walter Bright:
  handle byte, short, and int cases for bug 19946

https://github.com/dlang/dmd/pull/14770
Comment 20 ryuukk_ 2023-01-03 13:38:22 UTC
Finally, thanks!!
Comment 21 Walter Bright 2023-01-05 01:38:13 UTC
More progress: https://github.com/dlang/dmd/pull/14788
Comment 22 Dlang Bot 2023-01-11 20:42:03 UTC
@WalterBright updated dlang/dmd pull request #14790 "fix Issue 23218 - cgxmm.d:1373 assert fail" mentioning this issue:

- handle byte, short, and int cases for bug 19946 (#14770)

https://github.com/dlang/dmd/pull/14790
Comment 23 Dlang Bot 2023-01-14 03:45:21 UTC
@WalterBright updated dlang/dmd pull request #14805 "fix Issue 23614 - ImportC: __int128 not supported" mentioning this issue:

- handle byte, short, and int cases for bug 19946 (#14770)

https://github.com/dlang/dmd/pull/14805
Comment 24 Dlang Bot 2023-02-24 11:07:22 UTC
@RazvanN7 updated dlang/dmd pull request #14884 "Fix Issue 23709 - Cannot use synchronized on shared class with -preview=nosharedaccess" mentioning this issue:

- handle byte, short, and int cases for bug 19946 (#14770)

https://github.com/dlang/dmd/pull/14884
Comment 25 Dlang Bot 2023-02-24 11:54:34 UTC
@RazvanN7 updated dlang/dmd pull request #14909 "Fix Issue 23732 - Cannot create shared instance of class with -preview=nosharedaccess" mentioning this issue:

- handle byte, short, and int cases for bug 19946 (#14770)

https://github.com/dlang/dmd/pull/14909