D issues are now tracked on GitHub. This Bugzilla instance remains as a read-only archive.
Issue 8091 - Optimizer generates wrong code when reducing comparisons.
Summary: Optimizer generates wrong code when reducing comparisons.
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: dmd (show other issues)
Version: D2
Hardware: All All
: P2 critical
Assignee: No Owner
URL:
Keywords: wrong-code
Depends on:
Blocks:
 
Reported: 2012-05-12 13:24 UTC by Kasumi Hanazuki
Modified: 2012-05-17 20:01 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Kasumi Hanazuki 2012-05-12 13:24:17 UTC
DMD optimizer generates a wrong code for the program below.
The second assertion unexpectedly fails if you specify -O flag to DMD,
while it passes as expected without optimization.

 $ dmd -m32 -run test.d      # fine
 $ dmd -m32 -O -run test.d   # AssertError@test(8): Assertion failure

Tested against DMDv2.060 (git HEAD) on 32-bit Linux.
Found by hos_lyric <https://twitter.com/#!/hos_lyric_/status/201270597265793024>.

----

int solve(int n) {
    int a = (n % 3 == 0) ? 1 : (n % 3 == 1) ? 1 : 0;
    return (a != 0) ? a : 0;
}

void main() {
    assert(solve(0) ==  1);
    assert(solve(1) ==  1); // line 8, fails with -O
    assert(solve(2) ==  0);
}
Comment 1 Kasumi Hanazuki 2012-05-12 23:58:16 UTC
more simplification:

----

int solve(int n) {
    int a = (n == 0) ? 1 : (n == 1) ? 1 : 0;
    return (a != 0) ? a : 0;
}

void main() {
    assert(solve(0) ==  1);
    assert(solve(1) ==  1); // fails with -O
    assert(solve(2) ==  0);
}
Comment 2 Don 2012-05-16 12:17:27 UTC
Marginally reduced (this is what the early optimizer passes turn this into):

int solve(int n) {
    int a = n ? (n>=1u) : 1;
    return a ? a : 0;
}

void main() {
    assert(solve(1) ==  1);
}

It looks as after common subexpression elimination of a, the branch address from the >= is out by 1. The generated code is:

		push	EAX
		cmp	dword ptr -4[EBP],0
		je	L1C
		mov	ECX,1
		cmp	dword ptr -4[EBP],1
		jae	L1F
		mov	ECX,0
		jmp short	L1F <--- wrong address
L1C:		xor	ECX,ECX
		inc	ECX
L1F:		je	L25        <---- shouldn't jump here
		mov	EAX,ECX    <---- should jump here instead
		jmp short	L27
L25:		xor	EAX,EAX
L27:		mov	ESP,EBP
Comment 3 github-bugzilla 2012-05-17 19:59:04 UTC
Commit pushed to dmd-1.x at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/8a689e26e1ba6565d76b05d045674967e83aef60
fix Issue 8091 - Optimizer generates wrong code when reducing comparisons
Comment 4 github-bugzilla 2012-05-17 20:01:21 UTC
Commit pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/40f81b41931eea6768b38b44802e35cd4f32f419
fix Issue 8091 - Optimizer generates wrong code when reducing comparisons