Issue 23361 - std.uni.normalize should be pure
Summary: std.uni.normalize should be pure
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D2
Hardware: All All
: P2 major
Assignee: No Owner
URL:
Keywords: pull
Depends on:
Blocks:
 
Reported: 2022-09-23 10:23 UTC by Ate Eskola
Modified: 2023-06-18 12:45 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Ate Eskola 2022-09-23 10:23:40 UTC
-----------
@safe pure auto fun()
{ import std.uni;
  return "a\u0308".normalize;
}
-----------

Errors, with both nightly and stable DMD:
onlineapp.d(3): Error: `pure` function `onlineapp.fun` cannot call impure function `std.uni.normalize!(NormalizationForm.NFC, char).normalize`

There is nothing in `normalize` that should alter global state, so this should compile.

I'm classifying this as major, because this is the only way to normalize Unicode strings in D without a DIY or third-party library. The function not being available in pure code is a serious hinderance.
Comment 1 Imperatorn 2022-10-30 22:50:27 UTC
I took a look at this.

If you mark the following as pure:
normalize
decompose
decomposeHangul

Then there's only this that needs to be pure (in normalize)

() @trusted {
    decomposed.assumeSafeAppend();
    ccc.length = 0;
    ccc.assumeSafeAppend();
} ();

To cheat and not make the appenders pure but the block pure we can just add pure:

() @trusted pure {
    decomposed.assumeSafeAppend();
    ccc.length = 0;
    ccc.assumeSafeAppend();
} ();

But, we still need some way there to allow calling the impure append from the pure block.

Maybe we could cast some function pointer to pure, I don't know.

I tested wrapping it in a debug block and that makes it work, but I'm not sure what the "real" solution would look like.

() @trusted pure {
    debug {
        decomposed.assumeSafeAppend();
        ccc.length = 0;
        ccc.assumeSafeAppend();
    }
} ();
Comment 2 Imperatorn 2022-10-30 23:11:46 UTC
One way I guess is to cast the function to pure.
The other is to actually mark _d_arrayshrinkfit as pure.

Which of those seems most reasonable?
Comment 3 Dlang Bot 2023-06-14 17:27:22 UTC
@dukc created dlang/phobos pull request #8763 "fix issue 23361 - std.uni.normalize made pure" fixing this issue:

- fix issue 23361 - std.uni.normalize made pure

https://github.com/dlang/phobos/pull/8763
Comment 4 Ate Eskola 2023-06-14 17:35:35 UTC
Oh, it seems I have missed your comment - I just figured all this out myself creating a fix for this when I could just have read your message.

I solved the assumeSafeAppend thing by marking the calls pure.

(But with a cast - does simply marking the lambda pure really work? I thought even `@trusted` code lets to treat impure calls as pure only with an explicit cast.)

I think it's okay because... well let me copy my comment from the code as explaination:

---
// assumeSafeAppend isn't considered pure as of writing, hence the
// cast. It isn't pure in the sense that the elements after
// the array in question are affected, but we don't use those
// making the call pure for our purposes.
---
Comment 5 Dlang Bot 2023-06-18 12:45:18 UTC
dlang/phobos pull request #8763 "fix issue 23361 - std.uni.normalize made pure" was merged into master:

- e6ed40182187ea4f3caf1127102136f9aa5fd0ac by Ate Eskola:
  fix issue 23361 - std.uni.normalize made pure

https://github.com/dlang/phobos/pull/8763