Issue 19587 - std.range.generate's range calls its argument one time too many
Summary: std.range.generate's range calls its argument one time too many
Status: NEW
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D2
Hardware: All All
: P4 enhancement
Assignee: No Owner
URL:
Keywords: pull
: 23094 (view as issue list)
Depends on:
Blocks:
 
Reported: 2019-01-15 21:59 UTC by Paul Backus
Modified: 2024-12-01 16:34 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Paul Backus 2019-01-15 21:59:11 UTC
Example:

---
int count = 0;

int counter()
{
    return count++;
}

void main()
{
    import std.algorithm;
    import std.range;
    import std.stdio;
    
    auto gen = generate!counter;
    gen.take(3).each!writeln;
    assert(count == 3); // fails, count == 4
}
---

This behavior leads to two related problems:

1) Ranges, in general, are expected to be lazy. Eagerly calling the generator function on construction and in popFront violates that expectation.

2) As a result, writing optimal code using ranges created with `generate` (that is, code which does no unnecessary work) requires special-case handling, since Phobos' algorithms (for example, `take`, above) "naively" assume that construction and popFront are cheap.

These problems are especially bothersome when the generator function is expensive to call (for example, if it accesses the network).
Comment 1 Dennis 2021-05-06 22:53:27 UTC
Just got hit by this today, my `generate!(() => readln()).take(10).array` discarded the line after the 10 lines put into the array.

This is really annoying, but changing this would be a breaking change, so I'm afraid the fix would be either:

- a template parameter 'caching' with default value of `true`
- a new synonym symbol, similar to `approxEqual => isClose`, with the possibility to deprecate the old symbol (caching can still be achieved by doing `generate!(f).cache`). I'm not sure what that synonym would be, things that come to mind are `generateLazy`, `successiveCalls`, `repeatEvaluate`, but I'm not a big fan of them.
Comment 2 Remi Thebault 2022-05-06 16:50:40 UTC
*** Issue 23094 has been marked as a duplicate of this issue. ***
Comment 3 Dlang Bot 2022-05-06 17:37:06 UTC
@rtbo created dlang/phobos pull request #8453 "fix issue 19587" fixing this issue:

- fix issue 19587
  
  avoid that std.range.generate calls
  fun one time too many

https://github.com/dlang/phobos/pull/8453
Comment 4 dlangBugzillaToGithub 2024-12-01 16:34:47 UTC
THIS ISSUE HAS BEEN MOVED TO GITHUB

https://github.com/dlang/phobos/issues/10366

DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB