D issues are now tracked on GitHub. This Bugzilla instance remains as a read-only archive.
Issue 14469 - file.readText on Win64 doesn't work for files > 4GB.
Summary: file.readText on Win64 doesn't work for files > 4GB.
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D2
Hardware: x86_64 Windows
: P1 normal
Assignee: No Owner
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-04-20 06:40 UTC by Kenny Alive
Modified: 2017-07-19 17:43 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Kenny Alive 2015-04-20 06:40:11 UTC
In DMD 2.067 this function calls read(in char[], size_t) which in turns uses GetFileSize like this:

auto size = trustedGetFileSize(h, null);

One solution is to pass not null second parameter but it seems that it's even better to use GetFileSizeEx (us/library/windows/desktop/aa364957(v=vs.85).aspx).
Comment 1 Kenny Alive 2015-04-21 19:49:12 UTC
Additionally since ReadFile (and ReadFileEx too) can read file in chunks not larger than 4GB the file reading code should be updated to something like this:

            ulong totalNumRead = 0;
            while (totalNumRead != size)
            {
                uint chunkSize = (size - totalNumRead) & 0xffffffff;
                DWORD numRead = void;
                cenforce(ReadFile(hFile, cast(ubyte*)lpBuffer + totalNumRead, chunkSize, &numRead, null) != 0
                        && numRead == chunkSize, name);
                totalNumRead += chunkSize;
            }
Comment 2 Kenny Alive 2015-04-21 21:51:37 UTC
The above algorithm for reading file has a bug. chunkSize should be calculated like this:

uint chunkSize = min(size - totalNumRead, 0xffffffff);
Comment 3 github-bugzilla 2015-05-25 07:48:53 UTC
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/95fd043dc6d0c945c171f8874761e1399a11252c
Fix Issue 14469 - read > 4GB file on Windows x64.

https://github.com/D-Programming-Language/phobos/commit/75f068773ccc0750c2cf865fe2f621a9947723f3
Merge pull request #3218 from artemalive/issue_14469

Fix Issue 14469 - read > 4GB file on Windows x64.
Comment 4 github-bugzilla 2017-07-19 17:43:27 UTC
Commits pushed to dmd-cxx at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/95fd043dc6d0c945c171f8874761e1399a11252c
Fix Issue 14469 - read > 4GB file on Windows x64.

https://github.com/dlang/phobos/commit/75f068773ccc0750c2cf865fe2f621a9947723f3
Merge pull request #3218 from artemalive/issue_14469