Issue 13996 - Function for returning a temporary file with a randomly generated name where the name can be accessed
Summary: Function for returning a temporary file with a randomly generated name where ...
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D2
Hardware: All All
: P1 enhancement
Assignee: No Owner
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-01-17 22:57 UTC by Jonathan M Davis
Modified: 2017-07-19 17:42 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Jonathan M Davis 2015-01-17 22:57:08 UTC
We need a function which creates a file with a temporary name where we have access to the name and can close and reopen the file. We have std.stdio.tmpfile, which calls the C tmpfile function and gives us a temporary file, but we have no access to the name of the file, and it gets deleted when its closed, which is completely useless for many situations, especially in unit tests where you need to do something like write to a file, close it, and then read from it.

POSIX's mkstemp does something similar to this, but on some systems, it doesn't allow for very many file names (e.g. the windows implementation is restricted to 26; some POSIX implementations have similar limitations), and it doesn't provide any control over where the file is written to.

So, I propose that we create something like

    static File tempFile(string prefix = "", string dir = tempDir()) {...}

in std.stdio.File which allows for adding a prefix like mkstemp does but also allows control over where the file goes and does not have the same limitations on the number of randomly generated files as mkstemp sometimes does. And, of course, it's a normal File which won't delete itself when being closed, so it will be able to be used like any other File, unlike the one returned by std.stdio.File.tmpfile.

And with that added, I'd suggest that we deprecate tmpfile, since IMHO it's pretty useless, and it'll reduce confusion if there's only one function returns a temporary file with a randomly generated name, but that's obviously up for debate.
Comment 1 Martin Nowak 2015-02-04 00:06:10 UTC
You forgot suffix argument :).

It's missing for ages, we just needed it for dub https://github.com/D-Programming-Language/dub/pull/497#issuecomment-72763326.

The python implementation takes a reasonable approach, generate random names and try to exclusively open the file.

https://github.com/python/cpython/blob/1196330dedbe741f8a0c418abcd2956fad29837f/Lib/tempfile.py#L191

The new function could be called namedTempFile. It's still useful to have a tempFile function that directly unlink the file.
Comment 2 Jonathan M Davis 2015-02-04 00:21:05 UTC
(In reply to Martin Nowak from comment #1)
> You forgot suffix argument :).

What suffix? As suggested, it takes an optional prefix and randomly generates the rest. You mean that it should take an extension? Or do you think that it should take a suffix rather than a prefix? I suppose that if it takes a suffix rather than a prefix, the extension could just be part of the suffix.

> It's missing for ages, we just needed it for dub
> https://github.com/D-Programming-Language/dub/pull/497#issuecomment-72763326.
> 
> The python implementation takes a reasonable approach, generate random names
> and try to exclusively open the file.
> 
> https://github.com/python/cpython/blob/
> 1196330dedbe741f8a0c418abcd2956fad29837f/Lib/tempfile.py#L191
> 
> The new function could be called namedTempFile. It's still useful to have a
> tempFile function that directly unlink the file.

I have a working implementation for Linux. The problem is that I can't figure out how to get _wsopen or any of its relatives to work on Windows. The linker fails to find them, even when I verify that they're in snn.lib (or whatever the name of the library is with the stdio stuff in it that comes with dmc). So, I'm not quite sure what to do. I could easily create a PR, but it's kind of pointless if it doesn't work on Windows.

As for keeping tmpfile, I suppose that we could - to avoid forcing anyone to change their code if nothing else - but in my experience, the function is utterly useless. And replacing it with tempFile would be trivial. instead of

auto file
Comment 3 Jonathan M Davis 2015-02-04 00:23:31 UTC
Drat. I managed to post by accident. In any case, tmpfile could easily be replaced such that

auto file = File.tmpfile();

became

auto file = File.tempFile();
scope(exit) std.file.remove(file.name);

so IMHO tmpfile becomes redundant on top of being pretty useless. But it _does_ mean deprecating it if we're getting rid of it, so it might be better to just keep it around rather than dealing with that.
Comment 4 Martin Nowak 2015-02-04 02:17:43 UTC
(In reply to Jonathan M Davis from comment #3)
> auto file = File.tempFile();
> scope(exit) std.file.remove(file.name);

There is a security aspect, because the current tempfile is unlinked at the moment it's created. So no other process can access it.
But as that's the advanced use-case it deserves the longer name, anonTempFile or so.
Comment 5 Martin Nowak 2015-02-04 02:21:54 UTC
> What suffix? As suggested, it takes an optional prefix and randomly generates the rest. You mean that it should take an extension?

Yes, prefix and suffix, random in between.
Comment 6 Martin Nowak 2015-02-04 02:22:05 UTC
A mkdtemp for directories is missing as well.
Comment 7 Jonathan M Davis 2015-02-04 09:42:36 UTC
Okay. I now have a pull request for this, and I added a suffix parameter to what I had so that now both a prefix and suffix can be set:

https://github.com/D-Programming-Language/phobos/pull/2956
Comment 8 github-bugzilla 2015-02-04 10:35:26 UTC
Commits pushed to master at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/0f389cb0aa5a8f5fd1a36dbdd338f1560d08ee73
Add declaration for _wsopen. It's needed for issue# 13996.

https://github.com/D-Programming-Language/druntime/commit/af9cabcd48a4fef66213ad84e4eaf8959ce36306
Merge pull request #1154 from jmdavis/_wsopen

Add declaration for _wsopen. It's needed for issue# 13996.
Comment 10 github-bugzilla 2015-03-31 02:43:54 UTC
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/e3e9f2965f438170157338767700bc1d7292dd67
Implement issue# 13996. Add File.scratchFile.

This adds a File.scratchFile, which generates a random file name and
returns an open std.stdio.File for it. Unlike with File.tmpfile, it's a
normal file which is _not_ deleted when the file is closed, and you
actually have access to the file's name, which is necessary in many
situations - particularly when writing unit tests that need to write to
a file and then read from it.

https://github.com/D-Programming-Language/phobos/commit/8a65ff8db444f35f6322debbb970166b8c587beb
Merge pull request #2956 from jmdavis/tempFile

Implement issue# 13996. Add File.tempFile.
Comment 11 github-bugzilla 2015-05-10 23:22:31 UTC
Commit pushed to revert-2956-tempFile at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/8087f354414ed39f23eec94049a1a154fcbb412c
Revert "Implement issue# 13996. Add File.tempFile."
Comment 12 github-bugzilla 2015-05-19 04:06:36 UTC
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/8087f354414ed39f23eec94049a1a154fcbb412c
Revert "Implement issue# 13996. Add File.tempFile."

https://github.com/D-Programming-Language/phobos/commit/c8c5a3255def4274dcc807e479f5aca1f8177cdf
Merge pull request #3273 from D-Programming-Language/revert-2956-tempFile

Revert "Implement issue# 13996. Add File.tempFile."
Comment 13 Timothee Cour 2015-07-26 20:34:47 UTC
the pull seems to cause this regression: https://issues.dlang.org/show_bug.cgi?id=14828
Comment 14 Jonathan M Davis 2015-07-26 21:08:35 UTC
(In reply to Timothee Cour from comment #13)
> the pull seems to cause this regression:
> https://issues.dlang.org/show_bug.cgi?id=14828

I'm not sure how it could be, since it was reverted. Maybe the changes for scratchFile triggered that bug, were reverted, and now something later is triggering it. I don't know. But since the changes for this were already reverted, it's not like we can revert them to fix the regression.
Comment 15 github-bugzilla 2017-07-19 17:42:54 UTC
Commits pushed to dmd-cxx at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/e3e9f2965f438170157338767700bc1d7292dd67
Implement issue# 13996. Add File.scratchFile.

https://github.com/dlang/phobos/commit/8a65ff8db444f35f6322debbb970166b8c587beb
Merge pull request #2956 from jmdavis/tempFile

https://github.com/dlang/phobos/commit/8087f354414ed39f23eec94049a1a154fcbb412c
Revert "Implement issue# 13996. Add File.tempFile."

https://github.com/dlang/phobos/commit/c8c5a3255def4274dcc807e479f5aca1f8177cdf
Merge pull request #3273 from D-Programming-Language/revert-2956-tempFile