Issue 5548 - Efficient std.conv.to conversions
Summary: Efficient std.conv.to conversions
Status: REOPENED
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D2
Hardware: All All
: P4 enhancement
Assignee: No Owner
URL:
Keywords: bootcamp, performance
Depends on:
Blocks:
 
Reported: 2011-02-08 12:47 UTC by Tomasz Sowiński
Modified: 2024-12-01 16:13 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Tomasz Sowiński 2011-02-08 12:47:58 UTC
Currently most to!T conversions allocate behind the scenes. I propose that the conversions where T is dynamically sized (like an array) get a speedy overload of the form:

void to(T, S, O)(S source, O output) if (isOutputRange!(O, T));

Example: to!string(9234820934, appender);

For further investigation: consider assuming the output range to be buffered.
Comment 1 Steven Schveighoffer 2011-02-08 12:52:10 UTC
Do we not have the functionality for this (at least for strings) in std.format?

Are there use cases for this beyond strings?
Comment 2 Kenji Hara 2012-06-14 02:33:25 UTC
I agree with Steven. If you want to represent objects with output range of characters, you can use std.format.formatValue family directly.
Comment 3 Orvid King 2013-11-01 08:26:19 UTC
After a bit of testing, std.format.fomatValue is not a valid alternative to having std.conv.to provide overloads that accept an output buffer. The reason for this is simple, the following code is 3x slower than simply using to!string(int):


auto toStr = benchmark!(() {
	import std.format;
	import std.range : Appender;
	
	auto ret = Appender!string();
	auto fmt = FormatSpec!char("%s");
	ret.reserve(4096);
	for (auto i = 0; i < ObjectCount * 11; i++)
	{
		ret.formatValue(i, fmt);
		ret.clear();
	}
})(1);
	 
writefln("Took %s ms (%s) to serialize 100k SimpleObjects with an average payload of %s bytes (%s).", res[0].msecs, toStr[0].msecs, cast(real)totalPayload / ObjectCount, totalPayload);


In my tests where ObjectCount was 100k, it takes 400ms for to!string(int) to create all the strings, and 1100ms for formatValue to do the same. formattedWrite is even worse, 1500ms. In my current implementation of a dynamic JSON (de)serializer, more than half of my time is eaten up by converting integers to strings when performing deserialization. I use a pre-allocated output range as the destination, so I know I'm not doing any allocations within my code.
Comment 4 Jack Stouffer 2017-02-02 16:03:27 UTC
Definitely agree that this should exist. Currently in the array overloads, there are a lot of uses of functions that use appender internally and then return the managed array with app.data(), only to be appended to another appender in the parent function. Having OutputRange overloads would represent a significant performance boost.

Once https://github.com/dlang/phobos/pull/5018 is pulled, I'll start working on this.

BTW, in the mean time, converting to string can be used with OutputRanges via std.range.put and std.conv.toChars.
Comment 5 dlangBugzillaToGithub 2024-12-01 16:13:56 UTC
THIS ISSUE HAS BEEN MOVED TO GITHUB

https://github.com/dlang/phobos/issues/9897

DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB