D issues are now tracked on GitHub. This Bugzilla instance remains as a read-only archive.
Issue 8730 - writeln stops on a nul character, even if passed a D string
Summary: writeln stops on a nul character, even if passed a D string
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D2
Hardware: All All
: P2 minor
Assignee: No Owner
URL:
Keywords: pull
: 13065 (view as issue list)
Depends on:
Blocks:
 
Reported: 2012-09-26 13:12 UTC by Adam D. Ruppe
Modified: 2014-07-17 17:14 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Adam D. Ruppe 2012-09-26 13:12:00 UTC
import std.stdio;

void main() {
        writeln("test\0gone");
}


only prints "test". writefln("%s") prints the whole thing.

$ ./test | xxd
0000000: 7465 7374 0a

as you can see the data is indeed not being printed; it isn't just invisible on my screen.
Comment 1 Andrej Mitrovic 2012-10-04 17:11:33 UTC
I'm just guessing, but:

void writeln(T...)(T args)
{
    static if (T.length == 0)
    {
        enforce(fputc('\n', .stdout.p.handle) == '\n');
    }
    else static if (T.length == 1 &&
                    is(typeof(args[0]) : const(char)[]) &&
                    !is(typeof(args[0]) == enum) && !is(typeof(args[0]) == typeof(null)) &&
                    !isAggregateType!(typeof(args[0])))
    {
        // Specialization for strings - a very frequent case
        enforce(fprintf(.stdout.p.handle, "%.*s\n",
                        cast(int) args[0].length, args[0].ptr) >= 0);
    }
    else
    {
        // Most general instance
        stdout.write(args, '\n');
    }
}

The specialization is probably to blame. I think 'args[0].length' probably sets the max limit rather than min, but I don't know enough about fprintf internals. :)
Comment 2 Adam D. Ruppe 2012-10-09 19:24:49 UTC
(In reply to comment #1)
> The specialization is probably to blame. I think 'args[0].length' probably sets
> the max limit rather than min, but I don't know enough about fprintf internals.
> :)


Yeah, you're right. The man page for printf says "the maximum number of
       characters to be printed from a string for s and S conversions."


I'm not sure what is best here. I really think it should work, but the specialization has got to be there for a reason too.
Comment 3 Andrej Mitrovic 2012-10-09 19:38:15 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > The specialization is probably to blame. I think 'args[0].length' probably sets
> > the max limit rather than min, but I don't know enough about fprintf internals.
> > :)
> 
> 
> Yeah, you're right. The man page for printf says "the maximum number of
>        characters to be printed from a string for s and S conversions."
> 
> 
> I'm not sure what is best here. I really think it should work, but the
> specialization has got to be there for a reason too.

It could me off guard just recently. E.g. git uses two strings separated by nul in it's object format, and when I've tried to print out the contents as a char[] using writeln it would only print out a small portion of it, even though printing it as a byte[] would print much more.

Anyway this *is* a bug. We can't have it both ways:
writeln("bla\0bla");  // bla
writefln("%s", "bla\0bla");  // bla[NUL]bla
Comment 4 Brad Roberts 2012-10-09 20:33:42 UTC
Shouldn't this be trivial to fix:

Replace:
fprintf(.stdout.p.handle, "%.*s\n",
                        cast(int) args[0].length, args[0].ptr)

with:
fwrite(args[0].ptr, 1, args[0].length, .stdout.p.handle)
Comment 5 Brad Roberts 2012-10-09 20:36:53 UTC
oops, followed by the same code as in the length == 0 code to get the \n.  At which point it's questionable that specialization is all that special.
Comment 6 Musashi Tamura 2013-12-15 03:53:31 UTC
write("test\0gone\n");
write("test\0gone", '\n');
writeln("test\0gon", "e");

All of the above code outputs NUL.
I expect writeln("test\0gone") has the same behavior.
Comment 7 monarchdodra 2013-12-15 10:25:36 UTC
Has anybody done anything to fix this? If not, then IMO, it's simply invalid.

I think this is "just" an OS output issue: When printing a null character to console, the console seizes to print for the current line.

For example, on windows,
writeln("test\0gone");
prints
test

Yet
writeln("test\0gone\n");
prints
test gone


and
writeln("test\0gon", 'e');
prints
test gone

Weird, right?

So I decided to simply print to file, and check what is actually being *passed* to the stream (NOT what the console prints). Sure enough, everything is correctly placed in the stream, null and all.

Conclusion => Output stream is the one to blame; D passes everything correctly to the stream.

Gonna see how this behaves on linux next.
Comment 8 Musashi Tamura 2013-12-16 03:13:29 UTC
This problem is only in "std.stdio.writeln".
"std.stdio.File.writeln" doesn't have it.

// abc.d
import std.stdio;
void main() {
    writeln("test\0gone"); // output "test\n"
    stdout.writeln("test\0gone"); // output "test\0gone\n"
}

$ rdmd abc | od -a
0000000   t   e   s   t  nl   t   e   s   t nul   g   o   n   e  nl
0000017
Comment 9 bearophile_hugs 2014-07-07 11:14:37 UTC
*** Issue 13065 has been marked as a duplicate of this issue. ***
Comment 10 hsteoh 2014-07-15 04:10:15 UTC
Wow, has this trivial bug really been around for 2 years?!

Anyway, here's the fix:

https://github.com/D-Programming-Language/phobos/pull/2334
Comment 11 github-bugzilla 2014-07-17 16:51:36 UTC
Commit pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/4f81791438d283edbe508125bbc13ff00e959dd2
Merge pull request #2334 from quickfur/issue8730

std.stdio.writeln should write strings with embedded nulls correctly
Comment 12 hsteoh 2014-07-17 17:14:56 UTC
Verified fixed in git HEAD.