Reading from and writing to the Windows console should be lossy

[ChrisDenton]

The Windows console works with text whereas our Read and Write traits deal with arbitrary bytes. We currently use the Unicode Windows APIs and assume the bytes are in fact UTF-8. If they're not then we return an error. Similarly we expect that the input from the console will be actual Unicode text (I'd be surprised if it ever wasn't but I guess it's possible) and again we error if it's not. See here, here and here.

Seeing as the console is a text interface, rather than a binary one, why not lossily replace any invalid Unicode to the replacement character?

[ChrisDenton]

This was very briefly touched on in a recent libs api meeting though it wasn't the main topic of conversation.

One issue that was brought up was that maybe we do want to return an error in order to help catch programmer mistakes. I think if that's true then this shouldn't be a Windows-only thing; it should be a check available when writing to a tty on all platforms.

There's no particular reason to treat Windows as special here. In a glorious UTF-8 everywhere future (coming soon) we'd just use the normal WriteFile function for writing bytes to the console (just like a normal file), rather than converting to UTF-16 and using WriteConsoleW. They would be no reason to do a UTF-8 check because this is handled by the console (almost certainly by doing the replacement thing for us but it could error if it wants to).

[QuineDot]

I think if that's true then this shouldn't be a Windows-only thing; it should be a check available when writing to a tty on all platforms.

Don't do that on all platforms without introducing a new type or introducing new methods and macros to do so. I write OsString to StdOut (as bytes) even when it's a tty in order to print out what the user has passed in losslessly. And yes, I work on platforms where the default locale and definitely the filenames are not UTF8.

In a glorious UTF-8 everywhere future (coming soon)

I don't see unix disallowing the scenario above ever, really, but if happens I imagine it's decades off.