Re: Feedback on the proposal to change U+FFFD generation when decoding ill-formed UTF-8

From: Asmus Freytag via Unicode <>
Date: Thu, 1 Jun 2017 07:21:28 -0700

On 6/1/2017 2:32 AM, Henri Sivonen via Unicode wrote:
> O
> On Wed, May 31, 2017 at 10:38 PM, Doug Ewell via Unicode
> <> wrote:
>> Henri Sivonen wrote:
>>> If anything, I hope this thread results in the establishment of a
>>> requirement for proposals to come with proper research about what
>>> multiple prominent implementations to about the subject matter of a
>>> proposal concerning changes to text about implementation behavior.
>> Considering that several folks have objected that the U+FFFD
>> recommendation is perceived as having the weight of a requirement, I
>> think adding Henri's good advice above as a "requirement" seems
>> heavy-handed. Who will judge how much research qualifies as "proper"?

I agree with Henri on these general points:

1) Requiring extensive research on implementation practice is crucial in
dealing with any changes to long standing definitions, algorithms,
properties and recommendations.
2) Not having a perfect definition of what "extensive" means is not an
excuse to do nothing.
3) Evaluating only the proposer's implementation (or only ICU) is not
4) Changing a recommendation that many implementers (or worse, an
implementers' collective) have chosen to adopt is a breaking change.
5) Breaking changes to fundamental algorithms require extraordinarily
strong justification including, but not limited to "proof" that the
existing definition/recommendation is not workable or presents grave
security risks that cannot be mitigated any other way.

I continue to see a disturbing lack of appreciation of these issues in
some of the replies to this discussion (and some past decisions by the UTC).

Received on Thu Jun 01 2017 - 09:22:00 CDT

This archive was generated by hypermail 2.2.0 : Thu Jun 01 2017 - 09:22:00 CDT