L2/08-045
Title: Proposal for Handling Rejected and Under Investigation Status
Date: January 25, 2008
Source: Ken Whistler
Action: For discussion by joint UTC/L2 meetingBackground
For many years now, the Unicode website has had several pages dedicated to tracking UTC decisions taken about characters (or scripts) proposed for encoding:
http://www.unicode.org/alloc/Pipeline.html
The "Pipeline" page, which tracks characters accepted for encoding by the UTC and/or WG2, as well as any variation sequences.
http://www.unicode.org/alloc/rejected.html
The Rejected Characters and Scripts page, which keeps a historical record of characters (or proposed scripts) which didn't make the grade, and for which the UTC decided to go on record to note a non-approval (which would indicate a UTC precedent, among other things).
http://www.unicode.org/alloc/investigation.html
The Characters Under Investigation page, which has only ever had a few entries (and none added since 2002), for some problematic characters that weren't approved, but also not rejected with a precedent established. The table was to indicate characters where more information might clarify the situation and lead to a different decision. (Actually, the investigation.html page itself is rather new -- it was simply split off from the rejected.html page, so as not to mix the status of rejected and under investigation in the same table. But the entries themselves date from 2002.)
The Problems
The Pipeline page has always served its function well, and has become a crucial part of the UTC process for tracking new characters before they go into a formally released version of the standard. And there is very little confusion about what its entries mean. So I think that page is working fine.
The Rejected and Under Investigation pages, on the other hand, have a number of problems, as noted by some other UTC members on occasions, and periodically by the general public accessing those pages.
One of the problems was that it was confusing to have rejected status and under investigation status tracked on a single page. The editorial committee dealt with that be splitting the original table into the current two pages.
The more substantial, ongoing problem is that unlike the Pipeline page, which is complete and for which the UTC has active, ongoing maintenance process, the Rejected and Under Investigation pages only ever record certain of the UTC decisions, on a somewhat arbitrary basis related to how contentious an issue seems at the time. And once entries are added to those tables, there isn't any active process for the UTC to review entries to see if they are still contain current and relevant information, or if they have been overtaken by further events.
An example: Egyptian hieroglyphics sat in the table for a long time listed as "under investigation", even after a substantially revised proposal was submitted, approved, and started ballotting.
Furthermore, our public pages about submitting proposals and FAQs on the process point people to these pages, among others, to help determine whether a character has already been proposed and accepted or rejected, before they submit a new proposal. As it stands now, the entries for rejected and under investigation don't really supply useful or consistent enough information to be very helpful for that purpose.
And the marginal usefulness of the Under Investigation status has long been overtaken by other UTC process. In particular, the UTC now generally issues a Public Review Issue for proposals where it is seeking active feedback that may contribute to the investigation of appropriateness for encoding. Or, in other cases, it simply records an action item for someone to communicate back to a proposal submitter with UTC feedback and suggestions -- without tracking any such status for a proposal as requiring it to be listed in an "Under Investigation" table.
Ordinarily a problem with our web pages like this would simply be taken up by the editorial committee to get them fixed, but in this case, the content of the rejected and under investigation pages is the cumulative result of a number of explicit decisions by the UTC to add such-and-such entries to the rejection list, and so on. So before just udpating the pages further, it seems appropriate to get some UTC discussion and consensus about the course forward.
The Proposal
I suggest obsoleting both the rejected.html and investigation.html pages and replacing them with a better maintained page modeled a little more closely on our Resolved Public Review Issues pages, such as:
http://www.unicode.org/review/resolved-pri-100.html
The page would keep a sequentially numbered and dated series of what could be called, perhaps, "Non-approval Notices", which would serve as an archive of specific decisions taken by the UTC which did not result in approvals for encoding.
The initial content of the page's entries would consist of the entries from the rejected and investigation pages, but restructured into discursive entries that explain slightly more about what the proposed characters (or script) were, and then for each gives a resolved status -- whether formally rejected or whatever.
For each entry, there should be a link to the actual proposal document(s), so we can get back to what the item was about. A date of the proposal. And a date of the decision taken.
Then instead of directing that a particular character be "added to the Rejected table", for example, when the UTC decides to make a precedent-setting rejection, it would create an action item to "issue a Non-approval Notice". That would imply drafting the complete entry for addition to the list, together with rationale, and would also be the basis for communicating back to the proposer that a precedent-setting non-approval decision was taken. This would differ from the more run-of-the-mill decisions to just continue working with a proposal author to improve a proposal before it gets approval.
Further, when approving characters or scripts, one of the routine steps the UTC should take would be to review the Non-approval Notices list to see if any precedent was involved, or whether a subsequent decision impacts an earlier Non-approval. Such would be the case, for example, in the long trail of Malalayalam chillu letter decisions. Any action to approve something subject to a precedent-setting Non-approval Notice would then also require an action item to go back and update the relevant Non-approval Notice with the later decision. This would be much, much less confusing that simply having characters like the Malayalam chillus sitting indefinitely in the Rejected table, even if annotated.
I would volunteer, with the help of the editorial committee, to do the research required on the current Rejected and Under Investigation table entries, to dig out the original proposals for reference and to flesh out the descriptions and resolutions, so as to provide the initial list of Non-approval Notices (or whatever we decide to call them), which the UTC could then start using as a part of a more generic maintenance process.