The ISO / Unicode Merger

An Interview with Ed Hart *

Conducted by Laura Wideburg ** on October 16, 1995

“I worked on a position paper for Share Inc. We were working on ASCII and EBCDIC. The IBM version had 95 characters, 3 were not in ASCII and vice versa. There were problems just converting between these two sets. At that time there was 7-bit ISO 646, and they were all country specific. There were 3 different versions for Switzerland.

In 1991 you had ISO 10646 and Unicode 1.0. Both codes wanted to be the one code. They were also incompatible, so the translation between the codes would be impossible. For instance, you had the floating diacritics in Unified CJK. There were different ways of encoding, and so round-trip integrity was in question. Both 10646 and Unicode were going full bore. Several companies were already going Unicode. Governments were deciding which standard to choose as their standard. I realized that two standards may exist and there’d be attempts to make round-trip conversion. Our main concern in SHARE was round-trip integrity. We had all these problems in the US with just 6 characters. What kind of problems would you have with 65 000? I termed it a disaster and got people’s attention.

In September 1990, I got a call from Isai [Isai Scheinberg, IBM's representative to Unicode]. He asked, “What are we going to do about it? How is SHARE going to vote on the [10646-DIS 1] ballot?”

I knew there were good reasons for choosing Unicode, but also good reasons for choosing ISO. I really didn’t know which one would be better.

One of the SHARE members called me. “This isn’t an academic question. It will take millions of dollars to install. The conversion is expensive, the storage is expensive, and the memory is expensive.” Multi-byte code was just not academic.

So we went out to the SHARE membership. Isai and Monty [???] sent out a survey about the two codes. Do you need a standard? Which one would you choose? Or both? Or a combination of the two?”

We had Mike Ksar, Masami Hasegawa [Editor of 10646], Lee Collins [Unicode's Han-Unification Expert] and Ken Whistler describe the differences.

We had 13% participation in the survey. We had 260,000 members, many Fortune 500 companies. Many people didn’t vote because they felt they didn’t know enough about the technical issues. But one thing was clear. They wanted just one standard.

We wrote a summary paper with bar charts, and we sent it to ISO and probably to Unicode as well.

In May, ISO [JTC1/SC2/WG2] was going to meet in San Francisco. This was the perfect spot to discuss a merger. I brought it up to Jerry Andersen. Mike Ksar had just been elected convener of the ISO group and was concerned about doing his job correctly. Jerry and I talked about the merger every week. I wanted to go for the merger at ISO, but Mike said as soon as ISO was over, we could use the H-P facilities afterwards, and “Ed, why don’t you lead the discussion?”

It was like trying to get the Hatfields and McCoys together. I prayed about it, I can tell you. I talked to my manager about it. We realized that we couldn’t even begin to talk about a merger until there was trust created between the two parties. We had to build the trust, even while appearing to make progress on the technical issues.

Jerry made an announcement, and put the merger on the table. My goal was to pull it together. I was neutral, I didn’t care whether it was Unicode or ISO and that gave me some advantage.

First, I went out just with Unicode. I meet Lee, Joe and Isai. I’m not sure I met Mark then, but maybe not until the actual meeting.

The next night, I went out with Mike, Jerry and Hasegawa.

Both groups said the same thing: “We don’t trust ‘em.”

At IBM, we had people in both camps, and the IBM corporate direction had to be set. If we pulled off the merger, IBM would buy it.

The ISO meeting broke up on Wednesday morning. On Wednesday afternoon, I started with some trust building exercises. I only allowed one person to speak at a time. First we had a spokesperson describe: “Why we did it this way”. Ken spoke for Unicode, Jerry for ISO. Really, 10646 was more big-frame, and Unicode was more workstation. A big issue was how wide the code was. ISO said, 8, 16, 24 or 32. Unicode said 16, period. We got over that one.

We went back and forth and I said, “Give me an advantage from the other person’s scheme.” So this way, they better understood each other technically and each side gained credibility.

At one in the afternoon on Thursday, I said, “It’s time to talk about a merger.” I asked questions, “What is the shape of 16-bit? Are you going to reserve space for control characters? How many characters are you going to include?” The number of characters would determine how big a player Unicode was going to be.

Willy Bohn from IBM in Germany said, “So, why don’t we just go with 65 000 characters?” Another ISO guy said, “Do you know what you just said?”

Isai volunteered to investigate control characters, and we had about 5 other issues.

On Friday, Jaing said, “You’ll have this for me tomorrow.” I spent the night until 3 a.m. finishing the minutes!

At that meeting, we (one) built trust (two) respected each other and (three) had agreement on five issues.

We published the minutes and had them distributed. So many countries voted asking for a merger. Now how to handle this merger?

Europe would be a real mess. Their idea was, “They have just a bunch of independent cowboys out there.” Avery [Bishop] called me. “We don’t have a snowball’s chance in hell.” I said, “If we don’t try, we’ll have even less of a chance.”

In the end, it was a success. There was a real incentive to have just one code. Really, it was a common sense thing. Once people put aside their egos and realized the other side also had reasonable people, it went fine. We had a rational, professional discussion. I remember Mark was at those meetings.

Now how are we going to get ISO to ratify it in Geneva? Could we keep the same feeling we had in San Francisco? There were lots of people in Europe who didn’t like Unicode. We had an ad hoc meeting in the morning, and people continued to respect what the other side had to say. There were lots of informal discussions over coffee. Mark was calm and deliberate and respectful, and I think that that had a lot to do with Unicode’s acceptance. If he’d been smart-alecky, it would have failed. The way he came across, the Europeans just had to respect him.

So we worked through several issues. Masami brought in some documents which had already accounted for a merger, and he got some flack for that. So I don’t think that they were distributed as documents.

After the October meeting in Paris, we had a working group meeting under subcommittee 2 (SC2). WG2 was worried about technical issues. One of our goals was to produce a draft for the ballot. We resolved the comments and went to the next level.

In Paris, there were no showstoppers, and we got approval for the merger. Meanwhile, Unicode published their book, and in the book they mentioned they were discussing a merger with ISO 10646.

Eventually we had the merger. Unicode 1.1 and ISO were conformed.

As for my role, I saw myself more as a facilitator, getting the merger started. People realized that they could come to an agreement. I remember the first time they came to an agreement, and I took a break and I said, “Yes!”

If the people hadn’t been willing to come together, it never would have happened. And like I said, I did A LOT of praying.
 

---

* Ed Hart - The Johns Hopkins University Applied Physics Laboratory
Mr. Hart is a senior engineer at the Johns Hopkins University Applied Physics Laboratory where he coordinates networking for the Computing Systems Group. Prior to this, he worked as a systems programmer at Bell Telephone Laboratories. He has a BES in EE from the Johns Hopkins University, an MS in EE from Columbia University, and a MS in Computer Science from the Johns Hopkins University.
Since 1990, he has represented SHARE Inc., an association of IBM customers, to the US L2 technical standards committee for codes and character sets. There he served as the vice-chairman and later the chairman. While vice- chairman, in 1991 he facilitated the first discussion between the Unicode Consortium and ISO, and represented the US at subsequent discussions. These talks led to an agreement to merge ISO/IEC 10646 and Unicode. Since 1994, he has represented SHARE Inc. to the Unicode Technical Committee. Mr. Hart was co-editor of the ISO Technical Report, ISO/IEC TR 15285, "An operational model for characters and glyphs" on which this presentation is based.

** This interview is printed with permission from the interviewer. © Laura Wideburg 1998. It may not be reprinted without Wideburg's permission.
All other material is ©Unicode, Inc. 1998 and may be reprinted, for purposes of educational or press releases.  Any other reprinting requires permission of the Unicode Consortium, which can be reached at 650-693-3921.  

---