Purpose

Describe the use of Unicode as infrastructure for shareable Japanese character data support on a very large database engine.

Description

The original Teradata data base system was the first commercial database engine based upon the client-server model. It used cutting edge technology for the time. Heterogeneous ASCII and EBCDIC clients could share character data which was stored on the server utilizing an extended form of ASCII. As time passed, there has been continuous updating of both the hardware and the software, and significant inroads have been made in the area of internationalization, including support for Western Europe and Japan. However, much of the infrastructure remains from the original system. NCR has approved an upgrade of this infrastructure, converting aging Pascal code to C, and making Unicode support intrinsic within the database engine. These two efforts have proceeded independently, but writing Unicode support in Pascal is quite challenging. Therefore, the decision was made to implement Unicode inside, rather than throughout, the database engine, until the two projects merge. Meanwhile, the difficulties of providing Unicode support are exacerbated by an inadequate and ambiguous SQL internationalization model. Still, extensions to the SQL model, innovative use of previous Japanese support, entirely new translation and collation processing, and a new Unicode knowledgeable hash function for dividing the workload amongst the many processors, provides significant heterogeneous multilingual support.

Conclusion

Even in the situation where external Unicode support is not feasible, the use of internal Unicode infrastructure provides many benefits to the users of a system. The new C based parser now available should allow us to complete the transition to Unicode throughout the system.