Jump to content

GBK (character encoding)

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by JCBradfield (talk | contribs) at 19:36, 7 July 2009 (removed disputed accuracy tag - queried point addressed by edits in last year). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

GBK is an extension of the GB2312 character set for simplified Chinese characters, used in the People's Republic of China.

GB stands for National Standard, while K stands for Extension. GBK not only extended the old standard GB2312 with Traditional Chinese characters, but also with Chinese characters that were simplified after the establishment of GB2312 in 1981. With the arrival of GBK, certain names with characters formerly unrepresentable, like the "rong" (镕) character in former Chinese Premier Zhu Rongji's name, are now representable.

History

In 1993, the Unicode 1.1 standard was released, including 20,902 characters used in mainland China, Taiwan, Japan and Korea. Following this, China released GB13000.1-93, a national standard (guóbiāo) equivalent of Unicode 1.1.

The GBK character set was defined in 1993 as an extension of GB2312-80, while also including the characters of GB13000.1-93 through the unused codepoints available in GB2312. Hence GBK is upward compatible with GB2312.

Microsoft implemented GBK in Windows 95 as Code Page 936. While GBK was never an official standard, widespread usage of Windows 95 led to GBK becoming the de facto standard. While GBK included all the Chinese characters defined in Unicode 1.1 and GB13000.1-93, these standards used different code tables. The primary reason for its existence was simply to bridge the gap between GB2312-80 and GB13000.1-93.

In 1995, China National Information Technology Standardization Technical Committee set down the Chinese Internal Code Specification (Chinese: 汉字内码扩展规范(GBK); pinyin: Hànzì Nèimǎ Kuòzhǎn Guīfàn (GBK)), Version 1.0, known as GBK 1.0, which is a slight extension of Codepage 936. The newly added 95 characters were not found in GB 13000.1-1993, and were provisionally assigned Unicode PUA code points.

Microsoft later added the euro sign to Codepage 936 and assigned the code 0x80 to it. This is not a valid code point in GBK 1.0.

In 2000, the GB18030-2000 standard was released, superseding yet maintaining compatibility with GBK 1.0. It increased the number of definitions of Chinese characters and extended the number of possible characters through the implementation of four-byte character spaces. The subset of GB 18030 consisting of one-byte and two-byte characters is sometimes also referred to as GBK. Mapping to Unicode has been slightly changed, though, as some characters are now defined in Unicode. In the most up-to-date form of the standard, GB 18030-2005, only 14 characters are still mapped to Unicode PUA.

Encoding

A character is encoded as 1 or 2 bytes. A byte in the range 007F is a single byte that means the same thing as it does in ASCII. Strictly speaking, there are 96 characters and 32 control codes in this range.

A byte with the high bit set indicates that it is the first of 2 bytes. Loosely speaking, the first byte is in the range 81FE (that is, never 80 or FF), and the second byte is 40FE for some areas and 80FE for others.

More specifically, the following ranges of bytes are defined:

GBK Encoding Ranges
range byte 1 byte 2 code points characters
GB 18030 GBK 1.0 Codepage 936 GB 2312
Level GBK/1 A1A9 A1FE 846 728 717 702 682
Level GBK/2 B0F7 A1FE 6,768 6,763 6,763 6,763
Level GBK/3 81A0 40FE except 7F 6,080 6,080 6,080
Level GBK/4 AAFE 40A0 except 7F 8,160 8,160 8,080
Level GBK/5 A8A9 40A0 except 7F 192 166 166
user-defined AAAF A1FE 564
user-defined F8FE A1FE 658
user-defined A1A7 40A0 except 7F 672
total: 23,940 21,897 21,886 21,791 7,445

In graphical form, the following figure shows the space of all 64K possible 2-byte codes. Green and yellow areas are assigned GBK codepoints, red are for user-defined characters. The uncolored areas are invalid byte combinations.

Relationship to other encodings

The areas indicated in the previous section as GBK/1 and GBK/2, taken by themselves, is simply GB2312-80 in its usual encoding. GB2312, or more properly the EUC-CN encoding thereof, takes a pair of bytes from the range A1FE, like any 94² ISO-2022 character set loaded into GR. This corresponds to the lower-right quarter of the illustration above. However, GB2312 does not assign any code points to the rows located at ABB0 and F8FE, even though it had staked out the territory.

GBK added extensions to this. You can see that the two gaps were filled in with user-defined areas.

More significantly, it extended the range of the bytes. Having two-byte characters in the ISO-2022 GR range gives a limit of 94²=8,836 possibilities. Abandoning the ISO-2022 model of strict regions for graphics and control characters, but retaining the feature of low bytes being 1-byte characters and pairs of high bytes denoting a character, you could potentially have 128²=16,384 positions. GBK takes part of that, extending the range from A1FE (94 choices for each byte) to 81FE (126 choices) for the first byte and 40FE (191 choices) for the second byte, for a total of 24,066 positions.

Microsoft's Code Page 936 is generally thought of as being GBK. It has bytes in the same range, with assignments that seem to match if you compare them. However, the total number of two-byte code points defined is 21,791 so there must be some differences—at the very least, 95 are missing.

GBK's successor, GB18030-2000, uses the remaining range available to the second byte to further expand the number of possibilities while retaining GBK as a subset.