Character Sets

Overview

In computing, a character set is a standardised way to represent text and symbols in binary so that computers can process and display them correctly. Each character (e.g., letters, digits, punctuation) is assigned a unique binary code. Understanding how character sets like ASCII and Unicode work is crucial for handling text data across different systems and languages.

What is a Character Set?

• A character set maps characters (letters, numbers, symbols) to specific binary values.
• This enables computers to store, transmit, and display text correctly, even across different devices and platforms.
• Each character is assigned a unique numeric code, which is then converted to binary.

ASCII (American Standard Code for Information Interchange)

Overview: One of the earliest character sets, ASCII uses 7 bits to represent characters.

7-bit ASCII

Can represent 128 characters ($2^7$ = 128), including:

• Uppercase and lowercase English letters (A–Z, a–z)
• Digits (0–9)
• Punctuation and special symbols (e.g., !, @, #)
• Control characters (e.g., newline, tab)

8-bit ASCII (Extended ASCII)

Extends the set to 256 characters ($2^8$ = 256), adding support for additional symbols and simple graphical characters.

Usage: Suitable for English text and basic symbols but limited for international use.

Unicode

Overview: A more comprehensive character set designed to support a wide range of characters from multiple languages and scripts.

• 16-bit Unicode (UTF-16): Initially supported 65,536 characters.

• UTF-8 Encoding: Variable-length encoding that uses 1 to 4 bytes, ensuring compatibility with ASCII for the first 128 characters. Why Unicode?:

• Supports thousands of characters, including non-Latin scripts (e.g., Chinese, Arabic).

• Includes emojis, mathematical symbols, and more. Usage: Essential for global applications, such as web development, where diverse languages must be supported.

Differences Between ASCII and Unicode

Feature	ASCII	Unicode
Bit Length	7-bit (or 8-bit extended)	8 to 32 bits (variable length)
Character Support	128 (7-bit) or 256 (8-bit)	Over 1 million characters
Scope	English and basic symbols	Global, supports all languages
Compatibility	Not suitable for international use	Backward-compatible with ASCII

Why Character Sets Matter

1. Data Interoperability:

• Ensures consistent representation of text across different systems.

2. Internationalisation:

• Unicode enables software to support multiple languages and scripts.

3. Storage and Transmission:

• Efficient storage of text data in binary format, critical for data processing and network communication.

Examples

Note Summary