how many bytes did each character require for extended ascii characters
a single byte
12361110
multi-byte character encodings (character encodings that can handle more than 256 different characters) are also true extended ASCII. That means all ASCII characters are encoded with a single byte with the value that is used in ASCII to encode that character. They can be used in file formats where only ASCII bytes are used for keywords and file format syntax, while bytes 0x80-0xFF might be used for free text, including most programming languages, where language keywords, variable names, and function names must be in ASCII, but string constants and comments can use non-ASCII characters. This makes it much easier to
Extended ASCII
12361110
multi-byte character encodings (character encodings that can handle more than 256 different characters) are also true extended ASCII. That means all ASCII characters are encoded with a single byte with the value that is used in ASCII to encode that character. They can be used in file formats where only ASCII bytes are used for keywords and file format syntax, while bytes 0x80-0xFF might be used for free text, including most programming languages, where language keywords, variable names, and function names must be in ASCII, but string constants and comments can use non-ASCII characters. This makes it much easier to
Extended ASCII
2921084
of the 256 possible bytes are passed through unencoded as a single byte, whether that result is a printable ASCII character or not. Only NUL, LF, CR, and = are escaped. LF and CR are escaped because the RFCs that define Internet messages still require that carriage returns and line feeds have special meaning in a mail message. = is the escape character, so it itself is escaped. NUL is also escaped because of problems handling null characters in common code, although as an optimization yEnc adds 42 to every source byte so that data with large stretches of 0x00
YEnc
17757860
literal is to use a single quote (codice_1) for character literals, as contrasted by the use of a double quote (codice_2) for string literals. For example, codice_3 indicates the single character codice_4 while codice_5 indicates the string codice_4 of length 1. The representation of a character within the computer memory, in storage, and in data transmission, is dependent on a particular character encoding scheme. For example, an ASCII (or extended ASCII) scheme will use a single byte of computer memory, while a UTF-8 scheme will use one or more bytes, depending on the particular character being encoded. Alternative ways to
Character literal
606331
more primitive as well as a much longer high-level encoding scheme was also used – BNPF (Begin-Negative-Positive-Finish). In BNPF encoding, a single byte (8 bits) would be represented by a highly redundant character framing sequence starting with a single ASCII "B", eight ASCII characters where a "0" would be represented by a "N" and a "1" would be represented by a "P", followed by an ending ASCII "F". These ten-character ASCII sequences were separated by one or more whitespace characters, therefore using at least eleven ASCII characters for each byte stored (9% efficiency). The ASCII "N" and "P" characters differ
Punched tape
5435630
published in 2015 defines a GBK encoder as a GB 18030 encoder with a single-byte euro sign and without four-byte sequences. A character is encoded as 1 or 2 bytes. A byte in the range codice_1–codice_2 is a single byte that means the same thing as it does in ASCII. Strictly speaking, there are 95 characters and 33 control codes in this range. A byte with the high bit set indicates that it is the first of 2 bytes. Loosely speaking, the first byte is in the range codice_3–codice_4 (that is, never codice_5 or codice_6), and the second byte is
GBK (character encoding)
12361099
storage. (Assuming that the unused 8th bit of each byte was not reused in some way, such as error checking, Boolean fields, or packing 8 characters into 7 bytes.) An eight-bit character set (using one byte per character) encodes 256 characters, so it can include ASCII plus 128 more characters. (Variable multi-byte schemes based on ASCII can add many more additional characters. Unmodified code will provide valid results searching for sequences of ASCII and extended characters. But software must be more sophisticated, only accepting or creating valid multi-byte characters, and always treating multi-byte characters as one character whenever counting, copying,
Extended ASCII
12361112
(0x00-0x7F). Shift JIS can directly be used in programming languages and languages such as HTML (if accepting that backslash looks like ¥), because the bytes used for free text delimiters are not used as part of non-ASCII characters. UTF-16 is even less extended ASCII because ASCII characters are stored as two bytes with one byte equal to 0x00. Porting an existing system to support character sets as Shift JIS or UTF-16 is complicated and bug prone. For programming languages and document languages such as C and HTML, the principle of Extended ASCII is important, since it enables many different encodings
Extended ASCII
5775791
Legacy programs can generally handle UTF-8 encoded files, even if they contain non-ASCII characters. For instance, the C printf function can print a UTF-8 string, as it only looks for the ASCII '%' character to define a formatting string, and prints all other bytes unchanged, thus non-ASCII characters will be output unchanged. UTF-16 and UTF-32 are incompatible with ASCII files, and thus require Unicode-aware programs to display, print and manipulate them, even if the file is known to contain only characters in the ASCII subset. Because they contain many zero bytes, the strings cannot be manipulated by normal null-terminated string
Comparison of Unicode encodings
12361111
introduce a multi-byte character set into existing systems that use extended ASCII. UTF-8 is true extended ASCII, as are some Extended Unix Code encodings. ISO/IEC 6937 is not extended ASCII because its code point 0x24 corresponds to the general currency sign (¤) rather than to the dollar sign ($), but it is an extended version of the International Reference Version of ISO 646. Shift JIS is not true extended ASCII, because although ASCII characters are encoded the same as in ASCII (except for the backslash; its position is used for the yen character), multi-byte characters can also include ASCII bytes
Extended ASCII
12361098
"÷". C interprets the two-character sequences "<%" and "%>" as equivalent to "{" and "}" because "{" and "}" are not available on all keyboards.) The English alphabet was generally useful on computers around the world for programming, operation, and international communication, so extended character sets that added to the original ASCII set were a good idea. Seven-bit ASCII encodes only 128 characters (including control characters). When computers and peripherals standardized on eight-bit bytes, it became obvious that computers and software could handle text that uses 256-character sets at almost no additional cost in programming, and no additional cost for
Extended ASCII
13939097
characters. The following table shows the final bytes in hexadecimal and the corresponding ASCII characters after the intermediate bytes. The EACC is the only multibyte encoding of MARC-8, it encodes each CJK character in three ASCII bytes. For example, to encode the U+4EBA CJK character (人) you will need the following bytes The \x1B\x24\x31 switches to EACC/CJK, and the \x21\x30\x64 corresponds to the U+4EBA. In addition to the ISO-2022 character sets, the following custom sets are available too. The byte designation follows the escape byte (hexadecimal 1B). There is no intermediate byte. MARC-8 The MARC-8 charset is a MARC standard
MARC-8
62341
character set (the collection of characters which it can represent), though some character sets have multiple character encodings and vice versa. Character encodings may be broadly grouped according to the number of bytes required to represent a single character: there are single byte encodings, multibyte (also called wide) encodings, and variable-width (also called variable-length) encodings. The earliest character encodings were single-byte, the best known example of which is ASCII. ASCII remains in use today, for example in HTTP headers. However, single-byte encodings cannot model character sets with more than 256 characters. Scripts which require large character sets such as Chinese,
Code
4077190
a complete copy of the affected program. In most hex editor applications, the data of the computer file is represented as hexadecimal values grouped in 4 groups of 4 bytes (or two groups of 8 bytes), followed by one group of 16 printable ASCII characters which correspond to each pair of hex values (each byte). Non-printable ASCII characters (e.g., Bell) and characters that would take more than one character space (e.g., tab) are typically represented by a dot (".") in the following ASCII field. Since the invention of computers and their different uses, a variety of file formats has been
Hex editor
16012
more importantly, forward compatibility is ensured as software that recognizes only 7-bit ASCII characters as special and does not alter bytes with the highest bit set (as is often done to support 8-bit ASCII extensions such as ISO-8859-1) will preserve UTF-8 data unchanged. ASCII ASCII ( ), abbreviated from American Standard Code for Information Interchange, is a character encoding standard for electronic communication. ASCII codes represent text in computers, telecommunications equipment, and other devices. Most modern character-encoding schemes are based on ASCII, although they support many additional characters. ASCII is the traditional name for the encoding system; the Internet Assigned
ASCII
123375
likely to be adopted by systems with 36 bits (as five seven-bit ASCII characters fit into one word). As eight-bit bytes became widespread, ASCII systems sometimes used the "unused" bit for other purposes, such as metacharacters to mark the borders of records or words. This made it difficult to change the code to work with EBCDIC. On the PDP-11 bytes with the high bit set were treated as negative numbers, behavior that was copied to C, causing unexpected problems with EBCDIC. Both of these problems also hindered the adoption of extended ASCII character sets. The table below shows the "invariant
EBCDIC
12361092
Extended ASCII Extended ASCII (EASCII or high ASCII) character encodings are eight-bit or larger encodings that include the standard seven-bit ASCII characters, plus additional characters. The use of the term is sometimes criticized, because it can be mistakenly interpreted to mean that the ASCII standard has been updated to include more than 128 characters or that the term unambiguously identifies a single encoding, neither of which is the case. There are many extended ASCII encodings (more than 220 DOS and Windows codepages). EBCDIC ("the other" major 8-bit character code) likewise developed many extended variants (more than 186 EBCDIC codepages) over
Extended ASCII
12361103
0xAF) that implemented low-resolution block graphics. (Each block-graphic character displayed as a 2x3 grid of pixels, with each block pixel effectively controlled by one of the lower 6 bits.) IBM introduced eight-bit extended ASCII codes on the original IBM PC and later produced variations for different languages and cultures. IBM called such character sets "code pages" and assigned numbers to both those they themselves invented as well as many invented and used by other manufacturers. Accordingly, character sets are very often indicated by their IBM code page number. In ASCII-compatible code pages, the lower 128 characters maintained their standard US-ASCII
Extended ASCII
4665651
each possible 32-bit value. Because five radix-84 digits only provide 84 = 4,182,119,424 representable values, 85 is the minimum possible integral base that will represent four bytes in five characters, hence its choice. When encoding, each group of 4 bytes is taken as a 32-bit binary number, most significant byte first (Ascii85 uses a big-endian convention). This is converted, by repeatedly dividing by 85 and taking the remainder, into 5 radix-85 digits. Then each digit (again, most significant first) is encoded as an ASCII printable character by adding 33 to it, giving the ASCII characters 33 ("codice_1") through 117 ("codice_2").
Ascii85
6571439
characters. For example, the capital letter "A" is ASCII character 65, the numeral "2" is ASCII 50, the character "<nowiki>}</nowiki>" is ASCII 125, and the metacharacter "carriage return" is ASCII 13. Systems based on ASCII use seven bits to represent these values digitally. In contrast, most computers store data in memory organized in eight-bit bytes. Files that contain machine-executable code and non-textual data typically contain all 256 possible eight-bit byte values. Many computer programs came to rely on this distinction between seven-bit "text" and eight-bit "binary" data, and would not function properly if non-ASCII characters appeared in data that was
Binary-to-text encoding
4568968
Telepen Telepen is a name of a barcode symbology designed in 1972 in the UK to express all 128 ASCII characters without using shift characters for code switching, and using only two different widths for bars and spaces. (Unlike Code 128, which uses shifts and four different element widths.) Unlike most linear barcodes, Telepen does not define independent encodings for each character, but instead operates on a stream of bits. It is able to represent any bit stream containing an even number of 0 bits, and is applied to ASCII bytes with even parity, which satisfy that rule. Bytes are
Telepen
2921083
by converting them into printable ASCII characters, because the range of printable ASCII characters is supported by most operating systems. However, since this reduces the available character set considerably, there is significant overhead (wasted bandwidth) over 8bit-byte networks. For example, in uuencode and Base64, three bytes of data are encoded into four printable ASCII characters, which equals four bytes, a 33% overhead (not including the overhead from headers). yEnc uses one character (one byte) to represent one byte of the file, with a few exceptions. yEnc assumes that binary data mostly can be transmitted through Usenet and email. Therefore, 252
YEnc
5037457
results in shorter URLs. The procedure for percent-encoding binary data has often been extrapolated, sometimes inappropriately or without being fully specified, to apply to character-based data. In the World Wide Web's formative years, when dealing with data characters in the ASCII repertoire and using their corresponding bytes in ASCII as the basis for determining percent-encoded sequences, this practice was relatively harmless; it was just assumed that characters and bytes mapped one-to-one and were interchangeable. The need to represent characters outside the ASCII range, however, grew quickly and URI schemes and protocols often failed to provide standard rules for preparing character
Percent-encoding
2805781
ASCII. However, these encodings often use byte values outside the ASCII range so they need to be encoded further before they are suitable for use in a non-8-bit-clean environment. Quoted-Printable encoding is one method used for mapping arbitrary bytes into sequences of ASCII characters. So, Quoted-Printable is not a character encoding scheme itself, but a data coding layer to be used under some byte-oriented character encoding. QP encoding is reversible, meaning the original bytes and hence the non-ASCII characters they represent can be identically recovered. Quoted-Printable and Base64 are the two basic MIME content transfer encodings, if a trivial "8bit"
Quoted-printable
2255816
cleanly divisible. These bytes are subtracted from the line's <length character> so that the decoder does not append unwanted characters to the file. uudecoding is reverse of the above, subtract 32 from each character's ASCII code to get a 6-bit value, concatenate 4 6-bit groups to get 24 bits, then output 3 bytes. The encoding process is demonstrated by this table, which shows the derivation of the above encoding for "Cat". The following table shows the conversion of the decimal value of the 6-bit fields obtained during the conversion process and their corresponding ASCII character output code and character. Note
Uuencoding
433557
UTF-16, and UTF-32, and several other encodings are in use. The most commonly used encodings are UTF-8, UTF-16, and UCS-2, a precursor of UTF-16. UTF-8, the dominant encoding on the World Wide Web (used in over 92% of websites), uses one byte for the first 128 code points, and up to 4 bytes for other characters. The first 128 Unicode code points are the ASCII characters, which means that any ASCII text is also a UTF-8 text. UCS-2 uses two bytes (16 bits) for each character but can only encode the first 65,536 code points, the so-called Basic Multilingual Plane
Unicode
18804
ASCII art ASCII art is a graphic design technique that uses computers for presentation and consists of pictures pieced together from the 95 printable (from a total of 128) characters defined by the ASCII Standard from 1963 and ASCII compliant character sets with proprietary extended characters (beyond the 128 characters of standard 7-bit ASCII). The term is also loosely used to refer to text based visual art in general. ASCII art can be created with any text editor, and is often used with free-form languages. Most examples of ASCII art require a fixed-width font (non-proportional fonts, as on a traditional
ASCII art
4665649
Ascii85 Ascii85, also called Base85, is a form of binary-to-text encoding developed by Paul E. Rutter for the btoa utility. By using five ASCII characters to represent four bytes of binary data (making the encoded size ¹⁄₄ larger than the original, assuming eight bits per ASCII character), it is more efficient than uuencode or Base64, which use four characters to represent three bytes of data (¹⁄₃ increase, assuming eight bits per ASCII character). Its main modern uses are in Adobe's PostScript and Portable Document Format file formats, as well as in the patch encoding for binary files used by Git.
Ascii85
2130981
by the decoder and decoding resumes with the next character. Otherwise decoding resumes with the character after the base64. First, an encoder must decide which characters to represent directly in ASCII form, which codice_3 has to be escaped as codice_4, and which to place in blocks of Unicode characters. A simple encoder may encode all characters it considers safe for direct encoding directly. However the cost of ending a Unicode sequence, outputing a single character directly in ASCII and then starting another Unicode sequence is 3 to 3⅔ bytes. This is more than the 2⅔ bytes needed to represent the
UTF-7
5037458
data for inclusion in a URI. Web applications consequently began using different multi-byte, stateful, and other non-ASCII-compatible encodings as the basis for percent-encoding, leading to ambiguities and difficulty interpreting URIs reliably. For example, many URI schemes and protocols based on RFCs 1738 and 2396 presume that the data characters will be converted to bytes according to some unspecified character encoding before being represented in a URI by unreserved characters or percent-encoded bytes. If the scheme does not allow the URI to provide a hint as to what encoding was used, or if the encoding conflicts with the use of ASCII
Percent-encoding
2716358
"invoked" to interpret bytes in the stream. Character codes from the 7-bit ASCII graphic range (0x20–0x7F), being on the left side of a character code table, are referred to as "GL" codes "(with "GL" standing for "graphics left")" while codes from the "high ASCII" range (0xA0–0xFF), if available, are referred to as the "GR" codes "("graphics right")". By default, GL codes specify G0 characters, and GR codes specify G1 characters, but this may be modified with control codes or by prior agreement: Each of the four working sets may be a 94-character set or a 94-character set. Additionally, G1 through
ISO/IEC 2022
2130975
of headers using byte values above the ASCII range. Although MIME allows encoding the message body in various character sets (broader than ASCII), the underlying transmission infrastructure (SMTP, the main E-mail transfer standard) is still not guaranteed to be 8-bit clean. Therefore, a non-trivial content transfer encoding has to be applied in case of doubt. Unfortunately base64 has a disadvantage of making even US-ASCII characters unreadable in non-MIME clients. On the other hand, UTF-8 combined with quoted-printable produces a very size-inefficient format requiring 6–9 bytes for non-ASCII characters from the BMP and 12 bytes for characters outside the BMP. Provided
UTF-7
437531
to ASCII. Characters outside the ASCII range are stored in 2-4 bytes. It is also possible to use UTF-16 where most characters are stored as two bytes with varying endianness, which is supported by modern browsers but less commonly used. In order to work around the limitations of legacy encodings, HTML is designed such that it is possible to represent characters from the whole of Unicode inside an HTML document by using a numeric character reference: a sequence of characters that explicitly spell out the Unicode code point of the character being represented. A character reference takes the form codice_2Ncodice_3,
Unicode and HTML
2858671
page 10008 or codice_1). EUC-JP is a variable-width encoding used to represent the elements of three Japanese character set standards, namely JIS X 0208, JIS X 0212, and JIS X 0201. 0.1% of all web pages use EUC-JP since August 2018. Other names for this encoding include Unixized JIS (or UJIS) and AT&T JIS. It is called Code page 954 by IBM. This encoding scheme allows the easy mixing of 7-bit ASCII and 8-bit Japanese without the need for the escape characters employed by ISO-2022-JP, which is based on the same character set standards, and without ASCII bytes appearing as
Extended Unix Code
18841
done by sampling the image down to grayscale with less than 8-bit precision, so that each value corresponds to different ASCII character. ASCII art ASCII art is a graphic design technique that uses computers for presentation and consists of pictures pieced together from the 95 printable (from a total of 128) characters defined by the ASCII Standard from 1963 and ASCII compliant character sets with proprietary extended characters (beyond the 128 characters of standard 7-bit ASCII). The term is also loosely used to refer to text based visual art in general. ASCII art can be created with any text editor,
ASCII art
2858669
in newspaper typesetting. GBK is an extension to GB2312. It defines an extended form of the EUC-CN encoding capable of representing a larger array of CJK characters sourced largely from Unicode 1.1, including traditional Chinese characters and characters used only in Japanese. It is not, however, a true EUC code, because ASCII bytes may appear as trail bytes (and C1 bytes, not limited to the single shifts, may appear as lead or trail bytes), due to a larger encoding space being required. Variants of GBK are implemented by Windows code page 936 (the Microsoft Windows code page for simplified Chinese),
Extended Unix Code
12361113
and therefore many human languages to be supported with little extra programming effort in the software that interprets the computer-readable language files. The principle of Extended ASCII means that: Extended ASCII Extended ASCII (EASCII or high ASCII) character encodings are eight-bit or larger encodings that include the standard seven-bit ASCII characters, plus additional characters. The use of the term is sometimes criticized, because it can be mistakenly interpreted to mean that the ASCII standard has been updated to include more than 128 characters or that the term unambiguously identifies a single encoding, neither of which is the case. There are
Extended ASCII
16007
to use an 8-bit byte to store each character in memory, providing an opportunity for extended, 8-bit relatives of ASCII. In most cases these developed as true extensions of ASCII, leaving the original character-mapping intact, but adding additional character definitions after the first 128 (i.e., 7-bit) characters. Encodings include ISCII (India), VISCII (Vietnam). Although these encodings are sometimes referred to as ASCII, true ASCII is defined strictly only by the ANSI standard. Most early home computer systems developed their own 8-bit character sets containing line-drawing and game glyphs, and often filled in some or all of the control characters from
ASCII
12361102
extensions of ASCII appeared on non-EBCDIC mainframe computers and minicomputers, especially in universities. Hewlett-Packard started to add European characters to their extended 7-bit / 8-bit ASCII character set HP Roman Extension around 1978/1979 for use with their workstations, terminals and printers. This later evolved into the widely used regular 8-bit character sets HP Roman-8 and HP Roman-9 (as well as a number of variants). Atari and Commodore home computers added many graphic symbols to their non-standard ASCII (Respectively, ATASCII and PETSCII, based on the original ASCII standard of 1963). The TRS-80 home computer additions included 64 semigraphics characters (0x80 through
Extended ASCII
12361100
comparing, inserting, or deleting characters.) Many manufacturers devised 8-bit character sets consisting of ASCII plus up to 128 of the unused codes, with varying degrees of cooperation or coordination by national and international standards bodies. 128 additional characters is not enough to cover all purposes, all languages, or even all European languages, so the emergence of "many" proprietary and national ASCII-derived 8-bit character sets was inevitable. Drawbacks of multiple 8-bit character sets are that conversion of text data from one character encoding to another (transcoding) is complex, and almost always has the possibility of changing non-ASCII characters in the text.
Extended ASCII
7103431
Alt code On IBM compatible personal computers, many characters not directly associated with a key can be entered using the Alt Numpad input method or Alt code: pressing and holding the "Alt" key while typing the number identifying the character with the keyboard's numeric keypad. Similar or extended forms of this feature are also available in many operating systems, including DOS and Microsoft Windows. Often this is the best or only method many computer users know for entering non-ASCII characters. Many Wikipedia articles on various characters will include how to type that character using Alt codes for code page 437.
Alt code
2741892
VISCII VISCII is a character encoding for using the Vietnamese language with computers. Unicode and the Windows-1258 code page are now used for virtually all Vietnamese computer data, but legacy VSCII and VISCII files may need conversion. VISCII is a modified ASCII. It keeps the 95 printable characters of ASCII unmodified, but it replaces 6 of the 33 control characters with printable characters. It adds 128 precomposed characters. A traditional extended ASCII character set consists of the ASCII set plus up to 128 characters. Vietnamese requires 134 additional letter-diacritic combinations, which is six too many. There are essentially three different
VISCII
2255814
by adding 32 to the actual byte count, with the sole exception of a grave accent "`" (ASCII code 96) signifying zero bytes. All data lines except the last (if the data was not divisible by 45), have 45 bytes of encoded data (60 characters after encoding). Therefore, the vast majority of length values is 'M', (32 + 45 = ASCII code 77 or "M"). <formatted characters> are encoded characters. See Formatting Mechanism for more details on the actual implementation. The file ends with two lines: The second to last line is also a character indicating the line length with
Uuencoding
4665661
highest possible representable address, 2−1 = 74×85 + 53×85 + 5×85 + ..., would be encoded as codice_19. This character set excludes the characters codice_20, making it suitable for use in JSON strings (where codice_21 and codice_22 would require escaping). However, for SGML-based protocols, notably including XML, string escapes may still be required (to accommodate codice_23, codice_24 and codice_25). Ascii85 Ascii85, also called Base85, is a form of binary-to-text encoding developed by Paul E. Rutter for the btoa utility. By using five ASCII characters to represent four bytes of binary data (making the encoded size ¹⁄₄ larger than the original,
Ascii85
7165401
bytes per code point. Byte-oriented protocol Byte-oriented framing protocol is "a communications protocol in which full bytes are used as control codes. Also known as character-oriented protocol." For example UART communication is byte-oriented. The term "character-oriented" is deprecated, since the notion of character has changed. An ASCII character fits to one byte (octet) in terms of the amount of information. With the internationalization of computer software, wide characters became necessary, to handle texts in different languages. In particular, Unicode characters (or strictly speaking code points) can be 1, 2, 3 or 4 bytes in UTF-8, and other encodings of Unicode
Byte-oriented protocol
960810
high byte), the end address (in the same format), the actual data, one byte with the value 0x2F ("/" character ), a two-byte checksum, and two bytes with the value 0x04 (EOT, End Of Transmission). Each byte of memory is stored as two sequential ASCII characters on tape, for example, hexadecimal B5 in memory (181 decimal) would be stored as two sequential ASCII characters "B" and "5" (42 and 35 hexadecimal). KIM-1 The KIM-1, short for "Keyboard Input Monitor", is a small 6502-based single-board computer developed and produced by MOS Technology, Inc. and launched in 1976. It was very successful
KIM-1
410687
of one, two, or four bytes, or as a variable-length sequence of one to four bytes, in accordance to specific character encoding conventions, such as ASCII, ISO/IEC 2022, UTF-8, or Unicode. These conventions define many printable characters, but also non-printing characters that control the flow of the text, such space, line break, and page break. Plain text contains no other information about the text itself, not even the character encoding convention employed. Plain text is stored in text files, although text files do not exclusively store plain text. In the early days of computers, plain text was displayed using a
Text editor
706950
8-bit clean 8-bit clean describes a computer system that correctly handles 8-bit character encodings, such as the ISO 8859 series and the UTF-8 encoding of Unicode. Up to the early 1990s, many programs and data transmission channels assumed that all characters would be represented as numbers between 0 and 127 (7 bits); for example, the ASCII standard used only 7 bits per character, avoiding an 8-bit representation in order to save on data transmission costs. On computers and data links using 8-bit bytes this left the top bit of each byte free for use as a parity, flag bit, or
8-bit clean
20879478
as they are mapped to ASCII commands codice_8, codice_9, codice_10, codice_11, codice_12, codice_13 and codice_14, respectively. Each character is encoded as a single seven-bit code value. It contains all 96 printable ASCII characters along with 27 mathematical symbols and 6 Greek letters. Stanford/ITS character set Stanford/ITS character set is an extended ASCII character set based on SEASCII with modifications allowing compatibility with 1968 ASCII. It is used as an alternate character set of the SUPDUP protocol for terminals with codice_1 and codice_2 bits set. It is also recommended for TeX implementations on systems with large character sets. The default plain
Stanford/ITS character set
2712193
unencoded parts in UCS. GB 18030 inherits the bad aspects of GBK, most notably needing special code to safely find ASCII characters in a GB18030 sequence. The one- and two-byte code points are essentially GBK with the euro sign, PUA mappings for unassigned/user-defined points, and vertical punctuations. The four byte scheme can be thought of as consisting of two units, each of two bytes. Each unit has a similar format to a GBK two byte character but with a range of values for the second byte of 0x30–0x39 (the ASCII codes for decimal digits). The first byte has the range
GB 18030
5881666
a multi-byte encoding like UTF-8; a single Unicode code point can be encoded in one, two, three, or five bytes. The ASCII range is encoded as one byte (all code points from U+0000 to U+009F are). UTF-1 does not use the C0 and C1 control codes or the space character in multi-byte encodings, the bytes 0 - 0x20 or 0x7F - 0x9F always stand for the corresponding code point. This design with 66 "protected" characters tried to be ISO 2022 compatible. UTF-1 uses "modulo 190" arithmetic (256 − 66 = 190). For comparison, UTF-8 protects all 128 ASCII characters and
UTF-1
2557454
characters was read from memory at a specified location. This location was one of the VIC-II registers, which allowed programmers to construct their own characters sets by placing the appropriate data in memory; each character was an 8x8 grid, a byte represented 8 bits horizontally, so 8 bytes were required for a single character and thus the complete 256-character set used a total of 2,048 bytes. There was no limit to how many such sets could be constructed, although they had to be aligned on a 2K boundary within the 16K video bank. In addition to charsets, the VIC-II also
MOS Technology VIC-II
1712530
strings encoded in UTF-16 use two bytes to encode each character (or four bytes for some special characters). When an ASCII string is transformed into UTF-16, a zero byte is inserted after each byte in the original string. Obscou proved in Phrack 61 that it is possible to write shellcode that can run successfully after this transformation. Programs that can automatically encode any shellcode into alphanumeric UTF-16-proof shellcode exist, based on the same principle of a small self-modifying decoder that decodes the original shellcode. Most shellcode is written in machine code because of the low level at which the vulnerability
Shellcode
440593
UTF-8 UTF-8 is a variable width character encoding capable of encoding all 1,112,064 valid code points in Unicode using one to four 8-bit bytes. The encoding is defined by the Unicode Standard, and was originally designed by Ken Thompson and Rob Pike. The name is derived from "Unicode" (or "Universal Coded Character Set") "Transformation Format 8-bit". It was designed for backward compatibility with ASCII. Code points with lower numerical values, which tend to occur more frequently, are encoded using fewer bytes. The first 128 characters of Unicode, which correspond one-to-one with ASCII, are encoded using a single octet with the
UTF-8
10583353
other variable-width character sets that used more bytes per character. On the other hand, early Japanese computing used half-width kana characters instead of normal-sized kana in a single-byte code page called JIS X 0201. Some IBM code pages used a similar treatment for Korean jamo. Some terminals and editing programs could not deal with Han characters starting at odd columns, only even ones. Therefore if ASCII or digits were to be inserted between the Han characters, a fullwidth alternative was desirable. So the DBCS encodings usually included a fullwidth versions of all ASCII characters. In Unicode, if a certain grapheme
Halfwidth and fullwidth forms
5388404
Code point In character encoding terminology, a code point or code position is any of the numerical values that make up the "code space". Many code points represent single characters but they can also have other meanings, such as for formatting. For example, the character encoding scheme ASCII comprises 128 code points in the range 0 to 7F, Extended ASCII comprises 256 code points in the range 0 to FF, and Unicode comprises 1,114,112 code points in the range 0 to 10FFFF. The Unicode code space is divided into seventeen planes (the basic multilingual plane, and 16 supplementary planes), each
Code point
18816
"break animations" by the Atari sceners. The Commodore 64, which was released in 1982, also did not follow the ASCII standard. The C-64 character set is called PETSCII, an extended form of ASCII-1963. As with the Atari's ATASCII art, C-64 fans developed a similar scene that used PETSCII for their creations. So-called "block ASCII" or "high ASCII" uses the extended characters of the 8-bit code page 437, which is a proprietary standard introduced by IBM in 1979 (ANSI Standard x3.16) for the IBM PC DOS and MS-DOS operating systems. "Block ASCIIs" were widely used on the PC during the 1990s
ASCII art
3628870
A hex editor or viewer may be used to view file data as a sequence of hexadecimal (or decimal, binary or ASCII character) values for corresponding bytes of a binary file. If a binary file is opened in a text editor, each group of eight bits will typically be translated as a single character, and the user will see a (probably unintelligible) display of textual characters. If the file is opened in some other application, that application will have its own use for each byte: maybe the application will treat each byte as a number and output a stream of
Binary file
7165400
Byte-oriented protocol Byte-oriented framing protocol is "a communications protocol in which full bytes are used as control codes. Also known as character-oriented protocol." For example UART communication is byte-oriented. The term "character-oriented" is deprecated, since the notion of character has changed. An ASCII character fits to one byte (octet) in terms of the amount of information. With the internationalization of computer software, wide characters became necessary, to handle texts in different languages. In particular, Unicode characters (or strictly speaking code points) can be 1, 2, 3 or 4 bytes in UTF-8, and other encodings of Unicode use two or four
Byte-oriented protocol
1340058
all the files in an archive are ASCII text files, and have ASCII names, then the archive is essentially an ASCII text file (containing many NUL characters). The fields defined by the original Unix tar format are listed in the table below. The link indicator/file type table includes some modern extensions. When a field is unused it is filled with NUL bytes. The header uses 257 bytes, then is padded with NUL bytes to make it fill a 512 byte record. There is no "magic number" in the header, for file identification. Pre-POSIX.1-1988 (i.e. v7) tar header: The pre-POSIX.1-1988 Link
Tar (computing)
1349454
of DBCS characters and single-byte characters in Big5-encoded text. Bytes in the range 0x00 to 0x7f that are not part of a double-byte character are assumed to be single-byte characters. The meaning of non-ASCII single bytes outside the permitted values that are not part of a double-byte character varies from system to system. In old MSDOS-based systems, they are likely to be displayed as 8-bit characters; in modern systems, they are likely to either give unpredictable results or generate an error. In the original Big5, the encoding is compartmentalized into different zones: The "graphical characters" actually comprise punctuation marks, partial
Big5
706952
8-bit message had high chances to be garbled during transmission in the 20th century. But some implementations really did not care about formal discouraging of 8-bit data and allowed high bit set bytes to pass through. Many early communications protocol standards, such as , , for SMTP, for NNTP, , , , were designed to work over such "7-bit" communication links. They specifically mention the use of ASCII character set "transmitted as an 8-bit byte with the high-order bit cleared to zero" and some of these explicitly restrict "all" data to 7-bit characters. For the first few decades of email
8-bit clean
2858667
applications are more likely to use UTF-8, which supports all of the glyphs of the EUC codes, and more, and is generally more portable with fewer vendor deviations and errors. EUC-CN is the usual way to use the GB2312 standard for simplified Chinese characters. Unlike the case of Japanese, the ISO-2022 form of GB2312 is not normally used, though a variant form called HZ was sometimes used on USENET. An ASCII character is represented in its usual encoding. A character from GB 2312 is represented by two bytes in the range 0xA1 – 0xFE. An encoding related to EUC-CN is
Extended Unix Code
9595076
character data. Six-bit BCD was used by IBM on early computers such as the IBM 704 in 1954. This encoding was replaced by the 8-bit EBCDIC code when System/360 standardized on 8-bit bytes. There are some variants of this type of code (see below). Six-bit character codes generally succeeded the five-bit Baudot code and preceded seven-bit ASCII. One popular variant was DEC SIXBIT. This is simply the ASCII character codes from 32 to 95 coded as 0 to 63 by subtracting 32 (i.e., columns 2, 3, 4, and 5 of the ASCII table (16 characters to a column), shifted to
Six-bit character code
1349453
codes are frequently given as a 4-digit hexadecimal number, which describes the two bytes that comprise the Big5 code as if the two bytes were a big endian representation of a 16-bit number. For example, the Big5 code for a full-width space, which are the bytes 0xa1 0x40, is usually written as 0xa140 or just A140. Strictly speaking, the Big5 encoding contains only DBCS characters. However, in practice, the Big5 codes are always used together with an unspecified, system-dependent single-byte character set (ASCII, or an 8-bit character set such as code page 437), so that you will find a mix
Big5
89353
operating systems, files are organized into one-dimensional arrays of bytes. The format of a file is defined by its content since a file is solely a container for data, although, on some platforms the format is usually indicated by its filename extension, specifying the rules for how the bytes must be organized and interpreted meaningfully. For example, the bytes of a plain text file (.txt in Windows) are associated with either ASCII or UTF-8 characters, while the bytes of image, video, and audio files are interpreted otherwise. Most file types also allocate a few bytes for metadata, which allows a
Computer file
2570037
Hangul does not contain as many characters, but KS X 1001 supports both Hangul and Hanja, and uses two bytes per character. The term "DBCS" traditionally refers to a character encoding where each graphic character is encoded in two bytes. The DBCS always has "lead bytes" with the most significant bit set (i.e., being greater than 7 bits), and is always paired up with a single-byte character-set (SBCS). Furthermore, for the practical reason of maintaining compatibility with unmodified, off-the-shelf software, the SBCS is associated with half-width characters and the DBCS with full-width characters. Sometimes, the use of the term "DBCS"
DBCS
12361107
unused range. Code page 1252 is the standard character encoding of western European language versions of Microsoft Windows, including English versions. ISO 8859-1 is the common 8-bit character encoding used by the X Window System, and most Internet standards used it before Unicode. The meaning of each extended code point can be different in every encoding. In order to correctly interpret and display text data (sequences of characters) that includes extended codes, hardware and software that reads or receives the text must use the "specific" extended ASCII encoding that applies to it. Applying the wrong encoding causes irrational substitution of
Extended ASCII
9237217
used a minimum of two ASCII "SYN" characters (0x16…0x16) to achieve character synchronization in an undifferentiated bit stream, then other special characters to synchronize to the beginning of a frame of characters. The syncwords can be seen as a kind of delimiter. Various techniques are used to avoid delimiter collision, orin other wordsto "disguise" bytes of data at the data link layer that might otherwise be incorrectly recognized as the syncword. For example, HDLC uses bit stuffing or "octet stuffing", while other systems use ASCII armor or Consistent Overhead Byte Stuffing (COBS). In some communication systems, a receiver can achieve
Syncword
2192621
does not always result in an intuitive API. Null-terminated strings require that the encoding does not use a zero byte (0x00) anywhere, therefore it is not possible to store every possible ASCII or UTF-8 string. However, it is common to store the subset of ASCII or UTF-8 – every character except the NUL character – in null-terminated strings. Some systems use "modified UTF-8" which encodes the NUL character as two non-zero bytes (0xC0, 0x80) and thus allow all possible strings to be stored. This is not allowed by the UTF-8 standard as it is a security risk. A 0xC0, 0x80
Null-terminated string
19391735
Code page 932 (Microsoft Windows) Microsoft Windows code page 932 (abbreviated MS932, Windows-932 or ambiguously CP932), also called Windows-31J amongst other names (see § Terminology below), is the Microsoft Windows code page for the Japanese language, which is an extended variant of the Shift JIS Japanese character encoding. It contains standard 7-bit ASCII codes, and Japanese characters are indicated by the high bit of the first byte being set to 1. Some code points in this page require a second byte, so characters use either 8 or 16 bits for encoding. IBM offer the same extended double-byte codes in their
Code page 932 (Microsoft Windows)
514936
of encoding characters of BMP. It is supported by many programs. Early in this process it became increasingly clear that 2 characters would not suffice, and IEEE introduced a larger 31-bit space and an encoding (UCS-4) that would require 4 bytes per character. This was resisted by the Unicode Consortium, both because 4 bytes per character wasted a lot of disk space and memory, and because some manufacturers were already heavily invested in 2-byte-per-character technology. The UTF-16 encoding scheme was developed as a compromise to resolve this impasse in version 2.0 of the Unicode standard in July 1996 and is
UTF-16
4665650
The basic need for a binary-to-text encoding comes from a need to communicate arbitrary binary data over preexisting communications protocols that were designed to carry only English language human-readable text. Those communication protocols may only be 7-bit safe (and within that avoid certain ASCII control codes), and may require line breaks at certain maximum intervals, and may not maintain whitespace. Thus, only the 95 printable ASCII characters are "safe" to use to convey data. Four bytes can represent 2 = 4,294,967,296 possible values. Five radix-85 digits provide 85 = 4,437,053,125 possible values, enough to provide for a unique representation for
Ascii85
2446541
of a set of 63 graphic characters intended for use with ASCII, the American National Standard Code for Information Interchange, ANSI X3.4-1986, including 29 combining diacritic characters. A combining diacritic character precedes the spacing character on which it should be superimposed. The following table shows ANSI/NISO Z39.47-1993 (R2003). Each character is shown with its Unicode equivalent. The GEDCOM specification for exchanging genealogical data refers to ANSEL (ANSI/NISO Z39.47-1985) as a valid text encoding for GEDCOM files and extends it with additional characters which are shown in the following table. The MARC 21 Extended Latin character set is synchronized with ANSEL
ANSEL
10146552
Unicode control characters Many Unicode control characters are used to control the interpretation or display of text, but these characters themselves have no visual or spatial representation. For example, the null character () is used in C-programming application environments to indicate the end of a string of characters. In this way, these programs only require a single starting memory address for a string (as opposed to a starting address and a length), since the string ends once the program reads the null character. The control characters U+0000–U+001F and U+007F come from ASCII. Additionally, U+0080–U+009F were used in conjunction with ISO
Unicode control characters
375741
of representing strings as lists of character codes. Representations of strings depend heavily on the choice of character repertoire and the method of character encoding. Older string implementations were designed to work with repertoire and encoding defined by ASCII, or more recent extensions like the ISO 8859 series. Modern implementations often use the extensive repertoire defined by Unicode along with a variety of complex encodings such as UTF-8 and UTF-16. The term "byte string" usually indicates a general-purpose string of bytes, rather than strings of only (readable) characters, strings of bits, or such. Byte strings often imply that bytes can
String (computer science)
8038149
selected character codes are encapsulated, which reduces the performance by about 0.4% + additional 0.4% for every selected character code. In case the data link is limited to seven data bits, a special bit collection mode can be activated. The highest bits for every seven bytes are collected to one byte (which is eight bits under normal conditions). This collection reduces the performance about 12.4% and doubles the character encapsulation capacity. The maximum transfer line correction mode of MSLP uses five characters to transmit four characters that use only printable 7-bit ASCII codes (Ascii85). This mode should work on almost
SMODEM
2130979
direct characters are safe to include literally. The other main group, known as "optional direct characters", contains all other printable characters in the range –U+007E except codice_2 and space. Using the optional direct characters reduces size and enhances human readability but also increases the chance of breakage by things like badly designed mail gateways and may require extra escaping when used in encoded words for header fields. Space, tab, carriage return and line feed may also be represented directly as single ASCII bytes. However, if the encoded text is to be used in e-mail, care is needed to ensure that
UTF-7
4754104
The C40 and Text modes have four separate sets. Set 0 is the default, and contains codes that temporarily select a different set for the next character. The only difference is that they reverse upper-and lower-case letters. C40 is primarily upper-case, with lower-case letters in Set 3. Text is the other way around. Set 1, containing ASCII control codes, and set 2, containing punctuation symbols are identical in C40 and Text mode. EDIFACT mode uses six bits per character, with four characters packed into three bytes. It can store digits, upper-case letters, and many punctuation marks, but has no support
Data Matrix
202676
start operation at the "current" buffer address, and executing an order or writing data will update this address. The buffer address can be set directly using the "Set Buffer Address (SBA)" order, often followed by "Start Field". For a device with a 1920 character display a twelve bit address is sufficient. Later 3270s with larger screen sizes use fourteen or sixteen bits. Addresses are encoded in orders in two bytes. For twelve bit addresses the high order two bits of each byte are normally set to form valid EBCDIC (or ASCII) characters. For example, address 0 is coded as X'4040',
IBM 3270
2858517
Shift JIS Shift JIS (Shift Japanese Industrial Standards, also SJIS, MIME name Shift_JIS) is a character encoding for the Japanese language, originally developed by a Japanese company called ASCII Corporation in conjunction with Microsoft and standardized as JIS X 0208 Appendix 1. 0.4% of all web pages used Shift JIS in September 2018, a decline from 1.3% in July 2014. Shift JIS is based on character sets defined within JIS standards JIS X 0201:1997 (for the single-byte characters) and JIS X 0208:1997 (for the double-byte characters). The lead bytes for the double-byte characters are "shifted" around the 64 halfwidth katakana
Shift JIS
12361105
ASCII codes in Mac OS, such as Mac OS Roman. Digital Equipment Corporation (DEC) developed the Multinational Character Set, which had fewer characters but more letter and diacritic combinations. It was supported by the VT220 and later DEC computer terminals. This later became the basis for other character sets such as the Lotus International Character Set (LICS), ECMA-94 and ISO 8859-1. Eventually, ISO released this standard as ISO 8859 describing its own set of eight-bit ASCII extensions. The most popular is ISO 8859-1, also called "ISO Latin 1", which contained characters sufficient for the most common Western European languages. Variations
Extended ASCII
1698186
what characters were safe to use. For instance, uuencode uses uppercase letters, digits, and many punctuation characters, but no lowercase. The Base64 index table: The example below uses ASCII text for simplicity, but this is not a typical use case, as it can already be safely transferred across all systems that can handle Base64. The more typical use is to encode binary data (such as an image); the resulting Base64 data will only contain 64 different ASCII characters, all of which can reliably be transferred across systems that may corrupt the raw source bytes. A quote from Thomas Hobbes' "Leviathan":
Base64
41385
Some implementations, for example IBM mainframe systems, support zoned decimal numeric representations. Each decimal digit is stored in one byte, with the lower four bits encoding the digit in BCD form. The upper four bits, called the "zone" bits, are usually set to a fixed value so that the byte holds a character value corresponding to the digit. EBCDIC systems use a zone value of 1111 (hex F); this yields bytes in the range F0 to F9 (hex), which are the EBCDIC codes for the characters "0" through "9". Similarly, ASCII systems use a zone value of 0011 (hex 3),
Binary-coded decimal
2716354
before the end of a line. To represent large character sets, ISO/IEC 2022 builds on ISO/IEC 646's property that one seven bit character will normally define 94 graphic (printable) characters (in addition to space and 33 control characters). Using two bytes, it is thus possible to represent up to 8836 (94×94) characters; and, using three bytes, up to 830584 (94×94×94) characters. Though the standard defines it, no registered character set uses three bytes (although EUC-TW's unregistered G2 is). For the two-byte character sets, the code point of each character is normally specified in so-called "kuten" (Japanese: ) form (sometimes called
ISO/IEC 2022
19391803
Code page 949 (IBM) IBM code page 949 (IBM-949) is IBM's PC Data KS code. It implements EUC-KR in addition to encodings for IBM extensions including user defined characters. This code page supports the Korean language. It is a combination of the single-byte Code page 1088 and the double-byte Code page 951. Bytes 0x00 through 0x7F are used for single byte KS X 1003 (ISO 646:KR) characters, a similar set to ASCII. Bytes 0x80 through 0x84 are used for IBM single byte extension characters. Lead bytes 0x8F through 0xA0 are used for IBM double byte extension characters. Lead bytes 0xA1
Code page 949 (IBM)
1698190
two bytes. The example below illustrates how truncating the input of the above quote changes the output padding: The same characters will be encoded differently depending on their position within the three-octet group which is encoded to produce the four characters. For example: The ratio of output bytes to input bytes is 4:3 (33% overhead). Specifically, given an input of "n" bytes, the output will be formula_1 bytes long, including padding characters. In theory, the padding character is not needed for decoding, since the number of missing bytes can be calculated from the number of Base64 digits. In some implementations,
Base64
3897681
bytes to represent each character (primarily used in source code and external files), from "wide characters", which are run-time representations of characters in single objects (typically, greater than 8 bits). UTF-16 little-endian is the encoding standard at Microsoft (and in the Windows operating system). Yet with surrogate pairs it supports 32-bit as well. The .NET Framework platform supports multiple wide-character implementations including UTF7, UTF8, UTF16 and UTF32. The Java platform requires that wide character variables be defined as 16-bit values, and that characters be encoded using UTF-16 (due to former use of UCS-2), while modern Unix-like systems generally require UTF-8
Wide character
7910659
three bytes left. These 24 bits are split into four 6-bit numbers, each of which is then translated to the formula_1th character in the following table: Each group of sixty output characters (corresponding to 45 input bytes) is output as a separate line preceded by an encoded character giving the number of encoded bytes on that line. For all lines except the last, this will be the character 'h' (the character mapping to the value 45). If the input is not evenly divisible by 45, the last line will contain the remaining N output characters, preceded by the number of
Xxencoding
7127525
rather, blanks after the shebang have traditionally been allowed, and sometimes documented with a space (see the 1980 email in history section below). The shebang characters are represented by the same two bytes in extended ASCII encodings, including UTF-8, which is commonly used for scripts and other text files on current Unix-like systems. However, UTF-8 files may begin with the optional byte order mark (BOM); if the "exec" function specifically detects the bytes 0x23 and 0x21, then the presence of the BOM () before the shebang will prevent the script interpreter from being executed. Some authorities recommend against using the
Shebang (Unix)
12361097
points were modified by at least one modified set, leaving only 82 "invariant" codes. There was always temptation to make a larger character set (preferably by extending a standard character set), but there was no easy path forward. There was no way to define a universal extension. There was no hardware to print or display an extended set. For years, applications were designed around the 64-character set and/or the 95-character set, so several characters acquired new uses. (For example, ASCII lacks "÷", so most programming languages use "/" (and sometimes "\") to indicate division, and most still do not recognize
Extended ASCII
5775806
so should be used for estimation and comparison only. To accurately determine the size of text in an encoding, see the actual specifications. Endianness does not affect sizes (UTF-16BE and UTF-32BE have the same size as UTF-16LE and UTF-32LE, respectively). The use of UTF-32 under quoted-printable is highly impractical, but if implemented, will result in 8–12 bytes per code point (about 10 bytes in average), namely for BMP, each code point will occupy exactly 6 bytes more than the same code in quoted-printable/UTF-16. Base64/UTF-32 gets bytes for "any" code point. An ASCII control character under quoted-printable or UTF-7 may be
Comparison of Unicode encodings
2082252
Multinational Character Set The Multinational Character Set (DMCS or MCS) is a character encoding created in 1983 by Digital Equipment Corporation (DEC) for use in the popular VT220 terminal. It was an 8-bit extension of ASCII that added accented characters, currency symbols, and other character glyphs missing from 7-bit ASCII. It is only one of the code pages implemented for the VT220 National Replacement Character Set (NRCS). MCS is registered as IBM code page 1100 (Multinational Emulation) since 1992. Depending on associated sorting Oracle calls it WE8DEC, N8DEC, DK8DEC, S8DEC, or SF8DEC. Such "extended ASCII" sets were common (the National
Multinational Character Set
1992001
screen was limited to plain and simple text, such as that found in the ASCII code set. In the early 1980s, users of IBM PC compatible computers began to experiment with ways of forming simple pictures and designs using only the 255 characters within the Extended ASCII character set, specifically known as code page 437, created by IBM. Modems and networking technology allowed computer users to communicate with each other over bulletin board systems (BBSes); the operators of these BBSes used ASCII art to enhance the aesthetic appearance of their systems. The common user interface or video mode shared by
Computer art scene
5775793
by definition) and UTF-16. UTF-8 requires 8, 16, 24 or 32 bits (one to four octets (bytes) to encode a Unicode character, UTF-16 requires either 16 or 32 bits to encode a character, and UTF-32 always requires 32 bits to encode a character. The first 128 Unicode code points, U+0000 to U+007F, used for the C0 Controls and Basic Latin characters and which correspond one-to-one to their ASCII-code equivalents, are encoded using 8 bits in UTF-8, 16 bits in UTF-16, and 32 bits in UTF-32. The next 1,920 characters, U+0080 to U+07FF (encompassing the remainder of almost all Latin-script alphabets,
Comparison of Unicode encodings
3452210
within a word, and the size of each byte. Instructions existed to load and store bytes via this descriptor, and to increment the descriptor to point at the next byte (bytes were not split across word boundaries). Much DEC software used five 7-bit bytes per word (plain ASCII characters), with one bit per word unused. Implementations of C had to use four 9-bit bytes per word, since the 'malloc' function in C assumes that the size of an "int" is some multiple of the size of a "char"; the actual multiple is determined by the system-dependent compile-time operator sizeof. The
Addressing mode
20019950
Code for Information Interchange” (ASCII). While ASCII is a 7-bit code, BraSCII is an 8-bit code, where the characters from 160 to 255 were configured to support extended characters. It is nearly identical to ECMA-94 (1985) and ISO 8859-1 (1987) except that the characters × and ÷ are replaced by Œ and œ, as they still were in the Multinational Character Set (MCS, 1983) and Lotus International Character Set (LICS, 1985), whereas these code points were empty in the earliest versions of ECMA-94 (1985) and ISO 8859-1. In some other devices, this character set is simply referred as “ABNT”. This
BraSCII
15971
differ in bit pattern from the upper case by a single bit, which simplified case-insensitive character matching and the construction of keyboards and printers. The X3 committee made other changes, including other new characters (the brace and vertical bar characters), renaming some control characters (SOM became start of header (SOH)) and moving or removing others (RU was removed). ASCII was subsequently updated as USAS X3.4-1967, then USAS X3.4-1968, ANSI X3.4-1977, and finally, ANSI X3.4-1986. Revisions of the ASCII standard: In the X3.15 standard, the X3 committee also addressed how ASCII should be transmitted (least significant bit first), and how it
ASCII
15972
should be recorded on perforated tape. They proposed a 9-track standard for magnetic tape, and attempted to deal with some punched card formats. The X3.2 subcommittee designed ASCII based on the earlier teleprinter encoding systems. Like other character encodings, ASCII specifies a correspondence between digital bit patterns and character symbols (i.e. graphemes and control characters). This allows digital devices to communicate with each other and to process, store, and communicate character-oriented information such as written language. Before ASCII was developed, the encodings in use included 26 alphabetic characters, 10 numerical digits, and from 11 to 25 special graphic symbols. To
ASCII
2431658
G and T. Different definitions of an edit distance use different sets of string operations. The Levenshtein distance operations are the removal, insertion, or substitution of a character in the string. Being the most common metric, the Levenshtein distance is usually what is meant by "edit distance". Given two strings and on an alphabet (e.g. the set of ASCII characters, the set of bytes [0..255], etc.), the edit distance is the minimum-weight series of edit operations that transforms into . One of the simplest sets of edit operations is that defined by Levenshtein in 1966: In Levenshtein's original definition, each
Edit distance
63202
had no official assigned glyph, many computer equipment vendors used it as an additional printable character (often an all-black "box" character useful for erasing text by overprinting with ink). Non-erasable Programmable ROMs are typically implemented as arrays of fusible elements, each representing a bit, which can only be switched one way, usually from one to zero. In such PROMs, the DEL and NUL characters can be used in the same way that they were used on punched tape: one to reserve meaningless fill bytes that can be written later, and the other to convert written bytes to meaningless fill bytes.
Control character