text: ADC Home > Reference Library > Technical Notes > Carbon > File Management > Technical Note TN1150 HFS Plus Volume Format CONTENTS. [Mar 05, 2004] HFS Plus Basics: efficient use of disk space, international-friendly file names, future support for named forks, and ease booting on non-Mac OS operating systems. The following sections describes these goals, and the differences between HFS and HFS Plus required to meet these goals. Efficient Use of Disk Space. International-Friendly File Names). Future Support for Named Forks. Whenever possible, an application should delete named forks rather than orphan them. Easy Startup of Alternative. In many respects, the startup file is a generalization of the HFS boot blocks, one that provides a much larger, variable-sized amount of storage. Back to top Core Concepts HFS Plus uses a number of interrelated structures to manage the organization of data on the volume. These structures include: the volume header the catalog file the extents overflow file the attributes file the allocation file (bitmap) the startup file Each of these complex structures is described in its own section. The purpose of this section is to give an overview of the volume format, describe how the structures fit together, and define the primitive data types used by HFS Plus. Terminology. Broad Structure The bulk of an HFS Plus volume consists of seven types of information or areas: user file forks, the allocation file (bitmap), the catalog file, the extents overflow file, the attributes file, the startup file, and unused space.. Primitive Data Types). Reserved and Pad Fields In many places this specification describes a field, or bit within a field, as reserved. This has a definite meaning, namely: When creating a structure with a reserved field, an implementation must set the field to zero. When reading existing structures, an implementation must ignore any value in the field. When modifying a structure with a reserved field, an implementation must preserve the value of the reserved field. This definition allows for backward-compatible enhancements to the volume format. Pad fields have exactly the same semantics as a reserved field. The different name merely reflects the designer's goals when including the field, not the behavior of the implementation. Integer Types. HFS Plus Names File and folder names on HFS Plus consist of up to 255 Unicode characters with a preceding 16-bit length, defined by the type. Text Encodings. The valid values for the textEncoding field are defined in Table 2. Table 2 Text Encodings Encoding Name Value Encoding Name Value MacRoman Dates. HFS Plus Permissions For each file and folder, HFS Plus maintains a record containing access permissions, defined by the HFSPlusBSDInfo structure. The Mac OS X user ID of the owner of the file or folder. Mac OS X versions prior to 10.3 treats user ID 99 as if it was the user ID of the user currently logged in to the console. If no user is logged in to the console, user ID 99 is treated as user ID 0 (root). Mac OS X version 10.3 treats user ID 99 as if it was the user ID of the process making the call (in effect, making it owned by everyone simultaneously). These substitutions happen at run time. The actual user ID on disk is not changed. groupID The Mac OS X group ID of the group associated with the file or folder. Mac OS X typically maps group ID 99 to the group named "unknown." There is no run time substitution of group IDs in Mac OS X. adminFlags BSD flags settable by the super-user only. This field corresponds to bits 16 through 23 of the st_flags field of struct stat in Mac OS X. See the manual page for chflags(2) for more information. The following table gives the bit position in the adminFlags field and the name of the corresponding mask used in the st_flags field. Bit st_flags mask Meaning 0 SF_ARCHIVED File has been archived 1 SF_IMMUTABLE File may not be changed 2 SF_APPEND Writes to file may only append ownerFlags BSD flags settable by the owner of the file or directory, or by the super-user. This field corresponds to bits 0 through 7 of the st_flags field of struct stat in Mac OS X. See the manual page for chflags(2) for more information. The following table gives the bit position in the ownerFlags field and the name of the corresponding mask used in the st_flags field. Bit st_flags mask Meaning 0 UF_NODUMP Do not dump (back up or archive) this file 1 UF_IMMUTABLE File may not be changed 2 UF_APPEND Writes to file may only append 3 UF_OPAQUE Directory is opaque (see below) fileMode BSD file type and mode bits. Note that the constants from the header shown below are in octal (base eight), not hexadecimal. #define S_ISUID 0004000 /* set user id on execution */ #define S_ISGID 0002000 /* set group id on execution */ #define S_ISTXT 0001000 /* sticky bit */  */ In some versions of Unix, the sticky bit, S_ISTXT, is used to indicate that an executable file's code should remain in memory after the executable finishes; this can help performance if the same executable is used again soon. Mac OS X does not use this optimization. If the sticky bit is set for a directory, then Mac OS X restricts movement, deletion, and renaming of files in that directory. Files may be removed or renamed only if the user has write access to the directory; and is the owner of the file or the directory, or is the super-user. special This field is used only for certain special kinds of files. For directories, and most files, this field is unused and reserved. When used, this field is used as one of the following: iNodeNum For hard link files, this field contains the link reference number. See the Hard Links section for more information. linkCount For indirect node files, this field contains the number of hard links that point at this indirect node file. See the Hard Links section for more information. rawDevice For block and character special devices files (when the S_IFMT field contains S_IFCHR or S_IFBLK), this field contains the device number.. Fork Data Structure. An unused extent descriptor in an extent record would have both startBlock and blockCount set to zero. For example, if a given fork occupied three extents, then the last five extent descriptors would be all zeroes.]; The fields have the following meaning: logicalSize The size, in bytes, of the valid data in the fork. clumpSize For HFSPlusForkData structures in the volume header, this is the fork's clump size, which is used in preference to the default clump size in the volume header. For HFSPlusForkData structures in a catalog record, this field was intended to store a per-fork clump size to override the default clump size in the volume header. However, Apple implementations prior to Mac OS X version 10.3 ignored this field. As of Mac OS X version 10.3, this field is used to keep track of the number of blocks actually read from the fork. See the Hot Files section for more information. totalBlocks The total number of allocation blocks used by all the extents in this fork. extents An array of extent descriptors for the fork. This array holds the first eight extent descriptors. If more extent descriptors are required, they are stored in the extents overflow file. IMPORTANT: The HFSPlusExtentRecord is also the data record used in the extents overflow file (the extent record). The HFSPlusExtentDescriptor structure is used to hold information about a specific extent. struct HFSPlusExtentDescriptor { UInt32 startBlock; UInt32 blockCount; }; typedef struct HFSPlusExtentDescriptor HFSPlusExtentDescriptor; The fields have the following meaning: startBlock The first allocation block in the extent. blockCount The length, in allocation blocks, of the extent. Back to top Volume Header; The fields have the following meaning: signature The volume signature, which must be kHFSPlusSigWord ('H+') for an HFS Plus volume, or kHFSXSigWord ('HX') for an HFSX volume. version The version of the volume format, which is currently 4 (kHFSPlusVersion) for HFS Plus volumes, or 5 (kHFSXVersion) for HFSX volumes. attributes Volume attributes, as described below. lastMountedVersion A value which uniquely identifies the implementation that last mounted this volume for writing. This value can be used by future implementations to detect volumes that were last mounted by older implementations and check them for deficiencies. Any code which modifies the on disk structures must also set this field to a unique value which identifies that code. Third-party implementations of HFS Plus should place a registered creator code in this field. The value used by Mac OS 8.1 to 9.2.2 is '8.10'. The value used by Mac OS X is '10.0'. The value used by a journaled volume (including HFSX) in Mac OS X is 'HFSJ'. The value used by fsck_hfs on Mac OS X is  The allocation block number of the allocation block which contains the JournalInfoBlock for this volume's journal. This field is valid only if bit kHFSVolumeJournaledBit is set in the attribute field; otherwise, this field is reserved. createDate The date and time when the volume was created. See HFS Plus Dates for a description of the format. modifyDate The date and time when the volume was last modified. See HFS Plus Dates for a description of the format. backupDate The date and time when the volume was last backed up. The volume format requires no special action on this field; it simply defines the field for the benefit of user programs. See HFS Plus Dates for a description of the format. checkedDate The date and time when the volume was last checked for consistency. Disk checking tools, such as Disk First Aid, must set this when they perform a disk check. A disk checking tool may use this date to perform periodic checking of a volume. fileCount The total number of files on the volume. The fileCount field does not include the special files. It should equal the number of file records found in the catalog file. folderCount The total number of folders on the volume. ThefolderCount field does not include the root folder. It should equal the number of folder records in the catalog file, minus one (since the root folder has a folder record in the catalog file). blockSize The allocation block size, in bytes. totalBlocks The total number of allocation blocks on the disk. For a disk whose size is an even multiple of the allocation block size, all areas on the disk are included in an allocation block, including the volume header and alternate volume header. For a disk whose size is not an even multiple of the allocation block size, only the allocation blocks that will fit entirely on the disk are counted here. The remaining space at the end of the disk is not used by the volume format (except for storing the alternate volume header, as described above). freeBlocks The total number of unused allocation blocks on the disk. nextAllocation Start of next allocation search. The nextAllocation field is used by Mac OS as a hint for where to start searching for free allocation blocks when allocating space for a file. It contains the allocation block number where the search should begin. An implementation that doesn't want to use this kind of hint can just treat the field as reserved. [Implementation details: traditional Mac OS implementations typically set it to the first allocation block of the extent most recently allocated. It is not set to the allocation block immediately following the most recently allocated extent because of the likelihood of that extent being shortened when the file is closed (since a whole clump may have been allocated but not actually used).] See Allocation File section for details. rsrcClumpSize The default clump size for resource forks, in bytes. This is a hint to the implementation as to the size by which a growing file should be extended. All Apple implementations to date ignore the rsrcClumpSize and use dataClumpSize for both data and resource forks. dataClumpSize The default clump size for data forks, in bytes. This is a hint to the implementation as to the size by which a growing file should be extended. All Apple implementations to date ignore the rsrcClumpSize and use dataClumpSize for both data and resource forks. nextCatalogID The next unused catalog ID. See Catalog File for a description of catalog IDs. writeCount This field is incremented every time a volume is mounted. This allows an implementation to keep the volume mounted even when the media is ejected (or otherwise inaccessible). When the media is re-inserted, the implementation can check this field to determine when the media has been changed while it was ejected. It is very important that an implementation or utility change the writeCount field if it modifies the volume's structures directly. This is particularly important if it adds or deletes items on the volume. encodingsBitmap This field keeps track of the text encodings used in the file and folder names on the volume. This bitmap enables some performance optimizations for implementations that don't use Unicode names directly. See the Text Encoding sections for details. finderInfo This array of 32-bit items contains information used by the Mac OS Finder, and the system software boot process.[1] contains the parent directory ID of the startup application (for example, Finder), or zero if the volume is not bootable. finderInfo[2] contains the directory ID of a directory whose window should be displayed in the Finder when the volume is mounted, or zero if no directory window should be opened. In traditional Mac OS, this is the first in a linked list of windows to open; the frOpenChain field of the directory's Finder Info contains the next directory ID in the list. The open window list is deprecated. The Mac OS X Finder will open this directory's window, but ignores the rest of the open window list. The Mac OS X Finder does not modify this field. finderInfo[3] contains the directory ID of a bootable Mac OS 8 or 9 System Folder, or zero if there isn't one. finderInfo[4] is reserved. finderInfo[5] contains the directory ID of a bootable Mac OS X system (the /System/Library/CoreServices directory), or zero if there is no bootable Mac OS X system on the volume. finderInfo[6] and finderInfo[7] are used by Mac OS X to contain a 64-bit unique volume identifier. One use of this identifier is for tracking whether a given volume's ownership (user ID) information should be honored. These elements may be zero if no such identifier has been created for the volume. allocationFile Information about the location and size of the allocation file. See Fork Data Structure for a description of the HFSPlusForkData type. extentsFile Information about the location and size of the extents file. See Fork Data Structure for a description of the HFSPlusForkData type. catalogFile Information about the location and size of the catalog file. See Fork Data Structure for a description of the HFSPlusForkData type. attributesFile Information about the location and size of the attributes file. See Fork Data Structure for a description of the HFSPlusForkData type. startupFile Information about the location and size of the startup file. See Fork Data Structure for a description of the HFSPlusForkData type. Volume Attributes: bits 0-7 An implementation must treat these as reserved fields. kHFSVolumeUnmountedBit (bit 8) This bit is set if the volume was correctly flushed before being unmounted or ejected. An implementation must clear this bit on the media when it mounts a volume for writing. An implementation must set this bit on the media as the last step of unmounting a writable volume, after all other volume information has been flushed. If an implementation is asked to mount a volume where this bit is clear, it must assume the volume is inconsistent, and do appropriate consistency checking before using the volume. kHFSVolumeSparedBlocksBit (bit 9) This bit is set if there are any records in the extents overflow file for bad blocks (belonging to file ID kHFSBadBlockFileID). See Bad Block File for details. kHFSVolumeNoCacheRequiredBit (bit 10) This bit is set if the blocks from this volume should not be cached. For example, a RAM or ROM disk is actually stored in memory, so using additional memory to cache the volume's contents would be wasteful. kHFSBootVolumeInconsistentBit (bit 11) This bit is similar to kHFSVolumeUnmountedBit, but inverted in meaning. An implementation must set this bit on the media when it mounts a volume for writing. An implementation must clear this bit on the media as the last step of unmounting a writable volume, after all other volume information has been flushed. If an implementation is asked to mount a volume where this bit is set, it must assume the volume is inconsistent, and do appropriate consistency checking before using the volume. kHFSCatalogNodeIDsReusedBit (bit 12) This bit is set when the nextCatalogID field overflows 32 bits, forcing smaller catalog node IDs to be reused. When this bit is set, it is common (and not an error) for catalog records to exist with IDs greater than or equal to nextCatalogID. If this bit is set, you must ensure that IDs assigned to newly created catalog records do not conflict with IDs used by existing records. kHFSVolumeJournaledBit (bit 13) If this bit is set, the volume has a journal, which can be located using the journalInfoBlock field of the Volume Header. bit 14 An implementation must treat this bit as reserved. kHFSVolumeSoftwareLockBit (bit 15) This bit is set if the volume is write-protected due to a software setting. Any implementations must refuse to write to a volume with this bit set. This flag is especially useful for write-protecting a volume on a media that cannot be write-protected otherwise, or for protecting an individual partition on a partitioned device. bits 16-31 An implementation must treat these bits as reserved..) Back to top B-Trees. Each B-tree contains a single header node. The header node is always the first node in the B-tree. It contains the information needed to find other any other node in the tree. Map nodes contain map records, which hold any allocation data (a bitmap that describes the free nodes in the B-tree) that overflows the map record in the header node. Index nodes hold pointer records that determine the structure of the B-tree. Leaf nodes hold data records that contain the data associated with a given key. The key for each data record must be unique. All nodes share a common structure, described in the next section. Node Structure. struct BTNodeDescriptor { UInt32 fLink; UInt32 bLink; SInt8 kind; UInt8 height; UInt16 numRecords; UInt16 reserved; }; typedef struct BTNodeDescriptor BTNodeDescriptor; The fields have the following meaning: fLink The node number of the next node of this type, or 0 if this is the last node. bLink The node number of the previous node of this type, or 0 if this is the first node. kind The type of this node. There are four node kinds, defined by the constants listed below. height The level, or depth, of this node in the B-tree hierarchy. For the header node, this field must be zero. For leaf nodes, this field must be one. For index nodes, this field is one greater than the height of the child nodes it points to. The height of a map node is zero, just like for a header node. (Think of map nodes as extensions of the map record in the header node.) numRecords The number of records contained in this node. reserved An implementation must treat this as a reserved field.. Header Nodes. Header Record The B-tree header record contains general information about the B-tree such as its size, maximum key length, and the location of the first and last leaf nodes. The data type BTHeaderRec describes the structure of a header record.. The fields have the following meaning: treeDepth The current depth of the B-tree. Always equal to the height field of the root node. rootNode The node number of the root node, the index node that acts as the root of the B-tree. See Index Nodes for details. There is a possibility that the rootNode is a leaf node. See Inside Macintosh: Files, pp. 2-69 for details. leafRecords The total number of records contained in all of the leaf nodes. firstLeafNode The node number of the first leaf node. This may be zero if there are no leaf nodes. lastLeafNode The node number of the last leaf node. This may be zero if there are no leaf nodes. nodeSize The size, in bytes, of a node. This is a power of two, from 512 through 32,768, inclusive. maxKeyLength The maximum length of a key in an index or leaf node. HFSVolumes.h has the maxKeyLength values for the catalog and extents files for both HFS and HFS Plus (kHFSPlusExtentKeyMaximumLength, kHFSExtentKeyMaximumLength, kHFSPlusCatalogKeyMaximumLength, kHFSCatalogKeyMaximumLength). The maximum key length for the attributes B-tree will probably be a little larger than for the catalog file. In general, maxKeyLength has to be small enough (compared to nodeSize) so that a single node can fit two keys of maximum size plus the node descriptor and offsets. totalNodes The total number of nodes (be they free or used) in the B-tree. The length of the B-tree file is this value times the nodeSize. freeNodes The number of unused nodes in the B-tree. reserved1 An implementation must treat this as a reserved field. clumpSize Ignored for HFS Plus B-trees. The clumpSize field of the HFSPlusForkData record is used instead. For maximum compatibility, an implementation should probably set the clumpSize in the node descriptor to the same value as the clumpSize in the HFSPlusForkData when initializing a volume. Otherwise, it should treat the header records's clumpSize as reserved. btreeType The value stored in this field is of type. keyCompareType For HFSX volumes, this field in the catalog B-tree header defines the ordering of the keys (whether the volume is case-sensitive or case-insensitive). In all other cases, an implementation must treat this as a reserved field. Constant name Value Meaning kHFSCaseFolding 0xCF Case folding (case-insensitive) kHFSBinaryCompare 0xBC Binary compare (case-sensitive) attributes A set of bits used to describe various attributes of the B-tree. The meaning of these bits is given below. reserved3 An implementation must treat this as a reserved field. The following constants define the various bits that may be set in the attributes field of the header record. enum { kBTBadCloseMask = 0x00000001, kBTBigKeysMask = 0x00000002, kBTVariableIndexKeysMask = 0x00000004 }; The bits have the following meaning: kBTBadCloseMask This bit indicates that the B-tree was not closed properly and should be checked for consistency. This bit is not used for HFS Plus B-trees. An implementation must treat this as reserved. kBTBigKeysMask If this bit is set, the keyLength field of the keys in index and leaf nodes is UInt16; otherwise, it is a UInt8. This bit must be set for all HFS Plus B-trees. kBTVariableIndexKeysMask If this bit is set, the keys in index nodes occupy the number of bytes indicated by their keyLength field; otherwise, the keys in index nodes always occupy maxKeyLength bytes. This bit must be set for the HFS Plus Catalog B-tree, and cleared for the HFS Plus Extents B-tree. Bits not specified here must be treated as reserved. User Data Record. Map Record. Map Nodes. Keyed Records. Index Nodes). Leaf Nodes. Searching for Keyed Records: Keyed records must be placed in a node such that their keys are in ascending order. All the nodes in a given level (whose height field is the same) must be chained via their fLink and bLink field. The node with the smallest keys must be first in the chain and its bLink field must be zero. The node with the largest keys must be last in the chain and its fLink field must be zero. For any given node, all the keys in the node must be less than all the keys in the next node in the chain (pointed to by fLink). Similarly, all the keys in the node must be greater than all the keys in the previous node in the chain (pointed to by bLink). HFS and HFS Plus B-Trees Compared). Node Sizes: 4 KB (8KB in Mac OS X) for the catalog file 1 KB (4KB in Mac OS X) for the extents overflow file 4 KB for the attributes file). IMPORTANT: The node size of the attributes file must be at least kHFSPlusAttrMinNodeSize (4096). Key Size in an Index Node. Back to top Catalog File. typedef UInt32 HFSCatalogNodeID; The first 16 CNIDs are reserved for use by Apple Computer, Inc., and include the following standard assignments: }; These constants have the following meaning: kHFSRootParentID Parent ID of the root folder. kHFSRootFolderID Folder ID of the root folder. kHFSExtentsFileID File ID of the extents overflow file. kHFSCatalogFileID File ID of the catalog file. kHFSBadBlockFileID File ID of the bad block file. The bad block file is not a file in the same sense as a special file and a user file. See Bad Block File for details. kHFSAllocationFileID File ID of the allocation file (introduced with HFS Plus). kHFSStartupFileID File ID of the startup file (introduced with HFS Plus). kHFSAttributesFileID File ID of the attributes file (introduced with HFS Plus). kHFSRepairCatalogFileID Used temporarily by fsck_hfs when rebuilding the catalog file. kHFSBogusExtentFileID Used temporarily during ExchangeFiles operations. kHFSFirstUserCatalogNodeID First CNID available for use by user files and folders.: the format of the key used both in index and leaf nodes, and the format of the leaf node data records (file, folder, and thread records). Catalog File Key For a given file, folder, or thread record, the catalog file key consists of the parent folder's CNID and the name of the file or folder. This structure is described using the HFSPlusCatalogKey type. struct HFSPlusCatalogKey { UInt16 keyLength; HFSCatalogNodeID parentID; HFSUniStr255 nodeName; }; typedef struct HFSPlusCatalogKey HFSPlusCatalogKey; The fields have the following meaning: keyLength The keyLength field is required by all keyed records in a B-tree. The catalog file, in common with all HFS Plus B-trees, uses a large key length (UInt16). parentID For file and folder records, this is the folder containing the file or folder represented by the record. For thread records, this is the CNID of the file or folder itself. nodeName This field contains Unicode characters, fully decomposed and in canonical order. For file or folder records, this is the name of the file or folder inside the parentID folder. For thread records, this is the empty string.. Catalog File Data A catalog file leaf node can contain four different types of data records: A folder record contains information about a single folder. A file record contains information about a single file. A folder thread record provides a link between a folder and its parent folder, and lets you find a folder record given just the folder ID. A file thread record provides a link between a file and its parent folder, and lets you find a file record given just the file ID. (In both the folder thread and the file thread record, the thread record is used to map the file or folder ID to the actual parent directory ID and name.) Each record starts with a recordType field, which describes the type of catalog data record. The recordType field contains one of the following values: enum { kHFSPlusFolderRecord = 0x0001, kHFSPlusFileRecord = 0x0002, kHFSPlusFolderThreadRecord = 0x0003, kHFSPlusFileThreadRecord = 0x0004 }; The values have the following meaning: kHFSPlusFolderRecord This record is a folder record. You can use the HFSPlusCatalogFolder type to interpret the data. kHFSPlusFileRecord This record is a file record. You can use the HFSPlusCatalogFile type to interpret the data. kHFSPlusFolderThreadRecord This record is a folder thread record. You can use the HFSPlusCatalogThread type to interpret the data. kHFSPlusFileThreadRecord This record is a file thread record. You can use the HFSPlusCatalogThread type to interpret the data. }; The values have the following meaning: kHFSFolderRecord This record is an HFS folder record. You can use the HFSCatalogFolder type to interpret the data. kHFSFileRecord This record is an HFS file record. You can use the HFSCatalogFile type to interpret the data. kHFSFolderThreadRecord This record is an HFS folder thread record. You can use the HFSCatalogThread type to interpret the data. kHFSFileThreadRecord This record is an HFS file thread record. You can use the HFSCatalogThread type to interpret the data. Catalog Folder Records; The fields have the following meaning: recordType The catalog data record type. For folder records, this is always kHFSPlusFolderRecord. flags This field contains bit flags about the folder. No bits are currently defined for folder records. An implementation must treat this as a reserved field. valence The number of files and folders directly contained by this folder. This is equal to the number of file and folder records whose key's parentID is equal to this folder's. folderID The CNID of this folder. Remember that the key for a folder record contains the CNID of the folders parent, not the CNID of the folder itself. createDate The date and time the folder was created. See HFS Plus Dates for a description of the format. Again, the createDate of the Volume Header is NOT stored in GMT; it is local time. (Further, if the volume has an HFS wrapper, the creation date in the MDB should be the same as the createDate in the Volume Header). contentModDate The date and time the folder's contents were last changed. This is the time when a file or folder was created or deleted inside this folder, or when a file or folder was moved in or out of this folder. See HFS Plus Dates for a description of the format. Note: The traditional Mac OS APIs use the contentModDate when getting and setting the modification date. The traditional Mac OS APIs treat attributeModDate as a reserved field. attributeModDate The last date and time that any field in the folder's catalog record was changed. An implementation may treat this field as reserved. In Mac OS X, the BSD APIs use this field as the folder's change time (returned in the st_ctime field of struct stat). All versions of Mac OS 8 and 9 treat this field as reserved. See HFS Plus Dates for a description of the format. accessDate The date and time the folder's contents were last read. This field has no analog in the HFS catalog record. It represents the last time the folder's contents were. Folders created by traditional Mac OS have an accessDate of zero. backupDate The date and time the folder was last backed up. The volume format requires no special action on this field; it simply defines the field for the benefit of user programs. See HFS Plus Dates for a description of the format. permissions This field contains folder permissions, similar to those defined by POSIX or AFP. See HFS Plus Permissions for a description of the format. IMPORTANT: The traditional Mac OS implementation of HFS Plus does not use the permissions field. Folders created by traditional Mac OS have the entire field set to 0. userInfo This field contains information used by the Mac OS Finder. The contents of this structure are not strictly part of the HFS Plus specification, but general information is in folder name was derived. This hint can be used to improve the quality of the conversion of the name to a Mac OS-encoded Pascal string. See Text Encodings for details. reserved An implementation must treat this as a reserved field. Catalog File Records; The fields have the following meaning: recordType The catalog data record type. For files records, this is always kHFSPlusFileRecord. flags This field contains bit flags about the file. The currently defined bits are described below. An implementation must treat undefined bits as reserved. reserved1 An implementation must treat this as a reserved field. fileID The CNID of this file. createDate The date and time the file was created. See HFS Plus Dates for a description of the format. contentModDate The date and time the file's contents were last changed by extending, truncating, or writing either of the forks. See HFS Plus Dates for a description of the format. attributeModDate The last date and time that any field in the file's catalog record was changed. An implementation may treat this field as reserved. In Mac OS X, the BSD APIs use this field as the file's change time (returned in the st_ctime field of struct stat). All versions of Mac OS 8 and 9 treat this field as reserved. See HFS Plus Dates for a description of the format. accessDate The date and time the file's contents were last read. This field has no analog in the HFS catalog record. It represents the last time either of a file's forks was. Files created by traditional Mac OS have an accessDate of zero. backupDate The date and time the file was last backed up. The volume format requires no special action on this field; it simply defines the field for the benefit of user programs. See HFS Plus Dates for a description of the format. permissions This field contains file permissions, similar to those defined by POSIX. See HFS Plus Permissions for a description of the format. IMPORTANT: The traditional Mac OS implementation of HFS Plus does not use the permissions field. Files created by traditional Mac OS have the entire field set to 0. userInfo This field contains information used by the Mac OS Finder. For more information, see file name was derived. This hint can be used to improved the quality of the conversion of the name to a Mac OS encoded Pascal string. See Text Encodings for details. reserved2 An implementation must treat this as a reserved field. dataFork Information about the location and size of the data fork. See Fork Data Structure for a description of the HFSPlusForkData type. resourceFork Information about the location and size of the resource fork. See Fork Data Structure for a description of the HFSPlusForkData type. }; The values have the following meaning: kHFSFileLockedBit, kHFSFileLockedMask If kHFSFileLockedBit is set, then none of the forks may be extended, truncated, or written to. They may only be opened for reading (not for writing). The catalog information (like finderInfo and userInfo) may still be changed. kHFSThreadExistsBit, kHFSThreadExistsMask This bit incidates that the file has a thread record. As all files in HFS Plus have thread records, this bit must be set. Catalog Thread Records fields have the following meaning: recordType The catalog data record type. For thread records, this is kHFSPlusFileRecord or kHFSPlusFolderRecord, depending on whether the thread record refers to a file or a folder. Both types of thread record contain the same data. reserved1 An implementation must treat this as a reserved field. parentID The CNID of the parent of the file or folder referenced by this thread record. nodeName The name of the file or folder referenced by this thread record. The next section explains how thread records can be used to find a file or folder using just its CNID. Catalog Tree Usage. Finder Info; Back to top Extents Overflow File. Extents Overflow File Key The structure of the key for the extents overflow file is described by the HFSPlusExtentKey type. struct HFSPlusExtentKey { UInt16 keyLength; UInt8 forkType; UInt8 pad; HFSCatalogNodeID fileID; UInt32 startBlock; }; typedef struct HFSPlusExtentKey HFSPlusExtentKey; The fields have the following meaning: keyLength The keyLength field is required by all keyed records in a B-tree. The extents overflow file, in common with all HFS Plus B-trees, uses a large key length (UInt16). Keys in the extents overflow file always have the same length, kHFSPlusExtentKeyMaximumLength (10). forkType The type of fork for which this extent record applies. This must be either 0 for the data fork or 0xFF for the resource fork. pad An implementation must treat this as a pad field. fileID The CNID of the file for which this extent record applies. startBlock The offset, in allocation blocks, into the fork of the first extent described by this extent record. The startBlock field lets you directly find the particular extents for a given offset into a fork.. Extents Overflow File Data). Extents Overflow File Usage.) Bad Block File. Back to top Allocation File. Using a file allows the bitmap itself to be allocated from allocation blocks. This simplifies the design, since volumes are now comprised of only one type of block -- the allocation block. The HFS is slightly more complex because it uses 512-byte blocks to hold the allocation bitmap and allocation blocks to hold file data. The allocation file does not have to be contiguous, which allows allocation information and user data to be interleaved. Many modern file systems do this to reduce head travel when growing files. The allocation file can be extended, which makes it significantly easier to increase the number of allocation blocks on a disk. This is useful if you want to either decrease the allocation block size on a disk, or increase the total disk size. The allocation file may be shrunk. This makes it easy to create a disk images suitable for volumes of varying sizes. The allocation file in the disk image is sized to hold enough allocation data for the largest disk, and shrunk back when the disk is written to a smaller disk.. Back to top Attributes File: the structure of the keys in the attribute B-tree has not been finalized and is subject to change, and addition attribute's file data record types may be defined. An implementation written to this specification may use the details that are final to perform basic consistency checks on attributes. These checks will be compatible with future implementations written to a final attributes specification. See Attributes and the Allocation File Consistency Check. Attributes File Data: Fork data attributes are used for attributes whose data is large. The attribute's data is stored in extents on the volume and the attribute merely contains a reference to those extents. Extension attributes augment fork data attributes, allowing an fork data attribute to have more than eight extents. Each record starts with a recordType field, which describes the type of attribute data record. The recordType fieldData type to interpret the data. kHFSPlusAttrExtents This record is an extension attribute. You can use the HFSPlusAttrExtents type to interpret the data. A record of type kHFSPlusAttrExtents is really just overflow extents for a corresponding record of type kHFSPlusAttrForkData. (Think of kHFSPlusAttrForkData as being like a catalog record and kHFSPlusAttrExtents as being like an extents overflow record.) The next two sections describe the fork data and extension attributes in detail. Fork Data Attributes fields have the following meaning: recordType The attribute data record type. For fork data attributes, this is always kHFSPlusAttrForkData. reserved An implementation must treat this as a reserved field. theFork Information about the location and size of the attribute data. See Fork Data Structure for a description of the HFSPlusForkData type. Extension Attributes; The fields have the following meaning: recordType The attribute data record type. For extension attributes, this is always kHFSPlusAttrExtents. reserved An implementation must treat this as a reserved field. extents The eight extents of the attribute data described by this record. See Fork Data Structure for a description of the HFSPlusExtentRecord type. Attributes and the Allocation File Consistency Check. Back to top Startup File. Back to top Hard Links. Repairing the Metadata Directory When repairing an HFS Plus volume with hard links or a metadata directory, there are several conditions that might need to be repaired: Opened but deleted files (which are now orphaned). Orphaned indirect node files (no hard links refer to them). Broken hard link (hard link exists, but indirect node file does not). Incorrect link count. Link reference was 0.. Back to top Symbolic Links. Back to top Journal. Background). Overview of Journal Data Structures. Journal Info Block; The fields have the following meaning: flags Contains a set of one-bit flags. The currently defined bits are described below. An implementation must treat the undefined bits as reserved. device_signature This space is reserved for describing the device containing the journal when the journal is not stored in the volume itself (when kJIJournalOnOtherDeviceMask is set). offset The offset in bytes from the start of the device to the start of the journal header. When the journal is stored in the volume itself (kJIJournalInFSMask is set), this offset is relative to the start of the volume. size The size in bytes of the journal, including the journal header and the journal buffer. This size does not include the journal info block. reserved This space is reserved. The following constants define bit flags in the flags field: enum { kJIJournalInFSMask = 0x00000001, kJIJournalOnOtherDeviceMask = 0x00000002, kJIJournalNeedInitMask = 0x00000004 }; The values have the following meaning: kJIJournalInFSMask When set, the space for the journal header and transactions resides inside the volume being journaled. The offset field of the journal info block is relative to the start of the volume (allocation block number zero). kJIJournalOnOtherDeviceMask When set, the space for the journal header and journal buffer does not reside inside the volume being journaled. The device_signature field in the journal info block describes the device containing the journal. kJIJournalNeedInitMask This bit is set to indicate that the journal header is invalid and needs to be initialized. This bit is typically set when the journal is first created, and the space has been allocated; the first mount of the journaled volume typically initializes the journal header and clears this bit. When this bit is set, there are no valid transactions in the journal. Note: Implementations must currently set kJIJournalInFSMask, but not kJIJournalOnOtherDeviceMask. Journals stored on a separate device are not currently supported. The format of the device_signature field is not yet defined. Journal Header The journal begins with a journal header, whose main purpose is to describe the location of transactions in the journal buffer. The journal header is stored using the journal_header data type. The fields have the following meaning: magic Contains the value JOURNAL_HEADER_MAGIC (0x4a4e4c78). This is used to verify the integrity of the journal header. endian Contains the value ENDIAN_MAGIC (0x12345678). This is used to verify the integrity of the journal header. start Contains the offset in bytes from the start of the journal header to the start of the first (oldest) transaction. end Contains the offset in bytes from the start of the journal header to the end of the last (newest) transaction. Note that this field may be less than the start field, indicating that the transactions wrap around the end of the journal's circular buffer. If end equals start, then the journal is empty, and there are no transactions that need to be replayed. size The size of the journal, in bytes. This includes the journal header and the journal buffer. This value must be equal to the value in the size field of the journal info block. blhdr_size The size of one block list header, in bytes. This value typically ranges from 4096 to 16384. checksum The checksum of the journal header, computed as described below. jhdr_size The size of the journal header, in bytes. The journal header always occupies exactly one sector so that it can be updated atomically. Therefore, this value is equal to the sector size (for example, 2048 on many types of optical media). Block List Header The block list header describes a list of blocks included in a transaction. A transaction may include several block lists if it modifies more blocks than can be represented in a single block list. The block list header is stored in a structure of type block_list_header. typedef struct block_list_header { UInt16 max_blocks; UInt16 num_blocks; UInt32 bytes_used; UInt32 checksum; UInt32 pad; block_info binfo[1]; } block_list_header; The fields have the following meaning: max_blocks The maximum number of blocks (block_info items) this block list can describe. This field is used while in memory to keep track of journal buffer sizes. On disk, this field is reserved. num_blocks The number of elements in the binfo array. Since the first element of the binfo array is used to chain multiple block lists into a single transaction, the actual number of data blocks is num_blocks - 1. bytes_used The number of bytes occupied in the journal for this block list, including the block list header and the data for each of the blocks in the list. The next block list header (if any) will be bytes_used bytes from the start of the current block list header, wrapping from the end of the journal buffer to the start of the journal buffer if needed. checksum The checksum of the block list header, including the first element of the binfo array (a total of 32 bytes). pad Alignment padding. An implementation must treat this field as reserved. binfo A variable-sized array of blocks. The array contains num_blocks+1 entries. The first entry is used when a single transaction occupies multiple block lists, using the next field as described above. The remaining num_blocks entries describe where the data from this block list will be written to disk. The sector number where the data in this block must be written. If this field is 0xFFFFFFFFFFFFFFFF (all 64 bits set), then this block must be skipped and not written. This field is reserved for the first element of the binfo array. bsize The number of bytes to be copied from the journal buffer to the above sector number. This value will be a multiple of 512. This field is reserved for the first element of the binfo array. next This field is used while in memory to keep track of transactions that span multiple block lists. If this field is zero in the first block_info of a block list, then the transaction ends with this block list; otherwise, the transaction has one or more additional block lists. This field is meaningful only for the first element of the block list array. The actual on-disk value has no meaning beyond testing for zero or non-zero. Journal Checksums. static int calc_checksum(unsigned char *ptr, int len) { int i, cksum=0; for(i=0; i < len; i++, ptr++) { cksum = (cksum << 8) ^ (cksum + *ptr); } return (~cksum); } Replaying the Journal: Read the volume header into variable vhb. The volume may have an HFS wrapper; if so, you will need to use it to determine the location of the volume header. Test the kHFSVolumeJournaledBit in the attributes field of the volume header. If it is not set, there is no journal to replay, and you are done. Read the journal info block from the allocation block number vhb.journalInfoBlock, into variable jib. If kJIJournalNeedsInitMask is set in jib.flags, the journal was never initialized, so there is no journal to replay. Verify that kJIJournalInFSMask is set and kJIJournalOnOtherDeviceMask is clear in jib.flags. Read the journal header at jib.offset bytes from the start of the volume, and place it in variable jhdr. If jhdr.start equals jhdr.end, the journal does not have any transactions, so there is nothing to replay. Set the current offset in the journal (typically a local variable) to the start of the journal buffer, jhdr.start. While jhdr.start does not equal jhdr.end, perform the following steps: Read a block list header of jhdr.blhdr_size bytes from the current offset in the journal into variable blhdr. For each block in bhdr.binfo[1] to bhdr.binfo[blhdr.num_blocks], inclusive, copy bsize bytes from the current offset in the journal to sector bnum on the volume (to byte offset bnum*jdhr.jhdr_size). Remember that jhdr_size is the size of a sector, in bytes. If bhdr.binfo[0].next is zero, you have completed the last block list of the current transaction; set jhdr.start to the current offset in the journal.. Back to top HFSX". HFSX Version 5. Back to top Metadata Zone: Item Contribution to the Metadata Zone size Allocation Bitmap File Physical size (totalBlocks times the volume's allocation block size) of the allocation bitmap file. Extents Overflow File 4MB, plus 4MB per 100GB (up to 128MB maximum) Journal File 8MB, plus 8MB per 100GB (up to 512MB maximum) Catalog File 10 bytes per KB (1GB minimum) Hot Files 5 bytes per KB (10MB minimum; 512MB maximum) Quota Users File Described below Quota Groups File Described below. Hot Files: Recording Watch file accesses to determine which files are used most Evaluation Merge recently used hot files with previously found hot files Eviction Move older and less frequently used hot files out of metadata zone to make room for newer, hotter files Adoption Move newer and hotter files into the metadata zone Hot File B-Tree; The fields have the following meaning: magic Must contain the value HFC_MAGIC (0xFF28FF26). version Contains the version of the HotFilesInfo structure. Version 1 of the structure is described here. If your implementation encounters any other version number, it should not read or modify the hot file B-tree. duration Contains the duration of the current recording phase, in seconds. In Mac OS X 10.3, this value is typically HFC_DEFAULT_DURATION (60 hours). timebase Contains the time that the current recording phase began, in seconds since Jan 1, 1970 GMT. timeleft Contains the time remaining in the current recording phase, in seconds. threshold Contains the minimum temperature for a file to be eligible to be moved into the hot file area. Files whose temperature is less than this value will be moved out of the hot file area. maxfileblks Contains the maximum file size, in allocation blocks, for a file to be eligible to be moved into the hot file area. Files larger than this size will not be moved into the hot file area. In Mac OS X 10.3, this value is typically HFC_MAXIMUM_FILESIZE divided by the volume's allocation block size. maxfilecnt Contains the maximum number of files to place into the hot file area. Note that the hot file area may actually contain more than this number of files, especially if they previously existed in the hot file area before the beginning of the recording phase. This number represents the number of files that the hot file recording code intents to track and eventually place into the hot file area. tag Contains the null-terminated (C-style) string containing the ASCII text "CLUSTERED HOT FILES B-TREE " (not including the quotes). Note that the last six bytes are five spaces and the null (zero) byte. This field exists to make it easier to recognize the hot file B-tree when debugging or using a disk editor. An implementation should not attempt to verify or change this field. Hot File Record Key A key in the hot file B-tree is of type)) The fields have the following meaning: keyLength The length of a hot file key, not including the keyLength field itself. Hot file keys are of fixed size. This field must contain the value 10. forkType Indicates whether the fork being tracked is a data fork (value 0x00) or a resource fork (value 0xFF). In Mac OS X version 10.3, only data forks are eligible for placement into the hot file area. pad An implementation must treat this as a pad field. temperature The fork's temperature. For hot file thread records, this field contains the value HFC_LOOKUPTAG (0xFFFFFFFF). fileID The catalog node ID of the file being tracked. Hot file keys are compared first by temperature, then fileID, and lastly by forkType. All of these comparisons are unsigned. Hot File Records. Recording Hot File Temperatures. Back to top Unicode Subtleties. Canonical Decomposition). To reduce complexity in the B-tree key comparison routines (which have to compare Unicode strings), HFS Plus defines that Unicode strings will be stored in fully decomposed form, with composing characters stored in canonical order. The other equivalent forms are illegal in HFS Plus strings. An implementation must convert these equivalent forms to the fully decomposed form before storing the string on disk.. Case-Insensitive String Comparison Algorithm In HFS Plus and case-insensitive HFSX, strings must be compared in a case-insensitive fashion. The Unicode standard does not strictly define upper and lower case equivalence, although it does suggest some equivalences. The HFS Plus string comparison algorithm (defined below) include a concrete case equivalence definition. An implementation must use the equivalence expressed by this algorithm. Furthermore, Unicode requires that certain formatting characters be ignored (skipped over) during string comparisons. The algorithm and tables used for case equivalence also arrange to ignore these characters. An implementations must ignore the characters that are ignored by this algorithm.. */ }; Back to top HFS Wrapper An HFS Plus volume may be contained within an HFS volume in a way that makes the volume look like an HFS volume to systems without HFS Plus support. This has a two important advantages: It allows a computer with HFS (but no HFS Plus) support in ROM to start up from an HFS Plus volume. When creating the wrapper, Mac OS includes a System file containing the minimum code to locate and mount the embedded HFS Plus volume and continue booting from its System file. It improves the user experience when an HFS Plus volume is inserted in a computer that has HFS support but no HFS Plus support. On such a computer, the HFS wrapper will be mounted as a volume, which prevents error dialogs that might confuse the user into thinking the volume is empty, damaged, or unreadable. The HFS wrapper may also contain a Read Me document to explain the steps the user should take to access their files.. HFS Master Directory Block. Allocating Space for the Embedded Volume. Read Me and System Files. Read Me -- The Read Me file, whose name is actually "Where_have_all_my_files_gone?", contains text explaining that this volume is really an HFS Plus volume but the contents cannot be accessed because HFS Plus is not currently installed on the computer. It also describes the steps needed to install HFS Plus support. Localized system software will also create a localized version of the file with localized file name and text content. System and Finder (invisible) -- The System file contains the minimum code to locate and mount the embedded HFS Plus volume, and to continue booting from the System file in the embedded volume. The Finder file is empty; it is there to prevent older versions of the Finder from de-blessing the wrapper's root directory, which would prevent booting from the volume. Desktop DB and Desktop DF (invisible) -- The Desktop DB and Desktop DF files are an artifact of the way the files on the wrapper volume are created. In addition, the root folder is set as the blessed folder by placing its folder ID in the first SInt32 of the drFndrInfo (Finder information) field of the MDB. Back to top Volume Consistency Checks. Next Catalog Node ID Consistency Check For an HFS Plus volume to work correctly with many implementations, it is vital that the nextCatalogID field of the volume header be greater than all CNIDs currently used in the catalog file. The algorithm to ensure this is as follows. The implementation must iterate through all the leaf nodes of the catalog file, looking for file and folder records, determining the maximum CNID of any file or folder in the catalog. Once it knows the maximum CNID value, the implementation must set nextCatalogID to a value greater than it. Note: The consistency check of nextCatalogID must be skipped if kHFSCatalogNodeIDsReusedBit is set in the attributes field of the volume header. Allocation File Consistency Check For an HFS Plus volume to work correctly, it's vital that any allocation block in use by file system structures be marked as allocated in the allocation file. The algorithm to ensure this is as follows: The implementation must first walk the allocation file, marking every allocation block as free. (This step can be skipped to improve the performance of the consistency check. All that will happen is that some allocation blocks may have been marked as in-use, though they are not really in use by any extent.) The implementation must then mark the allocation blocks containing the first 1536 bytes and the last 1024 bytes as allocated. These areas are either reserved or used by the volume header. The implementation must then mark the allocation blocks used by all extents in all special files (the catalog file, the extents overflow file, the allocation file, the attributes file, and the startup file) as allocated. These extents are all described in the volume header. The implementation must then walk the leaf nodes of the catalog file, marking all allocation blocks used by extents in file records (in the HFSPlusForkData structures for the data and resource forks). The implementation must then walk the leaf nodes of the extents overflow file, marking all allocation blocks used by all extents in all extent records as allocated. The implementation must then walk the leaf nodes of the attributes file, marking all allocation blocks used by all extents described in fork data attributes and extension attributes as allocated.. Back to top Summary Volume format specifications are fun exhausting. Back to top References. Change History. Back to top Downloadables FastUnicodeCompare.c (43 KB). Download Back to top