text: Extended attributes
  Tiger includes support for the "xattr" family of functions in the BSD layer: getxattr(), setxattr(), removexattr(), listxattr(), plus "f"-prefixed variants that deal with file descriptors instead of paths. The "xattr" functions have existed in various forms for many years on other flavors of Unix, but this is their first appearance in Mac OS X. The Unix manual page explains.
  Extended attributes extend the basic attributes associated with files and directories in the file system. They are stored as name:data pairs associated with file system objects (files, directories, symlinks, etc).
  An extended attribute's name is a simple NULL-terminated UTF-8 string. Value is a pointer to a data buffer containing textual or binary data to be associated with the extended attribute.
  In case it hasn't clicked yet: after over four years of waiting, Mac OS X now includes support for arbitrarily extensible file system metadata.
  (I'm a big enough geek that I wanted to end that sentence with an exclamation point in addition to making it bold. I've decided to spare the casual reader. But if you read it that way on your own, right on, brother.)
  Yes, it's really true. I commissioned long-time Macintoshian Achaia regular Marquis Logan ("nibs") of The Big Nerd Ranch to create a simple command-line tool that lets you see for yourself. He went beyond the call of duty, wrapping the BSD APIs in a set of CoreFoundation-like classes. You can download the source or the binary. (You'll need Xcode to build the source.) Here's an example of the program, appropriately named xattr, in action.
  First, create a small text file to play with.
  % cat > file Testing 1 2 3. ^D % cat file Testing 1 2 3.
  Next, add, list, modify, and delete some arbitrary attributes.
  % xattr --set name John file % xattr --set color red file % xattr --list file file color red name John % xattr --set color blue file % xattr --list file file color blue name John % xattr --delete color file % xattr --list file file name John
  There's nothing up my sleeve, honest.
  % ls -la total 8 drwxrwx--- 3 john john 102 Mar 24 21:36 ./ drwxr-xr-x 66 john john 2244 Mar 24 21:36 ../ -rw-rw---- 1 john john 15 Mar 24 21:36 file
  No, there's no resource fork either.
  % ls -l file/..namedfork/rsrc -rw-rw---- 1 john john 0 Mar 24 21:36 file/..namedfork/rsrc
  In Tiger, most Unix command-line tools are metadata-savvy too...
  % cp file file2 % xattr --list file2 file2 name John
  ...but some require an extra flag. (RTFM to be sure.)
  % mkdir backup % cp file backup % rsync -E file backup % xattr --list backup/file backup/file name John
  This is all happening on an HFS+ volume—not "HFSX" but plain old HFS+. As foreshadowed in March of 2004, an unmodified HFS+ volume does indeed have the ability to store an arbitrary number of attributes. The xattr APIs in Tiger (finally) leverage this ability. No new volume format has been introduced, and no reformatting is required.
  Support on lesser volume formats is provided by the venerable dot-underscore file.
  % hdiutil create -size 50K -fs MS-DOS -volname DOS dos.dmg created: /Users/john/dos.dmg % hdiutil attach dos.dmg /dev/disk5 /Volumes/DOS % cd /Volumes/DOS % touch file % ls -l total 0 -rwxrwxrwx 1 john john 0 Mar 24 22:07 file % xattr --set name John file % xattr --list file file name John % ls -la total 49 ... -rwxrwxrwx 1 john john 4096 Mar 24 22:07 ._file -rwxrwxrwx 1 john john 0 Mar 24 22:07 file
  As in earlier versions of Mac OS X, files prefixed with "._" contain all of the metadata for their un-prefixed partner that the native volume format cannot contain in a single file: resource forks, flags, dates, and now in Tiger, extended attributes.
  The extended attribute APIs hide these details, and the other command-line tools have been updated to use the new APIs when needed. For example, look at what happens when the cp command is used to copy a file with extended attributes ("file2" from the earlier example) from an HFS+ volume onto a FAT (MS-DOS) volume.
  % cd /Volumes/DOS % cp /Users/john/file2 . % xattr --list file2 file2 name John % ls -la ... -rwxrwxrwx 1 john john 4096 Mar 24 22:09 ._file2 -rwxrwxrwx 1 john john 15 Mar 24 22:09 file2
  Boom. Instant dot-underscore file. (Now there's a demo you'll likely never see Jobs do in a keynote presentation.)
  Extended attributes: uses and limits
  The Tiger implementation of extended attributes is "preliminary" in many senses of the word. There are some limitations on the number and size of attribute names and values. Attribute names can be up to 128 bytes long and are made up of Unicode (UTF-8) characters. Values are up to 4KB of arbitrary data. (There is actually no hard limit on the size of attribute values, but Apple recommends staying below 4KB for now.)
  While there appears to be no hard limit on the number of attributes that can be associated with a file, the time it takes to add an attribute increases tremendously once a few thousand attributes are added. Running something like this
  % touch file % perl -e 'for(1..50000) { system "xattr --set a$i v$i file"; $i++ }'
  for a few minutes results in a file with about 7,000 attributes. I thought maybe the attribute insertion performance was falling off due to hashing collisions in the implementation, so I tried adding a random string to the front of each attribute name. The performance didn't really improve. I'm not sure what the problem is. I ran it some more and got a file with over 12,000 attributes, but it took a while.
  Interestingly, only the addition of attributes seems to slow down so precipitously. Listing all 12,000+ attribute names and values was nearly instant. Deleting an attribute was similarly quick.
  This kind of torture testing may seem academic, but I'm trying to get a feel for how well HFS+ fares in its new role. I had hoped for, requested, and even now (in optimistic moments) still expect a new high-performance metadata-enabled file system from Apple. Tapping HFS+ to do the job in the meantime is fine with me, but I can't help pining for some more of the strengths of the Be file system to make their way to the Mac platform. (Of particular note is BFS's support for the creation of file-system-resident indexes on attributes, and its real-time search and notification features. But I'll have more to say about those topics later.)
  The reverse-DNS naming style first gained widespread use in the Java programming language as a means to separate code into namespaces. It has become increasingly common since.
  The style is called reverse-DNS because it uses piecewise-backwards identifiers from the domain name system (DNS). For example, Apple owns the developer.apple.com domain name, and the reverse-DNS equivalent is com.apple.developer.
  The domain name system already has a well-established controlling authority and dispute arbitration system. The reverse-DNS naming style effectively piggybacks on this infrastructure. This is such a big win than any new naming system intended for widespread use that chooses not to use reverse-DNS names is probably making a mistake.
  The 128-byte limit on attribute names is reasonable, but remember that there is just one flat namespace for attributes. Some sort of policy for isolation is necessary; we can't have 5 programs all trying to write to the "name" attribute, after all.
  To solve this problem, Apple recommends the use of a reverse-DNS naming scheme (e.g., com.apple.itunes.artist), but with the dots replaced with underscores in order to avoid conflicting with the namespace convention used by Cocoa's key/value coding system.
  The final, and perhaps most important limitation of Tiger's extended attribute implementation can also be seen as a feature. The extended attribute APIs exist only at the BSD layer. There are no Carbon, Cocoa, or even Core Foundation interfaces to them at this time. If an application wants to use extended attributes, it has to use the (decidedly low-level) BSD APIs.
  Historically, Apple has introduced many technologies "from the bottom up," providing BSD implementations first and adding the higher-level APIs later. The BSD layer usually has to come first, or at least simultaneously, since most high-level APIs in Mac OS X eventually trigger one or more BSD-level function calls. This is especially true for file i/o.
  In Tiger, extended attributes are essentially being "field tested." The APIs exist, HFS+ supports them natively, and most BSD-level tools have been updated to honor them. But until there are Carbon, Cocoa, or at least Core Foundation APIs that abstract and explicitly leverage these new features, extended attributes will likely fly under the radar of traditional (GUI) Mac OS X applications.
  That's a shame because there are a ton of potential uses for extended attributes, as the next topic will demonstrate.
  You must login or create an account to comment.