What is OS X's equivalent of Linux's NSS?
I'm not very familiar with Linux, but it sounds like you're looking for Open Directory.  Warning: Apple uses this term to refer to all of the various parts of their directory service architecture, so if you google "Open Directory" you'll find both Apple's LDAP-based directory server, and the daemon that runs on OS X and does the function you're looking for.

Let me try to give you a historical overview of its development (and hopefully I'll remember which changes happened in which version):

NextSTEP (the OS that OS X was based on) had a daemon called lookupd that was, as I understand it, very similar to NSS.  Most of the actual data was stored in NetInfo databases (which could be either local on the computer, or accessed over the net from a server).

Early versions of OS X added another deamon, named DirectoryService, which did similar things (but a little differently); since it didn't do everything lookupd did, any query it couldn't find an answer to would get handed off to lookupd.  Result: a certain amount of redundancy and confusion.  IIRC, there were actually some queries that got directed to lookupd first, and them it might have to hand them off to DirectoryService.  There were also two different sets of plug-ins (equiv to NSS modules): those for DirectoryService and those for lookupd. Local users &amp; groups were still stored in NetInfo, but it was deprecated for network accounts in favor of LDAP.

I don't remember the details anymore, but over the versions responsibilities gradually got shifted from lookupd to DirectoryService.

In 10.5, NetInfo was removed; local users, groups, etc were stored in XML property list files under /var/db/dslocal/.  The plug-in (excuse me, they're called "connectors" now) to access them has the not-at-all-generic name "Local" (it's sometimes referred to as "dslocal" to disambiguate it a bit).  At this point, the only thing lookupd is still responsible for is DNS lookups (although it hands off multicast lookups to mDNSResponder).

In 10.6, lookupd was removed; mDNSResponder now handles all DNS lookups, both standard (unicast) and multicast.

In 10.7, the Open Directory daemon was renamed from DirectoryService to opendirectoryd.