How much does the build number of an Android OS matter? I have seen several builds on OS 4.3 and 4.4-- what are the key differences? I tried finding documentation on the exact changes, but couldn't find any.

I set up two new Nexus 7 tablets (2013) and even brand new ones come with different build versions.
How much does the build number of an Android OS matter?

As a developer, I have never used it for anything significant (other than maybe analytics).
Between the Vendor and Google, it does correlate to a branch/release of the AOSP platform. 

Your other question on OS 4.3 vs 4.4 and what are they key differences?
Search "Android 4.4 vs Android 4.3 feature list". It is quite significant. But as a general rule, each release typically comes with new API's for developers, improved security, and improved efficiency.