Suggested edible quantity/scaling of Habanero Pepper per pound of meat
One habanero per six quarts of chili, containing approximately one quart meat, provides a solid heat that an average palate can handle. I have cooked chili on numerous occasions for groups of people and found this formula works for most people. Typically I stack it with other, lower-Scoville peppers to produce a well-bodied heat.

Other things to bear in mind:


one habanero per six quarts will not really showcase the subtler flavors of the habanero, only bring out its heat. 
capsaicin is fat soluble, more fats in the chili gives you more wiggle room to add more heat. Up the meat or oil (bacon fat), up the peppers
browning your meat in the diced peppers will lock away some flavor in the meat as long as you don't stew it forever
using the bulb's flesh, rather than up by the stem will lower the capsaicin in the pepper, discard seeds and pith as well; this will allow more pepper flavor without overwhelming spiciness
some habanero peppers are weaker than others, you might seek out some that are less spicy if trying to add heat
If all you're looking for is heat, a plurality of pepper types/cultivars will yield better heat; single pepper chili can be hotter, but have a thin kind of heat as opposed to a whole-bodied punch in the jaw kind of heat