What is the evolutionary tradeoff against primates producing multiple offsprings (litter)?
There is a tradeoff between having multiple progeny, with relatively low parental investment in each, and few progeny, with high investment in each.  This used to be called "r" and "K" selection respectively, though the terms are now used less.  Very simplistically speaking, long-lived species within a fairly stable environment tend to produce few offspring and invest large amounts of care in each; this description fits primates very well, and so it's not surprising that they follow this paradigm.  The bottom line is that because of primate life history, having small litters is a more successful evolutionary strategy.

The Wikipedia article on r/K selection theory is reasonably good and points to updates and some further reading on the subject.