Drupal Forum Solutions?
The Advanced forum module might be a good starting point, since Drupal has a modular approach you'll have to use a lot of building blocks (drupal contrib modules) to get to the extended functionality that you desire. There might be some custom code needed. In short: there is no out of the box solution for this nor a short answer to help you build this forum but here is a good start:


Has staff moderation. Volunteer moderators can be assigned. 
-> Use Workbench
Private messaging between members.
-> Use Privatemsg
Users can earn or be given “badges” and have a nice leaderboard – we would like to use it to create an "expert" vs "new user" atmosphere for Q&amp;A threads, with new contributors asking more seasoned ones (or staff moderators) for advice
-> Badge/score system might be custom, the overviews you can build with Views
[Advanced] A stub of the most popular discussion(s), i.e. linked title (# of replies) can be pulled into a relevant landing page, using categories or similar to define which forums belong to which landing pages. This is important to bring user generated content into landing pages. An extra bonus would be the feature of over-riding the 'most popular' discussion with another staff featured discussion at landing page level.
-> Probably some custom code with also views to pull the lists
Doesn't require subdomain or iframes to integrate - can reside at something like domain.com/forum/thread or domain.com/landingpage/forum/thread
-> Use PathAuto and again views for the landingpages overviews with dynamic urls
Clean and modern UI out of the box or can be themed, responsive for mobile
-> Omega's example theme Ohm might be a good starting point or you could build a child theme from scratch on omega
Can use Drupal user authentication or replace/supplement it with OpenID or Facebook/Twitter logins
-> Try OpenID connect, Facebook connect, not sure if there is something like this for twitter
Includes spam control, though this isn't a deal breaker.
-> Use Mollom or Honeypot
Comment upvoting
-> Use Vote Up/Down