Looking at potentially 100,000 users/entries to capture data
We've seen Craft installs with several hundred thousand users/assets/entries running along fine.

You'll just need to be smart with how you're pulling data out on the template side and use caching where appropriate and you should be fine.