Why does adding LIMIT to my query make it crawl?
Use correct ANSI group by (not the MySQL abomination extension) and see what happens
select sum(score) total,name,gender,dob,country  
from users join scores on users.id = scores.user_id
where date between '2012-01-01' and '2012-01-31 23:59:59'
group by name,gender,dob,country
having sum(score)&gt;=1000
order by sum(score) desc limit 50

Why?
GROUP BY in MySQL implies ORDER BY which causes the filesort here
Changing the query to be more standard may help the optimiser
More on the MySQL abominations extensions:

https://stackoverflow.com/a/6642253/27535
https://stackoverflow.com/a/10894876/27535
https://stackoverflow.com/a/6457144/27535

And from the MySQL Docs on ORDER BY Optimization

In some cases, MySQL cannot use indexes to resolve the ORDER BY, although it still uses indexes to find the rows that match the WHERE clause. These cases include the following:
...
You have different ORDER BY and GROUP BY expressions.