Caching 1600 integers and searching from treeSet takes a lot of time
You claim the lookup in the TreeSet takes a long time, why don't you use a HashSet which guarantees constant time instead of the log(n) time of TreeSet.

Also did you profile your code to verify that it is the lookup that is taking so much time? And that it isn't something else?