Majorly refactored my code, is this good enough?
Personally I'd make the UserFinder class source-independent.
This could be either by seeing the Xml and Test sources as a mapper and the UserFinder as a model (some kind of DataMapper):

public interface UserMapperInterface {}
public class XmlSource : UserMapperInterface { // ... }
public class TestSource : UserMapperInterface { // ... }

public class UserFinder {
     public UserFinder(UserMapperInterface $source) { // ... }
     public UserFinder() { // use default source }
}


This way you make the GetUserById method source-independent. Basic usage would be:

UserFinder t = new UserFinder(new TestSource());
// UserFinder t = new UserFinder(); // using the default source
t.GetUserById(43);


This way you seperate the source logic from the operations and you can easily add new sources without touching the UserFinder class.