(Feature selection) In which cases it is legitimate to remove features manually?
Is it reasonable to remove this feature before building an estimator? I've experimented with the case when including and excluding this feature, and the later case showed a slightly(very slightly...) better performance.


This, ultimately, is the best way to get an answer. Trust in cross-validation; if you're doing it robustly and it says that ditching the feature improves performance, then ditch the feature.


  I am wondering if there is any machine learning principle, such as we should avoid 'human learning' and just let 'machine learning' do. What would be the best way to deal with this sort of features? I am currently using Scikit-learn.


Nope, human selection of features plays a part too. Some algorithms are resistant to junk features and can effectively ignore them, but they always present the potential for overfitting. The best way to deal with them is what you've done; try the model both with and without and see which performs better.