Recently, online input selection has gained an increasing attention in evolving fuzzy models. In this paper, we proposed a new evolving fuzzy system referred to as evolving heterogeneous fuzzy inference system (eHFIS), which can simultaneously perform local input selection and system identification in an evolving and integrative manner. The introduced eHFIS is structured by some fuzzy rules with different effective input variables. This was achieved through inclusion of some parameters (local input selectors) in the structure of the Takagi-Sugeno system. An online learning algorithm is proposed to identify the eHFIS, where 1) the premise parameter learning and rule evolution take place with the usage of an incremental and evolving clustering for partitioning the data space; 2) a local input selection strategy based on switching to a neighboring model is adopted, and then, all fuzzy rules with the same input structure form a new category; and 3) for each category, the parameters of linear models, in consequent parts, are updated by weighted recursive fuzzily weighted least-squares estimator. The performance of the proposed eHFIS is evaluated and compared through several simulations on hand made as well as real-life datasets.