How P2P search engines prevent corruption of distributed index by malicious peers?
When tackling such problems, think in general terms. Researchers have studied the impact of malicious peers in numerous papers. Most of these systems should work. 

However, define your problem clearly. Ex: What are the malicious peers trying to do in this case? do you want to:


protect your system from unwanted outsiders (security), 
protect your data from faulty peers (fault tolerance), 
protect your data from greedy peers (greedy peers), 
or others?


Also note that Kademlia is very similar to Chord, SkipNet, Pastry and others (some researchers call these networks "common prefix networks - I can't remember", others: ring-based p2p networks, etc ... Some solutions may be designed specifically for these networks.  However, I don't see why a solution made for a structured P2P networks would not work for your architecture.