how to make a kelvin probe?
If you are talking about a Kelvin test lead- they work like a pair of clamps where each side of each clamp is brought out independently, sometimes back through four coaxial cables to four BNC connectors, sometimes just to banana plugs. 



The two sides of the clamp shown above are not connected together like they would be in an alligator clip- they are brought out to separate leads. The idea is that having separate force and sense connections allows the effects of contact resistance to be minimized. 

A "Kelvin Probe" used in a Kelvin Force Probe Microscope is completely different. If this is what you are interested in you might want to look at the design of cantilever probes for Atomic Force Microscopy. I believe there are some DIY projects along this line.