Is there a characteristic that characterizes all, and only, metals?
The Universe is full of continuous and smoothly-transitioning properties, yet which we often stubbornly try to assign a discrete yes/no answer in order to make the world easier to cope with. Metallicity is one of them. The Van Arkel-Ketelaar triangle is a diagram which attempts to show the transition between covalent, ionic and metallic types of bonding. Here is one populated by a variety of compounds:  

The closer a substance is to one of the vertices, the more that vertex is important in describing the nature of bonding in the material. As you can see, the entire triangle is populated quite uniformly. There are certainly many more materials which would fit in the gaps between the materials shown, and even simply varying the ambient conditions (pressure, temperature, etc), a same substance can be put in different regions of the triangle.

Given this behaviour, it should be clear that any attempt to rigorously define a line dividing substances between metals and non-metals will be fraught with difficulty and likely not useful in the first place. No matter where the line is drawn, there will be some compounds that will look strange for "barely being a metal", while others will be strange for "barely being a non-metal".