Can I drastically undervolt/underpower a relay?
For the actual contacts, it's a good idea to provide a minimum wetting current to maintain good contact (the datasheet will sometimes specify a minimum load current). Other than that you can run them at whatever current you like under the maximum value (10A in your example)  

The coil should be driven at it's specified rating (24V in the example you link to)

Reference on switch contacts
Reference 2

You should really make sure that your relay has a decent datasheet that provides enough info - the one you link does not appear to have one, so I'd avoid it (unless you can google the part number and get it elsewhere)

Here are examples of the contact and coil ratings from a typical datasheet (in the contact section, note the minimum switching load and the different maximum switching ratings for AC/DC):

Contact Ratings: 

  

Coil Ratings (note the release voltage is much less than the operate voltage - once the relay has switched, it takes less power to keep it in position):