The contractarian argument is unassailable all the time it is accepted that abilities can 'acquire' an external relation to an individual, and can be treated as if they were property.
In a modern liberal-capitalist society, the employment contract is enforced while the enslavement contract is not; the former being considered valid because of its consensual/non-coercive nature, and the latter being considered inherently invalid, consensual or not.