proto: Use .priv_data_size to allocate the private context

This simplifies the open functions by avoiding one function
call that needs error checking, reducing the amount of
extra bulk code.

Signed-off-by: Martin Storsjö <martin@martin.st>
