vhost: fail backend intialization early

Don't initialize vhost backend if memslots number exceeds the supported
limit. This prevents failures down the road when backend
is actually started.

[MST: rewrite commit log]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
