How do I apply patch in vendor directory using composer?
You will need cweagans vendor to make composer able to parse the extra->patches lines. 

This line is wrong I think: "vendor/behat/web-api-extension"
You always need the package name (without version) and not the relative path.
In your case it should be "behat/behat"