Is it possible to remove a border from a Webm video losslessly?
See here https://www.webmproject.org/docs/container/ under section Track. There you find PixelCropBottom, PixelCropTop, PixelCropLeft and PixelCropRight - all of which are supported. These parameters should do what you want. They also exist for Matroska.
Look for a tool that can manipulate those parameters.