Is there any way set the render border to be a different shape?
The output image will always be rectangular.
But you can mask some parts out using the holdout shader.


Add a mask in front of your camera - it can be a simple plane with a hole
Give it the Holdout shader
In Render Tab check Film > Transparent


Your masked area (and background) will become fully transparent, and will render waaay faster.