Can I tell MacPorts to build against its own X11 or Xorg?
No, there's no such possibility within MacPorts. This port does not come with such a choice.

kitty is based on OpenGL, and doesn't really use ordinary UI controls from a library such as GTK, QT, Aqua or similar.

kitty is a native Mac-application, as well as existing as a native Linux application. When you compile kitty on a Mac, the backend used is Cocoa (i.e. native Mac). X11 does not come into the picture at all.

You could try manually recompiling kitty and then force it to use its X11 backend. I don't think that backend has been tested on macOS though, as the author intends the user to use the native Carbon backend.