Can I use a floor patch over this small area of slightly damaged subfloor?
I think you will be ok patching if it is still solid underfoot, as you say it is.
I would use a hard-setting plaster patch (sometimes called by trade names like Fixall, Fix-It All or Water Putty) after removing all crumbly unsound material. It can be difficult to get setting plaster smooth and level, so what I would do is use it as a base for regular floor-leveler, which can be troweled perfectly level and smooth for blend with surrounding floor. Then install your LVT on top.