Eager loading category data from a Matrix block
Apologies, I misread the question the first time around.

When eager loading via a matrix block, you need to provide a blocktype as well as the field name.

See: https://docs.craftcms.com/v3/dev/eager-loading-elements.html#eager-loading-elements-related-to-matrix-blocks for more details