Can custom nodes be written for the compositor?
While it is possible to define a pynode that can be added to existing node trees, they are considered unsupported and have limited functionality - mostly they don't get an event to automatically update. I don't think anything has changed since this previous answer.

The two main pynode projects are animation nodes and sverchok - both of these projects have implemented bpy.app.handlers that trigger an update of their node trees for various events. You will need to experiment to see if you are able to get some results using one of the render related handlers to update your node, but I don't think that any of the handlers fall between rendering and compositing.

Depending on your needs, you may be able to use a script that renders a frame, updates your node to get the result and then save the render result or viewer node output.

You may also want to try manipulating the render after it is completed instead of during compositing.

bpy.ops.render.render()
img = bpy.data.images['Viewer Node']
w = img.size[0]
h = img.size[1]
arr = np.array(img.pixels[:])
arr = arr.reshape((w,h,4))