Use filmic and output mask with absolute color values
Ok I found a way to do it. Instead of doing an alpha over with the ID Mask between black and white, I use black and a value node. The value node is set to a high value, high enough for the filmic color space transformation to map it to the maximum value 255/#FFFFFF in the output.

This has two problems though, 1) it depends on the "alpha over"-node to accept higher values than usually available (this should be ok, since internal representation is floating point), 2) I just took some high value instead of basing it on what value is mapped to white.