Why do manufacturers strongly discourage using X7R capacitors in AC signal and mains filtering applications?
Q1)  X7R capacitors are Class 2.  Class 2 ceramic capacitors are ferroelectric and contain dipoles.  The dipoles group together into domains where they point in the same direction.  Application of an electric field (AC or DC Voltage) will force the dipoles in the domains to try and align with the field.  This movement of the dipoles in the domain cause a release of energy in the form of heat along the walls of each domain.  Full polarity reversal of the field (AC Voltage) will generate the most heat because the dipoles have to completely flip their direction each time.  Continually applying this condition will heat up the component until it eventually fails.  It is very similar to hysteresis losses in a ferromagnetic core.

Q2)  The conditions where this will not lead to failure are difficult to define, which is why the manufacturers completely avoid the situation and put the statement you mentioned in their datasheets.  It is a function of frequency, voltage, footprint size, and the physical construction of the capacitor.  It is particularly bad below 1kHz because this gives the dipoles time to flip with each polarity reversal and build up heat.  At higher frequencies, it is more difficult for the dipoles to flip quickly enough and less heat is generated as a result.  Again, it is difficult for them to clearly define it in a way that they can guarantee no failures.  So, they just won't recommend it.