Does a CFI have to be a CFII to fly with a student in IMC?
No, a pilot with CP-ASEL-IA and CFI-ASE who is instrument current can take a student into actual IMC, and can even log it in both logbooks as dual training if the instruction is on the area of operation described in FAR 61.107(b)(1)(ix) "Basic Instrument Maneuvers" (etc) because that AOA is towards a private pilot certificate. However, this would be a Dumb Thing to Do because a CFI with no -I does not have any training flying instruments from the right seat, which is a skill that takes quite a bit of practice.

The thing that a CFI-I allows a person to do is to provide training on the areas of operation in FAR 61.65 towards an instrument rating.