Is a helicopter ride unsafe for young children?
A few notes of update.


Evidence from other operators that there is no hard age limit



  There is no upper or lower age limit for pleasure flight however we recommend that children under 12 are accompanied by an adult.
  Central Helicopters T&amp;C
  
  Generally anyone can be a passenger on a pleasure flight. Young children should be accompanied by an adult.
  Hields Aviation FAQ



GUIDANCE FOR EVENT ORGANISERS
AND HELICOPTER OPERATORS AT SPECIAL EVENTS covers Local flights (They call it A to A) and makes no special mention of minimum or maximum age limits. You can be fairly sure this document would be where any such specification would be made. Along with CAP 789, which is linked below, which also has no special mention of age limits for flights on helicopters


It is very hard to prove a negative, but I hope this is enough to convince you. Operators are under no obligation to justify their rules.



Original answer follows:

There is nothing inherently stopping an infant taking a flight on board a helicopter.

I suspect this has everything to do with the operators of these flights do not have the equipment to properly ensure the security and safety of infants. 

Airlines typically provide extender belts for infants on laps. They also have provision for dealing with emergency situations with infants on board. This is because it is what their customers require and makes commercial sense to provide those services.

CAP 789 (which is guidance for operators) has something to say about this (emphasis mine)


  EU-OPS, JAR-OPS 3 and the ANO require the aircraft commander to ensure that at
  certain times each passenger on board occupies a seat or berth with his safety belt
  (or harness where provided) properly secured. These regulations also require the
  aircraft commander to ensure that multiple occupancy of seats does not occur except
  by one adult and one infant who is properly secured by a supplementary loop belt or
  other restraint device.


The clientele of helicopter pleasure flights probably do not regularly require bringing infants aboard. So there is little requirement or commercial pressure for them to do so. 

The "must be 4 years of age" is quite possibly entirely arbitrary - in that they are probably large enough to use the standard harnesses/belts/noise protection provided. Operators probably do not routinely carry things like infant noise protection, and helicopters are noisy environments.

Some child restraint systems used in cars are appropriate for use in aircraft, and some operators may have allowed you to bring your car child seat on board for use with your infant (More info is available here). You would need to call around a few operators to find out.