Freezing cold cabin housing just under a window
It's not a structural problem, and nothing to worry about. The cabin of an airliner is a pressure vessel, the pressure outside of the airplane is far lower,  if there was a leak air would be going out, not in. The air outside is very cold, which makes the skin of the airplane cold, there's insulation between the skin and the cabin fixtures but if that comes off a bit you get cold spots. 

By all means tell the cabin crew so they can log it for the next time the airplane's in for maintenance, it's important for passenger comfort.