The Welsh name Pen-coed means end of the wood, and refers to the site's location at the southwestern extremity of the Wentwood forest. It was the site of a Norman castle, the property of Sir Richard de la More in 1270. The tower at the south-west corner of the extant ruins dates from the late 13th century. The manor of Llanmartin was owned by the Kemeys family around 1300, but it is not clear whether Pencoed was a separate manor at that time.