On Kilmar Tor's northern flank is Twelve Men's Moor with Trewortha Tor and Hawk's Tor beyond the saddle. To the southeast is Bearah Tor and, to the south, Langstone Downs. Withy Brook runs roughly north to south past the western end of the hill and, to the east, open moorland descends to the valley of the River Lynher.