Term for/information about manufacturing a non-rectangular PCB?
You need to supply a board outline in the CAD design file for the PCB. This gets plotted into the Gerber data along with all of the other artwork features. 

I normally setup my projects in such a way that the board outline appears in each of the artwork layers and also in a layer by itself in a layer called the outline layer. The PC board manufacturer can choose then to remove the outline from those layers where it interferes with the board manufacturing process. For example they may remove the board outline from the silkscreen layer so that there is not a sliver of the printing ink left at the routed edges of the finished board. 

You definitely do want to include a board fabrication drawing along with your Gerber data sent to the manufacturer. This can show the dimensions of the board and any other special information deemed applicable to the production of your board. Do make sure that when you select a board fab shop that you select one that will actually look at and comprehend any special instructions contained in your fabrication drawing. The drawing is often delivered in an AutoCad compatible format or can be exported to PDF format. I have personally had a bad experience with popular a prototyping shop that refused to consider a fab drawing and thus provided boards that were built based upon assumptions and had to be rejected (the shop thankfully admitted the mistake in the end and re-made the assemblies at no additional cost). 

If you ever intend to build boards in a mass production process, where pick and place machinery is used to place parts on the board and then through a re-flow oven to solder the parts, it is common that irregular shaped boards have outriggers designed into the assembly that extend the overall shape to a rectangular outline. These normally are designed with routed breakaways and mouse bite attachment to the circuit board itself so that they can be broken off the assembly after the board level assembly process is complete.

Be aware that some of the PC board houses that specialize in the lowest cost prototyping builds will only provide boards in a rectangular cutout format. That has changed some in the more recent years as more and more shops have adapted to using routing machinery to separate boards from the manufacturing panels. Most shops that do support routing will be happy to produce boards of almost any shape that you can prepare in your board outline. There may be minimum size requirements and the outline be reasonably simple. I routinely take advantage of this to prepare all of my boards with 100 mil radius rounded corners. 

Communications with the manufacturer is the all important factor here!!