Microcontroller GPIO ports referred to as Parallel Ports
Microcontrollers typically have several groups of GPIO pins, labeled PORTA, PORTB, PORTC etc. or something similar (perhaps P0, P1, etc.).

Each of these ports is referred to as a parallel I/O port in the manufacturers' datasheets, because all pins of the port can be acted upon in parallel by one machine instruction.  The width of the port dictated by the data path width of the microcontroller (8, 16, or 32-bits).



               (taken from the datasheet for the PIC32MX795F512L)


So the pins on the first port of an 8-bit µC might be referred to as RA0 through RA7, and on a 32-bit µC, they would be referred to as pins RA0 through RA31 (where in this example R stands for Register, the next letter is the port name, followed by the bit number -- other microcontrollers may have different naming conventions, such as P0.0 through P0.7 etc.).

Typically each port will have at least three registers associated with it (sometimes more); one to set the direction of the pins to either input or output, another to read the current input state of the digital pins, and another to set the output state of the pins. There may be another to specify whether each pin is open-drain or push-pull output.

Virtually all microcontrollers will be able to read or write to each of these registers all at once; they may or may not have the ability to read individual bits, or to write individual bits of a register without affecting the others in the port, depending on the instruction set of the processor. 

Even if a processor does not have an instruction to read or write an individual bit, I/O libraries may hide this from the user; for example the Arduino digital I/O library has a function called digitalWrite(pin, value) which is implemented on a variety of different microcontrollers, not all of which may have an instruction to write to just one bit.  In which case, it would need to either to a read-modify-write of the output register, or keep a shadow register in memory to remember the last state of the output port, so when updating just one bit, it could mask in the new status without affecting the other pins.  Likewise, Arduinos have functions to read a digital pin, and set a pin to input or output mode.

The Arduino I/O library only allows digital pins to be written individually, since the ports used will vary from one board to another and the pins may not even be assigned consecutively.  So instead, the functions just use an abstract digital pin number (0-13).  So for an Arduino, the concept of a "parallel port" doesn't really exist, which is where some of the confusion may come from.  But under its layer of abstraction, they are still there on the embedded microcontroller.

The actual port pins on the processor can be quite different from one processor to another, for example, here are the assignments for the Arduino Uno (8-bit ATmega 328P) and Arduino DUE (32-bit ARM SAM3X8E):

Pin #    Uno    DUE

0        PD0    A.8
1        PD1    A.9
2        PD2    B.25
3        PD3    C.28
4        PD4    C.26/A.29
5        PD5    C.25
6        PD6    C.24
7        PD7    C.23
8        PB0    C.22
9        PB1    C.21
10       PB2    C.29/A.28
11       PB3    D.7
12       PB4    D.8
13       PB5    B.27


So while the assignments for the Uno are straightforward (using all of Port D first, then six bits of Port B), the port assignments for the DUE are all over the place.  Two of the digital pins (4 and 10) are even connected to more than one port pin of the ARM.

Finally, there may be another set of registers on a microcontroller that allow some of the port pins to be used as analog inputs.  Any pins so designated will always be addressed individually; you cannot read a group of analog pins in parallel.  And for almost all microcontrollers, peripherals such as UARTs, I²C, SPI, etc. usually share pins with the parallel ports; when these peripherals are enabled, the pins are taken over by the peripheral and are no longer available from the parallel port registers.