Is a capacitor considered a solid state device?
For the 1st half or so of the 20th century, vacuum tubes (or "valves") were used for rectification (diodes), amplification (triodes, tetrodes, pentodes, etc.), and, for most of the 20th century, displays (CRTs).

In these devices, a space current of electrons exists between the cathode and anode, i.e., the charge carriers are transported through a vacuum*.

In the 2nd half of the 20th century, semiconductor technology advanced and rapidly rendered vacuum technology obsolete (except for CRTs).  In semiconductors, the charge carriers are transported through the crystal lattice of the solid semiconductor, thus the term "solid-state" to distinguish these types of devices from the vacuum devices.

Generally, this is the context in which "solid-state" is used and understood.

In the case of capacitors, the dielectric doesn't support charge transport, i.e., there isn't a flow of charge carriers through the dielectric in a properly functioning capacitor so the dielectric falls out of the scope of the definition you've quoted.

*with some exceptions.