Why do we have a limited bandwidth in electronics?
I feel like there are almost two questions here

What limits the bandwidth of electronics? (Let's focus on solid-state electronics)

Why do optical waveguides provide such large bandwidth?


Side Note: Comparing the bandwidth of electronics and that of optical fiber is somewhat odd to me. I say this because electronics, such as LEDs and LASER Diodes, are frequently used to send signals down optical fibers. It would be more common to compare optical fiber to other options for carrying electromagnetic waves such as Coaxial cable, twisted pair, metallic waveguide, free-space etc. I touch on this somewhat in point 2 below.
For 1. you can spend a long time reading up on the semiconductor physics behind this but switching on and off a transistor, LED or LASER diode is not an instantaneous process. These structures have capacitance which means there are voltages that cannot change instantaneously. This limits the rise and fall time of signals. The slower the envelope of a signal changes, the smaller its bandwidth is.
You correctly noted that dispersion will limit the bandwidth in an optical fiber. Chromatic dispersion for instance means that you can not use too broad a range of wavelengths as the components will travel at different speeds and you will get Inter-symbol interference on your channel. It's similar with say a transistor driving a length of coax, the capacitance and inductance of the circuit will result in different phase shifts for different frequency components, and therefore transmitted symbols will broaden in time and interfere with one another.
For 2. Ultimately, it is all electromagnetic waves and governed by the same fundamental equations. However, there is substantially more bandwidth available at optical frequencies than say microwave frequencies. A given device structure may support a 10% 3db-bandwidth, but 10% of 1000THz is far larger than 10% of 10GHz.