Having trouble understanding the iron-carbon phase diagram
For any steel with less than 0.02% carbon we should see only ferrite, and for steel with >0.02% carbon we should see only pearlite.


You have got a small misunderstanding here. The maximum solubility of carbon at $723\ \mathrm{^\circ C}$ is 0.02%. At room temperature it is very low ($0.005%$ at $0\ \mathrm{^\circ C}$). Also for carbon level between 0.02 and 0.8 you wont see only pearlite, you will see a combination of ferrite and pearlite for sure. Only at 0.8% you can observe pure pearlite(100%). Also note that the phase diagram is in such a way that if you cool from the ferrite+ austenite region at $800\ \mathrm{^\circ C}$ with carbon content &lt; 0.02% we can still get a ferrite + pearlite + even austenite/martensite.
Phase diagrams give an ideal picture. In real situations, it is very tough to get very slow cooling rates to get the equilibrium structure.