Burned out potentiometer
simulate this circuit &ndash; Schematic created using CircuitLab

Figure 1. LED and pot smoker.

You forgot to tell us the voltage of the battery and the value of the pot. I'm going to assume 6 V and 1k.

A typical potentiometer has a power rating of about 1/8 W. That's the amount of power it can safely dissipate along the whole resistance track without overheating. We can now work out the maximum current that can safely be run through the pot. The relationship between power, current and resistance is given by the formula \$ P = I^2R \$ so \$ I_{MAX} = \sqrt {\frac {P}{R}} \$. For a 1k, 0.125 W pot that works out at \$ \sqrt {\frac {0.125}{1000}} = 0.011~A = 11~mA \$. 



Figure 2. LED current vs voltage curves. Source: Electronic Tutorials.

Next we turn to the LED. From the graph in Figure 2 we can see that once the LED turns on a small increase in applied voltage results in a large change in current. This is why LEDs are always powered from current limited supplies rather than constant voltage.


  What did I do?


You decreased the pot resistance allowing more and more current to flow through the LED. At 20 mA the LED was probably at full brightness and the pot was already at twice its rated current. By the sound of things you kept on going so you probably had 50 to 100 mA in your circuit which is above the rating for both components. Smoke is the typical result.