How to interpret s parameters including S21 on Smith chart on VNA display
It looks like for S11, the chart plots the impedance, from which the reflection coefficient is calculated, 


The way the Smith chart works, if you took the lines of the Smith chart away, and put ordinary polar coordinate lines on the chart instead, these lines would give you the reflection coefficient.

If you then go back to the Smith chart lines instead of the polar coordinate lines, you have the impedance.

In paper and pencil days, you'd use a compass and protractor (or angle markings printed around the perimeter of the chart) to plot the reflection coefficient onto a pre-printed Smith chart. Then the Smith chart lines would give you the impedance without having to do any tedious calculations. 

So really what's going on is ADS is plotting the reflection coefficient against lines that aren't shown on the chart, and you're reading the impedance off the lines that are on the chart.


  For S21, the magnitude plot makes sence, because it goes to 0, meaning no energy from port 1 is appearing at port 2. However, I don't understand why the Smith chart would show 1 + j0 at the 5 Ghz point.


The Smith chart isn't really meaningful for transmission paramters (\$S_{21}\$ or \$S_{12}\$). Just plot these parameters on an ordinary polar plot, not a Smith chart.

The reason the chart shows "1+j0" at the point where the thing you plotted on it goes to zero is because if you were plotting reflections, then the point where the reflection goes to zero is the point where the input impedance of the DUT is matched (\$Z = (1+j0)\times Z_0\$).

But if you're plotting a transmission parameter, the only thing that's special about this point is that transmission is 0, which you would see more clearly on a polar chart than on a Smith chart.