what could this mean if your "elbow curve" looks like this? This is from running kmeans clustering with k on the x-axis (ranging from 2 to 10) and the silhouette distance on the y-axis. 

Clearly there's peaks at k=3, k=4 and it seems to decline from there. It doesn't resemble an elbow and thought it should rise as k gets larger (due to over fitting on he training set). Do I just lack data?  

I'm computing the silhouette distance using a 80-20 train test split.
First of all, you do have two elbows: one at $k=4$ and a large one at $k=8$. The second isn't very apparent because you haven't drawn out the plot for larger values of $k$. If you do you might get a figure like this:



Secondly, you aren't meant to look for an elbow when computing the silhouette score! The silhouette score accounts for both inter- and intra-cluster distance, as such it can be used for selecting $k$ on its own (i.e. select the $k$ that produces the best silhouette score).

Note: I'm not familiar with the "silhouette distance", I assume it is somewhat related to the silhouette score (maybe its inverse).

The "elbow" criterion should be used when dealing with metrics that tend to improve as $k$ increases (e.g. inertia).