To answer these questions, we use Robocrystallographer to generate descriptions for all stable materials available in the Materials Project. We use this corpus of text as the input data set for a mat2vec-like unsupervised word embedding model to obtain feature vectors encoded with structural information for all the elements of the periodic table. We then construct numerous machine learning models for a variety of materials property predictions in order to assess how valuable these feature vectors are compared to other approaches such as the composition-based feature vector or feature-free deep learning approaches. Finally, we incrementally alter the data set size to see if improvements coming from feature engineering persist into large data set scenarios, or only add value in the limit of sparse data.