In a real discovery context, solubility prediction is usually applied towards a solute extrapolation task, wherein it is desirable to know the solubility of novel candidate compound in a variety of standard solvents for a given temperature. Thus, we stringently trained and evaluated our model performance with this task in mind. However, attention to the ability of a model to extrapolate to new solutes is not typically heeded in the literature, which makes benchmarking our model performance challenging.