A foundational assumption used in the development of machine learning algorithms is that the samples in a dataset are identically distributed. This means that there exists a data generating probability distribution that can produce all samples of both the training and test sets. The learning task is then to characterize this probability distribution within the structure of a model, allowing useful and accurate predictions to be made on unseen data.