Creating an easy but not trivial dataset
You might use a Markov chain for generating a simple enough dataset. You can estimate the probabilities of punctuation by scanning a large corpus of text (e.g. wikipedia), then you can use the model to generare fictious sentences with punctuation marks. You can set the order of the Markov chain so as to make the generated dataset increasingly closer to real-world text. The Markov chain should be defined on the set of words, however, so you have to carefully choose the vocabulary.