Overfitting and cross-validation
It definitely can be associated with over-fitting. I would suggest plotting the training and validation accuracy in a single graph. So you see how both depend on the number of splits. If it is over-fitting one would expect that the training accuracy continues rising, while the validation accuracy gets to a maximum and then drops. 

However, you have to make sure that your training and validation sets are OK. I assume you do something like k-fold cross validation. Where every sample is k-times in a training set and one time in a validation set. And you have enough samples to train your model etc.