Is it safe to eat baby-potatoes?
Planting will get you a better return. Cut the potatoes up into sections with a sprout each, let the cuts callus at 70 F/21C for a few days, and plant.

If you remove the sprouts and the potatoes are not green, or any green parts are removed, they should be safe, if not of particularly good quality.

Planting the sprouted ones and buying others to eat would be your best bet, better quality, and not too costly. Next year, put some apples in the root cellar with the potatoes to help prevent/reduce potato sprouting during storage. 

In the well-managed root cellar we're supposed to somehow find time to to go through every couple of weeks to check for and remove the various sprouting or spoiling food before the process gets too far along. I can't say I have arrived there myself yet...but if you at least look for the sprouted ones when you are grabbing tubers to use, you can remove small sprouts from firmish potatoes and use them up, and be aware if the whole batch is sprouting away and needs to be used up soon, before it gets to this point.