Trying to Get the Most Cooling Power out of TEC Peltier Element: Smaller or Larger Peltier?
Look at the datasheet, if you can't find it, have a look at the place you bought it from, see if they provide any information about the correct way and voltage to drive the Peltier module.


The easiest way to check if a larger Peltier module will work better than two smaller ones is to check what wattage each small one is rated for and whether the larger one's wattage (50x50mm is usually 60w) is larger than the sum of the two smaller ones.
If you have not already, add a heatsink, these things get really hot really fast, and you can damage them if you are not careful, if you add a fan to the heatsink, you will get even colder temperatures on the other side.
I don't know about the size you have, but you need 12v 60w to power a 50x50mm Peltier module, as I mentioned earlier, double check if the voltage and current ratings are corre4ct for your module.
If I might offer a suggestion, for prototyping, get a bench power supply, that plugs into the wall, if this is too expensive, you can always an old desktop/laptop power supply with a powerful buck-boost converter. This way, you won't have to waste money on batteries.
It is also worth mentioning that Peltier modules are not very energy efficient, so unfortunately, you probably won't get the kind of cooling that you are hoping for.