Lumps in pancake batter
Bob1's answer has some of the basics -- "lumps" are mostly caused by bits of starch that don't dissolve completely in the liquid to blend with the rest of the batter.

But the potential "downsides" depend on the size of the lumps.  It is true that if they are excessively large, you can end up with loose dry flour in your pancakes, which is unpleasant.  But I usually have only found that to happen when I haven't sufficiently mixed in flour on the bottom of a mixing bowl or something.  Unless you have an exceptionally thick batter or haven't mixed up large "streaks" of flour in the batter, it's more rare to find a large free lump of dry flour floating in the middle of the batter that is big enough not to be "dissolved" during cooking.

Small lumps will usually be broken up in the cooking process as air from the leavening rises and causes circulation in the batter.  The warmer temperatures during cooking will also allow liquid to penetrate dry flour bits more readily.  (Note that most small lumps in batter that sits for any length of time won't even have dry flour at all: they'll just be "goopy" bits of dough where some moisture has penetrated, kind of like "lumps" in gravy, which won't be very noticeable in the final cooked pancakes.)

Just a note about why many sources recommend "lumpy" batter: it has to do with gluten.  If you beat your batter for a long time, more gluten chains will form.  This is great if you're trying to build up strength in a batter to make bread or something, but undesirable if you want a tender cake or pancake.  Hence, many sources will tell you to mix pancake batter only until dry flour and other dry ingredients are incorporated (i.e., no longer with dry streaks), but small lumps are still acceptable.  If you beat the batter long enough to get rid of ALL lumps, you may be toughening the batter too much, so your pancakes will be rubbery, flat, and less tender, rather than high-rising, light, and fluffy.  You will also likely be causing some of the bubbles produced by the initial burst of gas from the leavening to burst and leave the batter, which will leave the pancakes more flat and dense.

So, yes, you want to mix enough that you don't actually have huge lumps containing dry flour, but any other negative impact of small lumps is unnoticeable in the final product.  On the other hand, overmixing will generally result in dense, rubbery pancakes, probably not a desirable result.



If you're truly worried about the lumps, I can recommend a couple suggestions to mitigate them:


Mix with a whisk or similar utensil rather than a spoon.  You want a utensil that will break up bits of flour most effectively, so something with multiple strands that cuts through the dry ingredients will facilitate that.
Use lower-gluten flour, like cake flour or pastry flour.  Often cake flour is milled more finely, making it easier to dissolve small lumps in liquid.  Less gluten makes it less likely for lumps to stick together.  In addition, your final pancakes will be guaranteed to be more tender.
Let pancake batter rest.  Many recipes recommend a rest of around 15-20 minutes at room temperature, or up to an hour (or more) in the refrigerator.  This will allow time for the flour to dissolve more completely and allow most of the lumps to dissipate or at least soften before you cook the batter.  (The fridge is recommended for longer rests, as eventually gases from the leavening will start to bubble out, and the fridge will slow this process.)  A rest usually also results in more tender pancakes with a smoother texture, though be careful to add enough liquid as rested batter will generally thicken a bit.