Does milk make cakes lighter or tougher?
Some people say that it gives a lighter texture, while others say it
  gives a more rubbery/tougher texture (due to more gluten being
  developed).


I would challenge the notion that these two aspects are necessarily opposed.  Lighter doesn't necessarily mean "softer" or "more tender"; in cakes, it generally means "rising higher with more air."  The way that cakes rise higher with more air is partly by having a stronger gluten structure, along with more gas released during baking to inflate the batter.  Yes, having a stronger gluten structure will increase "toughness," but cakes already tend to have a very high amount of fat and sugar, which both serve to soften the texture.  The two options being compared here are not "lighter" vs. "rubbery."  Rather, if milk is added, the cake may be both lighter and stronger in structure, which may make it slightly more "spongy," rather than a cake without milk that is denser and more crumbly.

Also, the question as asked brings up an implicit, "Compared to what?"  If one doesn't add milk, what is added to replace it?  Because most cakes need a certain amount of moisture/water, whether that comes from water or from components milk or eggs, etc.  In the CIA's Baking and Pastry book where the role of ingredients in baking is described, they state:


  Water acts to dilute or liquefy water-soluble ingredients such as
  sugar and salt. [...] In addition, water helps to develop the proteins
  in flour, necessary for proper leavening.  Water also acts as a
  leavener when it changes to steam and expands.
  
  Milk performs many of the same functions as water, but because of its
  additional components (fat, sugar, minerals, and protein), it also
  serves a number of other functions and adds flavor as well.  [...] The
  lactic acid in milk has a tightening effect on the proteins in flour
  that serves to increase stability, resulting in a product with a fine
  grain and texture.


In sum, milk, like water, will thus help to develop the proteins in flour (gluten), but that's a necessary step to some extent to produce a "light" cake, rather than a dense and crumbly one.  It adds steam which also helps the cake to rise more.  But milk, unlike plain water, also comes with things like fat and sugar that will mitigate some of the "toughness" that comes with the added structure of gluten.  The protein in milk will provide both some structure and tenderness.  Furthermore, the acid component helps create a fine structure that adds to the sensation of a softer, more cottony texture.

I think the perception that milk (and water) contribute to "toughness" or a "rubbery" texture comes out of pastry-making, where adding too much can indeed produce a tough product.  As Harold McGee writes in On Food and Cooking regarding pastry:


  Water is essential for binding the flour particles into a dough, and
  the water content is especially critical because there is so little. 
  Pastry cooks say that as little as 1/2 teaspoon/3 ml variation in
  water in 1 cup/120 gm flour can make the difference between a crumbly
  texture and a tough one.


McGee (and others) note that in pastry-making, milk is often a better choice for the reasons stated above: it contains fat and sugar and acid that improve effects on dough, in addition to the water it provides.  However, it's important to remember that cake batter is a lot different from pastry dough.  The low water content of pastry dough is essential to allow the fat/flour mixture to maintain its integrity and create the flaky layers most prized.  A little more water will cause the dough to mix too much, gum together, and remain tough after baking.

Cake batter, however, has a significantly greater amount of moisture, which (as noted above) not only develops gluten to allow a higher rise, but also produces significant steam during baking that contributes to the leavening effect.

So, the question is: if not milk, where are you getting that water from in your cake batter?  Some ingredients (like plain water) may be worse than milk in creating more gluten structure while providing less fat/sugar/acid that softens the dough, but other sources of water (like eggs, buttermilk, various forms of cream, etc.) may provide other benefits that come from their additional components.  The choice really depends on the specific type of cake, as well as the desired flavor and texture.