What controls the min/max brightness of a dimmer switch light
First check that your dimmer does not have a dimming range adjustment.  Several of the Lutron CFL/LED dimmers have this ability, check for an adjustment dial. LUTRON ADJUSTMENT 

But lower dimmable levels depend on what type of bulb you have (CFL, LED, incandescent, halogen).
Lutron Dimmable Bulbs identifies the following 


  What does “compatible” mean?
  Compatible means that the bulbs can be dimmed, although the bulb’s dimming performance may not
  match what you’re used to seeing when dimming an incandescent or halogen bulb. These bulbs have also
  been electrically tested to confirm that the control may be used up to its full rated CFL/LED load without
  affecting the life of the control.
  Products on this list are considered compatible with the respective Lutron C·L dimmers. Depending on the
  quality of the bulb you select, compatible bulbs may still:
  - have less dimming range than an incandescent or halogen bulb
  - dim less smoothly than an incandescent or halogen bulb
  - flicker or shimmer at certain light levels
  - buzz


As you notice LED and CFLs are not able to dim to the same levels that incandescents and halogens can achieve.  So if your using LEDs or CFLs and want to dim to a lower level you could switch to incandescent or halogen.  But if you must use LEDs or CFLs then the only way to have less "brightness" is to move to a lower lumen bulb or try changing color temperature.