Why are HOTP and TOTP implementations all using 6 digits and not more? I was checking the RFC's after wondering for some time. In RFC 4226 - HOTP: An HMAC-Based One-Time Password Algorithm, it states on the appendix:


  A simple enhancement in terms of security would be to extract more
  digits from the HMAC-SHA-1 value.
  
  For instance, calculating the HOTP value modulo 10^8 to build an 8-
  digit HOTP value would reduce the probability of success of the
  adversary from sv/10^6 to sv/10^8.
  
  This could give the opportunity to improve usability, e.g., by
  increasing T and/or s, while still achieving a better security
  overall.  For instance, s = 10 and 10v/10^8 = v/10^7 &lt; v/10^6 which is
  the theoretical optimum for 6-digit code when s = 1.


While checking implementations, I don't realize what's the problem with extracting 8 (or more) digits from the HMAC-SHA1 result.

Is this merely a matter of improving the user experience by saving the user from writing 10 digits instead of 4? or is there a mathematical reason that I'm missing?
It is for user experience reasons, as you surmise, but the security is not compromised as much as you may think. 

Most implementations use 6 digit HOTP/TOTP schemes and design their implementation of the scheme to give them a security level they are comfortable with.

For HOTP, the key parameter that allows 6 digits to be secure enough is the throttling behavior. When combined with a sensible look-ahead window size $s$, the probability of a brute force attack succeeding is low enough as the account or client will be unable to attempt a large number of guesses.

The throttling argument for TOTP is the same, as it is based on HOTP. The security calculation differs but the same principles apply.

The converse of course is that inappropriate selection of look-ahead/behind or throttling behavior does indeed open up a 6 digit decimal OTP to brute force attacks with high probability of success.