Extendable format provider
public IDictionary&lt;Type, IFormatProvider&gt; FormatProviders { get; set; }  



This is bad. Having a public setter for the dictionary will let a user of that code pass in null resulting in the Format() method throwing a NullReferenceException.  



You don't check the passed in parameters of the Format() method if they are valid. By omiting the validation of the arguments you are exposing implementation details for the case they aren't valid. E.g if the passed object arg is null then the call to arg.GetType() will throw and exposes that you are requesting the type of that object.  

You only want to show the user that the passed in arguments are invalid, but you shouldn't let the code do it for you.