1. A method for performing an experiment on a software application, the method comprising: under control of a computing system comprising computing hardware including one or more processors: during execution of an instance of a software application on the computing system, receiving, from a remote system, an allocation of a treatment configuration for an experiment for the software application, wherein the treatment configuration includes: at least one test factor; at least one test value corresponding to the at least one test factor; and a treatment distribution parameter indicating how the treatment configuration is to be distributed to one or more computing systems executing instances of the software application, wherein the treatment distribution parameter specifies a percent value indicating a percentage of running instances of the software application that should receive the treatment configuration, and wherein the treatment configuration is allocated based on the treatment distribution parameter; receiving the at least one test value corresponding to the at least one test factor associated with the treatment configuration; and modifying the instance of the software application to reference the at least one test value during execution.