1. A method comprising: extracting features from an application of a plurality of applications linked to an online system, the plurality of applications operate independently of and external to the online system, are used by users of the online system, and are configured to post content to the online system; identifying a set of features from a set of training applications from the plurality of applications, the quality scores of each of the set of features from the set of training applications being manually assigned and indicating whether each training application satisfies one or more of a set of criteria; and generating an application quality score for the application based on the quality scores assigned to the set of features of the set of training applications, the generated application quality score representing a probability of the application satisfying the set of criteria.