1. A computer implemented constructive response grading method for spoken language grading, the method implemented using a processor, the method comprising: collecting, through a communication network, electronically recorded speech samples, wherein the electronically recorded speech samples include a plurality of responses of one or more candidates from a plurality of data processing devices associated with the one or more candidates; providing, by a crowdsourcing engine, the electronically recorded speech samples to crowdsource volunteers; receiving, by the crowdsourcing engine, a crowdsourced transcription and crowd scores from each of the crowdsource volunteers for one or more of a plurality of features for each of the plurality of responses of the one or more candidates, wherein the plurality of features are a function of pronunciation, fluency, content organization, and grammar; automatically combining, with a transcription combination algorithm, crowdsourced transcriptions from each of the crowdsource volunteers to form a combined transcription; generating, by a feature generation module, for each of the plurality of responses of the one or more candidates, individual scores for a first feature generated pertaining to grades given by the crowdsource volunteers and second features generated by force aligning free speech on the crowdsourced transcriptions and natural language processing features from the crowdsourced transcriptions; generating, by a machine learning module, a composite score for each of the plurality of responses of the one or more candidates based on the individual scores for each of the plurality of features together with the crowd scores given by the crowdsource volunteers; communicating the individual scores for each of the plurality of features and the composite score through a communication interface to each candidate of the one or more candidates; providing the crowdsource volunteers with gold standard tasks with known answers; comparing crowdsource answers from the crowdsource volunteers to the known answers; assigning a respective risk level to each of the crowdsource volunteers based on a comparison of the crowdsource answers to the known answers; and responsive to the respective risk level exceeding a predetermined risk threshold, automatically removing a corresponding crowdsource volunteer from evaluations of the electronically recorded speech samples.
1. A computer implemented constructive response grading method for spoken language grading, the method implemented using a processor, the method comprising: collecting, through a communication network, electronically recorded speech samples, wherein the electronically recorded speech samples include a plurality of responses of one or more candidates from a plurality of data processing devices associated with the one or more candidates; providing, by a crowdsourcing engine, the electronically recorded speech samples to crowdsource volunteers; receiving, by the crowdsourcing engine, a crowdsourced transcription and crowd scores from each of the crowdsource volunteers for one or more of a plurality of features for each of the plurality of responses of the one or more candidates, wherein the plurality of features are a function of pronunciation, fluency, content organization, and grammar; automatically combining, with a transcription combination algorithm, crowdsourced transcriptions from each of the crowdsource volunteers to form a combined transcription; generating, by a feature generation module, for each of the plurality of responses of the one or more candidates, individual scores for a first feature generated pertaining to grades given by the crowdsource volunteers and second features generated by force aligning free speech on the crowdsourced transcriptions and natural language processing features from the crowdsourced transcriptions; generating, by a machine learning module, a composite score for each of the plurality of responses of the one or more candidates based on the individual scores for each of the plurality of features together with the crowd scores given by the crowdsource volunteers; communicating the individual scores for each of the plurality of features and the composite score through a communication interface to each candidate of the one or more candidates; providing the crowdsource volunteers with gold standard tasks with known answers; comparing crowdsource answers from the crowdsource volunteers to the known answers; assigning a respective risk level to each of the crowdsource volunteers based on a comparison of the crowdsource answers to the known answers; and responsive to the respective risk level exceeding a predetermined risk threshold, automatically removing a corresponding crowdsource volunteer from evaluations of the electronically recorded speech samples. 5. The method of claim 1 further comprising: responsive to the respective risk level exceeding the predetermined risk threshold, sending a notification to a corresponding crowdsource volunteer wherein the notification includes a suggested corrective course of action.