1. An active learning decision engine, comprising: hypotheses storage configured to store hypotheses, where a hypothesis comprises a possible solution to a problem set; equivalence class storage configured to store equivalence classes, where an equivalence class comprises at least one hypothesis in the hypotheses having a shared class of solution to the problem set; test storage configured to store a set of tests, where a test in the set of tests comprise a set of related hypotheses, a test cost, and a test outcome; observation storage configured to store observations, where an observation comprises a test result observed from the running of a test in the set of tests stored in the test storage; edge storage configured to store a set of edges, where an edge comprises a test related to a first hypothesis and a second hypothesis in the hypotheses, where the first hypothesis and the second hypothesis are in distinct equivalence classes; and a processor, where the processor is configured to: determine a plurality of equivalence classes containing one or more hypotheses based on the problem set, where the plurality of equivalence classes are a partition of the hypotheses in the hypotheses storage based on the class of the hypotheses and the possible solution to the problem set described by the hypotheses; determine a set of edges utilizing the tests, where the edges in the set of edges span equivalence classes in the plurality of equivalence classes; determine weights for the determined edges in the set of edges, where the weight for an edge in the set of edges is a measure of the test cost associated with running the test represented by the edge; select a test based on the determined weights; perform the selected test to generate at least one test result describing the likelihood that at least one hypothesis related to the test is a solution to the problem set; observe the results of the performed test, where the observed results for the performed test comprise the at least one test result associated with the performed test; remove edges from the set of edges based on the observed results; and select at least one hypothesis from the one or more hypotheses using the set of edges, where the selected at least one hypothesis is consistent with the observed results describing the likelihood that the selected at least one hypothesis is a solution to the problem set.