class_docstr: Experiment which uses one experiment for the optimization,
  and another for the evaluation
docstr: Returns a list of MultiExperiment, corresponding to MOO cross-validation:
  Each experiments consist in optimizing all but one of the input ExperimentFunction's,
  and then considering that the score is the performance of the best solution in the
  approximate Pareto front for the excluded ExperimentFunction.
  
  Parameters
  ----------
  experiments: sequence of ExperimentFunction
      iterable of experiment functions, used for creating the crossvalidation.
  training_only_experiments: sequence of ExperimentFunction
      iterable of experiment functions, used only as training functions in the crossvalidation and never for test..
  pareto_size: int
      if provided, selects a subset of the full pareto front with the given maximum size
  subset: str
      method for selecting the subset (see optimizer.pareto_front)
docstr_tok: ["Returns","a","list","of","MultiExperiment",",","corresponding","to","MOO","cross-validation",":","Each","experiments","consist","in","optimizing","all","but","one","of","the","input","ExperimentFunction","'s",",","and","then","considering","that","the","score","is","the","performance","of","the","best","solution","in","the","approximate","Pareto","front","for","the","excluded","ExperimentFunction",".","Parameters","--","--","--","--","--","experiments",":","sequence","of","ExperimentFunction","iterable","of","experiment","functions",",","used","for","creating","the","crossvalidation",".","training_only_experiments",":","sequence","of","ExperimentFunction","iterable","of","experiment","functions",",","used","only","as","training","functions","in","the","crossvalidation","and","never","for","test","..","pareto_size",":","int","if","provided",",","selects","a","subset","of","the","full","pareto","front","with","the","given","maximum","size","subset",":","str","method","for","selecting","the","subset","(","see","optimizer.pareto_front",")"]
code: def create_crossvalidation_experiments(
      cls,
      experiments: tp.List[base.ExperimentFunction],
      training_only_experiments: tp.Sequence[
          base.ExperimentFunction
      ] = (),
      pareto_size: int = 12,
      pareto_subset_methods: tp.Sequence[str] = (
          "random",
          "loss-covering",
          "EPS",
          "domain-covering",
          "hypervolume",
      ),
  ) -> tp.List["SpecialEvaluationExperiment"]:
      """Returns a list of MultiExperiment, corresponding to MOO cross-validation:
      Each experiments consist in optimizing all but one of the input ExperimentFunction's,
      and then considering that the score is the performance of the best solution in the
      approximate Pareto front for the excluded ExperimentFunction.
  
      Parameters
      ----------
      experiments: sequence of ExperimentFunction
          iterable of experiment functions, used for creating the crossvalidation.
      training_only_experiments: sequence of ExperimentFunction
          iterable of experiment functions, used only as training functions in the crossvalidation and never for test..
      pareto_size: int
          if provided, selects a subset of the full pareto front with the given maximum size
      subset: str
          method for selecting the subset (see optimizer.pareto_front)
  
      """
      funcs: tp.List["SpecialEvaluationExperiment"] = []
      if "PYTEST_NEVERGRAD" in os.environ:
          pareto_subset_methods = ("random",)  # override for speed
      for pareto_subset in pareto_subset_methods:
          params: tp.Dict[str, tp.Any] = dict(
              pareto_size=pareto_size, pareto_subset=pareto_subset
          )
          for eval_xp in experiments:
              trainxps = [xp for xp in experiments if xp != eval_xp]
              if training_only_experiments is not None:
                  trainxps += training_only_experiments
              if len(trainxps) == 1:  # singleobjective
                  experiment = trainxps[0]
              else:  # multiobjective
                  # uses origin as upper bound
                  param = eval_xp.parametrization
                  upper_bounds = [
                      xp(*param.args, **param.kwargs) for xp in trainxps
                  ]
                  experiment = base.MultiExperiment(trainxps, upper_bounds)  # type: ignore
              funcs.append(cls(experiment, eval_xp, **params))
      return funcs
  
file_path: nevergrad/nevergrad/functions/helpers.py
imports: import os
  import nevergrad.common.typing
  from nevergrad.parametrization import parameter
  from nevergrad.optimization import multiobjective
  from .None import base
label_desc: Use image node_id 7 for calling the SpecialEvaluationExperiment obj's underlying member method code with example usage: obj.create_crossvalidation_experiments(cls, experiments, training_only_experiments, pareto_size, pareto_subset_methods) and returns: funcs