src/estimagic/batch_evaluators.py
def pathos_mp_batch_evaluator(
    func,
    arguments,
    n_cores=N_CORES,
    error_handling="continue",
    unpack_symbol=None,
):
    """Batch evaluator based on pathos.multiprocess.ProcessPool

    This uses a patched but older version of python multiprocessing that replaces
    pickling with dill and can thus handle decorated functions.

    Args:
        func (Callable): The function that is evaluated.
        arguments (Iterable): Arguments for the functions. Their interperation
            depends on the unpack argument.
        n_cores (int): Number of cores used to evaluate the function in parallel.
            Value below one are interpreted as one. If only one core is used, the
            batch evaluator disables everything that could cause problems, i.e. in that
            case func and arguments are never pickled and func is executed in the main
            process.
        error_handling (str): Can take the values "raise" (raise the error and stop all
            tasks as soon as one task fails) and "continue" (catch exceptions and set
            the traceback of the raised exception.
            KeyboardInterrupt and SystemExit are always raised.
        unpack_symbol (str or None). Can be "**", "*" or None. If None, func just takes
            one argument. If "*", the elements of arguments are positional arguments for
            func. If "**", the elements of arguments are keyword arguments for func.


    Returns:
        list: The function evaluations.

    """
    if not pathos_is_available:
        raise NotImplementedError(
            "To use the pathos_mp_batch_evaluator, install pathos with "
            "conda install -c conda-forge pathos."
        )

    _check_inputs(func, arguments, n_cores, error_handling, unpack_symbol)
    n_cores = int(n_cores)

    reraise = error_handling == "raise"

    @unpack(symbol=unpack_symbol)
    @catch(default="__traceback__", reraise=reraise)
    def internal_func(*args, **kwargs):
        return func(*args, **kwargs)

    if n_cores <= 1:
        res = [internal_func(arg) for arg in arguments]
    else:
        p = ProcessPool(nodes=n_cores)
        try:
            res = p.map(internal_func, arguments)
        except Exception as e:
            p.terminate()
            raise e

    return res
Batch evaluator based on pathos.multiprocess.ProcessPool

    This uses a patched but older version of python multiprocessing that replaces
    pickling with dill and can thus handle decorated functions.

    Args:
        func (Callable): The function that is evaluated.
        arguments (Iterable): Arguments for the functions. Their interperation
            depends on the unpack argument.
        n_cores (int): Number of cores used to evaluate the function in parallel.
            Value below one are interpreted as one. If only one core is used, the
            batch evaluator disables everything that could cause problems, i.e. in that
            case func and arguments are never pickled and func is executed in the main
            process.
        error_handling (str): Can take the values "raise" (raise the error and stop all
            tasks as soon as one task fails) and "continue" (catch exceptions and set
            the traceback of the raised exception.
            KeyboardInterrupt and SystemExit are always raised.
        unpack_symbol (str or None). Can be "**", "*" or None. If None, func just takes
            one argument. If "*", the elements of arguments are positional arguments for
            func. If "**", the elements of arguments are keyword arguments for func.


    Returns:
        list: The function evaluations.

    
Batch evaluator based on pathos.multiprocess.ProcessPool
This uses a patched but older version of python multiprocessing that replaces
pickling with dill and can thus handle decorated functions.

func (Callable): The function that is evaluated.
arguments (Iterable): Arguments for the functions. Their interperation
depends on the unpack argument.
n_cores (int): Number of cores used to evaluate the function in parallel.
Value below one are interpreted as one. If only one core is used, the
batch evaluator disables everything that could cause problems, i.e. in that
case func and arguments are never pickled and func is executed in the main
process.
error_handling (str): Can take the values "raise" (raise the error and stop all
tasks as soon as one task fails) and "continue" (catch exceptions and set
the traceback of the raised exception.

The function evaluations.
Batch evaluator based on pathos . multiprocess . ProcessPool This uses a patched but older version of python multiprocessing that replaces pickling with dill and can thus handle decorated functions . func ( Callable ) : The function that is evaluated . arguments ( Iterable ) : Arguments for the functions . Their interperation depends on the unpack argument . n_cores ( int ) : Number of cores used to evaluate the function in parallel . Value below one are interpreted as one . If only one core is used the batch evaluator disables everything that could cause problems i . e . in that case func and arguments are never pickled and func is executed in the main process . error_handling ( str ) : Can take the values " raise " ( raise the error and stop all tasks as soon as one task fails ) and " continue " ( catch exceptions and set the traceback of the raised exception . The function evaluations .
def pathos_mp_batch_evaluator(
    func,
    arguments,
    n_cores=N_CORES,
    error_handling="continue",
    unpack_symbol=None,
):
    if not pathos_is_available:
        raise NotImplementedError(
            "To use the pathos_mp_batch_evaluator, install pathos with "
            "conda install -c conda-forge pathos."
        )
    _check_inputs(func, arguments, n_cores, error_handling, unpack_symbol)
    n_cores = int(n_cores)
    reraise = error_handling == "raise"
    @unpack(symbol=unpack_symbol)
    @catch(default="__traceback__", reraise=reraise)
    def internal_func(*args, **kwargs):
        return func(*args, **kwargs)
    if n_cores <= 1:
        res = [internal_func(arg) for arg in arguments]
    else:
        p = ProcessPool(nodes=n_cores)
        try:
            res = p.map(internal_func, arguments)
        except Exception as e:
            p.terminate()
            raise e
    return res
def pathos_mp_batch_evaluator ( func , arguments , n_cores = N_CORES , error_handling = "continue" , unpack_symbol = None , ) : if not pathos_is_available : raise NotImplementedError ( "To use the pathos_mp_batch_evaluator, install pathos with " "conda install -c conda-forge pathos." ) _check_inputs ( func , arguments , n_cores , error_handling , unpack_symbol ) n_cores = int ( n_cores ) reraise = error_handling == "raise" @ unpack ( symbol = unpack_symbol ) @ catch ( default = "__traceback__" , reraise = reraise ) def internal_func ( * args , ** kwargs ) : return func ( * args , ** kwargs ) if n_cores <= 1 : res = [ internal_func ( arg ) for arg in arguments ] else : p = ProcessPool ( nodes = n_cores ) try : res = p . map ( internal_func , arguments ) except Exception as e : p . terminate ( ) raise e return res
Batch evaluator based on pathos.multiprocess.ProcessPool
This uses a patched but older version of python multiprocessing that replaces
pickling with dill and can thus handle decorated functions.
Batch evaluator based on pathos . multiprocess . ProcessPool This uses a patched but older version of python multiprocessing that replaces pickling with dill and can thus handle decorated functions .
"""Batch evaluator based on pathos.multiprocess.ProcessPool

    This uses a patched but older version of python multiprocessing that replaces
    pickling with dill and can thus handle decorated functions.

    Args:
        func (Callable): The function that is evaluated.
        arguments (Iterable): Arguments for the functions. Their interperation
            depends on the unpack argument.
        n_cores (int): Number of cores used to evaluate the function in parallel.
            Value below one are interpreted as one. If only one core is used, the
            batch evaluator disables everything that could cause problems, i.e. in that
            case func and arguments are never pickled and func is executed in the main
            process.
        error_handling (str): Can take the values "raise" (raise the error and stop all
            tasks as soon as one task fails) and "continue" (catch exceptions and set
            the traceback of the raised exception.
            KeyboardInterrupt and SystemExit are always raised.
        unpack_symbol (str or None). Can be "**", "*" or None. If None, func just takes
            one argument. If "*", the elements of arguments are positional arguments for
            func. If "**", the elements of arguments are keyword arguments for func.


    Returns:
        list: The function evaluations.

    """
func
arguments
n_cores
error_handling
unpack_symbol