src/estimagic/differentiation/finite_differences.py
def hessian(evals, steps, f0, method):
    """Calculate a Hessian estimate with finite differences according to method.

    Notation: f:R^dim_x -> R^dim_f. We compute the derivative at x0, with f0 = f(x0).

    The formulae in Rideout [2009] which are implemented here use three types of
    function evaluations:

    1. f(theta + delta_j e_j)
    2. f(theta + delta_j e_j + delta_k e_k)
    3. f(theta + delta_j e_j - delta_k e_k)

    Which are called here: 1. ``evals_one``, 2. ``evals_two`` and 3. ``evals_cross``,
    corresponding to the idea that we are moving in one direction, in two directions and
    in two cross directions (opposite signs). Note that theta denotes x0, delta_j the
    step size for the j-th variable and e_j the j-th standard basis vector.

    Note also that the brackets in the finite difference formulae are not arbitrary but
    improve the numerical accuracy, see Rideout [2009].

    Args:
        evals (dict[namedtuple]): Dictionary with keys "one_step" for function evals in
            a single step direction, "two_step" for evals in two steps in the same
            direction, and "cross_step" for evals in two steps in the opposite
            direction. Each dict item has the fields called pos and neg for evaluations
            with positive and negative steps, respectively. Each field is a numpy array
            of shape (n_steps, dim_f, dim_x). It contains np.nan for evaluations that
            failed or were not attempted because a one-sided derivative rule was chosen.
        steps (namedtuple): Namedtuple with the fields pos and neg. Each field
            contains a numpy array of shape (n_steps, dim_x) with the steps in
            the corresponding direction. The steps are always symmetric, in the sense
            that steps.neg[i, j] = - steps.pos[i, j] unless one of them is NaN.
        f0 (numpy.ndarray): Numpy array of length dim_f with the output of the function
            at the user supplied parameters.
        method (str): One of {"forward", "backward", "central_average", "central_cross"}
            These correspond to the finite difference approximations defined in
            equations [7, x, 8, 9] in Rideout [2009], where ("backward", x) is not found
            in Rideout [2009] but is the natural extension of equation 7 to the backward
            case.

    Returns:
        hess (numpy.ndarray): Numpy array of shape (n_steps, dim_f, dim_x, dim_x) with
            estimated Hessians. I.e. there are n_step hessian estimates.

    """
    n_steps, dim_f, dim_x = evals["one_step"].pos.shape
    f0 = f0.reshape(1, dim_f, 1, 1)

    # rename variables to increase readability in formulas
    evals_one = namedtuple_from_kwargs(
        pos=np.expand_dims(evals["one_step"].pos, axis=3),
        neg=np.expand_dims(evals["one_step"].neg, axis=3),
    )
    evals_two = evals["two_step"]
    evals_cross = evals["cross_step"]

    if method == "forward":
        outer_product_steps = _calculate_outer_product_steps(steps.pos, n_steps, dim_x)
        diffs = (evals_two.pos - evals_one.pos.swapaxes(2, 3)) - (evals_one.pos - f0)
        hess = diffs / outer_product_steps
    elif method == "backward":
        outer_product_steps = _calculate_outer_product_steps(steps.neg, n_steps, dim_x)
        diffs = (evals_two.neg - evals_one.neg.swapaxes(2, 3)) - (evals_one.neg - f0)
        hess = diffs / outer_product_steps
    elif method == "central_average":
        outer_product_steps = _calculate_outer_product_steps(steps.pos, n_steps, dim_x)
        forward = (evals_two.pos - evals_one.pos.swapaxes(2, 3)) - (evals_one.pos - f0)
        backward = (evals_two.neg - evals_one.neg.swapaxes(2, 3)) - (evals_one.neg - f0)
        hess = (forward + backward) / (2 * outer_product_steps)
    elif method == "central_cross":
        outer_product_steps = _calculate_outer_product_steps(steps.pos, n_steps, dim_x)
        diffs = (evals_two.pos - evals_cross.pos) - (evals_cross.neg - evals_two.neg)
        hess = diffs / (4 * outer_product_steps)
    else:
        raise ValueError(
            "Method has to be 'forward', 'backward', 'central_average' or ",
            "'central_cross'.",
        )
    return hess
Calculate a Hessian estimate with finite differences according to method.

    Notation: f:R^dim_x -> R^dim_f. We compute the derivative at x0, with f0 = f(x0).

    The formulae in Rideout [2009] which are implemented here use three types of
    function evaluations:

    1. f(theta + delta_j e_j)
    2. f(theta + delta_j e_j + delta_k e_k)
    3. f(theta + delta_j e_j - delta_k e_k)

    Which are called here: 1. ``evals_one``, 2. ``evals_two`` and 3. ``evals_cross``,
    corresponding to the idea that we are moving in one direction, in two directions and
    in two cross directions (opposite signs). Note that theta denotes x0, delta_j the
    step size for the j-th variable and e_j the j-th standard basis vector.

    Note also that the brackets in the finite difference formulae are not arbitrary but
    improve the numerical accuracy, see Rideout [2009].

    Args:
        evals (dict[namedtuple]): Dictionary with keys "one_step" for function evals in
            a single step direction, "two_step" for evals in two steps in the same
            direction, and "cross_step" for evals in two steps in the opposite
            direction. Each dict item has the fields called pos and neg for evaluations
            with positive and negative steps, respectively. Each field is a numpy array
            of shape (n_steps, dim_f, dim_x). It contains np.nan for evaluations that
            failed or were not attempted because a one-sided derivative rule was chosen.
        steps (namedtuple): Namedtuple with the fields pos and neg. Each field
            contains a numpy array of shape (n_steps, dim_x) with the steps in
            the corresponding direction. The steps are always symmetric, in the sense
            that steps.neg[i, j] = - steps.pos[i, j] unless one of them is NaN.
        f0 (numpy.ndarray): Numpy array of length dim_f with the output of the function
            at the user supplied parameters.
        method (str): One of {"forward", "backward", "central_average", "central_cross"}
            These correspond to the finite difference approximations defined in
            equations [7, x, 8, 9] in Rideout [2009], where ("backward", x) is not found
            in Rideout [2009] but is the natural extension of equation 7 to the backward
            case.

    Returns:
        hess (numpy.ndarray): Numpy array of shape (n_steps, dim_f, dim_x, dim_x) with
            estimated Hessians. I.e. there are n_step hessian estimates.

    
Calculate a Hessian estimate with finite differences according to method.

The formulae in Rideout [2009] which are implemented here use three types of
function evaluations.

1.

Which are called here: 1.

Note also that the brackets in the finite difference formulae are not arbitrary but
improve the numerical accuracy, see Rideout [2009].
Calculate a Hessian estimate with finite differences according to method . The formulae in Rideout [ 2009 ] which are implemented here use three types of function evaluations . 1 . Which are called here : 1 . Note also that the brackets in the finite difference formulae are not arbitrary but improve the numerical accuracy see Rideout [ 2009 ] .
def hessian(evals, steps, f0, method):
    n_steps, dim_f, dim_x = evals["one_step"].pos.shape
    f0 = f0.reshape(1, dim_f, 1, 1)
    evals_one = namedtuple_from_kwargs(
        pos=np.expand_dims(evals["one_step"].pos, axis=3),
        neg=np.expand_dims(evals["one_step"].neg, axis=3),
    )
    evals_two = evals["two_step"]
    evals_cross = evals["cross_step"]
    if method == "forward":
        outer_product_steps = _calculate_outer_product_steps(steps.pos, n_steps, dim_x)
        diffs = (evals_two.pos - evals_one.pos.swapaxes(2, 3)) - (evals_one.pos - f0)
        hess = diffs / outer_product_steps
    elif method == "backward":
        outer_product_steps = _calculate_outer_product_steps(steps.neg, n_steps, dim_x)
        diffs = (evals_two.neg - evals_one.neg.swapaxes(2, 3)) - (evals_one.neg - f0)
        hess = diffs / outer_product_steps
    elif method == "central_average":
        outer_product_steps = _calculate_outer_product_steps(steps.pos, n_steps, dim_x)
        forward = (evals_two.pos - evals_one.pos.swapaxes(2, 3)) - (evals_one.pos - f0)
        backward = (evals_two.neg - evals_one.neg.swapaxes(2, 3)) - (evals_one.neg - f0)
        hess = (forward + backward) / (2 * outer_product_steps)
    elif method == "central_cross":
        outer_product_steps = _calculate_outer_product_steps(steps.pos, n_steps, dim_x)
        diffs = (evals_two.pos - evals_cross.pos) - (evals_cross.neg - evals_two.neg)
        hess = diffs / (4 * outer_product_steps)
    else:
        raise ValueError(
            "Method has to be 'forward', 'backward', 'central_average' or ",
            "'central_cross'.",
        )
    return hess
def hessian ( evals , steps , f0 , method ) : n_steps , dim_f , dim_x = evals [ "one_step" ] . pos . shape f0 = f0 . reshape ( 1 , dim_f , 1 , 1 ) evals_one = namedtuple_from_kwargs ( pos = np . expand_dims ( evals [ "one_step" ] . pos , axis = 3 ) , neg = np . expand_dims ( evals [ "one_step" ] . neg , axis = 3 ) , ) evals_two = evals [ "two_step" ] evals_cross = evals [ "cross_step" ] if method == "forward" : outer_product_steps = _calculate_outer_product_steps ( steps . pos , n_steps , dim_x ) diffs = ( evals_two . pos - evals_one . pos . swapaxes ( 2 , 3 ) ) - ( evals_one . pos - f0 ) hess = diffs / outer_product_steps elif method == "backward" : outer_product_steps = _calculate_outer_product_steps ( steps . neg , n_steps , dim_x ) diffs = ( evals_two . neg - evals_one . neg . swapaxes ( 2 , 3 ) ) - ( evals_one . neg - f0 ) hess = diffs / outer_product_steps elif method == "central_average" : outer_product_steps = _calculate_outer_product_steps ( steps . pos , n_steps , dim_x ) forward = ( evals_two . pos - evals_one . pos . swapaxes ( 2 , 3 ) ) - ( evals_one . pos - f0 ) backward = ( evals_two . neg - evals_one . neg . swapaxes ( 2 , 3 ) ) - ( evals_one . neg - f0 ) hess = ( forward + backward ) / ( 2 * outer_product_steps ) elif method == "central_cross" : outer_product_steps = _calculate_outer_product_steps ( steps . pos , n_steps , dim_x ) diffs = ( evals_two . pos - evals_cross . pos ) - ( evals_cross . neg - evals_two . neg ) hess = diffs / ( 4 * outer_product_steps ) else : raise ValueError ( "Method has to be 'forward', 'backward', 'central_average' or " , "'central_cross'." , ) return hess
Calculate a Hessian estimate with finite differences according to method.
Calculate a Hessian estimate with finite differences according to method .
"""Calculate a Hessian estimate with finite differences according to method.

    Notation: f:R^dim_x -> R^dim_f. We compute the derivative at x0, with f0 = f(x0).

    The formulae in Rideout [2009] which are implemented here use three types of
    function evaluations:

    1. f(theta + delta_j e_j)
    2. f(theta + delta_j e_j + delta_k e_k)
    3. f(theta + delta_j e_j - delta_k e_k)

    Which are called here: 1. ``evals_one``, 2. ``evals_two`` and 3. ``evals_cross``,
    corresponding to the idea that we are moving in one direction, in two directions and
    in two cross directions (opposite signs). Note that theta denotes x0, delta_j the
    step size for the j-th variable and e_j the j-th standard basis vector.

    Note also that the brackets in the finite difference formulae are not arbitrary but
    improve the numerical accuracy, see Rideout [2009].

    Args:
        evals (dict[namedtuple]): Dictionary with keys "one_step" for function evals in
            a single step direction, "two_step" for evals in two steps in the same
            direction, and "cross_step" for evals in two steps in the opposite
            direction. Each dict item has the fields called pos and neg for evaluations
            with positive and negative steps, respectively. Each field is a numpy array
            of shape (n_steps, dim_f, dim_x). It contains np.nan for evaluations that
            failed or were not attempted because a one-sided derivative rule was chosen.
        steps (namedtuple): Namedtuple with the fields pos and neg. Each field
            contains a numpy array of shape (n_steps, dim_x) with the steps in
            the corresponding direction. The steps are always symmetric, in the sense
            that steps.neg[i, j] = - steps.pos[i, j] unless one of them is NaN.
        f0 (numpy.ndarray): Numpy array of length dim_f with the output of the function
            at the user supplied parameters.
        method (str): One of {"forward", "backward", "central_average", "central_cross"}
            These correspond to the finite difference approximations defined in
            equations [7, x, 8, 9] in Rideout [2009], where ("backward", x) is not found
            in Rideout [2009] but is the natural extension of equation 7 to the backward
            case.

    Returns:
        hess (numpy.ndarray): Numpy array of shape (n_steps, dim_f, dim_x, dim_x) with
            estimated Hessians. I.e. there are n_step hessian estimates.

    """ # rename variables to increase readability in formulas
hess (numpy.ndarray): Numpy array of shape (n_steps, dim_f, dim_x, dim_x) with
estimated Hessians. I.e. there are n_step hessian estimates.
hess ( numpy . ndarray ) : Numpy array of shape ( n_steps dim_f dim_x dim_x ) with estimated Hessians . I . e . there are n_step hessian estimates .
evals
Dictionary with keys "one_step" for function evals in
a single step direction, "two_step" for evals in two steps in the same
direction, and "cross_step" for evals in two steps in the opposite
direction. Each dict item has the fields called pos and neg for evaluations
with positive and negative steps, respectively. Each field is a numpy array
of shape (n_steps, dim_f, dim_x). It contains np.nan for evaluations that
failed or were not attempted because a one-sided derivative rule was chosen.
Dictionary with keys " one_step " for function evals in a single step direction " two_step " for evals in two steps in the same direction and " cross_step " for evals in two steps in the opposite direction . Each dict item has the fields called pos and neg for evaluations with positive and negative steps respectively . Each field is a numpy array of shape ( n_steps dim_f dim_x ) . It contains np . nan for evaluations that failed or were not attempted because a one - sided derivative rule was chosen .
steps
Namedtuple with the fields pos and neg. Each field
contains a numpy array of shape (n_steps, dim_x) with the steps in
the corresponding direction. The steps are always symmetric, in the sense
that steps.neg[i, j] = - steps.pos[i, j] unless one of them is NaN.
Namedtuple with the fields pos and neg . Each field contains a numpy array of shape ( n_steps dim_x ) with the steps in the corresponding direction . The steps are always symmetric in the sense that steps . neg [ i j ] = - steps . pos [ i j ] unless one of them is NaN .
f0
Numpy array of length dim_f with the output of the function
at the user supplied parameters.
Numpy array of length dim_f with the output of the function at the user supplied parameters .
method
One of {"forward", "backward", "central_average", "central_cross"}
These correspond to the finite difference approximations defined in
equations [7, x, 8, 9] in Rideout [2009], where ("backward", x) is not found
in Rideout [2009] but is the natural extension of equation 7 to the backward
case.
One of { " forward " " backward " " central_average " " central_cross " } These correspond to the finite difference approximations defined in equations [ 7 x 8 9 ] in Rideout [ 2009 ] where ( " backward " x ) is not found in Rideout [ 2009 ] but is the natural extension of equation 7 to the backward case .