"""
    nsol(F!, x0, FS, FPS, J!=diffjac!; rtol=1.e-6, atol=1.e-12,
               maxit=20, solver="newton", sham=5, armmax=10, resdec=.1,
               dx = 1.e-7, armfix=false,
               pdata = nothing, jfact = klfact,
               printerr = true, keepsolhist = false, stagnationok=false)
)

C. T. Kelley, 2020

Julia versions of the nonlinear solvers from my SIAM books.
Herewith: nsol

You must allocate storage for the function and Jacobian in advance
--> in the calling program <-- ie. in FS and FPS

Inputs:\n
- F!: function evaluation, the ! indicates that F! overwrites FS, your
    preallocated storage for the function.\n
    So FS=F!(FS,x) or FS=F!(FS,x,pdata) returns FS=F(x)

- x0: initial iterate\n

- FS: Preallocated storage for function. It is an N x 1 column vector\n
  You may store it as (n,) or (n,1), depending on what F! likes to see.

- FPS: preallocated storage for Jacobian. It is an N x N matrix\n

- J!: Jacobian evaluation, the ! indicates that J! overwrites FPS, your
    preallocated storage for the Jacobian. If you leave this out the
    default is a finite difference Jacobian.\n
    So, FP=J!(FP,FS,x) or FP=J!(FP,FS,x,pdata) returns FP=F'(x). \n
    (FP,FS, x) must be the argument list, even if FP does not need FS.
    One reason for this is that the finite-difference Jacobian
    does and that is the default in the solver.

- Precision: Lemme tell ya 'bout precision. I designed this code for
    full precision functions and linear algebra in any precision you want.
    You can declare
    FPS as Float64, Float32, or Float16 and nsol will do the right thing if
    YOU do not destroy the declaration in your J! function. I'm amazed
    that this works so easily. If the Jacobian is reasonably well
    conditioned, you can cut the cost of Jacobian factorization and
    storage in half with no loss. For large dense Jacobians and inexpensive
    functions, this is a good deal.\n
    BUT ... There is very limited support for direct sparse solvers in
    anything other than Float64. I recommend that you only use Float64
    with direct sparse solvers unless you really know what you're doing. I
    have a couple examples in the notebook, but watch out.

----------------------

Keyword Arguments (kwargs):\n

rtol and atol: relative and absolute error tolerances\n

maxit: limit on nonlinear iterations\n

solver: default = "newton"\n
Your choices are "newton" or "chord". However,
you have sham at your disposal only if you chose newton. "chord"
will keep using the initial derivative until the iterate converges,
uses the iteration budget, or the line search fails. It is not the
same as sham=Inf, which is smarter.\n

sham: default = 5 (ie Newton)\n
This is the Shamanskii method. If sham=1, you have Newton.
The iteration updates the derivative every sham iterations.
The convergence rate has local q-order sham+1 if you only count
iterations where you update the derivative. You need not
provide your own derivative function to use this option. sham=Inf
is chord only if chord is converging well.\n

I made sham=1 the default for scalar equations. For systems I'm
more aggressive and want to invest as little energy in linear algebra
as possible. So the default is sham=5.

armmax: upper bound on step size reductions in line search\n

resdec: default = .1\n
This is the target value for residual reduction.
The default value is .1. In the old MATLAB codes it was .5.
I only turn Shamanskii on if the residuals are decreasing
rapidly, at least a factor of resdec, and the line search is quiescent.
If you want to eliminate resdec from the method ( you don't ) then set
resdec = 1.0 and you will never hear from it again.

dx: default = 1.e-7\n
difference increment in finite-difference derivatives
      h=dx*norm(x,Inf)+1.e-8

armfix: default = false\n
The default is a parabolic line search (ie false). Set to true and
the step size will be fixed at .5. Don't do this unless you are doing
experiments for research.\n

pdata:\n
precomputed data for the function/Jacobian.
Things will go better if you use this rather than hide the data
in global variables within the module for your function/Jacobian

If you use pdata in either of F! or J!, you must use in in the
calling sequence of both.

jfact: default = klfact (tries to figure out best choice) \n
If your Jacobian has any special structure, please set jfact
to the correct choice for a factorization.

I use jfact when I call PrepareJac! to evaluate the
Jacobian (using your J!) and factor it. The default is to use
klfact (an internal function) to do something reasonable.
For general matrices, klfact picks lu! to compute an LU factorization
and share storage with the Jacobian.  You may change LU to something else by,
for example, setting jfact = cholseky! if your Jacobian is spd.

klfact knows about banded matrices and picks qr. You should,
however RTFM, allocate the extra two upper bands, and use jfact=qr!
to override klfact.

If you give me something that klfact does not know how to dispatch on,
then nothing happens. I just return the original Jacobian matrix and
nsol will use backslash to compute the Newton step.
I know that this is probably not optimal in your situation, so it is
good to pick something else, like jfact = lu.

If you want to manage your own factorization within your Jacobian
evaluation function, then set\n
jfact = nofact\n
and nsol will not attempt to factor your Jacobian. That is also what
happens when klfact does not know what to do. Your Jacobian is sent
directly to Julia's \\  operation

Please do not mess with the line that calls PrepareJac!.

        FPF = PrepareJac!(FPS, FS, x, ItRules)

FPF is not the same as FPS (the storage you allocate for the Jacobian)
for a reason. FPF and FPS do not have the same type, even though they
share storage. So, FPS=PrepareJac!(FPS, FS, ...) will break things.

printerr: default = true\n
I print a helpful message when the solver fails. To suppress that
message set printerr to false.

keepsolhist: default = false\n
Set this to true to get the history of the iteration in the output
tuple. This is on by default for scalar equations and off for systems.
Only turn it on if you have use for the data, which can get REALLY LARGE.

stagnationok: default = false\n
Set this to true if you want to disable the line search and either
observe divergence or stagnation. This is only useful for research
or writing a book.

Output:\n
- A named tuple (solution, functionval, history, stats, idid,
               errcode, solhist)
where

   -- solution = converged result

   -- functionval = F(solution)

   -- history = the vector of residual norms (||F(x)||) for the iteration

   -- stats = named tuple of the history of (ifun, ijac, iarm), the number
of functions/derivatives/steplength reductions at each iteration.

I do not count the function values for a finite-difference derivative
because they count toward a Jacobian evaluation.

   -- idid=true if the iteration succeeded and false if not.

   -- errcode = 0 if if the iteration succeeded

        = -1 if the initial iterate satisfies the termination criteria

        = 10 if no convergence after maxit iterations

        = 1  if the line search failed

   -- solhist:\n
This is the entire history of the iteration if you've set
keepsolhist=true\n

solhist is an N x K array where N is the length of x and K is the number of
iteration + 1. So, for scalar equations, it's a row vector.

------------------------

### Examples from the docstrings for nsol

#### World's easiest problem example. Test 64 and 32 bit Jacobians. No meaningful difference in the residual histories or the converged solutions.

```jldoctest
 julia> function f!(fv,x)
       fv[1]=x[1] + sin(x[2])
       fv[2]=cos(x[1]+x[2])
       end
f (generic function with 1 method)

julia> x=ones(2,); fv=zeros(2,); jv=zeros(2,2); jv32=zeros(Float32,2,2);
julia> nout=nsol(f!,x,fv,jv; sham=1);
julia> nout32=nsol(f!,x,fv,jv32; sham=1);
julia> [nout.history nout32.history]
5×2 Array{Float64,2}:
 1.88791e+00  1.88791e+00
 2.43119e-01  2.43120e-01
 1.19231e-02  1.19231e-02
 1.03266e-05  1.03265e-05
 1.46416e-11  1.50402e-11

julia> [nout.solution nout.solution-nout32.solution]
2×2 Array{Float64,2}:
 -7.39085e-01  5.69433e-13
  2.30988e+00  2.89546e-13
```

#### H-equation example. I'm taking the sham=5 default here, so the convergence is not quadratic. The good news is that we evaluate the Jacobian only once.

```jldoctest
julia> n=16; x0=ones(n,); FV=ones(n,); JV=ones(n,n);
julia> hdata=heqinit(x0, .5);
julia> hout=nsol(heqf!,x0,FV,JV;pdata=hdata);
julia> hout.history
4-element Array{Float64,1}:
 6.17376e-01
 3.17810e-03
 2.75227e-05
 2.35817e-07
```
"""
function nsol(
    F!,
    x0,
    FS,
    FPS,
    J! = diffjac!;
    rtol = 1.e-6,
    atol = 1.e-12,
    maxit = 20,
    solver = "newton",
    sham = 5,
    armmax = 10,
    resdec = 0.1,
    dx = 1.e-7,
    armfix = false,
    pdata = nothing,
    jfact = klfact,
    printerr = true,
    keepsolhist = false,
    stagnationok = false,
)

    itc = 0
    idid = true
    iline = false
    #
    #   If I'm letting the iteration stagnate and turning off the
    #   linesearch, then the line search cannot fail.
    #
    stagflag = stagnationok && (armmax == 0)
    #=
    First evaluation of the function. I evaluate the derivative when
    Shamanskii tells me to, at the first iteration (duh!), and when
    the rate of residual reduction is below the target value of resdec.
    =#
    (ItRules, x, n, solhist) = Newtoninit(
        x0,
        dx,
        F!,
        J!,
        solver,
        sham,
        armmax,
        armfix,
        resdec,
        maxit,
        printerr,
        pdata,
        jfact,
        keepsolhist,
    )
    #    keepsolhist ? (solhist = solhistinit(n, maxit, x)) : (solhist = [])
    #
    # First Evaluation of the function. Initialize the iteration stats.
    # Fix the tolerances for convergence and define the derivative FPF
    # outside of the main loop for scoping.
    #
    FS = EvalF!(F!, FS, x, pdata)
    resnorm = norm(FS)
    tol = rtol * resnorm + atol
    FPF = []
    ItData = ItStats(resnorm)
    newiarm = -1
    newfun = 0
    newjac = 0
    derivative_is_old = false
    residratio = 1.0
    armstop = true
    #
    # Preallocate a few vectors for the step, trial step, trial function
    #
    step = copy(x)
    xt = copy(x)
    FT = copy(x)
    #
    # If the initial iterate satisfies the termination criteria, tell me.
    #
    toosoon = (resnorm <= tol)
    #
    # The main loop stops on convergence, too many iterations, or a
    # line search failure after a derivative evaluation.
    #
    T = eltype(FPS)
    while resnorm > tol && itc < maxit && (armstop || stagnationok)
        #
        # Evaluate and factor the Jacobian.
        #
        newfun = 0
        newjac = 0
        #
        # Evaluate and factor the Jacobian if (1) you are using the chord
        # method and it's the intial iterate, or
        # (2) it's Newton and you are on the right part of the Shamaskii loop,
        # or the line search failed with a stale deriviative, or the residual
        # reduction ratio is too large. This leads to a tedious barrage
        # of conditionals that I have parked in a function.
        #
        # I'm storing the factorization in FPF unless you have asked for
        # jfact = nofact or nsol can't figure out how to factor the Jacobian.
        # In those case you just get the Jacobian back and I use \\.
        #
        evaljac = test_evaljac(ItRules, itc, newiarm, residratio)
        if evaljac
            FPF = PrepareJac!(FPS, FS, x, ItRules)
            newfun += solver == "secant"
            newjac += ~(solver == "secant")
        end
        derivative_is_old = (newjac == 0) && (solver == "newton")
        if n > 1
            # If the Jacobian precision is worse than Float32, you'll have to
            # do some scaling to avoid underflow in the terminal phase of
            # the nonlinear iteratinon. So, I do it for anything worse that
            # Float64 to make the logic simple.
            T == Float64 ? (step .= -(FPF \ FS)) :
            (ns = norm(FS, Inf); step .= -ns * (FPF \ T.(FS / ns)))
        else
            step = -FS / FPF
        end
        #
        # Compute the trial point, evaluate F and the residual norm.
        #
        AOUT = armijosc(xt, x, FT, FS, step, resnorm, ItRules, derivative_is_old)
        #
        # update solution/function value
        #
        if n > 1
            x .= AOUT.ax
            FS .= AOUT.afc
        else
            x = AOUT.ax
            FS = AOUT.afc
        end
        #
        # If the line search fails and the derivative is current,
        # stop the iteration. Print an error message unless
        # stagnationok == true and armmax=0
        #
        armstop = AOUT.idid || derivative_is_old
        iline = ~armstop && ~stagflag
        #
        # Keep the books.
        #
        residm = resnorm
        resnorm = AOUT.resnorm
        residratio = resnorm / residm
        updateStats!(ItData, newfun, newjac, AOUT)
        newiarm = AOUT.aiarm
        itc += 1
        ~keepsolhist || (@views solhist[:, itc+1] .= x)
    end
    (idid, errcode) = NewtonOK(resnorm, iline, tol, toosoon, itc, ItRules)
    newtonout = CloseIteration(x, FS, ItData, idid, errcode, keepsolhist, solhist)
    return newtonout
end


"""
    pcdc!(rbm, data)

Trains the RBM on data using Persistent Contrastive divergence, with centered gradients.
See:

J. Melchior, A. Fischer, and L. Wiskott. JMLR 17.1 (2016): 3387-3447.
<http://jmlr.org/papers/v17/14-237.html>
"""
function pcdc!(rbm::RBM, data::AbstractArray;
    batchsize = 1,
    epochs = 1,
    optim = Flux.ADAM(),
    history::MVHistory = MVHistory(),
    wts = nothing,
    steps::Int = 1,
    center_h::Bool = true, # center hidden?
    damping::Real = true # running average for λh
)
    @assert size(data) == (size(rbm.visible)..., size(data)[end])
    @assert isnothing(wts) || _nobs(data) == _nobs(wts)

    # data statistics
    stats = RBMs.suffstats(rbm.visible, data; wts)
    avg_h = RBMs.batchmean(rbm.hidden, RBMs.mean_h_from_v(rbm, data); wts)

    # initialize fantasy chains by sampling visible layer
    vm = RBMs.transfer_sample(rbm.visible, falses(size(rbm.visible)..., batchsize))

    for epoch in 1:epochs
        batches = minibatches(data, wts; batchsize = batchsize)
        Δt = @elapsed for (vd, wd) in batches
            # update fantasy chains
            vm = RBMs.sample_v_from_v(rbm, vm; steps = steps)
            # compute centered gradients
            ∂ = ∂contrastive_divergence_centered(
                rbm, vd, vm; wd, stats, center_h, damping, avg_h
            )
            # update parameters using gradient
            RBMs.update!(optim, rbm, ∂)
            # store gradient norms
            push!(history, :∂, RBMs.gradnorms(∂))
        end
        push!(history, :epoch, epoch)
        push!(history, :Δt, Δt)
        @debug "epoch $epoch/$epochs ($(round(Δt, digits=2))s)"
    end

    return history
end

function ∂contrastive_divergence_centered(
    rbm::RBM, vd::AbstractArray, vm::AbstractArray;
    wd = nothing, wm = nothing,
    stats = RBMs.suffstats(rbm.visible, vd; wts = wd),
    center_h::Bool = true, damping::Real = 0.01, avg_h::AbstractArray
)
    ∂d = RBMs.∂free_energy(rbm, vd; wts = wd, stats)
    ∂m = RBMs.∂free_energy(rbm, vm; wts = wm)
    ∂ = RBMs.subtract_gradients(∂d, ∂m)

    # extract moment estimates from the gradients
    λv = grad2mean(visible(rbm), ∂d.visible)   # <v>_d, uses full data from sufficient_statistics
    λh = grad2mean(hidden(hidden), ∂d.hidden)  # <h>_d, uses minibatch

    # since <h>_d uses minibatches, we keep a running average
    @assert size(avg_h) == size(λh)
    λh .= avg_h .= (1 - damping) * avg_h .+ damping .* λh

    ∂c = center_gradients(rbm, ∂, λv, center_h * λh)
    return ∂c
end

function center_gradients(rbm::RBM, ∂::NamedTuple, λv::AbstractArray, λh::AbstractArray)
    @assert size(rbm.visible) == size(λv)
    @assert size(rbm.hidden) == size(λh)
    @assert size(∂.w) == size(rbm.w)

    ∂wmat = reshape(∂.w, length(rbm.visible), length(rbm.hidden))
    ∂cwmat = ∂wmat - vec(λv) * vec(∂.hidden.θ)' - vec(∂.visible.θ) * vec(λh)'
    ∂cw = reshape(∂cwmat, size(rbm.w))

    shift_v = reshape(∂cwmat  * vec(λh), size(rbm.visible))
    shift_h = reshape(∂cwmat' * vec(λv), size(rbm.hidden))
    ∂cv = center_gradients(rbm.visible, ∂.visible, shift_v)
    ∂ch = center_gradients(rbm.hidden,  ∂.hidden,  shift_h)

    return (visible = ∂cv, hidden = ∂ch, w = ∂cw)
end

function center_gradients(
    layer::Union{Binary,Spin,Potts,Gaussian,ReLU,pReLU,xReLU},
    ∂::NamedTuple, λ::AbstractArray
)
    @assert size(layer) == size(∂.θ) == size(λ)
    return (∂..., θ = ∂.θ - λ)
end

function center_gradients(layer::dReLU, ∂::NamedTuple, λ::AbstractArray)
    @assert size(layer) == size(∂.θp) == size(∂.θn) == size(λ)
    return (θp = ∂.θp - λ, θn = ∂.θn - λ, γp = ∂.γp, γn = ∂.γn)
end

# get moments from layer gradients, e.g. <v> = -derivative w.r.t. θ
grad2mean(::Union{Binary,Spin,Potts,Gaussian,ReLU,pReLU,xReLU}, ∂::NamedTuple) = -∂.θ
grad2mean(::dReLU, ∂::NamedTuple) = -(∂.θp + ∂.θn)


mutable struct LinprogSolution
    status
    objval
    sol
    attrs
end

InputVector{T<:Union{Real,Char}} = Union{Vector{T},Real,Char}

function expandvec(x,len::Integer)
    if isa(x,Vector)
        if length(x) != len
            error("Input size mismatch. Expected vector of length $len but got $(length(x))")
        end
        return x
    else
        return fill(x,len)
    end
end

function no_lp_solver()
    error("""
          A linear programming solver must be specified as the final argument.

          You can use any solver listed on the solvers table of http://www.juliaopt.org/ that has a checkmark in the Linear/Quadratic column.

          A (free) example is Clp.jl. Once Clp is installed and imported via "using Clp", you can specify ClpSolver() as the solver.
          Solver options are specified by using keyword arguments to ClpSolver().
          """)
end

function buildlp(c::InputVector, A::AbstractMatrix, rowlb::InputVector, rowub::InputVector, lb::InputVector, ub::InputVector, solver::AbstractMathProgSolver)
    m = LinearQuadraticModel(solver)
    nrow,ncol = size(A)

    c = expandvec(c, ncol)
    rowlbtmp = expandvec(rowlb, nrow)
    rowubtmp = expandvec(rowub, nrow)
    lb = expandvec(lb, ncol)
    ub = expandvec(ub, ncol)

    # rowlb is allowed to be vector of senses
    if eltype(rowlbtmp) == Char
        realtype = eltype(rowubtmp)
        warn_no_inf(realtype)
        sense = rowlbtmp
        rhs = rowubtmp
        @assert realtype <: Real

        rowlb = Array{realtype}(undef, nrow)
        rowub = Array{realtype}(undef, nrow)
        for i in 1:nrow
            if sense[i] == '<'
                rowlb[i] = typemin(realtype)
                rowub[i] = rhs[i]
            elseif sense[i] == '>'
                rowlb[i] = rhs[i]
                rowub[i] = typemax(realtype)
            elseif sense[i] == '='
                rowlb[i] = rhs[i]
                rowub[i] = rhs[i]
            else
                error("Unrecognized sense '$(sense[i])'")
            end
        end
    else
        rowlb = rowlbtmp
        rowub = rowubtmp
    end

    loadproblem!(m, A, lb, ub, c, rowlb, rowub, :Min)

    return m
end

function solvelp(m)
    optimize!(m)
    stat = status(m)
    if stat == :Optimal
        attrs = Dict()
        attrs[:redcost] = getreducedcosts(m)
        attrs[:lambda] = getconstrduals(m)
        return LinprogSolution(stat, getobjval(m), getsolution(m), attrs)
    elseif stat == :Infeasible
        attrs = Dict()
        try
            attrs[:infeasibilityray] = getinfeasibilityray(m)
        catch
            @warn("Problem is infeasible, but infeasibility ray (\"Farkas proof\") is unavailable; check that the proper solver options are set.")
        end
        return LinprogSolution(stat, nothing, [], attrs)
    elseif stat == :Unbounded
        attrs = Dict()
        try
            attrs[:unboundedray] = getunboundedray(m)
        catch
            @warn("Problem is unbounded, but unbounded ray is unavailable; check that the proper solver options are set.")
        end
        return LinprogSolution(stat, nothing, [], attrs)
    else
        return LinprogSolution(stat, nothing, [], Dict())
    end
end

function linprog(c::InputVector, A::AbstractMatrix, rowlb::InputVector, rowub::InputVector, lb::InputVector, ub::InputVector, solver::AbstractMathProgSolver)
    m = buildlp(c, A, rowlb, rowub, lb, ub, solver)
    return solvelp(m)
end

linprog(c,A,rowlb,rowub, solver::AbstractMathProgSolver) = linprog(c,A,rowlb,rowub,0,Inf, solver)

buildlp(c,A,rowlb,rowub, solver::AbstractMathProgSolver) = buildlp(c,A,rowlb,rowub,0,Inf, solver)

# Old versions
linprog(c::InputVector, A::AbstractMatrix, rowlb::InputVector, rowub::InputVector, lb::InputVector, ub::InputVector) = no_lp_solver()
linprog(c,A,rowlb,rowub) = no_lp_solver()

export linprog, buildlp, solvelp


const totalname = "Total"
const cellsname = "Cells"
const errorname = "Error"
const remaindername = "Remainder"

"""
    anova(observations::Array{Union{Number, Vector{Number}}}, factortypes = FactorType[]; factornames = String[], hasreplicates = true)
    anova(observations::Vector{Number}, factorassignments::Vector{Vector{Any}}, factortypes = FactorType[]; factornames = String[], hasreplicates = true)
    anova(df::DataFrame, observationscolumn::Symbol, factorcolumns::Vector{Symbol}, factortypes = FactorType[]; factornames = String[])

Performs an Analysis of Variance (ANOVA) calculation.

Operates on up to 3 crossed factors (fixed or random) and arbitrarily many random nested factors, with or without
replicates, on balanced data.

# Arguments
- `observations`: Array containing the values to test. For the array, each dimension is a factor level, such that observations[2,5,3] indicates the 2nd level of the first factor, the 5th level of the second factor, and the 3rd level of the third factor. May contain values or vectors of values, where the vector contains replicates. Factors should be ordered with least significant first. For the vector, must provide `factorassignments` to specify factor levels.
- `factorassignments`: Vector of vectors of integers specifying how each observation is assigned to a factor level. Provide this when `observations` is given as a vector. Factor levels do not have to be consecutive or ordered. Nested factors must reuse factor levels currently.
- `factortypes`: Vector indicating the `FactorType` for each factor. If present, `replicates` must appear first, any `nested` after, and then `random` or `fixed` in any order. Specify `replicates` if the first dimension of the `observations` matrix contains replicate values (vs. contained in vectors). If too few values are provided, remaining are assumed to be `fixed`.
- `factornames`: Vector of names for each factor, excluding the replicate factor. If empty, will be automatically populated alphabetically.

Notes: The last index will be the top factor in the table.

Output: `AnovaData` structure containing the test results for each factor.

# Examples
```julia
anova(observations)                        # N-way fixed-effects ANOVA with replicates (vectors or first dimension)
anova(observations, hasreplicates = false) # N-way fixed-effects ANOVA without replicates (first dimension)
anova(observations, [random])              # N-way ANOVA with lower random factor and 1 or 2 upper fixed factors
anova(observations, [random])              # N-way ANOVA with lower random factor and 1 or 2 upper fixed factors
anova(observations, [fixed, random])       # N-way ANOVA with 1 lower fixed factor, 1 random factor, and 0 or 1 upper fixed factor
anova(observations, [nested, random])      # N-way fixed-effects ANOVA with 1 random nested factor, 1 random factor, and 1-2 fixed factors
```

# Glossary
- observation: The dependent variable.
- factor: An independent variable.
- factor level: A value of a factor.
- balanced: All combinations of factor levels have the same number of observations.
- crossed factor: A factor with levels that combine with the levels of all other crossed factors.
- fixed factor: A factor with fixed effects (e.g. treatment, concentration, exposure time).
- random factor: A factor with random effects (e.g. location, individual).
- nested factor: A random factor where the levels are unique to a combination of crossed factor levels (e.g. replicate).
- sum of squares (SS): A measure of variance that is dependent on sample size. Also called "sum of squared deviations."
- degrees of freedom (DF, ν): The number of bins in which the values could have been moved, if random.
- mean square (MS): SS / DF. Corrects for the larger variance expected if random values can be assigned to more bins. Also called "mean squared error" or "mean squared deviation."
- F-statistic: The division of MS values produce a result belonging to the "F distribution", the shape of which depends on the DF of the numerator and denominator. The location of this value on the distribution provides the p-value.
- p-value: The probability that, if all measurements had been drawn from the same population, you would obtain data at least as extreme as contained in your observations.
- effect size: The standardized difference in the measurement caused by the factor.
"""
function anova(observations::AbstractArray{T}, factortypes::Vector{FactorType} = FactorType[]; factornames::Vector{<:AbstractString} = String[], hasreplicates = true) where {T <: Union{Number, AbstractVector{<:Number}}}
    length(observations) > 0 || return

    firstlevelreplicates = eltype(observations) <: Number ? hasreplicates : false
    nfactors = ndims(observations) - (firstlevelreplicates ? 1 : 0)

    # defaults to assuming all unspecified factors are fixed
    if length(factortypes) < nfactors
        nremaining = nfactors - length(factortypes)
        append!(factortypes, repeat([fixed], nremaining))
    end

    validate(factortypes, factornames, nfactors)

    # automatically assigns alphabetical names if not provided.
    if isempty(factornames)
        factornames = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z"][1:nfactors]
        reverse!(factornames)
    end

    nnestedfactors = count(f -> f == nested, factortypes)
    crossedfactortypes = filter(f -> f ∈ [fixed, random], factortypes)
    reverse!(crossedfactortypes)
    ncrossedfactors = length(crossedfactortypes)
    ncrossedfactors < 4 || error("ANOVA with 4 or more crossed factors is not supported.")
    nreplicates = firstlevelreplicates ? size(observations, 1) : length(observations[1])
    firstlevelreplicates || all(c -> length(c) == nreplicates, observations) || throw(ErrorException("All cells must have the same number of replicates."))
    ncells = Int.(length(observations) / (firstlevelreplicates ? nreplicates : 1))
    nfactorlevels = firstlevelreplicates ? [size(observations)...][Not(1)] : [size(observations)...]

    crossedfactornames = factornames[factortypes .≠ nested]
    nestedfactornames = factornames[factortypes .== nested]

    #10kb allocated before this point
    anovakernel(observations, nreplicates, ncells, nnestedfactors, ncrossedfactors, nfactorlevels, crossedfactortypes, crossedfactornames, nestedfactornames)
end

#=
function anova(data::AnovaData, crossedfactors::Vector{Int}, )
    # performs a subtest of the specified crossed factors within level of the remaing crossed factors, using the original denominators
end

# possible bug: function anova(data::AnovaData, crossedfactors::Vector{Int}, ) with normal functions ==> hang when called?
=#

function anova(observations::AbstractVector{T}, factorassignments::AbstractVector{<:AbstractVector}, factortypes::Vector{FactorType} = FactorType[]; factornames::Vector{<:AbstractString} = String[]) where {T <: Number}
    length(observations) > 0 || return
    nfactors = length(factorassignments)
    N = length(observations)
    N % nfactors == 0 || error("Design is unbalanced.")
    all(length.(factorassignments) .== N) || error("Each observation must have an assignment for each factor.")

    factorlevels = factorassignments .|> unique .|> sort
    nfactorlevels = length.(factorlevels)
    all(N .& nfactorlevels .== 0) || error("Design is unbalanced.")
    factorlevelcounts = [[count(l -> l == factorlevels[i][j], factorassignments[i]) for j ∈ 1:nfactorlevels[i]] for i ∈ 1:nfactors]
    nperfactorlevel = factorlevelcounts .|> unique
    all(nperfactorlevel .|> length .== 1) || error("Design is unbalanced.")
    nperfactorlevel = nperfactorlevel .|> first

    if !(isa(factorassignments, Number)) || any(maximum.(factorlevels) .> nfactorlevels)
        compressedfactorlevels = [1:i for i ∈ nfactorlevels]
        factorlevelremapping = [factorlevels[i] .=> compressedfactorlevels[i] for i ∈ 1:nfactors]
        factorassignments = [replace(factorassignments[i], factorlevelremapping[i]...) for i ∈ 1:nfactors]
    end

    nreplicates = Int(N / prod(nfactorlevels))

    nlevels = [nreplicates; nfactorlevels]
    sortorder = sortperm(repeat(1:nreplicates, Int(N / nreplicates)) .+
                         sum([factorassignments[i] .* prod(nlevels[1:i]) for i ∈ 1:nfactors]))
    observationsmatrix = reshape(observations[sortorder], nlevels...)
    anova(observationsmatrix, factortypes, factornames = factornames, hasreplicates = nreplicates > 1)
end

function validate(factortypes::Vector{FactorType}, factornames::Vector{<:AbstractString}, nfactors)
    if !isempty(factortypes)
        length(factortypes) == nfactors || error("factortypes must have an entry for each factor.")
        nested ∉ factortypes || factortypes[1:count(t -> t == nested, factortypes)] |> unique |> length == 1 || error("nested entries must come before crossed factors")
    end
    if !isempty(factornames)
        nfactors ≤ 26 || error("Can only automatically name up to 26 factors. Provide names explicitly.")
        length(factornames) == nfactors || error("factornames must have an entry for each factor.")
    end
end

function anovakernel(observations, nreplicates, ncells, nnestedfactors, ncrossedfactors, nfactorlevels, crossedfactortypes, crossedfactornames, nestedfactornames)
    N = ncells * nreplicates
    nfactors = nnestedfactors + ncrossedfactors

    # collapse replicate dimension
    cellsums = eltype(observations) <: Number && nreplicates == 1 ? observations : sumfirstdim(observations)
    constant = sum(cellsums) ^ 2 / N
    total = totalcalc(observations, N, constant)
    amongallnested, crossedcellsums, ncrossedfactorlevels, nnestedfactorlevels = amongnestedfactorscalc(cellsums, nfactorlevels, nnestedfactors, nreplicates, constant)
    cells = cellscalc(cellsums, nreplicates, ncells, constant)

    npercrossedcell = nreplicates * prod(nnestedfactorlevels)

    crossedfactors = factorscalc(crossedcellsums, ncrossedfactors, ncrossedfactorlevels, N, constant, crossedfactornames) # 3kb allocated here, possibly can't be avoided
    interactions = interactionscalc(cells, crossedcellsums, crossedfactors, ncrossedfactors, ncrossedfactorlevels, nnestedfactorlevels, nreplicates, constant, crossedfactornames, npercrossedcell) # 4 kb allocated here, down from 9 kb
    nestedfactors = nestedfactorscalc(amongallnested, nnestedfactors, crossedfactors, interactions, nestedfactornames)

    reverse!(crossedfactors)
    reverse!(ncrossedfactorlevels)
    if ncrossedfactors == 3 && nreplicates > 1
        interactions[1:3] = reverse(interactions[1:3])
    end

    error = errorcalc(total, amongallnested, cells, [crossedfactors; interactions[1:end-1]], nnestedfactors, nreplicates) # could probably condense the otherfactor here
    numerators = getnumerators(crossedfactors, ncrossedfactors, nnestedfactors, nestedfactors, interactions)
    denominators = getdenominators(interactions, nestedfactors, error, crossedfactortypes, ncrossedfactors, nnestedfactors) # 2 kb allocated here

    # drop least significant test if nreplicates == 1; lowest nesting level. Lowest interaction previously dropped if no nested factors. Should look to do same for nesting.
    if nnestedfactors > 0 && nreplicates == 1
        droppedfactor = pop!(numerators)
        pop!(denominators)
    end

    # perform test
    results = ftest.(numerators, denominators)

    crossedcellmeans = crossedcellsums ./ npercrossedcell
    results = effectsizescalc(results, denominators, total, ncrossedfactors, npercrossedcell, ncrossedfactorlevels, crossedfactortypes, nnestedfactors, nnestedfactorlevels, nreplicates) # 12 kb allocated
    data = AnovaData([total; results], total, ncrossedfactors, ncrossedfactorlevels, npercrossedcell, crossedfactors, denominators[1:ncrossedfactors], crossedcellmeans) # 1 kb allocated here
    nnestedfactors > 0 && nreplicates == 1 && push!(data.effects, droppedfactor)
    error.df > 0 && push!(data.effects, error)

    return data
end

function sumfirstdim(observations::T) where {T <: AbstractArray{<:AbstractVector}}
    map(sumfirstdim, observations)
end

function sumfirstdim(observations::T) where {T <: AbstractArray{<:Number}}
    dropdims(sum(observations, dims = 1), dims = 1)
end

function sumfirstdim(observations::T) where {T <: AbstractVector{<:Number}}
    sum(observations, dims = 1) |> first
end

function totalcalc(observations, N, C)
    ss = sum(c -> sum(c.^2), observations) - C
    df = N - 1
    AnovaValue(totalname, ss, df)
end

function factorscalc(cellsums, nfactors, nfactorlevels, N, C, factornames)
    factorindices = 1:nfactors
    ss = map(i -> sum(sum(cellsums, dims = factorindices[Not(i)]) .^ 2) / (N / nfactorlevels[i]), factorindices) .- C
    df = nfactorlevels .- 1
    AnovaFactor.(factornames, ss, df)
end

function cellscalc(cellsums, nreplicates, ncells, C)
    ss = sum(cellsums .^ 2) / nreplicates - C
    df = ncells - 1
    AnovaValue(cellsname, ss, df)
end

function errorcalc(total, amongallnested, cells, otherfactors, nnestedfactors, nreplicates)
    if nnestedfactors > 0
        otherfactor = amongallnested[1]
        name = errorname
    elseif nreplicates > 1
        otherfactor = cells
        name = errorname
    else
        otherfactor = AnovaValue("", sum(f -> f.ss, otherfactors), sum(f -> f.df, otherfactors))
        name = remaindername
    end

    ss = total.ss - otherfactor.ss
    df = total.df - otherfactor.df
    AnovaFactor(name, ss, df)
end

function amongnestedfactorscalc(cellsums, nfactorlevels, nnestedfactors, nreplicates, C)
    nestedsums = cellsums
    nlowerfactorlevels = 1
    nupperfactorlevels = nfactorlevels
    amongallnested = Vector{AnovaValue}(undef, nnestedfactors)
    for i ∈ 1:nnestedfactors
        # collapse each nested factor
        ss = sum(nestedsums .^ 2 ./ (nreplicates * prod(nlowerfactorlevels))) - C
        df = prod(nupperfactorlevels) - 1
        amongallnested[i] = AnovaValue(ss, df)
        nlowerfactorlevels = nfactorlevels[1:i]
        nupperfactorlevels = nfactorlevels[(i+1):end]
        nestedsums = sumfirstdim(nestedsums)
    end
    amongallnested, nestedsums, nupperfactorlevels, nlowerfactorlevels
end

function interactionscalc(cells, nestedsums, crossedfactors, ncrossedfactors, ncrossedfactorlevels, nnestedfactorlevels, nreplicates, C, crossedfactorlabels, npercrossedcell)
    interactions = AnovaFactor[]
    if (ncrossedfactors == 2 && npercrossedcell > 1) || ncrossedfactors == 3
        pairwise = pairwisecalc(nestedsums, crossedfactors, ncrossedfactors, ncrossedfactorlevels, nnestedfactorlevels, nreplicates, C, crossedfactorlabels)
        push!(interactions, pairwise[1,2])
        if ncrossedfactors == 3
            push!(interactions, pairwise[1,3])
            push!(interactions, pairwise[2,3])
            if npercrossedcell > 1
                threewise = threewisecalc(cells, crossedfactors, pairwise, crossedfactorlabels)
                push!(interactions, threewise)
            end
        end
    end
    interactions
end

function pairwisecalc(nestedsums, crossedfactors, ncrossedfactors, ncrossedfactorlevels, nnestedfactorlevels, nreplicates, C, crossedfactorlabels)
    pairwise = Array{AnovaFactor,2}(undef, ncrossedfactors, ncrossedfactors)
    factorindexes = 1:ncrossedfactors
    for i ∈ factorindexes
        for j ∈ (i+1):ncrossedfactors
            otherfactorindexes = factorindexes[Not([i, j])]
            ss = sum(sum(nestedsums, dims = otherfactorindexes) .^ 2 ./ (prod(ncrossedfactorlevels[otherfactorindexes]) * prod(nnestedfactorlevels) * nreplicates)) - C - crossedfactors[i].ss - crossedfactors[j].ss
            df = crossedfactors[i].df * crossedfactors[j].df
            pairwise[i,j] = pairwise[j,i] = AnovaFactor("$(crossedfactorlabels[j]) × $(crossedfactorlabels[i])", ss, df)
        end
    end
    pairwise
end

function threewisecalc(cells, factors, pairwise, crossedfactorlabels)
    ss = cells.ss - sum(f -> f.ss, factors) - pairwise[1,2].ss - pairwise[1,3].ss - pairwise[2,3].ss
    df = prod(f -> f.df, factors)
    AnovaFactor("$(crossedfactorlabels[3]) × $(crossedfactorlabels[2]) × $(crossedfactorlabels[1])", ss, df)
end

function nestedfactorscalc(amongallnested, nnestedfactors, crossedfactors, interactions, nestedfactorlabels)
    nestedfactors = []
    if nnestedfactors > 0
        otherfactors = [crossedfactors; interactions]
        otherss = sum(f -> f.ss, otherfactors)
        otherdf = sum(f -> f.df, otherfactors)
        for i ∈ nnestedfactors:-1:1
            ss = amongallnested[i].ss - otherss
            df = amongallnested[i].df - otherdf
            otherss += ss
            otherdf += df
            push!(nestedfactors, AnovaFactor(nestedfactorlabels[i], ss, df))
        end
    end
    nestedfactors
end

function getnumerators(crossedfactors, ncrossedfactors, nnestedfactors, nestedfactors, interactions)
    numerators = copy(crossedfactors)
    if ncrossedfactors > 1 && length(interactions) > 0
        push!(numerators, interactions[1])
        ncrossedfactors > 2 && push!(numerators, interactions[2], interactions[3], interactions[4])
    end
    nnestedfactors > 0 && append!(numerators, nestedfactors)
    numerators
end

function getdenominators(interactions, nestedfactors, error, crossedfactortypes, ncrossedfactors, nnestedfactors)
    nesteddenominators = Vector{AnovaFactor}(undef, nnestedfactors)

    if nnestedfactors > 0
        crossedbasedenominator = nestedfactors[1]
        nesteddenominators[1:end-1] = nestedfactors[2:end]
        nesteddenominators[end] = error
    else
        crossedbasedenominator = error
    end

    if ncrossedfactors == 1
        denominators = [crossedbasedenominator]
    elseif length(interactions) == 0
        denominators = repeat([crossedbasedenominator], ncrossedfactors)
    else # 2 or 3 crossed factors
        ninteractions = ncrossedfactors == 2 ? 1 : 4
        crosseddenominators = Vector{AnovaFactor}(undef, ncrossedfactors)
        pairinteractiondenominators = Vector{AnovaFactor}(undef, ninteractions - 1)

        if all(f -> f == fixed, crossedfactortypes)
            crosseddenominators = repeat([crossedbasedenominator], ncrossedfactors)
            pairinteractiondenominators = repeat([crossedbasedenominator], ninteractions - 1)

        elseif all(f -> f == random, crossedfactortypes)
            if ncrossedfactors == 2
                crosseddenominators = repeat(interactions, ncrossedfactors)

            else # 3 crossed factors
                crosseddenominators = Vector{AnovaFactor}(undef, 3)
                for i ∈ 1:3
                    j1, j2 = (1:3)[Not(i)]
                    crosseddenominators[i] = threewayinteraction(interactions[i + j1 - 2], interactions[i + j2 - 2], interactions[4])
                end
                pairinteractiondenominators = repeat([interactions[4]], ninteractions - 1)

            end

        else # has both random and fixed factors
            if ncrossedfactors == 2
                crosseddenominators = map(f -> f == fixed ? interactions[1] : crossedbasedenominator, crossedfactortypes)

            else # 3 crossed factors
                randomcount = count(f -> f == random, crossedfactortypes)
                if randomcount == 1
                    randomi = findfirst(f -> f == random, crossedfactortypes)
                    fixedi = (1:3)[Not(randomi)]
                    crosseddenominators = Vector{AnovaFactor}(undef, 3)
                    crosseddenominators[fixedi] = [interactions[i + randomi - 2] for i ∈ fixedi]
                    crosseddenominators[randomi] = crossedbasedenominator

                    fixedinteractioni = sum(fixedi) - 2
                    pairinteractiondenominators = Vector{AnovaFactor}(undef, 3)
                    pairinteractiondenominators[fixedinteractioni] = interactions[4]
                    pairinteractiondenominators[(1:3)[Not(fixedinteractioni)]] .= crossedbasedenominator

                else # 2 random factors
                    fixedi = findfirst(f -> f == fixed, crossedfactortypes)
                    randomi = (1:3)[Not(fixedi)]
                    crosseddenominators = Vector{AnovaFactor}(undef, 3)
                    crosseddenominators[fixedi] = threewayinteraction(interactions[fixedi + randomi[1] - 2], interactions[fixedi + randomi[2] - 2], interactions[4])
                    crosseddenominators[randomi] .= interactions[randomi[1] + randomi[2] - 2]

                    randominteractioni = sum(randomi) - 2
                    pairinteractiondenominators = Vector{AnovaFactor}(undef, 3)
                    pairinteractiondenominators[(1:3)[Not(randominteractioni)]] .= interactions[4]
                    pairinteractiondenominators[randominteractioni] = crossedbasedenominator

                end
            end
        end
        denominators = [crosseddenominators; pairinteractiondenominators; crossedbasedenominator]
    end
    append!(denominators, nesteddenominators)
    denominators
end

function threewayinteraction(interaction_ab, interaction_bc, interaction_abc)
    reducedmeansquare(factor::AnovaFactor) = factor.ms ^ 2 / factor.df
    ms = interaction_ab.ms + interaction_bc.ms - interaction_abc.ms
    df = ms ^ 2 / (reducedmeansquare(interaction_ab) + reducedmeansquare(interaction_bc) + reducedmeansquare(interaction_abc))
    AnovaFactor("", ms * df, df, ms)
end

function ftest(x, y)
    f = x.ms / y.ms
    fdist = FDist(x.df, y.df)
    p = ccdf(fdist, f)
    AnovaResult(x, f, p)
end

function effectsizescalc(results, denominators, total, ncrossedfactors, npercrossedcell, ncrossedfactorlevels, crossedfactortypes, nnestedfactors, nnestedfactorlevels, nreplicates)
    differences = [results[i].ms - denominators[i].ms for i ∈ eachindex(results)] # 1 kb between this line and next
    crossedfactordfs = [r.df for r ∈ results[1:ncrossedfactors]]

    if nreplicates == 1 && nnestedfactors > 0
        nnestedfactors -= 1
        nnestedfactorlevels = nnestedfactorlevels[1:(end-1)]
    end

    if ncrossedfactors == 1
        if nnestedfactors == 0
            ω² = [(results[1].ss - results[1].df * denominators[1].ms) / (total.ss + denominators[1].ms)]
        else
            effectdenominators = repeat([nreplicates], nnestedfactors + 1)
            nfactorlevels = [ncrossedfactorlevels; nnestedfactorlevels]
            effectdenominators[1] *= prod(nfactorlevels)
            factors = ones(Int, nnestedfactors + 1)
            factors[1] = crossedfactordfs[1]
            for i ∈ 2:nnestedfactors
                effectdenominators[2:(end - i + 1)] .*= nfactorlevels[end - i + 2]
            end
            σ² = factors .* differences ./ effectdenominators
            σ²total = sum(σ²) + denominators[end].ms
            ω² = σ² ./ σ²total
        end
    else
        if ncrossedfactors == 2 # this whole block not quite 1 kb
            if npercrossedcell > 1
                interactionindexes = [[1,2]]
                imax = 3
            else
                interactionindexes = []
                imax = 2
            end
        else
            if npercrossedcell > 1
                interactionindexes = [[1,2], [1,3], [2,3], [1,2,3]]
                imax = 7
            else
                interactionindexes = [[1,2], [1,3], [2,3]]
                imax = 6
            end
        end

        icrossed = 1:ncrossedfactors # this whole block 1 kb
        iother = ncrossedfactors < imax ? ((ncrossedfactors + 1):imax) : []
        factors = Vector{Int}(undef, imax)
        factors[icrossed] = [crossedfactortypes[i] == fixed ? crossedfactordfs[i] : 1 for i ∈ icrossed]
        factors[iother] = [prod(factors[x]) for x ∈ interactionindexes]

        effectsdenominators = repeat([npercrossedcell], imax)
        israndom = [x == random for x ∈ crossedfactortypes] # Originally used broadcasted equality (.==) but causes high allocations as of 1.3.0-rc3
        isfixed = [x == fixed for x ∈ crossedfactortypes]
        crossedeffectsdenominators = effectsdenominators[icrossed]
        crossedeffectsdenominators[isfixed] .*= prod(ncrossedfactorlevels)
        crossedeffectsdenominators[israndom] .*= [prod(ncrossedfactorlevels[Not(i)]) for i ∈ icrossed[israndom]]
        effectsdenominators[icrossed] = crossedeffectsdenominators
        effectsdenominators[iother] .*= [prod(ncrossedfactorlevels[Not(icrossed[israndom] ∩ x)]) for x ∈ interactionindexes] # 7kb - set intersection is 1kb, has to be done for each interaction

        σ² = factors .* differences[1:imax] ./ effectsdenominators

        if nnestedfactors > 0
            nestedrange = (length(results) .- nnestedfactors .+ 1):length(results)
            nestedeffectdenominators = repeat([nreplicates], nnestedfactors)
            for i ∈ 1:(nnestedfactors - 1)
                nestedeffectdenominators[1:(end - i + 1)] .*= nnestedfactorlevels[end - i + 2]
            end
            σ²nested = differences[nestedrange] ./ nestedeffectdenominators
            σ² = [σ²; σ²nested]
        end

        σ²total = sum(σ²) + denominators[end].ms
        ω² = σ² ./ σ²total
    end
    AnovaResult.(results, ω²)
end


#Mapping functions
##################
# function index2d(k,nx)
#     j = floor(Int,(k-1)/nx)+1
#     # TODO: add consistency check with ny?
#     return k-(j-1)*nx,j
# end

function index1d(i,j,nx)
    # TODO: add consistency check with ny?
    return i+(j-1)*nx
end

"""
    compute_equilibrium!(...)

* rel_error is the tolerance for global temperature equilibrium (default is 2e-5).
* max_years is the maximum number of annual cycles to be computed when searching for equilibrium
"""
function compute_equilibrium!(discretization; max_years=100, rel_error=2e-5, verbose=true)
    @unpack mesh, model, lu_decomposition, num_steps_year, annual_temperature, rhs, last_rhs  = discretization
    @unpack nx, dof = mesh

    average_temperature = average_temperature_old = area_weighted_average(view(annual_temperature, :, num_steps_year), mesh)

    if verbose
      println("year","  ","Average Temperature")
      println(0,"  ",average_temperature_old)
    end

    for year in 1:max_years
        average_temperature = 0.0
        for time_step in 1:num_steps_year
            old_time_step = (time_step == 1) ? num_steps_year : time_step - 1
            update_rhs!(rhs, mesh, num_steps_year, time_step, view(annual_temperature, :, old_time_step), model, last_rhs)

            # Use in-place operation `ldiv!` instead of `\` to avoid allocations
            ldiv!(view(annual_temperature, :, time_step), lu_decomposition, rhs)

            annual_temperature[1:nx, time_step] .= annual_temperature[1, time_step]
            annual_temperature[dof-nx+1:dof, time_step] .= annual_temperature[dof, time_step]

            average_temperature += area_weighted_average(view(annual_temperature, :, time_step), mesh)
        end
        average_temperature = average_temperature/num_steps_year
        if verbose
          println(year,"  ",average_temperature)
        end

        if (abs(average_temperature-average_temperature_old)<rel_error)
            if verbose
              println("EQUILIBRIUM REACHED!")
            end
            break
        end

        average_temperature_old = average_temperature
    end

    return average_temperature
end

"""
    compute_evolution!(...)

Compute the evolution of the mean temperature with varying CO2 levels.
"""
function compute_evolution!(discretization, co2_concentration_at_step, year_start, year_end; verbose=true)
    @unpack mesh, model, lu_decomposition, num_steps_year, annual_temperature, rhs, last_rhs  = discretization
    @unpack nx, dof = mesh

    if verbose
      println("year","  ","Average Temperature")
    end

    max_years = year_end - year_start + 1

    # Allocate output arrays
    year_at_step = zeros(Float64,max_years*num_steps_year+1)
    mean_temperature_at_step = zeros(Float64,max_years*num_steps_year+1)
    year_array = zeros(Float64,max_years)
    mean_temperature_yearly = zeros(Float64,max_years)

    # Fill year arrays
    year_at_step[1] = year_start + 0.21644  # The simulation starts at the vernal equinox
    for year in 2:size(year_at_step,1)
        year_at_step[year] = year_at_step[year-1] + 1.0/num_steps_year
    end
    year_array[1] = year_start + 0.71644    # The simulation starts at the vernal equinox
    for year in 2:max_years
        year_array[year] = year_array[year-1] + 1.0
    end

    # Initialize the temperature array
    mean_temperature_at_step[1] = area_weighted_average(view(annual_temperature, :, num_steps_year), mesh)
    step = 1

    for year in 1:max_years
        average_temperature = 0.0
        for time_step in 1:num_steps_year
            set_co2_concentration!(model, co2_concentration_at_step[step])
            old_time_step = (time_step == 1) ? num_steps_year : time_step - 1
            update_rhs!(rhs, mesh, num_steps_year, time_step, view(annual_temperature, :, old_time_step), model, last_rhs)

            # Use in-place operation `ldiv!` instead of `\` to avoid allocations
            ldiv!(view(annual_temperature, :, time_step), lu_decomposition, rhs)

            annual_temperature[1:nx, time_step] .= annual_temperature[1, time_step]
            annual_temperature[dof-nx+1:dof, time_step] .= annual_temperature[dof, time_step]

            # Compute mean temperature
            step += 1
            mean_temperature_at_step[step] = area_weighted_average(view(annual_temperature, :, time_step), mesh)
            average_temperature += mean_temperature_at_step[step]
        end
        average_temperature = average_temperature/num_steps_year
        mean_temperature_yearly[year] = average_temperature
        if verbose
          println(year_start+year-1,"  ",average_temperature)
        end
    end

    return (; year_at_step, mean_temperature_at_step, year_array, mean_temperature_yearly)
end

"""
Computes mean temperature in the globe at a specific time
"""
function area_weighted_average(vector,mesh)
    @unpack nx,ny,area,dof = mesh

    average_vector = 0.0

    # Contribution of inner points
    for j=2:ny-1
        for i=1:nx
            row_idx = index1d(i,j,nx)
            average_vector += area[j] * vector[row_idx]
        end
    end

    # Poles
    average_vector += area[1] * (vector[1] + vector[dof])

    return average_vector
end

function compute_matrix(mesh,num_steps_year,model)
    @unpack nx,ny,dof,h,geom,csc2,cot,area = mesh
    @unpack diffusion_coeff,heat_capacity,radiative_cooling_feedback = model
    matrix = spzeros(Float64,dof,dof)
    sh2 = 1 / h^2
    # Inner DOFs (c coefficients are divided by h^2)
    for j=2:ny-1
        for i=1:nx

            # Compute coefficients
            c0 = 2 * sh2 * diffusion_coeff[i,j] * (1 + csc2[j]) + 2 * heat_capacity[i,j] * num_steps_year + radiative_cooling_feedback

            # Get diffusion coefficient
            if (i == 1) # Periodic BC
                d_phi = (diffusion_coeff[2,j] - diffusion_coeff[nx,j]) / (2 * h)
            elseif (i == nx) # Periodic BC
                d_phi = (diffusion_coeff[1,j] - diffusion_coeff[nx-1,j]) / (2 * h)
            else # Inner DOFs
                d_phi = (diffusion_coeff[i+1,j] - diffusion_coeff[i-1,j]) / (2 * h)
            end

            c1 = sh2 * csc2[j] * (diffusion_coeff[i,j] - 0.5 * h * d_phi)
            c3 = sh2 * csc2[j] * (diffusion_coeff[i,j] + 0.5 * h * d_phi)

            d_theta= (diffusion_coeff[i,j+1] - diffusion_coeff[i,j-1]) / (2 * h)
            c2 = sh2 * (diffusion_coeff[i,j] - 0.5 * h * (diffusion_coeff[i,j] * cot[j] + d_theta))
            c4 = sh2 * (diffusion_coeff[i,j] + 0.5 * h * (diffusion_coeff[i,j] * cot[j] + d_theta))

            # Fill matrix A
            row_idx = index1d(i,j,nx)
            col_idx_c0 = row_idx

            if (i==1) # Periodic BC
                col_idx_c1 = index1d(nx,j,nx)
                col_idx_c3 = row_idx + 1
            elseif (i==nx) # Periodic BC
                col_idx_c1 = row_idx - 1
                col_idx_c3 = index1d(1,j,nx)
            else
                col_idx_c1 = row_idx - 1
                col_idx_c3 = row_idx + 1
            end
            col_idx_c2 = row_idx - nx
            col_idx_c4 = row_idx + nx

            matrix[row_idx,col_idx_c0] = c0
            matrix[row_idx,col_idx_c1] = -c1
            matrix[row_idx,col_idx_c2] = -c2
            matrix[row_idx,col_idx_c3] = -c3
            matrix[row_idx,col_idx_c4] = -c4

            # Get rhs
            # rhs[row_idx] = -(c0 * Temp[col_idx_c0] - c1 * Temp[col_idx_c0] - c2 * Temp[col_idx_c2] - c3 * Temp[col_idx_c3] - c4 * Temp[col_idx_c4]) + 4 * heat_capacity[i,j]
        end
    end

    # Poles
    total_area = area[1] + area[2]

    diff_total_np = zeros(Float64,nx)
    diff_total_sp = zeros(Float64,nx)
    for i=1:nx
        diff_total_np[i] = (area[1] * diffusion_coeff[1,1]  + area[2] * diffusion_coeff[i,2]) / total_area
        diff_total_sp[i] = (area[1] * diffusion_coeff[1,ny] + area[2] * diffusion_coeff[i,ny-1]) / total_area
    end
    diff_total_sum_np = sum(diff_total_np)
    diff_total_sum_sp = sum(diff_total_sp)

    gc_np = geom * diff_total_sum_np + radiative_cooling_feedback + 2 * heat_capacity[1,1] * num_steps_year     # north pole
    gc_sp = geom * diff_total_sum_sp + radiative_cooling_feedback + 2 * heat_capacity[1,ny] * num_steps_year    # south pole

    for i=1:nx
        #North pole
        row_idx = i
        matrix[row_idx,row_idx] = gc_np
        for ii=1:nx
            matrix[row_idx,nx+ii] = - geom * diff_total_np[ii]
        end

        #South pole
        row_idx = dof + 1 - i
        matrix[row_idx,row_idx] = gc_sp
        for ii=1:nx
            matrix[row_idx,dof-2*nx+ii] = - geom * diff_total_sp[ii]
        end
    end
    return matrix
end

function update_rhs!(rhs, mesh, num_steps_year, time_step, temperature, model,  last_rhs)
    @unpack nx,ny = mesh
    @unpack heat_capacity, solar_forcing, radiative_cooling_co2 = model
    for j=1:ny
        for i=1:nx
            row_idx = index1d(i,j,nx)

            rhs[row_idx] = 4 * heat_capacity[i,j] * temperature[row_idx] * num_steps_year  - last_rhs[row_idx] + solar_forcing[i,j,time_step] - 2 * radiative_cooling_co2
            if (time_step == 1)
                rhs[row_idx] += solar_forcing[i,j,num_steps_year]
            else
                rhs[row_idx] += solar_forcing[i,j,time_step-1]
            end
        end
    end
    last_rhs .= rhs
end


"""
Minimization wrapper for unsupervised learning.
"""
function minimize_unsupervised(L::LossUnsupervised, R::RegularizerUnsupervised, X, k,
                               beta=0.8, alpha=0.5, init=nothing, t_init=1.0;
                               max_iters=5000, verbose=false, tol=1e-8)

    theta_X, theta_Y, losses = optimize_unsupervised(L, R, X, k,
                                                     alpha, beta, init, t_init,
                                                     t_min=1e-15,
                                                     max_iters=max_iters, verbose=verbose, tol=tol)
    return theta_X, theta_Y, losses
end

"""
Alternating minimization for unsupervised problems.
"""
function optimize_unsupervised(L::LossUnsupervised, R::RegularizerUnsupervised,
                               C, k, alpha=.5, beta=.8, init=nothing, t_init=1.0;
                               t_min=1e-15, verbose=true, max_iters=5000, tol=1e-8)

    n, d = size(C)
    decay = true

    if verbose
        info("Solving problem. Shape is $n by $d.")
    end

    # convenience functions
    LOSS(u, v) = eval(L, C, u, v)
    GRAD_X(u, v) = deriv(L, C, u, v, "X")
    GRAD_Y(u, v) = deriv(L, C, u, v, "Y")
    PROX(u) = prox(R, u)

    theta_X, theta_Y, losses = [], [], []

    if init == nothing
        init_X = rand(n, k)
        init_Y = rand(d, k)
    else
        init_X = init["X"]
        init_Y = init["Y"]
    end

    if verbose
        info("The initial loss is $(LOSS(init_X, init_Y))")
        info("With gradient X $(GRAD_X(init_X, init_Y))")
        info("With gradient Y $(GRAD_Y(init_X, init_Y))")
    end

    assert(size(init_X)[1] == n)
    assert(size(init_Y)[1] == d)
    assert(size(init_X)[2] == k && size(init_Y)[2] == k)

    push!(theta_X, init_X)
    push!(theta_Y, init_Y)

    tx, ty = t_init, t_init

    progress_x, progress_y = true, true

    for k = 1:max_iters
        # Step for X
        curr_grad_x = GRAD_X(theta_X[end], theta_Y[end])
        grad_step_X = PROX(theta_X[end] - tx*curr_grad_x)

        progress_x = true
        prev_loss = LOSS(theta_X[end], theta_Y[end])

        while (LOSS(grad_step_X, theta_Y[end]) >= prev_loss)
            tx *= beta
            grad_step_X = PROX(theta_X[end] - tx*curr_grad_x)
            if tx < t_min
                progress_x = false
                break
            end
        end
        tx *= 2

        push!(theta_X, grad_step_X)

        # Step for Y
        curr_grad_y = GRAD_Y(theta_X[end], theta_Y[end])
        grad_step_Y = PROX(theta_Y[end] - ty*curr_grad_y)

        progress_y = true
        prev_loss = LOSS(theta_X[end], theta_Y[end])

        while (LOSS(theta_X[end], grad_step_Y) >= prev_loss)
            ty *= beta
            grad_step_Y = PROX(theta_Y[end] - ty*curr_grad_y)
            if ty < t_min
                progress_y = false
                break
            end
        end
        ty *= 2

        push!(theta_Y, grad_step_Y)
        push!(losses, LOSS(theta_X[end], theta_Y[end]))

        if verbose
            info("Iteration: $k,  Loss: $(losses[end])")
        end

        if (!progress_x && !progress_y)
            warn("Algorithm is not making any progress... breaking.")
            return theta_X, theta_Y, losses
        end

        if k > 4 && maximum(abs.(losses[end-4:end-1] - losses[end-3:end])) < tol
            info("Done.")
            return theta_X, theta_Y, losses
        end
    end
    print("Did not converge. Loss: $(losses[end])")
    return -1
end


@testset "Polygon" begin
    @testset "Triangle2D" begin
        for T in [Float32, Float64, BigFloat]
            p₁ = Point2D{T}(0, 0)
            p₂ = Point2D{T}(1, 0)
            p₃ = Point2D{T}(0, 1)
            tri = Triangle(p₁, p₂, p₃)
            @test tri.points == SVector(p₁, p₂, p₃)

            @test isconvex(tri)
        end
    end

    @testset "Triangle3D" begin
        for T in [Float32, Float64, BigFloat]
            p₁ = Point3D{T}(0, 0, 0)
            p₂ = Point3D{T}(0, 1, 0)
            p₃ = Point3D{T}(0, 0, 1)
            tri = Triangle(p₁, p₂, p₃)
            @test tri.points == SVector(p₁, p₂, p₃)

            @test isplanar(tri)
        end
    end

    @testset "Quadrilateral2D" begin
        for T in [Float32, Float64, BigFloat]
            p₁ = Point2D{T}(0, 0)
            p₂ = Point2D{T}(1, 0)
            p₃ = Point2D{T}(1, 1)
            p₄ = Point2D{T}(0, 1)
            quad = Quadrilateral(p₁, p₂, p₃, p₄)
            @test quad.points == SVector(p₁, p₂, p₃, p₄)

            @test isconvex(quad)
            @test !isconvex(Quadrilateral(p₄, p₃, p₂, p₁))
        end
    end

    @testset "Quadrilateral3D" begin
        for T in [Float32, Float64, BigFloat]
            p₁ = Point3D{T}(0, 0, 0)
            p₂ = Point3D{T}(0, 1, 0)
            p₃ = Point3D{T}(0, 1, 1)
            p₄ = Point3D{T}(0, 0, 1)
            quad = Quadrilateral(p₁, p₂, p₃, p₄)
            @test quad.points == SVector(p₁, p₂, p₃, p₄)

            @test isplanar(quad)
            @test !isplanar(Quadrilateral(p₁, p₂, p₃, Point3D{T}(1//10, 0, 1)))
        end
    end
end


@testset "transformedkernel" begin
    rng = MersenneTwister(123456)
    x = rand(rng) * 2
    v1 = rand(rng, 3)
    v2 = rand(rng, 3)

    s = rand(rng)
    v = rand(rng, 3)
    P = rand(rng, 3, 2)
    k = SqExponentialKernel()
    @test k ∘ IdentityTransform() === k

    kt = TransformedKernel(k, ScaleTransform(s))
    ktard = TransformedKernel(k, ARDTransform(v))
    @test kt ∘ IdentityTransform() === kt
    @test ktard ∘ IdentityTransform() === ktard
    @test kt(v1, v2) == (k ∘ ScaleTransform(s))(v1, v2)
    @test kt(v1, v2) ≈ k(s * v1, s * v2) atol = 1e-5
    @test ktard(v1, v2) == (k ∘ ARDTransform(v))(v1, v2)
    @test ktard(v1, v2) == k(v .* v1, v .* v2)
    @test (k ∘ LinearTransform(P') ∘ ScaleTransform(s))(v1, v2) ==
          ((k ∘ LinearTransform(P')) ∘ ScaleTransform(s))(v1, v2) ==
          (k ∘ (LinearTransform(P') ∘ ScaleTransform(s)))(v1, v2)

    @test repr(kt) == repr(k) * "\n\t- " * repr(ScaleTransform(s))

    TestUtils.test_interface(k, Float64)
    test_ADs(x -> SqExponentialKernel() ∘ ScaleTransform(x[1]), rand(1))

    # Test implicit gradients
    @testset "Implicit gradients" begin
        k = SqExponentialKernel() ∘ ScaleTransform(2.0)
        ps = Flux.params(k)
        X = rand(10, 1)
        x = vec(X)
        A = rand(10, 10)
        # Implicit
        g1 = Flux.gradient(ps) do
            tr(kernelmatrix(k, X; obsdim=1) * A)
        end
        # Explicit
        g2 = Flux.gradient(k) do k
            tr(kernelmatrix(k, X; obsdim=1) * A)
        end

        # Implicit for a vector
        g3 = Flux.gradient(ps) do
            tr(kernelmatrix(k, x) * A)
        end
        @test g1[first(ps)] ≈ first(g2).transform.s
        @test g1[first(ps)] ≈ g3[first(ps)]
    end

    @testset "Parameters" begin
        k = ConstantKernel(; c=rand(rng))
        c = Chain(Dense(3, 2))

        test_params(k ∘ ScaleTransform(s), (k, [s]))
        test_params(k ∘ ARDTransform(v), (k, v))
        test_params(k ∘ LinearTransform(P), (k, P))
        test_params(k ∘ LinearTransform(P) ∘ ScaleTransform(s), (k, [s], P))
        test_params(k ∘ FunctionTransform(c), (k, c))
    end
end


using Plots
using BenchmarkTools

include("random_sky.jl")

dir=false

mappa = zeros(NN, NN)
hit   = zeros(NN, NN)
N_baselines = 15

dataset_baselines = Array{Main.strip2.TOD_fake_pointing, 1}(undef, N_baselines)

for i = 1:N_baselines
    dataset = strip2.observe_sky(NN, conv_map; noise=true, direction_l_r=dir)
    m, h = strip2.binned_map(NN, dataset.time_order_data, dataset.pointing)
    mappa .+= m
    hit   .+= h
    dataset_baselines[i] = dataset
    global dir =  true ⊻ dir
end

tod = Array{Float64, 1}(undef, 0)
puntamenti = Array{Int64}(undef, 0)
for i in dataset_baselines
    append!(tod, i.time_order_data)
    append!(puntamenti, i.pointing[:,1])
end

for i in dataset_baselines
    append!(puntamenti, i.pointing[:,2])
end

puntamenti = reshape(puntamenti, NN*NN*10, 2)

#obs_map  = strip2.Plot_Sky(NN, pix_size, mappa ./ hit, cmin=-600, cmax=600)
#true_map = strip2.Plot_Sky(NN, pix_size, conv_map)

#plot(obs_map, true_map, layout = grid(2,1),size=(590,940))

map_des, a, b = strip2.destriper(NN, dataset_baselines, tolerance=1e-6)
map_d = reshape(map_des, NN, NN)
residui =  (map_d .- (sum(map_d)/(NN*NN))) .- (conv_map .- (sum(conv_map)/(NN*NN)))

#p2 = strip2.Plot_Sky(NN, pix_size, conv_map .- (sum(conv_map)/(NN*NN)), cmin = -400, cmax = 400, lab_title="T Map")
p3 = strip2.Plot_Sky(NN, pix_size, map_d .- (sum(map_d)/(NN*NN)) .-  (conv_map .- (sum(conv_map)/(NN*NN))), cmin=-20, cmax=20, lab_title="Destriped Map")
#p1 = strip2.Plot_Sky(NN, pix_size, map_d .- (sum(map_d)/(NN*NN)) .- (conv_map .- (sum(conv_map)/(NN*NN))) , cmin=-8, cmax=8, lab_title="Differences between destriped and true maps")

#plot(p1, p2, layout = grid(2,1), size=(590,940))

#strip2.Plot_Sky(NN, pix_size, map_d, cmin = -400, cmax = 400)


"""
ctmcst
"""

# struct CTMCModel{MatrixT,Tv}
#     Q::MatrixT{Tv}
#     initv::Vector{Tv}
#     reward::Dict{Symbol,Vector{Tv}}
#     states::Vector{Symbol}
#     index::Dict{Symbol,Ti}
# end

# function exrss(m::CTMCModel{MatrixT,Tv}; reward = :default) where {MatrixT,Tv}
# end

"""
macro
"""

mutable struct Markov
    Tv::DataType
    state::Set{Symbol}
    initial::Dict{Symbol,Any}
    trans::Dict{Tuple{Symbol,Symbol},Any}
    reward::Dict{Symbol,Dict{Symbol,Any}}
    ireward::Dict{Symbol,Dict{Tuple{Symbol,Symbol},Any}}
end

function Base.show(io::IO, x::Markov)
    Base.show(io, "Markov $(objectid(x))")
end

function Markov()
    Markov(Float64, Set{Symbol}(), Dict{Symbol,Any}(), Dict{Tuple{Symbol,Symbol},Any}(),
        Dict{Symbol,Dict{Symbol,Any}}(), Dict{Symbol,Dict{Tuple{Symbol,Symbol},Any}}())
end

function trans!(m::Markov, src::Symbol, dest::Symbol, tr)
    push!(m.state, src)
    push!(m.state, dest)
    m.Tv = promote_type(m.Tv, typeof(tr))
    m.trans[(src,dest)] = tr
end

function reward!(m::Markov, label::Symbol, s::Symbol, r)
    push!(m.state, s)
    d = get(m.reward, label) do
        m.reward[label] = Dict{Symbol,Any}()
    end
    m.Tv = promote_type(m.Tv, typeof(r))
    d[s] = r
end

function reward!(m::Markov, label, src, dest, r)
    push!(m.state, src)
    push!(m.state, dest)
    d = get(m.ireward, label) do
        m.ireward[label] = Dict{Tuple{Symbol,Symbol},Any}()
    end
    m.Tv = promote_type(m.Tv, typeof(r))
    d[(src,dest)] = r
end

function initial!(m::Markov, s::Symbol, p)
    push!(m.state, s)
    m.Tv = promote_type(m.Tv, typeof(p))
    m.initial[s] = p
end

function generate(m; modeltype::Symbol = :SparseCTMC)
    _generate(Val(modeltype), m)
end

function _generate(::Val{:SparseCTMC}, m::Markov)
    states = [x for x = m.state]
    index = Dict([states[i] => i for i = 1:length(states)]...)
    Q = spzeros(m.Tv, length(states), length(states))
    for ((src,dest),t) = m.trans
        Q[index[src],index[dest]] = t
        Q[index[src],index[src]] -= t
    end
    initv = zeros(m.Tv, length(states))
    for (s,p) = m.initial
        initv[index[s]] = p
    end
    rwd = Dict{Symbol,Vector{m.Tv}}()
    for (k,v) = m.reward
        rwdv = zeros(m.Tv, length(states))
        for (s,r) = v
            rwdv[index[s]] = r
        end
        rwd[k] = rwdv
    end
    Q, initv, rwd, states
end

function _generate(::Val{:DenseCTMC}, m::Markov)
    states = [x for x = m.state]
    index = Dict([states[i] => i for i = 1:length(states)]...)
    Q = zeros(m.Tv, length(states), length(states))
    for ((src,dest),t) = m.trans
        Q[index[src],index[dest]] = t
        Q[index[src],index[src]] -= t
    end
    initv = zeros(m.Tv, length(states))
    for (s,p) = m.initial
        initv[index[s]] = p
    end
    rwd = Dict{Symbol,Vector{m.Tv}}()
    for (k,v) = m.reward
        rwdv = zeros(m.Tv, length(states))
        for (s,r) = v
            rwdv[index[s]] = r
        end
        rwd[k] = rwdv
    end
    Q, initv, rwd, states
end

macro tr(m, block)
    if Meta.isexpr(block, :block)
        body = [_gentrans(x, m) for x = block.args]
        esc(Expr(:block, body...))
    else
        esc(_gentrans(block, m))
    end
end

function _gentrans(x::Any, m)
    x
end

function _gentrans(x::Expr, m)
    if Meta.isexpr(x, :tuple) && Meta.isexpr(x.args[1], :call) && x.args[1].args[1] == :(=>) && length(x.args) == 2
        src = x.args[1].args[2]
        dest = x.args[1].args[3]
        t = x.args[2]
        :(trans!($m, $src, $dest, $t))
    else
        throw(TypeError(x, "Invalid format for the transition"))
    end
end

macro init(m, block)
    if Meta.isexpr(block, :block)
        body = [_geninitial(x, m) for x = block.args]
        esc(Expr(:block, body...))
    else
        esc(_geninitial(block, m))
    end
end

function _geninitial(x::Any, m)
    x
end

function _geninitial(x::Expr, m)
    if Meta.isexpr(x, :tuple) && length(x.args) == 2
        s = x.args[1]
        p = x.args[2]
        :(initial!($m, $s, $p))
    else
        throw(TypeError(x, "Invalid format for the initial probability"))
    end
end

macro reward(m, label, block)
    if Meta.isexpr(block, :block)
        body = [_genreward(x, label, m) for x = block.args]
        esc(Expr(:block, body...))
    else
        esc(_genreward(block, label, m))
    end
end

function _genreward(x::Any, label, m)
    x
end

function _genreward(x::Expr, label, m)
    if Meta.isexpr(x, :tuple) && length(x.args) == 2
        s = x.args[1]
        r = x.args[2]
        :(reward!($m, $label, $s, $r))
    elseif Meta.isexpr(x, :tuple) && Meta.isexpr(x.args[1], :call) && x.args[1].args[1] == :(=>) && length(x.args) == 2
        src = x.args[1].args[2]
        dest = x.args[1].args[3]
        r = x.args[2]
        :(reward!($m, $label, $src, $dest, $r))
    else
        throw(TypeError(x, "Invalid format for the reward vector"))
    end
end

###

macro markov(f, block)
    body = []
    push!(body, Expr(:(=), :tmp, Expr(:call, :Markov)))
    if Meta.isexpr(block, :block)
        for x = block.args
            push!(body, _replace_macro(x))
        end
    end
    push!(body, :tmp)
    esc(Expr(:function, f, Expr(:block, body...)))
end

function _replace_macro(x::Any)
    x
end

function _replace_macro(x::Expr)
    if Meta.isexpr(x, :macrocall) && (x.args[1] == Symbol("@tr") || x.args[1] == Symbol("@init") || x.args[1] == Symbol("@reward"))
        Expr(:macrocall, x.args[1], x.args[2], :tmp, [_replace_macro(u) for u = x.args[3:end]]...)
    else
        Expr(x.head, [_replace_macro(u) for u = x.args]...)
    end
end

macro s(x...)
    esc(Expr(:call, :Symbol, x...))
end


using Tracker, Test, Random, Statistics

Random.seed!(0)

@testset "Tracker" begin

include("tracker.jl")

using Tracker: jacobian

@testset "Jacobian" begin
  A = param(randn(2,2))
  x = randn(2)
  m(x) = A*x
  y = m(x)
  J = jacobian(m,x)
  @test J ≈ A.data
end

end


"""FM Signal"""
function fmWave(P::Process)
    # Only parameter is the FM signal
    T = P.transient_t0:P.savedt:P.tmax
    p = parameter_functions(P)
    Δ𝑓 = 1.0
    sol = zeros(size(T))
    pint = 0.0
    for i ∈ 2:lastindex(T)
        t = T[i]
        pint += (p(t-P.savedt) + p(t))*P.savedt/2 # Crude integration, should be fine
        sol[i] = cos(2π*(t + Δ𝑓*pint))
    end
    return sol
end

fmWaveSim = Process(
    process = fmWave,
    parameter_profile = (stepNoise),
    parameter_profile_parameters = ((0.0, 100.0), 10.0, 0.5, 0.0),
    transient_t0 = 0.0,
    t0 = 0.0,
    savedt = 0.01,
    tmax = 100.0)
export fmWaveSim

"""Noisy Sine"""
function noisySine(P::Process)
    seed(P.solver_rng)
    sol = [sin(t) + parameter_function(P)(t)*randn() for t in P.transient_t0:P.savedt:P.tmax]
end

noisySineSim = Process(
    process = noisySine,
    X0 = [0.0],
    parameter_profile = unitStep,
    parameter_profile_parameters = (100.0, 0.0, 0.1), # (threshold, baseline, stepHeight)
    transient_t0 = -10.0,
    dt = 0.01,
    savedt = 0.01,
    tmax = 200.0,
    alg = nothing,
    solver_opts=Dict())
export noisySineSim

"""Noisy Shifty Scaly Sine"""
function noisyShiftyScalySine(P::Process)
    # Parameters (η, C, A)
    seed(P.solver_rng)
    (η, C, A) = parameter_functions(P)
    sol = [A(t)*sin(t) + η(t)*randn() + C(t) for t in P.transient_t0:P.savedt:P.tmax]
end

function noisyShiftyShcalySine(P::Process)
    # Parameters (η, C, A)
    seed(P.solver_rng)
    (η, C, A) = parameter_functions(P)
    sol = [A(t)*(sin(t) + η(t)*randn() + C(t)) for t in P.transient_t0:P.savedt:P.tmax]
end

"""Shifty Scaly Sine"""
function shcalySine(P::Process)
    seed(P.solver_rng)
    A = parameter_function(P)
    sol = [A(t)*(sin(t) + 1.0*randn() + 1.0) for t in P.transient_t0:P.savedt:P.tmax]
end

noisyShiftyScalySineSim = Process(
    process = noisyShiftyScalySine,
    X0 = [0.0],
    parameter_profile = (ramp, constantParameter, constantParameter), # (η, C, A)
    parameter_profile_parameters = ((0.001, 2.0, 0.0), (1.0,), (1.0,)),
    transient_t0 = 0.0,
    t0 = 0.0,
    dt = 0.1,
    savedt = 0.1,
    tmax = 1000.0,
    alg = nothing,
    solver_opts=Dict())
export noisyShiftyScalySineSim

noisyShiftyShcalySineSim = Process(
    process = noisyShiftyShcalySine,
    X0 = [0.0],
    parameter_profile = (constantParameter, constantParameter, constantParameter), # (η, C, A)
    parameter_profile_parameters = ((1.0,), (1.0,), (1.0,)),
    transient_t0 = 0.0,
    t0 = 0.0,
    dt = 0.05,
    savedt = 0.05,
    tmax = 500.0,
    alg = nothing,
    solver_opts=Dict())
export noisyShiftyShcalySineSim

shcalySineSim = Process(
    process = shcalySine,
    X0 = [0.0],
    parameter_profile = ramp, # (η, C, A)
    parameter_profile_parameters = (0.001, 1.0, 0.0),
    transient_t0 = 0.0,
    t0 = 0.0,
    dt = 0.01,
    savedt = 0.01,
    tmax = 1000.0,
    alg = nothing,
    solver_opts=Dict())
export shcalySineSim


# Bogumił Kamiński, 2019-03-25

# Start Julia run with the following command:
#     julia -p auto
#
# .int file format specification:
#     * Each image is encoded by 1+28^2 bytes.
#     * Images are stored consecutively.
#     * First byte for each image is its class from 0 to 9
#     * Then each of remaining 28^2 has value from 0 to 255
#       and encodes one entry in a 28x28 image

using Statistics
using Distributed

try
    using PyPlot
    @info "PyPlot loaded"
catch
    @warn "PyPlot missing"
end

@everywhere const TRAIN = (class=UInt8[], image=Matrix{Int32}[])
@everywhere const TEST = (class=UInt8[], image=Matrix{Int32}[])

@everywhere for (filename, data) in [("mnist_train.int", TRAIN),
                                     ("mnist_test.int", TEST)]
    open(filename) do f
        while !eof(f)
            c = read(f, UInt8)
            v = read(f, 28^2)
            push!(data.class, c)
            push!(data.image, reshape(v, 28, 28))
        end
    end
end

@everywhere function distance(a, b)
    d = 0
    @simd for i in 1:length(a)
        @inbounds d += (a[i] - b[i]) ^ 2
    end
    d
end

@everywhere function knnacc(i)
    dist = distance.(Ref(TEST.image[i]), TRAIN.image)
    knn_loc = partialsortperm(dist, 1:20)
    TRAIN.class[knn_loc]
end

@time neivec = pmap(knnacc, axes(TEST.class, 1))
const NEIGHBOURS = reduce(hcat, neivec)

function acc(classes, testclass)
    v = zeros(Int, 10)
    for c in classes
        v[c + 1] += 1
    end
    m = maximum(v)
    (v[testclass + 1] == m) / count(==(m), v)
end

function evalk(k)
    mean(axes(NEIGHBOURS, 2)) do i
        classes = view(NEIGHBOURS, 1:k, i)
        testclass = TEST.class[i]
        acc(classes, testclass)
    end
end

evalk.(axes(NEIGHBOURS, 1))


# Copyright(c) Rafael Dias
# MIT Licensed

module PropriedadesSecoes
    mutable struct Coordenada
        x::Float32
        y::Float32
    end

    function comprimentoEntrePontos(coordenadas::Array{Coordenada})::Float32
        quantidade = length(coordenadas)
        atual = 1
        proximo = 2
        comprimento = 0f0

        for contador in 1:quantidade - 1
            comprimento += sqrt(((coordenadas[proximo].x - coordenadas[atual].x)^2) + ((coordenadas[proximo].y - coordenadas[atual].y)^2))

            if proximo < quantidade
                proximo += 1
            else
                proximo = 1
            end
            if contador == quantidade
                atual = 1
            else
                atual += 1
            end
        end

        return comprimento
    end

    function areaPoligono(coordenadas::Array{Coordenada})::Float32
        quantidade = length(coordenadas)
        atual = 1
        proximo = 2
        area = 0f0

        for contador in 1:quantidade
            area += (coordenadas[proximo].x * coordenadas[atual].y) - (coordenadas[atual].x * coordenadas[proximo].y)

            if proximo < quantidade
                proximo += 1
            else
                proximo = 1
            end
            if contador == quantidade
                atual = 1
            else
                atual += 1
            end
        end

        return area / 2f0
    end

    function perimetroPoligono(coordenadas::Array{Coordenada})::Float32
        quantidade = length(coordenadas)
        atual = 1
        proximo = 2
        perimetro = 0f0

        for contador in 1:quantidade
            perimetro += sqrt(((coordenadas[proximo].x - coordenadas[atual].x)^2) + ((coordenadas[proximo].y - coordenadas[atual].y)^2))

            if proximo < quantidade
                proximo += 1
            else
                proximo = 1
            end
            if contador == quantidade
                atual = 1
            else
                atual += 1
            end
        end

        return perimetro
    end

    function centroGravidadePoligonoX(coordenadas::Array{Coordenada})::Float32
        quantidade = length(coordenadas)
        atual = 1
        proximo = 2
        xgc = 0f0

        for contador in 1:quantidade
            xgc += (coordenadas[proximo].x + coordenadas[atual].x) * ((coordenadas[proximo].x * coordenadas[atual].y) - (coordenadas[atual].x * coordenadas[proximo].y))

            if proximo < quantidade
                proximo += 1
            else
                proximo = 1
            end
            if contador == quantidade
                atual = 1
            else
                atual += 1
            end
        end

        return xgc * (1 / (6 * areaPoligono(coordenadas)))
    end

    function centroGravidadePoligonoY(coordenadas::Array{Coordenada})::Float32
        quantidade = length(coordenadas)
        atual = 1
        proximo = 2
        ygc = 0f0

        for contador in 1:quantidade
            ygc += (coordenadas[proximo].y + coordenadas[atual].y) * ((coordenadas[proximo].x * coordenadas[atual].y) - (coordenadas[atual].x * coordenadas[proximo].y))

            if proximo < quantidade
                proximo += 1
            else
                proximo = 1
            end
            if contador == quantidade
                atual = 1
            else
                atual += 1
            end
        end

        return ygc * (1 / (6 * areaPoligono(coordenadas)))
    end
end


# Copyright (c) 2021, Eric Sabo
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.

using AbstractAlgebra
using Nemo

import LinearAlgebra: tr

function ⊕(A::T, B::T) where T <: Union{fq_nmod_mat, gfp_mat}
    base_ring(A) == base_ring(B) || error("Matrices must be over the same base ring in directsum.")

    return vcat(hcat(A, zero_matrix(base_ring(B), size(A, 1), size(B, 2))),
        hcat(zero_matrix(base_ring(A), size(B, 1), size(A, 2)), B))
end

directsum(A::T, B::T) where T <: Union{fq_nmod_mat, gfp_mat} = A ⊕ B
⊗(A::T, B::T) where T <: Union{fq_nmod_mat, gfp_mat} = kronecker_product(A, B)
kron(A::T, B::T) where T <: Union{fq_nmod_mat, gfp_mat} = kronecker_product(A, B)
tensorproduct(A::T, B::T) where T <: Union{fq_nmod_mat, gfp_mat} = kronecker_product(A, B)
kroneckerproduct(A::T, B::T) where T <: Union{fq_nmod_mat, gfp_mat} = kronecker_product(A, B)
# nrows(A::T) where T = size(A, 1)
# ncols(A::T) where T = size(A, 2)

# I think we should avoid length checking here and return it for entire matrix if given
# Hammingweight(v::T) where T <: Union{fq_nmod_mat, gfp_mat, Vector{S}} where S <: Integer = count(i->(i != 0), v)
function Hammingweight(v::T) where T <: Union{fq_nmod_mat, gfp_mat, Vector{S}} where S <: Integer
    count = 0
    for i in 1:length(v)
        if !iszero(v[i])
            count += 1
        end
    end
    return count
end
weight(v::T) where T <: Union{fq_nmod_mat, gfp_mat, Vector{S}} where S <: Integer = Hammingweight(v)
wt(v::T) where T <: Union{fq_nmod_mat, gfp_mat, Vector{S}} where S <: Integer = Hammingweight(v)
Hammingdistance(u::T, v::T) where T <: Union{fq_nmod_mat, gfp_mat, Vector{S}} where S <: Integer = Hammingweight(u .- v)
distance(u::T, v::T) where T <: Union{fq_nmod_mat, gfp_mat, Vector{S}} where S <: Integer = Hammingweight(u .- v)
dist(u::T, v::T) where T <: Union{fq_nmod_mat, gfp_mat, Vector{S}} where S <: Integer = Hammingweight(u .- v)

function tr(x::fq_nmod, K::FqNmodFiniteField, verify::Bool=false)
    L = parent(x)
    if verify
        # shouldn't need Int casting here but just in case...
        Int64(characteristic(L)) == Int64(characteristic(K)) || error("The given field is not a subfield of the base ring of the element.")
        degree(L) % degree(K) == 0 || error("The given field is not a subfield of the base ring of the element.")
    end
    n = div(degree(L), degree(K))
    return sum([x^(q^i) for i in 0:(n - 1)])
end

function _expandelement(x::fq_nmod, K::FqNmodFiniteField, basis::Vector{fq_nmod}, verify::Bool=false)
    return [tr(x * i, K, verify) for i in basis]
end

function _expandrow(row::Vector{fq_nmod}, K::FqNmodFiniteField, basis::Vector{fq_nmod}, verify::Bool=false)
    return hcat([_expandelement(row[i], K, basis, verify)]...)
end

function expandmatrix(M::fq_nmod_mat, K::FqNmodFiniteField, basis::Vector{fq_nmod})
    L = base_ring(M)
    L != K || return M
    # shouldn't need Int casting here but just in case...
    Int64(characteristic(L)) == Int64(characteristic(K)) || error("The given field is not a subfield of the base ring of the element.")
    degree(L) % degree(K) == 0 || error("The given field is not a subfield of the base ring of the element.")
    n = div(degree(L), degree(K))
    n == length(basis) || error("Provided basis is of incorrect size for the given field and subfield.")
    # should really check if it is a basis
    return vcat([_expandrow(M[r, :], K, basis) for r in 1:size(M, 1)])
end

function symplecticinnerproduct(u::fq_nmod_mat, v::fq_nmod_mat)
    (size(u, 1) == 1 || size(u, 2) == 1) || error("First argument of symplectic inner product is not a vector: dims = $(size(u, 1)).")
    (size(v, 1) == 1 || size(v, 2) == 1) || error("Second argument of symplectic inner product is not a vector: dims = $(size(v, 1)).")
    length(u) == length(v) || error("Vectors must be the same length in symplectic inner product.")
    iseven(length(u)) || error("Vectors must have even length in symplectic inner product.")
    base_ring(u) == base_ring(v) || error("Vectors must be over the same field in symplectic inner product.")
    ncols = div(length(u), 2)
    return sum([u[i + ncols] * v[i] - v[i + ncols] * u[i] for i in 1:ncols])
end
# SIP(u::fq_nmod_mat, v::fq_nmod_mat) = symplecticinnerproduct(u, v)

# since changed method here, need to add error checks on dimensions
function aresymplecticorthogonal(A::fq_nmod_mat, B::fq_nmod_mat)
    AEuc = hcat(A[:, div(size(A, 2), 2) + 1:end], -A[:, 1:div(size(A, 2), 2)])
    iszero(AEuc * transpose(B)) || return false
    return true
end

# function traceinnerproduct(u::fq_nmod_mat, v::fq_nmod_mat)
#
# end
# TIP(u::fq_nmod_mat, v::fq_nmod_mat) = traceinnerproduct(u, v)

function Hermitianinnerproduct(u::fq_nmod_mat, v::fq_nmod_mat)
    (size(u, 1) == 1 || size(u, 2) == 1) || error("First argument of Hermitian inner product is not a vector: dims = $(size(u, 1)).")
    (size(v, 1) == 1 || size(v, 2) == 1) || error("Second argument of Hermitian inner product is not a vector: dims = $(size(v, 1)).")
    length(u) == length(v) || error("Vectors must be the same length in Hermitian inner product.")
    base_ring(u) == base_ring(v) || error("Vectors must be over the same field in Hermitian inner product.")
    q2 = order(base_ring(u))
    issquare(q2) || error("The Hermitian inner product is only defined over quadratic field extensions.")
    q = Int64(sqrt(q2))
    return sum([u[i] * v[i]^q for i in 1:length(u)])
end
# HIP(u::fq_nmod_mat, v::fq_nmod_mat) = Hermitianinnerproduct(u, v)

function Hermitianconjugatematrix(A::fq_nmod_mat)
    B = copy(A)
    q2 = order(base_ring(A))
    issquare(q2) || error("The Hermitian conjugate is only defined over quadratic field extensions.")
    q = Int64(sqrt(q2))
    return B .^ q
end

function printstringarray(A::Vector{String}, withoutIs=false)
    for a in A
        if !withoutIs
            println(a)
        else
            for i in a
                if i == 'I'
                    print(' ')
                else
                    print(i)
                end
            end
            print('\n')
        end
    end
end
printchararray(A::Vector{Vector{Char}}, withoutIs=false) = printstringarray(setchartostringarray(A), withoutIs)
printsymplecticarray(A::Vector{Vector{T}}, withoutIs=false) where T <: Integer = printstringarray(setsymplectictostringarray(A), withoutIs)

function entropy(x::Real)
    x != 0 || return 0
    (0 < x <= 1 - 1 / q) || error("Number should be in the range [0, 1 - 1/order(field)].")
    F = parent(x)
    q = order(F)
    return x * (log(q, q - 1) - log(q, x)) - (1 - x) * log(q, 1 - x)
end

function FpmattoJulia(M::fq_nmod_mat)
    degree(base_ring(M)) == 1 || error("Cannot promote higher order elements to the integers.")
    # Fp = [i for i in 0:Int64(characteristic(base_ring(M)))]
    A = zeros(Int64, size(M))
    for r in 1:size(M, 1)
        for c in 1:size(M, 2)
            # A[r, c] = Fp[findfirst(x->x==M[r, c], Fp)]
            A[r, c] = coeff(M[r, c], 0)
        end
    end
    return A
end

function pseudoinverse(M::fq_nmod_mat, verify::Bool=true)
    # let this fail elsewhere if not actually over a quadratic extension
    if degree(base_ring(M)) != 1
        M = deepcopy(quadratictosymplectic(M)')
    else
        M = deepcopy(M')
    end

    nr, nc = size(M)
    MS = MatrixSpace(base_ring(M), nr, nr)
    _, E = rref(hcat(M, MS(1)))
    E = E[:, (nc + 1):end]
    pinv = E[1:nc, :]
    dual = E[nc + 1:nr, :]

    if verify
        _, Mrref = rref(M)
        MScols = MatrixSpace(base_ring(M), nc, nc)
        E * M == Mrref || error("Pseudoinverse calculation failed (transformation incorrect).")
        Mrref[1:nc, 1:nc] == MScols(1) || error("Pseudoinverse calculation failed (failed to get I).")
        iszero(Mrref[nc + 1:nr, :]) || error("Pseudoinverse calculation failed (failed to get zero).")
        pinv * M == MScols(1) || error("Pseudoinverse calculation failed (eq 1).")
        M' * pinv' == MScols(1) || error("Pseudoinverse calculation failed (eq 2).")
        iszero(M' * dual') || error("Failed to correctly compute dual (rhs).")
        iszero(dual * M) || error("Failed to correctly compute dual (lhs).")
    end
    return pinv
end

function istriorthogonal(G::fq_nmod_mat, verbose::Bool=false)
    Int(order(base_ring(G))) == 2 || error("Triothogonality is only defined over 𝔽_2.")
    nr, nc = size(G)
    for r1 in 1:nr
        for r2 in 1:nr
            @views g1 = G[r1, :]
            @views g2 = G[r2, :]
            @views if !iszero(sum([g1[1, i] * g2[1, i] for i in 1:nc]))
                verbose && println("Rows $r1 and $r2 are not orthogonal.")
                return false
            end
        end
    end

    for r1 in 1:nr
        for r2 in 1:nr
            for r3 in 1:nr
                @views g1 = G[r1, :]
                @views g2 = G[r2, :]
                @views g3 = G[r3, :]
                @views if !iszero(sum([g1[1, i] * g2[1, i] * g3[1, i] for i in 1:nc]))
                    verbose && println("Rows $r1, $r2, and $r3 are not orthogonal.")
                    return false
                end
            end
        end
    end
    return true
end

function istriorthogonal(G::Matrix{Int})
    nr, nc = size(G)
    for r1 in 1:nr
        for r2 in 1:nr
            @views g1 = G[r1, :]
            @views g2 = G[r2, :]
            @views if !iszero(sum([g1[1, i] * g2[1, i] for i in 1:nc]) % 2)
                return false
            end
        end
    end

    for r1 in 1:nr
        for r2 in 1:nr
            for r3 in 1:nr
                @views g1 = G[r1, :]
                @views g2 = G[r2, :]
                @views g3 = G[r3, :]
                @views if !iszero(sum([g1[1, i] * g2[1, i] * g3[1, i] for i in 1:nc]) % 2)
                    return false
                end
            end
        end
    end
    return true
end


export getRotate3D,rotateAndMove3D,rotateAndMove3DTranspose,smooth3D,rotateSensitivity3D

function loc2cs3D(loc1::Union{Int64,Array{Int64}},loc2::Union{Int64,Array{Int64}},loc3::Union{Int64,Array{Int64}},n::Array{Int64,1})
@inbounds cs = loc1 .+ (loc2.-1)*n[1] .+ (loc3.-1)*n[1]*n[2];
return cs;
end

function getRotate3D(theta_rad::Float64,phi_rad::Float64)
## theta rotates on the x-y plain (or x1-x2)
## phi rotates on the x-z plain (or x1-x3).
ct = cos(theta_rad); st = sin(theta_rad);
cp = cos(phi_rad); sp = sin(phi_rad);
Rxy = [ct -st 0.0; st ct 0.0; 0.0 0.0 1.0];
Rxz = [cp 0.0 sp; 0.0 1.0 0.0; -sp 0.0 cp];
R   = Rxz*Rxy;
return R;
end

function rotateAndMove3D(u::Array{Float64,3},theta_phi_rad::Vector{Float64},b::Vector{Float64},doTranspose=false,v::Array = copy(u),XT::Array = zeros(0),XTT = zeros(0))
###
# b here is in pixels - not in Mesh units (no mesh here...)
###
n = collect(size(u));
v[:] .= 0.0;
R = getRotate3D(theta_phi_rad[1],theta_phi_rad[2]);
invR = inv(R);
midphalf = n./2.0 .+ 0.5;

midphalf_plus_b = midphalf .+ b;

maxU = maximum(u);
thresh = 1e-4*maxU;

xt = zeros(3);
x = zeros(3);

if length(XT)==0
	XT = zeros(3,prod(n));
	XTT = zeros(3,prod(n));
	# println("Building XT");
else
	XT[:].=0.0;
	XTT[:].=0.0;
end
for i3 = 1:size(u,3)
	for i2 = 1:size(u,2)
		for i1 = 1:size(u,1)
			@inbounds xt[1] = i1; xt[2] = i2; xt[3] = i3;
			# x = xt - midphalf_plus_b;
			BLAS.axpy!(-1.0,midphalf_plus_b,xt); # xt = xt - midphalf_plus_b;
			ii = loc2cs3D(i1,i2,i3,n);
			@inbounds XT[:,ii] = xt;
		end
	end
end
# gemm!(tA, tB, alpha, A, B, beta, C)
XTT[1,:] .= midphalf[1];
XTT[2,:] .= midphalf[2];
XTT[3,:] .= midphalf[3];
BLAS.gemm!('N','N',1.0,invR,XT,1.0,XTT);
# I think that here we assume that the domain is square to simplify.
II = findall(vec(sum((XTT .> 0.5) .& (XTT .< n[1]),dims=1).>=3));

LOC = round.(Int64,XTT[:,II]);
LOC = loc2cs3D(LOC[1,:],LOC[2,:],LOC[3,:],n);

if !doTranspose
	v[II] = u[LOC];
else
	# The following two options are actually different in terms of repeatitions:
	# v[LOC] .+= u[II];
	for ii=1:length(LOC)
		v[LOC[ii]] += u[II[ii]];
	end
end

# the code below is an altenative to the code above.
# iii=1;
# for i3 = 1:size(u,3)
	# for i2 = 1:size(u,2)
		# for i1 = 1:size(u,1)
			# @inbounds xt[1] = i1; xt[2] = i2; xt[3] = i3;
			# x = xt - midphalf_plus_b;
			# BLAS.axpy!(-1.0,midphalf_plus_b,xt); # xt = xt - midphalf_plus_b;
			# BLAS.gemv!('N',1.0,invR,xt,0.0,x); # x = 0.0*x + invR*xt;
			# BLAS.axpy!(1.0,midphalf,x); # x = xt + midphalf;
			# @inbounds x[:] = XTT[:,iii];
			# iii+=1;

			# @inbounds if x[1] < 0.5 || (x[1] > n[1]) || (x[2] < 0.5) || (x[2] > n[2])|| x[3] < 1 || x[3] > n[3]
				# continue;
			# end
			# ii = round.(Int64,x);
			# @inbounds v[i1,i2,i3] = u[ii[1],ii[2],ii[3]];
		# end
	# end
# end
return v,XT,XTT;
end

# function rotateAndMove3DTranspose(v::Array,theta_phi_rad::Array{Float64},b::Array{Float64})
# ###
# # b here is in pixels - not in Mesh units (no mesh here...)
# ###
# n = collect(size(v));
# u = zeros(eltype(v),size(v));
# R = getRotate3D(theta_phi_rad[1],theta_phi_rad[2]);
# invR = inv(R);
# mid = n/2.0;
# maxU = maximum(v);
# thresh = 1e-4*maxU;
# for i3 = 1:size(v,3)
	# for i2 = 1:size(v,2)
		# for i1 = 1:size(v,1)
			# @inbounds vi = v[i1,i2,i3];
			# if abs(vi) > thresh
				# x = [i1;i2;i3] .- (mid .+ 0.5 ) .- b; ### here we actually do the inverse operation in a forward way
				# x = invR*x;
				# x .+= mid;
				# @inbounds if x[1] <=  0.5 || (x[1] >= n[1]-0.5) || (x[2] <= 0.5) || (x[2] >= n[2]-0.5)|| x[3] <= 0.5 || x[3] >= n[3]-0.5
					# warn("Rotation3D: rotation went out of bounds. Please increase zero padding.");
					# continue;
				# end
				# x .+= 0.5;
				# xloc = convert(Array{Int64},round.(x));
				# @inbounds u[xloc[1],xloc[2],xloc[3]] += vi;
			# end
		# end
	# end
# end
# return u;
# end

# function smooth3D(v::Array,numFilters = 1)
# N = collect(-1:1);
# E3 = ones(3,3,3);
# E2 = [1.0 ; 2.0 ; 1.0]*[1.0 ; 2.0 ; 1.0]';
# E3[:,:,1] = E2;
# E3[:,:,2] = 2*E2;
# E3[:,:,3] = E2;
# E = E ./ sum(E[:]);

# for k = 1:2
	# for i3 = 2:size(u,3)-1
		# for i2 = 2:size(u,2)-1
			# for i1 = 2:size(u,1)-1
				# v[i1, i2, i3] = sum(E.*v[i1 + N, i2 + N, i3 + N]);
			# end
		# end
	# end
# end
# return v;
# end


using Test, YAAD, YAAD.TestUtils
using DiffRules, SpecialFunctions, NaNMath

@testset "broadcast math.jl" begin

# exclude function whose domain is not in [0, 1)
exclusion = Symbol[
    :asec,
    :acosh,
    :acsc,
    :acoth,
    :asecd,
    :acscd,
]

for (mod, name, nargs) in keys(DiffRules.DEFINED_DIFFRULES)
    f_ex_head = Expr(:., mod, QuoteNode(name))

    if nargs == 1
        name in exclusion && continue

        @eval @test gradcheck(x->Broadcast.broadcasted($f_ex_head, x), Variable(rand(10, 10)))
    end
end

@test gradcheck(x->Broadcast.broadcasted(asec, x), Variable(rand(10, 10) .+ 2.0))
@test gradcheck(x->Broadcast.broadcasted(acosh, x), Variable(rand(10, 10) .+ 2.0))
@test gradcheck(x->Broadcast.broadcasted(acsc, x), Variable(rand(10, 10) .+ 2.0))
@test gradcheck(x->Broadcast.broadcasted(acoth, x), Variable(rand(10, 10) .+ 2.0))
@test gradcheck(x->Broadcast.broadcasted(asecd, x), Variable(rand(10, 10) .+ 2.0))
@test gradcheck(x->Broadcast.broadcasted(acscd, x), Variable(rand(10, 10) .+ 2.0))

end # math.jl

gradcheck(x->Broadcast.broadcasted(Base.csch, x), Variable(rand(10, 10)))


@testset "Sequence" begin
    schedules = (log, sqrt)
    step_sizes = (rand(1:10), rand(1:10))
    s = Sequence(schedules, step_sizes)
    @test s == Sequence([s => t for (s, t) in zip(schedules, step_sizes)]...)
    i = rand(UInt) + 1
    @test s(i) == ((i > step_sizes[1]) ? sqrt(i - step_sizes[1]) : log(i))
    @test all(p == ((t > step_sizes[1]) ? sqrt(t - step_sizes[1]) : log(t))
              for (t, p) in zip(1:(sum(step_sizes) + 10), s))
    @test Base.IteratorEltype(typeof(s)) == Base.EltypeUnknown()
    @test Base.IteratorSize(typeof(s)) == Base.SizeUnknown()

    params = [rand(), rand()]
    s = Sequence(params, step_sizes)
    @test s(i) == ((i > step_sizes[1]) ? params[2] : params[1])
    @test all(p == ((t > step_sizes[1]) ? params[2] : params[1])
              for (t, p) in zip(1:(sum(step_sizes) + 10), s))
end

@testset "Loop" begin
    period = rand(1:10)
    s = Loop(log, period)
    i = rand(UInt) + 1
    @test s(i) == log(mod1(i, period))
    @test all(p == log(mod1(t, period)) for (t, p) in zip(1:100, s))
    @test Base.IteratorEltype(typeof(s)) == Base.IteratorEltype(typeof(log))
    @test Base.IteratorSize(typeof(s)) == Base.IsInfinite()
    @test axes(s) == (OneToInf(),)

    @testset "utilities" begin
        reverse_f = ParameterSchedulers.reverse(log, period)
        @test all(reverse_f(period - t) == log(t) for t in 1:period)
        symmetric_f = ParameterSchedulers.symmetric(log, period)
        @test all(symmetric_f(t) == ((t < period / 2) ? log(t) : log(period - t)) for t in 1:period)
    end
end

@testset "Stateful" begin
    stateful_s = Stateful(log)
    @test all(next!(stateful_s) == log(i) for i in 1:100)
    reset!(stateful_s)
    @test log(1) == next!(stateful_s)
    stateful_s = Stateful(log; advance = s -> s == 1)
    @test log(1) == next!(stateful_s)
    @test log(2) == next!(stateful_s)
    @test log(2) == next!(stateful_s)
end


using Vec
using Base.Test

include("test_vecE2.jl")
include("test_vecE3.jl")
include("test_vecSE2.jl")

include("test_coordinate_transforms.jl")
include("test_geomE2.jl")