# # Incompressible Navier-Stokes Equations via DifferentialEquations.jl
#
# ![](https://user-images.githubusercontent.com/9196588/134514213-76d91d34-19ab-47c2-957e-16bb0c8669e1.gif)
#
#
# In this example we focus on a simple but visually appealing problem from
# fluid dynamics, namely vortex shedding. This problem is also known as
# [von-Karman vortex streets](https://en.wikipedia.org/wiki/K%C3%A1rm%C3%A1n_vortex_street). Within this example, we show how to utilize [DifferentialEquations.jl](https://github.com/SciML/DifferentialEquations.jl)
# in tandem with Ferrite.jl to solve this space-time problem. To keep things simple we use a naive approach
# to discretize the system.
#
# ## Remarks on DifferentialEquations.jl
#
# Many "time step solvers" of [DifferentialEquations.jl](https://github.com/SciML/DifferentialEquations.jl) assume that that the
# problem is provided in mass matrix form. The incompressible Navier-Stokes
# equations as stated above yield a DAE in this form after applying a spatial
# discretization technique - in our case FEM. The mass matrix form of ODEs and DAEs
# is given as:
# ```math
#   M(t) \mathrm{d}_t u = f(u,t)
# ```
# where $M$ is a possibly time-dependent and not necessarily invertible mass matrix,
# $u$ the vector of unknowns and $f$ the right-hand-side (RHS). For us $f$ can be interpreted as
# the spatial discretization of all linear and nonlinear operators depending on $u$ and $t$,
# but not on the time derivative of $u$.
#
# ## Some Theory on the Incompressible Navier-Stokes Equations
#
# ### Problem Description in Strong Form
#
# The incompressible Navier-Stokes equations can be stated as the system
# ```math
#  \begin{aligned}
#    \partial_t v &= \underbrace{\nu \Delta v}_{\text{viscosity}} - \underbrace{(v \cdot \nabla) v}_{\text{advection}} - \underbrace{\nabla p}_{\text{pressure}} \\
#               0 &= \underbrace{\nabla \cdot v}_{\text{incompressibility}}
#  \end{aligned}
# ```
# where $v$ is the unknown velocity field, $p$ the unknown pressure field,
# $\nu$ the dynamic viscosity and $\Delta$ the Laplacian. In the derivation we assumed
# a constant density of 1 for the fluid and negligible coupling between the velocity components.
# Finally we see that the pressure term appears only in combination with the gradient
# operator, so for any solution $p$ the function $p + c$ is also an admissible solution, if
# we do not impose Dirichlet conditions on the pressure. To resolve this we introduce the
# implicit constraint that $ \int_\Omega p = 0 $.
#
# Our setup is derived from [Turek's DFG benchmark](http://www.mathematik.tu-dortmund.de/~featflow/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark1_re20.html).
# We model a channel with size $0.41 \times 2.2$ and a hole of radius $0.05$ centered at $(0.2, 0.2)$.
# The left side has a parabolic inflow profile, which is ramped up over time, modeled as the time dependent
# Dirichlet condition
# ```math
#  v(x,y,t)
#  =
#  \begin{bmatrix}
#      4 v_{in}(t) y (0.41-y)/0.41^2 \\
#      0
#  \end{bmatrix}
# ```
# where $v_{in}(t) = \text{clamp}(t, 0.0, 1.0)$. With a dynamic viscosity of $\nu = 0.001$
# this is enough to induce turbulence behind the cylinder which leads to vortex shedding. The top and bottom of our
# channel have no-slip conditions, i.e. $v = [0,0]^{\textrm{T}}$, while the right boundary has the do-nothing boundary condtion
# $\nu \partial_{\textrm{n}} v - p n = 0$ to model outflow. With these boundary conditions we can choose the zero solution as a
# feasible initial condition.
#
# ### Derivation of Semi-Discrete Weak Form
#
# By multiplying test functions $\varphi$ and $\psi$ from a suitable test function space on the strong form,
# followed by integrating over the domain and applying partial integration to the pressure and viscosity terms
# we can obtain the following weak form
# ```math
#  \begin{aligned}
#    \int_\Omega \partial_t v \cdot \varphi &= - \int_\Omega \nu \nabla v : \nabla \varphi - \int_\Omega (v \cdot \nabla) v \cdot \varphi + \int_\Omega p (\nabla \cdot \varphi) + \int_{\partial \Omega_{N}} \underbrace{(\nu \partial_n v - p n )}_{=0} \cdot \varphi \\
#                                  0 &= \int_\Omega (\nabla \cdot v) \psi
#  \end{aligned}
# ```
# for all possible test functions from the suitable space.
#
# Now we can discretize the problem as usual with the finite element method
# utilizing Taylor-Hood elements (Q2Q1) to yield a stable discretization in
# mass matrix form:
# ```math
#  \underbrace{\begin{bmatrix}
#      M_v & 0 \\
#       0  & 0
#  \end{bmatrix}}_{:=M}
#  \begin{bmatrix}
#      \mathrm{d}_t\hat{v} \\
#      \mathrm{d}_t\hat{p}
#  \end{bmatrix}
#  =
#  \underbrace{\begin{bmatrix}
#       A & B^{\textrm{T}} \\
#       B & 0
#  \end{bmatrix}}_{:=K}
#  \begin{bmatrix}
#      \hat{v} \\
#      \hat{p}
#  \end{bmatrix}
#  +
#  \begin{bmatrix}
#      N(\hat{v}, \hat{v}, \hat{\varphi}) \\
#      0
#  \end{bmatrix}
# ```
# Here $M$ is the singular block mass matrix, $K$ is the discretized Stokes operator and $N$ the nonlinear advection term, which
# is also called trilinear form. $\hat{v}$ and $\hat{p}$ represent the time-dependent vectors of nodal values of the discretizations
# of $v$ and $p$ respectively, while $\hat{\varphi}$ is the choice for the test function in the discretization. The hats are dropped
# in the implementation and only stated for clarity in this section.
#
#
# ## Commented Implementation
#
# Now we solve the problem with Ferrite and [DifferentialEquations.jl](https://github.com/SciML/DifferentialEquations.jl). What follows is a program spliced with comments.
# The full program, without comments, can be found in the next [section](@ref ns_vs_diffeq-plain-program).
#
# First we load Ferrite and some other packages we need
using Ferrite, SparseArrays, BlockArrays, LinearAlgebra, UnPack
# Since we do not need the complete DifferentialEquations suite, we just load the required ODE infrastructure, which can also handle
# DAEs in mass matrix form.
using OrdinaryDiffEq

# We start off by defining our only material parameter.
ν = 1.0/1000.0; #dynamic viscosity

# Next a fine 2D rectangular grid has to be generated. We leave the cell size parametric for flexibility when
# playing around with the code. Note that the mesh is pretty fine, leading to a high memory consumption when
# feeding the equation system to direct solvers.
dim = 2
cell_scale_factor = 2.0
x_cells = round(Int, cell_scale_factor*220)
y_cells = round(Int, cell_scale_factor*41)
# CI chokes if the grid is too fine. :)     #src
x_cells = round(Int, 55/3)                  #hide
y_cells = round(Int, 41/3)                  #hide
grid = generate_grid(Quadrilateral, (x_cells, y_cells), Vec{2}((0.0, 0.0)), Vec{2}((2.2, 0.41)));

# Next we carve a hole $B_{0.05}(0.2,0.2)$ in the mesh by deleting the cells and update the boundary face sets.
# This code will be replaced once a proper mesh interface is avaliable.
cell_indices = filter(ci->norm(mean(map(i->grid.nodes[i].x-[0.2,0.2], Ferrite.vertices(grid.cells[ci]))))>0.05, 1:length(grid.cells))
hole_cell_indices = filter(ci->norm(mean(map(i->grid.nodes[i].x-[0.2,0.2], Ferrite.vertices(grid.cells[ci]))))<=0.05, 1:length(grid.cells));
hole_face_ring = Set{FaceIndex}()
for hci ∈ hole_cell_indices
    push!(hole_face_ring, FaceIndex((hci+1, 4)))
    push!(hole_face_ring, FaceIndex((hci-1, 2)))
    push!(hole_face_ring, FaceIndex((hci-x_cells, 3)))
    push!(hole_face_ring, FaceIndex((hci+x_cells, 1)))
end
grid.facesets["hole"] = Set(filter(x->x.idx[1] ∉ hole_cell_indices, collect(hole_face_ring)));
cell_indices_map = map(ci->norm(mean(map(i->grid.nodes[i].x-[0.2,0.2], Ferrite.vertices(grid.cells[ci]))))>0.05 ? indexin([ci], cell_indices)[1] : 0, 1:length(grid.cells))
grid.cells = grid.cells[cell_indices]
for facesetname in keys(grid.facesets)
    grid.facesets[facesetname] = Set(map(fi -> FaceIndex( cell_indices_map[fi.idx[1]] ,fi.idx[2]), collect(grid.facesets[facesetname])))
end;

# We test against full development of the flow - so regenerate the grid                              #src
grid = generate_grid(Quadrilateral, (x_cells, y_cells), Vec{2}((0.0, 0.0)), Vec{2}((0.55, 0.41)));   #hide

# ### Function Space
# To ensure stability we utilize the Taylor-Hood element pair Q2-Q1.
# We have to utilize the same quadrature rule for the pressure as for the velocity, because in the weak form the
# linear pressure term is tested against a quadratic function.
ip_v = Lagrange{dim, RefCube, 2}()
ip_geom = Lagrange{dim, RefCube, 1}()
qr = QuadratureRule{dim, RefCube}(4)
cellvalues_v = CellVectorValues(qr, ip_v, ip_geom);

ip_p = Lagrange{dim, RefCube, 1}()
cellvalues_p = CellScalarValues(qr, ip_p, ip_geom);

dh = DofHandler(grid)
push!(dh, :v, dim, ip_v)
push!(dh, :p, 1, ip_p)
close!(dh);

# ### Boundary Conditions
# As in the DFG benchmark we apply no-slip conditions to the top, bottom and
# cylinder boundary. The no-slip condition states that the velocity of the
# fluid on this portion of the boundary is fixed to be zero.
ch = ConstraintHandler(dh);

nosplip_face_names = ["top", "bottom", "hole"];
# No hole for the test present                                          #src
nosplip_face_names = ["top", "bottom"]                                  #hide
∂Ω_noslip = union(getfaceset.((grid, ), nosplip_face_names)...);
noslip_bc = Dirichlet(:v, ∂Ω_noslip, (x, t) -> [0,0], [1,2])
add!(ch, noslip_bc);

# The left boundary has a parabolic inflow with peak velocity of 1.0. This
# ensures that for the given geometry the Reynolds number is 100, which
# is already enough to obtain some simple vortex streets. By increasing the
# velocity further we can obtain stronger vortices - which may need additional
# refinement of the grid.
∂Ω_inflow = getfaceset(grid, "left");

vᵢₙ(t) = clamp(t, 0.0, 1.0)*1.0 #inflow velocity
vᵢₙ(t) = clamp(t, 0.0, 1.0)*0.3 #hide
parabolic_inflow_profile((x,y),t) = [4*vᵢₙ(t)*y*(0.41-y)/0.41^2,0]
inflow_bc = Dirichlet(:v, ∂Ω_inflow, parabolic_inflow_profile, [1,2])
add!(ch, inflow_bc);

# The outflow boundary condition has been applied on the right side of the
# cylinder when the weak form has been derived by setting the boundary integral
# to zero. It is also called the do-nothing condition. Other outflow conditions
# are also possible.
∂Ω_free = getfaceset(grid, "right");

close!(ch)
update!(ch, 0.0);

# ### Linear System Assembly
# Next we describe how the block mass matrix and the Stokes matrix are assembled.
#
# For the block mass matrix $M$ we remember that only the first equation had a time derivative
# and that the block mass matrix corresponds to the term arising from discretizing the time
# derivatives. Hence, only the upper left block has non-zero components.
function assemble_mass_matrix(cellvalues_v::CellVectorValues{dim}, cellvalues_p::CellScalarValues{dim}, M::SparseMatrixCSC, dh::DofHandler) where {dim}
    ## Allocate a buffer for the local matrix and some helpers, together with the assembler.
    n_basefuncs_v = getnbasefunctions(cellvalues_v)
    n_basefuncs_p = getnbasefunctions(cellvalues_p)
    n_basefuncs = n_basefuncs_v + n_basefuncs_p
    v▄, p▄ = 1, 2
    Mₑ = PseudoBlockArray(zeros(n_basefuncs, n_basefuncs), [n_basefuncs_v, n_basefuncs_p], [n_basefuncs_v, n_basefuncs_p])

    ## It follows the assembly loop as explained in the basic tutorials.
    mass_assembler = start_assemble(M)
    @inbounds for cell in CellIterator(dh)
        fill!(Mₑ, 0)
        Ferrite.reinit!(cellvalues_v, cell)

        for q_point in 1:getnquadpoints(cellvalues_v)
            dΩ = getdetJdV(cellvalues_v, q_point)
            ## Remember that we assemble a vector mass term, hence the dot product.
            for i in 1:n_basefuncs_v
                φᵢ = shape_value(cellvalues_v, q_point, i)
                for j in 1:n_basefuncs_v
                    φⱼ = shape_value(cellvalues_v, q_point, j)
                    Mₑ[BlockIndex((v▄, v▄), (i, j))] += φᵢ ⋅ φⱼ * dΩ
                end
            end
        end
        assemble!(mass_assembler, celldofs(cell), Mₑ)
    end

    return M
end;

# Next we discuss the assembly of the Stokes matrix.
# Remember that we use the same function spaces for trial and test, hence the
# matrix has the following block form
# ```math
#   K = \begin{bmatrix}
#       A & B^{\textrm{T}} \\
#       B & 0
#   \end{bmatrix}
# ```
# which is also called saddle point matrix. These problems are known to have
# a non-trivial kernel, which is a reflection of the strong form as discussed
# in the theory portion if this example.
function assemble_stokes_matrix(cellvalues_v::CellVectorValues{dim}, cellvalues_p::CellScalarValues{dim}, ν, K::SparseMatrixCSC, dh::DofHandler) where {dim}
    ## Again, some buffers and helpers
    n_basefuncs_v = getnbasefunctions(cellvalues_v)
    n_basefuncs_p = getnbasefunctions(cellvalues_p)
    n_basefuncs = n_basefuncs_v + n_basefuncs_p
    v▄, p▄ = 1, 2
    Kₑ = PseudoBlockArray(zeros(n_basefuncs, n_basefuncs), [n_basefuncs_v, n_basefuncs_p], [n_basefuncs_v, n_basefuncs_p])

    ## Assembly loop
    stiffness_assembler = start_assemble(K)
    @inbounds for cell in CellIterator(dh)
        ## Don't forget to initialize everything
        fill!(Kₑ, 0)

        Ferrite.reinit!(cellvalues_v, cell)
        Ferrite.reinit!(cellvalues_p, cell)

        for q_point in 1:getnquadpoints(cellvalues_v)
            dΩ = getdetJdV(cellvalues_v, q_point)
            # Assemble local viscosity block of $A$
            #+
            for i in 1:n_basefuncs_v
                ∇φᵢ = shape_gradient(cellvalues_v, q_point, i)
                for j in 1:n_basefuncs_v
                    ∇φⱼ = shape_gradient(cellvalues_v, q_point, j)
                    Kₑ[BlockIndex((v▄, v▄), (i, j))] -= ν * ∇φᵢ ⊡ ∇φⱼ * dΩ
                end
            end
            # Assemble local pressure and incompressibility blocks of $B^{\textrm{T}}$ and $B$.
            #+
            for j in 1:n_basefuncs_p
                ψ = shape_value(cellvalues_p, q_point, j)
                for i in 1:n_basefuncs_v
                    divφ = shape_divergence(cellvalues_v, q_point, i)
                    Kₑ[BlockIndex((v▄, p▄), (i, j))] += (divφ * ψ) * dΩ
                    Kₑ[BlockIndex((p▄, v▄), (j, i))] += (ψ * divφ) * dΩ
                end
            end
        end

        ## Assemble `Kₑ` into the Stokes matrix `K`.
        assemble!(stiffness_assembler, celldofs(cell), Kₑ)
    end
    return K
end;

# ### Solution of the semi-discretized system via DifferentialEquations.jl
# First we assemble the linear portions for efficiency. These matrices are
# assumed to be constant over time.
T = 10.0
Δt₀ = 0.01
Δt_save = 0.1

M = create_sparsity_pattern(dh);
M = assemble_mass_matrix(cellvalues_v, cellvalues_p, M, dh);

K = create_sparsity_pattern(dh);
K = assemble_stokes_matrix(cellvalues_v, cellvalues_p, ν, K, dh);

# These are our initial conditions. We start from the zero solution, because it
# is trivially admissible if the Dirichlet conditions are zero everywhere on the
# Dirichlet boundary for $t=0$. Note that the time stepper is also doing fine if the
# Dirichlet condition is non-zero and not too pathological.
u₀ = zeros(ndofs(dh))
apply!(u₀, ch);

# DifferentialEquations assumes dense matrices by default, which is not
# feasible for semi-discretization of finite element models. We communicate
# that a sparse matrix with specified pattern should be utilized through the
# `jac_prototyp` argument. It is simple to see that the Jacobian and the
# stiffness matrix share the same sparsity pattern, since they share the
# same relation between trial and test functions.
jac_sparsity = sparse(K);

# To apply the nonlinear portion of the Navier-Stokes problem we simply hand
# over the dof handler and cell values to the right-hand-side (RHS) as a parameter.
# Further the pre-assembled linear part (which is time independent) is
# passed to save some runtime. To apply the time-dependent Dirichlet BCs, we
# also hand over the constraint handler.
# The basic idea to apply the Dirichlet BCs consistently is that we copy the
# current solution `u`, apply the Dirichlet BCs on the copy, evaluate the
# discretized RHS of the Navier-Stokes equations with this vector
# and finally set the RHS to zero on every constraint. This way we obtain a
# correct solution for all dofs which are not Dirichlet constrained. These
# dofs are then corrected in a post-processing step, when evaluating the
# solution vector at specific time points.
# It should be finally noted that this **trick does not work** out of the box
# **for constraining algebraic portion** of the DAE, i.e. if we would like to
# put a Dirichlet BC on pressure dofs. As a workaround we have to set $f_{\textrm{i}} = 1$
# instead of $f_{\textrm{i}} = 0$, because otherwise the equation system gets singular.
# This is obvious when we remember that our mass matrix is zero for these
# dofs, such that we obtain the equation $0 \cdot \mathrm{d}_t p_{\textrm{i}} = 1 \cdot p_{\textrm{i}}$, which
# now has a unique solution.
struct RHSparams
    K::SparseMatrixCSC
    ch::ConstraintHandler
    dh::DofHandler
    cellvalues_v::CellVectorValues
end
p = RHSparams(K, ch, dh, cellvalues_v)

function navierstokes!(du,u_uc,p,t)
    # Unpack the struct to save some allocations.
    #+
    @unpack K,ch,dh,cellvalues_v = p

    # We start by applying the time-dependent Dirichlet BCs. Note that we are
    # not allowed to mutate `u_uc`! We also can not pre-allocate this variable
    # if we want to use AD to derive the Jacobian matrix, which appears in the
    # utilized implicit Euler. If we hand over the Jacobian analytically to
    # the solver, or when utilizing a method which does not require building the
    # Jacobian, then we could also hand over a buffer for `u` in our RHSparams
    # structure to save the allocations made here.
    #+
    u = copy(u_uc)
    update!(ch, t)
    apply!(u, ch)

    # Now we apply the rhs of the Navier-Stokes equations
    #+
    ## Linear contribution (Stokes operator)
    mul!(du, K, u) # du .= K * u

    ## nonlinear contribution
    n_basefuncs = getnbasefunctions(cellvalues_v)
    for cell in CellIterator(dh)
        Ferrite.reinit!(cellvalues_v, cell)
        all_celldofs = celldofs(cell)
        v_celldofs = all_celldofs[dof_range(dh, :v)]
        v_cell = u[v_celldofs]
        for q_point in 1:getnquadpoints(cellvalues_v)
            dΩ = getdetJdV(cellvalues_v, q_point)
            ∇v = function_gradient(cellvalues_v, q_point, v_cell)
            v = function_value(cellvalues_v, q_point, v_cell)
            for j in 1:n_basefuncs
                φⱼ = shape_value(cellvalues_v, q_point, j)
                # Note that in Tensors.jl the definition $\textrm{grad} v = \nabla v$ holds.
                # With this information it can be quickly shown in index notation that
                # ```math
                # [(v \cdot \nabla) v]_{\textrm{i}} = v_{\textrm{j}} (\partial_{\textrm{j}} v_{\textrm{i}}) = [v (\nabla v)^{\textrm{T}}]_{\textrm{i}}
                # ```
                # where we should pay attentation to the transpose of the gradient.
                #+
                du[v_celldofs[j]] -= v ⋅ ∇v' ⋅ φⱼ * dΩ
            end
        end
    end

    # For now we have to ingore the evolution of the Dirichlet BCs.
    # The DBC dofs in the solution vector will be corrected in a post-processing step.
    #+
    apply_zero!(du, ch)
end;
# Finally, together with our pre-assembled mass matrix, we are now able to
# define our problem in mass matrix form.
rhs = ODEFunction(navierstokes!, mass_matrix=M; jac_prototype=jac_sparsity)
problem = ODEProblem(rhs, u₀, (0.0,T), p);

# Now we can put everything together by specifying how to solve the problem.
# We want to use the adaptive implicit Euler method with our custom linear
# solver, which helps in the enforcement of the Dirichlet BCs. Further we
# enable the progress bar with the `progess` and `progress_steps` arguments.
# Finally we have to communicate the time step length and initialization
# algorithm. Since we start with a valid initial state we do not use one of
# DifferentialEquations.jl initialization algorithms.
# NOTE: At the time of writing this [no Hessenberg index 2 initialization is implemented](https://github.com/SciML/OrdinaryDiffEq.jl/issues/1019).
#
# To visualize the result we export the grid and our fields
# to VTK-files, which can be viewed in [ParaView](https://www.paraview.org/)
# by utilizing the corresponding pvd file.
timestepper = ImplicitEuler()
integrator = init(
    problem, timestepper, initializealg=NoInit(), dt=Δt₀,
    adaptive=true, abstol=1e-3, reltol=1e-3,
    progress=true, progress_steps=1,
    saveat=Δt_save);

pvd = paraview_collection("vortex-street.pvd");
integrator = TimeChoiceIterator(integrator, 0.0:Δt_save:T)
for (u_uc,t) in integrator
    # We ignored the Dirichlet constraints in the solution vector up to now,
    # so we have to bring them back now.
    #+
    update!(ch, t)
    u = copy(u_uc)
    apply!(u, ch)
    #compress=false flag because otherwise each vtk file will be stored in memory
    vtk_grid("vortex-street-$t.vtu", dh; compress=false) do vtk
        vtk_point_data(vtk,dh,u)
        vtk_save(vtk)
        pvd[t] = vtk
    end
end
vtk_save(pvd);

# Test the result for full proper development of the flow                   #src
using Test                                                                  #hide
function compute_divergence(dh, u, cellvalues_v)                            #hide
    divv = 0.0                                                              #hide
    @inbounds for (i,cell) in enumerate(CellIterator(dh))                   #hide
        Ferrite.reinit!(cellvalues_v, cell)                                 #hide
        for q_point in 1:getnquadpoints(cellvalues_v)                       #hide
            dΩ = getdetJdV(cellvalues_v, q_point)                           #hide
                                                                            #hide
            all_celldofs = celldofs(cell)                                   #hide
            v_celldofs = all_celldofs[dof_range(dh, :v)]                    #hide
            v_cell = u[v_celldofs]                                          #hide
                                                                            #hide
            divv += function_divergence(cellvalues_v, q_point, v_cell) * dΩ #hide
        end                                                                 #hide
    end                                                                     #hide
    return divv                                                             #hide
end                                                                         #hide
@testset "INS OrdinaryDiffEq" begin                                         #hide
    u = copy(integrator.integrator.u)                                       #hide
    apply!(u, ch)                                                           #hide
    Δdivv = abs(compute_divergence(dh, u, cellvalues_v))                    #hide
    @test isapprox(Δdivv, 0.0, atol=1e-12)                                  #hide
                                                                            #hide
    Δv = 0.0                                                                #hide
    for cell in CellIterator(dh)                                            #hide
        Ferrite.reinit!(cellvalues_v, cell)                                 #hide
        all_celldofs = celldofs(cell)                                       #hide
        v_celldofs = all_celldofs[dof_range(dh, :v)]                        #hide
        v_cell = u[v_celldofs]                                              #hide
        coords = getcoordinates(cell)                                       #hide
        for q_point in 1:getnquadpoints(cellvalues_v)                       #hide
            dΩ = getdetJdV(cellvalues_v, q_point)                           #hide
            coords_qp = spatial_coordinate(cellvalues_v, q_point, coords)   #hide
            v = function_value(cellvalues_v, q_point, v_cell)               #hide
            Δv += norm(v - parabolic_inflow_profile(coords_qp, T))^2*dΩ     #hide
        end                                                                 #hide
    end                                                                     #hide
    @test isapprox(sqrt(Δv), 0.0, atol=1e-3)                                #hide
end;                                                                        #hide

#md # ## [Plain Program](@id ns_vs_diffeq-plain-program)
#md #
#md # Below follows a version of the program without any comments.
#md # The file is also available here: [ns_vs_diffeq.jl](ns_vs_diffeq.jl)
#md #
#md # ```julia
#md # @__CODE__
#md # ```


using Gadfly

xs = 1:10.
ys = xs
zs = Float64[x^2*log(y) for x in xs, y in ys]
plot(x=xs, y=ys, z=zs, Geom.contour, Scale.color_none)


"""
    isPPT(x, sys :: Int, dims :: Vector) :: Bool

This function returns true if the input state x is PPT
with respect to the (sys)th system. False otherwise.
dims is a vector of the sizes of the subsystems.
"""
function isPPT(x,sys::Int,dims::Vector) :: Bool
    #We don't make sure the inputs are good because the partialtranspose function
    #will do that for us

    #Get PPT of x
    xPPT = partialtranspose(x,sys,dims)

    #Check if it is PSD
    if eigmin(xPPT) >= 0 || isapprox(0,eigmin(xPPT))
        return true
    else
        return false
    end
end

"""
    swapOperator(dim :: Int) :: Matrix{Float64}

This function is the swap operator ``\\mathbb{F}`` which is defined by the action
```math
\\mathbb{F}(u \\otimes v) = v \\otimes u \\hspace{5mm} u,v \\in \\mathcal{H}_{A} .
```
The function uses that ``\\mathbb{F} = \\sum_{a,b \\in \\Sigma} E_{a,b} \\otimes E_{b,a}``
where ``E_{a,b}`` is a square matrix of dimension ``\\Sigma`` with a one in the ``(a,b)``
entry and a ``0`` everywhere else.
"""
function swapOperator(dim :: Int) :: Matrix{Float64}
    swap_operator = zeros(dim^2,dim^2)

    for col_id in 1:dim^2
        # factoring col_id into Alice and Bob subsystem ids
        a_id = floor(Int64, (col_id-1)/dim)
        b_id = (col_id-1) % dim

        # swapping Alice and Bob subsystem ids to construct row_id
        row_id = b_id*dim + a_id + 1
        swap_operator[row_id, col_id] = 1
    end

    return swap_operator
end

"""
    permuteSubsystems(
        ρ:: Vector,
        perm::Vector{Int64},
        dims::Vector{Int64}
    ) :: Vector

This function returns the vector with the subsystems permuted. For example, given three
subspaces ``A,B,C``, and the permutation ``\\pi`` defined by ``(A,B,C) \\xrightarrow[]{\\pi} (C,A,B),``
the function implements the process:
```math
    |e_{i}\\rangle_{A} |e_j \\rangle_{B} |e_k \\rangle_{C} \\xrightarrow[]{\\pi}
    |e_{k} \\rangle_{C} |e_{i}\\rangle_{A}  |e_{j} \\rangle_{B}  ,
```
by re-indexing the vector, permuting the indices appropriately, and converting it
back into a vector.
"""
function permuteSubsystems(ρ:: Vector, perm::Vector{Int64},dims::Vector{Int64}) :: Vector
    #This is almost identical to Tony Cubitt's implementation of this function https://www.dr-qubit.org/matlab.html
    #This is largely because Julia does reshape column-wise as Matlab does
    orig_shape = size(ρ)
    num_subsys = length(perm)
    #Note certain things get reversed. This is because Julia does reshape column-wise
    permTup = Tuple((num_subsys+1) .- reverse(perm)) #Reshape requires tuples
    dimTup = Tuple(reverse(dims))
    result = reshape(permutedims(reshape(ρ,dimTup),permTup),orig_shape)
end
"""
    permuteSubsystems(
        ρ:: Matrix,
        perm::Vector{Int64},
        dims::Vector{Int64}
    ) :: Matrix

This function returns the matrix with the subsystems permuted. It is a generalization of the vector code.
For example, given three subspaces ``A,B,C``, and the permutation ``\\pi`` defined by ``(A,B,C) \\xrightarrow[]{\\pi} (C,A,B),``
the function implements the process:
```math
|e_{i}\\rangle\\langle e_{i}|_{A}\\otimes |e_j \\rangle\\langle e_j|_{B}\\otimes |e_k \\rangle\\langle e_k|_{C} \\xrightarrow[]{\\pi}
|e_{k} \\rangle\\langle e_{k}|_{C}\\otimes |e_{i}\\rangle\\langle e_{i}|_{A} \\otimes  |e_{j} \\rangle\\langle e_{j}|_{B} ,
```
by re-indexing the matrix, permuting the indices, and reconstructing the matrix.
Both bra and ket indices receive the same permutation.
"""
function permuteSubsystems(ρ:: Matrix,perm::Vector{Int64},dims::Vector{Int64}) :: Matrix
    #This is almost identical to Tony Cubitt's implementation of this function https://www.dr-qubit.org/matlab.html
    #This is largely because Julia does reshape column-wise as Matlab does
    if !isequal(size(ρ)...)
            throw(DomainError(ρ, "the input ρ is not a square matrix or a pure state"))
    end
    orig_shape = size(ρ)
    num_subsys = length(perm)
    #For the matrix version we do the same thing we just have twice as many indices
    #because we keep track of bras and kets
    permPrime = (num_subsys+1) .- reverse(perm)
    permTup = Tuple([permPrime num_subsys .+ permPrime])
    dimTup = Tuple([reverse(dims) reverse(dims)])
    result = reshape(permutedims(reshape(ρ,dimTup),permTup),orig_shape)

    return result
end
"""
    shiftOperator(d::Int64) :: Matrix

This function returns the operator that shifts the computational basis mod d
for complex Euclidean space of dimension d. That is, it returns the operator ``S``
defined by the action
```math
    S|k\\rangle = |k+1 \\mod d \\rangle
```
"""
function shiftOperator(d::Int64) :: Matrix
    id = Matrix(I,d,d)
    shift = id[[end,1:(end-1)...],:]

    return shift
end

"""
    discreteWeylOperator(m :: Int64, n :: Int64, d :: Int64)

This function returns the (m,n)^th unitary for generating the generalized
Bell basis. They are defined by their action on the computational basis:
```math
    U_{m,n}|e_{k}\\rangle = e^{2 \\pi mk i / d} |e_{k+n}\\rangle
```
These are the discrete Weyl Operator basis. See
[Section 3 of this paper](https://arxiv.org/abs/1004.1655) for further details.
!!! warning
    Different works define the discrete Weyl operators differently such that
    the phase or ordering may differ. Please check your reference.
"""
function discreteWeylOperator(m :: Int64, n :: Int64, d :: Int64) :: Matrix
    if m < 0 || n < 0
        throw(DomainError((n,m), "Make sure m,n ∈ [0,1,...,d-1]."))
    elseif m >= d || n >= d
        throw(DomainError((n,m), "Make sure m,n < d."))
    end
    λ = exp(2*pi*1im / d)
    U = zeros(ComplexF64,d,d)
    for k = 0 : d-1
        U[((k+n) % d) + 1,k+1] = λ^(k*m)
    end
    return U
end


using IntervalRootFinding
using Test

include("branch_and_bound.jl")
include("search_interface.jl")
include("roots.jl")
include("test_smiley.jl")
include("newton1d.jl")
include("quadratic.jl")
include("linear_eq.jl")
include("slopes.jl")


function ranks(x)
    ranks!(similar(x, Int), x)
end
function ranks!(out, x, temp=similar(x, Float64))
    perm = temp
    sortperm!(perm, x)
    b = out #zeros(length(x))#Float64.(a)
    i = 0
    while i < length(x)
        i += 1
        x1 = x[perm[i]]
        j = i
        while j+1 <= length(x) && x1 == x[perm[j+1]]
            j += 1
        end
        for k = i:j
            b[perm[k]] = (i+j)/2
        end
        i = j
    end
    b
end

"""
    erlenvelope(A, alpha=0.05)

Return envelope with global significance level alpha based on
Extreme Rank Length and an interval for alpha. The interval measures
the effect tie breaking

"""
function erlenvelope(A, alpha=0.05)
    N = length(A)
    ndims = length(size(A[1])) + 1
    colons = ntuple(i -> :, ndims-1)
    A1 = Array{eltype(A[1]), ndims}(undef, size(A[1])..., N)
    for (i, a) in enumerate(A)
        A1[colons..., i] = a
    end
    temp1 = similar(A, Int)
    temp2 = similar(A, Float64)
    Ar = mapslices(x -> ranks!(temp2, x, temp1), A1, dims=ndims)

    @. Ar = min(Ar, N - Ar + 1)
    Aerl = sort!(reshape(Ar, :, N), dims=1)
    n = round(Int, alpha*length(A))
    inds = partialsortperm(collect(eachcol(Aerl)), n+1:N)
    Ralpha = Aerl[1, inds[1]]
    alpha_interval = (count(x -> x < Ralpha, Aerl[1,:]), count(x -> x <= Ralpha, Aerl[1,:])) ./ N

    hi = similar(A[1])
    lo = similar(hi)
    for i in eachindex(CartesianIndices(hi))
        lo[i], hi[i] = extrema(view(A1, i, inds))
    end
    lo, hi, alpha_interval
end


# Implements a Multilinear Expression data structure

export
   Monomial,
   MultilinearExpression

# DATA TYPE DEFINITIONS
"""
    Monomial

Specifies a monomial as a coefficient of numeric type T and a sorted list of variable indices.
We assume the index 0 represents the constant (variable-free) monomial.
"""
struct Monomial #{T}
    #coef::T # coefficient
    vars::Tuple{Vararg{Int}} # list of variable indices
    # function Monomial{T}(coef::T,vars) where T <: Number
    #     @assert Base.issorted(vars)
    #     new(coef,vars)
    # end
    function Monomial(vars)
        @assert Base.issorted(vars)
        new(vars)
    end
end
"""
    MultilinearExpression

Implements a multilinear expression data type as a dictionary of monoids to coefficients.

### Examples
```julia
# Creates a constant expression
@show c = MultilinearExpression(7.4)
# Creates expressions with single term
@show e1 = MultilinearExpression(4.0,1)
@show e2 = MultilinearExpression(3.0,1,3)
````
"""
struct MultilinearExpression
    terms::Dict{Monomial,<:Number}
    MultilinearExpression(d::Dict{Monomial,<:Number}) = new(d)
end
# CONSTRUCTORS
"Creates an expression with a single constant c"
MultilinearExpression(c::Number) = MultilinearExpression(Dict(Monomial(tuple()) => c))
#Monomial(coef::T, vars::Tuple{Vararg{Int}}) where T <: Number = Monomial{T}(coef,vars)
"Creates an expression with a single term (given as a pair monomial => coefficient"
MultilinearExpression(t::Pair{Monomial,<:Number}) = MultilinearExpression(Dict(t))
# function MultilinearExpression(e::MultilinearExpression,t::Pair{Monomial,<:Number})
#     terms = copy(e.terms)
#     terms[t[1]] = t[2]
#     MultilinearExpression(terms)
# end
"Creates an expression with given coefficient and variables (indices)."
MultilinearExpression(c::Number,v1::Int,v...) = MultilinearExpression(Dict(Monomial((v1,v...)) => c))
#monomial(coef,v1,v...) = MultilinearExpression( Monomial((v1,v...)) => coef )
#monomial(coef,v1,v...) = MultilinearExpression( (Monomial(coef,(v1,v...)),) )
#monomial(c::Number) = MultilinearExpression(Monomial(tuple()) => c)
"Returns the aditivity identity expression."
Base.zero(::Type{MultilinearExpression}) = MultilinearExpression(0) # falls back to most basic identity constant
Base.zero(e::MultilinearExpression) = MultilinearExpression(zero(first(e.terms)[2]))
"Returns the multiplicative identity expression."
Base.one(::Type{MultilinearExpression}) = MultilinearExpression(1) # falls back to most basic numeric identity constant
Base.one(e::MultilinearExpression) = MultilinearExpression(one(first(e.terms)[2]))
# # GETTERS
Base.getindex(e::MultilinearExpression,m::Monomial) = get(e.terms,m,zero(first(e.terms)[2]))
Base.setindex!(e::MultilinearExpression,c::Number,m::Monomial) = setindex!(e.terms,c,m)
@inline terms(e::MultilinearExpression) = e.terms
# coeffficient(m::Monomial) = m.coef
@inline variables(m::Monomial) = m.vars
# # HELPERS
# "Verify if monomials are ordered in an expression"
# issorted(e::MultilinearExpression) = length(e) < 2 || ((variables(e[1]) < variables(e[2])) && issorted(Base.tail(e)))
"Iterate over the pairs monomials => coeeficient in the expression."
Base.iterate(e::MultilinearExpression) = Base.iterate(terms(e))
Base.iterate(e::MultilinearExpression,state) = Base.iterate(terms(e),state)
"Returns the number of monomials in the expression."
Base.length(e::MultilinearExpression) = length(terms(e))
# # INPUT/OUTPUT
Base.show(io::IO, m::Monomial) = print(io, "χ", join(variables(m),"χ"))
function Base.show(io::IO, t::Pair{Monomial,<:Number})
    if length(variables(t[1])) == 0
        print(io, t[2])
    else
        print(io, t[2], t[1])
    end
    nothing
end
#Base.show(io::IO, e::MultilinearExpression) = print(io, join(map(string,collect(terms(e)))," + "))
Base.show(io::IO, e::MultilinearExpression) = print(io, join(terms(e), " + "))
# # OPERATIONS
# """
#     Adds two expressions. Assumes the terms are ordered in each expression.
# """
function Base.:+(e1::MultilinearExpression,e2::MultilinearExpression)
    e = MultilinearExpression(copy(terms(e1)))
    for (m,c) in e2
        e[m] += c
    end
    e
#     @assert issorted(e1) && issorted(e2)
#     i, j = 1, 1
#     e = Vector{Monomial}()
#     while i <= length(e1) && j <= length(e2)
#         c1, c2 = coeffficient(e1[i]), coeffficient(e2[j])
#         v1, v2 = variables(e1[i]), variables(e2[j])
#         if v1 == v2
#             if c1 + c2 != zero(c1)
#                 push!(e,Monomial(c1 + c2, v1))
#             end
#             i += 1
#             j += 1
#         elseif v1 < v2
#             if c1 != zero(c1)
#                 push!(e,e1[i])
#             end
#             i += 1
#         else
#             if c2 != zero(c2)
#                 push!(e,e2[j])
#             end
#             j += 1
#         end
#     end
#     for k = i:length(e1)
#         push!(e,e1[k])
#     end
#     for k = j:length(e2)
#         push!(e,e2[k])
#     end
#     Tuple(e)
end
## Base.:+(e::MultilinearExpression,m::Monomial) = Base.:+(e,(m,))
# # Base.:+(m::Monomial,e::MultilinearExpression) = Base.:+((m,),e)
Base.:+(c::Number,e::MultilinearExpression) = MultilinearExpression(c) + e
Base.:+(e::MultilinearExpression,c::Number) = c + e
# Base.:*(c::T,m::Monomial{T}) where T <: Number = Monomial(c*coeffficient(m),variables(m))
Base.:*(c::Number,m::Monomial) = MultilinearExpression(m => c)
Base.:*(m::Monomial,c::Number) = c * m
function Base.:*(m1::Monomial,m2::Monomial)
    vars = union(variables(m1),variables(m2))
    @assert length(vars) == length(variables(m1)) + length(variables(m2)) "Multiplication of monomial with common variable is not multilinear"
    Monomial(Tuple(sort!(vars)))
end
function Base.:*(e1::MultilinearExpression,e2::MultilinearExpression)
    MultilinearExpression(Dict(
        m1 * m2 => c1 * c2
        for (m1,c1) in e1, (m2,c2) in e2
    ))
    # for (m1,c1) in e1
    #     for (m2,c2) in e2
    #         e[ m1 * m2 ] = c1 * c2
    #     end
    # end
    # e
    # @assert issorted(e1) && issorted(e2)
    # e = Dict{Tuple{Vararg{Int}},Monomial}()
    # Tuple( map(x -> x[2], sort!( collect(e), by = x -> x[1] ) ) )
end
Base.:*(c::Number,e::MultilinearExpression) = MultilinearExpression(c) * e
Base.:*(e::MultilinearExpression,c::Number) = c * e
## COMPARISONS
Base.:(==)(m1::Monomial,m2::Monomial) = variables(m1) == variables(m2)
function Base.:(==)(e1::MultilinearExpression,e2::MultilinearExpression)
    if length(e1) != length(e2)
        return false
    end
    for (m1,c1) in e1
        if e2[m1] != c1
            return false
        end
    end
    true
end
Base.hash(m::Monomial, h::UInt) = hash(variables(m), h)
Base.hash(t::Pair{Monomial,<:Number}, h::UInt) = hash(t[1],hash(t[2],h))
Base.hash(e::MultilinearExpression, h::UInt) = foldl( (t1,t2) -> hash(t1,hash(t2,h)), e ; init=hash(0))
## PROMOTION AND CONVERSION
Base.convert(::Type{MultilinearExpression},c::Number) = MultilinearExpression(c)
Base.convert(::Type{MultilinearExpression},p::Pair{Monomial,Number}) = MultilinearExpression(p)
Base.promote_rule(::Type{MultilinearExpression}, ::Type{T}) where T <: Number = MultilinearExpression


using Test

using MathOptInterface
const MOI = MathOptInterface
const MOIT = MathOptInterface.Test
const MOIU = MathOptInterface.Utilities
const MOIB = MathOptInterface.Bridges

include("../utilities.jl")

mock = MOIU.MockOptimizer(MOIU.UniversalFallback(MOIU.Model{Float64}()))
config = MOIT.TestConfig()

@testset "NormInfinity" begin
    bridged_mock = MOIB.Constraint.NormInfinity{Float64}(mock)

    MOIT.basic_constraint_tests(bridged_mock, config,
        include = [(F, MOI.NormInfinityCone) for F in [
            MOI.VectorOfVariables, MOI.VectorAffineFunction{Float64}, MOI.VectorQuadraticFunction{Float64}
        ]])

    @testset "norminf1test" begin
        mock.optimize! = (mock::MOIU.MockOptimizer) -> MOIU.mock_optimize!(mock,
            [1.0, 0.5, 1.0],
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives) => [[0.0, 1.0, 0.0, 0.0]],
            (MOI.VectorAffineFunction{Float64}, MOI.Zeros) => [[-1], [-1]])

        MOIT.norminf1vtest(bridged_mock, config)
        MOIT.norminf1ftest(bridged_mock, config)

        var_names = ["x", "y", "z"]

        @testset "Test mock model" begin
            MOI.set(mock, MOI.VariableName(), MOI.get(mock, MOI.ListOfVariableIndices()), var_names)
            nonneg = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())
            @test length(nonneg) == 1
            MOI.set(mock, MOI.ConstraintName(), nonneg[1], "nonneg")
            zeros = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Zeros}())
            @test length(zeros) == 2
            MOI.set(mock, MOI.ConstraintName(), zeros[1], "x_eq")
            MOI.set(mock, MOI.ConstraintName(), zeros[2], "y_eq")

            s = """
            variables: x, y, z
            nonneg: [x + -1.0y, x + -1.0z, x + y, x + z] in MathOptInterface.Nonnegatives(4)
            x_eq: [-1.0 + x] in MathOptInterface.Zeros(1)
            y_eq: [-0.5 + y] in MathOptInterface.Zeros(1)
            maxobjective: y + z
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(mock, model, var_names, ["nonneg", "x_eq", "y_eq"])
        end

        @testset "Test bridged model" begin
            MOI.set(bridged_mock, MOI.VariableName(), MOI.get(bridged_mock, MOI.ListOfVariableIndices()), var_names)
            norminf = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormInfinityCone}())
            @test length(norminf) == 1
            MOI.set(bridged_mock, MOI.ConstraintName(), norminf[1], "norminf")
            zeros = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Zeros}())
            @test length(zeros) == 2
            MOI.set(bridged_mock, MOI.ConstraintName(), zeros[1], "x_eq")
            MOI.set(bridged_mock, MOI.ConstraintName(), zeros[2], "y_eq")

            s = """
            variables: x, y, z
            norminf: [1.0x, y, z] in MathOptInterface.NormInfinityCone(3)
            x_eq: [-1.0 + x] in MathOptInterface.Zeros(1)
            y_eq: [-0.5 + y] in MathOptInterface.Zeros(1)
            maxobjective: y + z
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(bridged_mock, model, var_names, ["norminf", "x_eq", "y_eq"])
        end

        ci = first(MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormInfinityCone}()))
        nonneg = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())

        @testset "$attr" for attr in [MOI.ConstraintPrimalStart(), MOI.ConstraintDualStart()]
            @test MOI.supports(bridged_mock, attr, typeof(ci))
            value = [4.0, 1.0, -2.0]
            MOI.set(bridged_mock, attr, ci, value)
            @test MOI.get(bridged_mock, attr, ci) ≈ value
            if attr isa MOI.ConstraintPrimalStart
                nonneg_value = Float64[3, 6, 5, 2]
            else
                nonneg_value = [0.25, 2.25, 1.25, 0.25]
            end
            @test MOI.get(mock, attr, nonneg[1]) ≈ nonneg_value
        end

        test_delete_bridge(bridged_mock, ci, 3, (
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives, 0),))
    end

    @testset "norminf3test" begin
        mock.optimize! = (mock::MOIU.MockOptimizer) -> MOIU.mock_optimize!(mock,
            [2, -1, -1, -1],
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives) => [vcat(fill(inv(3), 3), zeros(3)), fill(inv(3), 3)])
        MOIT.norminf3test(bridged_mock, config)

        var_names = ["x", "y1", "y2", "y3"]

        @testset "Test mock model" begin
            MOI.set(mock, MOI.VariableName(), MOI.get(mock, MOI.ListOfVariableIndices()), var_names)
            nonneg = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())
            @test length(nonneg) == 2
            MOI.set(mock, MOI.ConstraintName(), nonneg[1], "nonneg1")
            MOI.set(mock, MOI.ConstraintName(), nonneg[2], "nonneg2")

            s = """
            variables: x, y1, y2, y3
            nonneg1: [x + -1.0y1 + -3.0, x + -1.0y2 + -3.0, x + -1.0y3 + -3.0, x + y1 + 1.0, x + y2 + 1.0, x + y3 + 1.0] in MathOptInterface.Nonnegatives(6)
            nonneg2: [y1 + 1.0, y2 + 1.0, y3 + 1.0] in MathOptInterface.Nonnegatives(3)
            minobjective: x
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(mock, model, var_names, ["nonneg1", "nonneg2"])
        end

        @testset "Test bridged model" begin
            MOI.set(bridged_mock, MOI.VariableName(), MOI.get(bridged_mock, MOI.ListOfVariableIndices()), var_names)
            norminf = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormInfinityCone}())
            @test length(norminf) == 1
            MOI.set(bridged_mock, MOI.ConstraintName(), norminf[1], "norminf")
            nonneg = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())
            @test length(nonneg) == 1
            MOI.set(bridged_mock, MOI.ConstraintName(), nonneg[1], "nonneg")

            s = """
            variables: x, y1, y2, y3
            norminf: [x + -1.0, y1 + 2.0, y2 + 2.0, y3 + 2.0] in MathOptInterface.NormInfinityCone(4)
            nonneg: [y1 + 1.0, y2 + 1.0, y3 + 1.0] in MathOptInterface.Nonnegatives(3)
            minobjective: x
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(bridged_mock, model, var_names, ["norminf", "nonneg"])
        end

        ci = first(MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormInfinityCone}()))
        nonneg = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())

        @testset "$attr" for attr in [MOI.ConstraintPrimalStart(), MOI.ConstraintDualStart()]
            @test MOI.supports(bridged_mock, attr, typeof(ci))
            value = Float64[5, 1, -2, 3]
            MOI.set(bridged_mock, attr, ci, value)
            @test MOI.get(bridged_mock, attr, ci) ≈ value
            if attr isa MOI.ConstraintPrimalStart
                nonneg_value = Float64[4, 7, 2, 6, 3, 8]
            else
                nonneg_value = [-1, 11, -1, 5, -1, 17] / 6
            end
            @test MOI.get(mock, attr, nonneg[1]) ≈ nonneg_value
        end

        test_delete_bridge(bridged_mock, ci, 4, (
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives, 1),))
    end
end

@testset "NormOne" begin
    bridged_mock = MOIB.Constraint.NormOne{Float64}(mock)

    MOIT.basic_constraint_tests(bridged_mock, config,
        include = [(F, MOI.NormOneCone) for F in [
            MOI.VectorOfVariables, MOI.VectorAffineFunction{Float64}, MOI.VectorQuadraticFunction{Float64}
        ]])

    @testset "norminf1test" begin
        mock.optimize! = (mock::MOIU.MockOptimizer) -> MOIU.mock_optimize!(mock,
            [1.0, 0.5, 0.5, 0.5, 0.5],
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives) => [[1.0, 1.0, 1.0, 0.0, 0.0]],
            (MOI.VectorAffineFunction{Float64}, MOI.Zeros) => [[-1], [0]])

        MOIT.normone1vtest(bridged_mock, config)
        MOIT.normone1ftest(bridged_mock, config)

        var_names = ["x", "y", "z"]
        MOI.set(bridged_mock, MOI.VariableName(), MOI.get(bridged_mock, MOI.ListOfVariableIndices()), var_names)

        nonneg = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())
        u, v = MOI.get(mock, MOI.ListOfVariableIndices())[4:5]
        @testset "Test mock model" begin
            MOI.set(mock, MOI.VariableName(), u, "u")
            MOI.set(mock, MOI.VariableName(), v, "v")
            @test length(nonneg) == 1
            MOI.set(mock, MOI.ConstraintName(), nonneg[1], "nonneg")
            zeros = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Zeros}())
            @test length(zeros) == 2
            MOI.set(mock, MOI.ConstraintName(), zeros[1], "x_eq")
            MOI.set(mock, MOI.ConstraintName(), zeros[2], "y_eq")

            s = """
            variables: x, y, z, u, v
            nonneg: [x + -1.0u + -1.0v, u + -1.0y, v + -1.0z, u + y, v + z] in MathOptInterface.Nonnegatives(5)
            x_eq: [-1.0 + x] in MathOptInterface.Zeros(1)
            y_eq: [-0.5 + y] in MathOptInterface.Zeros(1)
            maxobjective: y + z
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(mock, model, [var_names; "u"; "v"], ["nonneg", "x_eq", "y_eq"])
        end

        @testset "Test bridged model" begin
            normone = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormOneCone}())
            @test length(normone) == 1
            MOI.set(bridged_mock, MOI.ConstraintName(), normone[1], "normone")
            zeros = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Zeros}())
            @test length(zeros) == 2
            MOI.set(bridged_mock, MOI.ConstraintName(), zeros[1], "x_eq")
            MOI.set(bridged_mock, MOI.ConstraintName(), zeros[2], "y_eq")

            s = """
            variables: x, y, z
            normone: [1.0x, y, z] in MathOptInterface.NormOneCone(3)
            x_eq: [-1.0 + x] in MathOptInterface.Zeros(1)
            y_eq: [-0.5 + y] in MathOptInterface.Zeros(1)
            maxobjective: y + z
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(bridged_mock, model, var_names, ["normone", "x_eq", "y_eq"])
        end

        ci = first(MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormOneCone}()))

        @testset "$attr" for attr in [MOI.ConstraintPrimalStart(), MOI.ConstraintDualStart()]
            @test MOI.supports(bridged_mock, attr, typeof(ci))
            value = [4.0, 1.0, -2.0]
            MOI.set(bridged_mock, attr, ci, value)
            @test MOI.get(bridged_mock, attr, ci) ≈ value
            if attr isa MOI.ConstraintPrimalStart
                @test MOI.get(mock, MOI.VariablePrimalStart(), u) == 1
                @test MOI.get(mock, MOI.VariablePrimalStart(), v) == 2
                @test MOI.get(mock, attr, nonneg[1]) == Float64[1, 0, 4, 2, 0]
            else
                @test MOI.get(mock, attr, nonneg[1]) == Float64[4, 0, 2, 1, 0]
            end
        end

        test_delete_bridge(bridged_mock, ci, 3, (
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives, 0),))
    end

    @testset "normone3test" begin
        mock.optimize! = (mock::MOIU.MockOptimizer) -> MOIU.mock_optimize!(mock,
            [4, -1, -1, -1, 1, 1, 1],
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives) => [vcat(ones(4), zeros(3)), ones(3)])
        MOIT.normone3test(bridged_mock, config)

        var_names = ["x", "y1", "y2", "y3"]

        @testset "Test mock model" begin
            var_names_all = vcat(var_names, "z1", "z2", "z3")
            MOI.set(mock, MOI.VariableName(), MOI.get(mock, MOI.ListOfVariableIndices()), var_names_all)
            nonneg = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())
            @test length(nonneg) == 2
            MOI.set(mock, MOI.ConstraintName(), nonneg[1], "nonneg1")
            MOI.set(mock, MOI.ConstraintName(), nonneg[2], "nonneg2")

            s = """
            variables: x, y1, y2, y3, z1, z2, z3
            nonneg1: [x + -1.0 + -1.0z1 + -1.0z2 + -1.0z3, z1 + -1.0y1 + -2.0, z2 + -1.0y2 + -2.0, z3 + -1.0y3 + -2.0, z1 + y1 + 2.0, z2 + y2 + 2.0, z3 + y3 + 2.0] in MathOptInterface.Nonnegatives(7)
            nonneg2: [y1 + 1.0, y2 + 1.0, y3 + 1.0] in MathOptInterface.Nonnegatives(3)
            minobjective: x
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(mock, model, var_names_all, ["nonneg1", "nonneg2"])
        end

        @testset "Test bridged model" begin
            MOI.set(bridged_mock, MOI.VariableName(), MOI.get(bridged_mock, MOI.ListOfVariableIndices()), var_names)
            normone = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormOneCone}())
            @test length(normone) == 1
            MOI.set(bridged_mock, MOI.ConstraintName(), normone[1], "normone")
            nonneg = MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())
            @test length(nonneg) == 1
            MOI.set(bridged_mock, MOI.ConstraintName(), nonneg[1], "nonneg")

            s = """
            variables: x, y1, y2, y3
            normone: [x + -1.0, y1 + 2.0, y2 + 2.0, y3 + 2.0] in MathOptInterface.NormOneCone(4)
            nonneg: [y1 + 1.0, y2 + 1.0, y3 + 1.0] in MathOptInterface.Nonnegatives(3)
            minobjective: x
            """
            model = MOIU.Model{Float64}()
            MOIU.loadfromstring!(model, s)
            MOIU.test_models_equal(bridged_mock, model, var_names, ["normone", "nonneg"])
        end

        ci = first(MOI.get(bridged_mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.NormOneCone}()))
        nonneg = MOI.get(mock, MOI.ListOfConstraintIndices{MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives}())

        @testset "$attr" for attr in [MOI.ConstraintPrimalStart(), MOI.ConstraintDualStart()]
            @test MOI.supports(bridged_mock, attr, typeof(ci))
            value = (attr isa MOI.ConstraintPrimalStart ? vcat(3, ones(3)) : vcat(1, fill(-1, 3)))
            MOI.set(bridged_mock, attr, ci, value)
            @test MOI.get(bridged_mock, attr, ci) ≈ value
            nonneg_value = (attr isa MOI.ConstraintPrimalStart ? vcat(zeros(4), fill(2.0, 3)) : vcat(ones(4), zeros(3)))
            @test MOI.get(mock, attr, nonneg[1]) ≈ nonneg_value
        end

        test_delete_bridge(bridged_mock, ci, 4, (
            (MOI.VectorAffineFunction{Float64}, MOI.Nonnegatives, 1),))
    end
end


### A Pluto.jl notebook ###
# v0.12.10

using Markdown
using InteractiveUtils

# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error).
macro bind(def, element)
    quote
        local el = $(esc(element))
        global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing
        el
    end
end

# ╔═╡ 728d5546-7a96-11eb-1757-93681c4e7955
begin
	import Pkg
    Pkg.activate(".")
    Pkg.add("PlutoUI")
	Pkg.add("Plots")

	using PlutoUI
	using Plots
end

# ╔═╡ 8dfd7e3e-80b8-11eb-0a8b-99c7a0102380
md"""# SIR-Models

A collection of variations of SIR Models:

* Standard SIR with birth & death
* Inzidenz dependent infection rates
* 6D-SIR with two subpopulations

First set the time horizon and the step size for the euler method for all simulations below.

Start $(@bind T0 NumberField(0:10_000, default=0)) |
Step size Δt $(@bind Δt NumberField(0.0001:0.0001:1.0,default=0.01)) |
Stopp $(@bind Tend NumberField(0:10_000, default=150))

"""

# ╔═╡ e6d00224-8bfe-11eb-1b62-79ccf754d3d7
round(Int,1/Δt+0.1)

# ╔═╡ 62645b8c-7b7a-11eb-1b35-a3957a0252fb
md"""
---
### Classical SIR-Model

The standard SIR-Model with constant birth and death, infection and recovery rates.

${\frac{dS}{dt} = \nu N - \beta \frac{SI}{N} - \mu S}$

${\frac{dI}{dt} = \beta \frac{SI}{N} - \gamma I - \mu I}$

${\frac{dR}{dt} = \gamma I - \mu R \phantom{-\beta \frac{SI}{N}}}$

"""

# ╔═╡ e64e9d62-7b7b-11eb-3620-a7a57b954fcc
md"""
recovery rate γ $(@bind γ Slider(0.0:0.01:1,show_value=true,default=0.15)) |
infection rate β $(@bind β Slider(0.0:0.01:1,show_value=true,default=0.7))

death rate μ $(@bind μ Slider(0.0:0.01:1,show_value=true)) |
birth rate ν $(@bind ν Slider(0.0:0.01:1,show_value=true))
"""

# ╔═╡ 81608c56-80b9-11eb-00f0-3b50905ec998
md"""
---
### Incidence dependent infection rate

Here the infection rate $\beta=\beta(i)$ is dependent on the incidence in a population of constant size.

${\frac{dS}{dt} = - \beta(i) \frac{SI}{N} \phantom{- \gamma I} }$

${\frac{dI}{dt} = \phantom{-} \beta(i) \frac{SI}{N} - \gamma I}$

${\frac{dR}{dt} = \phantom{-\beta(i) \frac{SI}{N}-} \gamma I }$

with the following infection rate functions

* Constant: $\beta(i)\equiv\beta$
* Step Function: $\beta(i) = \beta (1 - \mathbb{1}_{[I_{c},\infty)}(I) (1-\frac{1}{r_s}))$
* Logistic Decay: $\beta(i) = \frac{\beta}{1+\exp(r_l(I-I_c))}$
* Inverse Proportional $\beta(i) = \beta - \frac{\beta}{I_c} I$

where $I_c$ is the critical incidence at which the measurements stat and $r_s, r_f$ is the factor resp. rate at which the infection rate decreases.
"""

# ╔═╡ 5e81dd2c-80b9-11eb-364e-63fa4ebd27da
md"""
Choice of Incidence Function $(@bind Β_name Select(["LogisticDecay", "StepFunction","Constant","InverseProportional"])) |
Delay for application of measurements $(@bind d NumberField(0:100,default=0))
"""

# ╔═╡ 4f8dee16-80ba-11eb-1d0f-f3ecdc9ac671
if Β_name == "LogisticDecay"
	md"""
	maximum infection rate β $(@bind b Slider(0.0:0.01:1,show_value=true,default=0.7))

	Maximum decrease of infection rate at $(@bind high_inc NumberField(0:1000,default=50)) per 10^5 | Rate of decrease $(@bind r_ld Slider(0.0:0.001:0.1,show_value=true,default=0.01))
	"""
elseif Β_name == "StepFunction"
	md"""
	high infection rate β $(@bind b Slider(0.0:0.01:1,show_value=true,default=0.7))

	Reduce infection rate if incidence is high by factor of: $(@bind high_inc_f NumberField(0:0.1:100,default=2)) |
	High incidence at $(@bind high_inc NumberField(0:1000,default=50)) per 10^5
	"""
elseif Β_name == "Constant"
	md"""
	infection rate β $(@bind b Slider(0.0:0.01:1,show_value=true,default=0.7))
	"""
elseif Β_name == "InverseProportional"
	md"""
	maximum infection rate β $(@bind b Slider(0.0:0.01:1,show_value=true,default=0.7)) |
	High incidence at $(@bind high_inc NumberField(0:1000,default=50)) per 10^5
	"""
else
	md""
end

# ╔═╡ 7da7ede2-8bf0-11eb-0829-49e179733bb7
begin
	#Infection Rate Functions
	StepFunction(i) = i ≥ high_inc ? b/high_inc_f : b
	LogisticDecay(i) = b / (1+exp(r_ld*((i-high_inc))))
	Constant(i) = b
	InverseProportional(i) = max(b - (b/high_inc)*i,0)

	function_choice = Dict(
		"StepFunction" => StepFunction,
		"InverseProportional" => InverseProportional,
		"LogisticDecay" => LogisticDecay,
		"Constant" => Constant
	)
end;

# ╔═╡ 59879918-8bea-11eb-25f1-0908e2cdad4b
begin
	function incidence(F::Array{Array{Float64,1},1},t;ndays=7,scale=100_000)
		#starting time
		t₀ = max(t-ndays,1)
		I₀ = t₀ > 1 ? F[t-ndays-1][2] : 0
		#set the range
		Days = @view F[t₀:t]
		#count the newly infected individuals
		I = sum(getindex.(Days,2))-I₀
		#calculate the rescaled, averaged total population size
		N = (sum(sum.(Days))/ndays) / scale
		#return Incidence
		return I/N
	end
	incidence(F::Array{Float64,1},t₀,ndays) = sum(view(F,t₀+1:t₀+ndays+1)) - F[t₀]
	function incidence(F::Array{Float64,1},pop_size;ndays=7,scale=100_000)
		I = cumsum(view(F,1:ndays))
		F_extended = vcat(F,fill(F[end],ndays+1))
		for t₀ ∈ ndays+1:length(F)
			append!(I,incidence(F_extended,t₀,ndays))
		end
		return I .* scale/pop_size
	end
end;

# ╔═╡ 79a2aad2-7c2a-11eb-00b0-6bebd6658619
md"""
---
### Six dimensional SIR-Model
"""

# ╔═╡ 968af7ba-7c2a-11eb-0aa4-6d51510020b1
md"""
Devide the population in two subgroups with different infection rates within and between these groups.

${\frac{dS_1}{dt} = - \beta_{11} \frac{S_1I_1}{N} - \beta_{12} \frac{S_1I_2}{N} \phantom{-\beta \frac{SI}{N}}}$

${\frac{dI_1}{dt} = \phantom{-} \beta_{11} \frac{S_1I_1}{N} + \beta_{12} \frac{S_1I_2}{N} -   \gamma_1 I_1 }$

${\frac{dR_1}{dt} = \phantom{- \beta_{11} \frac{S_1I_1}{N} - \beta_{12} \frac{S_1I_2}{N} -}\gamma_1 I_1 }$

${\frac{dS_2}{dt} = - \beta_{22} \frac{S_2I_2}{N} - \beta_{21} \frac{S_2I_1}{N} \phantom{-\beta \frac{SI}{N}}}$

${\frac{dI_2}{dt} = \phantom{-} \beta_{22} \frac{S_2I_2}{N} + \beta_{21} \frac{S_2I_1}{N} -   \gamma_2 I_2 }$

${\frac{dR_2}{dt} = \phantom{- \beta_{11} \frac{S_1I_1}{N} - \beta_{12} \frac{S_1I_2}{N} -}\gamma_2 I_2 }$

Here $\beta_{11}$ is the infection rate within group one, $\beta_{12}$ is the infection rate from group one to group two, and so on.
"""

# ╔═╡ cbf8bba4-7c26-11eb-3301-07233740f82a
md"""

β₁₁ = $(@bind β11 Slider(0.0:0.01:1.0, show_value=true, default=0.75)) |
β₁₂ = $(@bind β12 Slider(0.0:0.01:1.0, show_value=true, default=0.5))

β₂₁ = $(@bind β21 Slider(0.0:0.01:1.0, show_value=true, default=0.5)) |
β₂₂ = $(@bind β22 Slider(0.0:0.01:1.0, show_value=true, default=0.75))

γ₁ = $(@bind γ1 Slider(0.0:0.01:1.0, show_value=true, default=0.25)) |
γ₂ = $(@bind γ2 Slider(0.0:0.01:1.0, show_value=true, default=0.25))

"""

# ╔═╡ 32f4bc7c-80b9-11eb-0111-074cc68e892f
md"---"

# ╔═╡ ecad4e78-7a96-11eb-2b54-e7f910e83835
begin
	function euler(f,t0,tn,Δt,x₀)
		T = t0:Δt:tn
		F = Vector{typeof(x₀)}(undef,length(T))
		F[1] = x₀
		for (n,t) ∈ enumerate(T[2:end])
			F[n+1] = F[n] .+ f(t,F[n]) .* Δt
		end
		return F
	end

	function euler2(f,t0,tn,Δt,x₀)
		T = t0:Δt:tn
		F = Vector{typeof(x₀)}(undef,length(T))
		F[1] = x₀
		for (n,t) ∈ enumerate(T[2:end])
			F[n+1] = F[n] .+ f(t,F,n) .* Δt
		end
		return F
	end

	reshape_result(F) = [[f[i] for f ∈ F] for i ∈ 1:length(F[1])]

	T = T0:Δt:Tend
end;

# ╔═╡ 6b37eb20-7b7a-11eb-07de-a1042efa6721
begin
	function SIR(t,x)
		N = sum(x)
		S, I, R = x[1], x[2], x[3]
		return [
			(ν*N - β*S*I/N - μ*S),	# dS/dt
			(β*S*I/N - γ*I - μ*I),	# dI/dt
			(γ*I - μ*R)				# dR/dt
		]
	end

	SIR_0 = [9999.9,0.1,0.0]

	F = euler(SIR,T[1],T[end],Δt,SIR_0)
	rF = reshape_result(F)
	N = sum.(F)
end;

# ╔═╡ 68f66db2-7b7c-11eb-138d-2f40b3bfbb4e
let
	p = plot(framestlye=:zerolines,size=(680,450))

	plot!(p,T,rF,label=["S" "I" "R"])

	p
end

# ╔═╡ a9f94f00-8be9-11eb-36f9-d32e2ab1fca7
begin
	function SIR_H(t,F,n)
		#current state
		x = F[n]
		#current sizes
		S, I, R = x[1], x[2], x[3]
		#incidence rate
		n₀ = max(1,n-delay)
		β = Β(incidence(F,n₀))
		return [
			(- β*S*I/N_H ),		# dS/dt
			(β*S*I/N_H - γ*I),	# dI/dt
			(γ*I)				# dR/dt
		]
	end

	#---

	Β = function_choice[Β_name]
	delay = round(Int,1/Δt)*d

	SIR_H_0 = [9999.9,0.1,0.0]
	N_H = sum(SIR_H_0)

	F_H = euler2(SIR_H,T[1],T[end],Δt,SIR_H_0)
	rF_H = reshape_result(F_H)
end;

# ╔═╡ 59382348-8bf3-11eb-0fcf-73857b994092
let
	p = plot(framestlye=:zerolines,size=(680,450))

	Inc = incidence(rF_H[2],N_H)

	plot!(p,T,Inc,label="Incidence per 100.000",color=:red,legend=:topright)
	p2 = twinx()
	plot!(p2,T,Β.(Inc), label="Infection rate",color=:orange,legend=:bottomright)

	p
end

# ╔═╡ 58e3898c-7c21-11eb-3364-9168fcedb22c
begin
	function SIR6D(t,x)
		N = sum(x)
		S1, S2 = x[1], x[4]
		I1, I2 = x[2], x[5]
		R1, R2 = x[3], x[6]

		return [
			(-β11*S1*I1/N - β21*S1*I2/N),			# dS₁/dt
			(β11*S1*I1/N + β21*S1*I2/N - γ1*I1),	# dI₁/dt
			(γ1*I1),								# dR₁/dt
			(-β22*S2*I2/N - β12*S2*I1/N),			# dS₂/dt
			(β22*S2*I2/N + β12*S2*I1/N - γ2*I2),	# dI₂/dt
			(γ2*I2),								# dR₂/dt
		]
	end

	SIR6D_0 = [
		4999.0,1.0,0.0,
		5000.0,0.0,0.0,
	]

	F6D = euler(SIR6D,T[1],T[end],Δt,SIR6D_0)
	rF6D = reshape_result(F6D)
	N6D = sum.(F6D)
end;

# ╔═╡ d1644026-7c27-11eb-17ae-67c414382cdc
let
	p = plot(framestlye=:zerolines,size=(680,450))

	plot!(p,T,rF6D,
		#label=["S₁" "I₁" "R₁" "S₂" "I₂" "R₂"],
		label="",
		alpha=0.5,
		lw=1,
		color=[:darkblue :darkred :darkgreen :lightblue :orangered :lightgreen]
	)

	plot!(p,T,[rF6D[i] .+ rF6D[i+3] for i ∈ 1:3],
		label=["S" "I" "R"],
		color=[:blue :red :green],
		lw=2
	)

	p
end

# ╔═╡ Cell order:
# ╟─8dfd7e3e-80b8-11eb-0a8b-99c7a0102380
# ╠═e6d00224-8bfe-11eb-1b62-79ccf754d3d7
# ╟─62645b8c-7b7a-11eb-1b35-a3957a0252fb
# ╠═6b37eb20-7b7a-11eb-07de-a1042efa6721
# ╟─e64e9d62-7b7b-11eb-3620-a7a57b954fcc
# ╠═68f66db2-7b7c-11eb-138d-2f40b3bfbb4e
# ╟─81608c56-80b9-11eb-00f0-3b50905ec998
# ╟─5e81dd2c-80b9-11eb-364e-63fa4ebd27da
# ╟─4f8dee16-80ba-11eb-1d0f-f3ecdc9ac671
# ╟─59382348-8bf3-11eb-0fcf-73857b994092
# ╠═a9f94f00-8be9-11eb-36f9-d32e2ab1fca7
# ╠═7da7ede2-8bf0-11eb-0829-49e179733bb7
# ╠═59879918-8bea-11eb-25f1-0908e2cdad4b
# ╟─79a2aad2-7c2a-11eb-00b0-6bebd6658619
# ╟─968af7ba-7c2a-11eb-0aa4-6d51510020b1
# ╠═58e3898c-7c21-11eb-3364-9168fcedb22c
# ╟─cbf8bba4-7c26-11eb-3301-07233740f82a
# ╟─d1644026-7c27-11eb-17ae-67c414382cdc
# ╟─32f4bc7c-80b9-11eb-0111-074cc68e892f
# ╠═728d5546-7a96-11eb-1757-93681c4e7955
# ╠═ecad4e78-7a96-11eb-2b54-e7f910e83835


# ---
# title: 65. Valid Number
# id: problem65
# author: Tian Jun
# date: 2020-10-31
# difficulty: Hard
# categories: Math, String
# link: <https://leetcode.com/problems/valid-number/description/>
# hidden: true
# ---
#
# Validate if a given string can be interpreted as a decimal number.
#
# Some examples:
# `"0"` => `true`
# `" 0.1 "` => `true`
# `"abc"` => `false`
# `"1 a"` => `false`
# `"2e10"` => `true`
# `" -90e3   "` => `true`
# `" 1e"` => `false`
# `"e3"` => `false`
# `" 6e-1"` => `true`
# `" 99e2.5 "` => `false`
# `"53.5e93"` => `true`
# `" --6 "` => `false`
# `"-+3"` => `false`
# `"95a54e53"` => `false`
#
# **Note:** It is intended for the problem statement to be ambiguous. It would
# be best if you gathered all requirements up front before implementing one.
# However, here is a list of characters that can be in a valid decimal number:
#
#   * Numbers 0-9
#   * Exponent - "e"
#   * Positive/negative sign - "+"/"-"
#   * Decimal point - "."
#
# Of course, the context of these characters also matters in the input.
#
#
#
# **Example 1:**
#
#
#
#     Input: s = "0"
#     Output: true
#
#
# **Example 2:**
#
#
#
#     Input: s = "3"
#     Output: true
#
#
#
#
# **Constraints:**
#
#   * `1 <= s.length <= 20`
#   * `s` consists of only English letters, digits, space `' '`, plus `'+'`, minus `'-'`, or dot `'.'`.
#
#
## @lc code=start
using LeetCode

## add your code here:
## @lc code=end


#!/usr/bin/julia

function err()
   println("Usage: please input a non-negative integer")
end

function fac(n)
    if n < 0
        err()
        exit()
    end

    out = 1
    for i = 1:n
        out = out * i
    end
    return out
end

try
    println(fac(parse(Int, ARGS[1])))
catch e
    err()
end


"""

This module borrows heavily from pyamg. See [https://github.com/pyamg/pyamg/blob/master/pyamg/amg_core/smoothed_aggregation.h](https://github.com/pyamg/pyamg/blob/master/pyamg/amg_core/smoothed_aggregation.h).
"""
module Aggregation

"""
    standard_aggregation(A :: SparseMatrixCSC)

Returns a vector `v` where each entry in the vector gives the aggregate that
that vertex belongs to. Ex: v[1] = 2 means that vertex 1 belongs to aggregate
2.
"""
function standard_aggregation(A :: SparseMatrixCSC)
    m, n = size(A)
    rows = rowvals(A)
    vals = nonzeros(A)

    # hold aggregates. -1 is unaggregated
    aggregates = fill(-1, n)

    # find initial aggregates
    current_aggregate = 1
    for i in 1:n
        if aggregates[i] == -1 # not yet aggregated
            neighbors = rows[nzrange(A, i)]
            if all(x -> aggregates[x] == -1, neighbors) # no neighbors are aggregated
                aggregates[i] = current_aggregate
                aggregates[neighbors] = current_aggregate
                current_aggregate += 1
            end
        end
    end

    # Pull neighboring nodes into aggregates
    for i in 1:n
        if aggregates[i] == -1
            # find max weight edge
            max_weight = 0
            ind = -1
            for j in nzrange(A, i)
                if rows[j] != i && aggregates[rows[j]] != -1
                    w = vals[j]
                    if w > max_weight
                        max_weight = w
                        ind = rows[j]
                    end
                end
            end

            if ind == -1
                error("isolated node")
            else
                aggregates[i] = aggregates[ind]
            end
        end
    end

    aggregates
end

end


using skalman
using Plots, LinearAlgebra, FastGaussQuadrature

n = 8
nodes, weights = gausslobatto(n)

# Broadcast and store in a matrix
P = legendrepoly.(nodes,n-1)
P = reduce(hcat,P)
P = transpose(P)

# Broadcast and store in a matrix
PX = derlegendre.(nodes,n-1)
PX = reduce(hcat,PX)
PX = transpose(PX)

M = zeros(n,n)
S = similar(M)
for i = 1:n
    for j = 1:n
    M[i,j] = sum(P[:,i].*P[:,j].*weights)
    S[i,j] = sum(P[:,i].*PX[:,j].*weights)
    end
end

f = sin.(nodes)
b = similar(f)
for i = 1:n
    b[i] = sum(P[:,i].*f.*weights)
end
umodes = M\b
uxmodes = M\(S*umodes)

zz = collect(LinRange(-1.0, 1.0, 101))
# Broadcast and store in a matrix
PP = legendrepoly.(zz,n-1)
PP = reduce(hcat,PP)
PP = transpose(PP)

unodes = PP*umodes
uxnodes = PP*uxmodes
pl = plot(zz,uxnodes)
plot!(pl,zz,cos.(zz))


@reexport module _GroupAlgorithm

abstract type Algorithm end

struct BabyStepGiantStep <: Algorithm end
struct PollardRho        <: Algorithm end
struct Schoof            <: Algorithm end

end

# https://en.wikipedia.org/wiki/Counting_points_on_elliptic_curves#Baby-step_giant-step
order(ec::Curve{T}, ::Type{_GroupAlgorithm.BabyStepGiantStep}) where {T<:FinFieldElem} = begin
    o = order(field(ec))
    q = convert(BigInt, o)
    m = convert(BigInt, root(o, 4) + 1)

    L = big(1)
    N = big(0)

    while true
        # random element P=(x,y) ∈ E(𝐅q)
        P = rand(ec)
        while isideal(P)
            P = rand(ec)
        end

        # Pⱼ[x] == j => jP == (x, p(x)); Pm = mP
        Pⱼ, _, Pm = logpx(P, m)
        Q = (q+1) * P
        P2m = Pm + Pm

        # find k, j: Q + k(2mP) = R = ±Pⱼ (the x-coordinates are compared)
        k = zero(m)
        R = Q
        while true
            isideal(R) && break
            R.x ∈ keys(Pⱼ) && break
            k += 1
            R += P2m
        end
        j = isideal(R) ? 0 : Pⱼ[R.x]
        # (q + 1 + 2mk ∓ j)P = MP = 𝒪
        M = q + 1 + 2*m*k + (R == (j*P) ? -1 : 1)*j

        for (p, e) in factor(M)
            for _ in 1:e
                N = M ÷ p
                !isideal(N*P) && break
                M = N
            end
        end
        # M is the order of the point P
        L = lcm(L, M)

        # L divides more than one integer N in (q+1-2√q, q+1+2√q)?
        sq = isqrt(q)
        N = q + 1 - 2(sq-1)
        N₊ = Array{typeof(N), 1}()
        while N <= q + 1 + 2(sq+1)
            N % L == 0 && push!(N₊, N)
            length(N₊) > 1 && break
            N += 1
        end
        # if not => N is the cardinality of E(𝐅q)
        length(N₊) == 1 && return N₊[1]
    end
end

logpx(p::Point, m::Integer) = begin
    pᵢ = Dict{typeof(p.x), typeof(m)}()
    o = zero(m)
    q = ideal(curve(p))
    for i in 1:m
        q += p
        isideal(q) && (o = i; break)
        q.x ∈ keys(pᵢ) && continue
        pᵢ[q.x] = i
    end
    pᵢ, o, q
end

# http://andrea.corbellini.name/2015/05/23/elliptic-curve-cryptography-finite-fields-and-discrete-logarithms/
order(p::Point, N::Integer) = begin
    for n in factors(N)
        isideal(n*p) && return n
    end

    error("$(p) is not an element of a subgroup")
end

gen(ec::Curve, N::Integer, n::Integer) = begin
    isprime(n) || error("order of the subgroup ($n) must be prime")

    h, r = divrem(N, n)
    r == 0 || error("order of the subgroup ($n) must be a divisor of $N")

    for i in 1:100
        P = rand(ec)
        G = h*P
        isideal(G) || return G
    end

    error("no generator of a subgroup with order $n could be found")
end


function pcnet(X::AbstractMatrix{T}, p::Int=3;
              scalein::Bool=true, scaleout::Bool=false,
              symmout::Bool=false) where T<:Real
    if scalein
        σ=std(X,dims=1)
        σ[σ.==0].=1.0
        X=(X.-mean(X,dims=1))./σ
    end
    ℊ=size(X,2)
    A=1.0 .-Matrix(I,ℊ,ℊ)
    Threads.@threads for k in 1:ℊ
        y=X[:,k]
        𝒳=X[:,1:end.≠k]
        ϕ=TSVD.tsvd(𝒳,p)[3]
        s=𝒳*ϕ
        s ./= (vecnorm(s).^2)'
        b=sum(y.*s,dims=1)
        𝒷=ϕ*b'
        @inbounds A[k,A[k,:].==1.0]=𝒷
    end
    if symmout
        A=0.5*(A+A')
    end
    if scaleout
        A=A./maximum(abs.(A))
    end
    return convert(Matrix{Float16},A)
end


using ManifoldsBase
import ManifoldsBase: representation_size, manifold_dimension, inner
using LinearAlgebra
using Test

struct ComplexEuclidean{N} <: Manifold{ℂ} where {N} end
ComplexEuclidean(n::Int) = ComplexEuclidean{n}()
representation_size(::ComplexEuclidean{N}) where {N} = (N,)
manifold_dimension(::ComplexEuclidean{N}) where {N} = 2N
@inline inner(::ComplexEuclidean, x, v, w) = dot(v, w)

@testset "Complex Euclidean" begin
    M = ComplexEuclidean(3)

    x = complex.([1.0, 2.0, 3.0], [4.0, 5.0, 6.0])
    v1 = complex.([0.1, 0.2, 0.3], [0.4, 0.5, 0.6])
    v2 = complex.([0.3, 0.2, 0.1], [0.6, 0.5, 0.4])

    @test norm(M, x, v1) isa Real
    @test norm(M, x, v1) ≈ norm(v1)
    @test angle(M, x, v1, v2) isa Real
    @test angle(M, x, v1, v2) ≈ acos(real(dot(v1, v2)) / norm(v1) / norm(v2))

    vv1 = vcat(reim(v1)...)
    vv2 = vcat(reim(v2)...)
    @test angle(M, x, v1, v2) ≈ acos(dot(normalize(vv1), normalize(vv2)))
end


using Clines, SimpleDrawing, Plots

function apollo_work(C1::Circle, C2::Circle, C3::Circle, min_radius)
    S = soddy(C1, C2, C3)
    if radius(S) < min_radius
        return
    end
    draw(S, linewidth = 0.25)
    apollo_work(C1, C2, S, min_radius)
    apollo_work(C1, C3, S, min_radius)
    apollo_work(C2, C3, S, min_radius)
end

function apollo(C1::Circle, C2::Circle, C3::Circle, min_radius::Real = 0.01)
    for C in [C1, C2, C3]
        draw(C, linewidth = 0.25)
    end
    apollo_work(C1, C2, C3, min_radius)
    finish()
end

function apollo(z1::Number, z2::Number, z3::Number, min_radius::Real = 0.01)
    newdraw()
    CC = kiss(z1, z2, z3)
    apollo(CC..., min_radius)
end

function apollo(min_radius::Real = 0.01)
    angles = [(2 * pi * k / 3 + pi / 6) * im for k = 1:3]
    zz = exp.(angles)
    apollo(zz..., min_radius)
end


#module roundPerRound

function σtot(σ::Float64,ρ::Float64)
    ρ*σ*1e-3
end

function SSHP(projectile::AbstractPenetrator, target::TargetCirc)
    D = abs(projectile.position-target.position)
    Stot=σtot(projectile.σ,D)
    y = (x)->x[1]/Stot^2*exp(-0.5*x[1]^2/Stot^2)
    g=hcubature(y,0,0.5*target.ρ)[1]
    return g
end
"""
    SSHP(target,error)
Returns the single shot hit probability (sshp) for a rectangular target.
"""
function SSHP(target::TargetRect,error::RectErrorB)
    y = x->1/(sqrt(2*pi)*error.σ_x)*exp(-((x[1]-error.μ_x)^2)/(2*error.σ_x^2))*1/(sqrt(2*pi)*error.σ_y)*exp(-((x[2]-error.μ_y)^2)/(2*error.σ_y^2))
    g=hcubature(y,[-target.a,-target.b],[target.a, target.b])[1]
    return g
end

"""
    pHit(target,error)

Returns the probability of hitting the target defined by'target' if the estimated error is provided by 'error'.
```math
\\int_{-a}^{a}\\int_{-b}^{b}\\frac{1}{2\\pi\\sigma_x\\sigma_y}\\text{exp}[\\frac{(x-\\mu_x)^2}{2\\sigma_x^2}\\frac{(y-\\mu_y)^2}{2\\sigma_y^2}]dxdy
```
"""
function pHit(target::AbstractTarget, error::RectErrorB)
    SSHP(target, error)
end
#end
#\\[\\frac{(x-\\mu_x)^2}{2\\sigma_x^2}
#+\\frac{(y-\\mu_y)^2}{2\\sigma_y^2}\\]dxdy\$


const eightschools ="
data {
  int<lower=0> J; // number of schools
  real y[J]; // estimated treatment effects
  real<lower=0> sigma[J]; // s.e. of effect estimates
}
parameters {
  real mu;
  real<lower=0> tau;
  real eta[J];
}
transformed parameters {
  real theta[J];
  for (j in 1:J)
    theta[j] <- mu + tau * eta[j];
}
model {
  eta ~ normal(0, 1);
  y ~ normal(theta, sigma);
}
"


function solve(tree::NDTree, x::Vector{Float64})
    y = x[tree.p]
    maxLevel = length(tree.t)
    # Forward
    for l = 1:maxLevel
        for s = 1:2^(maxLevel-l)
            Lss = tree.t[l][s].APiv[1,1]
            p   = get_dofs(tree.t[l][s])
            # Pivot
            BLAS.trsv!('L', 'N', 'U', Lss, view(y, p)) # y[p] <- Lss^-1 y[p]
            # Panel
            for (Lnp, n) in zip(tree.t[l][s].ALow[:,1], tree.t[l][s].NbrRanges)
                BLAS.gemv!('N', -1.0, Lnp, view(y, p), 1.0, view(y, n)) # y[n] <- y[n] - Lnp * y[p]
            end
        end
    end
    # Backward
    for l = maxLevel:-1:1
        for s = 1:2^(maxLevel-l)
            Uss = tree.t[l][s].APiv[1,1]
            p   = get_dofs(tree.t[l][s])
            # Panel
            for (Upn, n) in zip(tree.t[l][s].AUpp[1,:], tree.t[l][s].NbrRanges)
                BLAS.gemv!('N', -1.0, Upn, view(y, n), 1.0, view(y, p)) # y[p] <- y[p] - Upn * y[n]
            end
            # Pivot
            BLAS.trsv!('U', 'N', 'N', Uss, view(y, p)) # y[p] <- Uss^-1 y[p]
        end
    end
    return y[invperm(tree.p)]
end


include("tt_tools.jl")

"""
sparse implementation of one_body and two_body matrices
"""

function undigits(x;base=2)
    return sum([x[k]*base^(k-1) for k in 1:length(x)])
end

function sp_one_body_to_matrix(k,l,L,N)
    Is = spzeros(2^L)
    Js = spzeros(2^L)
    Vs = spzeros(2^L)
    j = 0 #counter
    for occ_list in combinations(1:L,N)
        if (l in occ_list) && ((k==l) || !(k in occ_list))
            j+=1
            x = [m in occ_list for m in 1:L]
            Js[j] = undigits(x)+1
            Vs[j] = (-1)^sum(x[1:l-1])
            x[l] = 0
            x[k] = 1
            Is[j] = undigits(x)+1
            Vs[j] *= (-1)^sum(x[1:k-1])
        end
    end
    return sparse(Is[1:j],Js[1:j],Vs[1:j],2^L,2^L)
end

#assume k<l, m<n
function sp_two_body_to_matrix(k,l,m,n,L,N)
    Is = spzeros(2^L)
    Js = spzeros(2^L)
    Vs = spzeros(2^L)
    j = 0 #counter
    for occ_list in combinations(1:L,N)
        if (m in occ_list) && (n in occ_list) && ((k==m) || (k==n) || !(k in occ_list)) && ((l==m) || (l==n) || !(l in occ_list))
            j+=1
            x = [r in occ_list for r in 1:L]
            Js[j] = undigits(x)+1
            Vs[j] = (-1)^sum(x[1:n-1])
            x[n] = 0
            Vs[j] *= (-1)^sum(x[1:m-1])
            x[m] = 0
            Vs[j] *= (-1)^sum(x[1:l-1])
            x[l] = 1
            Vs[j] *= (-1)^sum(x[1:k-1])
            x[k] = 1
            Is[j] = undigits(x)+1
        end
    end
    return sparse(Is[1:j],Js[1:j],Vs[1:j],2^L,2^L)
end

#h and V sparse
function sp_sc_to_mat(h,V,N)
    L = size(h,1)
    A = spzeros(2^L,2^L)
    for i in findall(!iszero,h)
        H = sp_one_body_to_matrix(i[1],i[2],L,N)
        A += h[i]*(H+H')
    end
    for i in findall(!iszero,V)
        H = sp_two_body_to_matrix(i[1],i[2],i[3],i[4],L,N)
        A += V[i]*(H+H')
    end
    return dropzeros!(A)
end

function ham_to_mpo(h,V,N)
    L = size(h,1)
    H = sp_sc_to_mat(h,V,N)
    dim = 2*ones(Int,L)
    return tto_spdecomp(sparsetensor_mat(dim,dim,H),L)
end


module BasisTest
using Base.Test
using Quantum

xb = basis(:X,[1:5])
xbplus = basis(:X,[2:6])
xb2 = xb*xb
@test separate(xb2)==[xb,xb]
@test separate(xb)==[xb]
@test basis([Ket{:X,Int}(i) for i=1:5])==xb
@test basis([xb[j]*xb[i] for i=1:5, j=1:5])==xb2
@test isdual(xb, xb')
@test basis(:X, [1,3,5]) == filter(s->label(s)%2!=0, xb)
@test basis(xb[1]*xb[1]) == filter(s->s==xb[1]*xb[1], xb2)
@test map(s->ket(:X,label(s)+1), xb) == basis(:X,[2:6])
@test map(s->ket(:X,label(s[1])+1)*s[2], xb2) == xbplus*xb
@test setdiff(xbplus, xb) == [xbplus[5]]
@test xb+xb == xb
@test xbplus[5]+xb == basis(:X,[6, 1:5])
@test xb+xbplus[5] == basis(:X,[1:6])

end


f(x,y) = 2*x + 3y
f2(x,y) = 3*x + 2*y
dfdx(x,y) = ForwardDiff.derivative(x->f(x,y),x)


module _XPermEn
export XPermEn
using StatsBase: fit, Histogram
using Combinatorics: permutations

    """
        XPerm = XPermEn(Sig)

    Returns the cross-permuation entropy estimates (`XPerm`) estimated betweeen
    the data sequences contained in `Sig` using the default parameters:
    embedding dimension = 3, time delay = 1, logarithm = base 2,

        XPerm = XPermEn(Sig::AbstractArray{T,2} where T<:Real; m::Int=3, tau::Int=1, Logx::Real=exp(1))

    Returns the permutation entropy estimates (`XPerm`) estimated between the
    data sequences contained in `Sig` using the specified 'keyword' arguments:

    # Arguments:
    `m`     - Embedding Dimension, an integer > 2   [default: 3]  \n
            **Note: XPerm is undefined for embedding dimensions < 3.**\n
    `tau`   - Time Delay, a positive integer        [default: 1]    \n
    `Logx`  - Logarithm base, a positive scalar     [default: 2]
            ** enter 0 for natural log.**    \n

    # See also `PermEn`, `XApEn`, `XSampEn`, `XFuzzEn`, `XMSEn`

    # References:
        [1] Wenbin Shi, Pengjian Shang, and Aijing Lin,
            "The coupling analysis of stock market indices based on
            cross-permutation entropy."
            Nonlinear Dynamics
            79.4 (2015): 2439-2447.

    """
    function XPermEn(Sig::AbstractArray{T,2} where T<:Real; m::Int=3, tau::Int=1, Logx::Real=exp(1))

    (size(Sig,2) > size(Sig,1)) ? Sig = transpose(Sig) : nothing

    N = size(Sig,1)
    (N>10 && size(Sig,2)==2) ? nothing :  error("Sig:   must be a 2-columns matrix")
    (m > 2) ? nothing : error("m:     must be an integer > 1")
    (tau>0) ? nothing : error("tau:   must be an integer > 0")
    (Logx>0) ? nothing : error("Logx:     must be a positive number > 0")

    S1 = Sig[:,1]; S2 = Sig[:,2];
    N = length(S1)-(m-1)*tau
    Sx1 = zeros(N,m)
    Sx2 = zeros(N,m)
    for k = 1:m
        Sx1[:,k] = S1[1+(k-1)*tau:N+(k-1)*tau]
        Sx2[:,k] = S2[1+(k-1)*tau:N+(k-1)*tau]
    end

    Temp = sortind(Sx1[1:N,1:m])
    Gx = zeros(N,m)
    for k = 1:N
        Gx[k,:] = Sx2[k,Temp[k,:]]
    end

    Kt = zeros(m-2,m-2,N)
    for k = 1:m-2
        for j = k+1:m-1
            G1 = Gx[:,j+1] .- Gx[:,k]
            G2 = Gx[:,k] .- Gx[:,j]
            Kt[k,j-1,:] = (G1.*G2 .> 0)
        end
    end

    Di = sum(Kt,dims=(1,2))[:]
    Ppi = fit(Histogram, Di, -.5:((m-2)*(m-1) + 1)/2).weights/N
    Ppi = Ppi[Ppi.!=0]
    XPerm = -sum(Ppi.*log.(Logx,Ppi))
    if round(sum(Ppi),digits=6)!=1
        @warn("Potential error with probability calculation")
    end

    return XPerm
    end

    function sortind(X)
        Y = zeros(Int, size(X))
        for k = 1:length(X[:,1])
            Y[k,:] = sortperm(X[k,:])
        end
        return Y
    end
end

"""
Copyright 2021 Matthew W. Flood, EntropyHub

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

For Terms of Use see https://github.com/MattWillFlood/EntropyHub
"""


@doc raw"""
    UnbiasedSKCE(k)

Unbiased estimator of the squared kernel calibration error (SKCE) with kernel `k`.

Kernel `k` on the product space of predictions and targets has to be a `Kernel` from the
Julia package
[KernelFunctions.jl](https://github.com/JuliaGaussianProcesses/KernelFunctions.jl)
that can be evaluated for inputs that are tuples of predictions and targets.

# Details

The estimator is unbiased and not guaranteed to be non-negative. Its sample complexity
is ``O(n^2)``, where ``n`` is the total number of samples.

Let ``(P_{X_i}, Y_i)_{i=1,\ldots,n}`` be a data set of predictions and corresponding
targets. The estimator is defined as
```math
\frac{2}{n(n-1)} \sum_{1 \leq i < j \leq n} h_k\big((P_{X_i}, Y_i), (P_{X_j}, Y_j)\big),
```
where
```math
\begin{aligned}
h_k\big((μ, y), (μ', y')\big) ={}&   k\big((μ, y), (μ', y')\big)
                                   - 𝔼_{Z ∼ μ} k\big((μ, Z), (μ', y')\big) \\
                                 & - 𝔼_{Z' ∼ μ'} k\big((μ, y), (μ', Z')\big)
                                   + 𝔼_{Z ∼ μ, Z' ∼ μ'} k\big((μ, Z), (μ', Z')\big).
\end{aligned}
```

# References

Widmann, D., Lindsten, F., & Zachariah, D. (2019). [Calibration tests in multi-class
classification: A unifying framework](https://proceedings.neurips.cc/paper/2019/hash/1c336b8080f82bcc2cd2499b4c57261d-Abstract.html).
In: Advances in Neural Information Processing Systems (NeurIPS 2019) (pp. 12257–12267).

Widmann, D., Lindsten, F., & Zachariah, D. (2021). [Calibration tests beyond
classification](https://openreview.net/forum?id=-bxf89v3Nx).

See also: [`BiasedSKCE`](@ref), [`BlockUnbiasedSKCE`](@ref)
"""
struct UnbiasedSKCE{K<:Kernel} <: SKCE
    """Kernel of estimator."""
    kernel::K
end

function (skce::UnbiasedSKCE)(predictions::AbstractVector, targets::AbstractVector)
    @unpack kernel = skce

    # obtain number of samples
    nsamples = check_nsamples(predictions, targets, 2)

    @inbounds begin
        # evaluate the kernel function for the first pair of samples
        hij = unsafe_skce_eval(
            kernel, predictions[1], targets[1], predictions[2], targets[2]
        )

        # initialize the estimate
        estimate = hij / 1

        # for all other pairs of samples
        n = 1
        for j in 3:nsamples
            predictionj = predictions[j]
            targetj = targets[j]

            for i in 1:(j - 1)
                predictioni = predictions[i]
                targeti = targets[i]

                # evaluate the kernel function
                hij = unsafe_skce_eval(kernel, predictioni, targeti, predictionj, targetj)

                # update the estimate
                n += 1
                estimate += (hij - estimate) / n
            end
        end
    end

    return estimate
end

@doc raw"""
    BlockUnbiasedSKCE(k[, blocksize = 2])

Unbiased estimator of the squared kernel calibration error (SKCE) with kernel `k` that
uses blocks with `blocksize` samples.

Kernel `k` on the product space of predictions and targets has to be a `Kernel` from the
Julia package
[KernelFunctions.jl](https://github.com/JuliaGaussianProcesses/KernelFunctions.jl)
that can be evaluated for inputs that are tuples of predictions and targets.

The number of samples per block must be at least 2 and at most the total number of samples.
Note that samples in the last block are discarded if it is incomplete (see details below).

# Details

The estimator is unbiased and not guaranteed to be non-negative. Its sample complexity
is ``O(Bn)``, where ``B`` is the block size and ``n`` is the total number of samples.

Let ``(P_{X_i}, Y_i)_{i=1,\ldots,n}`` be a data set of predictions and corresponding
targets. The estimator with block size ``B`` is defined as
```math
{\bigg\lfloor \frac{n}{B} \bigg\rfloor}^{-1} \sum_{b=1}^{\lfloor n/B \rfloor}
\frac{2}{B(B-1)} \sum_{(b - 1) B < i < j \leq bB} h_k\big((P_{X_i}, Y_i), (P_{X_j}, Y_j)\big),
```
where
```math
\begin{aligned}
h_k\big((μ, y), (μ', y')\big) ={}&   k\big((μ, y), (μ', y')\big)
                                   - 𝔼_{Z ∼ μ} k\big((μ, Z), (μ', y')\big) \\
                                 & - 𝔼_{Z' ∼ μ'} k\big((μ, y), (μ', Z')\big)
                                   + 𝔼_{Z ∼ μ, Z' ∼ μ'} k\big((μ, Z), (μ', Z')\big).
\end{aligned}
```

# References

Widmann, D., Lindsten, F., & Zachariah, D. (2019). [Calibration tests in multi-class
classification: A unifying framework](https://proceedings.neurips.cc/paper/2019/hash/1c336b8080f82bcc2cd2499b4c57261d-Abstract.html).
In: Advances in Neural Information Processing Systems (NeurIPS 2019) (pp. 12257–12267).

Widmann, D., Lindsten, F., & Zachariah, D. (2021). [Calibration tests beyond
classification](https://openreview.net/forum?id=-bxf89v3Nx).

See also: [`BiasedSKCE`](@ref), [`UnbiasedSKCE`](@ref)
"""
struct BlockUnbiasedSKCE{K<:Kernel} <: SKCE
    """Kernel of estimator."""
    kernel::K
    """Number of samples per block."""
    blocksize::Int

    function BlockUnbiasedSKCE{K}(kernel::K, blocksize::Int) where {K}
        blocksize ≥ 2 || error("there must be at least two samples per block")
        return new{K}(kernel, blocksize)
    end
end

function BlockUnbiasedSKCE(kernel::Kernel, blocksize::Int=2)
    return BlockUnbiasedSKCE{typeof(kernel)}(kernel, blocksize)
end

function (skce::BlockUnbiasedSKCE)(predictions::AbstractVector, targets::AbstractVector)
    @unpack kernel, blocksize = skce

    # obtain number of samples
    nsamples = check_nsamples(predictions, targets, blocksize)

    # compute number of blocks
    nblocks = nsamples ÷ blocksize

    # create iterator of partitions
    blocks = Iterators.take(
        zip(
            Iterators.partition(predictions, blocksize),
            Iterators.partition(targets, blocksize),
        ),
        nblocks,
    )

    # compute average u-statistic
    unbiased = UnbiasedSKCE(kernel)
    estimate = mean(unbiased(_predictions, _targets) for (_predictions, _targets) in blocks)

    return estimate
end


function dnf_and(exprs::BoolExprVec)::Or
    dnf_exprs = map(dnf, exprs)
    # Each item in dnf_exprs is in DNF form, that is
    # * A variable
    # * A disjunction (Or) of conjunctions (And) of variables
    childDNFs = map(children, dnf_exprs)
    prod = cartprod(childDNFs)
    Or(map(a -> And(children(a)), prod))
end

function dnf_or(exprs::BoolExprVec)::Or
    dnf_exprs = map(dnf, exprs)
    Or(flatmap(children, dnf_exprs))
end

dnf_not(parent::Not, expr::BoolExpr) = parent
dnf_not(parent::Not, expr::And) = dnf(Or(negate(expr.exprs))) # DeMorgan
dnf_not(parent::Not, expr::Or)  = dnf(And(negate(expr.exprs))) # DeMorgan
dnf_not(parent::Not, expr::Not) = dnf(expr.expr) # Double negation
dnf_not(parent::Not, expr::True) = f
dnf_not(parent::Not, expr::False) = t

dnf(expr::Not) = dnf_not(expr, expr.expr)
dnf(expr::And) = dnf_and(expr.exprs)
dnf(expr::Or) = dnf_or(expr.exprs)
dnf(expr::BoolExpr) = expr


def arrcpy(a) {
    cpy = []
    for i=0, i<|a|, i=i+1
        cpy :<= a[i]
    cpy
}

def partition(a, lo, hi) {
	{p, i} = (a[hi], lo)
	for j=lo, j < hi, j=j+1
		if a[j] <= p {
			a[i] <-> a[j]
			i = i+1
		}

	a[i] <-> a[hi]
	i
}

def quicksort(a, lo, hi)
	if lo < hi {
		p = partition(a, lo, hi)
		quicksort(a, lo, p-1)
		quicksort(a, p+1, hi)
	}

def qsort(a) quicksort(a, 0, |a|-1)

def qsorted(a) {
    sorted = arrcpy(a)
    qsort(sorted)
    sorted
}


using PyPlot
A = imread("sample_photo.png")
imshow(A);

println("A is of type $(typeof(A)) with dimensions $(size(A))")
println("The values of A range between $(minimum(A)) and $(maximum(A))")

Ar = copy(A); Ar[:,:,[2,3]] .= 0.0;
Ag = copy(A); Ag[:,:,[1,3]] .= 0.0;
Ab = copy(A); Ab[:,:,[1,2]] .= 0.0;
subplot(2,2,1); imshow(Ar);
subplot(2,2,2); imshow(Ag);
subplot(2,2,3); imshow(Ab);
subplot(2,2,4); imshow(A);

using Statistics
B = mean(A, dims=3)[:,:,1]

function imshow_scale(A)
    # Like imshow(A) but scales the values to [0,1] and supports grayscale

    A .-= minimum(A)            # Scale and shift to [0,1]
    A ./= maximum(A)
    if ndims(A) < 3
        A = reshape(A, size(A,1), size(A,2), 1)
    end
    if size(A,3) == 1
        A = repeat(A, 1, 1, 3)  # Set R=G=B for grayscale
    end
    imshow(A)
end

imshow_scale(B);

function image_threshold(A, th)
    return Float64.(A .> th)
end

imshow_scale(1.0 .- image_threshold(B, 0.8));


### A Pluto.jl notebook ###
# v0.14.0

using Markdown
using InteractiveUtils

# This Pluto notebook uses @bind for interactivity. When running this notebook outside of Pluto, the following 'mock version' of @bind gives bound variables a default value (instead of an error).
macro bind(def, element)
    quote
        local el = $(esc(element))
        global $(esc(def)) = Core.applicable(Base.get, el) ? Base.get(el) : missing
        el
    end
end

# ╔═╡ e73aaef4-97e7-11eb-1722-9d275d445f1c
html"""

Section 2.8

 Pascal

Lecture Video

"""

# ╔═╡ 568374de-9717-11eb-3ad1-571475dfe65c
function pascal_row(n)
	if n == 1
		[1]
	else
		prev = pascal_row(n-1)
		[prev; 0] .+ [0; prev]
	end
end

# ╔═╡ a5cb88e1-95de-4a01-8738-1761052d4886
pascal_row(3)

# ╔═╡ 4c6bdeec-cb3b-42da-9cba-f625818bc9c5
pascal(n) = pascal_row.(1:n)

# ╔═╡ 1a651f93-5df0-4ce6-991d-9720da29a7e3
@bind N Slider(1:17; default=5)

# ╔═╡ 3c0baae7-6a34-4126-8d19-5915cf64d7bd
pascal(N)

# ╔═╡ 3a765d51-95d0-444d-adab-f794190914f2
pyramid(pascal(N))

# ╔═╡ fec4bcfb-b88a-4008-82fa-194b8f7264e0
md"""
We need this magical mini-package
"""

# ╔═╡ 116ffb13-8b9a-40b4-aa1e-88d645d1124a
begin
    import Pkg
    Pkg.activate(mktempdir())
    Pkg.add([
        Pkg.PackageSpec(name="HypertextLiteral", version="0.6"),
        Pkg.PackageSpec(name="PlutoUI", version="0.7"),
    ])
    using HypertextLiteral, PlutoUI
end

# ╔═╡ 8de19918-2a32-4b74-a6c4-3fa229c36afd
rand_triangle = [round.(vi,digits=5) for vi ∈ [rand(i) for i=1:6]]

# ╔═╡ dd50d34b-edb3-4e3f-9a2f-77bfe0ed3fd0
pyramid(rand_triangle)

# ╔═╡ bdbd30d7-8714-44bb-a81f-adf988a225a4
md"""
### Vertical and horizontal padding
"""

# ╔═╡ 87b16571-05a3-4bf0-bc12-3f7bb302e364
pyramid(rand_triangle; padding_y = 20)

# ╔═╡ 2b4bd381-87ce-4b0c-8753-4b5a68a7389e
pyramid(rand_triangle; padding_x = 40)

# ╔═╡ 76f08f06-9351-4815-8881-dc978b0e2031
md"""
### Horizontal pyramid
"""

# ╔═╡ c6b3192f-efa3-4ad5-92c0-2eac700f6b26
pyramid(rand_triangle; horizontal=true)

# ╔═╡ 35f75f0a-08c5-47ea-9930-e70f20b8a64a
function pyramid(rows::Vector{<:Vector};
		horizontal=false,
		padding_x=8,
		padding_y=2,
	)

	style = (; padding = "$(padding_y)px $(padding_x)px")
	render_row(xs) = @htl("""
<padder></padder>$([@htl("$(x)") for x in xs])<padder></padder>
""")

	@htl("""

		$(render_row.(rows))

		""")
end

# ╔═╡ Cell order:
# ╠═e73aaef4-97e7-11eb-1722-9d275d445f1c
# ╠═568374de-9717-11eb-3ad1-571475dfe65c
# ╠═a5cb88e1-95de-4a01-8738-1761052d4886
# ╠═4c6bdeec-cb3b-42da-9cba-f625818bc9c5
# ╠═1a651f93-5df0-4ce6-991d-9720da29a7e3
# ╠═3c0baae7-6a34-4126-8d19-5915cf64d7bd
# ╠═3a765d51-95d0-444d-adab-f794190914f2
# ╟─fec4bcfb-b88a-4008-82fa-194b8f7264e0
# ╠═116ffb13-8b9a-40b4-aa1e-88d645d1124a
# ╠═8de19918-2a32-4b74-a6c4-3fa229c36afd
# ╠═dd50d34b-edb3-4e3f-9a2f-77bfe0ed3fd0
# ╟─bdbd30d7-8714-44bb-a81f-adf988a225a4
# ╠═87b16571-05a3-4bf0-bc12-3f7bb302e364
# ╠═2b4bd381-87ce-4b0c-8753-4b5a68a7389e
# ╟─76f08f06-9351-4815-8881-dc978b0e2031
# ╠═c6b3192f-efa3-4ad5-92c0-2eac700f6b26
# ╠═35f75f0a-08c5-47ea-9930-e70f20b8a64a