# ---
# title: 1502. Can Make Arithmetic Progression From Sequence
# id: problem1502
# author: zhwang
# date: 2022-03-22
# difficulty: Easy
# categories: Array, Sort
# link: <https://leetcode.com/problems/can-make-arithmetic-progression-from-sequence/description/>
# hidden: true
# ---
#
# Given an array of numbers `arr`. A sequence of numbers is called an arithmetic
# progression if the difference between any two consecutive elements is the
# same.
#
# Return `true` if the array can be rearranged to form an arithmetic
# progression, otherwise, return `false`.
#
#
#
# **Example 1:**
#
#
#
#     Input: arr = [3,5,1]
#     Output: true
#     Explanation: We can reorder the elements as [1,3,5] or [5,3,1] with differences 2 and -2 respectively, between each consecutive elements.
#
#
# **Example 2:**
#
#
#
#     Input: arr = [1,2,4]
#     Output: false
#     Explanation: There is no way to reorder the elements to obtain an arithmetic progression.
#
#
#
#
# **Constraints:**
#
#   * `2 <= arr.length <= 1000`
#   * `-10^6 <= arr[i] <= 10^6`
#
#
## @lc code=start
using LeetCode

can_make_arith(arr::Vector{Int})::Bool = length(unique!(diff(sort(arr)))) == 1

## @lc code=end

## @lc test=start
@testset "1502.can-make-arithmetic-progression-from-sequence.jl" begin
    @test can_make_arith([3,5,1])
    @test !can_make_arith([1,2,4])
end
## @lc test=end

## @lc info=start
# link: [solution to 1502]()
## @lc info=end


# # Hydrostatic Equilibrium test for Richards Equation

# This tutorial shows how to use `ClimateMachine` code to solve
# Richards equation in a column of soil. We choose boundary
# conditions of zero flux at the top and bottom of the column,
# and then run the simulation long enough to see that the system
# is approaching hydrostatic equilibrium, where the gradient of the
# pressure head is equal and opposite the gradient of the
# gravitational head. Note that the [`SoilWaterModel`](@ref
# ClimateMachine.Land.SoilWaterModel) includes
# a prognostic equation for the volumetric ice fraction,
# as ice is a form of water that must be accounted for to ensure
# water mass conservation. If freezing and thawing are not turned on
# (the default), the amount of ice in the model is zero for all space and time
# (again by default).

# The equations are:

# ``
# \frac{ ∂ ϑ_l}{∂ t} = ∇ ⋅ K (T, ϑ_l, θ_i; ν, ...) ∇h( ϑ_l, z; ν, ...).
# ``

# ``
# \frac{ ∂ θ_i}{∂ t} = 0
# ``

# Here

# ``t`` is the time (s),

# ``z`` is the location in the vertical (m),

# ``T`` is the temperature of the soil (K),

# ``K`` is the hydraulic conductivity (m/s),

# ``h`` is the hydraulic head (m),

# ``ϑ_l`` is the augmented volumetric liquid water fraction,

# ``θ_i`` is the volumetric ice fraction, and

# ``ν, ...`` denotes parameters relating to soil type, such as porosity.

# We will solve this equation in an effectively 1-d domain with ``z ∈ [-10,0]``,
# and with the following boundary and initial conditions:

# ``- K ∇h(t, z = 0) = 0 ẑ ``

# `` -K ∇h(t, z = -10) = 0 ẑ``

# `` ϑ(t = 0, z) = ν-0.001 ``

# `` θ_i(t = 0, z) = 0.0. ``

# where ``\nu`` is the porosity.

# A word about the hydraulic conductivity: please see the
# [`hydraulic functions`](./hydraulic_functions.md) tutorial
# for options regarding this function. The user can choose to make it depend
# on the temperature and the amount of ice in the soil; the default, which we use
# here, is that `K` only depends on the liquid moisture content.

# Lastly, our formulation of this equation allows for a continuous solution in both
# saturated and unsaturated areas, following [1].

# # Preliminary setup

# - Load external packages

using MPI
using OrderedCollections
using StaticArrays
using Statistics

# - Load CLIMAParameters and ClimateMachine modules

using CLIMAParameters
struct EarthParameterSet <: AbstractEarthParameterSet end
const param_set = EarthParameterSet()

using ClimateMachine
using ClimateMachine.Land
using ClimateMachine.Land.SoilWaterParameterizations
using ClimateMachine.Mesh.Topologies
using ClimateMachine.Mesh.Grids
using ClimateMachine.DGMethods
using ClimateMachine.DGMethods.NumericalFluxes
using ClimateMachine.DGMethods: BalanceLaw, LocalGeometry
using ClimateMachine.MPIStateArrays
using ClimateMachine.GenericCallbacks
using ClimateMachine.ODESolvers
using ClimateMachine.VariableTemplates
using ClimateMachine.SingleStackUtils
using ClimateMachine.BalanceLaws:
    BalanceLaw, Prognostic, Auxiliary, Gradient, GradientFlux, vars_state

# - Define the float type desired (`Float64` or `Float32`)
const FT = Float64;

# - Initialize ClimateMachine for CPU
ClimateMachine.init(; disable_gpu = true);

# Load plot helpers:
const clima_dir = dirname(dirname(pathof(ClimateMachine)));
include(joinpath(clima_dir, "docs", "plothelpers.jl"));

# # Set up the soil model

# We want to solve Richards equation alone, without simultaneously
# solving the heat equation. Because of that, we choose a
# [`PrescribedTemperatureModel`](@ref
# ClimateMachine.Land.PrescribedTemperatureModel).
# The user can supply a function for temperature,
# depending on time and space; if this option is desired, one could also
# choose to model the temperature dependence of viscosity, or to drive
# a freeze/thaw cycle, for example. If the user simply wants to model
# Richards equation for liquid water, the defaults will allow for that.
# Here we ignore the effects of temperature and freezing and thawing,
# using the defaults.

soil_heat_model = PrescribedTemperatureModel();

# Define the porosity, Ksat, and specific storage values for the soil. Note
# that all values must be given in mks units. The soil parameters chosen
# roughly correspond to Yolo light clay.
soil_param_functions = SoilParamFunctions{FT}(
    porosity = 0.495,
    Ksat = 0.0443 / (3600 * 100),
    S_s = 1e-3,
);

# Define the boundary conditions. The user can specify two conditions,
# either at the top or at the bottom, and they can either be Dirichlet
# (on `ϑ_l`) or Neumann (on `-K∇h`). Note that fluxes are supplied as
# scalars, inside the code they are multiplied by ẑ. The two conditions
# not supplied must be set to `nothing`.

surface_flux = (aux, t) -> eltype(aux)(0.0)
bottom_flux = (aux, t) -> eltype(aux)(0.0)
surface_state = nothing
bottom_state = nothing
bc = GeneralBoundaryConditions(
    Dirichlet(surface_state = surface_state, bottom_state = bottom_state),
    Neumann(surface_flux = surface_flux, bottom_flux = bottom_flux),
)

# Define the initial state function. The default for `θ_i` is zero.
ϑ_l0 = (aux) -> eltype(aux)(0.494);

# Create the SoilWaterModel. The defaults are a temperature independent
# viscosity, and no impedance factor due to ice. We choose to make the
# hydraulic conductivity a function of the moisture content `ϑ_l`,
# and employ the vanGenuchten hydraulic model with `n` = 2.0. The van
# Genuchten parameter `m` is calculated from `n`, and we use the default
# value for `α`.
soil_water_model = SoilWaterModel(
    FT;
    moisture_factor = MoistureDependent{FT}(),
    hydraulics = vanGenuchten{FT}(n = 2.0),
    initialϑ_l = ϑ_l0,
    boundaries = bc,
);

# Create the soil model - the coupled soil water and soil heat models.
m_soil = SoilModel(soil_param_functions, soil_water_model, soil_heat_model);

# We are ignoring sources and sinks here, like runoff or freezing and thawing.
sources = ();

# Define the function that initializes the prognostic variables. This
# in turn calls the functions supplied to `soil_water_model`.
function init_soil_water!(land, state, aux, localgeo, time)
    state.soil.water.ϑ_l = eltype(state)(land.soil.water.initialϑ_l(aux))
    state.soil.water.θ_i = eltype(state)(land.soil.water.initialθ_i(aux))
end

# Create the land model - in this tutorial, it only includes the soil.
m = LandModel(
    param_set,
    m_soil;
    source = sources,
    init_state_prognostic = init_soil_water!,
);

# # Specify the numerical configuration and output data.

# Specify the polynomial order and vertical resolution.
N_poly = 5;
nelem_vert = 20;

# Specify the domain boundaries.
zmax = FT(0);
zmin = FT(-10);

# Create the driver configuration.
driver_config = ClimateMachine.SingleStackConfiguration(
    "LandModel",
    N_poly,
    nelem_vert,
    zmax,
    param_set,
    m;
    zmin = zmin,
    numerical_flux_first_order = CentralNumericalFluxFirstOrder(),
);

# Choose the initial and final times, as well as a timestep.
t0 = FT(0)
timeend = FT(60 * 60 * 24 * 4)
dt = FT(5);

# Create the solver configuration.
solver_config =
    ClimateMachine.SolverConfiguration(t0, timeend, driver_config, ode_dt = dt);

# Determine how often you want output.
const n_outputs = 5;

const every_x_simulation_time = ceil(Int, timeend / n_outputs);

# Create a place to store this output.
state_types = (Prognostic(), Auxiliary(), GradientFlux())
dons_arr = Dict[dict_of_nodal_states(solver_config, state_types; interp = true)]
time_data = FT[0] # store time data

callback = GenericCallbacks.EveryXSimulationTime(every_x_simulation_time) do
    dons = dict_of_nodal_states(solver_config, state_types; interp = true)
    push!(dons_arr, dons)
    push!(time_data, gettime(solver_config.solver))
    nothing
end;

# # Run the integration
ClimateMachine.invoke!(solver_config; user_callbacks = (callback,));

# Get the final state and create plots:
dons = dict_of_nodal_states(solver_config, state_types; interp = true)
push!(dons_arr, dons)
push!(time_data, gettime(solver_config.solver));

# Get z-coordinate
z = get_z(solver_config.dg.grid; rm_dupes = true);

# # Create some plots
# We'll plot the moisture content vs depth in the soil, as well as
# the expected slope of `ϑ_l` in hydrostatic equilibrium when the soil
# is saturated. For `ϑ_l` values above porosity, the soil is
# saturated, and the pressure head changes from being equal to the matric
# potential to the pressure generated by compression of water and the soil
# matrix. The pressure head is continuous at porosity, but the derivative
# is not.
slope = -1e-3

output_dir = @__DIR__;

t = time_data ./ (60 * 60 * 24);

plot(
    dons_arr[1]["soil.water.ϑ_l"],
    dons_arr[1]["z"],
    label = string("t = ", string(t[1]), "days"),
    xlim = [0.47, 0.501],
    ylabel = "z",
    xlabel = "ϑ_l",
    legend = :bottomleft,
    title = "Equilibrium test",
);
plot!(
    dons_arr[4]["soil.water.ϑ_l"],
    dons_arr[4]["z"],
    label = string("t = ", string(t[4]), "days"),
);
plot!(
    dons_arr[6]["soil.water.ϑ_l"],
    dons_arr[6]["z"],
    label = string("t = ", string(t[6]), "days"),
);
plot!(
    (dons_arr[1]["z"] .+ 10.0) .* slope .+ dons_arr[6]["soil.water.ϑ_l"][1],
    dons_arr[1]["z"],
    label = "expected",
);

plot!(
    1e-3 .+ dons_arr[1]["soil.water.ϑ_l"],
    dons_arr[1]["z"],
    label = "porosity",
);
# save the output.
savefig(joinpath(output_dir, "equilibrium_test_ϑ_l_vG.png"))
# ![](equilibrium_test_ϑ_l_vG.png)
# # References
# [1] Woodward, C. S., and C. N. Dawson (2000), Analysis of expanded mixed
# finite element methods for a nonlinear parabolic equation modeling
# flow into variably saturated porous media, SIAM J. Numer. Anal., 37, 701–724


import Base: rand
using Random: AbstractRNG, Sampler, SamplerType

rand(rng::AbstractRNG, ::SamplerType{Leaf{N}}) where {N} = Leaf{N}(rand(rng, 1:N))

"""
    BoltzmannSampler{N}(m, n)
Configuration for Boltzmann sampling `DecisionTree{N}`s with minimum size `m` and maximum size `n`.
"""
struct BoltzmannSampler{N} <: Sampler{Tree{N}}
    minsize::Int
    maxsize::Int
end

function boltzmann_ub(rng::AbstractRNG, sampler::BoltzmannSampler{N}, cursize) where {N}
    minsize = sampler.minsize
    maxsize = sampler.maxsize

    if cursize ≥ maxsize        # maximum bound exceeded
        return nothing, cursize
    elseif rand(rng) ≤ 0.5      # generate random leaf
        return rand(rng, Leaf{N}), cursize
    else                        # try generating a branch -- if its size is small enough
        # try generating children, or immediately fail
        left_children, cursize = boltzmann_ub(rng, sampler, cursize + 1)
        left_children === nothing && return nothing, cursize

        right_children, cursize = boltzmann_ub(rng, sampler, cursize + 1)
        right_children === nothing && return nothing, cursize

        # `cursize` is now the size of the whole thing, and less than maxsize
        return Branch{N}(left_children, right_children), cursize
    end
end

function rand(rng::AbstractRNG, sampler::BoltzmannSampler{N}) where {N}
    candidate, size = boltzmann_ub(rng, sampler, 0)
    while candidate === nothing || size ≤ sampler.minsize
        candidate, size = boltzmann_ub(rng, sampler, 0)
    end

    return candidate
end


@testset "$TEST $G" begin

g4 = PathDiGraph(5, DG)
d1 =EdgeMap(g4, float([ 0 1 2 3 4; 5 0 6 7 8; 9 10 0 11 12; 13 14 15 0 16; 17 18 19 20 0]))
d2 =EdgeMap(g4, sparse(float([ 0 1 2 3 4; 5 0 6 7 8; 9 10 0 11 12; 13 14 15 0 16; 17 18 19 20 0])))

y = bellman_ford_shortest_paths(g4, 2, d1)
z = bellman_ford_shortest_paths(g4, 2, d2)
@test y.dists == z.dists == [Inf, 0, 6, 17, 33]
@test enumerate_paths(z)[2] == []
@test enumerate_paths(z)[4] == enumerate_paths(z,4) == [2,3,4]
@test !has_negative_edge_cycle(g4)

z = bellman_ford_shortest_paths(g4, 2)
@test z.dists == [typemax(Int), 0, 1, 2, 3]

end # testset


"""copy a FEFunction on a FE Space"""
clone_fe_function(space,f)=FEFunction(space,copy(get_free_dof_values(f)))

"""
 Given an arbitrary number of FEFunction arguments,
 returns a tuple with their corresponding free DoF Values
"""
function Gridap.get_free_dof_values(functions...)
  map(get_free_dof_values,functions)
end

function setup_mixed_spaces(model, order)
  reffe_rt  = ReferenceFE(raviart_thomas, Float64, order)
  V = FESpace(model, reffe_rt ; conformity=:HDiv)
  U = TrialFESpace(V)
  reffe_lgn = ReferenceFE(lagrangian, Float64, order)
  Q = FESpace(model, reffe_lgn; conformity=:L2)
  P = TrialFESpace(Q)
  reffe_lgn = ReferenceFE(lagrangian, Float64, order+1)
  S = FESpace(model, reffe_lgn; conformity=:H1)
  R = TrialFESpace(S)

  R, S, U, V, P, Q
end

function setup_and_factorize_mass_matrices(dΩ, R, S, U, V, P, Q)
  amm(a,b) = ∫(a⋅b)dΩ
  H1MM = assemble_matrix(amm, R, S)
  RTMM = assemble_matrix(amm, U, V)
  L2MM = assemble_matrix(amm, P, Q)
  H1MMchol = lu(H1MM)
  RTMMchol = lu(RTMM)
  L2MMchol = lu(L2MM)

  H1MM, RTMM, L2MM, H1MMchol, RTMMchol, L2MMchol
end

function new_vtk_step(Ω,file,_cellfields)
  n = size(_cellfields)[1]
  createvtk(Ω,
            file,
            cellfields=_cellfields,
            nsubcells=n)
end


# Julia1_3.4.jl Methods
#
# restricting cylinder function parameters to Float64
# "multiple dispatch" means that all parameters are used
#   to decide which method to use.
# So for efficiency we restrict the call parameters using
#  type assertion operators so that the operations
#  in the expression π*h*r^2 won't have to decide.
cylinder(r::Float64,h::Float64) = π*h*r^2  # for cylinder volume
println(cylinder(5.0,20.0))
println(cylinder(5,20.0))
println(typeof(5))
#
# Generalizing but still efficient for Float64 since it
# will use method 1.
# The most specific specifications take dispatch priority
cylinder(r::Real,h::Real) = π*h*r^2   # for cylinder volume
println(cylinder(5,20.0))

methods(cylinder) # we defined 2 methods for cylinder
#
# What if we just defined cylinder(r,h) = π*h*r^2
#  r and h would be of type 'any' which includes
#  Number, String, Char or any user defined type, often
#    undesirable
cuboid(l,h,d)=l*h*d   # will even take strings without complaining
cuboid("This"," is ","wrong!")

# perhaps want functions to check if types match
sametype(x::T, y::T) where {T}=true
sametype(x,y)=false
println(sametype(3,3.0))
println(sametype(3,2))
println(sametype(1+2im,3.1+4im))
println(sametype(1.0+2.0im,3.1+4.0im))

# Making sure myappend appends data of the same type
myappend(v::Vector{T}, x::T) where {T} = [v..., x]
y=myappend([1,2,3],4)

w=myappend(["stop","forword","slow"],"reverse")

w[3]


@testset "Interval" begin
    @test 0..1 isa Interval
    interval = 0..1

    @test 0 ∈ interval
    @test 1 ∈ interval
    @test 2 ∉ interval
    @test -1 ∉ interval
    @test 0.5 ∈ interval
    @test 1.1 ∉ interval
    @test -0.1 ∉ interval
end


include("SingleIntegrator.jl")
include("LinearSystem.jl")
include("MultipleEnvs.jl")
include("ReferenceModel.jl")
include("TwoDimensionalNonlinearOscillator.jl")
include("TwoDimensionalNonlinearPolynomialSystem.jl")
include("TwoDimensionalNonlinearDTSystem.jl")


@testset "LinearRegressor" begin
    # Define some linear, noise-free, synthetic data:
    # with intercept = 0.0
    n = 1000
    rng = StableRNG(1234)
    X, y = MLJBase.make_regression(n, 3, noise=0, intercept=false, rng=rng)
    # Train model with intercept on all data
    linear = LinearRegressor()
    yhat, fr = test_regression(linear, X, y)
    # Training error
    @test norm(yhat - y)/sqrt(n) < 1e-12
    # Get the true intercept?
    @test abs(fr.intercept) < 1e-10
    # Check metadata
end

@testset "MultitargetLinearRegressor" begin
    # Define some linear, noise-free, synthetic data:
    # with intercept = 0.0
    n = 1000
    rng = StableRNG(1234)
    X, Y = make_regression2(n, 3, noise=0, intercept=false, rng=rng)
    Y_mat = MLJBase.matrix(Y)
    # Train model on all data
    multi_linear = MultitargetLinearRegressor()
    Yhat, fr = test_regression(multi_linear, X, Y)
    Yhat_mat = MLJBase.matrix(Yhat)
    # Check if the column names is same after predict
    MLJBase.schema(Yhat).names == MLJBase.schema(Y).names
    # Training error
    @test norm(Yhat_mat - Y_mat)/sqrt(n) < 1e-12
    # Get the true intercept?
    @test norm(fr.intercept .- zeros(size(Y_mat, 2))) < 1e-10
end

@testset "RidgeRegressor" begin
    # Define some linear, noise-free, synthetic data:
    # with intercept = 0.0
    n = 1000
    rng = StableRNG(1234)
    X, y = MLJBase.make_regression(n, 3, noise=0, intercept=false, rng=rng)
    # Train model with intercept on all data with no regularization
    # and no standardization of target.
    ridge = RidgeRegressor(lambda=0.0)
    yhat, fr = test_regression(ridge, X, y)
    # Training error
    @test norm(yhat - y)/sqrt(n) < 1e-12
    # Get the true intercept?
    @test abs(fr.intercept) < 1e-10
    # Check metadata
end

@testset "MultitargetRidgeRegressor" begin
    # Define some linear, noise-free, synthetic data:
    # with intercept = 0.0
    n = 1000
    rng = StableRNG(1234)
    X, Y = make_regression2(n, 3, noise=0, intercept=false, rng=rng)
    Y_mat = MLJBase.matrix(Y)
    # Train model with intercept on all data with no regularization
    # and no standardization of target.
    multi_ridge = MultitargetRidgeRegressor(lambda=0.0)
    Yhat, fr = test_regression(multi_ridge, X, Y)
    Yhat_mat = MLJBase.matrix(Yhat)
    # Check if the column names is same after predict
    MLJBase.schema(Yhat).names == MLJBase.schema(Y).names
    # Training error
    @test norm(Yhat_mat - Y_mat)/sqrt(n) < 1e-12
    # Get the true intercept?
    @test norm(fr.intercept .- zeros(size(Y_mat, 2))) < 1e-10
end


using Distances
using Clustering
using DelimitedFiles
using PdfFraud

# set up the paths
reprpath = "vae_runs/2"
reprpath = joinpath(PdfFraud.experiment_path, reprpath, "representations")

# collect the files and encodings
encfs = filter(x->x!="index.csv",readdir(reprpath));
encodings = map(x->readdlm(x, ','), joinpath.(reprpath, encfs));

# get labels
_, labels, _, _ = PdfFraud.load_pdf_data();
linds = map(x->x!=nothing, labels)
Y = Int.(labels[linds]);

# compute distances for one encoding and use it to compute silhouttes
enc = encodings[11];
X = Array(enc[linds,:]');
silhouettes_encs(data, labels) = silhouettes(labels, pairwise(Euclidean(), data, dims=2))
silhs = silhouettes_encs(X, Y);


using CurveProximityQueries
using ConvexBodyProximityQueries
using Test
using LinearAlgebra
using StaticArrays
using IntervalArithmetic
using Random: seed!

import CurveProximityQueries: differentiate, integrate

@testset "CurveProximityQueries" begin
    @testset "Bernstein Polynomials" begin
        B2 = rand(Bernstein{2,8})
        B3 = rand(Bernstein{3,5})
        @test typeof(B2) <: Bernstein{2,8}
        @test typeof(B3) <: Bernstein{3,5}
        @test sum(norm.((differentiate(integrate(B2.f)) - B2.f).control_points)) ≤ 1e-10
        @test sum(norm.((differentiate(integrate(B3.f)) - B3.f).control_points)) ≤ 1e-10
    end
    @testset "Curve - Polygon" begin
        seed!(1)
        obs = @point zeros(2)
        c = rand(Bernstein{2,7})
        @test abs(minimum_distance(obs, c) - 0.3683522584741768) ≤ 1e-5
        @test tolerance_verification(obs, c, 0.3) == true
        @test tolerance_verification(obs, c, 0.4) == false
        @test collision_detection(obs, c) == false
        @test abs(minimum_distance(c, obs) - 0.3683522584741768) ≤ 1e-5
        @test tolerance_verification(c, obs, 0.3) == true
        @test tolerance_verification(c, obs, 0.4) == false
        @test collision_detection(c, obs) == false

        seed!(1)
        obs = @point zeros(3)
        c = rand(Bernstein{3,11})
        @test abs(minimum_distance(obs, c) - 0.511627527056288) ≤ 1e-5
        @test tolerance_verification(obs, c, 0.5) == true
        @test tolerance_verification(obs, c, 0.6) == false
        @test collision_detection(obs, c) == false
        @test abs(minimum_distance(c, obs) - 0.511627527056288) ≤ 1e-5
        @test tolerance_verification(c, obs, 0.5) == true
        @test tolerance_verification(c, obs, 0.6) == false
        @test collision_detection(c, obs) == false
    end
    @testset "Curve - Curve" begin
        seed!(1)
        c = rand(Bernstein{2,7})
        d = rand(Bernstein{2,3})
        @test abs(minimum_distance(c, d)) ≤ 1e-5
        @test tolerance_verification(c, d, 0.3) == false
        @test collision_detection(c, d) == true

        seed!(1)
        obs = @point zeros(3)
        c = rand(Bernstein{3,11})
        d = rand(Bernstein{3,5})
        @test abs(minimum_distance(c, d) - 0.137873546917387) ≤ 1e-5
        @test tolerance_verification(c, d, 0.1) == true
        @test tolerance_verification(c, d, 0.3) == false
        @test collision_detection(c, d) == false
    end
end


module CleanCMB
using LinearAlgebra
export ilc_weights, ilc_clean_cij
export cilc_weights
export milca_weights
export tsz, dust1, synch
export loglike_beta
include("ilc.jl")
include("cilc.jl")
include("milca.jl")
include("libspectra.jl")
include("profile_likelihood.jl")
end


"""
    compute_mf_diffusion_tensor(mesh, M, lap_eig, gradient::ScalarGradient)

Compute effective diffusion tensors using the matrix formalism.
"""
function compute_mf_diffusion_tensor(mesh, M, lap_eig, gradient::ScalarGradient)
    volumes = get_cmpt_volumes(mesh)
    p = reduce(hcat, mesh.points)
    a = p * M * lap_eig.funcs
    j = j_integral.(gradient.profile, lap_eig.values)
    D = a * Diagonal(j) * a' / sum(volumes)
end


#!/usr/bin/env julia

using Eigenbroetler

function sunflower(x::Number, y::Number)
    if x == 0 && y == 0
        return 0.0
    end
    r = sqrt(x * x + y * y)
    ϕ = atan(y, x)
    return cos(30 * log(r) + 13 * ϕ) ^ 2 + cos(20 * log(r) - 11 * ϕ) ^ 2
end

eb = Eigenbrot(sunflower, 512, 512)
save("sunflower.png", eb, ImageSetting(RealPart, LinearScale), colours = :rainbow)


# ------------------------------------------------------------------
# Licensed under the ISC License. See LICENCE in the project root.
# ------------------------------------------------------------------

"""
    SmoothingProcess(σ=3)

A smoothing process (i.e. Gaussian filter) with `σ` parameter.
"""
struct SmoothingProcess <: TimelessProcess
  σ::Float64
end

SmoothingProcess() = SmoothingProcess(3.0)

evolve!(land::Matrix, proc::SmoothingProcess) =
  imfilter!(land, land, centered(Kernel.gaussian(proc.σ)), "replicate")


@testset "L-shape" begin
    Ns = 8:8:16
    optim_prec_final = 64

    RR = RealField(1024)
    result = RR("[9.6397238440219410527114592624 +/- 1.43e-26]")
    goalradius = 3e-2

    domain = LShape(RealField(128))
    u = LShapeEigenfunction(domain)
    interval = ball(RR(9.6), RR(0.5))

    println("Computing eigenvalue for the $domain")
    λs, _ = iteratemps(
        domain,
        u,
        interval,
        Ns,
        optim_prec_final = optim_prec_final,
        optim_prec_linear = true,
        show_trace = true,
    )
    rad = Float64(radius(λs[end]))

    @test overlaps(result, λs[end])
    @test rad < goalradius

    if rad < goalradius
        @printf "radius = %e < %e\n" rad goalradius
    else
        @printf "radius = %e >= %e\n" rad goalradius
    end
end


# Arvind Prasadan
# 2019 January
# DMF Package
# SOBI Function (R) Wrapper

"""
	 SOBI_Wrapper(X, lags)

A wrapper function for the Second Order Blind Identification (SOBI) Source Separation algorithm. This is a wrapper that calls the function from R, and depends on the presence of an R install as well as the jointDiag package being installed.

# Arguments
- `X`: Data matrix, p dimensions/sensors x n samples
- `lags`: List of (Integral) lags for the algorithm to operate at

# Outputs
- `Q_hat`: Eigenvector Estimates
- `S_hat: The recovered latent time series
"""
function SOBI_Wrapper(X, lags)

    n = size(X, 2)
    p = size(X, 1)

    # Sanity check lags
    lags = abs.(Int.(round.(lags)))
    if 1 < length(lags)
        lags = [min(x, n - 1) for x in lags]
        lags = sort(unique(lags))
    end

    # Whiten X
    mu = mean(X; dims = 2)
    X = X .- mu
    cov_X = X * X'
    cov_X = (cov_X + cov_X') / 2.0
    cW, cV = eigen(cov_X)
    cov_invsqrt = cV * Diagonal(sqrt.(abs.(pinv(cW)))) * cV'
    cov_sqrt = cV * Diagonal(sqrt.(abs.(cW))) * cV'
    Xw = cov_invsqrt * X

    # Store matrices at different lags
    A = zeros(p, p, length(lags))
    for ll = 1:1:length(lags)
	    A[:, :, ll] = Xw[:, (lags[ll] + 1):end] * pinv(Xw[:, 1:(end - lags[ll])]) + Diagonal(ones(p)) # Regularizer
    end

    # Put Variables in R
    R"library(jointDiag)"
    R"A = $(A)"

    # Run SOBI: Joint Diagonalization
    Q_hat = reval("ajd(A, method = \"jedi\")\$A")
    Q_hat = convert(Array{Float64, 2}, Q_hat)
    Q_hat = cov_sqrt * Q_hat

    S_hat = (pinv(Q_hat) * (X .+ mu))'

    return (Q_hat, S_hat)
end


#!/usr/bin/julia

# Copyright (C) 2021 Matheus Fernandes Bigolin <mfrdrbigolin@disroot.org>
# SPDX-License-Identifier: MIT

"Day Seventeen, Trick Shot."

#= I am working in an analytical solution, but, for the time being, I will
 = leave the brute-force solution in part two due to time constrainsts (my
 = personal time, not the program’s).  At the very least, I put some limitations
 = on the ranges.
 =#

include("Utils.jl")

using .Utils

#= Throught this entire code the target area is assumed to be to the
 = bottom-right of the thrower.
 =#

"Find the maximum probe height possible that still lands in `target`."
function solve1(target)
    #= When the probe crosses the y = 0 line for the second time it’s velocity
     = will always be -vy - 1 (assuming vy >= 0).  Therefore, we want the
     = probe’s next step to fall exactly within the outermost vertical line of
     = the target area to maximize it’s vy.
     =#
    max_y = -target[2].start - 1

    return div(max_y^2 + max_y, 2)
end

"Calculate the x position at time `t` given the horizontal speed `vx`."
calc_x(t, vx) = if t < vx t*vx - div(t^2 - t, 2) else div(vx^2 + vx, 2) end
"Calculate the y position at time `t` given the vertical speed `vy`."
calc_y(t, vy) = t*vy - div(t^2 - t, 2)
"Calculate the position at time `t` given the speed pair `(vx, vy)`."
calc_xy(t, (vx, vy)) = (calc_x(t, vx), calc_y(t, vy))

"Count all the initial velocities that fall within the `target`."
function solve2(target)
    found_count = 0

    max_x = target[1].stop

    min_y = target[2].start
    max_y = -target[2].start - 1

    #= The speed that takes the maximum time is the one with maximum vertical
     = velocity.
     =#
    max_time = 2max_y + 2

    for vx in 1:max_x
        for vy in min_y:max_y
            for t in 1:max_time
                if all(calc_xy(t, (vx, vy)) .∈ target)
                    found_count += 1

                    break
                end
            end
        end
    end

    return found_count
end

const AREA_REX = r"target area: x=([-]?\d+)\.\.([-]?\d+), y=([-]?\d+)\.\.([-]?\d+)"

function main()
    usage_and_exit(length(ARGS) != 1)

    data = readline(first(ARGS))
    x₀, x₁, y₀, y₁ = parse.(Int, match(AREA_REX, data).captures)
    target = [x₀:x₁, y₀:y₁]

    println(solve1(target))
    println(solve2(target))
end

if abspath(PROGRAM_FILE) == @__FILE__
    main()
end


# Estimate gradient from a small set of points.

function gradest3(x, y)
    @assert size(x,1)==size(y,1)
    (x1,x2)=size(x)
    global dx = similar(x, (x1-1, x2))
    global dy = similar(y, (x1-1, 1))
    ix = 0
    for i=2:x1
        dx[i-1,:] = x[1,:]-x[i,:]
        dy[i-1,:] = y[1,:]-y[i,:]
    end
    dx\dy
end

function gradest2(x, y)
    @assert size(x,1)==size(y,1)
    (x1,x2)=size(x)
    x3 = div(x1*(x1-1),2)
    global dx = similar(x, (x3, x2))
    global dy = similar(y, (x3, 1))
    ix = 0
    for i=1:x1
        for j=(i+1):x1
            ix += 1
            dx[ix,:] = x[i,:]-x[j,:]
            dy[ix,:] = y[i,:]-y[j,:]
        end
    end
    dx\dy
end

function gradest0()
    for t=1:10
        x=randn(2,7)
        x[:,1]=1
        b=randn(7,1)
        y=x*b
        bb=x\y
        @show t
        display(bb')
        display(x[1,:]-x[2,:])
        display(y[1]-y[2])
    end
end


export rotate, rotate!

"""
    rotate!(corrs,azi,baz)

Rotate cross-correlations from E,N,Z to R,T,Z coordinate system.
# Arguments
- `corrs::Array{CorrData,1}`: Array of EE, EN, EZ, NE, NN, NZ, ZE, ZN, ZZ CorrData
          for a single station-pair.
- `azi::Real`: Azimuth (in radians) from station 1 to station 2
- `baz::Real`: Back azimuth (in radians) from station 1 to station 2
"""
function rotate!(corrs::Array{CorrData,1},azi::Real,baz::Real)
    T = eltype(corrs[1].corr)
    azi = T(azi)
    baz = T(baz)
    N = length(corrs)
    # get time intersect of all components
    ts = [c.t for c in corrs]
    inter = intersect(ts)[1]

    # subset so all corrs match in time
    for ii = 1:N
        ind = findall(x -> x ∈ inter, corrs[ii].t)
        corrs[ii].t = corrs[ii].t[ind]
        corrs[ii].corr = corrs[ii].corr[:,ind]
    end

    comp_dict = Dict()
    for ii = 1:N
        comp_dict[corrs[ii].comp] = ii
    end

    # get EE EN NN NE comps for rotation to TT RR TR RT
    # see Lin et al., 2008: Surface wave tomography of the western
    # United States from ambient seismic noise, GJI
    big_rot = [[-cos(azi)*cos(baz) cos(azi)*sin(baz) -sin(azi)*sin(baz) sin(azi)*cos(baz)];
               [-sin(azi)*sin(baz) -sin(azi)*cos(baz) -cos(azi)*cos(baz) -cos(azi)*sin(baz)];
               [-cos(azi)*sin(baz) -cos(azi)*cos(baz) sin(azi)*cos(baz) sin(azi)*sin(baz)];
               [-sin(azi)*cos(baz) sin(azi)*sin(baz) cos(azi)*sin(baz) -cos(azi)*cos(baz)]]
    to_rot = Array{eltype(corrs[1].corr)}(undef,4,size(corrs[1].corr)[1],length(inter))
    to_rot[1,:,:] = corrs[comp_dict["EE"]].corr
    to_rot[2,:,:] = corrs[comp_dict["EN"]].corr
    to_rot[3,:,:] = corrs[comp_dict["NN"]].corr
    to_rot[4,:,:] = corrs[comp_dict["NE"]].corr
    rotated = rotation_kernel(big_rot,to_rot)

    # update correlations from EE EN NN NE to TT RR TR RT
    update_corr!(corrs[comp_dict["EE"]],rotated[1,:,:],"TT")
    update_corr!(corrs[comp_dict["EN"]],rotated[2,:,:],"RR")
    update_corr!(corrs[comp_dict["NN"]],rotated[3,:,:],"TR")
    update_corr!(corrs[comp_dict["NE"]],rotated[3,:,:],"RT")

    # rotate from EZ NZ to RZ TZ
    azi_rot = [[cos(azi) -sin(azi)];
               [sin(azi) cos(azi)]]
    to_rot = Array{eltype(corrs[1].corr)}(undef,2,size(corrs[1].corr)[1],length(inter))
    to_rot[1,:,:] = corrs[comp_dict["NZ"]].corr
    to_rot[2,:,:] = corrs[comp_dict["EZ"]].corr
    rotated = rotation_kernel(azi_rot,to_rot)
    update_corr!(corrs[comp_dict["NZ"]],rotated[1,:,:],"RZ")
    update_corr!(corrs[comp_dict["EZ"]],rotated[2,:,:],"TZ")

    # rotate from ZE ZN to ZR ZT
    baz_rot = [[-cos(baz) -sin(baz)];
               [sin(baz) -cos(baz)]]
    to_rot = Array{eltype(corrs[1].corr)}(undef,2,size(corrs[1].corr)[1],length(inter))
    to_rot[1,:,:] = corrs[comp_dict["ZN"]].corr
    to_rot[2,:,:] = corrs[comp_dict["ZE"]].corr
    rotated = rotation_kernel(baz_rot,to_rot)
    update_corr!(corrs[comp_dict["ZN"]],rotated[1,:,:],"ZR")
    update_corr!(corrs[comp_dict["ZE"]],rotated[2,:,:],"ZT")
    # update ZZ rotation
    corrs[comp_dict["ZZ"]].rotated = true

    return nothing
end
rotate(corrs::Array{CorrData,1},azi::Real,baz::Real) =
                      (U = deepcopy(corrs);rotate!(U,azi,baz);return U)

"""
  rotation_kernel(rot_matrix, to_rotate)

Rotation function for 2D correlation functions.

# Arguments
- `rot_matrix::Array`: Rotation matrix (either 2x2 or 4x4).
- `to_rotate::Array`: Array to be rotated column-wise.
"""
function rotation_kernel(rot_matrix::AbstractArray, to_rotate::AbstractArray)
    rotated = similar(to_rotate)
    for ii = 1:size(to_rotate,2)
        for jj = 1:size(to_rotate,3)
            rotated[:,ii,jj] .= rot_matrix * to_rotate[:,ii,jj]
        end
    end
    return rotated
end

"""
  update_corr!(C,comp)

Change component, name and corr of rotated correlation `C` to component `comp`.

# Arguments
- `C::CorrData`: CorrData object.
- `new_corr::Array`: Rotated correlation matrix.
- `comp::String`: New component after rotation, e.g. "RR"
"""
function update_corr!(C::CorrData,new_corr::AbstractArray,comp::String)
    # change comp of correlation
    C.comp = comp

    # change name of correlation
    newname = split(C.name,'.')
    newname[4] = newname[4][1:end-1] * comp[1]
    newname[8] = newname[8][1:end-1] * comp[2]
    C.name = join(newname,'.')

    # change rotated flag
    C.rotated = true

    # update correlation
    C.corr = new_corr
    return C
end


# This file contains functions for the Cost Efficiency DEA model
"""
    CostDEAModel
An data structure representing a cost DEA model.
"""
struct CostDEAModel <: AbstractCostDEAModel
    n::Int64
    m::Int64
    s::Int64
    rts::Symbol
    disposY::Symbol
    dmunames::Union{Vector{String},Nothing}
    eff::Vector
    lambda::SparseMatrixCSC{Float64, Int64}
    techeff::Vector
    alloceff::Vector
    Xtarget::Matrix
    Ytarget::Matrix
end

"""
    deacost(X, Y, W)
Compute cost efficiency using data envelopment analysis for
inputs `X`, outputs `Y` and price of inputs `W`.

# Optional Arguments
- `rts=:VRS`: chooses variable returns to scale. For constant returns to scale choose `:CRS`.
- `dispos=:Strong`: chooses strong disposability of outputs. For weak disposability choose `:Weak`.
- `names`: a vector of strings with the names of the decision making units.

# Examples
```jldoctest
julia> X = [5 3; 2 4; 4 2; 4 8; 7 9.0];

julia> Y = [7 4; 10 8; 8 10; 5 4; 3 6.0];

julia> W = [2 1; 2 1; 2 1; 2 1; 2 1.0];

julia> deacost(X, Y, W)
Cost DEA Model
DMUs = 5; Inputs = 2; Outputs = 2
Orientation = Input; Returns to Scale = VRS
──────────────────────────────────
       Cost  Technical  Allocative
──────────────────────────────────
1  0.615385      0.75     0.820513
2  1.0           1.0      1.0
3  1.0           1.0      1.0
4  0.5           0.5      1.0
5  0.347826      0.375    0.927536
──────────────────────────────────
```
"""
function deacost(X::Union{Matrix,Vector}, Y::Union{Matrix,Vector},
    W::Union{Matrix,Vector}; rts::Symbol = :VRS, dispos::Symbol = :Strong,
    names::Union{Vector{String},Nothing} = nothing,
    optimizer::Union{DEAOptimizer,Nothing} = nothing)::CostDEAModel

    # Check parameters
    nx, m = size(X, 1), size(X, 2)
    ny, s = size(Y, 1), size(Y, 2)

    nw, mw = size(W, 1), size(W, 2)

    if nx != ny
        throw(DimensionMismatch("number of rows in X and Y ($nx, $ny) are not equal"));
    end
    if nw != nx
        throw(DimensionMismatch("number of rows in W and X ($nw, $nx) are not equal"));
    end
    if mw != m
        throw(DimensionMismatch("number of columns in W and X ($mw, $m) are not equal"));
    end

    if dispos != :Strong && dispos != :Weak
        throw(ArgumentError("`disposX` must be :Strong or :Weak"));
    end

    # Default optimizer
    if optimizer === nothing
        optimizer = DEAOptimizer(:LP)
    end

    # Get minimum cost targets and lambdas
    n = nx

    Xtarget, clambdaeff = deamincost(X, Y, W, rts = rts, dispos = dispos, optimizer = optimizer)
    Ytarget = Y[:,:]

    # Cost, technical and allocative efficiency
    cefficiency  = vec( sum(W .* Xtarget, dims = 2) ./ sum(W .* X, dims = 2) )
    techefficiency = efficiency(dea(X, Y, orient = :Input, rts = rts, slack = false, disposY = dispos, optimizer = optimizer))
    allocefficiency = cefficiency ./ techefficiency

    return CostDEAModel(n, m, s, rts, dispos, names, cefficiency, clambdaeff, techefficiency, allocefficiency, Xtarget, Ytarget)

end

ismonetary(model::CostDEAModel)::Bool = false;

function Base.show(io::IO, x::CostDEAModel)
    compact = get(io, :compact, false)

    n = nobs(x)
    m = ninputs(x)
    s = noutputs(x)
    disposY = x.disposY
    dmunames = names(x)

    eff = efficiency(x)
    techeff = efficiency(x, :Technical)
    alloceff = efficiency(x, :Allocative)

    if !compact
        print(io, "Cost DEA Model \n")
        print(io, "DMUs = ", n)
        print(io, "; Inputs = ", m)
        print(io, "; Outputs = ", s)
        print(io, "\n")
        print(io, "Orientation = Input")
        print(io, "; Returns to Scale = ", string(x.rts))
        print(io, "\n")
        if disposY == :Weak print(io, "Weak disposability of outputs \n") end

        show(io, CoefTable(hcat(eff, techeff, alloceff), ["Cost", "Technical", "Allocative"], dmunames))
    end

end


### Day 01 - Problem 1
## No Time for a Taxicab
## Author: Thanasis Georgiou

# Puzzle input
input = "R4, R3, R5, L3, L5, R2, L2, R5, L2, R5, R5, R5, R1, R3, L2, L2, L1, R5, L3, R1, L2, R1, L3, L5, L1, R3, L4, R2, R4, L3, L1, R4, L4, R3, L5, L3, R188, R4, L1, R48, L5, R4, R71, R3, L2, R188, L3, R2, L3, R3, L5, L1, R1, L2, L4, L2, R5, L3, R3, R3, R4, L3, L4, R5, L4, L4, R3, R4, L4, R1, L3, L1, L1, R4, R1, L4, R1, L1, L3, R2, L2, R2, L1, R5, R3, R4, L5, R2, R5, L5, R1, R2, L1, L3, R3, R1, R3, L4, R4, L4, L1, R1, L2, L2, L4, R1, L3, R4, L2, R3, L1, L5, R4, R5, R2, R5, R1, R5, R1, R3, L3, L2, L2, L5, R2, L2, R5, R5, L2, R3, L5, R5, L2, R4, R2, L1, R3, L5, R3, R2, R5, L1, R3, L2, R2, R1"

# Unpack complex number
function unpack(num)
    return real(num), imag(num)
end

# Current position
pos = 0 + 0im
dir = 0 + 1im

# History
history = zeros(Int8, 2000, 2000) # Large enough array (TM)

# HQ Location
hqLoc = 0 + 0im
hqFound = false

# Parse the input
steps = split(input, ", ")

# For each command
for step in steps
    # Find direction
    dir = begin
        command = step[1]
        if command == 'L'
            dir * 1im
        else
            dir * (-1im)
        end
    end

    # Move blocks
    blockDelta = parse(Int64, step[2:end])
    for i=1:1:blockDelta
        pos += dir

        # Store history
        x, y = unpack(pos)
        history[1000 - x, 1000 - y] += 1

        # If we have been here before, this is the HQ
        if hqFound == false && history[1000 - x, 1000 - y] > 1
            hqLoc = pos
            hqFound = true
        end
    end
end

# Calculate distance
x, y = unpack(pos)
#println("Position: $(real(pos)), $(imag(pos))")
println("Position: $x, $y")
println("Distance to position: $(abs(x) + abs(y))")

hqX, hqY = unpack(hqLoc)
println("HQ Location: $hqX, $hqY")
println("HQ Distance: $(abs(hqX) + abs(hqY))")


using Flux
println("neural_adapter_tests")
using DiffEqFlux
println("Starting Soon!")
using ModelingToolkit
using DiffEqBase
using Test, NeuralPDE
println("Starting Soon!")
using GalacticOptim
using Optim
using Quadrature,Cubature, Cuba
using QuasiMonteCarlo
using SciMLBase
import ModelingToolkit: Interval, infimum, supremum
using Statistics

using Random
Random.seed!(100)

cb = function (p,l)
    println("Current loss is: $l")
    return false
end

## Example, 2D Poisson equation with Neural adapter
println("Example, 2D Poisson equation with Neural adapter")
@parameters x y
@variables u(..)
Dxx = Differential(x)^2
Dyy = Differential(y)^2

# 2D PDE
eq  = Dxx(u(x,y)) + Dyy(u(x,y)) ~ -sin(pi*x)*sin(pi*y)

# Initial and boundary conditions
bcs = [u(0,y) ~ 0.0, u(1,y) ~ -sin(pi*1)*sin(pi*y),
       u(x,0) ~ 0.0, u(x,1) ~ -sin(pi*x)*sin(pi*1)]
# Space and time domains
domains = [x ∈ Interval(0.0,1.0),
           y ∈ Interval(0.0,1.0)]
quadrature_strategy = NeuralPDE.QuadratureTraining(reltol=1e-2,abstol=1e-2,
                                                   maxiters =50, batch=100)
inner = 8
af = Flux.tanh
chain1 = Chain(Dense(2,inner,af),
               Dense(inner,inner,af),
               Dense(inner,1))
initθ = Float64.(DiffEqFlux.initial_params(chain1))
discretization = NeuralPDE.PhysicsInformedNN(chain1,
                                             quadrature_strategy;
                                             init_params = initθ)

@named pde_system = PDESystem(eq,bcs,domains,[x,y],[u(x,y)])
prob = NeuralPDE.discretize(pde_system,discretization)
sym_prob = NeuralPDE.symbolic_discretize(pde_system,discretization)
res = GalacticOptim.solve(prob, BFGS();  maxiters=2000)
phi = discretization.phi

inner_ = 10
af = Flux.tanh
chain2 = FastChain(FastDense(2,inner_,af),
                   FastDense(inner_,inner_,af),
                   FastDense(inner_,inner_,af),
                   FastDense(inner_,1))

initθ2 =Float64.(DiffEqFlux.initial_params(chain2))

function loss(cord,θ)
    chain2(cord,θ) .- phi(cord,res.minimizer)
end

grid_strategy = NeuralPDE.GridTraining(0.05)
quadrature_strategy = NeuralPDE.QuadratureTraining(reltol=1e-5,abstol=1e-5,
                                                   maxiters =50, batch=100)
stochastic_strategy = NeuralPDE.StochasticTraining(400)
quasirandom_strategy = NeuralPDE.QuasiRandomTraining(400,resampling =false,minibatch = 200)
quasirandom_strategy_resampling = NeuralPDE.QuasiRandomTraining(250)

strategies1 = [grid_strategy,quadrature_strategy]

reses_1 = map(strategies1) do strategy_
    println("Neural adapter Poisson equation, strategy: $(nameof(typeof(strategy_)))")
    prob_ = NeuralPDE.neural_adapter(loss,initθ2,pde_system, strategy_)
    res_ = GalacticOptim.solve(prob_, ADAM(0.01); maxiters=8000)
    prob_ = remake(prob_,u0=res_.minimizer)
    res_ = GalacticOptim.solve(prob_, BFGS(); maxiters=200)
end
strategies2 = [stochastic_strategy,quasirandom_strategy]# quasirandom_strategy_resampling]
reses_2 = map(strategies2) do strategy_
    println("Neural adapter Poisson equation, strategy: $(nameof(typeof(strategy_)))")
    prob_ = NeuralPDE.neural_adapter(loss,initθ2,pde_system, strategy_)
    res_ = GalacticOptim.solve(prob_, ADAM(0.01); maxiters=8000)
    prob_ = remake(prob_,u0=res_.minimizer)
    res_ = GalacticOptim.solve(prob_, BFGS(); maxiters=200)
end
reses_ = [reses_1;reses_2;]

discretizations = map(res_ -> NeuralPDE.PhysicsInformedNN(chain2,
                                                          grid_strategy;
                                                          init_params = res_.minimizer),reses_)

probs = map(discret -> NeuralPDE.discretize(pde_system,discret),discretizations)
phis = map(discret -> discret.phi ,discretizations)

xs,ys = [infimum(d.domain):0.01:supremum(d.domain) for d in domains]
analytic_sol_func(x,y) = (sin(pi*x)*sin(pi*y))/(2pi^2)

u_predict = reshape([first(phi([x,y],res.minimizer)) for x in xs for y in ys],(length(xs),length(ys)))

u_predicts = map(zip(phis,reses_)) do (phi_,res_)
     reshape([first(phi_([x,y],res_.minimizer)) for x in xs for y in ys],(length(xs),length(ys)))
end

u_real = reshape([analytic_sol_func(x,y) for x in xs for y in ys], (length(xs),length(ys)))

@test u_predict ≈ u_real rtol = 0.1
@test u_predicts[1] ≈ u_real rtol = 0.2
map(u_predicts[2:end]) do upred
    @test upred ≈ u_real rtol = 0.3
end

#using Plots
# i=3
# diff_u = abs.(u_predict .- u_real)
# diff_u_ = abs.(u_predicts[i] .- u_real)
# p2 = plot(xs, ys, u_predict, linetype=:contourf,title = "predict");
# p1 = plot(xs, ys, u_real, linetype=:contourf,title = "analytic");
# p5 = plot(xs, ys, diff_u,linetype=:contourf,title = "error");
# p3 = plot(xs, ys, u_predicts[i],linetype=:contourf,title = "predict_");
# p6 = plot(xs, ys, diff_u_,linetype=:contourf,title = "error_");
# plot(p2,p1,p5,p3,p6)

## Example, 2D Poisson equation, domain decomposition
println("Example, 2D Poisson equation, domain decomposition")
@parameters x y
@variables u(..)
Dxx = Differential(x)^2
Dyy = Differential(y)^2

eq  = Dxx(u(x,y)) + Dyy(u(x,y)) ~ -sin(pi*x)*sin(pi*y)

bcs = [u(0,y) ~ 0.0, u(1,y) ~ -sin(pi*1)*sin(pi*y),
       u(x,0) ~ 0.0, u(x,1) ~ -sin(pi*x)*sin(pi*1)]

# Space
x_0 = 0.0
x_end = 1.0
x_domain = Interval(x_0, x_end)
y_domain = Interval(0.0, 1.0)
domains = [x ∈ x_domain,
           y ∈ y_domain]

count_decomp = 10

# Neural network
af = Flux.tanh
inner = 12
chains = [FastChain(FastDense(2, inner, af), FastDense(inner, inner, af), FastDense(inner, 1)) for _ in 1:count_decomp]
initθs = map(c -> Float64.(c), DiffEqFlux.initial_params.(chains))

xs_ = infimum(x_domain):1/count_decomp:supremum(x_domain)
xs_domain = [(xs_[i], xs_[i+1]) for i in 1:length(xs_)-1]
domains_map = map(xs_domain) do (xs_dom)
    x_domain_ = Interval(xs_dom...)
    domains_ = [x ∈ x_domain_,
                y ∈ y_domain]
end

analytic_sol_func(x,y) = (sin(pi*x)*sin(pi*y))/(2pi^2)
function create_bcs(x_domain_,phi_bound)
    x_0, x_e =  x_domain_.left, x_domain_.right
    if x_0 == 0.0
        bcs = [u(0,y) ~ 0.0,
               u(x_e,y) ~ analytic_sol_func(x_e,y),
               u(x,0) ~ 0.0,
               u(x,1) ~ -sin(pi*x)*sin(pi*1)]
        return bcs
    end
    bcs = [u(x_0,y) ~ phi_bound(x_0,y),
           u(x_e,y) ~ analytic_sol_func(x_e,y),
           u(x,0) ~ 0.0,
           u(x,1) ~ -sin(pi*x)*sin(pi*1)]
    bcs
end

reses = []
phis = []
pde_system_map = []

for i in 1:count_decomp
    println("decomposition $i")
    domains_ = domains_map[i]
    phi_in(cord) = phis[i-1](cord,reses[i-1].minimizer)
    # phi_bound(x,y) = if (x isa Matrix)  phi_in(vcat(x, fill(y,size(x)))) else  phi_in(vcat(fill(x,size(y)),y)) end
    phi_bound(x,y) = phi_in(vcat(x,y))
    @register_symbolic phi_bound(x,y)
    global phi_bound
    Base.Broadcast.broadcasted(::typeof(phi_bound), x,y) = phi_bound(x,y)
    bcs_ = create_bcs(domains_[1].domain, phi_bound)
    @named pde_system_ = PDESystem(eq, bcs_, domains_, [x, y], [u(x,y)])
    push!(pde_system_map,pde_system_)
    strategy = NeuralPDE.GridTraining([0.1/count_decomp, 0.1])

    discretization = NeuralPDE.PhysicsInformedNN(chains[i], strategy; init_params=initθs[i])

    prob = NeuralPDE.discretize(pde_system_,discretization)
    symprob = NeuralPDE.symbolic_discretize(pde_system_,discretization)
    res_ = GalacticOptim.solve(prob, BFGS(), maxiters=1500)
    @show res_.minimum
    phi = discretization.phi
    push!(reses, res_)
    push!(phis, phi)
end

# function plot_(i)
#     xs, ys = [infimum(d.domain):dx:supremum(d.domain) for (dx,d) in zip([0.001,0.01], domains_map[i])]
#     u_predict = reshape([first(phis[i]([x,y],reses[i].minimizer)) for x in xs for y in ys],(length(xs),length(ys)))
#     u_real = reshape([analytic_sol_func(x,y) for x in xs for y in ys], (length(xs),length(ys)))
#     diff_u = abs.(u_predict .- u_real)
#     p1 = plot(xs, ys, u_real, linetype=:contourf,title = "analytic");
#     p2 = plot(xs, ys, u_predict, linetype=:contourf,title = "predict");
#     p3 = plot(xs, ys, diff_u,linetype=:contourf,title = "error");
#     plot(p1,p2,p3)
# end
# ps =[plot_(i) for i in 1:count_decomp]

function compose_result(dx)
    u_predict_array = Float64[]
    diff_u_array = Float64[]
    ys = infimum(domains[2].domain):dx:supremum(domains[2].domain)
    xs_ = infimum(x_domain):dx:supremum(x_domain)
    xs = collect(xs_)
    function index_of_interval(x_)
        for (i,x_domain) in enumerate(xs_domain)
            if x_<= x_domain[2] && x_>= x_domain[1]
                return i
            end
        end
    end
    for x_ in xs
        i = index_of_interval(x_)
        u_predict_sub = [first(phis[i]([x_,y],reses[i].minimizer)) for y in ys]
        u_real_sub = [analytic_sol_func(x_,y)  for y in ys]
        diff_u_sub = u_predict_sub .- u_real_sub
        append!(u_predict_array,u_predict_sub)
        append!(diff_u_array,diff_u_sub)
    end
    xs,ys = [infimum(d.domain):dx:supremum(d.domain) for d in domains]
    u_predict = reshape(u_predict_array,(length(xs),length(ys)))
    diff_u = reshape(diff_u_array, (length(xs),length(ys)))
    u_predict, diff_u
end
dx= 0.01
u_predict, diff_u = compose_result(dx)

inner_ = 18
af = Flux.tanh
chain2 = FastChain(FastDense(2,inner_,af),
                   FastDense(inner_,inner_,af),
                   FastDense(inner_,inner_,af),
                   FastDense(inner_,inner_,af),
                   FastDense(inner_,1))

initθ2 =Float64.(DiffEqFlux.initial_params(chain2))

@named pde_system = PDESystem(eq, bcs, domains, [x, y], [u(x,y)])

losses = map(1:count_decomp) do i
    loss(cord,θ) = chain2(cord,θ) .- phis[i](cord,reses[i].minimizer)
end

prob_ = NeuralPDE.neural_adapter(losses,initθ2, pde_system_map,NeuralPDE.GridTraining([0.1/count_decomp,0.1]))
res_ = GalacticOptim.solve(prob_, BFGS(); maxiters=2000)
@show res_.minimum
prob_ = NeuralPDE.neural_adapter(losses,res_.minimizer, pde_system_map, NeuralPDE.GridTraining(0.01))
res_ = GalacticOptim.solve(prob_, BFGS(); maxiters=1000)
@show res_.minimum

parameterless_type_θ = DiffEqBase.parameterless_type(initθ2)
phi_ = NeuralPDE.get_phi(chain2,parameterless_type_θ)

xs,ys = [infimum(d.domain):dx:supremum(d.domain) for d in domains]
u_predict_ = reshape([first(phi_([x,y],res_.minimizer)) for x in xs for y in ys],(length(xs),length(ys)))
u_real = reshape([analytic_sol_func(x,y) for x in xs for y in ys], (length(xs),length(ys)))
diff_u_ = u_predict_ .- u_real

@test u_predict ≈ u_real rtol = 0.1
@test u_predict_ ≈ u_real rtol = 0.1

# p1 = plot(xs, ys, u_predict, linetype=:contourf,title = "predict 1");
# p2 = plot(xs, ys, u_predict_,linetype=:contourf,title = "predict 2");
# p3 = plot(xs, ys, u_real, linetype=:contourf,title = "analytic");
# p4 = plot(xs, ys, diff_u,linetype=:contourf,title = "error 1");
# p5 = plot(xs, ys, diff_u_,linetype=:contourf,title = "error 2");
# plot(p1,p2,p3,p4,p5)


function intergrid_1d_interp(N, m)

    ic = Int(ceil(m/2.0))
    Minv = stencil1d(m)
    sten = transpose(Minv) * Float64[(0.5)^(j-1)/factorial(j-1) for j=1:m]

    Ih = zeros(Float64, (N,N))
    Is = I + Ih
    for i=1:m
        k = i - ic
        vec = sten[i] * ones(N - abs(k))
        Ih += diagm(k => vec)
    end

    Iout = zeros(Float64, (2*N,N))
    Iout[1:2:end,:] = Is
    Iout[2:2:end,:] = Ih
    return Iout
end

function intergrid_transforms(Nx0, Ny0, mx, my, P0, P1)

    Ix = intergrid_1d_interp(Nx0, mx)
    Iy = intergrid_1d_interp(Ny0, my)
    Ic = kron(sparse(Iy), sparse(Ix))
    Rc = transpose(Ic) / 4.0

    Restr = P0 * Rc * transpose(P1)
    Interp = P1 * Ic * transpose(P0)
    return Restr, Interp
end

function edge_trim_box(walls::Vector{Wall}, deltas::Vector{Float64}, Nx::Int64, Ny::Int64, bottom_left::Vector{Float64}, top_right::Vector{Float64})

    deltaX = top_right[1] - bottom_left[1]
    deltaY = top_right[2] - bottom_left[2]
    dx = deltaX / Nx
    dy = deltaY / Ny

    points = zeros(Float64, (2, Nx*Ny))
    proj_rows = Int32[i for i=1:Nx*Ny]
    proj_cols = zeros(Int32, Nx*Ny)
    proj_vals = ones(Int32, Nx*Ny)
    nanmask = fill(NaN, (Nx, Ny))
    Nk = 0

    for j=1:Ny
        for i=1:Nx

            x = bottom_left[1] + (i-1)*dx
            y = bottom_left[2] + (j-1)*dx
            if inside(x, y, deltas, walls)
                Nk += 1
                points[:,Nk] = [x, y]
                proj_cols[Nk] = i + Nx*(j-1)
                nanmask[i,j] = 1.0
            end
        end
    end

    P = sparse(proj_rows[1:Nk], proj_cols[1:Nk], proj_vals[1:Nk], Nk, Nx*Ny)
    return points[:,1:Nk], P, nanmask, dx, dy
end

function edge_trim_double(points0::Matrix{Float64}, walls::Vector{Wall}, deltas::Vector{Float64}, dx0::Float64, dy0::Float64, Nx0::Int64, Ny0::Int64, bottom_left::Vector{Float64})

    dx = dx0 / 2.0
    dy = dy0 / 2.0
    Nk0 = size(points0)[2]

    points = zeros(Float64, (2, 4*Nk0))
    proj_rows = Int32[i for i=1:4*Nk0]
    proj_cols = zeros(Int32, 4*Nk0)
    proj_vals = ones(Int32, 4*Nk0)
    nanmask = fill(NaN, (2*Nx0, 2*Ny0))
    Nk = 0

    for l=1:Nk0
        for j=1:2
            for i=1:2
                x = points0[1,l] + (i-1)*dx
                y = points0[2,l] + (j-1)*dy
                deltaX = x - bottom_left[1]
                deltaY = y - bottom_left[2]
                ic = Int(round(deltaX/dx)) + 1
                jc = Int(round(deltaY/dy)) + 1

                if inside(x, y, deltas, walls)
                    Nk += 1
                    points[:,Nk] = [x, y]
                    proj_cols[Nk] = ic + 2*Nx0*(jc-1)
                    nanmask[ic, jc] = 1.0
                end

            end
        end
    end

    P = sparse(proj_rows[1:Nk], proj_cols[1:Nk], proj_vals[1:Nk], Nk, 4*Nx0*Ny0)
    return points[:,1:Nk], P, nanmask, dx, dy
end

#
#
#
#
#
#
# function edge_trim_double(Nx0::Int64, Ny0::Int64, dx0::Float64, dy0::Float64, points0::Matrix{Float64}, proj_cols0::Vector{Int32}, deltas::Vector{Float64}, walls::Vector{Wall})
#
#     dx = dx0 / 2.0
#     dy = dy0 / 2.0
#     Nk0 = size(points0)[2]
#
#     points = zeros(Float64, (2, 4*Nk0))
#     proj_rows = Int32[i for i=1:4*Nk0]
#     proj_cols = zeros(Int32, 4*Nk0)
#     proj_vals = ones(Int32, 4*Nk0)
#     nanmask = fill(NaN, (2*Nx0, 2*Ny0))
#     Nk = 0
#
#     for l=1:Nk0
#         for j=1:2
#             for i=1:2
#                 x = points0[1,l] + (i-1)*dx
#                 y = points0[2,l] + (j-1)*dy
#
#                 i0 = rem(proj_cols0[l], Nx0)
#                 j0 = div(proj_cols0[l], Ny0)
#                 i2 = (i0-1)*2 + i
#                 j2 = (j0-1)*2 + j
#
#                 if inside(x, y, deltas, walls)
#                     Nk += 1
#                     points[:,Nk] = [x, y]
#                     proj_cols[Nk] = i2 + 2*Nx0*(j2-1)
#                     nanmask[i2, j2] = 1.0
#                 end
#
#             end
#         end
#     end
#
#     P = sparse(proj_rows[1:Nk], proj_cols[1:Nk], proj_vals[1:Nk], Nk, 4*Nx0*Ny0)
#     return points[:,1:Nk], P, nanmask, dx, dy
# end

struct Grid

    points::Matrix{Float64}
    origin::Vector{Float64}
    projection::SparseMatrixCSC
    inverse_projection::SparseMatrixCSC
    restrictions::Vector{SparseMatrixCSC}
    interpolations::Vector{SparseMatrixCSC}
    nanmask::Matrix{Float64}
    mx::Int64
    my::Int64
    Nx::Int64
    Ny::Int64
    Nk::Int64
    Mxinv::Matrix{Float64}
    Myinv::Matrix{Float64}
    MxyinvT::Matrix{Float64}
    dx::Float64
    dy::Float64

end

function Grid(walls::Vector{Wall}, deltas::Vector{Float64}, corners::Matrix{Float64}, N_levels::Int64, N0::Vector{Int}, m::Vector{Int})

    Nx, Ny = N0
    mx, my = m

    # trim the bounding box, then double the resolutions and compute Restr and Interp
    points, P0, nanmask, dx, dy = edge_trim_box(walls, deltas, Nx, Ny, corners[:,1], corners[:,2])
    points, P1, nanmask, dx, dy = edge_trim_double(points, walls, deltas/2, dx, dy, Nx, Ny, corners[:,1])
    Restr, Interp = intergrid_transforms(Nx, Ny, mx+1, my+1, P0, P1)
    Nx = Nx * 2
    Ny = Ny * 2

    restrictions = SparseMatrixCSC[Restr]
    interpolations = SparseMatrixCSC[Interp]

    for l=3:N_levels

        P0 = P1
        points, P1, nanmask, dx, dy = edge_trim_double(points, walls, deltas/l, dx, dy, Nx, Ny, corners[:,1])
        Restr, Interp = intergrid_transforms(Nx, Ny, mx+1, my+1, P0, P1)
        prepend!(restrictions, [Restr])
        append!(interpolations, [Interp])
        Nx = Nx * 2
        Ny = Ny * 2
    end

    origin = corners[:,1]
    projection = P1
    inverse_projection = transpose(P1)
    Nk = size(points)[2]
    Mxinv = stencil1d(mx)
    Myinv = stencil1d(my)
    MxyinvT = stencil2d(mx, my)

    return Grid(points, origin, projection, inverse_projection, restrictions, interpolations,
                nanmask, mx, my, Nx, Ny, Nk, Mxinv, Myinv, MxyinvT, dx, dy)
end

function Grid(walls::Vector{Wall}, deltas::Vector{Float64}, corners::Matrix{Float64}, N_levels::Int64, N0::Int64, m::Int64)

    return Grid(walls, deltas, corners, N_levels, [N0, N0], [m, m])
end

function vec_to_mesh(vec, grid::Grid)

    vals = grid.inverse_projection * vec

    Nx = grid.Nx
    Ny = grid.Ny
    out = zeros(Float64, (Nx, Ny))

    for j=1:Ny
        out[:,j] = vals[1+(j-1)*Nx:j*Nx]
    end

    return out .* grid.nanmask
end

function f_to_grid(f::Function, grid::Grid)

    Nk = grid.Nk
    vec = zeros(Float64, Nk)

    for k=1:Nk
        x = grid.points[1,k]
        y = grid.points[2,k]
        vec[k] = f(x, y)
    end

    return vec
end

# overloading Base.map() to operate on the points in the grid, using pmap instead of usual map
function grid_map(f::Function, outsize::Tuple{Int64, Int64}, grd::Grid)

    n, m = outsize
    output = SharedArray{Float64}((n*grd.Nk,m))

    # @distributed unfortunately only works if nprocs > 1
    if nprocs() > 1
        @distributed for i=1:grd.Nk
            k = 1 + n*(i-1)
            output[k:k+n-1,:] = f(grd.points[:,i]..., grd)
        end
    else
        for i=1:grd.Nk
            k = 1 + n*(i-1)
            output[k:k+n-1,:] = f(grd.points[:,i]..., grd)
        end
    end

    return output[:,:]
end


"""
    intensity_sppa(pressures::Array{Unitful.Pressure, N}, medium::Medium, excitation::Excitation)

Spatial Peak, Pulse Averaged intensity [W/cm²]

`pressures` is expected to be an array with up to 4 dimensions. The first dimension is time,
and the remaining dimensions are space
"""
function intensity_sppa(pressures::Array{T, N}, medium, excitation) where {T <: Unitful.Pressure, N}
    # account for sampling frequency by multiplying pressure values with dt
    dt = 1/excitation.fs
    intensities = map(p -> intensity(p, medium), pressures)
    i_sppa = sum(intensities * dt; dims=1) / excitation.pulse_duration
    return format_intensity.(squeeze(i_sppa))
end

"""
    intensity(pressure::Unitful.Pressure, medium::Medium)

Instantaneous intensity [W/cm²]

`pressure` is expected to be a single number
"""
function intensity(pressure::Unitful.Pressure, medium::Medium)
    return format_intensity(pressure^2 / (medium.density * medium.c))
end

format_intensity(input) = uconvert(u"W/cm^2", input)

"""
    intensity_spta(pressures::Array{Unitful.Pressure, N}, medium::Medium, excitation::Excitation)

Spatial Peak, Time Averaged intensity [W/cm²]

`pressures` is expected to be an array with up to 4 dimensions. The first dimension is time,
and the remaining dimensions are space
"""
function intensity_spta(pressures::Array{T, N}, medium::Medium, excitation::Excitation) where {T <: Unitful.Pressure, N}
    return intensity_sppa(pressures, medium, excitation) * excitation.duty_cycle
end

"""
    energy_spta(pressures::Array{Unitful.Pressure, N}, medium::Medium, excitation::Excitation)

Spatial Peak, Time Averaged energy [J/cm²]
"""
function energy_spta(pressures::Array{T, N}, medium::Medium, excitation::Excitation) where {T <: Unitful.Pressure, N}
    return intensity_spta(pressures, medium, excitation) * excitation.total_duration
end

"""
    mechanical_index(pressures::Array{Unitful.Pressure, N}, excitation::Excitation)

Mechanical index [dimensionless]

`pressures` is expected to be an array with up to 4 dimensions. The first dimension is time,
and the remaining dimensions are space
"""
function mechanical_index(pressures::Array{T, N}, excitation::Excitation) where {T <: Unitful.Pressure, N}
    peak_rarefaction_pressure = pn_pressure(uconvert.(u"MPa", pressures))
    center_frequency = sqrt(uconvert(u"MHz", excitation.f0))
    return ustrip(peak_rarefaction_pressure / center_frequency)
end

"""
    pn_pressure(pressures::Array{Unitful.Pressure, N})

Peak Negative pressure [units unchanged]

`pressures` is expected to be an array with up to 4 dimensions. The first dimension is time,
and the remaining dimensions are space
"""
function pn_pressure(pressures::Array{T, N}) where {T <: Unitful.Pressure, N}
    peak_neg_pressure = abs.(minimum(pressures; dims=1))
    return squeeze(peak_neg_pressure)
end

"""
    pp_pressure(pressures::Array{Unitful.Pressure, N})

Peak Positive pressure [units unchanged]

`pressures` is expected to be an array with up to 4 dimensions. The first dimension is time,
and the remaining dimensions are space
"""
function pp_pressure(pressures::Array{T, N}) where {T <: Unitful.Pressure, N}
    peak_pos_pressure = abs.(maximum(pressures; dims=1))
    return squeeze(peak_pos_pressure)
end

"""
    ptp_pressure(pressures::Array{Unitful.Pressure, N})

Peak To Peak pressure [units unchanged]

`pressures` is expected to be an array with up to 4 dimensions. The first dimension is time,
and the remaining dimensions are space
"""
function ptp_pressure(pressures::Array{T, N}) where {T <: Unitful.Pressure, N}
    ptp_pressure = pp_pressure(pressures) .+ pn_pressure(pressures)
    return ptp_pressure
end


a = 0; b = 1
ϵ = 0.1
nₓ = 9; nₜ = 100; t_max = 1
left_bc = PeriodicBC(); right_bc = PeriodicBC()
u₀(x) = 1.

problem = AllenCahnProblem1D(a, b, ϵ, nₓ, nₜ, t_max, left_bc, right_bc, u₀)
assemble_system!(problem)
tsm = BackwardEulerMethod()
t, u = solve(problem, tsm)

error = abs.(ones(length(u[end])) - u[end])
norm(error, Inf) < 1e-14


function a3fn(verts::AbstractVector{<:AbstractPoint{3,T1}},
            triangles::AbstractVector{<:TriangleFace},
            n_samples::Int) where T1 <: AbstractFloat
    bins = calculate_area_bins(verts, triangles)
    angles = Vector{Float64}(undef, n_samples)
    @inbounds for i in 1:n_samples
        points = [point_sampling(verts, triangles, bins) for _ in 1:3]
        v1 = points[2] - points[1]
        v2 = points[3] - points[1]
        angles[i] = angle_between_vector(v1, v2)
    end
    return angles
end
function a3fn(mesh::Mesh, n_samples::Int)
    verts, triangles = decompose_vertex_and_face(mesh)
    return a3fn(verts, triangles, n_samples)
end
function d1fn(verts::AbstractVector{<:AbstractPoint{3,T1}},
            triangles::AbstractVector{<:TriangleFace},
            n_samples::Int) where T1 <: AbstractFloat
    bins = calculate_area_bins(verts, triangles)
    centroid_point = centroid(verts)
    distances = Vector{Float64}(undef, n_samples)
    @inbounds for i in 1:n_samples
        point = point_sampling(verts, triangles, bins)
        distances[i] = euclidean(centroid_point, point)
    end
    return distances
end
function d1fn(mesh::Mesh, n_samples::Int)
    verts, triangles = decompose_vertex_and_face(mesh)
    return d1fn(verts, triangles, n_samples)
end
function d2fn(verts::AbstractVector{<:AbstractPoint{3,T1}},
            triangles::AbstractVector{<:TriangleFace},
            n_samples::Int) where T1 <: AbstractFloat
    bins = calculate_area_bins(verts, triangles)
    distances = Vector{Float64}(undef, n_samples)
    @inbounds for i in 1:n_samples
        points = [point_sampling(verts, triangles, bins) for _ in 1:2]
        distances[i] = euclidean(points[1], points[2])
    end
    return distances
end
function d2fn(mesh::Mesh, n_samples::Int)
    verts, triangles = decompose_vertex_and_face(mesh)
    return d2fn(verts, triangles, n_samples)
end
function d3fn(verts::AbstractVector{<:AbstractPoint{3,T1}},
            triangles::AbstractVector{<:TriangleFace},
            n_samples::Int) where T1 <: AbstractFloat
    bins = calculate_area_bins(verts, triangles)
    sqrt_areas = Vector{Float64}(undef, n_samples)
    @inbounds for i in 1:n_samples
        points = [point_sampling(verts, triangles, bins) for _ in 1:3]
        area = triangle_area(points[1], points[2], points[3])
        sqrt_areas[i] = sqrt(area)
    end
    return sqrt_areas
end
function d3fn(mesh::Mesh, n_samples::Int)
    verts, triangles = decompose_vertex_and_face(mesh)
    return d3fn(verts, triangles, n_samples)
end
function d4fn(verts::AbstractVector{<:AbstractPoint{3,T1}},
            triangles::AbstractVector{<:TriangleFace},
            n_samples::Int) where T1 <: AbstractFloat
    bins = calculate_area_bins(verts, triangles)
    cbrt_vols = Vector{Float64}(undef, n_samples)
    @inbounds for i in 1:n_samples
        points = [point_sampling(verts, triangles, bins) for _ in 1:4]
        vol = tetrahedron_volume(points[1], points[2], points[3], points[4])
        cbrt_vols[i] = cbrt(vol)
    end
    return cbrt_vols
end
function d4fn(mesh::Mesh, n_samples::Int)
    verts, triangles = decompose_vertex_and_face(mesh)
    return d4fn(verts, triangles, n_samples)
end


# LSS solve
using SparseArrays
using BlockArrays
function lsssolve(R, b)
    d_u, n = size(b)
    ndu = n*d_u
    eye = sparse(Matrix([zeros(ndu, d_u) 1.0I(ndu)]))
    D = (BlockArray{Float64}(zeros(ndu,ndu),
               d_u*ones(Int64,n),
               d_u*ones(Int64,n)))
    [D[Block(i,i)] = R[:,:,i+1] for i =1:n-1]
    D = sparse([Array(D) zeros(ndu, d_u)])
    B = D - eye
    BB = B*transpose(B)
	a = -transpose(B)*(BB\b[:])
    a = reshape(a, d_u, n+1)[:,1:end-1]
	return a
end


using Ipopt
using SparseArrays
function solve_ipopt(P::AbstractMatrix, q, A::AbstractMatrix, b,
    r_max=Inf, r_min=-Inf, x_init=zeros(0);
    max_iter=5000, print_level=0)

    n = size(P, 1)
    A_ = [A; ones(n)']
    m = size(A_, 1)

    eval_f(x) = dot(x, Symmetric(P)*x)/2 + dot(q, x)
    eval_grad_f(x, grad_f) = grad_f[:] = P*x + q

    function eval_g(x, g)
        g[1:end-1] = A*x - b
        g[end] = dot(x, x)
    end

    function eval_jac_g(x, mode, rows, cols, values)
        if mode == :Structure
            k = 1
            for j = 1:n, i=1:m rows[k] = i
                cols[k] = j
                k += 1
            end
        else
            A_[end, :] .= 2*x
            values[:] .= reshape(A_, length(A_))
        end
    end

    function eval_h(x, mode, rows, cols, obj_factor, lambda, values)
        if mode == :Structure
            k = 1
            @inbounds for j = 1:n, i=j:n
                rows[k] = i; cols[k] = j
                k += 1
            end
        else
            k = 1
            @inbounds for j=1:n, i=j:n
                values[k] = obj_factor*P[j, i]
                if i == j
                    values[k] += 2*lambda[end]
                end
                k +=1
            end
        end
    end

    x_L = -Inf*ones(n)
    x_U = Inf*ones(n)

    g_L = -Inf*ones(m)
    g_L[end] = r_min^2
    g_U = zeros(m)
    g_U[end] = r_max^2

    nz_P = Int(n*(n + 1)/2)
    nz_A_ = m*n

    prob = createProblem(n, x_L, x_U, m, g_L, g_U, nz_A_, nz_P,
                eval_f, eval_g, eval_grad_f, eval_jac_g, eval_h)
    addOption(prob, "max_iter", max_iter)
    addOption(prob, "print_level", print_level)
    if length(x_init) == n
        prob.x .= x_init
    end

    timing = @elapsed status = solveProblem(prob)
    # println(Ipopt.ApplicationReturnStatus[status])
    # println(prob.obj_val)
    prob.mult_g[end] *= 2
    return prob.x, prob.mult_g, timing
end

function solve_ipopt(P::SparseMatrixCSC, q, A::SparseMatrixCSC, b,
    r_max=Inf, r_min=-Inf, x_init=zeros(0);
    max_iter=5000, print_level=0)

    # Ensure diagonal is present
    n = size(P, 1)
    P = P + 1e-60*I
    P_lower = SparseMatrixCSC(LowerTriangular(P))
    A_ = [A; ones(1, n)]
    m, n = size(A_)

    eval_f(x) = dot(x, P*x)/2 + dot(q, x)
    eval_grad_f(x, grad_f) = grad_f[:] = P*x + q

    function eval_g(x, g)
        # Bad: g    = zeros(2)  # Allocates new array
        # OK:  g[:] = zeros(2)  # Modifies 'in place'
        g[1:end-1] = A*x - b
        g[end] = dot(x, x)
    end

    function eval_jac_g(x, mode, rows, cols, values)
        if mode == :Structure
            rows[:], cols[:] = findnz(A_)
        else
            m, n = size(A_)
            @inbounds for i = 1:n
                A_[end, i] = 2*x[i]
            end
            values[:] .= A_.nzval
        end
    end

    function eval_h(x, mode, rows, cols, obj_factor, lambda, values)
        if mode == :Structure
            rows[:], cols[:] = findnz(P_lower)
        else
            P_ = obj_factor*P_lower + 2*lambda[end]*I
            values[:] .= P_.nzval
        end
    end

    x_L = -Inf*ones(n)
    x_U = Inf*ones(n)

    g_L = -Inf*ones(m)
    g_L[end] = r_min^2
    g_U = zeros(m)
    g_U[end] = r_max^2

    nz_P = nnz(P_lower)
    nz_A_ = nnz(A_)

    prob = createProblem(n, x_L, x_U, m, g_L, g_U, nz_A_, nz_P,
                eval_f, eval_g, eval_grad_f, eval_jac_g, eval_h)
    addOption(prob, "max_iter", max_iter)
    addOption(prob, "print_level", print_level)
    if length(x_init) == n
        prob.x .= x_init
    end
    timing = @elapsed status = solveProblem(prob)

    # println(Ipopt.ApplicationReturnStatus[status])
    # println(prob.obj_val)
    # println(prob.mult_g)
    prob.mult_g[end] *= 2
    return prob.x, prob.mult_g, timing
end


############################################################################################
# 							STATE BOUND CONSTRAINTS 										   #
############################################################################################
"""
	StateBoundConstraint{P,N,T}

Linear bound constraint on states
# Constructors
```julia
StateBoundConstraint(n; x_min, x_max)
```
Any of the bounds can be ±∞. The bound can also be specifed as a single scalar, which applies the bound to all states.
"""
struct StateBoundConstraint{P,N,T} <: TrajectoryOptimization.StateConstraint
	n::Int
	x_max::SVector{N,T}
	x_min::SVector{N,T}
	i_max::Vector{Int}
	i_min::Vector{Int}
	inds::SVector{P,Int}
end

Base.copy(bnd::StateBoundConstraint{P,n,T}) where {P,n,T} =
	StateBoundConstraint(bnd.n, bnd.x_max, bnd.x_min,
		copy(bnd.i_max), copy(bnd.i_min), bnd.inds)

function StateBoundConstraint(n; x_max=Inf*(@SVector ones(n)), x_min=-Inf*(@SVector ones(n)))

	# Check and convert bounds
	x_max, x_min = checkBounds(n, x_max, x_min)

	# Concatenate bounds
	b = [-x_max; x_min]
	inds = findall(isfinite, b)
	inds = SVector{length(inds)}(inds)

	# Get linear indices of 1s of Jacobian
	a_max = findall(isfinite, x_max)
	a_min = findall(isfinite, x_min)
	u = length(a_max)
	l = length(a_min)
	carts_u = [CartesianIndex(i,   j) for (i,j) in enumerate(a_max)]
	carts_l = [CartesianIndex(i+u, j) for (i,j) in enumerate(a_min)]
	∇c = zeros(u+l, n)
	linds_u = LinearIndices(zeros(u+l,n))[carts_u]
	linds_l = LinearIndices(zeros(u+l,n))[carts_l]

	StateBoundConstraint(n, x_max, x_min, linds_u, linds_l, inds)
end

function TrajectoryOptimization.con_label(con::StateBoundConstraint, ind::Int)
	i = con.inds[ind]
	n = state_dim(con)
	if 1 <= i <= n
		return "x max $i"
	elseif n < i <= 2n
		j = i - n
		return "x min $j"
	else
		throw(BoundsError())
	end
end

@inline TrajectoryOptimization.state_dim(con::StateBoundConstraint) = con.n
@inline TrajectoryOptimization.is_bound(::StateBoundConstraint) = true
@inline TrajectoryOptimization.lower_bound(bnd::StateBoundConstraint) = bnd.x_min
@inline TrajectoryOptimization.upper_bound(bnd::StateBoundConstraint) = bnd.x_max
@inline TrajectoryOptimization.sense(::StateBoundConstraint) = Inequality()
@inline Base.length(con::StateBoundConstraint) = length(con.i_max) + length(con.i_min)

function TrajectoryOptimization.primal_bounds!(zL, zU, bnd::StateBoundConstraint)
	for i = 1:length(zL)
		zL[i] = max(bnd.x_min[i], zL[i])
		zU[i] = min(bnd.x_max[i], zU[i])
	end
	return true
end

function TrajectoryOptimization.evaluate(bnd::StateBoundConstraint, z::AbstractKnotPoint)
	[(state(z) - bnd.x_max); (bnd.x_min - state(z))][bnd.inds]
end

function TrajectoryOptimization.jacobian!(∇c, bnd::StateBoundConstraint{U,L}, z::AbstractKnotPoint) where {U,L}
	for i in bnd.i_max
		∇c[i]  = 1
	end
	for i in bnd.i_min
		∇c[i] = -1
	end
	return true
end

TrajectoryOptimization.∇jacobian!(G, con::StateBoundConstraint, z::AbstractKnotPoint, λ::AbstractVector) = true # zeros

function TrajectoryOptimization.change_dimension(con::StateBoundConstraint, n::Int, ix=1:n)
	n0 = con.n
	x_max = fill(Inf,n)
	x_min = fill(Inf,n)
	x_max[ix] = con.x_max[1:n0]
	x_min[ix] = con.x_min[1:n0]
	StateBoundConstraint(n, x_max=x_max, x_min=x_min)
end


module PolynomialOperators
using Polynomials
using LinearOperators
using IterativeSolvers

type PolynomialOperator <: AbstractLinearOperator
    A::AbstractLinearOperator
    p::Poly

    nrow::Integer
    ncol::Integer
    dtype::DataType
    symmetric::Bool
    hermitian::Bool
    prod::Function
    tprod::Nullable{Function}
    ctprod::Nullable{Function}
end

function poly_prod(p::Poly, A::AbstractLinearOperator,
                   v::StridedVector, tmp::StridedVector)
    w = zeros(promote_type(eltype(v), A.dtype), A.ncol)
    tmp[:] = v
    for k = 1:(length(p.a)-1)
        w += p.a[k]*tmp
        tmp[:] = A*tmp[:]
    end
    w += p.a[end]*tmp
    w
end

solve!{T<:Number}(x::StridedVector{T}, A::PolynomialOperator, B::StridedVector{T}) = gmres!(x,A,B)

import Polynomials: polyval
function polyval(p::Poly, A::AbstractLinearOperator, tmp::StridedVector)
    PolynomialOperator(A, p,
                       A.nrow, A.ncol, A.dtype,
                       A.symmetric, A.hermitian,
                       v -> poly_prod(p, A, v, tmp),
                       Nullable{Function}(v -> poly_tprod(p, A, v, tmp)),
                       Nullable{Function}(v -> poly_ctprod(p, A, v, tmp)))
end

import Base: *
(*)(op::PolynomialOperator, M::Vector) = op.prod(M)
(*)(op::PolynomialOperator, M::StridedVecOrMat) = op.prod(M)

export PolynomialOperator, solve!, polyval
end # module


using Distributions
using LinearAlgebra
using StatsBase
using StaticArrays

import LinearAlgebra.isposdef

CovMat = MMatrix{2, 2, Float64, 4}
MeanVec = MVector{2, Float64}

struct MvNormalF
    μ::MeanVec;
    Σ::CovMat;
    MvNormalF(μ::MeanVec, Σ::CovMat=CovMat(diagm(0 => ones(2)))) = new(μ, Σ)
    MvNormalF(μ::T where T <: AbstractArray{Float64, 1}) = MvNormalF(MeanVec(μ))
    MvNormalF(μ::T1 where T1 <: AbstractArray{Float64, 1}, Σ::T2 where T2 <: AbstractArray{Float64, 2}) =
        MvNormalF(MeanVec(μ), CovMat(Σ))
end

function wmean!(μ::MeanVec, values::T where T <: AbstractMatrix{<:Real}, weights::T1 where T1 <: AbstractVector{<:Real})
    @assert length(μ) == size(values, 1)
    μ .= 0.0
    sum_w = 0.0
    for ci in 1:size(values, 2)
        w = fmax(weights[ci], 0.01)
        for ri in 1:length(μ)
            μ[ri] += values[ri, ci] * w
        end
        sum_w += w
    end

    μ ./= sum_w
    return μ
end

@inline @inbounds function log_pdf(d::MvNormalF, x::Float64, y::Float64)::Float64
    std_x = sqrt(d.Σ[1])
    std_y = sqrt(d.Σ[4])
    ρ = fmax(fmin(1 - 1e-2, d.Σ[3] / (std_x * std_y)), -1 + 1e-2)

    ndx = (x - d.μ[1]) / std_x
    ndy = (y - d.μ[2]) / std_y;

    div1 = 2 * (1 - ρ^2)
    log_div2 = log(2 * π * std_x * std_y * sqrt(1 - ρ^2))

    return -(ndx * ndx + ndy * ndy - 2 * ρ * ndx * ndy) / div1 - log_div2
end

pdf(d::MvNormalF, x::Float64, y::Float64) = exp(log_pdf(d, x, y))
shape(d::MvNormalF) = Matrix(d.Σ)

function robust_cov(x::Array{Float64, 2}; prop::Float64=0.1)
    v1 = mad(x[:,1], normalize=true)^2
    v2 = mad(x[:,2], normalize=true)^2

    dev1 = x[:,1] .- trim_mean(x[:,1]; prop=prop)
    dev2 = x[:,2] .- trim_mean(x[:,2]; prop=prop)
    covar = mean(winsor(dev1 .* dev2; prop=prop))

    return [v1 covar; covar v2]
end

estimate_sample_cov(args...; kwargs...) =
    estimate_sample_cov!(CovMat(zeros(2, 2)), args...; kwargs...)

function estimate_sample_cov!(Σ::CovMat, x::T where T <: AbstractMatrix{Float64}, weights::T2 where T2 <: AbstractVector{<:Real}; μ::MeanVec)
    # https://en.wikipedia.org/wiki/Estimation_of_covariance_matrices#Intrinsic_expectation
    Σ .= 0.0
    sum_w = 0.0
    for i in 1:size(x, 2)
        w = fmax(weights[i], 0.01)
        for c in 1:size(Σ, 1)
            for r in 1:size(Σ, 1)
                Σ[r, c] += w * (x[c, i] - μ[c]) * (x[r, i] - μ[r])
            end
        end
        sum_w += w
    end

    Σ ./= sum_w
    return Σ
end

function maximize!(dist::MvNormalF, x::T, confidences::T2 where T2 <: AbstractVector{<:Real})::MvNormalF where T <: AbstractMatrix{Float64}
    if size(x, 2) == 0
        return dist
    end

    wmean!(dist.μ, x, confidences)
    if size(x, 2) <= 2
        return dist
    end

    estimate_sample_cov!(dist.Σ, x, confidences; μ=dist.μ)
    return dist
end

isposdef(A::StaticMatrix) = LinearAlgebra.isposdef(cholesky(A))

function adjust_cov_matrix!(Σ::T; max_iter::Int=100, cov_modifier::Float64=1e-4, tol=1e-10)::T where T <: Union{CovMat, Matrix{Float64}}
    if !issymmetric(Σ)
        error("Non-symmetric matrix: $Σ")
    end

    if isposdef(Σ) && det(Σ) > tol
        return Σ
    end

    for it in 1:max_iter
        if isposdef(Σ) && det(Σ) > tol
            break
        end

        @inbounds Σ[diagind(Σ)] .+= cov_modifier
        cov_modifier *= 1.5
    end

    return Σ
end


include( "testingUtils.jl" )
include( "../src/bootstrapPosteriorEstimation.jl" )
include( "../src/directSolution.jl" )
using DataFrames

println( "running direct solution unit tests" );

predMatTraining, predMatEval, tTraining, tEval, errMatTraining = makeupPredictions();

predMat = convert( Matrix{Float64}, predMatTraining );
t       = Float64.( tTraining );

pUni    = zeros( Float64, 16 );
fill!( pUni, 1 / 16 );

pBayes       = bootstrapPosteriorEstimation( errMatTraining, 100, 10000 );
pDirectMSE   = directOptimNaiveMSE( predMat, t, pBayes );
pDirectHinge = directOptimHinge( predMat, t, pBayes );
yBayes       = predMatEval * pBayes;
yDirectMSE   = predMatEval * pDirectMSE;
yDirectHinge = predMatEval * pDirectHinge;
yUni         = predMatEval * pUni;

performanceBayes       = mean( lossFunctions.MSE( yBayes, tEval ) );
performanceDirectMSE   = mean( lossFunctions.MSE( yDirectMSE, tEval ) );
performanceDirectHinge = mean( lossFunctions.MSE( yDirectHinge, tEval ) );
performanceUni         = mean( lossFunctions.MSE( yUni, tEval ) );

#== test MSE optimization ==#
@test performanceBayes > performanceDirectMSE
@test performanceDirectHinge > performanceDirectMSE
@test performanceUni > performanceDirectMSE

performanceBayes       = mean( lossFunctions.hingeLoss( yBayes, tEval ) );
performanceDirectMSE   = mean( lossFunctions.hingeLoss( yDirectMSE, tEval ) );
performanceDirectHinge = mean( lossFunctions.hingeLoss( yDirectHinge, tEval ) );
performanceUni         = mean( lossFunctions.hingeLoss( yUni, tEval ) );

#== test hingeLoss optimization ==#
@test performanceBayes > performanceDirectHinge
@test performanceDirectMSE > performanceDirectHinge
@test performanceUni > performanceDirectHinge


using Printf
using Statistics
using LinearAlgebra
using MDBM
using PyPlot
pygui(true);
using Interpolations
# using Plots

#creating iteration array
function it(A)
    matrdim=size(A,2)-1
    scaleitp=real(A[:,1])
    knots=(scaleitp,)
    ARe=real(A)
    AIm=imag(A)

    imRe=[interpolate(knots, ARe[:,2],Gridded(Linear()))]
    for j = 3:matrdim+1
        imRetemp=[interpolate(knots, ARe[:,j], Gridded(Linear()))]
        imRe=vcat(imRe,imRetemp)
    end
    imIm=[interpolate(knots, AIm[:,2],Gridded(Linear()))]
    for j = 3:matrdim+1
        imImtemp=[interpolate(knots, AIm[:,j], Gridded(Linear()))]
        imIm=vcat(imIm,imImtemp)
    end
    return(hcat(imRe,imIm))
end

#substitution in interpolating function
function sub(it,t)
    subdim=size(it,1)
    out=zeros(ComplexF64,subdim)
    for j = 1:subdim
        out[j]=it[j,1](t)+it[j,2](t)*im
    end
    return(out)
end

#function for finding the greatest eigenvalue
function normmax(a)
    s=maximum(size(a))
    norma=zeros(s)
    for j = 1:s
        norma[j]=norm(a[j])
    end
    return(a[findmax(norma)[2]])
end

#function for finding the smallest eigenvalue
function normmin(a)
    s=maximum(size(a))
    norma=zeros(s)
    for j = 1:s
        norma[j]=norm(a[j])
    end
    return(a[findmin(norma)[2]])
end

#checking the difference between the greatest and the smallest eigenvalue
function checkconv(a,b,err)
    var=false
    amax=norm(normmax(a))
    amin=norm(normmin(a))
    bmax=norm(normmax(b))
    bmin=norm(normmin(b))
    if abs((amax-amin)-(bmax-bmin))/(amax-amin)<err
        var=true
    end
    return(var)
end

#matrix exponential
function expM(M)
    p=10
    n=size(M)[1]
    Mval=zeros(Float64,n,n)
    for j = 0:p
        Mval=Mval+(M^j)/(factorial(j))
    end
    return(Mval)
end
#matrix coefficients
function A(t,vv)
    Amatr=zeros(Float64,dim,dim)
    #vv[1]=kappa #vv[2]=delta #vv[3]=epsilon #vv[4]=omega
    Amatr=[-vv[1] -(vv[2]+vv[3]*cos(vv[4]*t)); 1 0]
    return(Amatr)
end

function B(t,bv)
    Bmatr=zeros(dim*dim)
    Bmatr=[0 bv; 0 0]
    return(Bmatr)
end

###################### ISIM #######################
function ISIM(kappa1,delta1,epsilon1,b1,tau1,k1,n)
        dt=tau1/(n-1)
        v=[kappa1, delta1, epsilon1, 2*pi/(k1*tau1)]
        eval=zeros(ComplexF64,mult,1)
        S=zeros(ComplexF64,n*dim,mult)
        V=zeros(ComplexF64,n*dim,mult)
        Vjnorm=zeros(ComplexF64,n*dim,mult)
        tvect0=collect(-tau1:dt:tau1*k1+dt/2) #discretized time vector
        sol=zeros(ComplexF64,k1*(n-1),mult*dim) #empty solution vector
        sol00=rand(n,mult*dim)*(-2)+2*ones(n,mult*dim) #S0 inital random array
        solreturn=zeros(ComplexF64,1,1+mult*dim)
        for g = 1:i #iteration
        sol0=hcat(tvect0,vcat(sol00,sol)) #constructing solution array
            #time integration
            for m = 2:dim:(dim*mult-(dim-2)) #choosing "one physical system"
            sol0m=hcat(sol0[:,1],sol0[:,m:m+dim-1])
            #choosing numerical method
            if method == "EE"
                for j = 0:k1*(n-1)-1
                    Y=sol0m[n+j,2:2+(dim-1)]+dt*(A(sol0m[n+j,1],v)*sol0m[n+j,2:2+(dim-1)]+B(sol0m[n+j,1],b1)*sol0m[1+j,2:2+(dim-1)])
                    sol0m[n+j+1,2:2+(dim-1)]=transpose(Y)
                end
            elseif method == "RK4_int"
                for j = 0:k1*(n-1)-1
                    sol0mittau=it(sol0m[j+1:j+3,:])

                    Ytau0=sub(sol0mittau,-tau1+j*dt)
                    Ytau05=sub(sol0mittau,-tau1+(j+0.5)*dt)
                    Ytau1=sub(sol0mittau,-tau1+(j+1)*dt)

                    Y1=sol0m[j+n,2:dim+1]
                    kk1=(dt)*(A(j*dt,v)*Y1+B(j*dt,b1)*Ytau0)
                    kk2=(dt)*(A((j+0.5)*dt,v)*(Y1+kk1/2)+B((j+0.5)*dt,b1)*Ytau05)
                    kk3=(dt)*(A((j+0.5)*dt,v)*(Y1+kk2/2)+B((j+0.5)*dt,b1)*Ytau05)
                    kk4=(dt)*(A((j+1.0)*dt,v)*(Y1+kk3)+B((j+1.0)*dt,b1)*Ytau1)

                    Y=Y1+(kk1 + 2*kk2 + 2*kk3 + kk4)/6

                    sol0m[n+j+1,2:2+(dim-1)]=transpose(Y)
                end
            elseif method == "RK4"
                for j = 0:k1*(n-1)-1
                    Ydtau=0.5*(sol0m[j+1,2:2+(dim-1)]+sol0m[j+2,2:2+(dim-1)])

                    Y1=sol0m[n+j,2:2+(dim-1)]
                    Y2=Y1+(dt/2)*(A(j*dt,v)*Y1+B(j*dt,b1)*sol0m[j+1,2:2+(dim-1)])
                    Y3=Y1+(dt/2)*(A((j+0.5)*dt,v)*Y2+B((j+0.5)*dt,b1)*Ydtau)
                    Y4=Y1+dt*(A((j+1.0)*dt,v)*Y3+B((j+1.0)*dt,b1)*Ydtau)

                    Y=Y1+(dt/6)*((A(j*dt,v)*Y1+B(j*dt,b1)*sol0m[j+1,2:2+(dim-1)])+2*(A((j+0.5)*dt,v)*Y2+B((j+0.5)*dt,b1)*Ydtau)+2*(A((j+0.5)*dt,v)*Y3+B((j+0.5)*dt,b1)*sol0m[j+1,2:2+(dim-1)])+(A((j+1)*dt,v)*Y4+B(j*dt,b1)*sol0m[j+2,2:2+(dim-1)]))

                    sol0m[n+j+1,2:2+(dim-1)]=transpose(Y)
                end
            elseif method == "IE"
                for j = 0:k1*(n-1)-1
                    X=sol0m[j+n,2:2+(dim-1)]
                    Yt=sol0m[j+n,2:2+(dim-1)]
                    Ytau=sol0m[j+1,2:2+(dim-1)]
                    for l=1:4
                        X=X-inv(dt*A(j*dt,v)-Matrix{Float64}(I,dim,dim))*(Yt+dt*(A((j+1+n)*dt,v)*X+B((j+1+n)*dt,b1)*Ytau)-X)
                    end
                    Y=X
                   sol0m[n+j+1,2:2+(dim-1)]=transpose(Y) #filling solution array of the physical system
                end
            elseif method == "RK2"
                for j = 0:k1*(n-1)-1

                    Ydtau=0.5*(sol0m[j+1,2:2+(dim-1)]+sol0m[j+2,2:2+(dim-1)])

                    Y1=sol0m[n+j,2:2+(dim-1)]
                    Y2=Y1+(dt/2)*(A((j+0.5)*dt,v)*Y1+B((j+0.5)*dt,b1)*Ydtau)

                    Y=Y1+dt*(A((j+0.5)*dt,v)*Y2+B((j+0.5)*dt,b1)*Ydtau)

                    sol0m[n+j+1,2:2+(dim-1)]=transpose(Y)
                end
            end

                sol0[:,m:m+dim-1]=sol0m[:,2:2+(dim-1)] #filling solution array
            end
            #solreturn=vcat(solreturn,sol0)
            #retrieving the solution of last and second-to-last periods
            S1=sol0[1:n,2:mult*dim+1]
            V1=sol0[end-(n-1):end,2:mult*dim+1]
            #restructuring the solution
                for p=1:n
                    for s=1:mult
                        for q=1:dim
                            S[1+(p-1)*dim+(q-1),s]=S1[p,1+(s-1)*dim+(q-1)]
                            V[1+(p-1)*dim+(q-1),s]=V1[p,1+(s-1)*dim+(q-1)]
                        end
                    end
                end
            H=pinv(S)*V #pseudo-inverse calculation
            #H=pinv(S,rtol = sqrt(eps(real(float(one(eltype(S)))))))*V
            (evm,VjT)=eigen(H) #eigenvalue calculationV
            Vj=V*VjT #calculating of new set of eigenvectors
            #normalizing the results
            for h = 1:mult
                Vjnorm[:,h]=normalize(Vj[:,h])
            end
            sol00=zeros(ComplexF64,n,mult*dim) #creating new initial solution array
            for p=1:n
                for s=1:mult
                    for q=1:dim
                             sol00[p,1+(s-1)*dim+(q-1)]=Vjnorm[1+dim*(p-1)+(q-1),s]
                    end
                end
            end
            eval=hcat(eval,evm)     #filling up the eigenvalue array
            #checking convergence
            if g>2 && checkconv(eval[:,end],eval[:,end-1],0.01)
                 break
            end
            #if check(evm) print(g-1); break end
        end
        # print(size(eval)[2]-1)
        # return(norm(normmax(eval[:,end]))-1)
        return eval[:,end]
        #return(eval[:,2:end]) #returning the global eigenvalue array
        #return(solreturn)
end

###################### SDM ########################
function SDM(kappa1, delta1, epsilon1, b1, tau1, k1)
        dt=tau1/m #definition of time-step
        m2=trunc(Int,m/2)
        dim2=trunc(Int,dim/2)
        dimg=dim*(m2+1)
        v=[kappa1, delta1, epsilon1, 2*pi/(k1*tau1)]
        P=zeros(Float64,dim,dim*m*k1) #construction of Pi matrices
        for j=1:m*k1
            P[:,1+dim*(j-1):dim*j]=expM(A((j-1)*dt,v)*dt)
        end
        R=zeros(Float64,dim,dim*m*k1) #construction of Ri matrices
        for j=1:m*k1
            R[:,1+dim*(j-1):dim*j]=0.5*((expM(A((j-1)*dt,v)*dt)-Matrix{Float64}(I, dim, dim))*inv(A((j-1)*dt,v))*B((j-1)*dt,b1))
        end
        Zw=zeros(Float64,dimg,dimg*m*k1) #construction of Zi matrices
        for j=1:m*k1
            P1=P[:,1+(j-1)*dim:j*dim]
            R1=R[:,1+(j-1)*dim:j*dim]
            Zw[:,1+(j-1)*dim*(m2+1):j*dim*(m2+1)]=vcat(hcat(P1,zeros(dim,dim*(m2-1)),R1[:,dim2+1:dim],R1[:,dim2+1:dim]),hcat(zeros(m2*dim,dim2),Matrix{Float64}(I, m2*dim,m2*dim),zeros(m2*dim,dim2)))
        end
        #construction of final Z matrix
        Z=Zw[:,1+dimg*(m-2)+m*dimg*(k1-1):dimg*(m-1)+m*dimg*(k1-1)]*Zw[:,1+dimg*(m-1)+m*dimg*(k1-1):dimg*m+m*dimg*(k1-1)]
        for j=1:(m-2)+(k1-1)*m
            Z=Z*Zw[:,1+dimg*(m-2-j)+m*dimg*(k1-1):dimg*(m-1-j)+m*dimg*(k1-1)]
        end
        #evaluating stability based on largest eigenvalue
        return(norm(normmax(eigvals(Z)))-1)
        #return(P)
end

#parameters of the delayed Mathieu equation
dim=2 #DoF of the system (after Cauchy transcription)
const tau=2*pi
const kappa=0.1
const delta=0.5
const epsilon=1.0
const b=0.2
const k=2
#const T=k*tau
#const omega=2*pi/T
#timestep
#dt=tau/(n-1)

n=5
mus_final=ISIM(kappa, 5.0, 1.0, 2.0 , tau, k,n)
n=5000
mus_final=ISIM(kappa, 5.0, 1.0, 2.0 , tau, k,n)
figure(1)
scatter(real.(mus_final),imag.(mus_final))
plot(sin.(0:0.01:2*pi),cos.(0:0.01:2*pi))
for ni in 2:0.5:14
#
###################### ISIM #######################
mult=16 #Ns=dim*mult
# n=30 #timestep number
#EE: Explicit Euler
#RK4: 4th order Runge-Kutta
#RK4_int: 4th order Runge-Kutta with interpolation function
#IE: Implicit Euler
#RK2: 2nd order Runge-Kutta
method="RK4" #choosing numerical simulation method
i=35 #number of iteration

###################### SDM ########################

mus=ISIM(kappa, 5.0, 1.0, 2.0 , tau, k,Int64(floor(2^ni)))
# figure(1)
# scatter(real.(mus),imag.(mus))
# plot(sin.(0:0.01:2*pi),cos.(0:0.01:2*pi))
figure(2)
# scatter(ones(size(mus)).*m,abs.(mus[1]-mus_final[1]))
println(mus[end])
scatter(ones(size(mus[end])).*ni,log(abs(abs.(mus[end])-abs.(mus_final[end]))))
# scatter(ones(size(mus[end])).*ni,abs.(mus[end]))
pause(0.001)
end

#
# ####### Multi-Dimensional Bisection Method #########
#
# function foo(x,y,z)
#     return(ISIM(kappa, x, y, z , tau, k))
# end
#
#
# # ax1=Axis(-1.0:0.5:5,"delta") # initial grid in x direction
# # ax2=Axis(-1.5:0.5:1.5,"b") # initial grid in y direction
# #
# # ax1=Axis(0.5:0.2:2.0,"tau") # initial grid in x direction
# # ax2=Axis(0.05:0.2:1.5,"b") # initial grid in y direction
# ax1=Axis(0.1:1.0:15.0,"delta") # initial grid in x direction
# ax2=Axis(-5.05:0.5:5.5,"epsilon") # initial grid in y direction
# ax3=Axis(-5.05:0.7:5.5,"b") # initial grid in y direction
#
#
# using PyPlot
# pygui(true)
#
# mymdbm=MDBM_Problem(foo,[ax1,ax2,ax3])
#
# for kinter in 1:1
# MDBM.solve!(mymdbm,1)
#
#
# x_eval,y_eval=getevaluatedpoints(mymdbm)
#
# x_sol,y_sol=getinterpolatedsolution(mymdbm)
#
# fig = figure(11);clf()
# PyPlot.scatter(x_eval,y_eval,s=5)
# PyPlot.scatter(x_sol,y_sol)
#
# pause(0.01)
# end
#
#
# x_eval,y_eval=getevaluatedpoints(mymdbm)
# x_sol,y_sol=getinterpolatedsolution(mymdbm)
# 3
# fig = figure(11);clf()
# PyPlot.scatter(x_eval,y_eval,s=5)
# PyPlot.scatter(x_sol,y_sol,s=5)


module MultiAssemblersTests

using Test
using Gridap
using SparseArrays
using SparseMatricesCSR

for T in [SparseMatrixCSC, SparseMatrixCSR, SymSparseMatrixCSR]
    order = 1
    diritag = "boundary"
    model = CartesianDiscreteModel(domain=(0.0,1.0,0.0,1.0), partition=(2,3))
    fespace = H1ConformingFESpace(Float64,model,order,diritag)

    ufun1(x) = x[1] + x[2]
    U1 = TrialFESpace(fespace,ufun1)

    ufun2(x) = x[1] + x[2]
    U2 = TrialFESpace(fespace,ufun2)

    V1 = TestFESpace(fespace)
    V2 = V1

    V = [V1,V2]
    U = [U1,U2]

    assem = SparseMatrixAssembler(T,V,U)

    @test isa(assem,MultiSparseMatrixAssembler)

    a(v,u) = varinner(∇(v),∇(u))

    m(v,u) = varinner(v,u)

    bfun1(x) = 1.0
    b1(v) = varinner(v,CellField(trian,bfun1))

    bfun2(x) = 2.0
    b2(v) = varinner(v,CellField(trian,bfun2))

    trian = Triangulation(model)
    quad = CellQuadrature(trian,degree=2)

    v1 = CellBasis(V1)
    v2 = CellBasis(V2)

    u1 = CellBasis(U1)
    u2 = CellBasis(U2)

    mat11 = integrate(a(v1,u1),trian,quad)
    mat12 = integrate(m(v1,u2),trian,quad)
    mat22 = integrate(m(v2,u2),trian,quad)

    mat = MultiCellMatrix([mat11,mat12,mat22],[(1,1),(1,2),(2,2)])

    vec1 = integrate(b1(v1),trian,quad)
    vec2 = integrate(b2(v2),trian,quad)

    vec = MultiCellVector([vec1,vec2],[(1,),(2,)])

    v = assemble(assem,vec)
    @test v ≈ [0.166666666666, 0.166666666666, 0.3333333333333, 0.333333333333]

    mm = assemble(assem,mat)

    assemble!(mm,assem,mat)

    cellids = IdentityCellNumber(Int,length(vec))

    v2 = assemble(assem,(vec,cellids),(vec,cellids))
    @test v2 ≈ 2*v

    assemble!(v2,assem,(vec,cellids),(vec,cellids))
    @test v2 ≈ 2*v

    mm2 = assemble(assem,(mat,cellids,cellids),(mat,cellids,cellids))
    @test mm2 ≈ 2*mm

    assemble!(mm2,assem,(mat,cellids,cellids),(mat,cellids,cellids))
    @test mm2 ≈ 2*mm
end

end # module MultiAssemblersTests


module FinOptions

## Dependencies

using DataArrays
using DataFrames
using Base.Dates

export

        Black76,
        BlackScholes,
        BlackScholesDP,
        GBlackScholes,
        KirkSpreadOption,
        GeoAveRate,
        GDelta,
        Delta,
        Elasticity,
        Gamma,
        DGammaDVol,
        GammaP,
        DDeltaDVol,
        Vega,
        Vomma,
        VegaP,
        DVegaDVol,
        Theta,
        Rho,
        RhoFut,
        Rho2,
        Carry,
        Speed,
        StrikeDelta,
        SABRVolatility,
        RNDensity,
        PutCallParity,
        VannaVolga,
        SwingOptionPricer,
        StrikeLadder,
        VVParams,
        Aug16, Sep16, Oct16, Nov16, Dec16,
        Jan17, Feb17, Mar17, Apr17, May17, Jun17,
        Jul17, Aug17, Sep17, Oct17, Nov17, Dec17,
        Jan18, Feb18, Mar18, Apr18, May18, Jun18,
        Jul18, Aug18, Sep18, Oct18, Nov18, Dec18,
        Q416, Q117, Q217, Q317, Q417, Q118, Q218, Q318, Q418,
        W16, S17, W17, S18,
        CND

include("Black76.jl")
include("BlackScholes.jl")
include("BlackScholesDP.jl")
include("CND.jl")
#include("CNDEV.jl"),
include("GBlackScholes.jl")
include("GDelta.jl")
include("Greeks.jl")
include("PutCallParity.jl")
include("SABR.jl")
include("VannaVolga.jl")
include("SwingOptionPricer.jl")
include("StrikeLadder.jl")
include("Expiry.jl")

# package code goes here

type VVParams
        putStrike
        ATMStrike
        callStrike
        putVol
        ATMVol
        callVol
end

end # module


s = open("input.txt") do file
    readlines(file);
end

function testInput(inp, dims)
    return reshape(replace(inp, "\n"=>"")|>collect, dims)
end

function GetOccupiedNeighbors(i, j, board)
    count = 0;
    directions = [
        [0, 1], [1, 1], [1, 0], [1, -1], [0, -1], [-1, -1], [-1, 0], [-1, 1]
    ];

    for dir in directions
        # println(dir);
        loc = [i, j] .+ dir;
        while 0 < loc[1] <= size(board, 1) && 0 < loc[2] <= size(board, 2)
            # println(board[loc[1], loc[2]]);
            if board[loc[1], loc[2]] == '#'
                count += 1;
                break;
            elseif board[loc[1], loc[2]] == 'L'
                break;
            end

            loc .+= dir;
        end
    end

    return count;
end

function SimulateSeating(startState)
    simulating = true;
    curState = startState;
    nextState = Array{Char}(undef, 96, 91);
    nextState .= curState;
    lastChanges = -1;
    while simulating
        changes = 0;
        for i = 1:size(startState,1)
            for j = 1:size(startState,2)
                if curState[i, j] == '.'
                    continue
                end

                nCount = GetOccupiedNeighbors(i, j, curState);

                if curState[i, j] == 'L' && nCount == 0
                    nextState[i, j] = '#';
                    changes += 1;
                elseif curState[i, j] == '#' && nCount >= 5
                    nextState[i, j] = 'L';
                    changes += 1;
                else
                    nextState[i,j] = curState[i, j];
                end
            end
        end
        println(changes);

        if all(nextState .== curState)
            simulating = false;
            break;
        end

        curState .= nextState;
    end

    println("=================");
    println(sum(map(x->x=='#', curState)));
end

rowLen = length(s[1]);
seating = vcat(map(collect, s)...);
seatingV = reshape(seating, (rowLen,91));
SimulateSeating(seatingV)


using Test
using MIPVerify
using MIPVerify: UnrestrictedPerturbationFamily, BlurringPerturbationFamily, LInfNormBoundedPerturbationFamily
@isdefined(TestHelpers) || include("../../../TestHelpers.jl")

@testset "Conv + FC + Softmax" begin
    ### Parameters for neural net
    batch = 1
    c1_in_height = 8
    c1_in_width = 8
    p1_stride_height = 2
    p1_stride_width = 2
    p1_strides = (1, p1_stride_height, p1_stride_width, 1)
    p1_height = round(Int, c1_in_height/p1_stride_height, RoundUp)
    p1_width = round(Int, c1_in_width/p1_stride_width, RoundUp)
    c1_in_channels = 1
    c1_filter_height = 2
    c1_filter_width = 2
    c1_out_channels = 2

    l1_height = 5
    l1_width = p1_height*p1_width*c1_out_channels

    l2_height = 3
    l2_width = l1_height

    ### Choosing data to be used
    input = gen_array(
        (batch, c1_in_height, c1_in_width, c1_in_channels),
        0, 1
    )

    c1_kernel = gen_array(
        (c1_filter_height, c1_filter_width, c1_in_channels, c1_out_channels),
        -1, 1
    )
    c1_bias = gen_array((c1_out_channels,), -1, 1)

    l1_kernel = gen_array((l1_width, l1_height), -1, 1)
    l1_bias = gen_array((l1_height,), -1, 1)

    l2_kernel = gen_array((l2_width, l2_height), -1, 1)
    l2_bias = gen_array((l2_height,), -1, 1)

    nn = Sequential(
        [
            Conv2d(c1_kernel, c1_bias), MaxPool(p1_strides), ReLU(),
            Flatten(4),
            Linear(l1_kernel, l1_bias), ReLU(),
            Linear(l2_kernel, l2_bias)
        ],
        "tests.integration.generated_weights.conv+fc+softmax"
    )

    @testset "BlurringPerturbationFamily" begin
        pp_blur = BlurringPerturbationFamily((5, 5))
        # TODO (vtjeng): Add example where blurring perturbation generates non-NaN results
        test_cases = [
            ((2, pp_blur, 1, 0), NaN),
            ((3, pp_blur, 1, 0), NaN),
        ]

        TestHelpers.batch_test_adversarial_example(nn, input, test_cases)
    end

    @testset "UnrestrictedPerturbationFamily" begin
        pp_unrestricted = UnrestrictedPerturbationFamily()
        @testset "Minimizing l1 norm" begin
            test_cases = [
                ((1, pp_unrestricted, 1, 0), 0),
                ((2, pp_unrestricted, 1, 0), 9.2825418),
            ]

            TestHelpers.batch_test_adversarial_example(nn, input, test_cases)
        end

        @testset "Minimizing lInf norm" begin
            test_cases = [
                ((1, pp_unrestricted, Inf, 0), 0),
                ((2, pp_unrestricted, Inf, 0), 0.4347241),
                ((3, pp_unrestricted, Inf, 0), 0.5045819),
            ]

            TestHelpers.batch_test_adversarial_example(nn, input, test_cases)
        end

        @testset "Increasing margin increases required distance" begin
            test_cases = [
                ((2, pp_unrestricted, 1, 0.1), 9.5642628),
            ]

            TestHelpers.batch_test_adversarial_example(nn, input, test_cases)
        end

        @testset "With multiple target labels specified, minimum target label found" begin
            test_cases = [
                (([2, 3], pp_unrestricted, Inf, 0), 0.4347241),
            ]

            TestHelpers.batch_test_adversarial_example(nn, input, test_cases)
        end
    end

    @testset "LInfNormBoundedPerturbationFamily" begin
        test_cases = [
            ((2, LInfNormBoundedPerturbationFamily(0.43), Inf, 0), NaN),  # restricting maximum perturbation to below minimum distance causes optimization problem to be infeasible
            ((2, LInfNormBoundedPerturbationFamily(0.435), Inf, 0), 0.4347241),  # restricting maximum perturbation to above minimum distance does not affect optimal value of problem
        ]

        TestHelpers.batch_test_adversarial_example(nn, input, test_cases)
    end
end