### A Pluto.jl notebook ###
# v0.17.1

using Markdown
using InteractiveUtils

# ╔═╡ 9c2b64cf-c236-46aa-a4d6-02a22c5b7faa
using Distributions

# ╔═╡ c5ef7103-4aca-49dc-b49b-4cf5ac157701
using Random.Random

# ╔═╡ f126a1f8-afc6-4b52-9021-39e946dd0ea2
md"""
# Cross-Entropy Method

STMO Exam Project by **Ceri-Anne Laureyssens**
"""

# ╔═╡ c629d020-2c17-477c-8517-68c39f46fbcb
md"""
### Cross-entropy

As the title states the method has to do with the cross-entropy. It's a metric used to measure the distance between two probability distributions. These two distributions, as you will read later on, are in fact the original distribution ($$f$$) and the optimized distribution based on elite samples ($$g$$). The distance used to define cross-entropy is called the Kullback-Leibler (KL) distance and measures how one probability distribution is different from a second, reference probability distribution. This KL distance or relative entropy can be easily used to derive the cross-entropy itself and is defined as follows:

```math
\begin{equation}
D_{KL}(f,g) = E_f[\log\frac{f(X)}{g(X)}]
= \int_{x\in χ} f(x)\log f(x)dx - \int_{x\in χ} f(x)\log g(x)dx
\end{equation}
```

With $$D$$ being the KL distance, $$E$$ the expectation, and $$X$$ a random variable with support $$χ$$.

Minimizing this KL distance in between distribution $$f$$ and $$g$$ (parameterized by $$θ$$) is equivalent to choosing $$θ$$ that minimizes the cross-entropy:

```math
\begin{equation}
H(f,g) = H(f) + D_{KL}(f,g)
= -E_f[\log g(X)]
= -\int_{x\in χ} f(x)\log g(x|θ)dx
\end{equation}
```

With $$H$$($$f$$) being the entropy of distribution $$f$$.

This assumes that $$f$$ and $$g$$ share the support $$χ$$ and are continuous with respect to $$x$$. The minimization problem becomes the following:

```math
\begin{equation}
minimize_θ - \int_{x\in χ} f(x)\log g(x|θ)dx
\end{equation}
```

Finding this minimum is the goal of the method explained in the next section.
"""

# ╔═╡ 2c923610-52c4-11ec-367b-2565c12740ab
md"""
### Method description and implementation

The cross-entropy or CE method is a Monte Carlo method for importance sampling and optimization.
Monte-Carlo algorithms rely on repeated sampling to obtain numerical results and use randomness as underlying concept to solve the given problem.

The CE method can be used for both combinatorial, such as the travelling salesman problem and the knapsack problem, as continuous problems, with either a static or noisy objective, and is also a valuable asset in rare-event simulation. In the examples called before a brute-force algorithm is not tractable. As such specialized algorithms, like the CE method, can be used because they quickly rule out large parts of the search space.

The simple two-step process involves generating a random data sample according to a specified mechanism and updating the parameters of the mechanism based on the data to produce a better sample in the next iteration as can be seen in the figure underneath.
"""

# ╔═╡ 2588e834-f2ee-41d1-b2c3-dbc43a0cda63
md"""
![](https://github.com/CeriAnneLaureyssens/CrossEntropy.jl/blob/master/notebook/figures/CE_visual.jpg?raw=true)
"""

# ╔═╡ 9b4da26a-6136-4be7-b3d8-ad573a89c7f9
md"""
We start with a specified distribution in the left panel from which a fixed amount of samples are randomly derived. These randomly derived samples can be seen in the middle panel. The cross-entropy method makes use of an objective, or loss (as mentioned further down), function ($$S$$) which minimizes the cross-entropy between the known distribution $$f$$ and a proposal distribution $$g$$ parameterized by $$θ$$.

```math
\begin{equation}
θ_g^* = \text{argmin}_{θ_g} - \int_{x\in χ} I_{(S(x)≥γ)}g(x|θ) \log g(x|θ_g)dx
= \text{argmin}_{θ_g} - E_θ[I_{(S(x)≥γ)} \log g(x|θ_g)]
\end{equation}
```

With $$I$$ being the indicator function, and $$γ$$ a threshold with which elite samples are determined.

The grey dots indicate the elite samples. One becomes the elite samples by sorting the outcome of the objective function (done for every sample in the distribution). These elite samples are then selected as the samples of the new distribution, which can be seen in the third panel, and the whole process, starting from the left panel, is repeated until the distribution is optimized.
"""

# ╔═╡ aba389ff-77b4-4683-9e41-35e8dc429797
md"""
The optimized distribution parameter $θ_g^*$ is estimated iteratively via the algorithm underneath which runs `max_iter` times and is used here for optimizing multivariate time series distributions.

Suppose we have a timeseries `X = {x₁, ..., xₙ}` where each `xᵢ` is a vector of dimension `m`. The `cross_entropy_method` function can handle two different scenarios:

1. The time series is sampled IID from a single distribution `p`: `xᵢ ~ p(x)`. In this case, the distribution is represented as a `Dict{Symbol, Tuple{Sampleable, Int64}}`. The dictionary will contain `m` symbols, one for each variable in the series. The `Sampleable` object represents `p` and the integer is the length of the timeseries (`N`).
2. The time series is sampled from a different distribution at each timestep `pᵢ`: `xᵢ ~ pᵢ(x)`. In this case, the distribution is also represented as a `Dict{Symbol, Tuple{Sampleable, Int64}}`.

The parameters $θ_{iteration'}$ are defined based on the parameter $θ_{iteration}$. The threshold $γ_{iteration}$ becomes smaller than its initial value, artificially making events less rare under $$X$$ ~ $$g$$($$x$$|$θ_{ iteration}$).
"""

# ╔═╡ cb36a92a-0888-4993-ba11-ebbc7b05a3f9
md"""
The `cross_entropy_method` function takes the following input:
- `loss`: The loss or objective function.
- `d_in`: The starting sampling distribution.
- `max_iter`: Maximum number of iterations.
- `N`: Population size (number of samples).
- `elite_thresh`: The threshold below which a sample will be considered elite. To have a fixed number of elite samples set this to `-Inf` and use the `min_elite_samples`.
- `min_elite_samples`: The minimum number of elite samples.
- `max_elite_samples`: The maximum number of allowed elite samples.
- `weight_fn`: A function that specifies the weight of each sample. Use the likelihood ratio when trying to perform importance sampling.
- `rng::AbstractRNG`: The random number generator used.
- `verbose`: Whether or not to print progress.
- `show_progress`: Whether or not to show the progress meter.
- `batched`: Indicates batched loss evaluation (loss function must return an array containing loss values for each sample).
- `add_entropy`: A function that transforms the sampling distribution after fitting. Use it to enforce a maximum level of entropy if converging too quickly.

And provides you with an output `d`, which is the optimized distribution.
"""

# ╔═╡ 4115dcd8-9931-4871-bc32-5e6e08450242
md"""
### Fun little example

Now the CE method itself is implemented we will try it out on the importance sampling technique.

**Importance sampling**

Importance sampling is a general technique for estimating properties of a particular distribution, while only having samples generated from a different distribution than the distribution of interest.

We start off with choosing a base distribution and a starting sampling distribution. Following this a random time series is sampled from the starting distribution.
"""

# ╔═╡ d799b586-f338-4614-9799-89ad9d04e695
md"""
The next two code chunks provide the functions for determining the loss function and determining the likelihood ratio weighting function. In the third code chunk underneath you can see the output of the weighting funtion.
"""

# ╔═╡ d77ca772-6b3b-43fb-a3a1-459e04bc37fc
# Define a loss function that identifies the values of 3,4,5 as those of interest
function l(d, s)
    v = s[:x][1]
    -(v in [3,4,5])
end

# ╔═╡ f9a0105c-b570-46b2-93a6-af1982fd4966
md"""
Now we can run the `cross_entropy_method` function with the determined loss function on the sampling distribution. This will return the optimized distribution and can then be plotted in the second code chunk underneath. The plot contains both the base distribution, from which we started, and the optimal distribution as the `cross_entropy_method` funtion returned. `Random.seed!(1234)` is used so the results stay the same each time the code is run. This is done because the CE method uses randomness as underlying concept and thus might provide slightly different results each time its run (sometimes even a vector with different length in comparison to the base distribution, which will provide problems in the underlying plot).
"""

# ╔═╡ e4cf6503-a9e1-4328-bdd5-d32b4330a6a0
md"""
### References

Kloek, T., and van Dijk, H.K. (1978). *Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo.* Econometrica, 46(1), 1-19.

Moss, R.J. (2020). *Cross-Entropy Variants for Optimization.*

Rubinstein, R.Y., and Kroese, D.P. (2004). *The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning.* Springer-Verlag, New York.

Wikipedia (2021). *Combinatorial optimization.* Checked on the 31 January 2022 via https://en.wikipedia.org/wiki/Combinatorial_optimization
"""

# ╔═╡ 04ef2558-ff4b-4a2d-9f59-2f8b6edde3f0
md"""
# Appendices

Underneath you can find all functions this notebook relies on. Please leave these as they are ☺.
"""

# ╔═╡ abb2b875-0439-49f8-accd-5475ae388a67
function Distributions.logpdf(d::Dict{Symbol, Vector{Sampleable}}, x, i)
    sum([logpdf(d[k][i], x[k][i]) for k in keys(d)])
end

# ╔═╡ dbcf3ddc-f4ad-40ed-8c0f-3d7ace8b5069
function Distributions.logpdf(d::Dict{Symbol, Vector{Sampleable}}, x)
    sum([logpdf(d, x, i) for i=1:length(first(x)[2])])
end

# ╔═╡ aeb1f33e-1b4b-4808-bcee-79e8c5e4438f
function Distributions.logpdf(d::Dict{Symbol, Tuple{Sampleable, Int64}}, x, i)
    sum([logpdf(d[k][1], x[k][i]) for k in keys(d)])
end

# ╔═╡ fcb9f3ce-c23a-41ee-9ad2-eaea4699d5bd
function Distributions.logpdf(d::Dict{Symbol, Tuple{Sampleable, Int64}}, x)
    sum([logpdf(d, x, i) for i=1:length(first(x)[2])])
end

# ╔═╡ c81f3bd0-b576-467c-8621-21e75fa587a5
function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Vector{Sampleable}})
    Dict(k => rand.(Ref(rng), d[k]) for k in keys(d))
end

# ╔═╡ 452fb99f-349e-45b0-bdf8-ada524b6d0db
function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Tuple{Sampleable, Int64}})
    Dict(k => rand(rng, d[k][1], d[k][2]) for k in keys(d))
end

# ╔═╡ 05cf58ae-b9f7-4f69-b7e7-64586f1e27a8
function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Vector{Sampleable}}, N::Int)
    [rand(rng, d) for i=1:N]
end

# ╔═╡ 88ea50a3-02bd-4a90-8d72-cf571273ba7f
function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Tuple{Sampleable, Int64}}, N::Int)
    [rand(rng, d) for i=1:N]
end

# ╔═╡ b0054103-d45f-401c-ba20-7b36729fc53e
begin
	# Actual distibution
	base_dist = Categorical([0.74899, 0.2, 0.05, 0.001, 0.00001])

	# Starting sampling distribution
	is_dist_0 = Dict{Symbol, Vector{Sampleable}}(:x => [Categorical(5)])

	# Sample a random time series from the starting distribution
	rand(is_dist_0)
end

# ╔═╡ 7427a6ac-44f1-4fe6-8d0d-e16088391764
# Define the likelihood ratio weighting function
function w(d, s)
    v = s[:x][1]
    exp(logpdf(base_dist, v) - logpdf(d, s))
end

# ╔═╡ 0160a7ed-7418-4061-bb80-cac0d93c0ac8
# Show what the weighting function does
begin
	s = Dict(:x => [5])
	w(is_dist_0, s)
end

# ╔═╡ 028f3359-5335-4a13-9b3d-f23084087b8e
function Distributions.fit(d::Dict{Symbol, Vector{Sampleable}}, samples, weights; add_entropy = (x) -> x)
    N = length(samples)
    new_d = Dict{Symbol, Vector{Sampleable}}()
    for s in keys(d)
        dtype = typeof(d[s][1])
        m = length(d[s])
        new_d[s] = [add_entropy(fit(dtype, [samples[j][s][i] for j=1:N], weights)) for i=1:m]
    end
    new_d
end

# ╔═╡ 199c2691-aeb2-4d0a-b30e-2dd130493038
function Distributions.fit(d::Dict{Symbol, Tuple{Sampleable, Int64}}, samples, weights; add_entropy = (x)->x)
    N = length(samples)
    new_d = Dict{Symbol, Tuple{Sampleable, Int64}}()
    for s in keys(d)
        dtype = typeof(d[s][1])
        m = d[s][2]
        all_samples = vcat([samples[j][s][:] for j=1:N]...)
        all_weights = vcat([fill(weights[j], length(samples[j][s][:])) for j=1:N]...)
        new_d[s] = (add_entropy(fit(dtype, all_samples, all_weights)), m)
    end
    new_d
end

# ╔═╡ 6259dd44-2c35-4faf-b340-c69070e7fc48
# This version uses a vector of distributions for sampling
# N is the number of samples taken
# m is the length of the vector

# If batched is set to true, loss function must return an array containing loss values for each sample
function cross_entropy_method(loss,
                              d_in;
                              max_iter,
                              N=100,
                              elite_thresh = -0.99,
                              min_elite_samples = floor(Int, 0.1*N),
                              max_elite_samples = typemax(Int64),
                              weight_fn = (d,x) -> 1.,
                              rng::AbstractRNG = Random.GLOBAL_RNG,
                              verbose = false,
                              show_progress = false,
                              batched = false,
                              add_entropy = (x)->x
                             )
    d = deepcopy(d_in)
    show_progress ? progress = Progress(max_iter) : nothing

    for iteration in 1:max_iter
        # Get samples -> Nxm
        samples = rand(rng, d, N)

        # Sort the samples by loss and select elite number
        if batched
            losses = loss(d,samples)
        else
            losses = [loss(d, s) for s in samples]
        end

        order = sortperm(losses)
        losses = losses[order]
        N_elite = losses[end] < elite_thresh ? N : findfirst(losses .> elite_thresh) - 1
        N_elite = min(max(N_elite, min_elite_samples), max_elite_samples)

        verbose && println("iteration ", iteration, " of ", max_iter, " N_elite: ", N_elite)

        # Update based on elite samples
        elite_samples = samples[order[1:N_elite]]
        weights = [weight_fn(d, s) for s in elite_samples]
        if all(abs.(weights) .< 1e8)
            println("Warning: all weights are zero")
        end
        d = fit(d, elite_samples, weights, add_entropy = add_entropy)
        show_progress && next!(progress)
    end
    d
end

# ╔═╡ 06af1708-5e0a-4c5c-a8a5-6d3ba97f2690
begin
	Random.seed!(1234)

	# Run the optimization
	is_dist_opt = cross_entropy_method(l, is_dist_0; max_iter = 4, N=10000, weight_fn = w, verbose = true)

	# See the final distribution
	is_dist_opt[:x][1].p
end

# ╔═╡ 0902be86-d083-4c30-815b-f1aed4e92102
# Plot the base distribution and the optimized distribution
begin
	using Plots
	x = 1:5
	y = [0.74899, 0.2, 0.05, 0.001, 0.00001] # Base distribution
	plot(x, y, title = "Importance sampling", label = "Base distribution")

	z = is_dist_opt[:x][1].p # Optimized distribution
	plot!(x, z, label = "Optimal distribution")
end

# ╔═╡ d60f308c-72c0-4eff-8b21-23ef6803be8d
# Gets a function that adds entropy h to categorical distribution c
function add_categorical_entropy(h::Vector{Float64})
    function add_entropy(c::Categorical)
        p = h
        p[1:length(c.p)] .+= c.p
        Categorical(p ./ sum(p))
    end
end

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[compat]
Distributions = "~0.25.37"
Plots = "~1.25.7"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "af92965fb30777147966f58acb05da51c5616b5f"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.3"

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "4b859a208b2397a7a623a03449e4636bdb17bcf2"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.1+1"

[[ChainRulesCore]]
deps = ["Compat", "LinearAlgebra", "SparseArrays"]
git-tree-sha1 = "d711603452231bad418bd5e0c91f1abd650cba71"
uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
version = "1.11.3"

[[ChangesOfVariables]]
deps = ["ChainRulesCore", "LinearAlgebra", "Test"]
git-tree-sha1 = "bf98fa45a0a4cee295de98d4c1462be26345b9a1"
uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0"
version = "0.1.2"

[[ColorSchemes]]
deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"]
git-tree-sha1 = "6b6f04f93710c71550ec7e16b650c1b9a612d0b6"
uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
version = "3.16.0"

[[ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "44c37b4636bc54afac5c574d2d02b625349d6582"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.41.0"

[[CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[Contour]]
deps = ["StaticArrays"]
git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7"
uuid = "d38c429a-6771-53c6-b99e-75d170b6e991"
version = "0.5.7"

[[DataAPI]]
git-tree-sha1 = "cc70b17275652eb47bc9e5f81635981f13cea5c8"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.9.0"

[[DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "3daef5523dd2e769dad2365274f760ff5f282c7d"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.11"

[[DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[DensityInterface]]
deps = ["InverseFunctions", "Test"]
git-tree-sha1 = "80c3e8639e3353e5d2912fb3a1916b8455e2494b"
uuid = "b429d917-457f-4dbc-8f4c-0cc954292b1d"
version = "0.4.0"

[[Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[Distributions]]
deps = ["ChainRulesCore", "DensityInterface", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsBase", "StatsFuns", "Test"]
git-tree-sha1 = "6a8dc9f82e5ce28279b6e3e2cea9421154f5bd0d"
uuid = "31c24e10-a181-5473-b8eb-7969acd0382f"
version = "0.25.37"

[[DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "b19534d1895d702889b219c382a6e18010797f0b"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.6"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[EarCut_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3f3a2501fa7236e9b911e0f7a588c657e822bb6d"
uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5"
version = "2.2.3+0"

[[Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b3bfd02e98aedfa5cf885665493c5598c350cd2f"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.2.10+0"

[[FFMPEG]]
deps = ["FFMPEG_jll"]
git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8"
uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a"
version = "0.4.1"

[[FFMPEG_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"]
git-tree-sha1 = "d8a578692e3077ac998b50c0217dfd67f21d1e5f"
uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5"
version = "4.4.0+0"

[[FillArrays]]
deps = ["LinearAlgebra", "Random", "SparseArrays", "Statistics"]
git-tree-sha1 = "8756f9935b7ccc9064c6eef0bff0ad643df733a3"
uuid = "1a297f60-69ca-5386-bcde-b61e274b549b"
version = "0.12.7"

[[FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.93+0"

[[Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.4+0"

[[FriBidi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91"
uuid = "559328eb-81f9-559d-9380-de523a88c83c"
version = "1.0.10+0"

[[GLFW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"]
git-tree-sha1 = "0c603255764a1fa0b61752d2bec14cfbd18f7fe8"
uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89"
version = "3.3.5+1"

[[GR]]
deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "RelocatableFolders", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "4a740db447aae0fbeb3ee730de1afbb14ac798a1"
uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71"
version = "0.63.1"

[[GR_jll]]
deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "aa22e1ee9e722f1da183eb33370df4c1aeb6c2cd"
uuid = "d2c73de3-f751-5644-a686-071e5b155ba9"
version = "0.63.1+0"

[[GeometryBasics]]
deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"]
git-tree-sha1 = "58bcdf5ebc057b085e58d95c138725628dd7453c"
uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
version = "0.4.1"

[[Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "a32d672ac2c967f3deb8a81d828afc739c838a06"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+2"

[[Graphite2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "344bf40dcab1073aca04aa0df4fb092f920e4011"
uuid = "3b182d85-2403-5c21-9c21-1e1f0cc25472"
version = "1.3.14+0"

[[Grisu]]
git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2"
uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe"
version = "1.0.2"

[[HTTP]]
deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"]
git-tree-sha1 = "0fa77022fe4b511826b39c894c90daf5fce3334a"
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
version = "0.9.17"

[[HarfBuzz_jll]]
deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "Graphite2_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg"]
git-tree-sha1 = "129acf094d168394e80ee1dc4bc06ec835e510a3"
uuid = "2e76f6c2-a576-52d4-95c1-20adfe4de566"
version = "2.8.1+1"

[[IniFile]]
deps = ["Test"]
git-tree-sha1 = "098e4d2c533924c921f9f9847274f2ad89e018b8"
uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f"
version = "0.5.0"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[InverseFunctions]]
deps = ["Test"]
git-tree-sha1 = "a7254c0acd8e62f1ac75ad24d5db43f5f19f3c65"
uuid = "3587e190-3f89-42d0-90ee-14403ec27112"
version = "0.1.2"

[[IrrationalConstants]]
git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151"
uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
version = "0.1.1"

[[IterTools]]
git-tree-sha1 = "fa6287a4469f5e048d763df38279ee729fbd44e5"
uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
version = "1.4.0"

[[IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.3.0"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "8076680b162ada2a031f707ac7b4953e30667a37"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.2"

[[JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "d735490ac75c5cb9f1b00d8b5509c11984dc6943"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.0+0"

[[LAME_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c"
uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d"
version = "3.100.1+0"

[[LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[LaTeXStrings]]
git-tree-sha1 = "f2355693d6778a178ade15952b7ac47a4ff97996"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.3.0"

[[Latexify]]
deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"]
git-tree-sha1 = "a8f4f279b6fa3c3c4f1adadd78a621b13a506bce"
uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316"
version = "0.15.9"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "0b4a5d71f3e5200a7dff793393e09dfc2d874290"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+1"

[[Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[Libglvnd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"]
git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf"
uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29"
version = "1.3.0+3"

[[Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "340e257aada13f95f98ee352d316c3bed37c8ab9"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+0"

[[Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[LogExpFunctions]]
deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"]
git-tree-sha1 = "e5718a00af0ab9756305a0392832c8952c7426c1"
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
version = "0.3.6"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "3d3e902b31198a27340d0bf00d6ac452866021cf"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.9"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MbedTLS]]
deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"]
git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe"
uuid = "739be429-bea8-5141-9913-cc70e7f3736d"
version = "1.0.3"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Measures]]
git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f"
uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e"
version = "0.3.1"

[[Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.2"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[NaNMath]]
git-tree-sha1 = "f755f36b19a5116bb580de457cda0c140153f283"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "0.3.6"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[Ogg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "887579a3eb005446d514ab7aeac5d1d027658b8f"
uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051"
version = "1.3.5+1"

[[OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"

[[OpenSSL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "648107615c15d4e09f7eca16307bc821c1f718d8"
uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
version = "1.1.13+0"

[[OpenSpecFun_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1"
uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e"
version = "0.5.5+0"

[[Opus_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720"
uuid = "91d4177d-7536-5919-b921-800302f37372"
version = "1.3.2+0"

[[OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[PDMats]]
deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"]
git-tree-sha1 = "ee26b350276c51697c9c2d88a072b339f9f03d73"
uuid = "90014a1f-27ba-587c-ab20-58faa44d9150"
version = "0.11.5"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "92f91ba9e5941fc781fecf5494ac1da87bdac775"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.2.0"

[[Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[PlotThemes]]
deps = ["PlotUtils", "Requires", "Statistics"]
git-tree-sha1 = "a3a964ce9dc7898193536002a6dd892b1b5a6f1d"
uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a"
version = "2.0.1"

[[PlotUtils]]
deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"]
git-tree-sha1 = "6f1b25e8ea06279b5689263cc538f51331d7ca17"
uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043"
version = "1.1.3"

[[Plots]]
deps = ["Base64", "Contour", "Dates", "Downloads", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs", "UnicodeFun", "Unzip"]
git-tree-sha1 = "7e4920a7d4323b8ffc3db184580598450bde8a8e"
uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
version = "1.25.7"

[[Preferences]]
deps = ["TOML"]
git-tree-sha1 = "2cf929d64681236a2e074ffafb8d568733d2e6af"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.3"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[Qt5Base_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"]
git-tree-sha1 = "ad368663a5e20dbb8d6dc2fddeefe4dae0781ae8"
uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1"
version = "5.15.3+0"

[[QuadGK]]
deps = ["DataStructures", "LinearAlgebra"]
git-tree-sha1 = "78aadffb3efd2155af139781b8a8df1ef279ea39"
uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc"
version = "2.4.2"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[RecipesBase]]
git-tree-sha1 = "6bf3f380ff52ce0832ddd3a2a7b9538ed1bcca7d"
uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
version = "1.2.1"

[[RecipesPipeline]]
deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"]
git-tree-sha1 = "37c1631cb3cc36a535105e6d5557864c82cd8c2b"
uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c"
version = "0.5.0"

[[Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[RelocatableFolders]]
deps = ["SHA", "Scratch"]
git-tree-sha1 = "cdbd3b1338c72ce29d9584fdbe9e9b70eeb5adca"
uuid = "05181044-ff0b-4ac5-8273-598c1e38db00"
version = "0.1.3"

[[Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.3.0"

[[Rmath]]
deps = ["Random", "Rmath_jll"]
git-tree-sha1 = "bf3188feca147ce108c76ad82c2792c57abe7b1f"
uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa"
version = "0.7.0"

[[Rmath_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "68db32dff12bb6127bac73c209881191bf0efbb7"
uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f"
version = "0.3.0+0"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Scratch]]
deps = ["Dates"]
git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.1.0"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[Showoff]]
deps = ["Dates", "Grisu"]
git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de"
uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f"
version = "1.0.3"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[SpecialFunctions]]
deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
git-tree-sha1 = "e08890d19787ec25029113e88c34ec20cac1c91e"
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
version = "2.0.0"

[[StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "2884859916598f974858ff01df7dfc6c708dd895"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.3.3"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[StatsAPI]]
git-tree-sha1 = "d88665adc9bcf45903013af0982e2fd05ae3d0a6"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.2.0"

[[StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "51383f2d367eb3b444c961d485c565e4c0cf4ba0"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.14"

[[StatsFuns]]
deps = ["ChainRulesCore", "InverseFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"]
git-tree-sha1 = "bedb3e17cc1d94ce0e6e66d3afa47157978ba404"
uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c"
version = "0.9.14"

[[StructArrays]]
deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"]
git-tree-sha1 = "d21f2c564b21a202f4677c0fba5b5ee431058544"
uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
version = "0.6.4"

[[SuiteSparse]]
deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"]
uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "TableTraits", "Test"]
git-tree-sha1 = "bb1064c9a84c52e277f1096cf41434b675cd368b"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.6.1"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[URIs]]
git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355"
uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
version = "1.3.0"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[UnicodeFun]]
deps = ["REPL"]
git-tree-sha1 = "53915e50200959667e78a92a418594b428dffddf"
uuid = "1cfade01-22cf-5700-b092-accc4b62d6e1"
version = "0.4.1"

[[Unzip]]
git-tree-sha1 = "34db80951901073501137bdbc3d5a8e7bbd06670"
uuid = "41fe7b60-77ed-43a1-b4f0-825fd5a5650d"
version = "0.1.2"

[[Wayland_jll]]
deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23"
uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89"
version = "1.19.0+0"

[[Wayland_protocols_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "66d72dc6fcc86352f01676e8f0f698562e60510f"
uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91"
version = "1.23.0+0"

[[XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[Xorg_libXcursor_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd"
uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724"
version = "1.2.0+4"

[[Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[Xorg_libXfixes_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4"
uuid = "d091e8ba-531a-589c-9de9-94069b037ed8"
version = "5.0.3+4"

[[Xorg_libXi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"]
git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246"
uuid = "a51aa0fd-4e3c-5386-b890-e753decda492"
version = "1.7.10+4"

[[Xorg_libXinerama_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"]
git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123"
uuid = "d1454406-59df-5ea1-beac-c340f2130bc3"
version = "1.1.4+4"

[[Xorg_libXrandr_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631"
uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484"
version = "1.5.2+4"

[[Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[Xorg_libxkbfile_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2"
uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a"
version = "1.1.0+4"

[[Xorg_xcb_util_image_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97"
uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b"
version = "0.4.0+1"

[[Xorg_xcb_util_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"]
git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1"
uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5"
version = "0.4.0+1"

[[Xorg_xcb_util_keysyms_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00"
uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7"
version = "0.4.0+1"

[[Xorg_xcb_util_renderutil_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e"
uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e"
version = "0.3.9+1"

[[Xorg_xcb_util_wm_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67"
uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361"
version = "0.4.1+1"

[[Xorg_xkbcomp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"]
git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b"
uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4"
version = "1.4.2+4"

[[Xorg_xkeyboard_config_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"]
git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d"
uuid = "33bec58e-1273-512f-9401-5d533626f822"
version = "2.27.0+4"

[[Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "cc4bf3fdde8b7e3e9fa0351bdeedba1cf3b7f6e6"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.0+0"

[[libass_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "HarfBuzz_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "5982a94fcba20f02f42ace44b9894ee2b140fe47"
uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0"
version = "0.15.1+0"

[[libfdk_aac_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "daacc84a041563f965be61859a36e17c4e4fcd55"
uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280"
version = "2.0.2+0"

[[libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[libvorbis_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"]
git-tree-sha1 = "b910cb81ef3fe6e78bf6acee440bda86fd6ae00c"
uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a"
version = "1.3.7+1"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

[[x264_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fea590b89e6ec504593146bf8b988b2c00922b2"
uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a"
version = "2021.5.5+0"

[[x265_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ee567a171cce03570d77ad3a43e90218e38937a9"
uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76"
version = "3.5.0+0"

[[xkbcommon_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"]
git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6"
uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd"
version = "0.9.1+5"
"""

# ╔═╡ Cell order:
# ╟─f126a1f8-afc6-4b52-9021-39e946dd0ea2
# ╟─c629d020-2c17-477c-8517-68c39f46fbcb
# ╟─2c923610-52c4-11ec-367b-2565c12740ab
# ╟─2588e834-f2ee-41d1-b2c3-dbc43a0cda63
# ╟─9b4da26a-6136-4be7-b3d8-ad573a89c7f9
# ╟─aba389ff-77b4-4683-9e41-35e8dc429797
# ╟─cb36a92a-0888-4993-ba11-ebbc7b05a3f9
# ╠═6259dd44-2c35-4faf-b340-c69070e7fc48
# ╟─4115dcd8-9931-4871-bc32-5e6e08450242
# ╠═b0054103-d45f-401c-ba20-7b36729fc53e
# ╟─d799b586-f338-4614-9799-89ad9d04e695
# ╠═d77ca772-6b3b-43fb-a3a1-459e04bc37fc
# ╠═7427a6ac-44f1-4fe6-8d0d-e16088391764
# ╠═0160a7ed-7418-4061-bb80-cac0d93c0ac8
# ╟─f9a0105c-b570-46b2-93a6-af1982fd4966
# ╠═06af1708-5e0a-4c5c-a8a5-6d3ba97f2690
# ╠═0902be86-d083-4c30-815b-f1aed4e92102
# ╟─e4cf6503-a9e1-4328-bdd5-d32b4330a6a0
# ╟─04ef2558-ff4b-4a2d-9f59-2f8b6edde3f0
# ╠═9c2b64cf-c236-46aa-a4d6-02a22c5b7faa
# ╠═abb2b875-0439-49f8-accd-5475ae388a67
# ╠═dbcf3ddc-f4ad-40ed-8c0f-3d7ace8b5069
# ╠═aeb1f33e-1b4b-4808-bcee-79e8c5e4438f
# ╠═fcb9f3ce-c23a-41ee-9ad2-eaea4699d5bd
# ╠═c5ef7103-4aca-49dc-b49b-4cf5ac157701
# ╠═c81f3bd0-b576-467c-8621-21e75fa587a5
# ╠═452fb99f-349e-45b0-bdf8-ada524b6d0db
# ╠═05cf58ae-b9f7-4f69-b7e7-64586f1e27a8
# ╠═88ea50a3-02bd-4a90-8d72-cf571273ba7f
# ╠═028f3359-5335-4a13-9b3d-f23084087b8e
# ╠═199c2691-aeb2-4d0a-b30e-2dd130493038
# ╠═d60f308c-72c0-4eff-8b21-23ef6803be8d
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


""" Mesh Tools
This file includes tools for importing delta sets from mesh
files supported by MeshIO and for converting delta sets to
meshes (for the purposes of plotting.

Meshes here are stored in the GeometryBasics.Mesh object.
"""
module MeshInterop
export EmbeddedDeltaSet2D, Mesh

using FileIO, MeshIO, GeometryBasics
import GeometryBasics: Mesh

using Catlab.CategoricalAlgebra: copy_parts!
using ..SimplicialSets, ..DualSimplicialSets
import ..SimplicialSets: EmbeddedDeltaSet2D

# Helper Functions (should these be exposed?)
##################

""" Construct a GeometryBasics.Mesh object from an embedded delta set
"""
function Mesh(ds::EmbeddedDeltaSet2D)
  points = Point{3, Float64}[point(ds)...]
  tris = TriangleFace{Int}[zip(triangle_vertices(ds)...)...]
  GeometryBasics.Mesh(points, tris)
end

""" Construct a GeometryBasics.Mesh object from a dual embedded delta set
"""
function Mesh(ds::EmbeddedDeltaDualComplex2D{O,R,P}; primal=false) where {O,R,P}
  if(primal)
    ds′ = EmbeddedDeltaSet2D{O,P}()
    copy_parts!(ds′, ds)
    Mesh(ds′)
  else
    points = Point{3, Float64}[dual_point(ds)...]
    tris = TriangleFace{Int}[zip(dual_triangle_vertices(ds)...)...]
    GeometryBasics.Mesh(points, tris)
  end
end

# Import Tooling
################

""" Constructor for EmbeddedDeltaSet2D from GeometryBasics.Mesh object

This operator should work for any triangular mesh object. Note that it will
not preserve any normal, texture, or other attributes from the Mesh object.
"""
function EmbeddedDeltaSet2D(m::GeometryBasics.Mesh)
  coords = coordinates(m)
  tris = faces(m)
  s = EmbeddedDeltaSet2D{Bool, eltype(coords)}()
  add_vertices!(s, length(coords), point=coords)
  for tri in tris
    tri = convert.(Int64, tri)
    glue_sorted_triangle!(s, tri...)
  end
  # Properly orient the delta set
  orient!(s)
  s
end

""" Constructor for EmbeddedDeltaSet2D from mesh file

This operator should work for any file support for import from MeshIO. Note
that it will not preserve any normal, texture, or other data beyond points and
triangles from the mesh file.
"""
function EmbeddedDeltaSet2D(fn::String)
	EmbeddedDeltaSet2D(FileIO.load(fn))
end
end


@testset "Functions to extend" begin

    @testset "predicates" begin
        @test GroupsCore.isabelian isa Function
        @test GroupsCore.issolvable isa Function
        @test GroupsCore.isnilpotent isa Function
        @test GroupsCore.isperfect isa Function
    end

    @testset "generic constructions" begin
        @test GroupsCore.derivedsubgroup isa Function
        @test GroupsCore.center isa Function
        @test GroupsCore.socle isa Function
        @test GroupsCore.sylowsubgroup isa Function
    end

    @testset "general subgroups" begin
        @test GroupsCore.centralizer isa Function
        @test GroupsCore.normalizer isa Function
        @test GroupsCore.stabilizer isa Function
    end
end


export E
export b00, b01, b10, b11, plus, minus, iplus, iminus

"""
    E(n::Integer, m::Integer=n, a::Integer, b::Integer)

Computational-basis vectors of `n`x`m` matrices.
"""
function E(n::Integer, m::Integer, a::Integer, b::Integer)
    (1 <= a <= n && 1 <= b <= m) || error("E: Invalid indices, violate 1 <= $a <= $n && 1 <= $b <= $m")
    Em = zeros(Int8,  n, m)
    Em[a, b] = 1
    return Em
end
E(n::Integer, a::Integer, b::Integer) = E(n, n, a, b)

####
#### Bell States
####

"""
    b00, b01, b10, b11

Bell states.
"""
const b00 = (ket(0, 0) + ket(1, 1)) / sqrt(2)
const b10 = (ket(0, 0) - ket(1, 1)) / sqrt(2)
const b01 = (ket(0, 1) + ket(1, 0)) / sqrt(2)
const b11 = (ket(0, 1) - ket(1, 0)) / sqrt(2)
@doc (@doc b00) b01
@doc (@doc b00) b10
@doc (@doc b00) b11

####
#### Plus and minus states
####

"""
    plus, minux, iplus, iminus

The states |+⟩, |-⟩, |+i⟩, |-i⟩. These are eigenstates of
the Pauli X and Y operators.
"""
const plus = (ket(0) + ket(1)) / sqrt(2)
const minus = (ket(0) - ket(1)) / sqrt(2)
const iplus = (ket(0) + im * ket(1)) / sqrt(2)
const iminus = (ket(0) - im * ket(1)) / sqrt(2)
@doc (@doc plus) minus
@doc (@doc plus) iplus
@doc (@doc plus) iminus


using IntervalArithmetic
using RevisedAffineArithmetic
using StaticArrays
using Test

const RAA = RevisedAffineArithmetic

include("test_affineform.jl")
include("test_arithmetic.jl")
include("test_conversions.jl")
include("test_vars.jl")


input = readlines("day13/input.txt")
# input = readlines("day13/small.txt")
depart_time = parse(Int, input[1])
buses = convert.(String, split(input[2], ","))

min_mod, min_bus = 10000000000, 0
for bus in buses
    global min_mod,min_bus
    if tryparse(Int, bus) !== nothing
        minimize = parse(Int, bus) - mod(depart_time, parse(Int, bus))
        if minimize < min_mod
            min_mod = minimize
            min_bus = parse(Int, bus)
        end
    end
end

println(min_mod)
println(min_bus)

println(min_mod*min_bus)


module Test_bouncingBall_no_additional_values_at_events

#=
SavedValues not stored at events #557

https://github.com/JuliaDiffEq/DifferentialEquations.jl/issues/557
=#

using DifferentialEquations, PyPlot

function f(du,u,p,t)
  du[1] = u[2]
  du[2] = -9.81
end

function outputs(u, t, integrator) # called at communication points
    return 0.5*u[1]
end

function condition(u,t,integrator) # Event when event_f(u,t) == 0
 z = u[1] + 1e-12
 return z
end

function affect_neg(integrator)
  integrator.u[2] = -0.8*integrator.u[2]
end

u0 = [1.0,0.0]
tspan = (0.0,3.0)

t_start = 0.0
t_inc   = 0.01
t_end   = 3.0
tspan   = (t_start, t_end)
tspan2  = t_start:t_inc:t_end

saved_values=SavedValues(Float64, Float64)
cb1 = SavingCallback(outputs, saved_values, saveat=tspan2)
cb2 = ContinuousCallback(condition,nothing,affect_neg! = affect_neg)

prob = ODEProblem(f,u0,tspan)
sol = solve(prob,Tsit5(),saveat=t_inc,callback=CallbackSet(cb1,cb2))

figure(1)
clf()
plot(sol.t, getindex.(sol.u, 1), label="h")
plot(saved_values.t, saved_values.saveval, label="outputs")
grid(true)
legend()

println("length(sol.t)          = ", length(sol.t))
println("length(saved_values.t) = ", length(saved_values.t))
end


"""
  Alternative calculation of incoherence using
"""
function pointspreadfunctionAlt(A::AbstractLinearOperator,shape::Tuple)

  tres = zeros(ComplexF64,prod(shape))
  unitvec_i = zeros(ComplexF64,prod(shape))
  unitvec_i[floor(Int64,prod(shape)/2 - shape[1]/2) ] = 1.

  println(floor(Int64,prod(shape)/2))
  unitvec_j = zeros(ComplexF64,prod(shape))

    B = zeros(ComplexF64,prod(shape), prod(shape) )
  for j=1:prod(shape)
    unitvec_j[j] = 1.
    uno = A * unitvec_j
    B[:,j] = uno
  end

  C = B'*B
  D = 1 ./diag(C)
  C = diagm(D) * B
  return C

end

function pointspreadfunction(A::AbstractLinearOperator
                            , shape::Tuple
                            ; verbose::Bool=true)
    numOfPixel = prod(shape)
    unitvec_i = zeros(ComplexF64,numOfPixel)
    unitvec_j = zeros(ComplexF64,numOfPixel)
    psfunc = zeros(ComplexF64,numOfPixel,numOfPixel)

    if verbose == true
        progr = Progress(numOfPixel^2+1,dt=0.1,desc="Calc PSF...";barglyphs=BarGlyphs("[=> ]"),barlen=50);
    end

    for i=1:numOfPixel
        unitvec_i = zeros(ComplexF64,numOfPixel)
        unitvec_i[i] = 1
        tmp = (vec(A\ (A.density .* (A * unitvec_i))) )
        for j=1:numOfPixel
            unitvec_j = zeros(ComplexF64,numOfPixel)
            unitvec_j[j] = 1
            psfunc[i,j] = dot(unitvec_j,tmp)
            if verbose == true
                next!(progr)
            end
        end
    end
    peak = maximum(abs(psfunc))
    println(peak)
    for i=1:numOfPixel
        psfunc[i,i] = 0.0
    end
    sidelobe = maximum(abs(psfunc))
    println("sidelobe to peak ration ",sidelobe/peak)
    return psfunc
end

"""
  Direct calculation of the point spread function
"""
function pointspreadfunction2(A::AbstractLinearOperator,shape::Tuple)
  numOfPixel = prod(shape)
  tres = zeros(ComplexF64,prod(shape))
  unitvec_i = zeros(ComplexF64,prod(shape))
  unitvec_i[floor(Int64,prod(shape)/2 - shape[1]/2) ] = 1.

  println(floor(Int64,prod(shape)/2))
  unitvec_j = zeros(ComplexF64,prod(shape))

  for j=1:prod(shape)
    unitvec_j[j] = 1.
    uno = A * unitvec_j
    unitvec_j[j] = 0.
    unouno = A\(uno)
    tres[j] = dot(unitvec_i , vec(unouno))
    unitvec_j[j] = 0.
  end

  backup = tres[floor(Int64,prod(shape)/2 - shape[1]/2)]
  tres[floor(Int64,prod(shape)/2 - shape[1]/2)] = 0.
  println("sidelobe-to-peak ratio:",maximum(abs(tres/backup)))
  tres[floor(Int64,prod(shape)/2 - shape[1]/2) ] = backup
  return reshape(tres,shape)
end


using LinearAlgebra

using DelimitedFiles
fdata = "C:\\Users\\jcai.AUTH\\Downloads\\SCODE-master\\SCODE-master\\data2\\data.txt" # ARGS[1]
ftime = "C:\\Users\\jcai.AUTH\\Downloads\\SCODE-master\\SCODE-master\\data2\\time.txt" # ARGS[2]
X=readdlm(fdata,'\t')
pseudotime=readdlm(ftime,'\t')[:,2]
pseudotime = pseudotime/maximum(pseudotime)

cnum=length(pseudotime)
tfnum = 100   #parse(Int32,ARGS[4])
pnum = 4      #parse(Int32,ARGS[5])
maxite = 10

maxB = 2.0
minB = -10.0

W = zeros(tfnum, pnum)
Z = zeros(pnum, cnum)
WZ = zeros(tfnum, cnum)

new_B = zeros(pnum)
old_B = zeros(pnum)

#initialization
RSS = Inf
for i in 1:pnum
    new_B[i] = rand()*(maxB-minB)+minB
    old_B[i] = new_B[i]
end

function sample_Z()
    for i in 1:pnum
        for j in 1:cnum
            Z[i,j] = exp(new_B[i]*pseudotime[j]) + (rand()*0.002-0.001)
        end
    end
end

#optimization
for ite in 1:maxite
    #sampling B
    target = rand(1:pnum)
    global new_B[target] = rand()*(maxB-minB)+minB

    #for last calc
    if ite==maxite
        global new_B = copy(old_B)
    end

    #sample Z from new B
    sample_Z()

    #regression
    for i in 1:tfnum
        W[i,:] = (Z * Z')^-1 * (Z * X[i,:])
    end
    #prediction
    WZ = W * Z

    #RSS
    tmp_RSS = 0
    for i in 1:tfnum
        tmp_RSS += sum((X[i,:]-WZ[i,:]).*(X[i,:]-WZ[i,:]))
    end

    if tmp_RSS < RSS
        global RSS = tmp_RSS
        global old_B[target] = new_B[target]
    else
        global new_B[target] = old_B[target]
    end
end

#open("RSS.txt", "w") do f
#    println(f, RSS)
#end

#infer A
B = zeros(pnum, pnum)
for i in 1:pnum
    B[i,i] = new_B[i]
end
invW = pinv(W)
A = W * B * invW

#writedlm("A.txt",A)
#writedlm("B.txt",B)


# Density test for manipulation in the running variable.
# This work is based on McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of econometrics, 142(2), 698-714.

"""
    McCraryTest(bin::Union{Real,Nothing} = nothing, bw::Union{Real,Nothing} = nothing)

A McCrary (2008) test for manipulation in the running variable for `SharpRD` estimator.

The `bin` object is the width of each bin. The default is 2*sd(runvar)*length(runvar)^(-0.5), following McCrary 2008.

The `bw` object is the bandwidth. The default uses the calculation from McCrary (2008), pg. 705.
"""
Base.@kwdef struct McCraryTest{K}
    bin::Union{Real,Nothing} = nothing
    bw::Union{Real,Nothing} = nothing
    kernel::K = SymTriangularDist()
end

Base.@kwdef struct FittedMcCraryTest{T,M,V,C}
    b::T
    h::T
    θhat::T
    σθ::T
    pval::T
    Js::M
    fitted_res::V
    coeftable::C
end

# 1. Descriptive Statistics

function descriptive_stats(RunVar::RunningVariable)
    n = Base.length(RunVar)
    sd = StatsBase.std(RunVar)
    rmax = Base.maximum(RunVar)
    rmin = Base.minimum(RunVar)
    return n, sd, rmax, rmin
end

# 2. Binsize (uses descriptive_stats)
# function Binsize(n, sd, bin::Union{Real,Nothing}=nothing)
function bin_width(n, sd)
    b = 2 * sd * n^(-.5)
end

function bin_width(Zs::RunningVariable)
    n, sd, rmax, rmin = descriptive_stats(Zs)
    return b = 2 * sd * (n)^(-.5)
end

# 3. Number of bins
function bin_number(c, b, rmin, rmax)
    Jl = convert(Int64, round(floor((c - rmin) / b) + 1))
    J = convert(Int64, floor((rmax - rmin) / b) + 2)
    Jr = J - Jl
    return J, Jl, Jr
end

function bin_number(Zs::RunningVariable)
    n, sd, rmax, rmin = descriptive_stats(Zs)
    b = bin_width(Zs)
    c = Zs.cutoff
    Jl = convert(Int64, round(floor((c - rmin) / b) + 1))
    J = convert(Int64, floor((rmax - rmin) / b) + 2)
    Jr = J - Jl
    return J, Jl, Jr
end

function bin_number(method::McCraryTest, Zs::RunningVariable)
    n, sd, rmax, rmin = descriptive_stats(Zs)

    if method.bin === nothing
        b = bin_width(Zs)
    else
        b = method.bin
    end
    c = Zs.cutoff
    Jl = convert(Int64, round(floor((c - rmin) / b) + 1))
    J = convert(Int64, floor((rmax - rmin) / b) + 2)
    Jr = J - Jl
    return J, Jl, Jr
end

# 4. Kernel Constant for McCrary
### According to footnote 18 of McCrary (2008), this constant cannot be computed by the standard formula in the book (Eq 4.3) because is a boundary case. What I understand is that the constant is calculted from 3.20 and 3.22 of Fan and Gijbels.

function kernel_constant(method::McCraryTest)
    # Here I need to see how to calculate this. For now its output is just for the Triangular Kernel
    # kernel = EquivalentKernel(method.kernel)

    κ = 3.348
end

# 5. Bandwidth, needs (4), (3), (2), (1)
# function bandwidth(mccrary::McCrary, kernel::SymTriangularDist, X, Y, Jl, c)

function bandwidth(method::McCraryTest, Zs::RunningVariable)
        # I will proceed to estimate the optimal bandwidth using McCrary (2008) optimal method.
    κ = kernel_constant(method)
    c = Zs.cutoff

    # Discretized Running Variable
    n, sd, rmin, rmax = descriptive_stats(Zs)

    #### NEED TO ADAPT THIS
    if method.bin === nothing
        b = bin_width(Zs)
    else
        b = method.bin
    end

    J, Jl, Jr = bin_number(method, Zs)

    DRV = DiscretizedRunningVariable(Zs, J + 1, b)
    X = collect(DRV)
    Y = DRV.weights / (n * b)

    # Left part
    ldf = DataFrame(Y=Y[1:Jl], X=X[1:Jl])
    lols = lm(@formula(Y ~ X + X^2 + X^3 + X^4 ), ldf)
    lcoef = coef(lols)
    lmse =  deviance(lols) / dof_residual(lols)

    lf = 2 * lcoef[3] .+ 6 * lcoef[4] * ldf.X + 12 * lcoef[5] * ldf.X.^2
    lh = κ * (lmse * (c - ldf.X[1]) / sum(lf'lf))^(0.2)

    # Right part
    rdf = DataFrame(Y=Y[Jl + 1:end], X=X[Jl + 1:end])
    rols = lm(@formula(Y ~ X + X^2 + X^3 + X^4 ), rdf)
    rcoef = coef(rols)
    rmse =  deviance(rols) / dof_residual(rols)

    rf = 2 * rcoef[3] .+ 6 * rcoef[4] * rdf.X + 12 * rcoef[5] * rdf.X.^2
    rh = κ * (rmse * (rdf.X[end] - c) / sum(rf'rf))^(0.2)

    # Taking the average
    h = 0.5 * (rh + lh)
end

# 6. McCraryTest
"""
    fit(method::McCraryTest, data::RunningVariable)

Test for manipulation in the running variable following McCrary (2008).

# Arguments

- `data::RunningVariable`: The running variable.

# Returns
Returns  a FittedMcCraryTest object with fields `b` and `h` which are the bin size and the bandwidth used, respectively. Also, it returns `θhat`, `σθ`, `pval`, `Js`, and `fitted_res` which are the estimated difference in the intercepts, the standard error of the estimate, the pval of the test H0: θhat = 0, the number of bins used, and the fitted values for the support of the running variable (as a DataFrame). Finally, `coeftable` provides an object of type CoefTable for the estimator.
"""
function fit(method::McCraryTest, Zs::RunningVariable)

    # Cutoff selection
    c = Zs.cutoff
    R = Zs
    kernel(t) = pdf(method.kernel, t)

    # Call the functions
    # 1.
    n, sd, rmax, rmin = descriptive_stats(Zs)

    # 2.
    if method.bin === nothing
        b = bin_width(Zs)
    else
        b = method.bin
    end

    # 3.
    J, Jl, Jr = bin_number(method, Zs)
    Js = [J; Jl; Jr]

    # 4.
    DRV = DiscretizedRunningVariable(Zs, J + 1, b)
    X = collect(DRV)
    Y = DRV.weights / (n * b)

    # 5.
    if method.bw === nothing
        h = bandwidth(method, Zs)
    else
        h = method.bw
    end

    ##############################################################
    # Smooth histograms to the left and to the right of the cutoff
    ldf = DataFrame(Y=Y[1:Jl], X=X[1:Jl], pred=zeros(Jl))
    rdf = DataFrame(Y=Y[Jl + 1:end], X=X[Jl + 1:end], pred=zeros(Jr))

    # Left side
    Sminus(k) = sum(kernel.((ldf.X .- c) ./ h) .* (ldf.X .- c).^k)
    lfhat = sum(kernel.((ldf.X .- c) ./ h) .* ((Sminus(2) .- Sminus(1) .* (ldf.X .- c)) ./ (Sminus(2) * Sminus(0) - (Sminus(1))^2 )) .* ldf.Y)

    # Right side
    Splus(k) = sum(kernel.((rdf.X .- c) ./ h) .* (rdf.X .- c).^k)
    rfhat = sum(kernel.((rdf.X .- c) ./ h) .* ((Splus(2) .- Splus(1) .* (rdf.X .- c)) ./ (Splus(2) * Splus(0) - (Splus(1))^2)) .* rdf.Y)

    # The estimates:
    θhat = log(rfhat) - log(lfhat)
    σθ = sqrt((1 / (n * h)) * (24 / 5) * ( (1 / rfhat) + (1 / lfhat)))
    z = θhat / σθ
    pval = 2(1 -  cdf(Normal(), abs(z)))

    # Creating the table
    res = [θhat σθ z pval]
    colnms = ["θ̂"; "σ̂"; "z"; "p-val"]
    rownms = ["McCrary Test"]
    coeftbl = CoefTable(res, colnms, rownms, 4, 3)

    # Fitted Values:
        # Left side
    for i in eachindex(ldf.X)
        ldf[!,:dist] = ldf.X  .-  ldf.X[i]
        ldf.pred[i] = predict(lm(@formula(Y ~ dist), ldf, wts=kernel.(ldf.dist / h)))[i]
    end

    m = min.((c .- ldf.X) / h, 1)

    lVarf = (12 / (5 * n * h)) .* ldf.Y .* (2.0 .+ 3 * m.^11 .- 24 * m.^10 .+ 83 * m.^9 .- 72 * m.^8 .- 42 * m.^7 .+ 18 * m.^6 .+ 18 * m.^5 .+ 18 * m.^4 .+ 3 * m.^3 .+ 18 * m.^2 .+ 15 * m) ./ ((1.0 .+ m.^6 .- 6 * m.^5 .- 3 * m.^4 .+ 4  * m.^3 .+ 9 * m.^2 .+ 6 * m))

    ldf[!, :se_pred] = sqrt.(lVarf)

        # Right side
    for i in eachindex(rdf.X)
        rdf[!,:dist] = rdf.X  .- rdf.X[i]
        rdf.pred[i] = predict(lm(@formula(Y ~ dist), rdf, wts=kernel.(rdf.dist / h)))[i]
    end

    m = max.(-1, (-rdf.X .+ c) / h)

    rVarf = (12 / (5 * n * h)) .* rdf.Y .* (2.0 .- 3 * m.^11 .- 24 * m.^10 .- 83 * m.^9 .- 72 * m.^8 .+ 42 * m.^7 .+ 18 * m.^6 .- 18 * m.^5 .+ 18 * m.^4 .- 3 * m.^3 .+ 18 * m.^2 .- 15 * m) ./ ((1.0 .+ m.^6 .+ 6 * m.^5 .- 3 * m.^4 .- 4  * m.^3 .+ 9 * m.^2 .- 6 * m).^2 )

    rdf[!, :se_pred] = sqrt.(rVarf)

fitted_res = DataFrame(Y=Y, X=X, fhat=vcat(ldf.pred, rdf.pred), se_fhat=vcat(ldf.se_pred, rdf.se_pred))

FittedMcCraryTest(
        b=b,
        h=h,
        θhat=θhat,
        σθ=σθ,
        pval=pval,
        Js=Js,
        fitted_res=fitted_res,
        coeftable=coeftbl
    )

end
function Base.show(io::IO, rdd_fit::FittedMcCraryTest)
    println(io, "The McCrary (2008) test for manipulation in the")
    println(io, "running variable for RDD.")
    println(io, "          ⋅⋅⋅⋅ " * "Bin size: ", string(round(rdd_fit.b, digits=4)))
    println(io, "          ⋅⋅⋅⋅ " * "Bandwidth size: ", string(round(rdd_fit.h, digits=4)))
    println(io, "          ⋅⋅⋅⋅ " * "Number of iterations: ", string(rdd_fit.Js[1]))
    Base.show(io, rdd_fit.coeftable)
end

# 7. Plot

@recipe function f(rdd_fit::FittedMcCraryTest)
    # Recovering the data
    X = rdd_fit.fitted_res[!,:X]
    Y = rdd_fit.fitted_res[!,:Y]
    f̂ = rdd_fit.fitted_res[!,:fhat]
    Jl = rdd_fit.Js[2]
    lX = X[1:Jl]
    rX = X[Jl + 1:end]
    lf̂ = f̂[1:Jl]
    rf̂ = f̂[Jl + 1:end]

    # set up
    legend := :false
    grid := false

    # scatter
    @series begin
        seriestype := :scatter
        seriescolor := :lightgray
        linecolor --> :lightgray
        seriesalpha := 0.99
        X, Y
    end

    # common to both series
    seriestype := :line
    linecolor --> :firebrick
    linewidth --> 2
    label --> "McCrary Test"

    # left side
    @series begin
        lX, lf̂
    end

    # left side
    @series begin
        rX, rf̂
end
end


using GeometricAlgebra
using BenchmarkTools
using Profile
Profile.init(500000000, 5e-8)

@basis "+++"

f(v1, v3, v12, v2) = (5v1 + 3v3 + 1v12) * 5v2
h(v1, v3, v12, v2) = 5v1 + 3v3 + 1v2
@benchmark f($v1, $v3, $v12, $v2)
# @profiler f(v1, v3, v12, v2)


# This file is part of RLinearAlgebra.jl

module ProceduralTestLSVCProjFO

using Test, RLinearAlgebra, LinearAlgebra, Random

@testset "LSVC Projection Full Orthogonal -- Procedural" begin

    # Supertype
    @test supertype(LinSysVecColProjFO) == LinSysVecColProjection

    # Testing context
    Random.seed!(1010)

    A = rand(15,10)
    x = rand(10)
    b = A * x

    z = rand(10)

    rsub = LinSysVecColProjFO()

    # Verify that S (storage) is nothing
    @test isnothing(rsub.S)

    # Verify that S (storage) is appropriately initialized
    @test let
        e = zeros(10)
        e[1] = 1.0
        res = dot(A[:,1], A * z - b)
        RLinearAlgebra.rsubsolve!(rsub, z, (e, A, res), 1)
        typeof(rsub.S) <: Matrix
    end

    # Verify that solution is orthogonalized against all previous hyperplanes
    @test let
        for i = 1:13
            e = zeros(10)
            e[mod(i, 1:10)] = 1.0
            res = dot(A[:,mod(i, 1:10)], A * z - b)
            RLinearAlgebra.rsubsolve!(rsub, z, (e, A, res), i)
        end

        norm(A' * (A * z - b) ) < 1e-11
    end
end

end # End Module


int_rules_5_1_1 = @theory begin

    #= ::Subsection::Closed:: =#
    #= 5.1.1*(a+b*arcsin(c*x))^n =#
    @apply_utils Antiderivative((~(a') + ~(b') * asin(~(c') * ~x)) ^ ~(n'), ~x) => ~x * (~a + ~b * asin(~c * ~x)) ^ ~n - ~b * ~c * ~n * Antiderivative((~x * (~a + ~b * asin(~c * ~x)) ^ (~n - 1)) / sqrt(1 - (~c) ^ 2 * (~x) ^ 2), ~x) <-- FreeQ([~a, ~b, ~c], ~x) && GtQ(~n, 0)
    @apply_utils Antiderivative((~(a') + ~(b') * acos(~(c') * ~x)) ^ ~(n'), ~x) => ~x * (~a + ~b * acos(~c * ~x)) ^ ~n + ~b * ~c * ~n * Antiderivative((~x * (~a + ~b * acos(~c * ~x)) ^ (~n - 1)) / sqrt(1 - (~c) ^ 2 * (~x) ^ 2), ~x) <-- FreeQ([~a, ~b, ~c], ~x) && GtQ(~n, 0)
    @apply_utils Antiderivative((~(a') + ~(b') * asin(~(c') * ~x)) ^ ~n, ~x) => (sqrt(1 - (~c) ^ 2 * (~x) ^ 2) * (~a + ~b * asin(~c * ~x)) ^ (~n + 1)) / (~b * ~c * (~n + 1)) + (~c / (~b * (~n + 1))) * Antiderivative((~x * (~a + ~b * asin(~c * ~x)) ^ (~n + 1)) / sqrt(1 - (~c) ^ 2 * (~x) ^ 2), ~x) <-- FreeQ([~a, ~b, ~c], ~x) && LtQ(~n, -1)
    @apply_utils Antiderivative((~(a') + ~(b') * acos(~(c') * ~x)) ^ ~n, ~x) => (-(sqrt(1 - (~c) ^ 2 * (~x) ^ 2)) * (~a + ~b * acos(~c * ~x)) ^ (~n + 1)) / (~b * ~c * (~n + 1)) - (~c / (~b * (~n + 1))) * Antiderivative((~x * (~a + ~b * acos(~c * ~x)) ^ (~n + 1)) / sqrt(1 - (~c) ^ 2 * (~x) ^ 2), ~x) <-- FreeQ([~a, ~b, ~c], ~x) && LtQ(~n, -1)
    @apply_utils Antiderivative((~(a') + ~(b') * asin(~(c') * ~x)) ^ ~n, ~x) => (1 / (~b * ~c)) * Subst(Antiderivative((~x) ^ ~n * cos(-(~a) / ~b + ~x / ~b), ~x), ~x, ~a + ~b * asin(~c * ~x)) <-- FreeQ([~a, ~b, ~c, ~n], ~x)
    @apply_utils Antiderivative((~(a') + ~(b') * acos(~(c') * ~x)) ^ ~n, ~x) => (-1 / (~b * ~c)) * Subst(Antiderivative((~x) ^ ~n * sin(-(~a) / ~b + ~x / ~b), ~x), ~x, ~a + ~b * acos(~c * ~x)) <-- FreeQ([~a, ~b, ~c, ~n], ~x)
end

src/rules/5 Inverse trig functions/5.1 Inverse sine/5.1.1 (a+b arcsin(c x))^n.jl


"""
_ultailL1(C, A, B)

gives L[Block(1,1)] of a block-tridiagonal Toeplitz operator. Based on Delvaux and Dette 2012.
"""

# need to solve [1 -B*inv(L); 0 1] * [C A B; 0 C L] == [C L 0; 0 C L]
# that is
# A - B*inv(L)*C == L
# In the scalar case this is quadratic
# L^2 - A*L + B*C == 0
# so that
# = L = (A ± sqrt(A^2 - 4B*C))/2
# we choose sign(A) to maximise the magnitude as we know inv(T)[1,1] -> 0, hence
# inv(L) -> 0
_ultailL1(c::Number, a::Number, b::Number) = (a + sign(a)*sqrt(a^2-4b*c))/2

# In the matrix case we write inv(L)*C = R (double check) to make it quadratic
# A - B*R == inv(C)inv(R)
# C*A*R - C*B*R^2 == I
# and do eigen decomposition of R to reduce this to a companion matrix problem.

function _ultailL1(C::AbstractMatrix, A::AbstractMatrix, B::AbstractMatrix)
    d = size(A,1)
    λs = filter!(λ -> abs(λ) ≤ 1, eigvals([zeros(d,d) -B; -C -A], [B zeros(d,d); zeros(d,d) -B]))
    @assert length(λs) == d
    V = Matrix{eltype(λs)}(undef, d, d)
    j = 1
    for λ in union(λs)
        c = count(==(λ), λs) # multiplicities
        V[:,j:j+c-1] = svd(A-λ*B - C/λ).V[:,end-c+1:end] # nullspace(A-λ*B - C/λ)
        j += c
    end
    C*(V*Diagonal(inv.(λs))/V)
end

function _ul(::TridiagonalToeplitzLayout, J::AbstractMatrix, ::Val{false}; check::Bool = true)
    C = getindex_value(subdiagonaldata(J))
    A = getindex_value(diagonaldata(J))
    B = getindex_value(supdiagonaldata(J))
    L = _ultailL1(C, A, B)
    U = B/L
    UL(Tridiagonal(Fill(convert(typeof(L),C),∞), Fill(L,∞), Fill(U,∞)), OneToInf(), 0)
end

function _ul(::TridiagonalToeplitzLayout, J::AbstractMatrix, ::Val{true}; check::Bool = true)
    C = getindex_value(subdiagonaldata(J))
    A = getindex_value(diagonaldata(J))
    B = getindex_value(supdiagonaldata(J))
    A^2 ≥ 4B*C || error("Pivotting not implemented")
    ul(J, Val(false))
end

function _ul(::BlockTridiagonalToeplitzLayout, J::AbstractMatrix, ::Val{false}; check::Bool = true)
    C = getindex_value(subdiagonaldata(blocks(J)))
    A = getindex_value(diagonaldata(blocks(J)))
    B = getindex_value(supdiagonaldata(blocks(J)))
    L = _ultailL1(C, A, B)
    U = B/L
    UL(mortar(Tridiagonal(Fill(convert(typeof(L),C),∞), Fill(L,∞), Fill(U,∞))), OneToInf(), 0)
end

_inf_getU(::TridiagonalToeplitzLayout, F::UL) = Bidiagonal(Fill(one(eltype(F)),∞),F.factors.du, :U)
_inf_getL(::TridiagonalToeplitzLayout, F::UL) = Bidiagonal(F.factors.d,F.factors.dl, :L)

function _inf_getU(::BlockTridiagonalToeplitzLayout, F::UL)
    d = size(F.factors.blocks.du[1],1)
    II = convert(eltype(F.factors.blocks.du), I(d))
    mortar(Bidiagonal(Fill(II,∞),F.factors.blocks.du, :U))
end

_inf_getL(::BlockTridiagonalToeplitzLayout, F::UL) = mortar(Bidiagonal(F.factors.blocks.d,F.factors.blocks.dl, :L))

getU(F::UL, ::NTuple{2,InfiniteCardinal{0}}) = _inf_getU(MemoryLayout(F.factors), F)
getL(F::UL, ::NTuple{2,InfiniteCardinal{0}}) = _inf_getL(MemoryLayout(F.factors), F)

getU(F::UL{T,<:Tridiagonal}, ::NTuple{2,InfiniteCardinal{0}}) where T = _inf_getU(MemoryLayout(F.factors), F)
getL(F::UL{T,<:Tridiagonal}, ::NTuple{2,InfiniteCardinal{0}}) where T = _inf_getL(MemoryLayout(F.factors), F)


using tele
using NCDatasets, Revise, Statistics, PyPlot, PyCall
using CFTime, DSP, FFTW
using DataFrames, NaNMath
#analyzing data from https://psl.noaa.gov/data/gridded/data.noaa.ersst.v5.html

#FILE PATHS
data_dir = "/home/brynn/Documents/JP/860/tsa_data/"
ersst_p = data_dir * "sst.mnmean.nc"
# cesm_p2 = data_dir * "tos_Omon_CCSM4_past1000_r1i1p1_135001-185012.nc"
cesm_p2 = data_dir * "tos_Omon_CCSM4_historical_r1i1p1_185001-200512.nc"

#READ NETCDFS TO BOUNDING BOX
bbox = [71, 55, 290, 350]
(late, lone, timee, datae) = read_netcdf(ersst_p, "sst", bbox)
(latc, lonc, timec, datac) = read_netcdf(cesm_p2, "tos", bbox)

#FILTER CHARACTERISTICS
#sampling = monthly
#cutoff all signals below 3 year periodicity
responsetype = Lowpass(0.1*10^(-7),fs = 3.80265176 * 10^(-7))
designmethod = Butterworth(4)

#CALCULATE ANOMALY AND FILTER
"""
all this permutedims! nonsense is because my anomaly code assumes lon x lon x time
and filter assumes time x lon x lat
"""

#ersst
dataep = zeros(length(timee), length(lone), length(late))
permutedims!(dataep, datae, [3,1,2])
filte = filt(digitalfilter(responsetype, designmethod), dataep)
permutedims!(datae, filte, [2,3,1])
anome = anomaly(datae, timee)
permutedims!(filte, anome, [3,1,2])
#filte holds filtered anomalized data

#cesm
timec = reinterpret(typeof(timee[begin]), timec) #convert from DatetimeNoLeap to DateTime
datac = datac .- 273.15 #convert to celsius

datacp = zeros(length(timec), length(lonc), length(latc))
permutedims!(datacp, datac, [3,1,2])
filtc = filt(digitalfilter(responsetype, designmethod), datacp)
permutedims!(datac, filtc, [2,3,1])
anomc = anomaly(datac, timec)
permutedims!(filtc, anomc, [3,1,2])
#filtc holds filtered, anomalized data

anomc = anomaly(datac, timec)
anomcp = zeros(length(timec), length(lonc), length(latc))
permutedims!(anomcp, anomc, [3,1,2])
filtc = filt(digitalfilter(responsetype, designmethod), anomcp)

#COMPUTE EOFS
eofs = pyimport("eofs")

#cesm
solver = eofs.standard.Eof(filtc)
savetype = ".pdf"
plot_eofs(solver, lonc, latc, timec, "images/eofc"*savetype)

#ersst
solver = eofs.standard.Eof(filte)
plot_eofs(solver, lone, late, timee, "images/eofe"*savetype)

#CORRELATION PLOT
#at every grid point, correlate the mean time series with the grid point's timeseries
tavgc = spatial_mean(anomc)
tavgc_filt = filt(digitalfilter(responsetype, designmethod), tavgc)
tavge = spatial_mean(anome)
tavge_filt = filt(digitalfilter(responsetype, designmethod), tavge)

xcc = zeros(length(timec)*2 - 1, length(lonc), length(latc))
xce = zeros(length(timee)*2 - 1, length(lone), length(late))
regc = zeros(length(lonc), length(latc))
rege = zeros(length(lone), length(late))

for i in 1:length(lonc)
    for j in 1:length(latc)
        ts = filtc[:, i, j]
        xcc[:, i, j] = xcorr(tavgc_filt, ts)
        regc[i, j] = normalequation(1:length(ts), ts)
    end
end

for i in 1:length(lone)
    for j in 1:length(late)
        ts = filte[:, i, j]
        xce[:, i, j] = xcorr(tavge_filt, ts)
        rege[i, j] = normalequation(1:length(ts), ts)
    end
end

plot_corr(xcc, latc, lonc, timec, "images/xcc"*savetype)
plot_corr(xce, late, lone, timee, "images/xce"*savetype)

plot_reg(regc, latc, lonc, "images/regc"*savetype)
plot_reg(rege, late, lone, "images/rege"*savetype)

#extreme value anlaysis
quant = quantile(tavgc, [0.25, 0.5, 0.75])
thresh1 = (quant[3] - quant[1])* 1 + quant[3]
thresh2 = quant[1] - (quant[3] - quant[1])* 1
ind1 = findall(>(thresh1), tavgc)
ind2 = findall(<(thresh2), tavgc)
extreme1 = temp_mean(anomc[:,:, ind1])
extreme2 = temp_mean(anomc[:,:, ind2])

#Pot extreme values
contplot(latc,lonc,extreme2, "images/ext_anomc_low"*savetype)
contplot(latc, lonc, extreme1, "images/ext_anomc_high"*savetype)

#extreme value anlaysis
quant = quantile(tavge, [0.25, 0.5, 0.75])
thresh1 = (quant[3] - quant[1])* 1 + quant[3]
thresh2 = quant[1] - (quant[3] - quant[1])* 1
ind1 = findall(>(thresh1), tavge)
ind2 = findall(<(thresh2), tavge)
extreme1 = temp_mean(anome[:,:, ind1])
extreme2 = temp_mean(anome[:,:, ind2])

#Pot extreme values
contplot(late,lone,extreme2, "images/ext_anome_low"*savetype)
contplot(late, lone, extreme1, "images/ext_anome_high"*savetype)


"""
    invariantManifolds(SYS, rv0, T; tf=1., nPts=20, α=1e-6, reltol=1e-12, integrator=TsitPap8())

Compute invariant manifolds of a periodic orbit in the CR3BP.
"""
function invariant_manifolds(sys::System, rv₀, T; tf=1., nPts=20, α=1e-6, reltol=1e-12, integrator=TsitPap8())

    Φ₀ = I(6)
    w₀ = vcat(rv₀,reshape(Φ₀,36,1))
    tspan = (0.,T)
    prob = ODEProblem(CR3BPstm!,w₀,tspan,sys)
    nomTraj = solve(prob, integrator, reltol=reltol)
    Φₜ = Matrix(reshape(nomTraj.u[end][7:42],6,6))
    rvₜ = nomTraj.u[end][1:6] # last time step
    Λ,V = eigen(Φₜ,sortby=isreal) # Λ is vector of eigenvalues, V is matrix of eigenvectors
    Yw = real(V[:,findall(isreal, Λ)]) # Eigenvectors corresponding to real eigenvalues
    Λ = Λ[findall(isreal, Λ)] # Purely real eigenvalues (have 0.0 imaginary component)
    Yws = Yw[:,findmin(real(Λ))[2]] # Eigenvector associated with stable eigenvalue λ < 1
    Ywu = Yw[:,findmax(real(Λ))[2]] # Eigenvector associated with unstable eigenvalue λ > 1

    tspan_forward = (0., tf)
    tspan_backward = (0., -tf)

    # Wsp = []
    # Wsn = []
    # Wup = []
    # Wun = []
    # for i = 1:length(nomTraj)
    #   Φᵢ = Matrix(reshape(nomTraj.u[i][7:42],6,6)) # do I need Matrix()?
    #   rv0sp = nomTraj[i][1:6] + α*Φᵢ*Yws/norm(Φᵢ*Yws);
    #   rv0sn = nomTraj[i][1:6] - α*Φᵢ*Yws/norm(Φᵢ*Yws);
    #   rv0up = nomTraj[i][1:6] + α*Φᵢ*Ywu/norm(Φᵢ*Ywu);
    #   rv0un = nomTraj[i][1:6] - α*Φᵢ*Ywu/norm(Φᵢ*Ywu);
    #
    #    prob_sp = ODEProblem(CR3BPdynamics!,rv0sp, tspan_backward, sys) # stable positive
    #    push!(Wsp, solve(prob_sp, integrator, reltol=reltol))
    #    prob_sn = ODEProblem(CR3BPdynamics!,rv0sn, tspan_backward, sys) # stable negative
    #    push!(Wsn, solve(prob_sn, integrator, reltol=reltol))
    #    prob_up = ODEProblem(CR3BPdynamics!,rv0up, tspan_forward, sys) # unstable positive
    #    push!(Wup, solve(prob_up, integrator, reltol=reltol))
    #    prob_un = ODEProblem(CR3BPdynamics!,rv0un, tspan_forward, sys) # unstable negative
    #    push!(Wun, solve(prob_un, integrator, reltol=reltol))
    # end

    Wsp = []
    Wsn = []
    Wup = []
    Wun = []
    for t = LinRange(0,T,nPts)
      Φᵢ = Matrix(reshape(nomTraj(t)[7:42],6,6)) # do I need Matrix()?
      rv0sp = nomTraj(t)[1:6] + α*Φᵢ*Yws/norm(Φᵢ*Yws);
      rv0sn = nomTraj(t)[1:6] - α*Φᵢ*Yws/norm(Φᵢ*Yws);
      rv0up = nomTraj(t)[1:6] + α*Φᵢ*Ywu/norm(Φᵢ*Ywu);
      rv0un = nomTraj(t)[1:6] - α*Φᵢ*Ywu/norm(Φᵢ*Ywu);

      prob_sp = ODEProblem(CR3BPdynamics!,rv0sp, tspan_backward, sys) # stable positive
      push!(Wsp, solve(prob_sp, integrator, reltol=reltol))
      prob_sn = ODEProblem(CR3BPdynamics!,rv0sn, tspan_backward, sys) # stable negative
      push!(Wsn, solve(prob_sn, integrator, reltol=reltol))
      prob_up = ODEProblem(CR3BPdynamics!,rv0up, tspan_forward, sys) # unstable positive
      push!(Wup, solve(prob_up, integrator, reltol=reltol))
      prob_un = ODEProblem(CR3BPdynamics!,rv0un, tspan_forward, sys) # unstable negative
      push!(Wun, solve(prob_un, integrator, reltol=reltol))
   end

    return Wsp, Wsn, Wup, Wun, Φₜ, nomTraj
end

function differential_corrector(sys::System, rv₀; myconst=3, iter=100, plot=false, t0=0., tf=1., dir=1, tol=1e-12)
   Φ₀ = I(6)

   tspan = (t0, tf)

   # event function
   condition(u, t, integrator) = u[2]
   affect!(integrator) = terminate!(integrator)
   cb = OrdinaryDiffEq.ContinuousCallback(condition, affect!)

   for k = 1:iter
      w₀ = vcat(rv₀, reshape(Φ₀,36,1))

      prob = ODEProblem(CR3BPstm!,w₀,tspan,sys)
      sol = solve(prob, TsitPap8(), reltol=tol, callback=cb)

      w = sol[end]
      rv = w[1:6]
      Φ = reshape(w[7:42],6,6)
      global T = 2*sol.t[end]

      ∂ẋ = 0 - rv[4]
      ∂ż = 0 - rv[6]

      if norm([∂ẋ, ∂ż]) < tol
         break
      end
      if k == iter
         @warn("Differential corrector did not converge")
      end

      ẋ,ẏ,ż,ẍ,ÿ,z̈ = CR3BPdynamics(rv, sys, 0)

      if myconst == 1
         ∂Φ = [Φ[4,3] Φ[4,5];
               Φ[6,3] Φ[6,5]]
         dyad = [ẍ;z̈]*[Φ[2,3] Φ[2,5]]

         ∂z, ∂ẏ = (∂Φ - dyad/ẏ)\[∂ẋ; ∂ż]
         rv₀[3] += ∂z
         rv₀[5] += ∂ẏ
      elseif myconst == 3
         ∂Φ = [Φ[4,1] Φ[4,5];
               Φ[6,1] Φ[6,5]]
         dyad = [ẍ;z̈]*[Φ[2,1] Φ[2,5]]

         ∂x, ∂ẏ = (∂Φ - dyad/ẏ)\[∂ẋ; ∂ż]
         rv₀[1] += ∂x
         rv₀[5] += ∂ẏ
      elseif myconst == 5
         ∂Φ = [Φ[4,1] Φ[4,3];
               Φ[6,1] Φ[6,3]]
         dyad = [ẍ;z̈]*[Φ[2,1] Φ[2,5]]

         ∂x, ∂z = (∂Φ - dyad/ẏ)\[∂ẋ; ∂ż]
         rv₀[1] += ∂x
         rv₀[3] += ∂ẏ
      else
         error("myconst should be 1, 3, or 5")
      end

   end

   return rv₀, T
end

""" condition(u, t, integrator) = u[2]
 affect!(integrator) = terminate!(integrator)
 cb = ContinousCallback(condition, affect!)
 sol = solve(prob, Tsit5(), calback=cb)

    rich3()
"""
function rich3(sys::System, Az, Lpt, NS, npts=10)
    μ = sys.μ
    γ = gammaL(sys, Lpt)

    if Lpt == 1;        won =  1;   primary = 1-μ;
    elseif Lpt == 2;    won = -1;   primary = 1-μ;
    elseif Lpt == 3;    won =  1;   primary = -μ;
    end

    c = zeros(4)

    if Lpt == 3
      for N = 2:4
        c[N]= (1/γ^3)*( 1-μ + (-primary*γ^(N+1))/((1+γ)^(N+1)) );
      end
    else
      for N = 2:4
        c[N]= (1/γ^3)*( (won^N)*μ + ((-1)^N)*((primary)*γ^(N+1))/((1+(-won)*γ)^(N+1)) );
      end
    end

    # polylambda = [1, 0, (c[2]-2), 0, -(c[2]-1)*(1+2*c[2])];
    polylambda = [-(c[2]-1)*(1+2*c[2]), 0, c[2]-2, 0, 1];
    lambdaroots = roots(polylambda); # lambda = frequency of orbit
 ## Stuck here 6/22/21 (need to select the right root)
    λ = real(sort(lambdaroots, by = x -> abs(imag(x))))[1]

    # if Lpt==3
    #    λ = lambdaroots[1];
    # else
    #    λ = lambdaroots[1];
    # end

    k   = 2*λ/(λ^2 + 1 - c[2]);

    del = λ^2 - c[2];

    d1  = ((3*λ^2)/k)*(k*( 6*λ^2 -1) - 2*λ);
    d2  = ((8*λ^2)/k)*(k*(11*λ^2 -1) - 2*λ);

    a21 = (3*c[3]*(k^2 - 2))/(4*(1 + 2*c[2]));
    a22 = 3*c[3]/(4*(1 + 2*c[2]));
    a23 = -(3*c[3]*λ/(4*k*d1))*( 3*(k^3)*λ - 6*k*(k-λ) + 4);
    a24 = -(3*c[3]*λ/(4*k*d1))*( 2 + 3*k*λ );

    b21 = -(3*c[3]*λ/(2*d1))*(3*k*λ - 4);
    b22 = 3*c[3]*λ/d1;
    d21 = -c[3]/(2*λ^2);

    a31 = -(9*λ/(4*d2))*(4*c[3]*(k*a23 - b21) + k*c[4]*(4 + k^2)) + ((9*λ^2 + 1 -c[2])/(2*d2))*(3*c[3]*(2*a23 - k*b21) + c[4]*(2 + 3*k^2));
    a32 = -(1/d2)*( (9*λ/4)*(4*c[3]*(k*a24 - b22) + k*c[4]) + 1.5*(9*λ^2 + 1 - c[2])*( c[3]*(k*b22 + d21 - 2*a24) - c[4]) );

    b31 = (.375/d2)*( 8*λ*(3*c[3]*(k*b21 - 2*a23) - c[4]*(2 + 3*k^2)) + (9*λ^2 + 1 + 2*c[2])*(4*c[3]*(k*a23 - b21) + k*c[4]*(4 + k^2)) );
    b32 = (1/d2)*( 9*λ*(c[3]*(k*b22 + d21 - 2*a24) - c[4]) + .375*(9*λ^2 + 1 + 2*c[2])*(4*c[3]*(k*a24 - b22) + k*c[4]) );

    d31 = (3/(64*λ^2))*(4*c[3]*a24 + c[4]);
    d32 = (3/(64*λ^2))*(4*c[3]*(a23- d21) + c[4]*(4 + k^2));

    s1  = (1/(2*λ*(λ*(1+k^2) - 2*k)))*( 1.5*c[3]*(2*a21*(k^2 - 2)-a23*(k^2 + 2) - 2*k*b21) - .375*c[4]*(3*k^4 - 8*k^2 + 8) );
    s2  = (1/(2*λ*(λ*(1+k^2) - 2*k)))*( 1.5*c[3]*(2*a22*(k^2 - 2)+a24*(k^2 + 2) + 2*k*b22 + 5*d21) + .375*c[4]*(12 - k^2) );

    a1  = -1.5*c[3]*(2*a21+ a23 + 5*d21) - .375*c[4]*(12-k^2);
    a2  =  1.5*c[3]*(a24-2*a22) + 1.125*c[4];

    l1 = a1 + 2*(λ^2)*s1;
    l2 = a2 + 2*(λ^2)*s2;

    # ADDITIONAL TERMS FROM GEOMETRY CENTER PAPER
    b33 = -k/(16*λ)*(12*c[3]*(b21-2*k*a21+k*a23)+3*c[4]*k*(3*k^2-4)+16*s1*λ*(λ*k-1));
    b34 = -k/(8*λ)*(-12*c[3]*k*a22+3*c[4]*k+8*s2*λ*(λ*k-1));
    b35 = -k/(16*λ)*(12*c[3]*(b22+k*a24)+3*c[4]*k);

    deltan = 2 - NS;

    Ax  = sqrt( (-del - l2*Az^2)/l1 );
    omg = 1+s1*Ax^2+s2*Az^2;
    freq=λ*omg;
    period=2*pi/freq;

    rvi  = zeros(npts,6);
    ss   = zeros(npts,1);
    if npts > 1
       dtau1= 2*pi/(npts-1);
    else
       dtau1= 2*pi;
    end
    tau1 = 0;
    for i=1:npts
       x = a21*Ax^2 + a22*Az^2 - Ax*cos(tau1) + (a23*Ax^2 - a24*Az^2)*cos(2*tau1) + (a31*Ax^3 - a32*Ax*Az^2)*cos(3*tau1);
       y = k*Ax*sin(tau1) + (b21*Ax^2 - b22*Az^2)*sin(2*tau1) + (b31*Ax^3 - b32*Ax*Az^2)*sin(3*tau1);
       z = deltan*Az*cos(tau1) + deltan*d21*Ax*Az*(cos(2*tau1) - 3) + deltan*(d32*Az*Ax^2 - d31*Az^3)*cos(3*tau1);
       y_plus = (b33*Ax^3 + b34*Ax*Az^2 - b35*Ax*Az^2)*sin(tau1);
       y = y + y_plus;     # ADD EXTRA TERMS FROM G.C. PAPER

       xdot = freq*Ax*sin(tau1) - 2*freq*(a23*Ax^2-a24*Az^2)*sin(2*tau1) - 3*freq*(a31*Ax^3 - a32*Ax*Az^2)*sin(3*tau1);
       ydot = freq*(k*Ax*cos(tau1) + 2*(b21*Ax^2 - b22*Az^2)*cos(2*tau1) + 3*(b31*Ax^3 - b32*Ax*Az^2)*cos(3*tau1));
       zdot = - freq*deltan*Az*sin(tau1) - 2*freq*deltan*d21*Ax*Az*sin(2*tau1) - 3*freq*deltan*(d32*Az*Ax^2 - d31*Az^3)*sin(3*tau1);
       ydot_plus = freq*(b33*Ax^3 + b34*Ax*Az^2 - b35*Ax*Az^2)*cos(tau1);
       ydot = ydot_plus + ydot; # ADD EXTRA TERMS FROM G.C. PAPER

       rvi[i,:]= γ*[ (primary+γ*(-won+x))/γ, y, z, xdot, ydot, zdot];
       ss[i]   = tau1/freq;
       tau1=tau1+dtau1;
    end

    return ss, rvi, period, Ax
end


module Hyperslice
export hyperslice

include("types.jl")
include("util.jl")
include("IntersectTest.jl")

using ..Meshes
using ..HypersliceSets
using Sobol: SobolSeq, next!
using JSON

function hyperslice(mesh::ConvexMesh, n::UInt32=UInt32(50))
  dims = size(mesh)[2]

  # Generate the focus points
  lbs = [b[1] for b=mesh.problemSpec.vals]
  ubs = [b[2] for b=mesh.problemSpec.vals]
  s = SobolSeq(lbs, ubs)
  fps = collect([next!(s) for _ = 1:n])

  hyperslice(mesh, fps)
end

function hyperslice(mesh::ConvexMesh, fps::Array{PointND})
  slices = []
  for fp in fps
    ss = sliceDims(mesh, fp)
    append!(slices, ss)
  end
  HypersliceSet(mesh.problemSpec, slices)
end

# slices of all dimensions
function sliceDims(mesh::ConvexMesh, fp::PointND)
  slices = []
  for dd in dimComb(UInt64(size(mesh)[2]))
    ss = slice(mesh, fp, dd[1], dd[2])
    hss = [HypersliceSegment(fp, simp, dd[1], dd[2], s.p1d1, s.p1d2, s.p2d1, s.p2d2) for (simp,s) in ss]
    append!(slices, hss)
  end
  return slices
end

function slice(mesh::ConvexMesh, fp::PointND, d1::Dim, d2::Dim)
  slices = Tuple{Simplex,Intersect2D}[]
  for s in mesh
    ts = [(s,t) for t in simplexPointIntersection(s, fp, d1, d2)]
    append!(slices, ts)
  end
  return slices
end

end


module FEAutodiffTests

using LinearAlgebra
using Gridap.FESpaces
using Gridap.Arrays
using Gridap.Fields
using Gridap.Geometry
using Gridap.TensorValues
using Gridap.CellData

domain = (0,1,0,1)
partition = (2,2)
model = CartesianDiscreteModel(domain,partition)

V = FESpace(model=model,valuetype=Float64,reffe=:Lagrangian,order=2,conformity=:H1)
U = TrialFESpace(V)

dv = get_cell_basis(V)
du = get_cell_basis(U)
uh = FEFunction(U,rand(num_free_dofs(U)))

trian = Triangulation(model)
quad = CellQuadrature(trian,2)

function user_uh_to_cell_energy(uh)
  cell_e = integrate(0.5*∇(uh)⋅∇(uh),trian,quad)
end

function user_uh_to_cell_residual(uh)
  cell_r = integrate(∇(uh)⋅∇(dv),trian,quad)
end

function user_uh_to_cell_jacobian(uh)
  cell_j = integrate(∇(du)⋅∇(dv),trian,quad)
end

cell_r = user_uh_to_cell_residual(uh)
cell_j = user_uh_to_cell_jacobian(uh)
cell_h = cell_j

cell_r_auto = autodiff_cell_residual_from_energy(user_uh_to_cell_energy,uh)
cell_h_auto = autodiff_cell_jacobian_from_energy(user_uh_to_cell_energy,uh)
cell_j_auto = autodiff_cell_jacobian_from_residual(user_uh_to_cell_residual,uh)

test_array(cell_r_auto,cell_r)
test_array(cell_j_auto,cell_j)
test_array(cell_h_auto,cell_h)

trian_Γ = BoundaryTriangulation(model)
quad_Γ = CellQuadrature(trian_Γ,2)
cell_ids = get_cell_id(trian_Γ)

function user_uh_to_cell_energy_Γ(uh)
  uh_Γ = restrict(uh,trian_Γ)
  cell_e = integrate(0.5*∇(uh_Γ)⋅∇(uh_Γ),trian_Γ,quad_Γ)
end

function user_uh_to_cell_residual_Γ(uh)
  uh_Γ = restrict(uh,trian_Γ)
  dv_Γ = restrict(dv,trian_Γ)
  cell_r = integrate(∇(uh_Γ)⋅∇(dv_Γ),trian_Γ,quad_Γ)
end

function user_uh_to_cell_jacobian_Γ(uh)
  uh_Γ = restrict(uh,trian_Γ)
  dv_Γ = restrict(dv,trian_Γ)
  du_Γ = restrict(du,trian_Γ)
  cell_j = integrate(∇(du_Γ)⋅∇(dv_Γ),trian_Γ,quad_Γ)
end

cell_e_Γ = user_uh_to_cell_energy_Γ(uh)
cell_r_Γ = user_uh_to_cell_residual_Γ(uh)
cell_j_Γ = user_uh_to_cell_jacobian_Γ(uh)
cell_h_Γ = cell_j_Γ

cell_r_Γ_auto = autodiff_cell_residual_from_energy(user_uh_to_cell_energy_Γ,uh,cell_ids)
cell_h_Γ_auto = autodiff_cell_jacobian_from_energy(user_uh_to_cell_energy_Γ,uh,cell_ids)
cell_j_Γ_auto = autodiff_cell_jacobian_from_residual(user_uh_to_cell_residual_Γ,uh,cell_ids)

test_array(cell_r_Γ_auto,cell_r_Γ)
test_array(cell_j_Γ_auto,cell_j_Γ)
test_array(cell_h_Γ_auto,cell_h_Γ)

const p = 3
@law j(∇u) = norm(∇u)^(p-2) * ∇u
@law dj(∇du,∇u) = (p-2)*norm(∇u)^(p-4)*inner(∇u,∇du)*∇u + norm(∇u)^(p-2)*∇du
f(x) = 0

res(u,v) = ∇(v)⋅j(∇(u)) - v*f
jac(u,du,v) = ∇(v)⋅dj(∇(du),∇(u))

function user_uh_to_cell_residual_2(uh)
  cell_r = integrate(res(uh,dv),trian,quad)
end

function user_uh_to_cell_jacobian_2(uh)
  cell_j = integrate(jac(uh,du,dv),trian,quad)
end

cell_j = user_uh_to_cell_jacobian_2(uh)

cell_j_auto = autodiff_cell_jacobian_from_residual(user_uh_to_cell_residual_2,uh)

test_array(cell_j_auto,cell_j)

end # module


using TensorNetworkAD
using Test
using Zygote, OMEinsum
using TensorNetworkAD: model_tensor, tensorfromclassical, trg_svd

@testset "exampletensor" begin
    β = rand()
    @test model_tensor(Ising(),β) ≈ tensorfromclassical([β -β; -β β])
end


function entropy( img )
	sum = 0.0
	@inbounds @simd for idx in 1:length(img)
		sum += img[idx]
	end

	entrpy = 0.0
	@inbounds @simd for idx in 1:length(img)
		entrpy += (img[idx] == 0) ? 0.0 : log( 2, img[idx]/sum )*img[idx]/sum
	end

	return -entrpy
end


# IMPORT CONTROL SYSTEMS
using ControlSystems
# LAPLACE OPERATOR
s = tf("s")
# SYSTEM DEFINITION
sys = 2/(s+1)
# IMPORT PLOTS
using Plots
# PLOT BODE FREQUENCY RESPONSE
bodeplot(sys)
# PLOT NICHOLS FREQUENCY RESPONSE
nicholsplot(sys)
# PLOT NYQUIST FREQUENCY RESPONSE
nyquistplot(sys)


"""
    `get_αβ_from_ϕ(ϕ,REG_STRUCT)`

    Get full α and β vectors from vector ϕ=[a,b,c] using Nl, ρ̄l, and dl, assuming energy function
    from Jenkinson et al 2017 and 2018.

        α_l = (a + b⋅ρ̄_l)⋅N_l , β_l = c/d_l

    # Examples
    ```julia-repl
    julia> rs=CpelNano.RegStruct(); rs.L=10; rs.Nl=fill(5.0,rs.L); rs.ρl=fill(0.1,rs.L); rs.dl=fill(10.0,rs.L-1);
    julia> CpelNano.get_αβ_from_ϕ([0.0,0.0,0.0],rs)
    ([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0])
    ```
"""
function get_αβ_from_ϕ(ϕ::Vector{Float64}, rs::RegStruct)::NTuple{2,Vector{Float64}}

    # Obtain full α & β vectors
    α = [rs.Nl[l] * (ϕ[1] + ϕ[2] * rs.ρl[l]) for l = 1:rs.L]
    β = [ϕ[3] / rs.dl[l] for l = 1:(rs.L - 1)]

    # Return α and β vector
    return α, β

end
"""
    `get_α_from_ex(exs)`

    Get α vector from E[X] vector. Only used for marginal model

        α_n = 0.5 * [log(E[X]+1) - log(1-E[X])]

    # Examples
    ```julia-repl
    julia> CpelNano.get_α_from_ex(zeros(5))

    ```
"""
function get_α_from_ex(exs::Vector{Float64})::Vector{Float64}

    # Obtain full α vector
    α = 0.5 * [log((exs[l] + 1) / (1 - exs[l])) for l = 1:length(exs)]
    α = max.(-AMAX, α)
    α = min.(AMAX, α)

    # Return α vector
    return α

end


struct Component{P,S}
    prior::P
    suffstats::S
end

Component(prior::Distribution) = Component(prior, empty_suffstats(prior))

Component(params::NTuple{N,<:Real}) where N = Component(NormalInverseChisq(params...))

Base.show(io::IO, c::Component{P,S}) where {P,S} = print(io, "$P$(params(c)) w/ n=$(nobs(c))")

posterior_predictive(gc::Component) =
    posterior_predictive(posterior_canon(gc.prior, gc.suffstats))

posterior_predictive(d::NormalInverseChisq) =
    LocationScale(d.μ, sqrt((1+d.κ)*d.σ2/d.κ), TDist(d.ν))

function posterior_predictive(d::NormalInverseWishart)
    df = d.nu - length(d.mu) + 1
    Λ = convert(AbstractMatrix, d.Lamchol) * (d.kappa+1)/(d.kappa*df)
    Λchol = cholesky(Λ)
    Distributions.GenericMvTDist(df, d.mu, PDMat(Λ, Λchol))
end

Distributions.logpdf(c::Component, x) = logpdf(posterior_predictive(c), x)
Distributions.pdf(c::Component, x) = logpdf(posterior_predictive(c), x)

Distributions.params(c::Component) = params(posterior_canon(c.prior, c.suffstats))

empty_suffstats(d::NormalInverseChisq) = NormalStats(0, 0, 0, 0)
function empty_suffstats(d::NormalInverseWishart)
    d = d.dim
    MvNormalStats(zeros(d), zeros(d), zeros(d,d), 0.)
end

# struct NormalStats <: SufficientStats
#     s::Float64    # (weighted) sum of x
#     m::Float64    # (weighted) mean of x
#     s2::Float64   # (weighted) sum of (x - μ)^2
#     tw::Float64    # total sample weight
# end

# based on teh wikipedia algorithm for online variance/mean
function add(ss::NormalStats, x)
    tw = ss.tw+1
    Δ = x - ss.m
    m = ss.m + Δ/tw
    s2 = tw>1 ? ss.s2 + Δ*(x-m) : 0
    return NormalStats(ss.s+x, m, s2, tw)
end

function sub(ss::NormalStats, x)
    tw = ss.tw-1
    if tw ≤ 0
        return NormalStats(0,0,0,0)
    else
        Δ = x - ss.m
        m = ss.m - Δ/tw
        s2 = ss.s2 - Δ*(x-m)
        return NormalStats(ss.s-x, m, s2, tw)
    end
end

function add(ss::MvNormalStats, x::AbstractVector)
    tw = ss.tw + 1
    Δ = x - ss.m
    m = ss.m + Δ/tw
    s2 = tw>1 ? ss.s2 + Δ*(x.-m)' : zeros(size(ss.s2))
    return MvNormalStats(ss.s+x, m, Hermitian(s2), tw)
end

function sub(ss::MvNormalStats, x::AbstractVector)
    tw = ss.tw-1
    if tw ≤ 0
        n = length(ss.m)
        return MvNormalStats(zeros(n), zeros(n), zeros(n,n), 0)
    else
        Δ = x - ss.m
        m = ss.m - Δ./tw
        s2 = ss.s2 - Δ*(x-m)'
        return MvNormalStats(ss.s-x, m, s2, tw)
    end
end

add(c::Component, x) = Component(c.prior, add(c.suffstats, x))
sub(c::Component, x) = Component(c.prior, sub(c.suffstats, x))

nobs(c::Component) = Int(c.suffstats.tw)
Base.isempty(c::Component) = nobs(c) == 0

"""
    marginal_lhood(c::Component)

The marginal likelihood of data fit so far by `o`, which is the integral of the
likelihood given the mean and variance under the prior on those parameters.
"""
marginal_lhood(c::Component) =
    marginal_lhood(c.prior, c.suffstats)

marginal_lhood(prior, suffstats) = exp(marginal_log_lhood(prior, suffstats))

function marginal_lhood(prior::NormalInverseChisq{Float64}, suffstats::NormalStats)
    μ0, σ20, κ0, ν0 = params(prior)
    μn, σ2n, κn, νn = params(posterior_canon(prior, suffstats))
    n = νn - ν0
    gamma(νn*0.5)/gamma(ν0*0.5) *
        sqrt(κ0/κn) *
        (ν0*σ20)^(ν0*0.5) * (νn*σ2n)^(-νn*0.5) /
        π^(n*0.5)
end

"""
    marginal_log_lhood(c::Component)

The marginal likelihood of data fit so far by `o`, which is the integral of the
likelihood given the mean and variance under the prior on those parameters.
"""
marginal_log_lhood(c::Component) =
    marginal_log_lhood(c.prior, c.suffstats)

function marginal_log_lhood(prior::NormalInverseChisq{Float64}, suffstats::NormalStats)
    μ0, σ20, κ0, ν0 = params(prior)
    μn, σ2n, κn, νn = params(posterior_canon(prior, suffstats))
    n = νn - ν0
    lgamma(νn*0.5) - lgamma(ν0*0.5) +
        0.5*(log(κ0)-log(κn)) +
        (0.5*ν0)*(log(ν0)+log(σ20)) - (0.5*νn)*(log(νn)+log(σ2n)) -
        (0.5*n)*log(π)
end

function marginal_log_lhood(prior::NormalInverseWishart, suffstats::MvNormalStats)
    μ0, Λchol0, κ0, ν0 = params(prior)
    μn, Λcholn, κn, νn = params(posterior_canon(prior, suffstats))
    n = νn - ν0
    d = length(μ0)

    (-0.5*n*d) * logπ +
        logmvgamma(d, νn*0.5) - logmvgamma(d, ν0*0.5) +
        logdet(Λchol0)*ν0*0.5 - logdet(Λcholn)*νn*0.5 +
        d*0.5*(log(κ0) - log(κn))
end