# # Fitting and Predicting - Time Data
#md # [![](https://mybinder.org/badge_logo.svg)](@__BINDER_ROOT_URL__/notebooks/fitpredictTime.ipynb)
#md # [![](https://img.shields.io/badge/show-nbviewer-579ACA.svg)](@__NBVIEWER_ROOT_URL__/notebooks/fitpredictTime.ipynb)

# A large majority of scientists and engineers who undertake rheological experiments fit their data with one or several viscoelastic models in order to classify materials, quantify their behaviour and predict their response to external perturbations.

# Standard linear viscoelastic models take the form of an ordinary differential equation between stress σ and strain ϵ. Fitting models and predicting their response in the time domain then requires computing viscoelastic hereditary integrals such as:

# ```math
# \sigma(t) = \int_{0}^t G(t - \tau) \frac{d \epsilon(\tau)}{d \tau} d \tau
# ```

# where G is the relaxation response (stress response to a step in strain) of the material, which can be analytically or numerically defined for each rheological model and its set of rheological parameters. A similar relation exists to calculate the strain from the stress history:

# ```math
# \epsilon(t) = \int_{0}^t J(t - \tau) \frac{d \sigma(\tau)}{d \tau} d \tau
# ```

# where J is the creep modulus of the material. Fitting and predicting behaviour then becomes non-trivial and standardised tools to process are needed.

using RHEOS

# ## Fitting

# #### Step 1: Loading in Experimental Data

# RHEOS has a convenient function to import data from CSV files (more information in [File I/O](@ref) section).

data = importcsv("assets/data_time.csv", t_col = 1, ϵ_col = 2, σ_col = 3)
rheotimedatatype(data)

# The time, strain and stress data are stored into a [`RheoTimeData`](@ref) struct.

# #### Step 2: Fitting Models

# Let's fit a Maxwell model via its relaxation modulus, G, as our data is from a stress relaxation test, thus the strain is imposed. The first argument is our data, the second argument tells RHEOS which model to fit ([`RheoModelClass`](@ref)) and the final argument tells RHEOS whether to fit the model using a relaxation modulus (*strain* imposed) or creep modulus (*stress* imposed). The function will return a [`RheoModel`](@ref) object, i.e. a [`RheoModelClass`](@ref) with fixed values for the parameters.

maxwell_model = modelfit(data, Maxwell, strain_imposed)

# For the fitting process RHEOS relies on the optimistion package [NLopt.jl](https://nlopt.readthedocs.io/en/latest/). RHEOS makes use of a local derivative free algorithm, specifically the Tom Rowan's "Subplex" algorithm originally introduced in his [PhD thesis at The University of Texas at Austin, 1990](https://dl.acm.org/doi/book/10.5555/100816).

# Next, we will fit a fractional Maxwell (spring) model (the only difference from the above model is that the dash-pot is replaced by a spring-pot). This time we will also add upper and lower bounds on the model parameters. This is highly recommended for fractional models in particular as values less than 0 or greater than 1 for the spring-pot parameter are unphysical and can cause errors in the Mittag-Leffler function used (more information about the fractional models in review TBA). To facilitate the convergence of the fitting algorithm, it is also suggested to define an initial set of parameters (the function uses 0.5 for all parameters not defined). For initial parameters, lower bounds and upper bounds, it is possible to only specify some of the parameters and the rest will be filled in automatically.

## Lower bounds
Lo = (cₐ = 0.0, a = 0.01, k = 0.0)
## Upper bounds
Hi = (cₐ = Inf, a = 0.99, k = Inf)
## Initial parameters
P0 = (cₐ = 0.1, a = 0.2, k = 2.0)

Fmaxwell_model = modelfit(data, FractS_Maxwell, strain_imposed, lo = Lo, hi = Hi, p0 = P0)

# Further optional parameters can be defined in the fitting function, such as the relative tolerance (default parameter of 1e-4) or the finite difference formula to use for the derivative (`"BD"` = Backward Difference (default), `"CD"` = Central Difference). If the user is interested in the model parameters on each optimisation iteration, they can be printed on the terminal by enabling `verbose = true`.

#md # !!! warning "Important"
#md #     Often an ideal step loading is assumed, which simplifies the equations in the first section to σ(t) = G(t)ϵ₀ and ϵ(t) = J(t)σ₀. This enables the modulus to be used directly for the fitting, which vastly reduces the computational burden. For this reason, RHEOS provides a dedicated function [`modelstepfit`](@ref) that requires the same parameters as [`modelfit`](@ref). If this assumption is appropriate for the data then fitting can be sped up greatly by use of this function.

#nb # Important Note: Often an ideal step loading is assumed, which simplifies the equations in the first section to σ(t) = G(t)ϵ₀ and ϵ(t) = J(t)σ₀. This enables the modulus to be used directly for the fitting, which vastly reduces the computational burden. For this reason, RHEOS provides a dedicated function `modelstepfit` that requires the same parameters as `modelfit`. If this assumption is appropriate for the data then fitting can be sped up greatly by use of this function.

# ## Predicting

# RHEOS allows the user to simulate the response of a model (with defined parameters, [`RheoModel`](@ref) struct) to an imposed loading.

# Given an incomplete data set (time data with only either stress or strain missing) and model with values substituted into parameters ([`RheoModel`](@ref)), return a new "complete" dataset based on the model with the simulated missing variable.

# #### Assess Quality of the Fit

# The ability of predicting model's response is first exploited to assess the quality of the fits above. The incomplete [`RheoTimeData`](@ref) variable is defined by extracting the time and the imposed variable (for the current example, the strain).

data_ext = onlystrain(data)
## alternatively: data_ext = extract(data, 1);

rheotimedatatype(data_ext)

# For the prediction, RHEOS' function [`modelpredict`](@ref) requires the incomplete data set and a model with fixed parameters ([`RheoModel`](@ref)). For the assessment of the fitting quality the [`RheoModel`](@ref) required is the output of the fitting function.

maxwell_predict = modelpredict(data_ext, maxwell_model)

## Now we can plot data and model together for comparison
using PyPlot

fig, ax = subplots(1, 1, figsize = (7, 5))
ax.plot(data.t, data.σ, "o", markersize = 5)
ax.plot(maxwell_predict.t, maxwell_predict.σ, color = "red")
ax.set_xlabel("Time")
ax.set_ylabel("Stress")
#!nb fig #hide

# #### Simulate different model behaviours

# The ability of predicting model's response can be exploited to simulate the behaviour of the model to other external loading conditions. For example, we will explore the response of the fitted model to a creep experiment.

# First we need to define an incomplete [`RheoTimeData`](@ref) struct, which can be achieved via the data generation functions provided in RHEOS (see section [Generating Data](@ref)).

## Define timeline
dσ = timeline(t_end = 10)
## and a step in stress
dσ = stressfunction(dσ, hstep())

## we can now predict the creep response of the Maxwell model
maxwell_creepPredict = modelsteppredict(dσ, maxwell_model)

## Init plot figure
fig, ax = subplots(1, 1, figsize = (7, 5))

## Visualisation of the imposed loading
ax.plot(maxwell_creepPredict.t, maxwell_creepPredict.σ, "--")
## Visualisation of the simulated response
ax.plot(maxwell_creepPredict.t, maxwell_creepPredict.ϵ)
ax.set_xlabel("Time")
ax.set_ylabel("Strain")
#!nb fig #hide

#md # !!! warning "Important"
#md #     As for the fitting procedure, if an ideal step loading is assumed, RHEOS provides a dedicated function [`modelsteppredict`](@ref) that requires the same parameters as [`modelpredict`](@ref) which significantly reduces the computation time.

#nb # Important Note: As for the fitting procedure, if an ideal step loading is assumed, RHEOS provides a dedicated function modelsteppredict that requires the same parameters as modelpredict which significantly reduces the computating time.

# Similarly, more complex behaviour can be simulated (such as a stairs loading).

## Define timeline
dσ = timeline(t_end = 10)
## and a step in stress
dσ = stressfunction(dσ, stairs())

## we can now predict the stairs response of the Maxwell model
maxwell_creepPredict = modelpredict(dσ, maxwell_model)
## Plotting
fig, ax = subplots(1, 1, figsize = (7, 5))
ax.plot(maxwell_creepPredict.t, maxwell_creepPredict.σ, "--")
ax.plot(maxwell_creepPredict.t, maxwell_creepPredict.ϵ)
ax.set_xlabel("Time")
ax.set_ylabel("Strain")
#!nb fig #hide


module TestSKPreprocessing

using Test
using AutoMLPipeline.SKPreprocessors
using AutoMLPipeline.Utils
using Statistics
using DataFrames

const IRIS = getiris()
const X = IRIS[:,1:4] |> DataFrame
const Y = IRIS[:,5] |> Vector

# "KernelCenterer","MissingIndicator","KBinsDiscretizer","OneHotEncoder",
const preprocessors = [
     "DictionaryLearning", "FactorAnalysis", "FastICA", "IncrementalPCA",
     "KernelPCA", "LatentDirichletAllocation", "MiniBatchDictionaryLearning",
     "MiniBatchSparsePCA", "NMF", "PCA",
     "TruncatedSVD",
     "VarianceThreshold",
     "SimpleImputer",
     "Binarizer", "FunctionTransformer",
     "MultiLabelBinarizer", "MaxAbsScaler", "MinMaxScaler", "Normalizer",
     "OrdinalEncoder", "PolynomialFeatures", "PowerTransformer",
     "QuantileTransformer", "RobustScaler", "StandardScaler"
 ]

function fit_test(preproc::String,in::DataFrame,out::Vector)
	_preproc=SKPreprocessor(Dict(:preprocessor=>preproc))
	fit!(_preproc,in,out)
	@test _preproc.model != Dict()
	return _preproc
end

function transform_test(preproc::String,in::DataFrame,out::Vector)
	_preproc=SKPreprocessor(Dict(:preprocessor=>preproc))
	fit!(_preproc,in,out)
	res = transform!(_preproc,in)
	@test size(res)[1] == size(out)[1]
end

@testset "scikit preprocessors fit test" begin
    for cl in preprocessors
	println(cl)
	fit_test(cl,X,Y)
    end
end

@testset "scikit preprocessors transform test" begin
    for cl in preprocessors
	println(cl)
	transform_test(cl,X,Y)
    end
end

function skptest()
    features = X
    labels = Y

    pca = SKPreprocessor(Dict(:preprocessor=>"PCA",:impl_args=>Dict(:n_components=>3)))
    fit!(pca,features)
    @test transform!(pca,features) |> x->size(x,2) == 3

    svd = SKPreprocessor(Dict(:preprocessor=>"TruncatedSVD",:impl_args=>Dict(:n_components=>2)))
    fit!(svd,features)
    @test transform!(svd,features) |> x->size(x,2) == 2

    ica = SKPreprocessor(Dict(:preprocessor=>"FastICA",:impl_args=>Dict(:n_components=>2)))
    fit!(ica,features)
    @test transform!(ica,features) |> x->size(x,2) == 2

    stdsc = SKPreprocessor(Dict(:preprocessor=>"StandardScaler",:impl_args=>Dict()))
    fit!(stdsc,features)
    @test abs(mean(transform!(stdsc,features) |> Matrix)) < 0.00001

    minmax = SKPreprocessor(Dict(:preprocessor=>"MinMaxScaler",:impl_args=>Dict()))
    fit!(minmax,features)
    @test mean(transform!(minmax,features) |> Matrix) ≈ 0.4486931104833648
end
@testset "scikit preprocessor fit/transform test with real data" begin
    skptest()
end

end


using UnivariateFunctions
const tol = 10*eps()

# Make undefined function
und = Undefined_Function()
ismissing(und(5))
results = und.([5, 9]) # Test broadcasting
ismissing(results[1])
ismissing(results[2])

# pe functions
function test_pe(a::Float64,b::Float64,base::Float64,d::Int,x::Float64)
    return a * exp(b*(x-base)) * (x-base)^d
end
f1 = PE_Function(1.0, 2.0,3.0, 4)
f1_test_result = test_pe(1.0, 2.0,3.0, 4,5.0)
abs(evaluate(f1, 5.0) - f1_test_result) < tol
f2 = PE_Function(0.0, 2.0,3.0, 4)
f2_test_result = 0.0
abs(evaluate(f2, 5.0) - f2_test_result) < tol
f3 = PE_Function{Float64,Int64}(1.0, 8.0,7.0, 8)
f3_test_result = test_pe(1.0, 8.0,7.0, 8,5.0)
abs(f3(5.0) - f3_test_result) < tol
f4 = PE_Function(1.0)
f4_test_result = test_pe(1.0, 0.0,0.0, 0,5.0)
abs(f4(5.0) - f4_test_result) < tol

# Sum of functions
sum0 =  Sum_Of_Functions([])
typeof(sum0) == UnivariateFunctions.Sum_Of_Functions
sum1 =  Sum_Of_Functions([f1])
sum1 = Sum_Of_Functions(sum1)
typeof(sum1) == UnivariateFunctions.Sum_Of_Functions
sum3 =  Sum_Of_Functions([f2,f3])
typeof(sum3) == UnivariateFunctions.Sum_Of_Functions
sum4 =  Sum_Of_Functions([f1,f2,f3])
typeof(sum4) == UnivariateFunctions.Sum_Of_Functions
length(sum4.functions_) == 2
sum5 =  Sum_Of_Functions([sum3, sum4])
typeof(sum5) == UnivariateFunctions.Sum_Of_Functions
length(sum5.functions_) == 2
abs(sum1(5.0) - f1_test_result) < tol
abs(evaluate(sum5, 5.0) - 2*f3_test_result - f1_test_result) < 100* tol
abs(evaluate(sum0, 5.0)) < tol
sum6 =  Sum_Of_Functions([sum3, sum4, und])
typeof(sum6)  == Undefined_Function

fl = 8.9
integ = 7

function test_result(func, expected_type, eval_to, len = 1)
    val_test = abs(evaluate(func, 5.0) - eval_to) < 1e-09
    if (!val_test)
        print("Failed Val Test")
    end
    type_test = isa(func, expected_type)
    if (!type_test)
        print("Failed Type Test")
    end
    if typeof(func) == UnivariateFunctions.Sum_Of_Functions
        len_test = length(func.functions_) == len
        if (!len_test)
            print("Failed Length Test")
        end
        return all([val_test,type_test,len_test])
    else
        return all([val_test,type_test])
    end
end

# pe and Float
test_result(f1 + fl, UnivariateFunctions.Sum_Of_Functions, f1_test_result + fl,2 )
test_result(f1 - fl, UnivariateFunctions.Sum_Of_Functions, f1_test_result - fl,2 )
test_result(f1 * fl, UnivariateFunctions.PE_Function, (f1_test_result) * fl )
test_result(f1 / fl, UnivariateFunctions.PE_Function, (f1_test_result) / fl )
test_result(fl + f1, UnivariateFunctions.Sum_Of_Functions, f1_test_result + fl,2 )
test_result(fl - f1, UnivariateFunctions.Sum_Of_Functions, fl - f1_test_result,2 )
test_result(fl * f1, UnivariateFunctions.PE_Function, (f1_test_result) * fl )
# pe and Int
test_result(f1 + integ, UnivariateFunctions.Sum_Of_Functions, f1_test_result + integ,2 )
test_result(f1 - integ, UnivariateFunctions.Sum_Of_Functions, f1_test_result - integ,2 )
test_result(f1 * integ, UnivariateFunctions.PE_Function, (f1_test_result) * integ )
test_result(f1 / integ, UnivariateFunctions.PE_Function, (f1_test_result) / integ )
test_result(integ + f1, UnivariateFunctions.Sum_Of_Functions, f1_test_result + integ,2 )
test_result(integ - f1, UnivariateFunctions.Sum_Of_Functions, -f1_test_result + integ,2 )
test_result(integ * f1, UnivariateFunctions.PE_Function, (f1_test_result) * integ )

# Sum and Float
test_result(sum4 + fl, UnivariateFunctions.Sum_Of_Functions, f1_test_result + f3_test_result + fl, 3 )
test_result(sum4 - fl, UnivariateFunctions.Sum_Of_Functions, f1_test_result + f3_test_result - fl, 3 )
test_result(sum4 * fl, UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result) * fl, 2 )
test_result(sum4 / fl, UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result) / fl, 2 )
test_result(fl + sum4, UnivariateFunctions.Sum_Of_Functions, f1_test_result + f3_test_result + fl, 3 )
test_result(fl - sum4, UnivariateFunctions.Sum_Of_Functions, -f1_test_result -f3_test_result + fl, 3 )
test_result(fl * sum4, UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result) * fl, 2 )
# Sum and Int
test_result(sum4 + integ, UnivariateFunctions.Sum_Of_Functions, f1_test_result + f3_test_result + integ, 3 )
test_result(sum4 - integ, UnivariateFunctions.Sum_Of_Functions, f1_test_result + f3_test_result - integ, 3 )
test_result(sum4 * integ, UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result) * integ, 2 )
test_result(sum4 / integ, UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result) / integ, 2 )
test_result(integ + sum4, UnivariateFunctions.Sum_Of_Functions, f1_test_result + f3_test_result + integ, 3 )
test_result(integ - sum4, UnivariateFunctions.Sum_Of_Functions, -f1_test_result -f3_test_result + integ, 3 )
test_result(integ * sum4, UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result) * integ, 2 )

### Making sums with addition and subtraction.
evaluate(sum4, 5.0) == evaluate(f1 + f2 + f3, 5.0)
abs(evaluate(sum4, 5.0) - 2*evaluate(f3, 5.0) - evaluate(f1 + f2 - f3, 5.0)) < 0.001

### Changing of base
f1_unchanged = change_base_of_PE_Function(f1, 3.0)
isa(f1_unchanged, UnivariateFunctions.PE_Function)
abs(f1.base_ - f1_unchanged.base_) < tol

f1_changed = change_base_of_PE_Function(f1, 4.0)
typeof(f1_changed) == UnivariateFunctions.Sum_Of_Functions
abs(f1_changed.functions_[1].base_ - 4.0) < tol
abs(f1_changed.functions_[3].base_ - 4.0) < tol
abs(evaluate(f1_changed, 5.0) - f1_test_result) < 100*tol # Changing bases should not change this.

f1_changed_again = change_base_of_PE_Function(f1, -1.0)
typeof(f1_changed_again) == UnivariateFunctions.Sum_Of_Functions
abs(f1_changed_again.functions_[1].base_ + 1.0) < tol
abs(f1_changed_again.functions_[3].base_ + 1.0) < tol
abs(evaluate(f1_changed_again, 5.0) - f1_test_result) < 1000000*tol # Changing bases should not change this.

f3_changed = change_base_of_PE_Function(f3, 12.0)
typeof(f3_changed) == UnivariateFunctions.Sum_Of_Functions
abs(f3_changed.functions_[1].base_ - 12.0) < tol
abs(f3_changed.functions_[3].base_ - 12.0) < tol
abs(evaluate(f3_changed, 5.0) - f3_test_result) < 100*tol # Changing bases should not change this.

### multiplication of functions
test_result( f1 * f3 , UnivariateFunctions.Sum_Of_Functions, (f1_test_result * f3_test_result), 9 )
test_result( f1 * PE_Function(1.0,2.0,3.0,4) , UnivariateFunctions.PE_Function, (f1_test_result * f1_test_result), 1 )
test_result( f1 * sum4 , UnivariateFunctions.Sum_Of_Functions, f1_test_result * (f1_test_result + f3_test_result), 10 )
test_result( sum4 * f1 , UnivariateFunctions.Sum_Of_Functions, f1_test_result * (f1_test_result + f3_test_result), 10 )
test_result( sum4 * sum4 , UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result)^2, 11 )

### Powers pe
test_result( f1 ^ 0 , UnivariateFunctions.PE_Function, 1.0 )
test_result( f1 ^ 1 , UnivariateFunctions.PE_Function, (f1_test_result) )
test_result( f1 ^ 2 , UnivariateFunctions.PE_Function, (f1_test_result * f1_test_result) )
abs(evaluate(f1 ^ 4 ,5.0) - (f1_test_result * f1_test_result * f1_test_result * f1_test_result)) < 1e-03 # This fails normal tol.

### Powers sums
test_result( sum4 ^ 0 , UnivariateFunctions.PE_Function, 1.0 )
test_result( sum4 ^ 1 , UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result), 2 )
test_result( sum4 ^ 2 , UnivariateFunctions.Sum_Of_Functions, (f1_test_result + f3_test_result)^2, 11 )
# Higher powers give an underflow problem with base changes.

## Testing Calculus.
pe_const    = PE_Function(2.0,0.0,5.0,0)
pe_lin      = PE_Function(2.0,0.0,5.0,1)
pe_quad     = PE_Function(2.0,0.0,2.0,2)
pe_exp      = PE_Function(2.0,2.0,1.0,0)
pe_exp_quad = PE_Function(2.0,2.0,2.0,2)

# Linear gradient constant
abs(evaluate(derivative(pe_lin),5.0) - evaluate(derivative(pe_lin),1.0) ) < tol
typeof(derivative(pe_lin)) == UnivariateFunctions.Sum_Of_Functions
# This is also linear
abs(evaluate(derivative(derivative(pe_quad)),5.0) -evaluate(derivative(derivative(pe_quad)),9.0) ) < tol
# derivative into a sum of functions
typeof(derivative(pe_exp_quad)) == UnivariateFunctions.Sum_Of_Functions
abs(evaluate(derivative(pe_exp_quad),5.0) - ( 2.0*exp(2.0*(5.0-2.0))*(5.0-2.0)*(2 + 2.0*(5.0-2.0)) ) ) < tol
# Derivaitve of sum
abs(evaluate(derivative(sum4),5.0) - evaluate(derivative(f1),5.0) - evaluate(derivative(f3),5.0) ) < 1e-09

# integral of constant
indefinite_integral(pe_const).d_ == 1
# Integral of quadratic
abs(evaluate_integral(pe_quad,1.0,2.0) - ((2/3)*2^3 - 4*2^2 + 8*2) + ((2/3)*1^3 - 4*1^2 + 8*1)) < tol
# Integral of exponential
abs(evaluate_integral(pe_exp,0.2,3.8) - (exp(2.0*(3.8-1.0)) - exp(2.0*(0.2-1.0))  )) < tol
# Integral of combined.
combined_analytical_integral = PE_Function(1.0,2.0,2.0,2) - PE_Function(1.0,2.0,2.0,1) + PE_Function(0.5,2.0,2.0,0)
abs(evaluate_integral(pe_exp_quad,0.2,3.8) - ( evaluate(combined_analytical_integral, 3.8) - evaluate(combined_analytical_integral, 0.2))) < tol

# left and right integrals
l_int = left_integral(pe_exp_quad, 0.2)
(evaluate(l_int, 3.8) - evaluate_integral(pe_exp_quad,0.2,3.8)) < tol
r_int = right_integral(pe_exp_quad, 3.8)
(evaluate(r_int, 0.2) - evaluate_integral(pe_exp_quad,0.2,3.8)) < tol

## Testing linear rescaling
# For PE_Functions
test_func = PE_Function(0.5,1.2,0.9,2)
inp = 4.0
alpha = 0.435
beta = -1.52
rescaled_input = alpha*inp + beta
converted_test_func = convert_to_linearly_rescale_inputs(test_func, alpha, beta)
abs(evaluate(test_func, inp) - evaluate(converted_test_func, rescaled_input)) < 1e-10
# For undefined.
typeof(convert_to_linearly_rescale_inputs(Undefined_Function(), alpha, beta)) == UnivariateFunctions.Undefined_Function
# SumOfFunctions
sumFunc = test_func + pe_exp_quad + pe_exp
converted_sumFunc = convert_to_linearly_rescale_inputs(sumFunc, alpha, beta)
abs(evaluate(sumFunc, inp) - evaluate(converted_sumFunc, rescaled_input)) < 1e-10

### Test undefined
typeof(und + und)  == Undefined_Function
typeof(und - und)  == Undefined_Function
typeof(und / und)  == Undefined_Function
typeof(und * und)  == Undefined_Function

# PE_Function
typeof(f1 + und)  == Undefined_Function
typeof(f1 - und)  == Undefined_Function
typeof(f1 / und)  == Undefined_Function
typeof(f1 * und)  == Undefined_Function
typeof(und + f1)  == Undefined_Function
typeof(und - f1)  == Undefined_Function
typeof(und / f1)  == Undefined_Function
typeof(und * f1)  == Undefined_Function

# Sum_Of_Functions
typeof(sum0 + und)  == Undefined_Function
typeof(sum0 - und)  == Undefined_Function
typeof(sum0 / und)  == Undefined_Function
typeof(sum0 * und)  == Undefined_Function
typeof(und + sum0)  == Undefined_Function
typeof(und - sum0)  == Undefined_Function
typeof(und / sum0)  == Undefined_Function
typeof(und * sum0)  == Undefined_Function
typeof(und + und)  == Undefined_Function
typeof(und - und)  == Undefined_Function
typeof(und / und)  == Undefined_Function
typeof(und * und)  == Undefined_Function

# Numbers
typeof(und + 4)  == Undefined_Function
typeof(und - 4)  == Undefined_Function
typeof(und / 4)  == Undefined_Function
typeof(und * 4)  == Undefined_Function
typeof(und ^ 4)  == Undefined_Function
typeof(4 + und)  == Undefined_Function
typeof(4 - und)  == Undefined_Function
typeof(4 * und)  == Undefined_Function

typeof(und + 4.0)  == Undefined_Function
typeof(und - 4.0)  == Undefined_Function
typeof(und / 4.0)  == Undefined_Function
typeof(und * 4.0)  == Undefined_Function
typeof(4.0 + und)  == Undefined_Function
typeof(4.0 - und)  == Undefined_Function
typeof(4.0 * und)  == Undefined_Function


using ForwardDiff

import Distributions: AbstractMvNormal

abstract type AbstractStateSpaceModel end
const AbstractSSM = AbstractStateSpaceModel
abstract type AbstractGaussianSSM <: AbstractStateSpaceModel end

# LinearGaussianSSM     LinearKalmanFilter
#                       NonlinearKalmanFilter
#                       NonlinearFilter
#
# NonlinearGaussianSSM  NonlinearKalmanFilter
#                       NonlinearFilter
#
# NonlinearSSM          NonlinearFilter

###########################################################################
# Linear Gaussian
###########################################################################

struct LinearGaussianSSM <: AbstractGaussianSSM
    # Process transition matrix, control matrix, and noise covariance
    F :: Function
    B :: Function
    V :: Function

    # Observation matrix and noise covariance
    G :: Function
    W :: Function

    # Model dimensions
    nx :: Int
    ny :: Int
    nu :: Int

    function LinearGaussianSSM( F :: Function,
                                B :: Function,
                                V :: Function,
                                G :: Function,
                                W :: Function )

        @assert ispossemidef(V(1))
        @assert ispossemidef(W(1))

        nx, ny, nu = confirm_matrix_sizes(F(1), B(1), V(1), G(1), W(1))
        new(F, B, V, G, W, nx, ny, nu)

    end

    ## Time-dependent constructor
    function LinearGaussianSSM( F :: Function,
                                V :: Function,
                                G :: Function,
                                W :: Function )

        B = _ -> begin zeros(T, (size(V(1), 1), 1)) end where T

        LinearGaussianSSM(F, B, V, G, W)

    end

    # Time-independent constructor
    function LinearGaussianSSM( F :: Matrix{T},
                                V :: Matrix{T},
                                G :: Matrix{T},
                                W :: Matrix{T}) where T

        B = zeros(T, size(F, 1), 1)

        LinearGaussianSSM(_->F, _->B, _->V, _->G, _->W)

    end

    function LinearGaussianSSM( F :: Matrix{T},
                                V :: Matrix{T},
                                G :: Matrix{T},
                                W :: Matrix{T},
                                B :: Matrix{T}) where T

        LinearGaussianSSM(_->F, _->B, _->V, _->G, _->W)

    end

end

import Base.show

function show(io::IO, mod::LinearGaussianSSM)

    dx, dy = mod.nx, mod.ny
    println("LinearGaussianSSM, $dx-D process x $dy-D observations")
    println("Process evolution matrix F:")
    show(mod.F(1))
    println("\n\nGontrol input matrix B:")
    show(mod.B(1))
    println("\n\nProcess error covariance V:")
    show(mod.V(1))
    println("\n\nObservation matrix G:")
    show(mod.G(1))
    println("\n\nObseration error covariance W:")
    show(mod.W(1))

end

## Core methods
process_matrix(m::LinearGaussianSSM, state::Vector, t::Real=0.0) = m.F(t)
process_matrix(m::LinearGaussianSSM, state::AbstractMvNormal, t::Real=0.0) = m.F(t)

observation_matrix(m::LinearGaussianSSM, state::Vector, t::Real=0.0) = m.G(t)
observation_matrix(m::LinearGaussianSSM, state::AbstractMvNormal, t::Real=0.0) = m.G(t)

control_input(m::LinearGaussianSSM, u, t::Real=0.0) = m.B(t) * u

###########################################################################
# Nonlinear Gaussian
###########################################################################

struct NonlinearGaussianSSM <: AbstractGaussianSSM
    # Process transition function, jacobian, and noise covariance matrix
    f :: Function
    V :: Function

    # Control input function
    b :: Function

    # Observation function,
    g :: Function # actual observation function
    W :: Function

    nx :: Int
    ny :: Int
    nu :: Int

    function NonlinearGaussianSSM( f  :: Function,
                                   V  :: Function,
                                   b  :: Function,
                                   g  :: Function,
                                   W  :: Function,
                                   nx :: Int,
                                   ny :: Int,
                                   nu :: Int)

        @assert ispossemidef(V(1))
        @assert ispossemidef(W(1))
        @assert (nx, nx) == size(V(1))
        @assert (ny, ny) == size(W(1))

        new(f, V, b, g, W, nx, ny, nu)

    end

    function NonlinearGaussianSSM( f  :: Function,
                                   V  :: Matrix{T},
                                   g  :: Function,
                                   W  :: Matrix{T};
                                   nu :: Int=0) where T

        b  = _ -> 0
        nx = size(V, 1)
        ny = size(W, 1)

        new(f, _-> V, b, g, _-> W, nx, ny, nu)

    end

end

## Core methods
function process_matrix( m :: NonlinearGaussianSSM,
                         x :: Vector,
                         t :: Real=0.0)

    ForwardDiff.jacobian(m.f, x)

end

function process_matrix( m :: NonlinearGaussianSSM,
                         x :: AbstractMvNormal,
                         t :: Real=0.0)

    process_matrix(m, mean(x), t)

end

function observation_matrix( m :: NonlinearGaussianSSM,
                             x :: Vector,
                             t :: Real=0.0)

    ForwardDiff.jacobian(m.g, x)

end

function observation_matrix( m :: NonlinearGaussianSSM,
                             x :: AbstractMvNormal,
                             t :: Real=0.0)

    process_matrix(m, mean(x), t)

end

function control_input( m :: NonlinearGaussianSSM,
                        u,
                        t :: Real=0.0 )
    m.b(u)

end


function WilkinsonTicks(k_ideal::Int; k_min = 2, k_max = 10,
        Q = [(1.0,1.0), (5.0, 0.9), (2.0, 0.7), (2.5, 0.5), (3.0, 0.2)],
        granularity_weight = 1/4,
        simplicity_weight = 1/6,
        coverage_weight = 1/3,
        niceness_weight = 1/4,
        min_px_dist = 50.0)

    if !(0 < k_min <= k_ideal <= k_max)
        error("Invalid tick number specifications k_ideal $k_ideal, k_min $k_min, k_max $k_max")
    end

    WilkinsonTicks(k_ideal, k_min, k_max, Q, granularity_weight, simplicity_weight,
        coverage_weight, niceness_weight, min_px_dist)
end

get_tick_labels(ticks::WilkinsonTicks, tickvalues) = linearly_spaced_tick_labels(tickvalues)

function compute_tick_values(ticks::WilkinsonTicks, vmin, vmax, pxwidth)

    min_px_dist = ticks.min_px_dist
    n_max_allowed_ticks = max(ticks.k_min, round(Int, pxwidth / min_px_dist))

    tickvalues, _ = PlotUtils.optimize_ticks(vmin, vmax;
        extend_ticks = false, strict_span=true, span_buffer = nothing,
        k_min = ticks.k_min,
        k_max = min(ticks.k_max, n_max_allowed_ticks),
        k_ideal = min(ticks.k_ideal, n_max_allowed_ticks),
        Q = ticks.Q,
        granularity_weight = ticks.granularity_weight,
        simplicity_weight = ticks.simplicity_weight,
        coverage_weight = ticks.coverage_weight,
        niceness_weight = ticks.niceness_weight)

    tickvalues
end


module TestSquares
using Test

using Catlab.Theories, Catlab.CategoricalAlgebra
using Catlab.CategoricalAlgebra.FinSets
using Catlab.CategoricalAlgebra.Squares

t  = FinFunction([1,2,4], 5)
l  = FinFunction([1,2,3], 3)
b = FinFunction([2,3,1], 3)
f  = FinFunction([2,3,2,1,3], 3)

α = SquareDiagram(t, b, l, f)

t₁ = FinFunction([1,1,1,1,1], 1)
r  = FinFunction([1], 1)
b₁ = FinFunction([1,1,1], 1)

β = SquareDiagram(t₁, b₁, f, r)
γ = composeH(α, β)
@test top(γ) == t⋅t₁
@test bottom(γ) == b⋅b₁
@test left(γ) == left(α)
@test right(γ) == right(β)

t = FinFunction([1,2,4], 5)
b = FinFunction([2,1,3], 3)
l = FinFunction([2,1,3], 3)
r = FinFunction([1,2,2,3,3], 3)
α = SquareDiagram(t,b,l,r)
β = SquareDiagram(FinFunction([1,2,1,2,1], 2),
                  FinFunction([1,1,1],1),
                  FinFunction([1,2,2,3,3], 3),
                  FinFunction([1,1],1)
)
γ = composeH(α, β)
@test collect(top(γ)) == [1,2,2]
@test collect(bottom(γ)) == [1,1,1]

α = SquareDiagram(l,r, t,b)
β = SquareDiagram(FinFunction([1,2,2,3,3], 3),
                  FinFunction([1,1],1),
                  FinFunction([1,2,1,2,1], 2),
                  FinFunction([1,1,1],1)
)
γ = composeV(α, β)
@test_throws AssertionError composeH(α, β)
@test collect(left(γ)) == [1,2,2]
@test collect(right(γ)) == [1,1,1]

@test collect(idH(FinSet(3))) == collect(1:3)
@test collect(idV(FinSet(5))) == collect(1:5)
@test collect(idH(FinSet(0))) == []

@test collect(composeH(FinFunction([2,1], 3), FinFunction([2, 1, 3]))) == [1,2]
@test collect(composeV(FinFunction([2,1], 3), FinFunction([2, 1, 3]))) == [1,2]

end


for a = 1:1000
  for b = (a+1):1000
    c = sqrt(a^2 + b^2)
    if c == floor(c) && a + b + c == 1000
      c = Int(floor(c))
      println(a * b * c)
    end
  end
end


"""
    Radial( nodes::NodesType, epsilon, gamma )

Radial interpolation

"""
mutable struct Radial <: InterpolationType

    nnodes     :: Int64
    epsilon    :: Float64
    nodes      :: Array{Float64,1}
    colloc_mat :: Array{Float64,2}

    function Radial( nodes_t::NodesType, epsilon::Float64 )

        nodes      = nodes_t.xk
        nnodes     = nodes_t.nx

        colloc_mat = exp.((-epsilon^2) * (nodes .- transpose(nodes)).^2)

        new( nnodes, epsilon, nodes, colloc_mat )

    end

end

"""
    Radial( xe )

returns evaluated function

"""
function (interp::Radial)( xe::Array{Float64,1} )

    epsilon = interp.epsilon
    xk      = interp.nodes

    (exp.((-epsilon^2) * (xe .- transpose(xk)).^2)) / interp.colloc_mat

end


include("euler/euler.jl")
using .Primes: prime_numbers

function compute(n::Int)::Int
    result = 1
    for prime ∈ prime_numbers(trunc(Int, √n))
        result *= prime
        if result * prime > n
            return result
        end
    end
end


"""
METROPOLIS-HASTING

The following codes explore the use of MH for Bayesian Inference.
"""
immutable MH
  logposterior::Function
  proposal    ::Function
  init_est    ::Array{Float64}
  d           ::Int64

  function default_proposal(theta::Array{Float64}, d::Int64)
    sigmas = ones(d);
    random = Float64[]

    for i in 1:length(theta)
      push!(random, rand(Normal(theta[i], sigmas[i])))
    end

    return random
  end

  MH(logposterior = Union{}; proposal::Function = default_proposal, init_est::Array{Float64} = [0.; 0.], d::Int64 = 2) = new(logposterior, proposal, init_est, d)
end

function mcmc(parameters::MH; r = 1000)
  chain = zeros(r, parameters.d)
  chain[1, :] = parameters.init_est

  for i in 1:(r - 1)
    propose = parameters.proposal(chain[i, :], parameters.d)
    probab = exp(parameters.logposterior(propose) - parameters.logposterior(chain[i, :]))

    if rand(Uniform()) < probab
      chain[i + 1, :] = propose
    else
      chain[i + 1, :] = chain[i, :]
    end
  end

  return chain

end


using LinearOptimizationSolver
using Plots
using Polyhedra

A = [2 1; 1 2]
b = [4, 4]
c = [4, 3]
input = create(A, b, c, tol=1E-10, solver = 1,verbose=false,standard=false)
output = solve(input)

x = output.X[1,:]
y = output.X[2,:]
n = length(x)
u = zeros(n)
v = zeros(n)
for i = 1:n-1
    u[i] = x[i+1] - x[i]
    v[i] = y[i+1] - y[i]
end

h = hrep([A; -1 0; 0 -1], [b; 0; 0])
p = polyhedron(h)

image = plot(p)
quiver!(x, y, quiver = (u, v))
quiver!([x[end]], [y[end]], quiver = ([c[1]]/4, [c[2]]/4))
plot!(x,y; seriestype = :scatter, title = "Problema da produção",legend = false)
savefig(image, "examples\\prod_path")

gap = output.gap

image = plot(log10.(gap),title = "Problema da produção",legend = false)
savefig(image, "examples\\prod_conv")


module IndianGPA

using SPPL

n = @sppl begin
    Nationality ~ SPPL.Choice([:India => 0.5, :USA => 0.5])
    if Nationality == :India
        Perfect ~ SPPL.Bernoulli(0.1)
        if Perfect == 1
            GPA ~ SPPL.Atomic(10)
        else
            GPA ~ SPPL.Uniform(0, 10)
        end
    else
        Perfect ~ SPPL.Bernoulli(0.15)
        if Perfect == 1
            GPA ~ SPPL.Atomic(4)
        else
            GPA ~ SPPL.Uniform(0, 4)
        end
    end
end

println(n.model.prob(n.Perfect << set(1)))
event = (n.Nationality << set(:USA)) & (n.GPA > 3) | ((8 < n.GPA) < 10)
model_condition = n.model.condition(event)

end # module


sphere(x, y) = x^2 + y^2
matyas(x, y) = 0.26 .* (x^2 + y^2) - 0.48 .* x .* y
goldstein(x, y) = (1 + (x + y + 1)^2 .* (19 - 14 .* x + 3 .* x^2 - 14 .* y + 6 .* x .* y + 3 .* y^2)) .*
    (30 + (2 .* x - 3 .* y)^2 .* (18 - 32 .* x + 12 .* x^2 + 48 .* y - 36 .* x .* y + 27 .* y^2))

function check(f, ans)
    x = Var(1)
    y = Var(1)
    z = f(x, y)
    gradient!(z)
    (x.grad.data, y.grad.data) == ans
end

@test check(sphere, ([2.0], [2.0]))
@test check(matyas, ([0.040000000000000036], [0.040000000000000036]))
@test check(goldstein, ([-5376.0], [8064.0]))


# Navigate an object through a series of obstacles, while minimizing the jerk of the trajectory.
# The domain is broken down into a polyhedral covering, and the time interval into subintervals.
# In each subinterval, we enforce the object is contained entirely within one of the polyhedral
# subregions, which enforces that we do not run into any obstacles.
#
# The trajectory is described by polynomials p_x and p_y, which describe the x and y position,
# respectively, as a function of time. This is a polynomial optimization problem, which we relax
# into a mixed-integer sum of squares problem. The problem is infeasible for polynomials of degree
# 4 or less, and feasible for degree 5.
#
# Example written and by Joey Huchette.

# source: https://github.com/JuliaOpt/Pajarito.jl/blob/master/examples/collisionavoidance.jl

using JuMP, PolyJuMP, SumOfSquares, MultivariatePolynomials
using Pajarito

# Specify Pajarito solver

mip_solver_drives = true
rel_gap = 1e-4

# using Cbc
# mip_solver = CbcSolver()

using CPLEX
mip_solver = CplexSolver(
    CPX_PARAM_SCRIND=(mip_solver_drives ? 1 : 0),
    CPX_PARAM_EPINT=1e-9,
    CPX_PARAM_EPRHS=1e-9,
    CPX_PARAM_EPGAP=(mip_solver_drives ? 1e-5 : 1e-9)
)

# using Mosek
# conic_solver = MosekSolver(LOG=0)
# conic_solver = MosekSolver(MSK_DPAR_INTPNT_CO_TOL_PFEAS=1e-6,MSK_DPAR_INTPNT_CO_TOL_DFEAS=1e-6,MSK_DPAR_INTPNT_CO_TOL_REL_GAP=1e-5,MSK_DPAR_INTPNT_TOL_INFEAS=1e-8,MSK_DPAR_INTPNT_CO_TOL_MU_RED=1e-6)

micp_solver = PajaritoSolver(
    mip_solver_drives=mip_solver_drives,
    log_level=3,
    rel_gap=rel_gap,
    mip_solver=mip_solver,
    # cont_solver=conic_solver,
    solve_relax=false,
    solve_subp=false,
    prim_cuts_only=true
)

# Create model

struct Box
    xl::Float64
    xu::Float64
    yl::Float64
    yu::Float64
end

boxes = Box[
    Box(0.0,1.0,0.0,0.3),
    Box(0.8,1.7,0.1,0.3),
    Box(1.4,1.9,0.2,0.4),
    Box(1.0,1.7,0.3,0.5),
    Box(0.5,1.4,0.4,0.6),
    Box(0.0,1.0,0.5,0.7),
    Box(0.2,1.0,0.6,0.8),
    Box(0.5,1.3,0.7,0.9),
    Box(1.0,2.0,0.7,1.0)
]

N = 8 # Number of trajectory pieces
d = 2 # dimension of space
r = 5 # dimension of polynomial trajectories
M = 2 # number of horizontal segments

domain = Box(0,M,0,1)

X₀ = Dict(:x=>0, :y=>0)
X₀′ = Dict(:x=>1, :y=>0)
X₀′′ = Dict(:x=>0, :y=>0)
X₁ = Dict(:x=>2, :y=>1)

T = linspace(0, 1, N+1)

Tmin = minimum(T)
Tmax = maximum(T)

model = SOSModel(solver=micp_solver)

@polyvar(t)
Z = monomials([t], 0:r)

@variable(model, H[1:N,boxes], Bin)

Mxl, Mxu, Myl, Myu = domain.xl, domain.xu, domain.yl, domain.yu
p = Dict()
for j in 1:N
    @constraint(model, sum(H[j,box] for box in boxes) == 1)

    p[(:x,j)] = @polyvariable(model, _, Z)
    p[(:y,j)] = @polyvariable(model, _, Z)
    for box in boxes
        xl, xu, yl, yu = box.xl, box.xu, box.yl, box.yu
        @polyconstraint(model, p[(:x,j)] >= Mxl + (xl-Mxl)*H[j,box], domain = (t >= T[j] && t <= T[j+1]))
        @polyconstraint(model, p[(:x,j)] <= Mxu + (xu-Mxu)*H[j,box], domain = (t >= T[j] && t <= T[j+1]))
        @polyconstraint(model, p[(:y,j)] >= Myl + (yl-Myl)*H[j,box], domain = (t >= T[j] && t <= T[j+1]))
        @polyconstraint(model, p[(:y,j)] <= Myu + (yu-Myu)*H[j,box], domain = (t >= T[j] && t <= T[j+1]))
    end
end

for axis in (:x,:y)
    @constraint(model, p[(axis,1)]([Tmin], [t]) == X₀[axis])
    @constraint(model, differentiate(p[(axis,1)], t)([Tmin], [t]) == X₀′[axis])
    @constraint(model, differentiate(p[(axis,1)], t, 2)([Tmin], [t]) == X₀′′[axis])

    for j in 1:N-1
        @constraint(model, p[(axis,j)]([T[j+1]], [t]) == p[(axis,j+1)]([T[j+1]], [t]))
        @constraint(model, differentiate(p[(axis,j)], t )([T[j+1]], [t]) == differentiate(p[(axis,j+1)], t)([T[j+1]], [t]))
        @constraint(model, differentiate(p[(axis,j)], t, 2)([T[j+1]], [t]) == differentiate(p[(axis,j+1)], t, 2)([T[j+1]], [t]))
    end

    @constraint(model, p[(axis,N)]([Tmax], [t]) == X₁[axis])
end

@variable(model, γ[keys(p)] ≥ 0)
for (key,val) in p
    @constraint(model, γ[key] ≥ norm(differentiate(val, t, 3)))
end
@objective(model, Min, sum(γ))

solve(model)

# Interpret solution: eval_poly(r) gives the trajectory location at time r

PP = Dict(key => getvalue(p[key]) for key in keys(p))
HH = getvalue(H)

function eval_poly(r)
    for i in 1:N
        if T[i] <= r <= T[i+1]
            return PP[(:x,i)]([r], [t]), PP[(:y,i)]([r], [t])
            break
        end
    end
    error("Time $r out of interval [$(minimum(T)),$(maximum(T))]")
end


using MDBM
using LinearAlgebra, Arpack
function fooeig(x)
       mymx=Matrix{Float64}(I,5,5);
       mymx[2,1]=x;
       mymx[1,2]=x+5;
       abs(eigs(mymx, nev = 2)[1][1])-3
end
eig_problem=MDBM_Problem(fooeig,[Axis(-10:10)])

println(" X solution  //   Function val")
for k=1:12
solve!(eig_problem,1)
Xsol=getinterpolatedsolution(eig_problem)
print(Xsol[1])
print("  //   ")
println(maximum(abs.(fooeig.(Xsol[1]))))
end


# tests based on AMLCR 2nd ed,
@testset "AMLCR Math" begin

    # §5.7 payment frequency and certain (Tables 5.1 and 5.2)
    function f(age)
        life = SingleLife(mortality = AMLCR_std, issue_age = age)
        int = Yields.Constant(0.05)
        lc = [
            AnnuityImmediate(life, int)
            AnnuityImmediate(life, int, frequency = 4)
            AnnuityDue(life, int)
            AnnuityDue(life, int, frequency = 4)
            AnnuityImmediate(life, int, 10)
            AnnuityImmediate(life, int, 10, frequency = 4)
            AnnuityDue(life, int, 10, frequency = 4)
        ]

        return present_value.(lc)
    end

    # atol only to two decimals b/c AMLCR uses rounded rates?
    @test all(isapprox.(f(20), [18.966, 19.338, 19.966, 19.588, 7.711, 7.855, 7.952], atol = 0.01))
    @test all(isapprox.(f(80), [7.548, 7.917, 8.548, 8.167, 6.128, 6.373, 6.539], atol = 0.01))

    life = SingleLife(mortality = AMLCR_std, issue_age = 20)
    int = Yields.Constant(0.05)

    # property tests

    @test present_value(AnnuityDue(life, int)) > present_value(AnnuityDue(life, int, start_time = 5))

    @test present_value(AnnuityDue(life, int, 5, certain = 5, start_time = 5)) == sum([1 for t = 5:9] .* [1 / 1.05^t for t = 5:9])

    # relation on pg 124
    @test issorted(present_value.([
        AnnuityImmediate(life, int)
        AnnuityImmediate(life, int, frequency = 4)
        AnnuityDue(life, int, frequency = 4)
        AnnuityDue(life, int)
    ]))

end#


using Empirikos

@test NormalSample(1.0, 2.0) < NormalSample(2.0, 1.0)
@test NormalSample(1.0, 2.0) < NormalSample(1.0, 3.0)
@test NormalSample(1.0, 2.0) <= NormalSample(1.0, 2.0)

@test BinomialSample(1, 3) < BinomialSample(2, 2)
@test BinomialSample(2, 2) < BinomialSample(2, 3)
@test BinomialSample(3, 4) > BinomialSample(2, 3)
@test BinomialSample(3, 3) <= BinomialSample(3, 3)

@test PoissonSample(1, 3) < PoissonSample(2, 2)
@test PoissonSample(2, 2) < PoissonSample(2, 3)
@test PoissonSample(3, 4) > PoissonSample(2, 3)
@test PoissonSample(3, 3.1) <= PoissonSample(3, 3.1)

Zs_comp = compound( [NormalSample(1,2), NormalSample(2,1),NormalSample(1,1), NormalSample(0.5,2)])
@test Zs_comp[1] < Zs_comp[2]
@test Zs_comp[1] == Zs_comp[3]

Zs = PoissonSample.(sample(1:100, 1000))

Zs_summary = summarize(Zs)

Zs_unique = collect(keys(Zs_summary))
@test issorted(Zs_unique)

@test collect(values(Zs_summary))[2] == sum(Zs .== Zs_unique[2])
@test collect(keys(Zs_summary.store)) == collect(keys(Zs_summary))

@test issorted([response(k) for (k, v) in Zs_summary.store])


# ---
# title: 996. Number of Squareful Arrays
# id: problem996
# author: Indigo
# date: 2021-05-31
# difficulty: Hard
# categories: Math, Backtracking, Graph
# link: <https://leetcode.com/problems/number-of-squareful-arrays/description/>
# hidden: true
# ---
#
# Given an array `A` of non-negative integers, the array is _squareful_ if for
# every pair of adjacent elements, their sum is a perfect square.
#
# Return the number of permutations of A that are squareful.  Two permutations
# `A1` and `A2` differ if and only if there is some index `i` such that `A1[i]
# != A2[i]`.
#
#
#
# **Example 1:**
#
#
#
#     Input: [1,17,8]
#     Output: 2
#     Explanation:
#     [1,8,17] and [17,8,1] are the valid permutations.
#
#
# **Example 2:**
#
#
#
#     Input: [2,2,2]
#     Output: 1
#
#
#
#
# **Note:**
#
#   1. `1 <= A.length <= 12`
#   2. `0 <= A[i] <= 1e9`
#
#
## @lc code=start
using LeetCode

function can_adj(u, v)
    sq = isqrt(u + v)
    return sq * sq == u + v
end

function num_squareful_perms(nums::Vector{Int})
    ctr = counter(nums)
    adj_set = Set{Tuple{Int, Int}}()
    for i1 in 1:length(nums), i2 in i1:length(nums)
        n1, n2 = nums[i1], nums[i2]
        if can_adj(n1, n2)
            push!(adj_set, (n1, n2))
            push!(adj_set, (n2, n1))
        end
    end
    function gen_perms(ctr, lst = nothing)
        sum(ctr) == 0 && return 1
        res = 0
        for k in keys(ctr)
            (ctr[k] == 0 || !isnothing(lst) && (lst, k) ∉ adj_set) && continue
            ctr[k] -= 1
            res += gen_perms(ctr, k)
            ctr[k] += 1
        end
        res
    end
    gen_perms(ctr)
end
## @lc code=end


"""
    initialize_matrices(graph::Dict{Int64,Array{Tuple{Int64,Int64},1}})

Initialize the matrices for distances and pathes.

# Arguments
- `graph::Dict{Int64,Array{Tuple{Int64,Int64},1}}`: Graph of the connected nodes with weights
"""
function initialize_matrices(graph::Dict{Int64,Array{Tuple{Int64,Int64},1}})
    # Define intial infinity weight & next matrix for given data type
    vertex_size = find_global_maximum_complex(graph)
    dist = fill(99999, (vertex_size, vertex_size))
    next = zeros(Int64, (vertex_size, vertex_size))

    # Updating the matrix with actual vertex values
    for (key, elements) in graph
        # Setting the diagonal-elements of the weight matrix to 0
        dist[key, key] = 0
        # Setting the diagonal-elements of the next matrix to vertex
        next[key, key] = key
        # Weigthing of the edges and pathes
        for value in elements
            dist[key, value[1]] = value[2]
            next[key, value[1]] = value[1]
        end
    end
    return dist, next, vertex_size
end

"""
    path_reconstruction(next::Array{Int64,2}, u::Int64, v::Int64)

Reconstruction of the actual path between any two endpoint vertices (u & v).

# Arguments
- `graph::Dict{Int64,Array{Tuple{Int64,Int64},1}}`: Graph of the connected nodes with weights
- `u::Int64`: Startpoint of the to investigated path
- `v::Int64`: Endpoint of the to investigated path
"""
function path_reconstruction(next::Array{Int64,2}, u::Int64, v::Int64)
    # Check if path exists
    if next[u, v] == 0
        return zeros(Int64, false)
    end

    # Initialize path
    path = [u]

    while u != v
        u = next[u, v]
        append!(path, u)
    end
    return path
end

"""
    shortest_path_tree(graph::Dict{Int64,Array{Tuple{Int64,Int64},1}}, u=nothing, v=nothing)

The Shortest Path Tree (SPT) algorithm solves the shortest path problem between every
pair of vertices in a given edge-weighted directed Graph based on the Floyd–Warshall
algorithm. Optional, the SPT also provides the total parts between a start- (u) and end-
point (v). For more information see: [https://en.wikipedia.org/wiki/Floyd–Warshall_algorithm](https://en.wikipedia.org/wiki/Floyd–Warshall_algorithm)

# Arguments
- `next::Array{Int64,2}`: Vertex matrix of the connected nodes
- `u::Int64`: Startpoint of the to investigated path; optional
- `v::Int64`: Endpoint of the to investigated path; optional

# Examples
```julia-repl
julia> import ClassicAlgorithmsCollections
julia> graph = Dict( 1 => [(3, -2)], 2 => [(1, 4),(3, 3)], 3 => [(4, 2)], 4 => [(2, -1)]
julia> ClassicAlgorithmsCollections.shortest_path_tree(graph, 2, 4)
([0 -1 -2 0; 4 0 2 4; 5 1 0 2; 3 -1 1 0], [2, 1, 3, 4])
```
"""
function shortest_path_tree(
    graph::Dict{Int64,Array{Tuple{Int64,Int64},1}},
    u = nothing,
    v = nothing,
)

    dist, next, vertex_size = initialize_matrices(graph)

    # Floyd–Warshall-Algorithm
    for k in 1:vertex_size
        for i in 1:vertex_size
            for j in 1:vertex_size
                # Checking for minimum
                if dist[i, j] > dist[i, k] + dist[k, j]
                    # Add new distance
                    dist[i, j] = dist[i, k] + dist[k, j]
                    # Add new path points
                    next[i, j] = next[i, k]
                end
            end
        end
    end

    # Path-Reconstruction
    if !(isnothing(u) && isnothing(v))
        return dist, path_reconstruction(next, u, v)
    else
        return dist
    end
end


import Makie
import CairoMakie
using Statistics: mean, std, var, cor
using Random: randn!, bitrand
using LogExpFunctions: logsumexp
using RestrictedBoltzmannMachines: BinaryRBM, energy, free_energy, metropolis!

N = 5
M = 2
rbm = BinaryRBM(randn(N), randn(M), randn(N, M) / √N);

β = 0.5
nsteps = 10000
nchains = 100
v = bitrand(N, nchains, nsteps)
metropolis!(v, rbm; β);

counts = Dict{BitVector, Int}()
for t in 1000:nsteps, n in 1:nchains
    counts[v[:,n,t]] = get(counts, v[:,n,t], 0) + 1
end
freqs = Dict(v => c / sum(values(counts)) for (v,c) in counts);

𝒱 = [BitVector(digits(Bool, x; base=2, pad=N)) for x in 0:(2^N - 1)];
ℱ = free_energy.(Ref(rbm), 𝒱);

fig = Makie.Figure()
ax = Makie.Axis(fig[1,1], xlabel="empirical freqs.", ylabel="log(p)", xscale=log10, yscale=log10)
Makie.scatter!(ax, [get(freqs, v, 0.0) for v in 𝒱], exp.(-β * ℱ .- logsumexp(-β * ℱ)))
Makie.scatter!(ax, [get(freqs, v, 0.0) for v in 𝒱], exp.(-ℱ .- logsumexp(-ℱ)))
Makie.abline!(ax, 0, 1, color=:red)
fig

# Correlation

cor([get(freqs, v, 0.0) for v in 𝒱], exp.(-β * ℱ .- logsumexp(-β * ℱ)))


#=================================
Methods using Interval Arithmetic
=================================#
using IntervalArithmetic

"""
    enclose_IntervalArithmetic(f::Function, dom::Interval_or_IntervalBox)

Compute a range enclosure using interval arithmetic substitution.

### Input

- `f`   -- function
- `dom` -- hyperrectangular domain, either a unidimensional  `Interval` or
           a multidimensional `IntervalBox`

### Output

An interval representing the range enclosure (minimum and maximum) of `f` over
its domain `dom`.

### Algorithm

The result is obtained by substitution of `dom` on the function `f` and
application of the rules of interval arithmetic.

We refer to the documentation and source code of `IntervalArithmetic` for details.
"""
function enclose_IntervalArithmetic(f::Function, dom::Interval_or_IntervalBox)
    return f(dom...)
end

#===================================
Methods using Interval Optimisation
===================================#
using IntervalOptimisation

"""
    range_and_optimisers(f::Function, dom::Interval_or_IntervalBox;
                         structure=HeapedVector, tol=1e-3)

Return an interval that contains the global minimum (resp. an interval that
contains the global maximum) of a univariate or multivariate function `f`,
as well as a list of boxes containing the minimisers (resp. minimisers).

### Input

- `f`         -- function
- `dom`       -- hyperrectangular domain, either a unidimensional  `Interval` or
                 a multidimensional `IntervalBox`
- `structure` -- (optional, default: `HeapedVector`) the way in which vector elements
                 are kept arranged; possible options are `HeapedVector` and `SortedVector`
- `tol`       -- tolerance to which the optima are computed

### Output

The 4-tuple `(global_min, minimisers, global_max, maximisers)`, where:

- `global_min`: interval that is guaranteed to contain the global minimum of `f`
                over `dom`
- `minimisers`: list of boxes containing the minimisers of `f` over `dom`
- `global_max`: interval that is guaranteed to contain the global maximum of `f`
                over `dom`
- `maximisers`: list of boxes containing the maximisers of `f` over `dom`

### Algorithm

The solver finds the global minimum of the function `f` over the `Interval` or
`IntervalBox` `dom` using the Moore-Skelboe algorithm.

We refer to the documentation and source code of `IntervalOptimisation` for details.
"""
function range_and_optimisers(f::Function, dom::Interval_or_IntervalBox;
                              structure=HeapedVector, tol=1e-3)
    _f = wrap_f(f, dom)
    return _range_and_optimisers(_f, dom, structure, tol)
end

@inline wrap_f(f::Function, dom::Interval) = f

# for multidimensional domains, it is required that f has a single argument
@inline wrap_f(f::Function, dom::IntervalBox) = X -> f(X...)

function _range_and_optimisers(f, dom::Interval_or_IntervalBox, structure, tol)
    global_min, minimisers = minimise(f, dom, structure=structure, tol=tol)
    global_max, maximisers = maximise(f, dom, structure=structure, tol=tol)
    return global_min, minimisers, global_max, maximisers
end

"""
    enclose_IntervalOptimisation(f::Function, dom::Interval_or_IntervalBox;
                                 structure=HeapedVector, tol=1e-3)

Compute a range enclosure using interval optimisation.

### Input

- `f`         -- function
- `dom`       -- hyperrectangular domain, either a unidimensional  `Interval` or
                 a multidimensional `IntervalBox`
- `structure` -- (optional, default: `HeapedVector`) the way in which vector elements
                 are kept arranged; possible options are `HeapedVector` and `SortedVector`
- `tol`       -- tolerance to which the optima are computed

### Output

An interval representing the range enclosure (minimum and maximum) of `f` over
its domain `dom`.

### Algorithm

The solver finds the global minimum of the function `f` over the `Interval` or
`IntervalBox` `dom` using the Moore-Skelboe algorithm. The method actually returns
an *interval* containing the global minimum (resp. an *interval* containing the
global maximum), as well as a list of boxes containing the minimisers
(resp. maximisers); they can be obtained calling [`range_and_optimisers`](@ref).

This function conservatively chooses the infimum (resp. supremum) over each interval.

We refer to the documentation and source code of `IntervalOptimisation` for details
on the implemented methods.
"""
function enclose_IntervalOptimisation(f::Function, dom::Interval_or_IntervalBox;
                                      structure=HeapedVector, tol=1e-3)

    global_min, _, global_max, _ = range_and_optimisers(f, dom,
                                                        structure=structure, tol=tol)

    return Interval(inf(global_min), sup(global_max))
end


function kerneldensity(xdata::RealVector; xeval::RealVector=xdata, lb::Real=-Inf, ub::Real=Inf, kernel::Function=gaussiankernel, h::Real=-Inf)

    xdata, xeval, kernel = boundit(xdata, xeval, kernel, lb, ub)
    if h <= 0
        h = bwlscv(xdata, kernel)
    end
    n = length(xdata)
    w = zeros(n)
    den = zeros(length(xeval))
    for i in 1:length(xeval)
        kernel(xeval[i], xdata, h, w, n)
        den[i] = mean(w)
    end
    return den
end

function kerneldensity(xdata::RealMatrix; xeval::RealMatrix=xdata,
    kernel::Vector=[gaussiankernel for i in 1:size(xdata)[2]], h::RealVector=-Inf .* ones(size(xdata, 2)))

    if any(h .<= 0)
        h = bwlcv(xdata, kernel)
        warn("The user are responsible for giving the bandwidth! The defaults may not work well.")
    end
    m, p=size(xeval)
    n, p1 = size(xdata)
    if length(h) != p || p1 != p
        error("xeval and h should have same dimension as xdata")
    end
    den=zeros(m)
    for i=1:m
        wtmp = zeros(n)
        w = ones(n)
        for j=1:p
            kernel[j](xeval[i, j], xdata[:, j], h[j], wtmp, n)
            for k in 1:n
                w[k] = w[k] * wtmp[k]
            end
        end
        den[i] = mean(w)
    end
    den
end

kde=kerneldensity


# PROJECT_CONSENSUS    Bring a set of points into consensus.
#
#   project_consensus(v1,v2,...,vn) returns the elementwise
#   average of v1, ..., vn, i.e., the projection of the vi
#   onto the consensus set.

function project_consensus(varargin)
    N = length(varargin)
    x = zeros(size(varargin[1]))

    for i = 1:N
        x = x + varargin[i]
    end
    x = x ./ N

    return x
end


export ContinuousWave
export GaussianPulse, DGaussianPulse
export BlackmanHarrisPulse, DBlackmanHarrisPulse

function fourier(t,signal)
	δt = step(t) ; N = length(t)
	signal_fft_mag = abs.(fft(signal))* δt
	freqs = fftfreq(N,1/δt)
	return freqs, signal_fft_mag
end

abstract type AbstractWaveform end

include("./Waveforms/ContinuousWaveForm.jl")
include("./Waveforms/GaussianWaveForm.jl")
include("./Waveforms/BlackmanHarrisWaveForm.jl")


#module Vector2d

importall Base
export Vector2D, convert, dot, getindex, norm, normsq, normalize

@doc """A simple type for a 2-component vector.
Previous tests seemed to show that this version was faster than
e.g. ImmutableArrays. But these performance tests should be redone.

This should presumably be a subtype of an abstract array type to avoid
so much rewriting of preexisting methods. """ ->
immutable Vector2D{T}
	x::T
	y::T
end

Vector2D{T}(v::Vector{T}) = Vector2D{T}(v[1], v[2])
#Vector2D{T}(v::Vector) = Vector2D{T}(v[1], v[2])
#Vector2D(v::Vector) = Vector2D(v[1], v[2])

+(v::Vector2D, w::Vector2D) = Vector2D(v.x+w.x, v.y+w.y)
+{T}(v::Vector2D{T}, w::Vector2D{T}) = Vector2D{T}(v.x+w.x, v.y+w.y)
#+{S,T}(v::Vector2D{S}, w::Vector2D{T}) = Vector2D{promote_rule(T,S)}(v.x+w.x, v.y+w.y)
-{T}(v::Vector2D{T}, w::Vector2D{T}) = Vector2D{T}(v.x-w.x, v.y-w.y)
*{T}(v::Vector2D{T}, lamb::Number) = Vector2D{T}(v.x*lamb, v.y*lamb)
*{T}(lamb::Number, v::Vector2D{T}) = v*lamb
/{T}(v::Vector2D{T}, lamb::Number) = Vector2D{T}(v.x/lamb, v.y/lamb)

=={T}(v::Vector2D{T}, w::Vector2D{T}) = v.x==w.x && v.y==w.y

size{T}(v::Vector2D{T}) = 2
dot{T}(v::Vector2D{T}, w::Vector2D{T}) = v.x*w.x + v.y*w.y
norm{T}(v::Vector2D{T}) = √(v ⋅ v)
normsq{T}(v::Vector2D{T}) = v ⋅ v
normalize{T}(v::Vector2D{T}) = v / norm(v)

getindex{T}(v::Vector2D{T}, i) = (i==1) ? v.x : v.y

atan2(v::Vector2D) = atan2(v[2], v[1])  # no need to remember the order in atan2

#There are problems with this `convert`
convert{T}(::Type{Vector2D{T}}, v::Vector{T}) = Vector2D{T}(v[1], v[2])
convert(::Type{Vector2D}, v::Vector) = Vector2D(v[1], v[2])

#end


# Test variational information

using Base.Test
using Clustering

a1 = [1, 1, 1, 2, 2, 2, 3, 3, 3, 3]
a2 = [1, 1, 1, 1, 2, 2, 2, 2, 2, 2]
a3 = [3, 3, 3, 2, 2, 2, 1, 1, 1, 1]

(ARI, RI, MI, HI) = randindex(a1, a1)

@test_approx_eq_eps ARI 1.0 1.0e-12
@test_approx_eq_eps RI  1.0 1.0e-12
@test_approx_eq_eps MI  0.0 1.0e-12
@test_approx_eq_eps HI  1.0 1.0e-12

(ARI, RI, MI, HI) = randindex(a1, a3)

@test_approx_eq_eps ARI 1.0 1.0e-12
@test_approx_eq_eps RI  1.0 1.0e-12
@test_approx_eq_eps MI  0.0 1.0e-12
@test_approx_eq_eps HI  1.0 1.0e-12

(ARI, RI, MI, HI) = randindex(a1, a2)

@test_approx_eq_eps ARI 0.403669 1.0e-5
@test_approx_eq_eps RI  0.711111 1.0e-5
@test_approx_eq_eps MI  0.288888 1.0e-5
@test_approx_eq_eps HI  0.422222 1.0e-5

@test randindex(a1, a2) == randindex(a2, a1)


include("euler/euler.jl")
using .Math: fast_min
using BenchmarkTools
BenchmarkTools.DEFAULT_PARAMETERS.samples = 100

function compute(grid::Array{Int,2})::Int
    edge = size(grid)[1]
    for i ∈ edge - 1:-1:1
        grid[edge, i] += grid[edge, i + 1]
        grid[i, edge] += grid[i + 1, edge]
    end
    for i ∈ edge - 1:-1:1 for j ∈ edge - 1:-1:1 grid[i, j] += fast_min(grid[i + 1, j], grid[i, j + 1]) end end
    return grid[1, 1]
end

matrix = [131 673 234 103 18;
          201 96 342 965 150;
          630 803 746 422 111;
          537 699 497 121 956;
          805 732 524 37 331]

path = "problems/0081/p081_matrix.txt"

compute(matrix)

compute(parse.(Int, hcat(split.(split(read(path, String), "\n"), ",")...)))

@benchmark compute(parse.(Int, hcat(split.(split(read(path, String), "\n"), ",")...)))


using Images, ImageView, Statistics

# function to return magnitude of image elements
function brightness(img_element::AbstractRGB)
    return mean((img_element.r + img_element.g + img_element.b))
end

function find_energy(img)
    energy_x = imfilter(brightness.(img), Kernel.sobel()[2])
    energy_y = imfilter(brightness.(img), Kernel.sobel()[1])
    return sqrt.(energy_x.^2 + energy_y.^2)
end

function find_energy_map(energy)
    energy_map = zeros(size(energy))
    energy_map[end,:] = energy[end,:]
    next_elements = zeros(Int, size(energy))

    for i = size(energy)[1]-1:-1:1
        for j = 1:size(energy)[2]
            left = max(1, j-1)
            right = min(j+1, size(energy)[2])

            local_energy. next_element = findmin(energy_map[i+1, left:right])

            energy_map[i,j] = local_energy + energy[i,j]

            next_elements[i,j] = next_element - 2

            if left == 1
                next_elements[i,j] += 1
            end
        end
    end
    return energy_map, next_elements
end

function find_seam_at(next_elements, element)
    seam = zeros(Int, size(next_elements)[1])
    seam[1] = element

    for i = 2:length(seam)
        seam[1] = seam[i-1] + next_elements[i, seam[i-1]]
    end
    return seam
end

# function to create seams and return seam of minimum energy
function find_seam(energy)
    energy_map, next_elements = find_energy_map(energy)
    _, min_element = findmin(energy_map[1,:])
    return find_seam_at(next_elements, min_element)
end

#function to remove seam
function remove_seam(img, seam)
    im_res = (size(img)[i], size(img)[2]-1)

    # preallocate image
    new_img = Array{RGB}(undef, img_res)

    for i = 1:length(seam)
        if seam[i] > 1 && seam[i] < size(img)[2]
            new_img[i, :] .= vcat(img[i, 1:seam[i]-1],
                                  img[i, seam[i]+1:end])
        elseif seam[i] == 1
            new_img[i, :] .= img[i,2:end]
        elseif seam[i] == size(img)[2]
            new_img[i, :] .= img[i, 1:end-1]
        end
    end

    return new_img
end

function seam_carver(img, res)
    if res < 0 || res > size(img)[2]
        error("resolution not acceptable")
    end

    for i = (1:size(img)[2] - res)
        energy = find_energy(img)
        seam = find_seam(energy)
        img = remove_seam(img, seam)
    end
    return img
end


#=
Generate random NormalFormGame instances.
=#

#
# Random Games Generating
#
"""
    random_game([rng=GLOBAL_RNG], nums_actions)

Return a random N-player NormalFormGame instance where the
payoffs are drawn independently from the uniform distribution
on [0, 1).

# Arguments

- `rng::AbstractRNG=GLOBAL_RNG`: Random number generator used.
- `nums_actions::NTuple{N,Int}`: Tuple of the numbers of actions,
  one for each player.

# Returns

- `::NormalFormGame`: The generated random N-player NormalFormGame.
"""
function random_game(rng::AbstractRNG, nums_actions::NTuple{N,Int}) where N
    if N == 0
        throw(ArgumentError("nums_actions must be non-empty"))
    end

    players::NTuple{N,Player{N,Float64}} =
        ntuple(i -> Player(rand(rng, Float64, tuple(nums_actions[i:end]...,
                                                    nums_actions[1:i-1]...))),
               N)

    return NormalFormGame(players)
end

random_game(nums_actions::NTuple{N,Int}) where {N} =
    random_game(Random.GLOBAL_RNG, nums_actions)

#
# Covariance Games Generating
#
"""
    covariance_game([rng=GLOBAL_RNG], nums_actions, rho)

Return a random N-player NormalFormGame instance with N>=2 where
the payoff profiles are drawn independently from the standard
multi-normal with the covariance of any pair of payoffs equal to
`rho`, as studied in Rinott and Scarsini (2000).

# Arguments

- `rng::AbstractRNG=GLOBAL_RNG`: Random number generator used.
- `nums_actions::NTuple{N,Int}`: Tuple of the numbers of actions,
  one for each player.
- `rho::Real`: Covariance of a pair of payoff values. Must be in
  [-1/(N-1), 1], where N is the number of players.

# Returns

- `::NormalFormGame`: The generated random N-player NormalFormGame.

# References

- Y. Rinott and M. Scarsini, "On the Number of Pure Strategy
  Nash Equilibria in Random Games," Games and Economic Behavior
  (2000), 274-293.
"""
function covariance_game(rng::AbstractRNG, nums_actions::NTuple{N,Int},
                         rho::Real) where N
    if N <= 1
        throw(ArgumentError("length of nums_actions must be at least 2"))
    end

    if !(-1 / (N - 1) <= rho <= 1)
        lb = (N == 2) ? "-1" : "-1/$(N-1)"
        throw(ArgumentError("rho must be in [$lb, 1]"))
    end

    mu = zeros(N)
    Sigma = fill(rho, (N, N))
    Sigma[diagind(Sigma)] = ones(N)

    d = MVNSampler(mu, Sigma)
    x = rand(rng, d, prod(nums_actions))

    x_T = Matrix{eltype(x)}(undef, prod(nums_actions), N)
    transpose!(x_T, x)
    payoff_profile_array =
        reshape(x_T, (nums_actions..., N))

    return NormalFormGame(payoff_profile_array)
end

covariance_game(nums_actions::NTuple{N,Int}, rho::Real) where {N} =
    covariance_game(Random.GLOBAL_RNG, nums_actions, rho)

#
# Random action profile
#
"""
    random_pure_actions([rng=GLOBAL_RNG], nums_actions)

Return a tuple of random pure actions (integers).

# Arguments

- `rng::AbstractRNG=GLOBAL_RNG`: Random number generator used.
- `nums_actions::NTuple{N,Int}`: N-tuple of the numbers of actions,
  one for each player.

# Returns

- `::NTuple{N,Int}`: N-tuple of random pure actions.

"""
random_pure_actions(rng::AbstractRNG, nums_actions::NTuple{N,Int}) where {N} =
    ntuple(i -> rand(rng, 1:nums_actions[i]), Val(N))

random_pure_actions(nums_actions::NTuple{N,Int}) where {N} =
    random_pure_actions(Random.GLOBAL_RNG, nums_actions)

"""
    random_mixed_actions([rng=GLOBAL_RNG], nums_actions)

Return a tuple of random mixed actions (vectors of floats).

# Arguments

- `rng::AbstractRNG=GLOBAL_RNG`: Random number generator used.
- `nums_actions::NTuple{N,Int}`: N-tuple of the numbers of actions,
  one for each player.

# Returns

- `::NTuple{N,Vector{Float64}}`: N-tuple of random mixed actions.
"""
random_mixed_actions(rng::AbstractRNG, nums_actions::NTuple{N,Int}) where {N} =
    ntuple(i -> QuantEcon.random_probvec(rng, nums_actions[i]), Val(N))

random_mixed_actions(nums_actions::NTuple{N,Int}) where {N} =
    random_mixed_actions(Random.GLOBAL_RNG, nums_actions)


"""
    gaussian_kernel_hyperparameters
This contains the hyperparameters for the gaussian kernel. The functional form for correlation between x_p and x_q   is   w_0 exp( -0.5 * \\sum_{d=1}^D ((x_{p,d} - x_{q,d})/w_d)^2)
    where D is the number of dimensions. and each term of the summation is a different dimension of x_p and x_q.
"""
struct GaussianKernelHyperparameters{T<:Real}
    w_0::T
    w_i::Array{T,1}
    function GaussianKernelHyperparameters(w_0::S,w_i::Array{G,1}) where S<:Real where G<:Real
        promo = promote_type(S,G)
        return new{promo}(promo(w_0), promo.(w_i))
    end
    function GaussianKernelHyperparameters{S}(aa::GaussianKernelHyperparameters{G}) where S<:Real where G<:Real
        return new{S}(S(aa.w_0), S.(aa.w_i))
    end
end

"""
    KrigingModel
    Our kriging model is defined by a level \\mu, a set of X coordinates and corresponding y values, a set of hyperparameters and a noise level.
In addition we save a number of matrices in order to faciliate computations. One is
This contains the hyperparameters for the gaussian kernel.
"""

struct KrigingModel{T<:Real}
    X::AbstractArray{T,2}
    f::AbstractArray{T,1}
    hyper::GaussianKernelHyperparameters{T}
    cov_func::Function
    noise::T
    # Now we have stuff that is calculated from the above. Saved in the model purely for efficiency reasons.
    K::Hermitian{T}
    invK::Hermitian{T}
    mu::T
    sigma2::T
    y_m_mu::Array{T,1}
    function KrigingModel(X::AbstractArray{T,2}, f::AbstractArray{R,1}, hyper::GaussianKernelHyperparameters{S};
                          cov_func::Function = gaussian_kernel, noise::U = 1000*eps(),
                          K::Hermitian{W} = K_matrix(X, cov_func, hyper, noise), invK::Hermitian{Z} = LinearAlgebra.inv(K)) where T<:Real where R<:Real where S<:Real where U<:Real where W<:Real where Z<:Real
            tt = promote_type(T, R, S, U, W, Z)

            dim = size(invK)[1]
            mu = tt(sum(invK * f) / sum(invK * ones(dim)))
            y_m_mu = f .- mu
            sigma2 = tt((y_m_mu' * (invK * y_m_mu)) / dim)
            return new{tt}(Array{tt,2}(X), Array{tt,1}(f), GaussianKernelHyperparameters{tt}(hyper) , cov_func, tt(noise), LinearAlgebra.Hermitian(Array{tt,2}(K)), LinearAlgebra.Hermitian(Array{tt,2}(invK)), mu, sigma2, y_m_mu)
    end
end


# Methods for HMM for parsing reads of sequences.

"""
Column index for each nucleotide in observation matrix.
"""
const NUCLEOTIDE_COLS = Dict('A' => UInt8(1),
                             'C' => UInt8(2),
                             'G' => UInt8(3),
                             'T' => UInt8(4));

"""
    viterbi_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})

Return viterbi path and log probability for that path. Takes logs of matrices.
`observations_given_states` has # rows = # states, # columns = # steps of markov process.
"""
function viterbi_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})
    numstates, numsteps = size(observations_given_states)
    scores = zeros(numstates, numsteps)
    paths = zeros(scores, UInt16)
    # Forward Score propogation
    scores[:, 1] = initials + observations_given_states[:, 1]
    for t in 2:numsteps
        for j in 1:numstates
            transitioned_scores = scores[:, t-1] + transitions[:, j]
            paths[j, t] = indmax(transitioned_scores)
            scores[j, t] = transitioned_scores[paths[j, t]] + observations_given_states[j, t]
        end
    end
    # Backtrack
    path = zeros(Int, numsteps)
    probs = zeros(path, Float64)
    path[end] = indmax(scores[:, end])
    probs[end] = scores[path[end], end]
    for t in numsteps-1:-1:1
        path[t] = paths[path[t+1], t+1]
        probs[t] = scores[path[t+1], t+1]
    end
    return path, maximum(scores[:, end])
end

"""
    trans_mat(; uniform_cycle_prob = 0.9999999999, homopoly_cycle_prob = 0.98)

Return 5x5 transition matrix with given transition probabilities
State 1 -- uniform observation distribution.
State 2 -- High "A" observation likelihood.
State 3 -- High "C" observation likelihood.
State 4 -- High "G" observation likelihood.
State 5 -- High "T" observation likelihood.
Each state has high likelihood to return to state 1.
States 2 through 5 represent high likelihood of homopolymer regions of respective nuc."""
function trans_mat(; uniform_cycle_prob = 0.9999999999, homopoly_cycle_prob = 0.98)
    uh_prob = (1-uniform_cycle_prob) / 4
    hu_prob = 1 - homopoly_cycle_prob
    T = [
        uniform_cycle_prob uh_prob uh_prob uh_prob uh_prob;

        hu_prob homopoly_cycle_prob 0 0 0;

        hu_prob 0 homopoly_cycle_prob 0 0;

        hu_prob 0 0 homopoly_cycle_prob 0;

        hu_prob 0 0 0 homopoly_cycle_prob;
    ]
    return T
end

"""
    obs_mat(; homopoly_prob = 0.99)

Return 5x4 observation matrix with given probabilities.
State 1 -- uniform observation distribution.
State 2 -- High "A" observation likelihood.
State 3 -- High "C" observation likelihood.
State 4 -- High "G" observation likelihood.
State 5 -- High "T" observation likelihood.
Columns are in same order as columns 2 through 5
(the last four rows give a symmetric matrix).
"""
function obs_mat(; homopoly_prob = 0.99)
    uniform_prob = 0.25
    not_hpoly_prob = (1-homopoly_prob) / 3
    O = [
        uniform_prob uniform_prob uniform_prob uniform_prob;

        homopoly_prob  not_hpoly_prob not_hpoly_prob not_hpoly_prob;

        not_hpoly_prob homopoly_prob  not_hpoly_prob not_hpoly_prob;

        not_hpoly_prob not_hpoly_prob homopoly_prob  not_hpoly_prob;

        not_hpoly_prob not_hpoly_prob not_hpoly_prob homopoly_prob;
    ]
    return O
end

"""
    initial_dist(; uniform_state = 0.99)

Initial state distributions: see `trans_mat` for state descriptions. 5x1 vector.
"""
function initial_dist(; uniform_state = 0.99)
    not_unistate = (1 - uniform_state) / 4
    return [uniform_state; not_unistate; not_unistate; not_unistate; not_unistate]
end

"""
    get_obs_given_state(observation_matrix::Array{Float64,2}, observation_seq::String)

Populate 5xT matrix with likelihood of observation at each of T time steps given each
nucleotide in `observation_seq`.
"""
function get_obs_given_state(observation_matrix::Array{Float64,2}, observation_seq::String)
    obs_given_state = zeros(length(observation_seq), 5)
    for (i, nuc) in enumerate(observation_seq)
        obs_given_state[i, :] = observation_matrix[:, NUCLEOTIDE_COLS[nuc]]
    end
    return transpose(obs_given_state)
end

homopolymer_filter(seqs::Array{String}) = homopolymer_filter(seqs, [], [])[1]
markov_filter = homopolymer_filter

"""
    homopolymer_filter(seqs::Array{String,1}, phreds, names;
                       transmat = nothing, obsmat = nothing,
                       initialdist = nothing)

Filter sequences with "bad" sections in the middle -- abnormally long runs of a single base, using
viterbi alg inference. If this homopolymer occurs on one end of a sequence, keeps sequence and trims
homopolymer region off end. `phreds` and/or `names` may be `nothing`, in which case `nothing` is returned
for the respective field. If transition, observation, initial distribution matrices not provided (default)
then the default values from the respective constructors are used.
"""
function homopolymer_filter(seqs::Array{String,1}, phreds, names;
                       transmat = nothing, obsmat = nothing,
                       initialdist = nothing)
    newseqs = String[]
    # return empty arrays for newphreds, newnames if params were empty arrays
    # otherwise if params were nothing, return nothings
    newphreds= phreds==nothing ? nothing : Vector{Phred}[]
    newnames = names==nothing ? nothing : String[]
    phredsexist = (phreds != nothing) && (length(phreds) == length(seqs))
    namesexist = (names != nothing) && (length(names) == length(seqs))
    if (!phredsexist && phreds != nothing && length(phreds) > 0) || (!namesexist && names != nothing && length(names) > 0)
        error("Dimension mismatch in names or phreds array")
    end
    if transmat == nothing
        transmat = trans_mat()
    end
    if obsmat == nothing
        obsmat = obs_mat()
    end
    if initialdist == nothing
        initialdist = initial_dist()
    end
    T_logs, O_logs, i_logs = log.(transmat), log.(obsmat), log.(initialdist)
    for i in 1:length(seqs)
        seq = seqs[i]
        path = viterbi_logs(get_obs_given_state(O_logs, seq), T_logs, i_logs)[1]
        # remove bad parts
        lo = 1
        while lo < length(path) && path[lo] != 1
            lo += 1
        end
        hi = length(path)
        while hi > 0 && path[hi] != 1
            hi -= 1
        end
        if lo != 1 || hi != length(path)
            path = path[lo:hi]
        end
        # make sure no bad parts in middle
        ind = findnext(x->x != 1, path, lo)
        if ind > hi || ind == 0
            push!(newseqs, seq[lo:hi])
            if phredsexist
                push!(newphreds, phreds[i][lo:hi])
            end
            if namesexist
                push!(newnames, names[i])
            end
        end
    end
    # may return empty arrays for newphreds and newnames
    return newseqs, newphreds, newnames
end

"""
    homopolymer_filter(sourcepath::String, destpath::String;
                       transmat = nothing, obsmat = nothing,
                       initialdist = nothing, format="fastq")

Filter sequences with "bad" sections in the middle -- abnormally long runs of a single base, using
viterbi alg inference. If this homopolymer occurs on one end of a sequence, keeps sequence and trims
homopolymer region off end. Takes a file path for each of a source file of type `format` (which may be "fasta" or "fastq")
and a destination file is written which is the same file type.
"""
function homopolymer_filter(sourcepath::String, destpath::String;
                       transmat = nothing, obsmat = nothing,
                       initialdist = nothing, format="fastq")
    seqs, phreds, names = String[], Vector{Phred}[], String[]
    if format == "fastq"
        seqs, phreds, names = read_fastq(sourcepath, seqtype=String)
    elseif format == "fasta"
        names, seqs = read_fasta_with_names(sourcepath, seqtype=String)
    else
        println("Invalid format! File not written.")
    end
    println("Cleaning $(length(seqs)) sequences")
    newseqs, newphreds, newnames = homopolymer_filter(seqs, phreds, names,
                                                 transmat=transmat, obsmat=obsmat,
                                                 initialdist=initialdist)
    println("Writing $(length(newseqs)) sequences")
    if format == "fastq"
        write_fastq(destpath, newseqs, newphreds, names=newnames, DNASeqType=false)
    else
        write_fasta(destpath, newseqs, names=newnames, DNASeqType=false)
    end
end

"""
    forward_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})

Compute logs of forward scores. Takes logs of matrices as inputs.
`observations_given_states` has # rows = # states, # columns = # steps of markov process.
"""
function forward_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})
    numstates, numsteps = size(observations_given_states)
    fwds = zeros(numstates, numsteps)
    # Forward pass - (unnormalized?) probs of states at time t given observations up to t
    fwds[:, 1] = initials + observations_given_states[:, 1]
    for t in 2:numsteps
        for j in 1:numstates
            prob_state_given_prev = reduce(logsum, fwds[:, t-1] + transitions[:, j])
            fwds[j, t] = prob_state_given_prev + observations_given_states[j, t]
        end
    end
    return fwds
end

"""
    backward_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})

Compute logs of backward scores and individual posterior probabilities. Takes logs of matrices as inputs.
`observations_given_states` has # rows = # states, # columns = # steps of markov process.
"""
function backward_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})
    numstates, numsteps = size(observations_given_states)
    bwds = zeros(numstates, numsteps)
    # Backwards pass - (unnormalized?) probs of states at time t given observations after t
    bwds[:, end] = 1  # arbitrarily initialized
    log_posteriors = zeros(numstates, numsteps)
    for t in numsteps-1:-1:1
        for i in 1:numstates
            bwds[i, t] = reduce(logsum, bwds[:, t+1] + transitions[i, :] + observations_given_states[:, t+1])
        end
    end
    return bwds
end

"""
    forward_backward_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})

Compute logs of forward-backward scores and individual probabilities. Takes logs of matrices as inputs.
`observations_given_states` has # rows = # states, # columns = # steps of markov process.
"""
function forward_backward_logs(observations_given_states::Array{Float64, 2}, transitions::Array{Float64, 2}, initials::Array{Float64})
    numstates, numsteps = size(observations_given_states)
    fwds = forward_logs(observations_given_states, transitions, initials)
    bwds = zeros(numstates, numsteps)
    # Backwards pass - Compute manually in order to compute individual probabilities simultaneously
    # - saves an extra pass through observations arrays
    bwds[:, end] = 1  # arbitrarily initialized
    log_posteriors = zeros(numstates, numsteps)
    for t in numsteps-1:-1:1
        for i in 1:numstates
            bwds[i, t] = reduce(logsum, bwds[:, t+1] + transitions[i, :] + observations_given_states[:, t+1])
        end
        # and combine with fwds to get posterior probability logs
        fb_probs = fwds[:, t] + bwds[:, t]
        norm = reduce(logsum, fb_probs)
        log_posteriors[:, t] = fb_probs - norm
    end
    # and for the end
    fb_probs = fwds[:, end] + bwds[:, end]
    norm = reduce(logsum, fb_probs)
    log_posteriors[:, end] = fb_probs - norm
    return log_posteriors
end

"""
    gen_seq_with_model(n::Int, trans_mat, obs_mat, initial_dists)

Generate a sequence given a markov model. May create bad reads (long runs of a single base).
"""
function gen_seq_with_model(n::Int, trans_mat, obs_mat, initial_dists)
    state_seq = zeros(Int, n)
    state_seq[1] = wsample(collect(1:5), trans_mat*initial_dists)
    obs_seq = wsample(["A", "C", "G", "T"], obs_mat[state_seq[1], :])
    for t in 2:n
        state_seq[t] = wsample(collect(1:5), trans_mat[state_seq[t-1], :])
        obs_seq *= wsample(["A", "C", "G", "T"], obs_mat[state_seq[t], :])
    end
    return obs_seq
end

"""
    viterbiprint(s::String)

Draw flagged sites with capital letters, safe sites with lowercase.
"""
function viterbiprint(s::String)
    obs = get_obs_given_state(obs_mat(), s)
    v = viterbi_logs(log.(obs),log.(trans_mat()),log.(initial_dist()))
    v = v[1]
    s = collect(s)
    for i in 1:length(v)
        if v[i]==1
            s[i] += 32
        end
    end
    return join(s)
end


using Manifolds, Manopt, Test, ManifoldsBase

@testset "Differentials (on Sphere(2))" begin
    # The Adjoint Differentials test using the same variables as the differentials
    # test
    x = [1.0, 0.0, 0.0]
    y = [0.0, 1.0, 0.0]
    M = Sphere(2)
    ξ = log(M, x, y)
    # Text differentials (1) Dx of Log_xy
    @test adjoint_differential_log_basepoint(M, x, x, ξ) == -ξ
    @test adjoint_differential_log_argument(M, x, x, ξ) == ξ
    @test adjoint_differential_exp_basepoint(M, x, zero_tangent_vector(M, x), ξ) == ξ
    @test adjoint_differential_exp_argument(M, x, zero_tangent_vector(M, x), ξ) == ξ
    for t in [0, 0.15, 0.33, 0.66, 0.9]
        @test adjoint_differential_geodesic_startpoint(M, x, x, t, ξ) == (1 - t) * ξ
        @test norm(M, x, adjoint_differential_geodesic_endpoint(M, x, x, t, ξ) - t * ξ) ≈ 0 atol =
            10.0^(-16)
    end
    Mp = PowerManifold(M, NestedPowerRepresentation(), 3)
    xP = [x, y, x]
    yP = [x, x, y]
    ξP = [ξ, zero_tangent_vector(M, x), -ξ]
    @test norm(
        Mp,
        xP,
        adjoint_differential_forward_logs(Mp, xP, ξP) -
        [-ξ, zero_tangent_vector(M, x), zero_tangent_vector(M, x)],
    ) ≈ 0 atol = 4 * 10.0^(-16)
end


using Colors, Distributions, LaTeXStrings, PGFPlotsX

lprob(d, x) = cdf(d, x) - cdf(d, x-1/1000)
xs = map(x -> x/1000, 1:10000)

function seir_var_susp(r0, d, g, k, ep, days, conn)
    gamma_d = Gamma(k, 1/k)
    lprobs = map((x) -> lprob(gamma_d, x), xs[1:9999])
    push!(lprobs, 1 - sum(lprobs))
    st = [(1-ep) .* lprobs]
    et = [ep .* lprobs]
    it = [fill(0.0, 10000)]
    rt = [fill(0.0, 10000)]
    if conn
        b = r0/(1+1/k) * g
    else
        b = r0 * g
    end
    for t in 2:days
        if conn
            l = b * sum(xs .* it[t-1])
        else
            l = b * sum(it[t-1])
        end
        sred = xs .* l .* st[t-1]
        ered = d .* et[t-1]
        ired = g .* it[t-1]
        push!(st, st[t-1] .- sred)
        push!(et, et[t-1] .+ sred .- ered)
        push!(it, it[t-1] .+ ered .- ired)
        push!(rt, rt[t-1] .+ ired)
    end
    Dict(:st => st, :et => et, :it => it, :rt => rt,
        :r0 => r0, :d => d, :g => g, :k => k,
        :ep => ep, :conn => conn)
end

sedec(x) = replace("$x", "."=>"{,}")

function seirset(r0, d, g, kexps, ep, days)
    seirs_s = map(kexp -> seir_var_susp(r0, d, g, 2.0^kexp, ep, days, false), kexps)
    seirs_c = map(kexp -> seir_var_susp(r0, d, g, 2.0^kexp, ep, days, true), kexps)
    Dict(:seirs_s => seirs_s, :seirs_c => seirs_c, :kexps => kexps)
end

seirset_cends(seirset, c) = map(r -> r[end], map(s -> map(sum, s[c]), seirset))

function seirplot(seir)
    r0lab = latexstring(*("\\mathcal{R}_0=", sedec(seir[:r0])))
    klab = latexstring(*("k=", sedec(seir[:k])))
    if seir[:conn]
        connlab = "kontakt"
    else
        connlab = "mottaglighet"
    end
    days = 1:length(seir[:st])
    p = @pgf Axis({"yticklabel style"={"/pgf/number format/use comma"},
        legend_pos="outer north east",
        title="SEIR för $(r0lab), $(klab), varierande $(connlab)", xlabel=L"t",
        ylabel=L"p",
        xmajorgrids, ymajorgrids})
    cs = [("S", seir[:st]), ("E", seir[:et]), ("I", seir[:it]), ("R", seir[:rt])]
    plotcolors = distinguishable_colors(length(cs)+1, [RGB(1,1,1)])[2:end]
    for (i, c) in enumerate(cs)
        @pgf push!(p, PlotInc({no_markers, color = plotcolors[i]},
            Table([days, map(sum, c[2])])), LegendEntry(c[1]))
    end
    p
end

function seirset_rendsplot(seirset)
    seir = seirset[:seirs_s][1]
    r0lab = latexstring(*("\\mathcal{R}_0=", sedec(seir[:r0])))
    rendlab = latexstring("\\mathrm{R}($(length(seir[:st])))")
    p = @pgf Axis({"yticklabel style"={"/pgf/number format/use comma"},
        legend_pos="outer north east",
        title="SEIR för $(r0lab)", xlabel=L"\lg(k)",
        ylabel=rendlab,
        xmajorgrids, ymajorgrids})
    @pgf push!(p, PlotInc(Table([seirset[:kexps], seirset_cends(seirset[:seirs_s], :rt)])),
        LegendEntry("mottaglighet"))
    @pgf push!(p, PlotInc(Table([seirset[:kexps], seirset_cends(seirset[:seirs_c], :rt)])),
        LegendEntry("kontakt"))
    p
end


module CovSelTest

using Test, Random, LinearAlgebra, Distributions
import CovSel, ProximalBase
using MathOptInterface
const MOI = MathOptInterface
using JuMP, SCS

Random.seed!(123)

@testset "random" begin
  for REP=1:10

      n = 1000
      p = 10

      ρ = 0.5
      covMat = Matrix(1.0I, p, p)
      for a = 1:p
        for b = a+1:p
          t = ρ^(b - a)
          covMat[a, b] = t
          covMat[b, a] = t
        end
      end
      precM = inv(covMat)

      sqCov = sqrt(covMat)
      X = randn(n, p) * sqCov

      S = X' * X / n

      λ = 0.5

      X = zeros(Float64, (p,p))
      Z = zeros(Float64, (p,p))
      U = zeros(Float64, (p,p))
      CovSel.covsel!(X, Z, U, S, λ; penalize_diag=true)

      # passing in verbose=0 to hide output from SCS
      problem = Model(JuMP.optimizer_with_attributes(SCS.Optimizer, "verbose" => 0))
      @variable(problem, Ω[1:p, 1:p], PSD)
      @variable(problem, lg_det)
      @variable(problem, B[1:p, 1:p])

      indOffDiag = Vector{Int64}(undef, 0)
      ind = 0
      for col = 1:p
          for row = 1:p
              ind += 1
              if row <= col
                  push!(indOffDiag, ind)
              end
          end
      end

      @constraint(problem, [lg_det; 1; Ω[indOffDiag]] in MOI.LogDetConeTriangle(p))
      @constraint(problem, Ω .<= B)
      @constraint(problem, -Ω .<= B)

      @objective(problem, Min, tr(Ω*S) - lg_det + λ * sum(B))
      optimize!(problem)

      Ωv = JuMP.value.(Ω)

      @test tr(Z*S) - logdet(Z) + λ * sum(abs, Z) ≈ tr(Ωv*S) - logdet(Ωv) + λ * sum(abs, Ωv) atol = 1e-3
      @test JuMP.value.(Ω) - Z ≈ zeros(p,p) atol = 1e-2
  end
end

@testset "non random" begin
    p = 10
    S = Matrix(1.0I, p, p)

    λ = 0.5

    X = zeros(Float64, (p,p))
    Z = zeros(Float64, (p,p))
    U = zeros(Float64, (p,p))
    CovSel.covsel!(X, Z, U, S, λ; penalize_diag=false)
    @test Z ≈ S atol=1e-3

    CovSel.covsel!(X, Z, U, S, λ; penalize_diag=false, options=CovSel.ADMMOptions(;abstol=1e-12,reltol=1e-12))
    @test Z ≈ S
end

end