"""
This  package,  which  depends  on  no other package, implements univariate
Laurent  polynomials  (type  `Pol{T}`),  and  univariate rational fractions
(type `Frac{Pol{T}}`).

The  initial motivation was to have a simple way to port GAP polynomials to
Julia. The reasons for still having my own package are multiple:

  - I need  to have  a simple  and flexible  interface, which  I hope this
    provides.
  - I need my polynomials to behave  well when coefficients are in a ring,
    in which case I use pseudo-division and subresultant gcd.
  - For my uses my polynomials are several times faster than those in the
    package `Polynomials`.
  - I need my polynomials to  work as well as possible with coefficients of
    types  `T` where elements have a `zero`  method but `T` itself does not
    have  one (because `T` does not  contain the necessary information). An
    example is modular arithmetic with a `BigInt` modulus which thus cannot
    be part of the type.
  - a final justification is that `LaurentPolynomials` is used by `PuiseuxPolynomials`.

Laurent polynomials have the parametric type `Pol{T}`, where `T`is the type
of   the  coefficients.  They  are  constructed   by  giving  a  vector  of
coefficients  of  type  `T`,  and  a  valuation  (an  `Int`).  We call true
polynomials those whose valuation is `≥0`.

There  is a  current variable  name (a  `Symbol`) used to print polynomials
nicely  at  the  repl  or  in  IJulia  or  Pluto.  This name can be changed
globally,  or just changed for printing a given polynomial. But polynomials
do not record individually with which symbol they should be printed.

# Examples
```julia-repl
julia> Pol(:q) # define symbol used for printing and return Pol([1],1)
Pol{Int64}: q

julia> @Pol q  # same as q=Pol(:q)  useful to start session with polynomials
Pol{Int64}: q

julia> Pol([1,2]) # valuation is taken to be 0 if omitted
Pol{Int64}: 2q+1

julia> 2q+1       # same polynomial
Pol{Int64}: 2q+1

julia> Pol()   # omitting all arguments gives Pol([1],1)
Pol{Int64}: q

julia> p=Pol([1,2,1],-1) # here the valuation is specified to be -1
Pol{Int64}: q+2+q⁻¹

julia> q+2+q^-1 # same polynomial
Pol{Int64}: q+2+q⁻¹
```

```julia-rep1
julia> print(p) # if not nice printing give an output which can be read back
Pol([1, 2, 1],-1)

# change the variable for printing just this time
julia> print(IOContext(stdout,:limit=>true,:varname=>"x"),p)
x+2+x⁻¹

julia> print(IOContext(stdout,:TeX=>true),p) # TeXable output (used in Pluto, IJulia)
q+2+q^{-1}
```

A  polynomial can be  taken apart with  the functions `valuation`, `degree`
and `getindex`. An index `p[i]` gives the coefficient of degree `i` of `p`.

```julia-repl
julia> valuation(p),degree(p)
(-1, 1)

julia> p[0], p[1], p[-1], p[10]
(2, 1, 1, 0)

julia> p[valuation(p):degree(p)]
3-element Vector{Int64}:
 1
 2
 1

julia> p[begin:end]  # the same as the above line
3-element Vector{Int64}:
 1
 2
 1

julia> coefficients(p)  # the same again
3-element Vector{Int64}:
 1
 2
 1
```

A  polynomial  is  a  *scalar*  if  the  valuation  and degree are `0`. The
function  `scalar` returns the constant coefficient  if the polynomial is a
scalar, and `nothing` otherwise.

```julia-repl
julia> Pol(1)
Pol{Int64}: 1

julia> convert(Pol{Int},1) # the same thing
Pol{Int64}: 1

julia> scalar(Pol(1))
1

julia> convert(Int,Pol(1)) # the same thing
1

julia> Int(Pol(1))         # the same thing
1

julia> scalar(q+1) # nothing; convert would give an error
```

In arrays `Pol{T}` of different types `T` are promoted to the same type `T`
(when  the `T`  involved have  a promotion)  and a  number is promoted to a
polynomial.

Usual  arithmetic (`+`, `-`,  `*`, `^`, `/`,  `//`, `one`, `isone`, `zero`,
`iszero`,  `==`) works. Elements  of type `<:Number`  or of type  `T` for a
`Pol{T}`   are  considered  as   scalars  for  scalar   operations  on  the
coefficients.

```julia-repl
julia> derivative(p)
Pol{Int64}: 1-q⁻²

julia> p=(q+1)^2
Pol{Int64}: q²+2q+1

julia> p/2
Pol{Float64}: 0.5q²+1.0q+0.5

julia> p//2
Pol{Rational{Int64}}: (1//2)q²+(1//1)q+1//2

julia> p(1//2) # value of p at 1//2
9//4

julia> p(0.5)
2.25

# interpolation: find p taking values [2.0,1.0,3.0] at [1,2,3]
julia> Pol([1,2,3],[2.0,1.0,3.0])
Pol{Float64}: 1.5q²-5.5q+6.0
```

Polynomials  are scalars  for broadcasting.  They can  be sorted (they have
`cmp`   and  `isless`  functions  which   compare  the  valuation  and  the
coefficients), they can be keys in a `Dict` (they have a `hash` function).

The  functions  `divrem`,  `div`,  `%`,  `gcd`,  `gcdx`,  `lcm`, `powermod`
operate  between  true  polynomials  over  a  field,  using  the polynomial
division.  Over a ring it is better  to use `pseudodiv` and `srgcd` instead
of  `divrem`  and  `gcd`  (by  default  `gcd`  between  integer polynomials
delegates to `srgcd`).

`exactdiv`  does  division  (over  a  field  or  a  ring) when it is exact,
otherwise gives an error.

```julia-repl
julia> divrem(q^3+1,2q+1) # changes coefficients to field elements
(0.5q²-0.25q+0.125, 0.875)

julia> divrem(q^3+1,2q+1//1) # case of coefficients already field elements
((1//2)q²+(-1//4)q+1//8, 7//8)

julia> pseudodiv(q^3+1,2q+1) # pseudo-division keeps the ring
(4q²-2q+1, 7)

julia> (4q^2-2q+1)*(2q+1)+7 # but multiplying back gives a multiple of the polynomial
Pol{Int64}: 8q³+8

julia> exactdiv(q+1,2.0) # exactdiv(q+1,2) would give an error
Pol{Float64}: 0.5q+0.5
```

Finally,   `Pol`s  have   methods  `conj`,   `adjoint`  which   operate  on
coefficients,  methods `positive_part`,  `negative_part` and  `bar` (useful
for  Kazhdan-Lusztig  theory)  and  a  method  `randpol`  to produce random
polynomials.

Inverting polynomials is a way to get a rational fraction  `Frac{Pol{T}}`.
Rational fractions are normalized so that the numerator and denominator are
true  polynomials prime to each other.  They have the arithmetic operations
`+`,  `-` ,  `*`, `/`,  `//`, `^`,  `inv`, `one`, `isone`, `zero`, `iszero`
(which can operate between a `Pol` or a `Number` and a `Frac{Pol{T}}`).

```julia-repl
julia> a=1/(q+1)
Frac{Pol{Int64}}: 1/(q+1)

julia> Pol(2/a) # convert back to `Pol`
Pol{Int64}: 2q+2

julia> numerator(a)
Pol{Int64}: 1

julia> denominator(a)
Pol{Int64}: q+1

julia> m=[q+1 q+2;q-2 q-3]
2×2 Matrix{Pol{Int64}}:
 q+1  q+2
 q-2  q-3

julia> n=inv(Frac.(m)) # convert to rational fractions to invert the matrix
2×2 Matrix{Frac{Pol{Int64}}}:
 (-q+3)/(2q-1)  (-q-2)/(-2q+1)
 (q-2)/(2q-1)   (q+1)/(-2q+1)

julia> map(x->x(1),n) # evaluate at 1 the inverse matrix
2×2 Matrix{Float64}:
  2.0   3.0
 -1.0  -2.0

julia> map(x->x(1;Rational=true),n) # evaluate at 1 using //
2×2 Matrix{Rational{Int64}}:
  2//1   3//1
 -1//1  -2//1
```

Rational fractions are also scalars for broadcasting and can be sorted
(have `cmp` and `isless` methods).
"""
module LaurentPolynomials
export degree, valuation, Pol, derivative, shift, positive_part, negative_part,
       bar, derivative, srgcd, Frac, @Pol, scalar, coefficients, exactdiv, root,
       randpol, pseudodiv

const varname=Ref(:x)

struct Pol{T}
  c::Vector{T}
  v::Int
  # Unexported inner constructor that bypasses all checks
  global Pol_(c::AbstractVector{T},v::Integer) where T=new{T}(c,v)
end

"""
  `Pol(c::AbstractVector,v::Integer=0;check=true,copy=true)`

  Make a polynomial of valuation `v` with coefficients `c`.

  Unless  `check` is `false`  normalize the result  by making sure that `c`
  has  no leading or  trailing zeroes (do  not set `check=false` unless you
  are sure this is already the case).

  Unless  `copy=false` the  contents of  `c` are  copied (you  can gain one
  allocation by setting `copy=false` if you know the contents can be shared)
"""
function Pol(c::AbstractVector{T},v::Integer=0;check=true,copy=true)where T
  if check # normalize c so there are no leading or trailing zeroes
    b=findfirst(!iszero,c)
    b===nothing && return Pol_(T[],0)
    e=findlast(!iszero,c)
    if b!=1 || e!=length(c) || copy return Pol_(view(c,b:e),v+b-1) end
  end
  Pol_(copy ? Base.copy(c) : c,v)
end

Pol()=Pol_([1],1)
Pol(a::T) where T<:Number=iszero(a) ? Pol_(T[],0) : Pol_([a],0)

"""
 `Pol(t::Symbol)`

 Sets the name of the variable for printing `Pol`s to `t`, and returns
 the polynomial of degree 1 equal to that variable.
"""
function Pol(t::Symbol)
  varname[]=t
  Pol()
end

"""
 `@Pol q`

 is equivalent to `q=Pol(:q)` excepted it creates `q` in the global scope of
 the current module, since it uses `eval`.
"""
macro Pol(t)
  if !(t isa Symbol) error("usage: @Pol <variable name>") end
  Base.eval(Main,:($t=Pol($(Core.QuoteNode(t)))))
end

Base.broadcastable(p::Pol)=Ref(p)
Base.copy(p::Pol)=Pol_(copy(p.c),p.v)

degree(a::Number)=0 # convenient
degree(p::Pol)=length(p.c)-1+p.v
valuation(p::Pol)=p.v
coefficients(p::Pol)=p.c
Base.lastindex(p::Pol)=degree(p)
Base.firstindex(p::Pol)=valuation(p)
@inbounds Base.getindex(p::Pol{T},i::Integer) where T=
i in firstindex(p):lastindex(p) ? p.c[i-p.v+1] : iszero(p) ? zero(T) : zero(p.c[1])

Base.getindex(p::Pol,i::AbstractVector{<:Integer})=getindex.(Ref(p),i)

Base.convert(::Type{Pol{T}},a::Number) where T=iszero(a) ? Pol_(T[],0) :
                                                           Pol_([T(a)],0)

(::Type{Pol{T}})(a) where T=convert(Pol{T},a)

# like convert for vectors does not always make a copy
Base.convert(::Type{Pol{T}},p::Pol{T1}) where {T,T1}=Pol_(Vector{T}(p.c),p.v)

function Base.promote_rule(a::Type{Pol{T1}},b::Type{Pol{T2}})where {T1,T2}
  Pol{promote_type(T1,T2)}
end

function Base.promote_rule(a::Type{Pol{T1}},b::Type{T2})where {T1,T2<:Number}
  Pol{promote_type(T1,T2)}
end

Base.isinteger(p::Pol)=isinteger(scalar(p))

function scalar(p::Pol{T})where T
  if iszero(p) T(0)
  elseif iszero(valuation(p)) && iszero(degree(p)) p[0]
  end
end

function Base.convert(::Type{T},p::Pol) where T<:Number
  res=scalar(p)
  if res===nothing throw(InexactError(:convert,T,p)) end
  convert(T,res)
end

(::Type{T})(p::Pol) where T<:Number=convert(T,p)

Base.cmp(a::Pol,b::Pol)=cmp((a.c,a.v),(b.c,b.v))
Base.isless(a::Pol,b::Pol)=cmp(a,b)==-1
Base.hash(a::Pol, h::UInt)=hash(a.v,hash(a.c,h))

(p::Pol{T})(x) where T=iszero(p) ? zero(T) : evalpoly(x,p.c)*x^p.v

# efficient p↦ qˢ p
shift(p::Pol{T},s) where T=Pol_(p.c,p.v+s)

positive_part(p::Pol)=p.v>=0 ? copy(p) : Pol(view(p.c,1-p.v:length(p.c)),0)

negative_part(p::Pol)=degree(p)<=0 ? copy(p) : Pol(view(p.c,1:1-p.v),p.v)

# q↦ q⁻¹ on p
bar(p::Pol)=Pol_(reverse(p.c),-degree(p))

Base.:(==)(a::Pol, b::Pol)= a.c==b.c && a.v==b.v
Base.:(==)(a::Pol,b)= (iszero(a) && iszero(b))||(b!==nothing && scalar(a)==b)
Base.:(==)(b,a::Pol)= a==b

Base.one(a::Pol{T}) where T=Pol_([iszero(a) ? one(T) : one(a.c[1])],0)
Base.one(::Type{Pol{T}}) where T=Pol_([one(T)],0)
Base.one(::Type{Pol})=one(Pol{Int})
Base.isone(a::Pol)=!iszero(a) && scalar(a)==1
Base.zero(::Type{Pol{T}}) where T=Pol_(T[],0)
Base.zero(::Type{Pol})=zero(Pol{Int})
Base.zero(a::Pol{T}) where T=Pol_(T[],0)
Base.iszero(a::Pol)=isempty(a.c)
# next 4 stuff to make transpose and inv using LU work (abs is stupid)
Base.conj(p::Pol{T}) where T=Pol_(conj.(p.c),p.v)
Base.abs(p::Pol)=p
Base.adjoint(a::Pol)=conj(a)
Base.transpose(a::Pol)=a

ismonomial(p::Pol)=length(p.c)==1

function Base.show(io::IO, ::MIME"text/html", a::Pol)
  print(io, "\$")
  show(IOContext(io,:TeX=>true),a)
  print(io, "\$")
end

function Base.show(io::IO, ::MIME"text/plain", a::Pol)
  if !haskey(io,:typeinfo)
    print(io,typeof(a),": ")
    io=IOContext(io,:typeinfo=>typeof(a))
  end
  show(io,a)
end

# 3 next methods copied from Util.jl in order to have a self-contained file.

# determines which coefficients should be bracketed for unambiguous display
function bracket_if_needed(c::String)
  if match(r"^[-+]?([^-+*/]|√-|{-)*(\(.*\))?$",c)!==nothing c
  else "("*c*")"
  end
end

function format_coefficient(c::String)
  if c=="1" ""
  elseif c=="-1" "-"
  else bracket_if_needed(c)
  end
end

const supvec=['⁰','¹','²','³','⁴','⁵','⁶','⁷','⁸','⁹']

function stringexp(io::IO,n::Integer)
  if isone(n) ""
  elseif get(io,:TeX,false)
    n in 0:9 ? "^"*string(n) : "^{"*string(n)*"}"
  elseif get(io,:limit,false)
    res=Char[]
    if n<0 push!(res,'⁻'); n=-n end
    for i in reverse(digits(n)) push!(res,supvec[i+1]) end
    String(res)
  else "^"*string(n)
  end
end

function Base.show(io::IO,p::Pol)
  if !get(io,:limit,false) && !get(io,:TeX,false)
    if ismonomial(p) && isone(p.c[1]) && p.v==1 print(io,"Pol()")
    else print(io,"Pol(",p.c)
      if !iszero(p.v) print(io,",",p.v) end
      print(io,")")
    end
  elseif iszero(p) print(io,"0")
  else
    var=string(get(io,:varname,varname[]))
    for deg in degree(p):-1:valuation(p)
      c=p[deg]
      if iszero(c) continue end
      c=repr(c; context=IOContext(io,:typeinfo=>typeof(c)))
      if !iszero(deg)
        c=format_coefficient(c)*var*stringexp(io,deg)
      end
      if c[1]!='-' && deg!=degree(p) c="+"*c end
      print(io,c)
    end
  end
end

function Base.:*(a::Pol{T1},b::Pol{T2})where {T1,T2}
  T=promote_type(T1,T2)
  if iszero(a) || iszero(b) return zero(Pol{T}) end
  # below not zero(T) for types T like Mod{T1} which have no zero method
  res=fill(zero(a.c[1]*b.c[1]),length(a.c)+length(b.c)-1)
  for i in eachindex(a.c), j in eachindex(b.c)
@inbounds res[i+j-1]+=a.c[i]*b.c[j]
  end
  Pol_(res,a.v+b.v)
end

Base.:*(a::Pol, b::Number)=Pol(a.c.*b,a.v)
Base.:*(a::Pol{T}, b::T) where T=Pol(a.c.*b,a.v;copy=false)
Base.:*(b::Number, a::Pol)=a*b
Base.:*(b::T, a::Pol{T}) where T=a*b

Base.:^(a::Pol, n::Real)=isinteger(n) ? a^Int(n) : root(a,1//n)

Base.:^(a::Pol, n::Integer)=ismonomial(a) ? Pol_([a.c[1]^n],n*a.v) :
         n>=0 ? Base.power_by_squaring(a,n) :
                Base.power_by_squaring(inv(a),-n)

function Base.:+(a::Pol,b::Pol)
  if iszero(a) return b elseif iszero(b) return a end
  z=zero(a.c[1]+b.c[1])
  d=b.v-a.v
  if d>=0
    res=fill(z,max(length(a.c),d+length(b.c)))
  @inbounds view(res,eachindex(a.c)).=a.c
  @inbounds view(res,d.+eachindex(b.c)).+=b.c
    Pol(res,a.v;copy=false)
  else
    res=fill(z,max(length(a.c)-d,length(b.c)))
  @inbounds view(res,eachindex(a.c).-d).=a.c
  @inbounds view(res,eachindex(b.c)).+=b.c
    Pol(res,b.v;copy=false)
  end
end

function Base.:-(a::Pol,b::Pol)
  if iszero(a) return -b elseif iszero(b) return a end
  z=zero(a.c[1]+b.c[1])
  d=b.v-a.v
  if d>=0
    res=fill(z,max(length(a.c),d+length(b.c)))
  @inbounds view(res,eachindex(a.c)).=a.c
  @inbounds view(res,d.+eachindex(b.c)).-=b.c
    Pol(res,a.v;copy=false)
  else
    res=fill(z,max(length(a.c)-d,length(b.c)))
  @inbounds view(res,eachindex(a.c).-d).=a.c
  @inbounds view(res,eachindex(b.c)).-=b.c
    Pol(res,b.v;copy=false)
  end
end

Base.:+(a::Pol, b::Number)=a+Pol(b)
Base.:+(b::Number, a::Pol)=Pol(b)+a
Base.:-(a::Pol)=Pol_(-a.c,a.v)
Base.:-(b::Number, a::Pol)=Pol(b)-a
Base.:-(a::Pol,b::Number)=a-Pol(b)
Base.div(a::Pol,b::Number)=Pol(div.(a.c,b),a.v;copy=false)

exactdiv(a,b)=a/b  # generic version for fields
function exactdiv(a::Integer,b::Integer) # define for integral domains
  (d,r)=divrem(a,b)
  if !iszero(r) error(b," does not exactly divide ",a) end
  d
end

function coeffexactdiv(a::Pol,b)
  if isone(b) return a end
  c=exactdiv.(a.c,b)
  Pol_(c,a.v)
end
exactdiv(a::Pol,b::Number)=coeffexactdiv(a,b)

Base.:/(p::Pol,q::Number)=Pol_(p.c./q,p.v)
Base.://(p::Pol,q::Number)=Pol_(p.c.//q,p.v)

derivative(a::Pol)=Pol([(i+a.v-1)*v for (i,v) in enumerate(a.c)],a.v-1,copy=false)

"""
`divrem(a::Pol, b::Pol)`

`a` and `b` should be true polynomials (nonnegative valuation).
Computes  `(q,r)` such  that `a=q*b+r`  and `degree(r)<degree(b)`.
Type stable if the coefficients of `b` are in a field.
"""
function Base.divrem(a::Pol, b::Pol)
  if iszero(b) throw(DivideError) end
  if degree(b)>degree(a) return (zero(a),a) end
  if a.v<0 || b.v<0 error("arguments should be true polynomials") end
  d=inv(b.c[end])
  z=zero(a.c[1]+b.c[1]+d)
  r=fill(z,1+degree(a))
  view(r,a.v+1:length(r)).=a.c
  q=fill(z,length(r)-degree(b))
  for i in length(r):-1:degree(b)+1
    if iszero(r[i]) c=zero(d)
    else c=r[i]*d
         view(r,i-length(b.c)+1:i) .-= c .* b.c
    end
    q[i-degree(b)]=c
  end
  Pol(q),Pol(r)
end

function exactdiv(a::Pol,b::Pol)
  if isone(b) || iszero(a) return a end
  if iszero(b) throw(DivideError) end
  d=a.v-b.v
  if !iszero(a.v) a=shift(a,-a.v) end
  if !iszero(b.v) b=shift(b,-b.v) end
  if degree(b)>degree(a) error(b," does not exactly divide ",a) end
  z=zero(a.c[1]+b.c[1])
  r=fill(z,1+degree(a))
  view(r,a.v+1:length(r)).=a.c
  q=fill(z,length(r)-degree(b))
  for i in length(r):-1:degree(b)+1
    c=exactdiv(r[i],b.c[end])
    view(r,i-length(b.c)+1:i) .-= c .* b.c
    q[i-length(b.c)+1]=c
  end
  if !iszero(r) error(b," does not exactly divide ",a) end
  res=Pol(q,d)
end

"""
`pseudodiv(a::Pol, b::Pol)`

pseudo-division  of `a` by `b`.  If `d` is the  leading coefficient of `b`,
computes   `(q,r)`   such   that   `d^(degree(a)+1-degree(b))a=q*b+r`   and
`degree(r)<degree(b)`. Does not do division so works over any ring.
For true polynomials (errors if the valuation of `a` or of `b` is negative).

See Knuth AOCP2 4.6.1 Algorithm R
"""
function pseudodiv(a::Pol, b::Pol)
 if isone(b) || iszero(a) return a,zero(a) end
  if iszero(b) throw(DivideError) end
  d=b.c[end]
  if degree(a)<degree(b) return (zero(a),d^(degree(a)+1-degree(b))*a) end
  if a.v<0 || b.v<0 error("arguments should be true polynomials") end
  z=zero(a.c[1]+b.c[1])
  r=fill(z,1+degree(a))
  view(r,a.v+1:length(r)).=a.c
  q=fill(z,length(r)-degree(b))
  for i in length(r):-1:degree(b)+1
    c=r[i]
    r.*=d
    q.*=d
    if !iszero(c)
      for j in eachindex(b.c) r[j+i-length(b.c)]-=c*b.c[j] end
    end
    q[i-degree(b)]=c
  end
  Pol(q),Pol(r)
end

"""
`srgcd(a::Pol,b::Pol)`

sub-resultant gcd: gcd of polynomials over a unique factorization domain

See Knuth AOCP2 4.6.1 Algorithm C
"""
function srgcd(a::Pol,b::Pol)
  if degree(b)>degree(a) a,b=b,a end
  if iszero(b) return a end
  ca=gcd(a.c);a=coeffexactdiv(a,ca)
  cb=gcd(b.c);b=coeffexactdiv(b,cb)
  d=gcd(ca,cb)
  g=1
  h=1
  while true
    δ=degree(a)-degree(b)
    q,r=pseudodiv(a,b)
    if iszero(r)
      cb=gcd(b.c);b=coeffexactdiv(b,cb)
      return isone(d) ? b : Pol_(b.c .*d,b.v)
    elseif degree(r)==0
      return Pol_([d],0)
    end
    a=b
    gh=g*h^δ
    b=coeffexactdiv(r,gh)
    g=a[end]
    if δ>0 h=exactdiv(g^δ,h^(δ-1)) end
  end
end

Base.gcd(p::Pol{<:Integer},q::Pol{<:Integer})=srgcd(p,q)
Base.gcd(v::AbstractArray{<:Pol})=reduce(gcd,v)
Base.lcm(p::Pol,q::Pol)=exactdiv(p*q,gcd(p,q))
Base.lcm(m::AbstractArray{<:Pol})=reduce(lcm,m)

Base.div(a::Pol, b::Pol)=divrem(a,b)[1]
Base.:%(a::Pol, b::Pol)=divrem(a,b)[2]
Base.:%(a::Pol{<:Integer}, b::Pol{<:Integer})=divrem(a,b)[2]

"""
`gcd(p::Pol,  q::Pol)` computes the  `gcd` of the  polynomials. It uses the
subresultant algorithms for the `gcd` of integer polynomials.

```julia-repl
julia> gcd(2q+2,q^2-1)
Pol{Int64}: q+1

julia> gcd(q+1//1,q^2-1//1)
Pol{Rational{Int64}}: (1//1)q+1//1
```
"""
function Base.gcd(p::Pol,q::Pol)
  if degree(q)>degree(p)
    p,q=q,p
  end
  p,q=promote(p,q)
  while !iszero(q)
    q=q/q.c[end]
    (q,p)=(divrem(p,q)[2],q)
  end
  p*inv(p.c[end])
end

"""
  `gcdx(a::Pol,b::Pol)`

for  polynomials  over  a  field  returns `d,u,v`  such  that `d=ua+vb` and
`d=gcd(a,b)`.

```julia-repl
julia> gcdx(q^3-1//1,q^2-1//1)
((1//1)q-1//1, 1//1, (-1//1)q)
```
"""
function Base.gcdx(a::Pol, b::Pol)
  a,b=promote(a, b)
  # a0, b0=a, b
  s0, s1=one(a), zero(a)
  t0, t1=s1, s0
  # The loop invariant is: s0*a0 + t0*b0 == a
  x,y=a,b
  while y != 0
    q,q1=divrem(x, y)
    x, y=y, q1
    s0, s1=s1, s0 - q*s1
    t0, t1=t1, t0 - q*t1
  end
  (x, s0, t0)./x[end]
end

"""
`powermod(p::Pol, x::Integer, q::Pol)` computes ``p^x \\pmod m``.
```julia-repl
julia> powermod(q-1,3,q^2+q+1)
Pol{Int64}: 6q+3
```
"""
function Base.powermod(p::Pol, x::Integer, q::Pol)
  x==0 && return one(q)
  b=p%q
  t=prevpow(2, x)
  r=one(q)
  while true
    if x>=t
     r=(r*b)%q
      x-=t
    end
    t >>>= 1
    t<=0 && break
    r=(r*r)%q
  end
  r
end

"""
`randpol(T,d)`

random polynomial of degree `d` with coefficients from `T`
"""
randpol(T,d::Integer)=Pol(rand(T,d+1))

"""
`Pol(x::AbstractVector,y::AbstractVector)`

Interpolation:  find a `Pol` (of  nonnegative valuation) of smallest degree
taking  values `y` at points  `x`. The values `y`  should be in a field for
the function to be type stable.

```julia-repl
julia> p=Pol([1,1,1])
Pol{Int64}: q²+q+1

julia> vals=p.(1:5)
5-element Vector{Int64}:
  3
  7
 13
 21
 31

julia> Pol(1:5,vals*1//1)
Pol{Rational{Int64}}: (1//1)q²+(1//1)q+1//1

julia> Pol(1:5,vals*1.0)
Pol{Float64}: 1.0q²+1.0q+1.0
```
"""
function Pol(pts::AbstractVector,vals::AbstractVector)
  vals=copy(vals)
  a=map(eachindex(pts))do i
    for k in i-1:-1:1
      if pts[i]==pts[k] error("interpolating points must be distinct") end
      vals[k]=(vals[k+1]-vals[k])/(pts[i]-pts[k])
    end
    vals[1]
  end
  p=Pol_([a[end]],0)
  for i in length(pts)-1:-1:1
    p=p*(Pol()-pts[i])+a[i]
  end
  p
end

Base.denominator(p::Pol)=iszero(p) ? 1 : lcm(denominator.(p.c))
Base.numerator(p::Pol{<:Rational{T}}) where T=convert(Pol{T},p*denominator(p))

function root(x::Pol,n::Union{Integer,Rational{<:Integer}}=2)
  n=Int(n)
  if iszero(x) return x end
  if !ismonomial(x) || !iszero(x.v%n) error("root($x,$n) not implemented") end
  c=x.c[1]
  Pol_([isone(c) ? c : root(c,n)],div(x.v,n))
end

root(x::AbstractFloat,n=2)=x^(1/n)
#---------------------- Frac-------------------------------------
## cannot use Rational{T} since it forces T<:Integer
struct Frac{T}
  num::T
  den::T
  # Unexported inner constructor that bypasses all checks
  global Frac_(num::T,den::T) where T=new{T}(num,den)
end

Base.numerator(a::Frac)=a.num
Base.denominator(a::Frac)=a.den

function Base.convert(::Type{Frac{T}},p::Frac{T1}) where {T,T1}
  Frac_(convert(T,p.num),convert(T,p.den))
end

(::Type{Frac{T}})(a::Frac{T}) where T=a
(::Type{Frac{T}})(a::Number) where T=convert(Frac{T},a)

Base.broadcastable(p::Frac)=Ref(p)

Base.copy(a::Frac)=Frac_(a.num,a.den)
Base.one(a::Frac)=Frac_(one(a.num),one(a.den))
Base.one(::Type{Frac{T}}) where T =Frac_(one(T),one(T))
Base.isone(a::Frac)=a.num==a.den
Base.zero(::Type{Frac{T}}) where T =Frac_(zero(T),one(T))
Base.zero(a::Frac)=Frac_(zero(a.num),one(a.num))
Base.iszero(a::Frac)=iszero(a.num)
# next 3 methods are to make inv using LU work (abs is stupid)
Base.abs(p::Frac)=p
Base.conj(p::Frac)=Frac_(conj(p.num),conj(p.den))
Base.adjoint(a::Frac)=conj(a)
Base.:(==)(a::Frac,b::Frac)=a.num==b.num && a.den==b.den
Base.cmp(a::Frac,b::Frac)=cmp((a.num,a.den),(b.num,b.den))
Base.isless(a::Frac,b::Frac)=cmp(a,b)==-1

function Base.show(io::IO, ::MIME"text/plain", a::Frac)
  if !haskey(io,:typeinfo)
    print(io,typeof(a),": ")
    io=IOContext(io,:typeinfo=>typeof(a))
  end
  show(io,a)
end

function Base.show(io::IO,a::Frac)
  if !get(io, :limit, false) && !get(io, :TeX, false)
    print(io,"Frac(",a.num,",",a.den,")")
    return
  end
  if haskey(io,:typeinfo) && isone(a.den)
    print(io,a.num)
    return
  end
  n=sprint(show,a.num; context=io)
  print(io,bracket_if_needed(n))
  n=sprint(show,a.den; context=io)
  print(io,"/",bracket_if_needed(n))
end

Base.inv(a::Frac)=Frac_(a.den,a.num)

Base.:*(a::Frac,b::Frac)=Frac(a.num*b.num,a.den*b.den)
Base.://(a::Frac,b::Frac)=Frac(a.num*b.den,a.den*b.num)
Base.:/(a::Frac,b::Frac)=a//b

Base.:^(a::Frac, n::Integer)= n>=0 ? Base.power_by_squaring(a,n) :
                              Base.power_by_squaring(inv(a),-n)
Base.:+(a::Frac,b::Frac)=Frac(a.num*b.den+a.den*b.num,a.den*b.den)
Base.:-(a::Frac)=Frac_(-a.num,a.den)
Base.:-(a::Frac,b::Frac)=Frac(a.num*b.den-a.den*b.num,a.den*b.den)
Base.:+(a::Frac{<:T},b::Union{Number,T}) where T=+(promote(a,b)...)
Base.:+(b::Union{Number,T},a::Frac{<:T}) where T=+(promote(a,b)...)
Base.:-(a::Frac{<:T},b::Union{Number,T}) where T=-(promote(a,b)...)
Base.:-(b::Union{Number,T},a::Frac{<:T}) where T=-(promote(b,a)...)
Base.:*(a::Frac{<:T},b::T) where T=Frac(a.num*b,a.den)
Base.:*(b::T,a::Frac{<:T}) where T=Frac(a.num*b,a.den)

#----------------------------------------------------------------------

# make both pols, one of valuation 0
function make_positive(a::Pol,b::Pol)
  v=a.v-b.v
  shift(a,max(v,0)-a.v),shift(b,-min(v,0)-b.v)
end

Frac(a::Pol,b::Pol;prime=false)=Frac(promote(a,b)...;prime)

"""
`Frac(a::Pol,b::Pol;prime=false)

Polynomials  `a` and `b` are promoted to same coefficient type, and checked
for  being true polynomials (otherwise they are both multiplied by the same
power  of  the  variable  so  they  become  true  polynomials),  and unless
`prime=true` they are checked for having a non-trivial `gcd`.
"""
function Frac(a::T,b::T;prime=false)::Frac{T} where T<:Pol
  if iszero(a) return Frac_(a,one(a))
  elseif iszero(b) error("division by 0")
  end
  a,b=make_positive(a,b)
  if !(prime || ismonomial(b) || ismonomial(a))
    d=gcd(a,b)
    a,b=exactdiv(a,d),exactdiv(b,d)
  end
  if scalar(b)==-1 a,b=(-a,-b) end
  Frac_(a,b)
end

function Frac(a::Pol{<:Rational},b::Pol{<:Rational};k...)
  Frac(numerator(a)*denominator(b),numerator(b)*denominator(a);k...)
end

function Pol(p::Frac{<:Pol})
  if ismonomial(p.den)==1
    if isone(p.den.c[1]^2) return Pol(p.num.c .*p.den.c[1],p.num.v-p.den.v)
    else return Pol(p.num.c .//p.den.c[1],p.num.v-p.den.v)
    end
  end
  error("cannot convert ",p," to Pol")
end

Base.convert(::Type{Pol{T}},p::Frac) where {T}=convert(Pol{T},Pol(p))

function Base.convert(::Type{Frac{T}},p::Pol) where T
  Frac(convert(T,p),one(T);prime=true)
end

function Base.convert(::Type{Frac{T}},p::Number) where {T<:Pol}
  Frac_(convert(T,p),convert(T,1))
end

function Base.promote_rule(a::Type{T1},b::Type{Frac{T2}})where {T1<:Pol,T2<:Pol}
  Frac{promote_type(T1,T2)}
end

function Base.promote_rule(a::Type{T1},b::Type{Frac{T2}})where {T1<:Number,T2<:Pol}
  Frac{promote_type(T1,T2)}
end

function Frac(a::Pol)
  if a.v>0 return Frac_(a,one(a)) end
  Frac(a,one(a);prime=true)
end

function Base.://(a::Pol,b::Pol)
  if ismonomial(b) return Pol(a.c.//b.c[1],a.v-b.v) end
  Frac(a,b)
end

bestinv(x)=isone(x) ? x : isone(-x) ? x : inv(x)

function Base.inv(p::Pol)
  if ismonomial(p) return Pol_([bestinv(p.c[1])],-p.v) end
  Frac_(make_positive(one(p),p)...)
end

Base.:/(p::Pol,q::Pol)=p*inv(q)

Base.://(a::Frac{<:Pol},b::Pol)=Frac(a.num,a.den*b)
Base.://(a::Frac{<:Pol},b::Number)=Frac(a.num,a.den*b;prime=true)
Base.:/(a::Frac{<:Pol},b::Union{Number,Pol})=a//b
Base.://(a::Union{Number,Pol},b::Frac{<:Pol})=a*inv(b)
Base.:/(a::Union{Number,Pol},b::Frac{<:Pol})=a//b
Base.://(p::Number,q::Pol)=Frac(Pol(p),q;prime=true)
Base.:/(p::Number,q::Pol)=p*inv(q)

Base.:*(a::Frac{<:Pol},b::Number)=Frac(a.num*b,a.den;prime=true)
Base.:*(b::Number,a::Frac{<:Pol})=a*b

(p::Frac{<:Pol})(x;Rational=false)=Rational ? p.num(x)//p.den(x) : p.num(x)/p.den(x)
# @btime inv(Frac.([q+1 q+2;q-2 q-3])) setup=(q=Pol())
# 32.556 μs (938 allocations: 57.22 KiB)
end


using PyPlot

"""
    bushiness_nesdistance()

This function returns a plot showing the multistage distance of trees of different height.

The bushiness of the tree is increased with increasing height of the tree and the multistage
distance is calculated.

These results are from one of the examples we worked on.
"""
function bushiness_nesdistance()
    bushns = [3, 4, 5, 6, 7, 8]
    nstdistance = [ 0.24868 0.16772 0.12339 0.09586 0.07725 0.06408;
                    0.21781 0.12874 0.08541 0.0611 0.04575 0.03653;
                    0.16249 0.08449 0.05089 0.03337 0.02382 0.01724;
                    0.11346 0.05236 0.0289 0.01813 0.01188 0.00855]

    pt = subplot2grid((1,5), (0,0), colspan=5)
    pt.spines["top"].set_visible(false) #remove the line of the box at the top
    pt.spines["right"].set_visible(false) #remove the line of the box at the right
    for i=1:size(nstdistance,1)
        pt.plot(bushns,nstdistance[i,:],linewidth=2,marker = ".",label ="Height = $i")
    end
    xlabel("Branches at each node")
    ylabel("Multistage distance")
    axis("tight") # Fit the axis tightly to the plot
    legend(loc="upper right",fancybox="true") # Create a legend of all the existing plots using their labels as names
end


using DelayEmbeddings, Statistics

"""
    construct_candidate_variables(
        source::Vector{AbstractVector},
        target::Vector{AbstractVector},
        [cond::Vector{AbstractVector}];
        k::Int = 1, include_instantaneous = true,
        τexclude::Union{Int, Nothing} = nothing,
        maxlag::Union{Int, Float64} = 0.05
    ) → ([τs_source, τs_target, τs_cond, ks_targetfuture], [js_source, js_target, js_cond, js_targetfuture])

Construct candidate variables from input time series. `source` is a vector of equal-length time series
assumed to represent the putative source process. `target` and `cond` are the same, but contains time series
of the target process and of the conditional processes, respectively. `k` is the desired prediction lag.

If `include_instantaneous == true`, then the analysis will also consider instantaneous interactions between
the variables.

If `maxlag` is an integer, `maxlag` is taken as the maximum allowed embedding lag. If `maxlag` is a float,
then the maximum embedding lag is taken as `maximum([length.(source); length.(target); length.(cond)])*maxlag`.

If `τexclude` is an integer, all variables whose embedding lag has absolute value equal to `exclude` will be
excluded.
"""
function construct_candidate_variables(source, target, cond;
        k::Int = 1,
        τexclude::Union{Int, Nothing} = nothing,
        include_instantaneous = true,
        method_delay = "ac_min",
        maxlag::Union{Int, Float64} = 0.05)

    # Ensure all time series are of the same length.
    Ls = [length.(source); length.(target); length.(cond)]
    @assert all(Ls .== maximum(Ls))

    if maxlag isa Int
        τs = 1:maxlag
    else
        τs = 1:ceil(Int, maximum(Ls)*maxlag)
    end

    # Find the maximum allowed embedding lag for each of the candidates.
    τsmax_source = [estimate_delay(s, method_delay, τs) for s in source]
    τsmax_target = [estimate_delay(t, method_delay, τs) for t in target]
    τsmax_cond = [estimate_delay(c, method_delay, τs) for c in cond]

    # Generate candidate set
    startlag = include_instantaneous ? 0 : -1

    τs_source = [[startlag:-1:-τ...,] for τ in τsmax_source]
    τs_target = [[startlag:-1:-τ...,] for τ in τsmax_target]
    τs_cond = [[startlag:-1:-τ...,] for τ in τsmax_cond]

    ks_targetfuture = [k for i in 1:length(target)]
    js_targetfuture = [i for i in length(τs_source)+1:length(τs_source)+length(τs_target)]
    τs = [τs_source..., τs_target..., τs_cond...]
    js = [[i for x in 1:length(τs[i])] for i = 1:length(τs)]

    # Variable filtering, if desired
    if τexclude isa Int
        τs = [filtered_τs(τsᵢ, jsᵢ, τexclude) for (τsᵢ, jsᵢ) in zip(τs, js)]
        js = [filtered_js(τsᵢ, jsᵢ, τexclude) for (τsᵢ, jsᵢ) in zip(τs, js)]
    end
    return [τs..., ks_targetfuture], [js..., js_targetfuture]
end

# Usaully, we use all lags from startlag:-\tau_max to construct variables. In some situations,
# we may want to exclude som of those variables.
function filtered_τs(τs::AbstractVector{Int}, js::AbstractVector{Int}, τexclude::Int)
    [τ for τ in τs if abs(τ) != abs.(τexclude)]
end

function filtered_js(τs::AbstractVector{Int}, js::AbstractVector{Int}, τexclude::Int)
    [j for (τ, j) in zip(τs, js) if abs(τ) != abs.(τexclude)]
end

# source & target variant
function construct_candidate_variables(source, target;
        k::Int = 1,
        τexclude::Union{Int, Nothing} = nothing,
        include_instantaneous = true,
        method_delay = "ac_min",
        maxlag::Union{Int, Float64} = 0.05)

    # Ensure all time series are of the same length.
    Ls = [length.(source); length.(target)]
    @assert all(Ls .== maximum(Ls))

    if maxlag isa Int
        τs = 1:maxlag
    else
        τs = 1:ceil(Int, maximum(Ls)*maxlag)
    end

    # Find the maximum allowed embedding lag for each of the candidates.
    τsmax_source = [estimate_delay(s, method_delay, τs) for s in source]
    τsmax_target = [estimate_delay(t, method_delay, τs) for t in target]

    # Generate candidate set
    startlag = include_instantaneous ? 0 : -1
    τs_source = [[startlag:-1:-τ...,] for τ in τsmax_source]
    τs_target = [[startlag:-1:-τ...,] for τ in τsmax_target]

    ks_targetfuture = [k for i in 1:length(target)]
    js_targetfuture = [i for i in length(τs_source)+1:length(τs_source)+length(τs_target)]
    τs = [τs_source..., τs_target...,]
    js = [[i for x in 1:length(τs[i])] for i = 1:length(τs)]

    # Variable filtering, if desired
    if τexclude isa Int
        τs = [filtered_τs(τsᵢ, jsᵢ, τexclude) for (τsᵢ, jsᵢ) in zip(τs, js)]
        js = [filtered_js(τsᵢ, jsᵢ, τexclude) for (τsᵢ, jsᵢ) in zip(τs, js)]
    end

    return [τs..., ks_targetfuture], [js..., js_targetfuture]
end

# source, target & cond variant
function embed_candidate_variables(source, target, cond;
        η::Int = 1,
        τexclude::Union{Int, Nothing} = nothing,
        include_instantaneous = true,
        method_delay = "mi_min",
        maxlag::Union{Int, Float64} = 0.05)

    τs, js = construct_candidate_variables(source, target, cond, k = η, τexclude = τexclude)

    # TODO: This is more efficient if not using datasets. Re-do manually.
    data = Dataset([source..., target..., cond...,]...,)
    ℰ = genembed(data, ((τs...)...,), ((js...)...,))

    # Get all variables except the target future (which are the last columns of ℰ)
    n_timeseries = size(ℰ, 2)
    n_timeseries_target = length(target)
    Ω = [ℰ[:, i] for i = 1:n_timeseries - n_timeseries_target]
    Y⁺ = ℰ[:, n_timeseries - n_timeseries_target+1:end]

    # We need to keep track of which variables are from the source, because
    # when computing the final TE, we need a marginal which is 𝒮 \ 𝒮_source.
    # Hence, we need to know which indices in `js` correspond to the source.
    idxs_source = 1:length(source)
    idxs_target = length(source)+1:length(source)+length(target)
    idxs_cond = length(source)+length(target)+1:length(source)+length(target)+length(cond)

    return Ω, Y⁺, τs, js, idxs_source, idxs_target, idxs_cond
end

# source & target variant
function embed_candidate_variables(source, target;
        η::Int = 1,
        τexclude::Union{Int, Nothing} = nothing,
        include_instantaneous = true,
        method_delay = "mi_min",
        maxlag::Union{Int, Float64} = 0.05)

    τs, js = construct_candidate_variables(source, target, k = η, τexclude = τexclude)

    # TODO: This is more efficient if not using datasets. Re-do manually.
    data = Dataset([source..., target...,]...,)
    ℰ = genembed(data, ((τs...)...,), ((js...)...,))

    # Get all variables except the target future (which are the last columns of ℰ)
    n_timeseries = size(ℰ, 2)
    n_timeseries_target = length(target)
    Ω = [ℰ[:, i] for i = 1:n_timeseries - n_timeseries_target]

    Y⁺ = ℰ[:, n_timeseries - n_timeseries_target+1:end]
    idxs_source = 1:length(source)
    idxs_target = length(source)+1:length(source)+length(target)
    idxs_cond = Int[]

    return Ω, Y⁺, τs, js, idxs_source, idxs_target, idxs_cond
end

function optim_te(Ω, Y⁺, τs, js, idxs_source, idxs_target, idxs_cond, est;
        uq = 0.95, nsurr = 100, q = 1, base = 2)

    τs_comb = [(τs...)...,]
    js_comb = [(js...)...,]

    npts = length(Y⁺)
    n_candidate_variables = length(Ω)

    𝒮 = Vector{Vector{Float64}}(undef, 0)
    𝒮_τs = Vector{Int}(undef, 0)
    𝒮_js = Vector{Int}(undef, 0)

    k = 1
    while k <= n_candidate_variables
        n_remaining_candidates = length(Ω)
        CMIs_between_Y⁺_and_candidates = zeros(n_remaining_candidates)

        # At first iteration, only loop through source variable. If no source variable is found that
        # yields significant TE, terminate.
        for i = 1:n_remaining_candidates
            if k == 1 || length(𝒮) == 0
                Cᵢ = Ω[i]
                CMI_Y⁺_Cᵢ =
                    genentropy(Dataset(Y⁺, Dataset(Cᵢ)), est, q = q, base = base) -
                    genentropy(Dataset(Cᵢ), est, q = q, base = base)
            else
                Cᵢ = [Ω[i], 𝒮...]
                CMI_Y⁺_Cᵢ =
                    genentropy(Dataset(Y⁺, Dataset(Cᵢ...,)), est, q = q, base = base) -
                    genentropy(Dataset(Cᵢ...,), est, q = q, base = base)
            end
            CMIs_between_Y⁺_and_candidates[i] = CMI_Y⁺_Cᵢ
        end

        idx = findfirst(x -> x == minimum(CMIs_between_Y⁺_and_candidates), CMIs_between_Y⁺_and_candidates)
        Wₖ = Ω[idx]

        # Test significance of this candidate by using a random permutation test
        CMI_permutations = zeros(nsurr)

        # A circular shift surrogate generator, to exclude effects of autocorrelation
        s = surrogenerator(Wₖ, CircShift(collect(1:npts - 1)))
        #s = surrogenerator(Wₖ, RandomShuffle())

        if k == 1
            cmiₖ = CMIs_between_Y⁺_and_candidates[idx]

            for i = 1:nsurr
                surr_wₖ = s() # Surrogate version of Wₖ
                CMI_permutations[i] = mutualinfo(Y⁺, surr_wₖ, est)
            end
        else
            # Precompute terms that do not change during surrogate loop
            H_Y⁺_𝒮 = genentropy(Dataset(Y⁺, Dataset(𝒮...,)), est, q = q, base = base)

            # ORIGIANL TE
            H_𝒮 = genentropy(Dataset(𝒮...), est, q = q, base = base)
            cmiₖ = H_Y⁺_𝒮 +
                    genentropy(Dataset([Wₖ, 𝒮...,]...,), est, q = q, base = base) -
                    genentropy(Dataset(Y⁺, Dataset([Wₖ, 𝒮...,]...,)), est, q = q, base = base) -
                    H_𝒮

            for i = 1:nsurr
                surr_wₖ = s() # Surrogate version of Wₖ
                CMI_permutations[i] = H_Y⁺_𝒮 +
                    genentropy(Dataset([surr_wₖ, 𝒮...]...,), est, q = q, base = base) -
                    genentropy(Dataset(Y⁺, Dataset([surr_wₖ, 𝒮...]...,)), est, q = q, base = base) -
                    H_𝒮
            end

        end
       # If the candidate passes the significance test
        if cmiₖ > quantile(CMI_permutations, uq)
            # Add the candidate to list of selected candidates
            push!(𝒮, Wₖ)
            push!(𝒮_τs, τs_comb[idx])
            push!(𝒮_js, js_comb[idx])

            # Delete the candidate from the list of remaining candidates
            deleteat!(Ω, idx)
            deleteat!(τs_comb, idx)
            deleteat!(js_comb, idx)

            k = k + 1
        else
            k = n_candidate_variables + 1
        end
    end

    # No variables were selected
    if length(𝒮) == 0
        return 0.0, Int[], Int[], idxs_source, idxs_target, idxs_cond
    end

    # No variables were selected from the source process
    n_source_vars_picked = count(x -> x ∈ idxs_source, 𝒮_js)
    if n_source_vars_picked == 0
        return 0.0, Int[], Int[], idxs_source, idxs_target, idxs_cond
    end

    # No variables were selected from the target or conditional processes.
    𝒮_nonX = [ts for (ts, j) in zip(𝒮, 𝒮_js) if j ∉ idxs_source]
    if length(𝒮_nonX) == 0
        return 0.0, Int[], Int[], idxs_source, idxs_target, idxs_cond
    end

    CE2 = genentropy(Dataset(Y⁺, Dataset(𝒮...,)), est, base = base, q = q) -
        genentropy(Dataset(𝒮...,), est, base = base, q = q)

    CE1 = genentropy(Dataset(Y⁺, Dataset(𝒮_nonX...,)), est, base = base, q = q) -
        genentropy(Dataset(𝒮_nonX...,), est, base = base, q = q)

    CMI = CE1 - CE2
    return CMI, 𝒮_js, 𝒮_τs, idxs_source, idxs_target, idxs_cond

end

process_input(ts::Vector{T}) where T <: Real = [ts]
process_input(ts::AbstractVector{Vector{T}}) where T <: Real = ts


import Temporal.acf  # used for running autocorrelation function

using Statistics

"""
```
runmean(x::Array{T}; n::Int=10, cumulative::Bool=true)::Array{T}
runmean(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix
```

Compute a running or rolling arithmetic mean of an array.
"""
function runmean(x::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T} where {T<:AbstractFloat}
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(size(x,1))
    out[1:n-1] .= NaN
    if cumulative
        fi = 1.0:size(x,1)
        @inbounds for i in n:size(x,1)
            out[i] = sum(x[1:i])/fi[i]
        end
    else
        # original shorter but slower version of the code below
        # @inbounds for i = n:size(x,1)
        #     out[i] = mean(x[i-n+1:i])
        # end
        n_current = 0
        s::T = 0
        @inbounds for i = 1:n
            if isfinite(x[i])
                s += x[i]
                n_current += 1
            end
        end
        out[n] = n_current == n ? s / n : NaN
        @inbounds for i = n+1:size(x,1)
            if isfinite(x[i])
                s += x[i]
                n_current += 1
            end
            if isfinite(x[i-n])
                s -= x[i-n]
                n_current -= 1
            end
            out[i] = (n_current == n) ? s / n_current : NaN
        end
    end
    return out
end
runmean(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix = hcat((runmean(X[:,j], n=n, cumulative=cumulative) for j in 1:size(X,2))...)

"""
```
runsum(x::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T}
runsum(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix
```

Compute a running or rolling summation of an array.
"""
function runsum(x::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T} where {T<:AbstractFloat}
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    if cumulative
        out = cumsum(x, dims=1)
        out[1:n-1] .= NaN
    else
        out = zeros(size(x))
        out[1:n-1] .= NaN
        # original shorter but slower version of the code below
        # @inbounds for i = n:size(x,1)
        #     out[i] = sum(x[i-n+1:i])
        # end
        n_current = 0
        s::T = 0
        @inbounds for i = 1:n
            if isfinite(x[i])
                s += x[i]
                n_current += 1
            end
        end
        out[n] = n_current == n ? s : NaN
        @inbounds for i = n+1:size(x,1)
            if isfinite(x[i])
                s += x[i]
                n_current += 1
            end
            if isfinite(x[i-n])
                s -= x[i-n]
                n_current -= 1
            end
            out[i] = (n_current == n) ? s : NaN
        end
    end
    return out
end
runsum(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix = hcat((runsum(X[:,j], n=n, cumulative=cumulative) for j in 1:size(X,2))...)

"""
```
runmad(x::Vector{T}; n::Int=10, cumulative::Bool=true, fun::Function=median)::Vector{T}
runmad(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix
```

Compute the running or rolling mean absolute deviation of an array
"""
function runmad(x::Vector{T}; n::Int=10, cumulative::Bool=true, fun::Function=median)::Vector{T} where {T<:AbstractFloat}
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(size(x))
    out[1:n-1] .= NaN
    center = 0.0
    if cumulative
        fi = collect(1.0:size(x,1))
        @inbounds for i = n:size(x,1)
            center = fun(x[1:i])
            out[i] = sum(abs.(x[1:i].-center)) / fi[i]
        end
    else
        fn = float(n)
        @inbounds for i = n:size(x,1)
            center = fun(x[i-n+1:i])
            out[i] = sum(abs.(x[i-n+1:i].-center)) / fn
        end
    end
    return out
end
runmad(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix = hcat((runmad(X[:,j], n=n, cumulative=cumulative) for j in 1:size(X,2))...)

"""
```
runvar(x::Vector{T}; n::Int=10, cumulative=true)::Vector{T}
runvar(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix
```

Compute the running or rolling variance of an array
"""
function runvar(x::Vector{T}; n::Int=10, cumulative=true)::Vector{T} where {T<:AbstractFloat}
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(size(x))
    out[1:n-1] .= NaN
    if cumulative
        @inbounds for i = n:size(x,1)
            out[i] = var(x[1:i])
        end
    else
        @inbounds for i = n:size(x,1)
            out[i] = var(x[i-n+1:i])
        end
    end
    return out
end
runvar(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix = hcat((runvar(X[:,j], n=n, cumulative=cumulative) for j in 1:size(X,2))...)

"""
```
runsd(x::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T}
runsd(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix
```

Compute the running or rolling standard deviation of an array
"""
function runsd(x::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T} where {T<:AbstractFloat}
    return sqrt.(runvar(x, n=n, cumulative=cumulative))
end
runsd(X::Matrix; n::Int=10, cumulative::Bool=true)::Matrix = sqrt.(hcat((runvar(X[:,j], n=n, cumulative=cumulative) for j in 1:size(X,2))...))

"""
```
runcov(x::Vector{T}, y::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T}
runcov(X::Matrix, Y::Matrix; n::Int=10, cumulative::Bool=true)::Matrix
```

Compute the running or rolling covariance of two arrays
"""
function runcov(x::Vector{T}, y::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T} where {T<:AbstractFloat}
    @assert length(x) == length(y) "Dimension mismatch: length of `x` not equal to length of `y`."
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(size(x))
    out[1:n-1] .= NaN
    if cumulative
        @inbounds for i = n:length(x)
            out[i] = cov(x[1:i], y[1:i])
        end
    else
        @inbounds for i = n:length(x)
            out[i] = cov(x[i-n+1:i], y[i-n+1:i])
        end
    end
    return out
end
runcov(X::Matrix, Y::Matrix; n::Int=10, cumulative::Bool=true)::Matrix = hcat((runcov(X[:,j], Y[:,j], n=n, cumulative=cumulative) for j in 1:size(X,2))...)

"""
```
runcor(x::Vector{T}, y::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T}
runcor(X::Matrix, y::Vector; n::Int=10, cumulative::Bool=true)::Matrix
runcor(X::Matrix, Y::Matrix; n::Int=10, cumulative::Bool=true)::Matrix
```

Compute the running or rolling correlation of two arrays
"""
function runcor(x::Vector{T}, y::Vector{T}; n::Int=10, cumulative::Bool=true)::Vector{T} where {T<:Float64}
    @assert length(x) == length(y) "Dimension mismatch: length of `x` not equal to length of `y`."
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(size(x))
    out[1:n-1] .= NaN
    if cumulative
        @inbounds for i = n:length(x)
            out[i] = cor(x[1:i], y[1:i])
        end
    else
        @inbounds for i = n:length(x)
            out[i] = cor(x[i-n+1:i], y[i-n+1:i])
        end
    end
    return out
end
runcor(X::Matrix, Y::Matrix; n::Int=10, cumulative::Bool=true)::Matrix = hcat((runcor(X[:,j], Y[:,j], n=n, cumulative=cumulative) for j in 1:size(X,2))...)
runcor(X::Matrix, y::Vector; n::Int=10, cumulative::Bool=true)::Matrix = hcat((runcor(X[:,j], y, n=n, cumulative=cumulative) for j in 1:size(X,2))...)

"""
```
runmax(x::Vector{T}; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Vector{T}
runmax(X::Matrix; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Matrix
```

Compute the running or rolling maximum of an array
"""
function runmax(x::Vector{T}; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Vector{T} where {T<:AbstractFloat}
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(size(x))
    if inclusive
        if cumulative
            out[n] = maximum(x[1:n])
            @inbounds for i = n+1:size(x,1)
                out[i] = max(out[i-1], x[i])
            end
        else
            @inbounds for i = n:size(x,1)
                out[i] = maximum(x[i-n+1:i])
            end
        end
        out[1:n-1] .= NaN
        return out
    else
        if cumulative
            out[n+1] = maximum(x[1:n])
            @inbounds for i = n+1:size(x,1)-1
                out[i+1] = max(out[i-1], x[i-1])
            end
        else
            @inbounds for i = n:size(x,1)-1
                out[i+1] = maximum(x[i-n+1:i])
            end
        end
        out[1:n] .= NaN
        return out
    end
end
runmax(X::Matrix; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Matrix = hcat((runmax(X[:,j], n=n, cumulative=cumulative, inclusive=inclusive) for j in 1:size(X,2))...)

"""
```
runmin(x::Vector{T}; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Vector{T}
runmin(X::Matrix; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Matrix
```

Compute the running or rolling minimum of an array
"""
function runmin(x::Vector{T}; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Vector{T} where {T<:AbstractFloat}
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(size(x))
    if inclusive
        if cumulative
            out[n] = minimum(x[1:n])
            @inbounds for i = n+1:size(x,1)
                out[i] = min(out[i-1], x[i])
            end
        else
            @inbounds for i = n:size(x,1)
                out[i] = minimum(x[i-n+1:i])
            end
        end
        out[1:n-1] .= NaN
        return out
    else
        if cumulative
            out[n+1] = minimum(x[1:n])
            @inbounds for i = n+1:size(x,1)-1
                out[i+1] = min(out[i-1], x[i-1])
            end
        else
            @inbounds for i = n:size(x,1)-1
                out[i+1] = minimum(x[i-n+1:i])
            end
        end
        out[1:n] .= NaN
        return out
    end
end
runmin(X::Matrix; n::Int=10, cumulative::Bool=true, inclusive::Bool=true)::Matrix = hcat((runmin(X[:,j], n=n, cumulative=cumulative, inclusive=inclusive) for j in 1:size(X,2))...)

"""
```
runquantile(x::Vector{T}; p::T=0.05, n::Int=10, cumulative::Bool=true)::Vector{T}
runquantile(X::Matrix; n::Int=10, cumulative::Bool=true, p::Real=0.05)::Matrix
```

Compute the running/rolling quantile of an array
"""
function runquantile(x::Vector{T}; p::T=0.05, n::Int=10, cumulative::Bool=true)::Vector{T} where {T<:AbstractFloat}
    @assert n<size(x,1) && n>1 "Argument n is out of bounds."
    out = zeros(T, size(x,1))
    if cumulative
        @inbounds for i in 2:size(x,1)
            out[i] = quantile(x[1:i], p)
        end
        out[1] = NaN
    else
        @inbounds for i in n:size(x,1)
            out[i] = quantile(x[i-n+1:i], p)
        end
        out[1:n-1] .= NaN
    end
    return out
end
runquantile(X::Matrix; n::Int=10, cumulative::Bool=true, p::Real=0.05)::Matrix = hcat((runquantile(X[:,j], n=n, cumulative=cumulative, p=p) for j in 1:size(X,2))...)

"""
```
function runacf(x::Vector{T};
                n::Int = 10,
                maxlag::Int = n-3,
                lags::AbstractVector{Int,1} = 0:maxlag,
                cumulative::Bool = true)::Matrix{T} where {T<:AbstractFloat}
runacf(X::Matrix; n::Int=10, cumulative::Bool=true, maxlag::Int=n-3, lags::AbstractVector{Int}=0:maxlag)::Matrix
```

Compute the running/rolling autocorrelation of a vector.
"""
function runacf(x::Vector{T};
                n::Int = 10,
                maxlag::Int = n-3,
                lags::AbstractVector{Int} = 0:maxlag,
                cumulative::Bool = true)::Matrix{T} where {T<:AbstractFloat}
    @assert size(x, 2) == 1 "Autocorrelation input array must be one-dimensional"
    N = size(x, 1)
    @assert n < N && n > 0
    if length(lags) == 1 && lags[1] == 0
        return ones((N, 1))
    end
    out = zeros((N, length(lags))) * NaN
    if cumulative
        @inbounds for i in n:N
            out[i,:] = acf(x[1:i], lags=lags)
        end
    else
        @inbounds for i in n:N
            out[i,:] = acf(x[i-n+1:i], lags=lags)
        end
    end
    return out
end
runacf(X::Matrix; n::Int=10, cumulative::Bool=true, maxlag::Int=n-3, lags::AbstractVector{Int}=0:maxlag)::Matrix = hcat((runacf(X[:,j], n=n, cumulative=cumulative, maxlag=maxlag, lags=lags) for j in 1:size(X,2))...)

"""
```
runfun(x::Vector{T}, f::Function; n::Int = 10, cumulative::Bool=false, args...)::Vector{T} where {T<:AbstractFloat}
runfun(X::Matrix{T}, f::Function; n::Int=10, cumulative::Bool=false, args...)::Matrix{T} where {T<:AbstractFloat}
```

Apply a general function `f` that returns a scalar over an array
"""
function runfun(x::Vector{T}, f::Function; n::Int = 10, cumulative::Bool=false, args...)::Vector{T} where {T<:AbstractFloat}
    N = size(x,1)
    out = zeros(T, N) .* NaN
    if cumulative
        for i in n:N
            result::T = f(x[1:i]; args...)
            out[i] = result
        end
    else
        for i in n:N
            result::T = f(x[i-n+1:i]; args...)
            out[i] = result
        end
    end
    return out
end

function runfun(X::Matrix{T}, f::Function; n::Int=10, cumulative::Bool=false, args...)::Matrix{T} where {T<:AbstractFloat}
    return hcat((runfun(X[:,j], f, n=n, cumulative=cumulative; args...) for j in 1:size(X,2))...)
end


# R̂ of Gelman: page 284
function Rhat_gelman(chain, Npop, Ngeneration, Npar)
    # split each chain in two
    n = Int(floor(Ngeneration/2)) # length of chains that are split
    m = Npop*2
    chainsplit = zeros(eltype(chain), (m, Npar, n))
    chainsplit[1:Npop, :, :] .= chain[:,:,1:n]
    chainsplit[Npop+1:end, :, :] .= chain[:,:,n+1:2*n]
    # avg overall and within chain
    avg_par = mean(chainsplit, dims = [1,3])
    avg_chains = mean(chainsplit, dims = 3)
    # variance btw chains and within chain
    B = n/(m-1) * sum((avg_chains .- avg_par).^2, dims = 1)
    sj = 1/(n-1)*sum((chainsplit .- avg_chains).^2, dims =  3)
    W = 1/m * sum(sj, dims = 1)
    varhat = (n-1)/n * W .+ 1/n*B
    # Rhat is the ratio of (between + within) / within variance -> 1 if correct
    Rhat = sqrt.(varhat./W)[:]
    return Rhat
end

function flatten_chain(chain, Npop, Ngeneration, Npar)
    flatchain = zeros(eltype(chain), Npar, Npop*Ngeneration)
    for i = 1:Npar
        count = 0
        for  ig = 1:Ngeneration, ic = 1:Npop
            count += 1
            flatchain[i, count] = chain[ic, i, ig]
        end
    end
    return flatchain
end

function convergence_check(chain, log_obj, figure_path; verbose = true, parnames = [])
    Npop, Npar, Ngeneration = size(chain)
    return convergence_check(chain, log_obj, Npop, Ngeneration, Npar, figure_path; verbose = verbose, parnames = parnames)
end

function convergence_check(chain, log_obj, Npop, Ngeneration, Npar, figure_path; verbose = true, parnames = [])
    nrow, ncol = size(log_obj)
    @assert nrow == Npop "number of rows of log_obj is not equal to number of chains"
    @assert ncol == Ngeneration "number of columns of log_obj is not equal to number of generations simulated"
    n1, n2, n3 = size(chain)
    @assert n1 == Npop "#elements in first dimension of chain is not equal to number of chains simulated"
    @assert n2 == Npar "#elements in second dimension is not equal to number of parameters"
    @assert n3 == Ngeneration "#elements in third dimension of chain is not equal to number of generations simulated"

    if parnames == []
        for ip = 1:Npar
            append!(parnames, [string("par ", ip)])
        end
    end
    if verbose
        try
            mkpath(string(figure_path))
        catch e
            println(e)
        end
    end
    # acceptance ratio
    accept_ratio = sum(diff(log_obj, dims = 2).!=0., dims = 2)./ (Ngeneration-1)
    # plot the trace of the obj function value of the chains, clean out extreme values
    if verbose
        p_trace = plot(log_obj')
        savefig(p_trace, string(figure_path, "trace_logobj.png"))

        traceplots = []
        for par in 1:Npar
            append!(traceplots, [plot(reshape(chain[:, par , :], Npop, Ngeneration)', title=parnames[par])])
        end
        for par in 1:Npar
            par_trace = plot(traceplots[par])
            savefig(par_trace, string(figure_path, "trace_par_", par, ".png"))
        end
        for par in 1:Npar
            hi = histogram(chain[:, par , :][:], nbins = 33, normed=true)
            savefig(hi, string(figure_path, "hist_par_", par, ".png"))
        end
    end

    # Rhat Gelman
    Rhat = Rhat_gelman(chain, Npop, Ngeneration, Npar)
    if verbose
        println("Summary Checks")
        println(" ")
        println("Acceptance Ratio of each chain:")
        println(accept_ratio)
        println(" ")
        println("Rhat Gelman: $Rhat")
        println(" ")
        display(p_trace)
    end
    return accept_ratio, Rhat
end

function mean_cov_chain(chain, Npop, Ngeneration, Npar)
    n1, n2, n3 = size(chain)
    @assert n1 == Npop "#elements in first dimension of chain is not equal to number of chains simulated"
    @assert n2 == Npar "#elements in second dimension is not equal to number of parameters"
    @assert n3 == Ngeneration "#elements in third dimension of chain is not equal to number of generations simulated"

    flatchain = flatten_chain(chain, Npop, Ngeneration, Npar)
    b = mean(flatchain, dims=2)[:]
    cov = 1 ./(Ngeneration*Npop) * (flatchain .- b) * (flatchain .- b)'
    return b, cov
end

function mean_cov_chain(chain)
    Npop, Npar, Ngeneration = size(chain)
    return mean_cov_chain(chain, Npop, Ngeneration, Npar)
end

function save_res(b, Σb, Rhat, accept_ratio, path)
    writecsv(string(path, "_b.csv"), b)
    writecsv(string(path, "_Sigmab.csv"), Σb)
    writecsv(string(path, "_Rhat.csv"), Rhat)
    writecsv(string(path, "_accept_ratio.csv"), accept_ratio)
end

function extract_best(mc)
    bestval = maximum(mc.log_obj)
    (rows, cols) = findn(bestval.==mc.log_obj)
    bestpar = mc.chain[rows[1], :, cols[1]]
    return bestval, bestpar
end


module Physt

import Base.<<, Base.deepcopy, Base.copy
export h1, update, update!, find_bin, 📊

abstract type Binning{AxisType} end
abstract type Histogram end

struct RegularBinning{AxisType} <: Binning{AxisType}
    nbins::Int64
end

struct Histogram1D{AxisType, ValueType} <: Histogram where AxisType
    edges::Array{AxisType, 1}
    values::Array{ValueType, 1}
end

function copy(old::Histogram1D{AxisType, ValueType}) where AxisType where ValueType
    new = Histogram1D{AxisType, ValueType}(copy(old.edges), copy(old.values))
end

deepcopy(histogram::Histogram) = copy(histogram)

function fit(binning::RegularBinning{AxisType}, entries::AbstractArray)::Array{AxisType, 1} where AxisType
    min = convert(AxisType, minimum(entries))
    max = convert(AxisType, maximum(entries))
    step = (max - min) / binning.nbins
    return collect(range(min, length=binning.nbins + 1, step=step))
end

@inline function find_bin(edges::AbstractArray{AxisType}, value::AxisType) where AxisType
    rightedge = findfirst(e -> e > value, edges)
    if rightedge == nothing
        # Last edge is inclusive
        if value == edges[end]
            return length(edges) - 1
        end
    elseif rightedge > 1
        return rightedge - 1
    end
    return nothing
end

function find_bin(histogram::Histogram1D, value)
    return find_bin(histogram.edges, value)
end

function transform(edges::AbstractArray{AxisType}, entries::AbstractArray{AxisType})::Array{Int64, 1} where AxisType
    # Least effective of all implementations
    nbins = length(edges) - 1
    result = zeros(Int64, nbins)
    for value in entries
        bin = find_bin(edges, value)
        if bin !== nothing
            result[bin] += 1
        end
    end
    return result
end

function fit_transform(binning::Binning, entries::AbstractArray)
    edges = fit(binning, entries)
    return edges, transform(edges, entries)
end

function update(histogram::Histogram, any)
    new = copy(histogram)
    update!(new, any)
    return new
end

function update!(histogram::Histogram1D{AxisType}, entry) where AxisType
    entry_converted = convert(AxisType, entry)
    bin = find_bin(histogram.edges, entry_converted)
    if bin != nothing
        histogram.values[bin] += 1
    end
    return histogram
end

function update!(histogram::Histogram1D{AxisType}, entries::AbstractArray) where AxisType
    new_values = transform(histogram.edges, entries)
    histogram.values .+= new_values
    return histogram
end

<<(histogram::Histogram, what_to_update) = update(histogram, what_to_update)

function h1(entries::AbstractArray{AxisType}, nbins::Integer=10) where AxisType
    binning = RegularBinning{AxisType}(nbins)
    edges, values = fit_transform(binning, entries)
    return Histogram1D{AxisType, Int64}(edges, values)
end

function h1(entries::AbstractArray{AxisType}, edges::AbstractArray{AxisType}) where AxisType
    values = transform(edges, entries)
    return Histogram1D{AxisType, Int64}(edges, values)
end

📊 = h1  # Because we can...

end # module


#Demo 6: Bode Plot
#-------------------------------------------------------------------------------

using CMDimData
using CMDimData.EasyPlot
using CMDimData.MDDatasets
using Colors
import Printf: @sprintf

#==Attributes
===============================================================================#
dfltline = cons(:a, line=set(style=:solid, color=:blue, width=3))
markerline = cons(:a, line=set(style=:dash, width=2.5))
markerline_light = cons(:a, line=set(style=:dash, width=2.5, color=RGB24(.4,.4,.4)))
dfltglyph = cons(:a, glyph=set(shape=:square, size=3))

#==Input data
===============================================================================#
j = im
G = 20.0; 𝑓_p1 = 50e6; 𝑓_p2 = .9e9; 𝑓_z = Inf #.8e9
𝑓step = 5e6; 𝑓max = 10e9
𝑓min_disp = 10e6

#==Helper functions
===============================================================================#
phase(x) = rad2deg(angle(x))
function roots(a,b,c)
	r = sqrt(b*b-4*a*c)
	result = (-b .+ [r, -r]) / (2a)
	return (result[1], result[2])
end
xf_o2(f, G, f_z, f_p1, f_p2) = #2nd order transfer function
	G*(1 + j*(f/f_z)) / ( (1 + j*(f/f_p1)) * (1 + j*(f/f_p2)) )
function f_unitygain(f, G, f_z, f_p1, f_p2) #Compute unity gain frequency
	invsq(x) = (result = 1/x; return result*result)
	_z² = invsq(f_z); G² = G*G
	_p1² = invsq(f_p1); _p2² = invsq(f_p2)
	r = roots(_p1²*_p2², _p1²+_p2²-G²*_z², 1-G²)
	return sqrt(max(r...)) #Select sqrt(positive root)
end
f_3dB(f, G, f_z, f_p1, f_p2) = #Compute 3dB frequency
	f_unitygain(f, sqrt(2), f_z, f_p1, f_p2)

#==Calculations
===============================================================================#
𝑓 = DataF1(0:𝑓step:𝑓max)
X = xf_o2(𝑓, G, 𝑓_z, 𝑓_p1, 𝑓_p2)
𝑓0 = f_unitygain(𝑓, G, 𝑓_z, 𝑓_p1, 𝑓_p2)
𝑓BW = f_3dB(𝑓, G, 𝑓_z, 𝑓_p1, 𝑓_p2)
phase0 = phase(xf_o2(𝑓0, G, 𝑓_z, 𝑓_p1, 𝑓_p2))
pmargin = 180+phase0
@show pmargin

#==Generate EasyPlot
===============================================================================#
gridconf = set(vmajor = true, vminor = true, hmajor=false, hminor=false)
plot = cons(:plot, nstrips=2, legend=false, title = "Sample Bode Plot",
	xaxis = set(scale=:log, label="Frequency [Hz]", min=𝑓min_disp), #Avoid issues with log scale
	ystrip1 = set(scale=:dB20, axislabel="Magnitude [dB]", min=-10),
	ystrip2 = set(scale=:lin, axislabel="Phase [°]", min=-180),
	grid1 = gridconf, grid2 = gridconf
)

push!(plot,
	cons(:wfrm, abs(X), dfltline, label="|X|", strip=1),
	cons(:wfrm, phase(X), dfltline, label="∠X", strip=2),
)

𝑓3dBstr = @sprintf("𝑓3dB=%.1f MHz", 𝑓BW/1e6)
𝑓0str = @sprintf("𝑓0=%.1f MHz", 𝑓0/1e6)
pmstr = @sprintf("PM=%.1f°", pmargin)

push!(plot,
	cons(:atext, 𝑓3dBstr, x=𝑓BW, offset=set(x=-3), reloffset=set(y=.5), angle=-90, align=:bc, strip=1),
	cons(:atext, 𝑓0str, x=𝑓0, offset=set(x=-3), reloffset=set(y=.5), angle=-90, align=:bc, strip=1),
	cons(:atext, "0dB", y=0, reloffset=set(x=0.5), offset=set(y=3), align=:bc, strip=1),
	cons(:atext, pmstr, y=(phase0-180)/2, reloffset=set(x=0.5), align=:cc, strip=2),
)

#Add H/V markers to plot:
push!(plot,
	cons(:vmarker, 𝑓0, markerline, strip=0),
	cons(:vmarker, 𝑓BW, markerline, strip=0),
	cons(:hmarker, 0, markerline_light, strip=1),
	cons(:hmarker, phase0, markerline, strip=2),
)

pcoll = push!(cons(:plotcoll, title="Bode Test"), plot)

CMDimData.@includepkg EasyPlotInspect
plotgui = EasyPlot.displaygui(:InspectDR, pcoll)
ploth = 500; plotw = round(Int, ploth*1.6)
EasyPlot._write(:png, "demo6_test.png", plotgui, dim=set(w=plotw, h=ploth))

#==Return pcoll to user (call evalfile(...))
===============================================================================#
pcoll


include("Voronoiarea.jl")
include("Polyarea.jl")
include("Clip.jl")
include("Density.jl")


@doc doc"""
    Euclidean{T<:Tuple} <: Manifold

Euclidean vector space $\mathbb R^n$.

# Constructor

    Euclidean(n)

Generate the $n$-dimensional vector space $\mathbb R^n$.

    Euclidean(n₁,n₂,...,nᵢ; field=ℝ)

Generate the vector space of $k=n_1n_2\cdot\ldots n_i$ values, i.e. the
$\mathbb F^{n_1, n_2, \ldots, n_d}$ whose
elements are interpreted as $n_1 \times,n_2\times\cdots\times n_i$ arrays.
For $d=2$ we obtain a matrix space.
The default `field=ℝ` can also be set to `field=ℂ`.
The dimension of this space is $k \dim_ℝ 𝔽$, where $\dim_ℝ 𝔽$ is the
[`real_dimension`](@ref) of the field $𝔽$.
"""
struct Euclidean{N<:Tuple,T} <: Manifold where {N,T<:AbstractNumbers} end

function Euclidean(n::Vararg{Int,N}; field::AbstractNumbers = ℝ) where {N}
    return Euclidean{Tuple{n...},field}()
end

"""
    EuclideanMetric <: RiemannianMetric

A general type for any manifold that employs the Euclidean Metric, for example
the [`Euclidean`](@ref) manifold itself, or the [`Sphere`](@ref), where every
tangent space (as a plane in the embedding) uses this metric (in the embedding).

Since the metric is independent of the field type, this metric is also used for
the Hermitian metrics, i.e. metrics that are analogous to the `EuclideanMetric`
but where the field type of the manifold is `ℂ`.

This metric is the default metric for example for the [`Euclidean`](@ref) manifold.
"""
struct EuclideanMetric <: RiemannianMetric end

det_local_metric(M::MetricManifold{<:Manifold,EuclideanMetric}, x) = one(eltype(x))

"""
    distance(M::Euclidean, x, y)

Compute the Euclidean distance between two points on the [`Euclidean`](@ref)
manifold `M`, i.e. for vectors it's just the norm of the difference, for matrices
and higher order arrays, the matrix and ternsor Frobenius norm, respectively.
"""
distance(::Euclidean, x, y) = norm(x .- y)

@doc doc"""
    exp(M::Euclidean, x, v)

Compute the exponential map on the [`Euclidean`](@ref) manifold `M` from `x` in direction
`v`, which in this case is just
````math
\exp_x v = x + v.
````
"""
exp(::Euclidean, ::Any...)

exp!(M::Euclidean, y, x, v) = (y .= x .+ v)

"""
    flat(M::Euclidean, x, w)

Transform a tangent vector into a cotangent. Since they can directly be identified in the
[`Euclidean`](@ref) case, this yields just the identity for a tangent vector `w` in the
tangent space of `x` on `M`. The result is returned also in place in `v`.
"""
flat(::Euclidean, ::Any...)

flat!(M::Euclidean, v::CoTFVector, x, w::TFVector) = copyto!(v, w)

function get_basis(M::Euclidean{<:Tuple,ℝ}, x, B::ArbitraryOrthonormalBasis)
    vecs = [_euclidean_basis_vector(x, i) for i in eachindex(x)]
    return PrecomputedOrthonormalBasis(vecs)
end
function get_basis(M::Euclidean{<:Tuple,ℂ}, x, B::ArbitraryOrthonormalBasis)
    vecs = [_euclidean_basis_vector(x, i) for i in eachindex(x)]
    return PrecomputedOrthonormalBasis([vecs; im * vecs])
end
function get_basis(M::Euclidean, x, B::DiagonalizingOrthonormalBasis)
    vecs = get_basis(M, x, ArbitraryOrthonormalBasis()).vectors
    kappas = zeros(real(eltype(x)), manifold_dimension(M))
    return PrecomputedDiagonalizingOrthonormalBasis(vecs, kappas)
end

function get_coordinates(M::Euclidean{<:Tuple,ℝ}, x, v, B::ArbitraryOrDiagonalizingBasis)
    S = representation_size(M)
    PS = prod(S)
    return reshape(v, PS)
end
function get_coordinates(M::Euclidean{<:Tuple,ℂ}, x, v, B::ArbitraryOrDiagonalizingBasis)
    S = representation_size(M)
    PS = prod(S)
    return [reshape(real(v), PS); reshape(imag(v), PS)]
end

function get_vector(M::Euclidean{<:Tuple,ℝ}, x, v, B::ArbitraryOrDiagonalizingBasis)
    S = representation_size(M)
    return reshape(v, S)
end
function get_vector(M::Euclidean{<:Tuple,ℂ}, x, v, B::ArbitraryOrDiagonalizingBasis)
    S = representation_size(M)
    N = div(length(v), 2)
    return reshape(v[1:N] + im * v[N+1:end], S)
end

function hat(M::Euclidean{N,ℝ}, x, vⁱ) where {N}
    return reshape(vⁱ, representation_size(TangentBundleFibers(M)))
end

hat!(::Euclidean{N,ℝ}, v, x, vⁱ) where {N} = copyto!(v, vⁱ)

@doc doc"""
    injectivity_radius(M::Euclidean)

Return the injectivity radius on the [`Euclidean`](@ref) `M`, which is $\infty$.
"""
injectivity_radius(::Euclidean) = Inf

@doc doc"""
    inner(M::Euclidean, x, v, w)

Compute the inner product on the [`Euclidean`](@ref) `M`, which is just
the inner product on the real-valued or complex valued vector space
of arrays (or tensors) of size $n_1\times n_2 \times \cdots \times n_i$, i.e.

````math
g_x(v,w) = \sum_{k\in I} \overline{v}_{k} w_{k},
````
where $I$ is the set of integer vectors $k\in\mathbb N^i$, such that for all
$1 \leq j \leq i$ it holds $1\leq k_j \leq n_j$.

For the special case of $i\leq 2$, i.e. matrices and vectors, this simplifies to
````math
g_x(v,w) = w^{\mathrm{H}}v,
````
where $\cdot^{\mathrm{H}}$ denotes the hermitian, i.e. complex conjugate transposed.
"""
inner(::Euclidean, ::Any...)
@inline inner(::Euclidean, x, v, w) = dot(v, w)
@inline inner(::MetricManifold{<:Manifold,EuclideanMetric}, x, v, w) = dot(v, w)

inverse_local_metric(M::MetricManifold{<:Manifold,EuclideanMetric}, x) = local_metric(M, x)

is_default_metric(::Euclidean, ::EuclideanMetric) = Val(true)

function local_metric(::MetricManifold{<:Manifold,EuclideanMetric}, x)
    return Diagonal(ones(SVector{size(x, 1),eltype(x)}))
end

@doc doc"""
    log(M::Euclidean, x, y)

Compute the logarithmic map on the [`Euclidean`](@ref) `M` from `x` to `y`,
which in this case is just
````math
\log_x y = y - x.
````
"""
log(::Euclidean, ::Any...)

log!(M::Euclidean, v, x, y) = (v .= y .- x)

log_local_metric_density(M::MetricManifold{<:Manifold,EuclideanMetric}, x) = zero(eltype(x))

@generated _product_of_dimensions(::Euclidean{N}) where {N} = prod(N.parameters)

"""
    manifold_dimension(M::Euclidean)

Return the manifold dimension of the [`Euclidean`](@ref) `M`, i.e.
the product of all array dimensions and the [`real_dimension`](@ref) of the
underlying number system.
"""
function manifold_dimension(M::Euclidean{N,𝔽}) where {N,𝔽}
    return _product_of_dimensions(M) * real_dimension(𝔽)
end

mean(::Euclidean{Tuple{1}}, x::AbstractVector{<:Number}; kwargs...) = mean(x)
function mean(
    ::Euclidean{Tuple{1}},
    x::AbstractVector{<:Number},
    w::AbstractWeights;
    kwargs...,
)
    return mean(x, w)
end
mean(::Euclidean, x::AbstractVector; kwargs...) = mean(x)

function mean!(M::Euclidean, y, x::AbstractVector, w::AbstractVector; kwargs...)
    return mean!(M, y, x, w, GeodesicInterpolation(); kwargs...)
end

function mean_and_var(::Euclidean{Tuple{1}}, x::AbstractVector{<:Number}; kwargs...)
    m, v = mean_and_var(x; kwargs...)
    return m, sum(v)
end
function mean_and_var(
    ::Euclidean{Tuple{1}},
    x::AbstractVector{<:Number},
    w::AbstractWeights;
    corrected = false,
    kwargs...,
)
    m, v = mean_and_var(x, w; corrected = corrected, kwargs...)
    return m, sum(v)
end
function mean_and_var(M::Euclidean, x::AbstractVector, w::AbstractWeights; kwargs...)
    return mean_and_var(M, x, w, GeodesicInterpolation(); kwargs...)
end

median(::Euclidean{Tuple{1}}, x::AbstractVector{<:Number}; kwargs...) = median(x)
function median(
    ::Euclidean{Tuple{1}},
    x::AbstractVector{<:Number},
    w::AbstractWeights;
    kwargs...,
)
    return median(x, w)
end

function median!(::Euclidean{Tuple{1}}, y, x::AbstractVector; kwargs...)
    return copyto!(y, [median(vcat(x...))])
end
function median!(::Euclidean{Tuple{1}}, y, x::AbstractVector, w::AbstractWeights; kwargs...)
    return copyto!(y, [median(vcat(x...), w)])
end

@doc doc"""
    norm(M::Euclidean, x, v)

Compute the norm of a tangent vector `v` at `x` on the [`Euclidean`](@ref)
`M`, i.e. since every tangent space can be identified with `M` itself
in this case, just the (Frobenius) norm of `v`.
"""
norm(::Euclidean, x, v) = norm(v)
norm(::MetricManifold{<:Manifold,EuclideanMetric}, x, v) = norm(v)

"""
    normal_tvector_distribution(M::Euclidean, x, σ)

Normal distribution in ambient space with standard deviation `σ`
projected to tangent space at `x`.
"""
function normal_tvector_distribution(M::Euclidean{Tuple{N}}, x, σ) where {N}
    d = Distributions.MvNormal(zero(x), σ)
    return ProjectedFVectorDistribution(TangentBundleFibers(M), x, d, project_vector!, x)
end

@doc doc"""
    project_point(M::Euclidean, x)

Project an arbitrary point `x` onto the [`Euclidean`](@ref) `M`, which
is of course just the identity map.
"""
project_point(::Euclidean, ::Any...)

project_point!(M::Euclidean, x) = x

"""
    project_tangent(M::Euclidean, x, v)

Project an arbitrary vector `v` into the tangent space of a point `x` on the
[`Euclidean`](@ref) `M`, which is just the identity, since any tangent
space of `M` can be identified with all of `M`.
"""
project_tangent(::Euclidean, ::Any...)

project_tangent!(M::Euclidean, w, x, v) = copyto!(w, v)

"""
    projected_distribution(M::Euclidean, d, [x])

Wrap the standard distribution `d` into a manifold-valued distribution. Generated
points will be of similar type to `x`. By default, the type is not changed.
"""
function projected_distribution(M::Euclidean, d, x)
    return ProjectedPointDistribution(M, d, project_point!, x)
end
function projected_distribution(M::Euclidean, d)
    return ProjectedPointDistribution(M, d, project_point!, rand(d))
end

"""
    representation_size(M::Euclidean)

Return the array dimensions required to represent an element on the
[`Euclidean`](@ref) `M`, i.e. the vector of all array dimensions.
"""
@generated representation_size(::Euclidean{N}) where {N} = size_to_tuple(N)

"""
    sharp(M::Euclidean, x, w)

since cotangent and tangent vectors can directly be identified in the [`Euclidean`](@ref)
case, this yields just the identity for a cotangent vector `w` in the tangent space
of `x` on `M`.
"""
sharp(::Euclidean, ::Any...)

sharp!(M::Euclidean, v::TFVector, x, w::CoTFVector) = copyto!(v, w)

"""
    vector_transport_to(M::Euclidean, x, v, y, ::ParallelTransport)

Parallely transport the vector `v` from the tangent space at `x` to the tangent space at `y`
on the [`Euclidean`](@ref) `M`, which simplifies to the identity.
"""
vector_transport_to(::Euclidean, ::Any, ::Any, ::Any, ::ParallelTransport)

vector_transport_to!(M::Euclidean, vto, x, v, y, ::ParallelTransport) = copyto!(vto, v)

var(::Euclidean, x::AbstractVector; kwargs...) = sum(var(x; kwargs...))
function var(::Euclidean, x::AbstractVector{T}, m::T; kwargs...) where {T}
    return sum(var(x; mean = m, kwargs...))
end

vee(::Euclidean{N,ℝ}, x, v) where {N} = vec(v)

vee!(::Euclidean{N,ℝ}, vⁱ, x, v) where {N} = copyto!(vⁱ, v)

"""
    zero_tangent_vector(M::Euclidean, x)

Return the zero vector in the tangent space of `x` on the [`Euclidean`](@ref)
`M`, which here is just a zero filled array the same size as `x`.
"""
zero_tangent_vector(::Euclidean, ::Any...)

zero_tangent_vector!(M::Euclidean, v, x) = fill!(v, 0)


module EquationsSolver

using Symbolics
import Symbolics:@variables,Equation

include("base.jl")

abstract type EquationsProblem end
struct LinearProblem <: EquationsProblem
    eqs::Vector{Equation}
    vars::Vector{Num}
    maxiters::Int
end

struct NonlinearProblem <: EquationsProblem
    eqs::Vector{Num}
    vars::Dict
    maxiters::Int
    abstol::Float64
end

function LinearProblem(eqs::Any, vars::Dict, maxiters=10000)
    checked_eqs = check_eqs(eqs)
    res = check_vars(checked_eqs, vars)
    return LinearProblem(eqs, collect(keys(vars)), maxiters)
end
function NLProblem(eqs::Any, vars::Dict; maxiters=10000,abstol = 1.0E-6)
    eqs = check_eqs(eqs)
    res = check_vars(eqs, vars)
    return NonlinearProblem(eqs, vars, maxiters,abstol)
end

export LinearProblem
export NLProblem
export solve
export @variables,Equation
include("solver.jl")
end


using Distributed
using DelimitedFiles
using Printf

@everywhere using RAFF

# This script runs the parallel version of RAFF

"""

    run_praff()

Load and run the parallel/distributed version of RAFF. It assumes that
there is a problem file `/tmp/output.txt`.

"""
function run_praff(maxms=1, initguess=nothing; model_str="logistic", foutliers=0.5)

    n, model, modelstr = RAFF.model_list[model_str]

    open("/tmp/output.txt") do fp

        global N = parse(Int, readline(fp))

        global data = readdlm(fp)

    end

    if initguess == nothing

        initguess = zeros(Float64, n)

    end

    rsol = praff(model, data[:, 1:end - 1], n; MAXMS=maxms, initguess=initguess,
                 noutliers=Int(round(foutliers * size(data)[1])))

    @printf("Solution found:
            fbest = %f
            p     = %d\n", rsol.f, rsol.p)
    println(rsol.solution)

    return rsol

end


using ITensors
using Random

using ITensors: AbstractMPS

function fill_trivial_coefficients(ψ)
  return ψ isa AbstractMPS ? (1, ψ) : ψ
end

function inner_add(α⃗ψ⃗::Tuple{<:Number,<:MPST}...) where {MPST<:AbstractMPS}
  Nₘₚₛ = length(α⃗ψ⃗)
  α⃗ = first.(α⃗ψ⃗)
  ψ⃗ = last.(α⃗ψ⃗)
  N⃡ = (conj(α⃗[i]) * α⃗[j] * inner(ψ⃗[i], ψ⃗[j]) for i in 1:Nₘₚₛ, j in 1:Nₘₚₛ)
  return sum(N⃡)
end

inner_add(ψ⃗...) = inner_add(fill_trivial_coefficients.(ψ⃗)...)

# TODO: this is no longer needed, use randomMPS
function makeRandomMPS(sites; chi::Int=4)::MPS
  N = length(sites)
  v = Vector{ITensor}(undef, N)
  l = [Index(chi, "Link,l=$n") for n in 1:(N - 1)]
  for n in 1:N
    s = sites[n]
    if n == 1
      v[n] = randomITensor(l[n], s)
    elseif n == N
      v[n] = randomITensor(l[n - 1], s)
    else
      v[n] = randomITensor(l[n - 1], l[n], s)
    end
    normalize!(v[n])
  end
  return MPS(v, 0, N + 1)
end

function makeRandomMPO(sites; chi::Int=4)::MPO
  N = length(sites)
  v = Vector{ITensor}(undef, N)
  l = [Index(chi, "Link,l=$n") for n in 1:(N - 1)]
  for n in 1:N
    s = sites[n]
    if n == 1
      v[n] = ITensor(l[n], s, s')
    elseif n == N
      v[n] = ITensor(l[n - 1], s, s')
    else
      v[n] = ITensor(l[n - 1], s, s', l[n])
    end
    randn!(v[n])
    normalize!(v[n])
  end
  return MPO(v, 0, N + 1)
end

# Based on https://discourse.julialang.org/t/lapackexception-1-while-svd-but-not-svdvals/23787
function make_illconditioned_matrix(T=5000)
  t = 0:(T - 1)
  f = LinRange(0, 0.5 - 1 / length(t) / 2, length(t) ÷ 2)
  y = sin.(t)
  function check_freq(f)
    zerofreq = findfirst(iszero, f)
    zerofreq !== nothing &&
      zerofreq != 1 &&
      throw(ArgumentError("If zero frequency is included it must be the first frequency"))
    return zerofreq
  end
  function get_fourier_regressor(t, f)
    zerofreq = check_freq(f)
    N = length(t)
    Nf = length(f)
    Nreg = zerofreq === nothing ? 2Nf : 2Nf - 1
    N >= Nreg || throw(ArgumentError("Too many frequency components $Nreg > $N"))
    A = zeros(N, Nreg)
    sinoffset = Nf
    for fn in 1:Nf
      if fn == zerofreq
        sinoffset = Nf - 1
      end
      for n in 1:N
        phi = 2π * f[fn] * t[n]
        A[n, fn] = cos(phi)
        if fn != zerofreq
          A[n, fn + sinoffset] = -sin(phi)
        end
      end
    end
    return A, zerofreq
  end
  A, z = get_fourier_regressor(t, f)
  return [A y]
end


function mgc2mgc!(dst_ceps::AbstractVector, dst_α, dst_γ,
                  src_ceps::AbstractVector, src_α, src_γ)
    dst_order = length(dst_ceps) - 1
    α = (dst_α-src_α) / (1.0-dst_α*src_α)

    if α == zero(α)
        dst_ceps = gnorm(src_ceps, src_γ)
        dst_ceps = gc2gc(dst_ceps, src_γ, dst_order, dst_γ)
        ignorm!(dst_ceps, dst_γ)
    else
        freqt!(dst_ceps, src_ceps, α)
        gnorm!(dst_ceps, src_γ)
        dst_ceps = gc2gc(dst_ceps, src_γ, dst_order, dst_γ)
        ignorm!(dst_ceps, dst_γ)
    end

    dst_ceps
end

function mgc2mgc(src_ceps::AbstractVector,
                 src_α=0.0,
                 src_γ=0.0,
                 dst_order=length(src_ceps) - 1,
                 dst_α=0.0,
                 dst_γ=0.0)
    mgc2mgc!(Array(eltype(src_ceps), dst_order+1), dst_α, dst_γ,src_ceps, src_α, src_γ)
end

function mgc2mgc{T<:MelGeneralizedCepstrum}(state::SpectralParamState{T},
                                            dst_order=param_order(paramdef(state)),
                                            dst_α=allpass_alpha(paramdef(state)),
                                            dst_γ=glog_gamma(paramdef(state)))
    assert_not_ready_to_filt(state)

    def = paramdef(state)
    src_α = allpass_alpha(def)
    src_γ = glog_gamma(def)
    normalized = (dst_γ == zero(dst_γ))
    data = rawdata(state)
    newdef = MelGeneralizedCepstrum(dst_order, dst_α, dst_γ)
    SpectralParamState(newdef, mgc2mgc(data, src_α, src_γ, dst_order, dst_α, dst_γ),
                       has_loggain(state), normalized)
end


# Bridge scoring: based on

"""
    points(hands, trump, contract, double, vulnerable)

Returns score for the declarer given:

* hands: total number of hands taken by declarer [0:13]
* trump: 1=clubs 2=diamonds 3=hearts 4=spades 5=notrump
* contract: level of contract [1:7]
* double: 1=none 2=doubled 4=redoubled
* vulnerable: true/false

References:
* http://www.rpbridge.net/2y66.htm
* http://www.rpbridge.net/1ub8.htm

"""
function points(hands, trump, contract, double, vul)
    @assert in(hands,0:13) && in(trump,1:5) && in(contract,1:7) && in(double,(1,2,4)) && isa(vul,Bool)
    if hands >= 6 + contract
        trickscore = (contract * (trump <= 2 ? 20 : 30) + (trump == 5 ? 10 : 0)) * double
        overtricks = ((hands - 6 - contract) *
                      (double == 1 ? (trump <= 2 ? 20 : 30) :
                       double == 2 ? (vul ? 200 : 100) :
                       double == 4 ? (vul ? 400 : 200) : error()))
        gamebonus = (trickscore < 100 ? 50 : vul ? 500 : 300)
        slambonus = (contract == 6 ? (vul ? 750 : 500) :
                     contract == 7 ? (vul ? 1500 : 1000) : 0)
        doublebonus = (double == 2 ? 50 : double == 4 ? 100 : 0)
        trickscore + overtricks + gamebonus + slambonus + doublebonus
    else # hands < 6 + contract
        down = 6 + contract - hands
        (double == 1 ? (vul ? -100 : -50) * down :
         vul ? -(down * 300 - 100) * div(double,2) :
         (down == 1 ? -100 : down == 2 ? -300 : -down * 300 + 400) * div(double,2))
    end
end

"""
    imps(points)

Return IMP score given (positive) difference in points.
Based on https://www.bridgehands.com/I/IMP.htm.

"""
function imps(points)
    @assert points > 0
    const a = [20,50,90,130,170,220,270,320,370,430,500,600,750,900,1100,1300,1500,1750,2000,2250,2500,3000,3500,4000,10581]
    for i=1:length(a)
        if points < a[i]; return i-1; end
    end
end

# See http://bridgecomposer.com/Par.htm

# Let us assume we have the following deal from library.gz:
# JT852.93.KQ7.J82 AQ97.JT654.T6.A5 43.AK8.A542.7643 K6.Q72.J983.KQT9:88887777A9A977778888
# Hands are given in W,N,E,S order.
# Results are N/S tricks in hex grouped by NT,S,H,D,C.
# Each group has 4 results with S,E,N,W leading.

# The final contract reached by the model should specify trump, level,
# double, vul info.  Vul and dealer can be picked randomly at the
# beginning of bidding. The hands won comes from the DD and points() can
# be used to calculate score.

# The par score is the highest bid on which neither side can
# improve. We can assume all down hands are doubled, and no final par
# contract is redoubled.  Vul determined at start, dealer not
# important, declarer should be the first person bidding the trump
# suit but we can assume it can be freely chosen by the declaring
# partnership.

# Wrong: dealer may be important if both sides can make 1N, for
# example, then the first one to bid gets the score.

# The input is the 20 results and vulnerability, the output is the par N/S score.

"""

    parscore(results; dealer, nsvul, ewvul)

Returns a tuple with the parscore and some other info.
* results[leader,trump] is the number of tricks NS can take with double dummy
* dealer=SOUTH: 1=west, 2=north, 3=east, 4=south
* nsvul=false: true/false for NS vulnerability
* ewvul=false: true/false for EW vulnerability

References:
* http://bridgecomposer.com/Par.htm

"""
function parscore(results; dealer=SOUTH, nsvul=false, ewvul=false)
    par,bid,iter,decl = 0,0,0,mod1(dealer-1,4)
    while true
        iter += 1
        saved = par,bid,decl
        @show saved
        decl1,decl2 = mod1(decl+1,4),mod1(decl-1,4)
        lead1,lead2 = mod1(decl+2,4),decl
        for newbid in bid+1:TRUMPBIDS
            level,trump = bidlevel(newbid),bidtrump(newbid)
            if iseven(decl1)
                make1,make2,vul = results[lead1,trump],results[lead2,trump],nsvul
                newdecl,makes = (make1 >= make2 ? (decl1,make1) : (decl2,make2))
                double = (makes >= level+6 ? 1 : 2)
                newpar = points(makes, trump, level, double, vul)
                if newpar >= par; par,bid,decl = newpar,newbid,newdecl; end
            else
                make1,make2,vul = 13-results[lead1,trump],13-results[lead2,trump],ewvul
                newdecl,makes = (make1 >= make2 ? (decl1,make1) : (decl2,make2))
                double = (makes >= level+6 ? 1 : 2)
                newpar = -points(makes, trump, level, double, vul)
                if newpar <= par; par,bid,decl = newpar,newbid,newdecl; end
            end
        end
        if par == 0
            decl = mod1(decl+1,4)
        elseif (par,bid,decl) == saved
            break
        elseif iter > 10
            error("Too many iterations")
        end
    end
    return par,bid,decl,iter
end


# This file is a part of BAT.jl, licensed under the MIT License (MIT).

using BAT
using Test

using Random, Statistics, LinearAlgebra
using Distributions
using QuadGK, ForwardDiff
using StableRNGs
using StatsBase, FillArrays

@testset "standard_normal" begin
    stblrng() = StableRNG(789990641)

    @test @inferred(BAT.LogUniform(0.1f0, 100)) isa BAT.LogUniform{Float32}
    @test_throws ArgumentError BAT.LogUniform(0, 100)
    @test_throws ArgumentError BAT.LogUniform(100, 10)

    d = BAT.LogUniform(0.1, 100)
    X = @inferred(rand(stblrng(), d, 10^5))

    @test BAT.LogUniform() == BAT.LogUniform{Float64}(1, 2)

    @test params(d) == (0.1, 100)
    @test @inferred(minimum(d)) == 0.1
    @test @inferred(maximum(d)) == 100

    @test Distributions.location(d) == d.a
    @test Distributions.scale(d) == d.b - d.a

    @test isapprox(@inferred(quantile(d, 0.3)), quantile(X, 0.3), rtol = 0.05)
    @test quadgk(x -> pdf(d, x), minimum(d), maximum(d))[1] ≈ 1
    @test cdf(d, minimum(d)) ≈ 0
    @test cdf(d, maximum(d)) ≈ 1

    @test isapprox(@inferred(mean(d)), mean(X), rtol = 0.05)
    @test isapprox(@inferred(median(d)), median(X), rtol = 0.05)
    @test @inferred(mode(d)) == minimum(d)
    @test isapprox(@inferred(var(d)), var(X), rtol = 0.05)
    @test isapprox(@inferred(std(d)), std(X), rtol = 0.05)

    @test StatsBase.modes(d) == Fill(mode(d),0)
    @test Distributions.logccdf(d, 90) == log(ccdf(d, 90))
    @test Distributions.ccdf(d, 90) == one(cdf(d, 90)) - cdf(d, 90)

    @test Distributions.cquantile(d, 0.5) == one(quantile(d, 0.5)) - quantile(d, 0.5)

    @test Distributions.truncated(d, 10 + 1 / 3, 90 + 1 / 3) == BAT.LogUniform(promote(max(10 + 1 / 3,d.a), min(90 + 1 /3, d.b))...)

    for x in (minimum(d), maximum(d), (minimum(d) + maximum(d)) / 2.9, (minimum(d) + maximum(d)) / 5.2)
        @test @inferred(quantile(d, @inferred(cdf(d, x)))) ≈ x
        @test ForwardDiff.derivative(x -> cdf(d, x), x) ≈ @inferred(pdf(d, x))
        @test log(pdf(d, x)) ≈ @inferred(logpdf(d, x))
        @test log(cdf(d, x)) ≈ @inferred(logcdf(d, x))
    end
end


using Plots
# physical coordinates
inner_radius = 6378100 # [m]
outer_radius = 6378100 + 100000 # [m] adding 100km for the atmosphere [about 1.6%] (its 3km for the ocean)
r1 = (inner_radius / outer_radius)
#

nr = 5
nh = 10
ξ¹ = reshape(collect(range(-1,1, length = nh)), (1,nh))
ξ² = reshape(collect(range(-1,1, length = nr)) , (nr,1))

r1 = 0.5 # inner radius
r2 = 1.0 # outer radius
npolygon = 5 # polygon approximation
φ = 2π/(2 * npolygon)
x¹ = @. ( (r2-r1)*sin(φ) * (ξ² + 1)/2 + r1 * sin(φ) ) * ξ¹
x² = @. (r2-r1)*cos(φ) * (ξ²+1)/2 + 0 * ξ¹ + r1*cos(φ)
p1 = scatter(x¹[:], x²[:], xlims = (-r2*1.1,r2*1.1), ylims = (-r2*1.1,r2*1.1), legend = false,aspectratio = 1)

##
for j in 1:1:(npolygon-1)
    θ = 2π/npolygon * j
    xrot = @. cos(θ) * x¹ - sin(θ) * x²
    yrot = @. sin(θ) * x¹ + cos(θ) * x²
    p1 = scatter!(xrot[:], yrot[:], xlims = (-r2*1.1,r2*1.1), ylims = (-r2*1.1,r2*1.1))
    display(p1)
    sleep(0.1)
end

plot(p1)

##
j=npolygon
θ = 2π/npolygon * j
xrot = @. cos(θ) * x¹ - sin(θ) * x²
yrot = @. sin(θ) * x¹ + cos(θ) * x²
p1 = scatter(xrot[:], yrot[:], xlims = (-r2*1.1,r2*1.1), ylims = (-r2*1.1,r2*1.1), aspectratio = 1, legend = false)
display(p1)
sleep(0.1)
scatter!(x¹[:], x²[:], xlims = (-r2*1.1,r2*1.1), ylims = (-r2*1.1,r2*1.1), legend = false,aspectratio = 1)


using Pkg
Pkg.activate(".")

# this is the code for the specific model
include("SVlib.jl")

# controls and characteristics of model
global const n = 1000
global const burnin = 100

# the following defines a function that returns a draw of the statistic, given the parameter
global const WileE_model = θ -> sqrt(n)*aux_stat(SVmodel(θ::Array{Float64,1}, n::Int64, burnin::Int64)[1])

global const θtrue = [exp(-0.736/2.0), 0.9, 0.363]
global const lb = [0.05, 0.0, 0.05]
global const ub = [2.0, 0.99, 1.0]
global const nParams = 3

# controls of raw draws from prior, for training net
global const TrainingTestingSize = Int64(1e4) # set fairly large, to limit monte carlo error
global const TrainingProportion = 0.5
global const Epochs = 1000 # passes through entire training set

println("Ready to run the SV project!");


immutable Node{T}
    isleaf::Bool
    index::Int
    radius::T
    inside::Node{T}
    outside::Node{T}

    Node{T}() where T = new(true, zero(Int))
    Node{T}(k::Int) where T = new(true, k)
    Node{T}(k::Int, d::T, l::Node{T}, r::Node{T}) where T = new(false, k, d, l, r)

    function Node{T}(metric::Function, indices::Vector{Int}, i::Int, j::Int) where T
        j <  i && return Node{T}()
        @inbounds j == i && return Node{T}(indices[i])

        # Choose a random element as the vantage point
        r = rand(i:j)
        @inbounds vp = indices[r]
        @inbounds indices[i], indices[r] = indices[r], indices[i]
        i += 1

        # Find the index of the median
        m = (i + j) >> 1

        d = quickselect!(T, metric, indices, vp, i, j, m)::T

        inside = Node{T}(metric, indices, i, m)
        outside = Node{T}(metric, indices, m + 1, j)
        Node{T}(vp, d, inside, outside)
    end
end

immutable VPTree
    num_points::Int
    root::Node
    metric::Function
end

function partition!(metric::Function, indices::Vector{Int}, vp::Int, d, i::Int, j::Int)
    while true
        @inbounds while metric(vp, indices[i]) <= d
            i == j && return j
            i += 1
        end

        @inbounds while metric(vp, indices[j]) > d
            i == j && return j - 1
            j -= 1
        end

        @inbounds indices[i], indices[j] = indices[j], indices[i]
    end
end

function quickselect!{T}(::Type{T}, metric::Function, indices::Vector{Int}, vp::Int, i::Int, j::Int, k::Int)
    while true
        # Choose a random element as the pivot
        r = rand(i:j)
        @inbounds d = metric(vp, indices[r])::T
        n = partition!(metric, indices, vp, d, i, j)

        if n > k
            j = n
        elseif n < k
            i = n + 1
        else
            return d
        end
    end
end

"""
    VPTree(metric, indices)

Construct a VPTree by providing

1) a function metric(i::Int, j::Int) that computes the distance between point i and point j
2) a list of point indices for which the VP Tree is constructed
"""
function VPTree(metric::Function, indices::Vector{Int})
    n = length(indices)
    T = typeof(metric(first(indices), first(indices)))
    root = Node{T}(metric, indices, 1, n)
    VPTree(n, root, metric)
end

function distsq(points::Matrix, i1::Int, i2::Int)
    x = points[1, i1] - points[1, i2]
    y = points[2, i1] - points[2, i2]
    z = points[3, i1] - points[3, i2]

    x^2 + y^2 + z^2
end


struct Differential <: AbstractOperator
    x::Union{Variable,Operation}
    order::Int
end
Differential(x) = Differential(x,1)

Base.show(io::IO, D::Differential) = print(io,"($(D.x),$(D.order))")
Base.Expr(D::Differential) = :($(Symbol("D_$(D.x.name)_$(D.order)")))

function Derivative end
Base.:*(D::Differential,x::Operation) = Operation(Derivative,Expression[x,D])
function Base.:*(D::Differential,x::Variable)
    if D.x === x
        return Constant(1)
    elseif D.x ∉ x.dependents
        return Constant(0)
    else
        return Variable(x,D)
    end
end
Base.:(==)(D1::Differential, D2::Differential) = D1.order == D2.order && D1.x == D2.x

Variable(x::Variable,D::Differential) = Variable(x.name,x.value,x.value_type,
                x.subtype,D,x.dependents,x.description,x.flow,x.domain,
                x.size,x.context)

function expand_derivatives(O::Operation)
    if O.op == Derivative
        D = O.args[2]
        o = O.args[1]
        simplify_constants(sum(i->Derivative(o,i)*expand_derivatives(D*o.args[i]),1:length(o.args)))
    else
        for i in 1:length(O.args)
            O.args[i] = expand_derivatives(O.args[i])
        end
    end
end
expand_derivatives(x::Variable) = x

# Don't specialize on the function here
function Derivative(O::Operation,idx)
    # This calls the Derivative dispatch from the user or pre-defined code
    Derivative(O.op,O.args,Val{idx})
end

# Pre-defined derivatives
import DiffRules, SpecialFunctions, NaNMath
for (modu, fun, arity) in DiffRules.diffrules()
    if arity ==  1 && !(fun in (:-, :+)) # :+ and :- are both unary and binary operators
        @eval begin
            function Derivative(::typeof($modu.$fun), arg, ::Type{Val{1}})
                M, f = $(modu, fun)
                @assert length(arg) == 1 "$M.$f is a unary function!"
                dx = DiffRules.diffrule(M, f, arg[1])
                parse(Operation,dx)
            end
        end
    elseif arity ==  2
        for i in 1:2
            @eval begin
                function Derivative(::typeof($modu.$fun), args, ::Type{Val{$i}})
                    M, f =  $(modu, fun)
                    if f in (:-, :+)
                        @assert length(args) in (1, 2) "$M.$f is a unary or a binary function!"
                    else
                        @assert length(args) == 2 "$M.$f is a binary function!"
                    end
                    dx = DiffRules.diffrule(M, f, args[1], args[2])[$i]
                    parse(Operation,dx)
                end
            end
        end
    end
end

function count_order(x)
    @assert !(x isa Symbol) "The variable $x must have an order of differentiation that is greater or equal to 1!"
    n = 1
    while !(x.args[1] isa Symbol)
        n = n+1
        x = x.args[1]
    end
    n, x.args[1]
end

function _differetial_macro(x)
    ex = Expr(:block)
    lhss = Symbol[]
    x = flatten_expr!(x)
    for di in x
        @assert di isa Expr && di.args[1] == :~ "@Deriv expects a form that looks like `@Deriv D''~t E'~t`"
        lhs = di.args[2]
        rhs = di.args[3]
        order, lhs = count_order(lhs)
        push!(lhss, lhs)
        expr = :($lhs = Differential($rhs, $order))
        push!(ex.args,  expr)
    end
    push!(ex.args, Expr(:tuple, lhss...))
    ex
end

macro Deriv(x...)
    esc(_differetial_macro(x))
end

function calculate_jacobian(eqs,vars)
    Expression[Differential(vars[j])*eqs[i] for i in 1:length(eqs), j in 1:length(vars)]
end

export Differential, Derivative, expand_derivatives, @Deriv, calculate_jacobian