### A Pluto.jl notebook ###
# v0.16.1

using Markdown
using InteractiveUtils

# ╔═╡ 7c78684c-5608-4184-b7ca-9fe42c09f3f9
begin
  using PlutoUI
  #using LaTeXStrings
  using TikzPictures
end

# ╔═╡ d8aa0d5c-273f-11ec-3d4e-d9c3dbb94278
md"""
# B.5 Trees
By definition, a _**free tree**_ is an undirected, connected, acyclic graph. At first glance, this seems to be
an object with many (`3` to be precise) adjectives. But, once we started to have a deeper understanding, we
might come to realize that a free tree is just an undirected graph that is
```math
\newcommand{\norm}[1]{\lVert{#1}\rVert}
\newcommand{\abs}[1]{\lvert{#1}\rvert}
```

- connected
  - (Exercise B.4-3) If ``G = (V, E)`` is an undirected, connected graph then ``\abs{E} \ge \abs{V} - 1``.
  - In day-to-day English, this simply says,
    > "For an undirected graph to be connected, there cannot be too few edges; there must exist at least ``\abs{V} - 1`` edges."
- acyclic
  - We prove an analogous result: If ``G = (V, E)`` is an undirected, acyclic graph then ``\abs{E} \le \abs{V} - 1``.
  - This simply says analogously,
    > "For an undirected graph to remain acyclic, there cannot be too many edges. Too many edges create cycles. There can be at most ``\abs{V} - 1`` edges."

_Proof._$(HTML("
"))
We show that for any acyclic undirected graph ``G = (V, E)``, we have ``\abs{E} \le \abs{V} - 1``.

Note that this is not saying that cyclic graphs tend to have many more edges than acyclic graphs do. Indeed, three edges btw three distinct vertices suffice to make a graph cyclic. I find it useful to picture in our head like this:
> We are given the vertex set ``V`` and an empty edge set ``E``.
> We search to add as many edges to ``E`` as possible in keeping ``G`` acyclic.

Again, we shall prove this by mathematical induction on ``\abs{V}``. It is easy to
verify the validity of the statement when ``\abs{V} = 1, 2``. When ``\abs{V} = 3``, assume that ``G`` is acyclic. If ``\abs{E} = 3``, then ``G`` contains a cycle; in fact, in this case, ``G`` is itself a cycle. So we can conclude ``\abs{E} \le 2 = \abs{V} - 1.`` (Three vertices admit at most ``\begin{pmatrix} 3 \\ 2\end{pmatrix} = 3`` edges, and,  since ``3`` is excluded, we must have ``\abs{E} \le 2``.)

Now, assume that the statement holds for ``\abs{V} = 1, 2, 3, \ldots, k``. Let ``G = (V, E)`` be an acyclic undirected graph with ``\abs{V} = k+1``.
First, we would like to
01. Find some vertex ``u \in V``
02. Exclude ``u`` from ``V`` and ``E``. That is, obtain a new graph ``G_{1} = (V_{1}, E_{1})``, subgraph of ``G``, such that ``u`` is neither in ``V_{1}`` nor in any edge from ``E_{1}``
03. Argue that ``\abs{E} = \abs{E_{1}} + 1 \le (\abs{V_{1}} - 1) + 1 = \abs{V} - 1``

We soon realize that not every vertex of ``V`` can play the role of ``u`` above. Indeed, for the third argument above to work, we need ``\deg(u) = 1``.
In other words, we would like to pick ``u`` such that it is adjent to only one other vertex. Now, we give the argument that such ``u`` exists. More precisely, we show that
```math
\exists\, u \in V \;\text{s.t.}\; \deg(u) \le 1.
```
This can be shown using proof by contradiction: Assume the contrary, i.e. that every vertex in ``V`` has degree greater than or equal to ``2``. Then starting from any vertex ``v`` of ``V``, which we put at the left of the following figure, we will always arrive at the conclusion that there exists a cycle in ``V``.

Once the existence of ``u`` s.t. ``\deg(u) \le 1`` is established, it is easy to prove the rest.

- If ``\deg(u) = 0``, then removing ``u`` makes the aforementioned ``G_{1} = (V_{1}, E_{1})`` an acyclic graph of ``k`` vertices, whence
  ```math
  \abs{E} = \abs{E_{1}} \le (\abs{V} - 1) - 1 \le \abs{V} - 1.
  ```
- Similarly, if ``\deg(u) = 1``, then removing ``u`` makes ``G_{1}`` an acyclic graphs of ``k`` vertices, whence
  ```math
  \abs{E} = \abs{E_{1}} + 1 \le (\abs{V_{1}} - 1) + 1 = \abs{V_{1}} = \abs{V} - 1.
  ```
This concludes the proof.
"""

# ╔═╡ 684a24c0-9ca9-46d6-923f-f35c9129d221

# ╔═╡ 1a24de70-4203-475d-b183-086c6e0e392c

# ╔═╡ 4e5a7a11-397a-4f02-835f-cf56ff8b2ead

# ╔═╡ 01a0665d-794c-4f72-80cc-237c59753f94

# ╔═╡ 638c65ab-3eb4-416b-8adf-110649dac5c4
md"""
## Theorem of The Section
Since a tree (i.e. a free tree. It seems that we often drop the adjective and speak of trees as free trees) is at the same time connected and acyclic, from the above relation btw number of edges and number of vertices, we should have
```math
\abs{E} = \abs{V} - 1.
```

**Theorem B.2 (Properties of free trees)**$(HTML("
"))
The followings are equiv:
"""

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
TikzPictures = "37f6aa50-8035-52d0-81c2-5a1d08754b2d"

[compat]
PlutoUI = "~0.7.14"
TikzPictures = "~3.4.1"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "f2202b55d816427cd385a9a4f3ffb226bee80f99"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.1+0"

[[CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b3bfd02e98aedfa5cf885665493c5598c350cd2f"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.2.10+0"

[[Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.93+0"

[[FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.4+0"

[[Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "7bf67e9a481712b3dbe9cb3dac852dc4b1162e02"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+0"

[[HypertextLiteral]]
git-tree-sha1 = "f6532909bf3d40b308a0f360b6a0e626c0e263a8"
uuid = "ac1192a8-f4b3-4bfe-ba22-af5b92cd3ab2"
version = "0.9.1"

[[IOCapture]]
deps = ["Logging", "Random"]
git-tree-sha1 = "f7be53659ab06ddc986428d3a9dcc95f6fa6705a"
uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89"
version = "0.2.2"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.3.0"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "8076680b162ada2a031f707ac7b4953e30667a37"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.2"

[[JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "d735490ac75c5cb9f1b00d8b5509c11984dc6943"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.0+0"

[[LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[LaTeXStrings]]
git-tree-sha1 = "c7f1c695e06c01b95a67f0cd1d34994f3e7db104"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.2.1"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "761a393aeccd6aa92ec3515e428c26bf99575b3b"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+0"

[[Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "340e257aada13f95f98ee352d316c3bed37c8ab9"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+0"

[[Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[LittleCMS_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pkg"]
git-tree-sha1 = "110897e7db2d6836be22c18bffd9422218ee6284"
uuid = "d3a379c0-f9a3-5b72-a4c0-6bf4d2e8af0f"
version = "2.12.0+0"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[OpenJpeg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libtiff_jll", "LittleCMS_jll", "Pkg", "libpng_jll"]
git-tree-sha1 = "76374b6e7f632c130e78100b166e5a48464256f8"
uuid = "643b3616-a352-519d-856d-80112ee9badc"
version = "2.4.0+0"

[[PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "a8709b968a1ea6abc2dc1967cb1db6ac9a00dfb6"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.0.5"

[[Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[PlutoUI]]
deps = ["Base64", "Dates", "HypertextLiteral", "IOCapture", "InteractiveUtils", "JSON", "Logging", "Markdown", "Random", "Reexport", "UUIDs"]
git-tree-sha1 = "d1fb76655a95bf6ea4348d7197b22e889a4375f4"
uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
version = "0.7.14"

[[Poppler_jll]]
deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "OpenJpeg_jll", "Pkg", "libpng_jll"]
git-tree-sha1 = "e11443687ac151ac6ef6699eb75f964bed8e1faa"
uuid = "9c32591e-4766-534b-9725-b71a8799265b"
version = "0.87.0+2"

[[Preferences]]
deps = ["TOML"]
git-tree-sha1 = "00cfd92944ca9c760982747e9a1d0d5d86ab1e5a"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.2"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "4036a3bd08ac7e968e27c203d45f5fff15020621"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.1.3"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[Tectonic]]
deps = ["Pkg"]
git-tree-sha1 = "acf12eccb390a78653ee805cd527898f01f78a85"
uuid = "9ac5f52a-99c6-489f-af81-462ef484790f"
version = "0.6.1"

[[TikzPictures]]
deps = ["LaTeXStrings", "Poppler_jll", "Requires", "Tectonic"]
git-tree-sha1 = "a08671c0979063a437378f6410bb75a465f3cd1c"
uuid = "37f6aa50-8035-52d0-81c2-5a1d08754b2d"
version = "3.4.1"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "cc4bf3fdde8b7e3e9fa0351bdeedba1cf3b7f6e6"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.0+0"

[[libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
"""

# ╔═╡ Cell order:
# ╠═7c78684c-5608-4184-b7ca-9fe42c09f3f9
# ╠═d8aa0d5c-273f-11ec-3d4e-d9c3dbb94278
# ╠═684a24c0-9ca9-46d6-923f-f35c9129d221
# ╠═1a24de70-4203-475d-b183-086c6e0e392c
# ╠═4e5a7a11-397a-4f02-835f-cf56ff8b2ead
# ╠═01a0665d-794c-4f72-80cc-237c59753f94
# ╟─638c65ab-3eb4-416b-8adf-110649dac5c4
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


using Test, DayCounts, Dates, ExcelFiles

@testset "Excel basis 0" begin
    table = load("yearfrac.xlsx","basis0")
    dc = DayCounts.Thirty360Excel()
    for r in ExcelFiles.getiterator(table)
        @test yearfrac(Date(r.startdate), Date(r.enddate), dc) == r.yearfrac
    end
end

@testset "Excel basis 1" begin
    table = load("yearfrac.xlsx","basis1")
    dc = DayCounts.ActualActualExcel()
    for r in ExcelFiles.getiterator(table)
        @test yearfrac(Date(r.startdate), Date(r.enddate), dc) == r.yearfrac
    end
end

@testset "Excel basis 2" begin
    table = load("yearfrac.xlsx","basis2")
    dc = DayCounts.Actual360()
    for r in ExcelFiles.getiterator(table)
        @test yearfrac(Date(r.startdate), Date(r.enddate), dc) == r.yearfrac
    end
end

@testset "Excel basis 3" begin
    table = load("yearfrac.xlsx","basis3")
    dc = DayCounts.Actual365Fixed()
    for r in ExcelFiles.getiterator(table)
        @test yearfrac(Date(r.startdate), Date(r.enddate), dc) == r.yearfrac
    end
end

@testset "Excel basis 4" begin
    table = load("yearfrac.xlsx","basis4")
    dc = DayCounts.ThirtyE360()
    for r in ExcelFiles.getiterator(table)
        @test yearfrac(Date(r.startdate), Date(r.enddate), dc) == r.yearfrac
    end
end


using Test, RangeEnclosures
using RangeEnclosures: relative_precision, available_solvers

using DynamicPolynomials: @polyvar

include("univariate.jl")
include("multivariate.jl")


# Bogumił Kamiński, 2022

# Codes for chapter 14

# Codes for section 14.1

using Plots
using Statistics

X = [1.0, 1.1, 1.3, 1.2, 1.2]
T = 1.0
m = 4
Y = mean(X)
K = 1.05
plot(range(0.0, T length=m+1), X;
           xlabel="T", legend=false, color="black")
hline!([Y], color="gray", lw=3, ls=:dash)
hline!([K], color="gray", lw=3, ls=:dot)
annotate!([(T, Y + 0.01, "Y"),
           (T, K + 0.01, "K"),
           (T, X[end] + 0.01, "X")])

using DataFrames
using Random
Random.seed!(1234);
X0, T, s, r, m = 1.0, 2.0, 0.2, 0.1, 4
gbm = DataFrame(X=X0, t=0.0)
for i in 1:m
    Z = randn()
    log_return = (r - s^2/2) * T/m + s * sqrt(T/m) * Z
    next_X = gbm.X[end] * exp(log_return)
    next_t = gbm.t[end] + T/m
    push!(gbm, (next_X, next_t))
end
gbm

# Codes for section 14.2

# start Julia with and additional -t4 command line switch

Threads.nthreads()

# Code for listing 14.1

function payoff_asian_sample(T, X0, K, r, s, m)::Float64
    X = X0
    sumX = X
    d = T / m
    for i in 1:m
        X *= exp((r - s^2 / 2) * d + s * sqrt(d) * randn())
        sumX += X
    end
    Y = sumX / (m + 1)
    return exp(-r * T) * max(Y - K, 0)
end

# Code for checking the results of the payoff simulation

payoff_asian_sample(1.0, 50.0, 55.0, 0.05, 0.3, 200)
payoff_asian_sample(1.0, 50.0, 55.0, 0.05, 0.3, 200)
payoff_asian_sample(1.0, 50.0, 55.0, 0.05, 0.3, 200)

# Benchmarking map

using BenchmarkTools
@btime map(i -> payoff_asian_sample(1.0, 50.0, 55.0, 0.05, 0.3, 200), 1:10_000);

using ThreadsX
@btime ThreadsX.map(i -> payoff_asian_sample(1.0, 50.0, 55.0, 0.05, 0.3, 200), 1:10_000);

# Codes for section 14.3

# Code for listing 14.2

using Statistics

function asian_value(T, X0, K, r, s, m, max_time)
    result = Float64[]
    start_time = time()
    while time() - start_time < max_time
        append!(result, ThreadsX.map(i -> payoff_asian_sample(T, X0, K, r, s, m), 1:10_000))
    end
    n = length(result)
    mv = mean(result)
    sdv = std(result)
    lo95 = mv - 1.96 * sdv / sqrt(n)
    hi95 = mv + 1.96 * sdv / sqrt(n)
    zero = mean(==(0), result)
    return (; n, mv, lo95, hi95, zero)
end

# Code for example of the mean function

mean(x -> x ^ 2, [1, 2, 3])

eq0 = ==(0)
eq0(1)
eq0(0)

# Code for shorthand NamedTuple notation

val1 = 10
val2 = "x"
(; val1, val2)

# Testing asian_value function

@time asian_value(1.0, 50.0, 55.0, 0.05, 0.3, 200, 0.25)
@time asian_value(1.0, 50.0, 55.0, 0.05, 0.3, 200, 0.25)
@time asian_value(1.0, 50.0, 55.0, 0.05, 0.3, 200, 0.25)

# Converting NamedTuple to JSON response

using Genie
Genie.Renderer.Json.json((firstname="Bogumił", lastname="Kamiński"))

# Codes for section 14.4

# Start a new Julia session

using HTTP
using JSON3
req = HTTP.post("http://127.0.0.1:8000",
                ["Content-Type" => "application/json"],
                JSON3.write((K=55.0, max_time=0.25)))
JSON3.read(req.body)

JSON3.write((K=55.0, max_time=0.25))

HTTP.post("http://127.0.0.1:8000",
          ["Content-Type" => "application/json"],
          JSON3.write((K="", max_time=0.25)))

using DataFrames
df = DataFrame(K=30:2:80, max_time=0.25)
df.data = map(df.K, df.max_time) do K, max_time
    @show K
    @time req = HTTP.post("http://127.0.0.1:8000",
                          ["Content-Type" => "application/json"],
                          JSON3.write((;K, max_time)))
    return JSON3.read(req.body)
end;

df

all(x -> x.status == "OK", df.data)

small_df = DataFrame(x=[(a=1, b=2), (a=3, b=4), (a=5, b=6)])
transform(small_df, :x => identity => AsTable)
transform(small_df, :x => AsTable)

df2 = select(df, :K, :data => ByRow(x -> x.value) => AsTable)

using Plots
plot(plot(df2.K, df2.mv; legend=false,
          xlabel="K", ylabel="expected value"),
     plot(df2.K, df2.zero; legend=false,
          xlabel="K", ylabel="probability of zero"))


using PyPlot

"""
    s = find2dPIS(u, ABCD, options)

Find a positively invariant set for the 2nd-order binary modulator whose
loop filter is described by `ABCD` and whose input is a constant `u`.
Return `s` = [] if no invariant set is found.

`options` = [dbg, itnLimit, expFactor, N, skip] defaults to [0, 100, 0.005, 1000, 100],
dbg=1 causes debugging information to be displayed.
"""
function find2dPIS(u, ABCD, options=[])
    n = size(ABCD, 1) - 1
    if n != 2
        error("The modulator must be second order.")
    end

    nlev = 2
    dbg = (length(options) >= 1) ? options[1] : false
    itnLimit = (length(options) >= 2) ? options[2] : 100
    expFactor = (length(options) >= 3) ? options[3] : 0.005
    N = (length(options) >= 4) ? options[4] : 1000
    skip = (length(options) >= 5) ? options[5] : 100

    # Compute a few iterations of difference equations
    if size(u) == ()
        un = u * ones(N+skip)
    else
        error("Argument 1 (u) has the wrong dimensions.")
    end
    v, x, = simulateDSM(un, ABCD, nlev)
    x = x[:,skip+1:skip+N]

    xmin = minimum(x[1,:])
    xmax = maximum(x[1,:])
    dx = xmax - xmin

    ymin = minimum(x[2,:])
    ymax = maximum(x[2,:])
    dy = ymax - ymin

    axis1 = [xmin-dx/4, xmax+dx/4, ymin-dy/4, ymax+dy/4]

    # Take the convex hull of the result
    s = hull2d(x')[1]'
    ec = mean(x, dims=2)

    for i = 1:itnLimit
        # Inflate the hull
        shift = ec * ones(1, size(s, 2))
        s = shift + (1 + expFactor) * (s - shift)

        # Split the set
        splus, eplus, sminus, eminus = dssplit2d(u, ABCD, s)
        # Map the two halves
        s1 = dsmap(u, ABCD, 2, splus, eplus, 1)
        s2 = dsmap(u, ABCD, 2, sminus, eminus, -1)
        ns = [s1[:,1:size(s1,2)-1] s2[:,1:size(s2,2)-1]]

        # Test for inclusion: ns inside s (the inflated hull)
        out = outconvex2d(ns, s)

        if dbg
            clf()
            grid()
            axis(axis1)
            dotplot(x, "k", ".")
            dotplot(ec, "r", "o")
            polyplot(s, "b")
            polyplot(s1, "m")
            polyplot(s2, "c")
            outi = (out .!= 0)[:]
            dotplot(ns[:,outi], "r", "s")
            str = @sprintf("Iteration %d: %d image vertices outside", i, sum(outi))
            title(str)
            sleep(0.1)
        end

        if all(out .== 0)
            break
        end

        s = hull2d(ns')[1]'
    end

    return s
end


#!/usr/bin/julia

function BubbleSort(arr)
    l = length(arr)
    swapped = true
    while swapped
        swapped = false
        for j = 2:l
            if arr[j-1] > arr[j]
                tmp = arr[j-1]
                arr[j-1] = arr[j]
                arr[j] = tmp
                swapped = true
            end
        end
    end
end

function error()
    println("Usage: please provide a list of at least two integers to sort in the format \"1, 2, 3, 4, 5\"")
end

function HandleInput(inp)

    a = split(inp,",")
    a = map(x->split(x," "),a)
    a = map(x->last(x),a)
    numbers = map(x->parse(Int,x),a)
    if length(numbers) == 1
        error()
        exit()
    end
    return numbers

end

function PrintOutput(out)
    for i = 1:length(out)
        print(out[i])
        if i != length(out)
            print(", ")
        end
    end
    println()
end

try

    n = HandleInput(ARGS[1])
    BubbleSort(n)
    PrintOutput(n)

catch e
    error()
end


### Day 3 - Problem 1
## Squares With Three Sides
## Author: Thanasis Georgiou

# Read triangle data
input = readdlm("triangles.csv")

# For each row
count = 0
for i in 1:size(input, 1)
    # Get a row and sort it
    triangle = sort(view(input, i, :))

    # Is this row a valid triangle?
    count += triangle[1] + triangle[2] > triangle[3]
end

# Print result
println("Possible Triangles: $(count)")


# ndgrid-unitr.jl

using LazyGrids: ndgrid, ndgrid_array, GridUR
using Test: @test, @testset, @test_throws, @inferred

@testset "unitr" begin
    (L, M, N) = (5, 6, 7)
    (xa, ya, za) = @inferred ndgrid_array(1:L, 1:M, 1:N)
    (xl, yl, zl) = @inferred ndgrid(1:L, 1:M, 1:N)

    @test eltype(xl) === Int
    @test size(xl) === (L, M, N)
    @test xl isa GridUR
    @test yl isa GridUR
    @test zl isa GridUR
    @test xl == xa
    @test yl == ya
    @test zl == za
    @test xl[2,3,4] == xa[2,3,4] == 2
    @test yl[2,3,4] == ya[2,3,4] == 3
    @test zl[2,3,4] == za[2,3,4] == 4

    x, y, z = (-1:2, -3:4, 5:9)
    (xa, ya, za) = @inferred ndgrid_array(x, y, z)
    (xl, yl, zl) = @inferred ndgrid(x, y, z)

    @test size(xl) === (length(x), length(y), length(z))
    @test xl isa GridUR
    @test yl isa GridUR
    @test zl isa GridUR
    @test xl == xa
    @test yl == ya
    @test zl == za
    @test xl[2,3,4] == xa[2,3,4] == x[2]
    @test yl[2,3,4] == ya[2,3,4] == y[3]
    @test zl[2,3,4] == za[2,3,4] == z[4]
end


@testset "transform_src_insert_dest_map.jl" begin

    @testset "test 1" begin

        f(X,θ̂) = θ̂[1] .+ θ̂[2]*X

        f_map = Map_From_VectFunc(2,f)
        hat_θ_map = Float64[0,-1] # change sign

        # insert -θ4, at position 2
        # insert -θ1, at position 3
        src  = [4,1]
        dest = [2,3]

        # delete insert dest [1,2,3,4,5] -> [1,4,5]
        hat_θ_model = Float64[1:5;]
        deleteat!(hat_θ_model,dest)

        map_s_d = NLS_Fit.Transform_Src_Insert_Dest_Map(f_map,src=>dest)

        θ = eval_map(map_s_d,hat_θ_model,hat_θ_map)

        @test parameter_size( map_s_d) == 2
        @test θ ≈ Float64[1,-4,-1,4,5]
    end

    @testset "test 2" begin
        # same as test 1 but with permuted dest

        f(X,θ̂) = θ̂[1] .+ θ̂[2]*X

        f_map = Map_From_VectFunc(2,f)
        hat_θ_map = Float64[0,-1] # change sign

        src  = [4,1]
        dest = [3,2]

        hat_θ_model = Float64[1:5;]
        deleteat!(hat_θ_model,sort(dest))

        map_s_d = NLS_Fit.Transform_Src_Insert_Dest_Map(f_map,src=>dest)

        θ = eval_map(map_s_d,hat_θ_model,hat_θ_map)

        @test parameter_size( map_s_d) == 2
        @test θ ≈ Float64[1,-1,-4,4,5]
    end
end


using LightGraphs

function sparse_beta_loglik(g::Graph, mu, beta)
    tot = 0
    for i in 1:n
        tot -= degree(g)[i] * beta[i]
        for j in i:n
            tot += log(1 + exp(mu + beta[i] + beta[j]))
        end
    end
    return -(g.ne * mu) + tot
end

function sparse_beta_conditional_loglik(g::Graph, mu, beta, i,)
    beta_i = beta[i]
    tot = degree(g)[i] * beta_i
    for b in beta[1:end .!= i]
        tot -= log1pexp(mu + beta_i + b)
    end
    return tot
end

function sparse_beta_conditional_loglik(g::Graph, mu, beta_i, beta_mi, deg)
    tot = deg * beta_i
    for b in beta[1:end .!= i]
        tot -= log1pexp(mu + beta_i + b)
    end
    return tot
end


struct FourierTridiagonalPoissonSolver{A, G, B, S, β, T}
                  architecture :: A
                          grid :: G
    batched_tridiagonal_solver :: B
                       storage :: S
                        buffer :: β
                    transforms :: T
end

@kernel function compute_diagonals!(D, grid, λx, λy)
    i, j = @index(Global, NTuple)
    Nz = grid.Nz

    D[i, j, 1] = -1/ΔzF(i, j, 1, grid) - ΔzC(i, j, 1, grid) * (λx[i] + λy[j])

    @unroll for k in 2:Nz-1
        D[i, j, k] = - (1/ΔzF(i, j, k-1, grid) + 1/ΔzF(i, j, k, grid)) - ΔzC(i, j, k, grid) * (λx[i] + λy[j])
    end

    D[i, j, Nz] = -1/ΔzF(i, j, Nz-1, grid) - ΔzC(i, j, Nz, grid) * (λx[i] + λy[j])
end

function FourierTridiagonalPoissonSolver(arch, grid, planner_flag=FFTW.PATIENT)
    TX, TY, TZ = topology(grid)
    TZ != Bounded && error("FourierTridiagonalPoissonSolver can only be used with a Bounded z topology.")

    Nx, Ny, Nz = size(grid)

    # Compute discrete Poisson eigenvalues
    λx = poisson_eigenvalues(grid.Nx, grid.Lx, 1, TX())
    λy = poisson_eigenvalues(grid.Ny, grid.Ly, 2, TY())

    λx = arch_array(arch, λx)
    λy = arch_array(arch, λy)

    # Plan required transforms for x and y
    sol_storage = arch_array(arch, zeros(complex(eltype(grid)), size(grid)...))
    transforms = plan_transforms(arch, grid, sol_storage, planner_flag)

    # Lower and upper diagonals are the same
    CUDA.@allowscalar begin
        ld = arch_array(arch, [1/ΔzF(1, 1, k, grid) for k in 1:Nz-1])
        ud = ld
    end

    # Compute diagonal coefficients for each grid point
    D = arch_array(arch, zeros(Nx, Ny, Nz))
    event = launch!(arch, grid, :xy, compute_diagonals!, D, grid, λx, λy,
                    dependencies=Event(device(arch)))
    wait(device(arch), event)

    # Set up batched tridiagonal solver
    rhs_storage = arch_array(arch, zeros(complex(eltype(grid)), size(grid)...))
    btsolver = BatchedTridiagonalSolver(arch, dl=ld, d=D, du=ud, f=rhs_storage, grid=grid)

    # Need buffer for index permutations and transposes.
    buffer_needed = arch isa GPU && Bounded in (TX, TY) ? true : false
    buffer = buffer_needed ? similar(sol_storage) : nothing

    return FourierTridiagonalPoissonSolver(arch, grid, btsolver, sol_storage, buffer, transforms)
end

function solve_poisson_equation!(solver::FourierTridiagonalPoissonSolver)
    ϕ = solver.storage
    RHS = solver.batched_tridiagonal_solver.f

    # Apply forward transforms in order
    [transform!(RHS, solver.buffer) for transform! in solver.transforms.forward]

    # Solve tridiagonal system of linear equations in z at every column.
    solve_batched_tridiagonal_system!(ϕ, solver.architecture, solver.batched_tridiagonal_solver)

    # Apply backward transforms in order
    [transform!(ϕ, solver.buffer) for transform! in solver.transforms.backward]

    ϕ .= real.(ϕ)

    # Set the volume mean of the solution to be zero.
    # Solutions to Poisson's equation are only unique up to a constant (the global mean
    # of the solution), so we need to pick a constant. We choose the constant to be zero
    # so that the solution has zero-mean.
    ϕ .= ϕ .- mean(ϕ)

    return nothing
end


meancov(u) = u[1], u[2]
function correct(u::T, v, H; droptol=1e-7) where T
    x, Ppred = meancov(u)
    y, R = meancov(v)
    yres = y - H*x # innovation residual

    S = (H*Ppred*H' + R) # innovation covariance

    # not sure what is the right thing.
    if false # Dense matrix algebra
        K1 = Ppred*H'*inv(Matrix(S)) # Kalman gain
        @show sparsity(K1)
        K = droptol!(sparse(K1), droptol)
        @show sparsity(K)
    elseif true # Sparse cholesky and dense H*Ppred
        K = (cholesky(Hermitian(S))\(H*Ppred))' # Kalman gain # sparse backsolve is missing
        @show sparsity(K')
        droptol!(K', droptol)
        @show sparsity(K')
    else # Krylov method
        #K = Ppred*H'*inv(S)
        Pre = preconditioner(S, 16, 0.02)+0.01I
        K = srsolve(Pre*S, Pre*Ppred*H', tol = 5e-5, droptol=droptol)
    end
    #sparse(SuiteSparse.CHOLMOD.spsolve(0, cholesky(outer(sprand(10,10,0.5))), SuiteSparse.CHOLMOD.Sparse(sprand(10,10,0.1))))
    x = x + K*yres
    P = (I - K*H)*Ppred*(I - K*H)' + K*R*K'
    T((x, P)), yres, S
end


@testset "@map macro(ex)" begin
    n = 3
    A = rand(2, 2);
    b = rand(2)
    identitymap = @map x -> I(n)*x
    @test identitymap == MathematicalSystems.IdentityMap(n)
    linearmap = @map x -> A*x
    @test linearmap == MathematicalSystems.LinearMap(A)
    affinemap = @map x -> A*x + b
    @test affinemap == MathematicalSystems.AffineMap(A, b)
end

@testset "@map macro(ex, args)" begin
    n = 3
    identitymap1 = @map(x->x, dim=n)
    identitymap2 = @map(x->x, dim:3)
    @test identitymap1 == identitymap2 == IdentityMap(n)
    X = BallInf(zeros(3), 1.0)
    identitymap3 = @map(x->x, dim:3, x ∈ X)
    @test identitymap3 == ConstrainedIdentityMap(3, X)
end


@testset "Check field membership" begin
    R, (x, y, z) = Nemo.PolynomialRing(Nemo.QQ, ["x", "y", "z"])

    @test check_field_membership(
        [[R(1), x + y], [R(1), x * y], [z, (x + y)^2]],
        [(x^2 + y^2) // R(1), (x^3 + y^3) // (z - x * y), R(1) // (z + x + y), z // x],
        0.99
    ) == [true, true, true, false]

    @test check_field_membership(
        [[x + y + z, x^2 + y^2 + z^2, (x + y + z)^2, x^3 + y^3 + z^3]],
        [x + y + z, x * y * z, x + y + 2 * z, x // (y + z)],
        0.99
    ) == [true, true, false, false]
end


struct CurlSingleLayerDP3D{T,U} <: MaxwellOperator3D
    gamma::T
    alpha::U
end

function integrand(op::CurlSingleLayerDP3D, kernel_vals,
    test_vals, test_nbhd, trial_vals, trial_nbhd)

    α = op.alpha

    gx = test_vals.value
    fy = trial_vals.value

    ∇G = kernel_vals.gradgreen
    nx = normal(test_nbhd)

    return -α * dot(nx × gx, ∇G * fy)
end

function momintegrals!(op::CurlSingleLayerDP3D,
    test_local_space::RTRefSpace, trial_local_space::LagrangeRefSpace,
    test_triangular_element, trial_triangular_element, out, strat::SauterSchwabStrategy)

    I, J, K, L = SauterSchwabQuadrature.reorder(
        test_triangular_element.vertices,
        trial_triangular_element.vertices, strat)

    test_triangular_element  = simplex(test_triangular_element.vertices[I]...)
    trial_triangular_element = simplex(trial_triangular_element.vertices[J]...)

    function igd(u,v)
        α = op.alpha
        γ = op.gamma

        x = neighborhood(test_triangular_element,u)
        y = neighborhood(trial_triangular_element,v)

        nx = normal(x)

        r = cartesian(x) - cartesian(y)
        R = norm(r)
        G = exp(-γ*R)/(4π*R)
        GG = -(γ + 1/R) * G / R * r

        f = test_local_space(x)
        g = trial_local_space(y)

        jx = jacobian(x)
        jy = jacobian(y)
        j = jx*jy

        αjGG = α*j*GG

        R = @SVector[αjGG*g[i].value for i in 1:3]
        return @SMatrix[dot(f[i].value × nx, R[j]) for i in 1:3, j in 1:3]
    end

    Q = sauterschwab_parameterized(igd, strat)
    for j ∈ 1:3
        for i ∈ 1:3
            out[i,j] += Q[K[i],L[j]]
        end
    end

    nothing
end

function quadrule(op::BEAST.CurlSingleLayerDP3D,
    test_local_space::BEAST.RTRefSpace, trial_local_space::BEAST.RTRefSpace,
    test_index, test_chart, trial_index, trial_chart, quadrature_data)

    qrdf(op, test_local_space, trial_local_space, test_index, test_chart, trial_index, trial_chart, quadrature_data)
end


# Test that the maximum sphere radius is correctly calculated

@testset "Maximum sphere radius" begin
@testset "fcc" begin
    @test MkCell.score_sphere(fcc1*cellfcc)   ≈ 0.7905694150420949
    @test MkCell.score_sphere(fcc4*cellfcc)   ≈ 0.8660254037844386
    @test MkCell.score_sphere(fcc7*cellfcc)   ≈ 1.118033988749895
    @test MkCell.score_sphere(fcc13*cellfcc)  ≈ 1.50
    @test MkCell.score_sphere(fcc23*cellfcc)  ≈ 1.7677669529663689
    @test MkCell.score_sphere(fcc26*cellfcc)  ≈ 1.7320508075688772
    @test MkCell.score_sphere(fcc32*cellfcc)  ≈ 1.7320508075688772
    @test MkCell.score_sphere(fcc38*cellfcc)  ≈ 1.8708286933869707
    @test MkCell.score_sphere(fcc50*cellfcc)  ≈ 2.1213203435596424
    @test MkCell.score_sphere(fcc70*cellfcc)  ≈ 2.4238399287081647
    @test MkCell.score_sphere(fcc92*cellfcc)  ≈ 2.5495097567963922
    @test MkCell.score_sphere(fcc96*cellfcc)  ≈ 2.500
    @test MkCell.score_sphere(fcc100*cellfcc) ≈ 2.692582403567252
    @test MkCell.score_sphere(fcc108*cellfcc) ≈ 2.598076211353316
end;
@testset "cubic" begin
    @test MkCell.score_sphere(cubic1*cellcubic)   ≈ 0.8660254037844386
    @test MkCell.score_sphere(cubic3*cellcubic)   ≈ 1.50
    @test MkCell.score_sphere(cubic7*cellcubic)   ≈ 2.0615528128088303
    @test MkCell.score_sphere(cubic8*cellcubic)   ≈ 1.7320508075688772
    @test MkCell.score_sphere(cubic13*cellcubic)  ≈ 2.29128784747792
    @test MkCell.score_sphere(cubic14*cellcubic)  ≈ 2.23606797749979
    @test MkCell.score_sphere(cubic24*cellcubic)  ≈ 2.50
    @test MkCell.score_sphere(cubic27*cellcubic)  ≈ 2.598076211353316
    @test MkCell.score_sphere(cubic33*cellcubic)  ≈ 2.8722813232690143
    @test MkCell.score_sphere(cubic38*cellcubic)  ≈ 3.2015621187164243
    @test MkCell.score_sphere(cubic52*cellcubic)  ≈ 3.24037034920393
    @test MkCell.score_sphere(cubic56*cellcubic)  ≈ 3.500
    @test MkCell.score_sphere(cubic64*cellcubic)  ≈ 3.4641016151377544
    @test MkCell.score_sphere(cubic73*cellcubic)  ≈ 3.905124837953327
    @test MkCell.score_sphere(cubic76*cellcubic)  ≈ 3.7416573867739413
    @test MkCell.score_sphere(cubic93*cellcubic)  ≈ 4.06201920231798
    @test MkCell.score_sphere(cubic100*cellcubic) ≈ 4.031128874149275
end;
@testset "bcc" begin
    @test MkCell.score_sphere(bcc1*cellbcc)   ≈ 0.82915619758885
    @test MkCell.score_sphere(bcc2*cellbcc)   ≈ 0.8660254037844386
    @test MkCell.score_sphere(bcc6*cellbcc)   ≈ 1.479019945774904
    @test MkCell.score_sphere(bcc9*cellbcc)   ≈ 1.479019945774904
    @test MkCell.score_sphere(bcc16*cellbcc)  ≈ 1.7320508075688772
    @test MkCell.score_sphere(bcc21*cellbcc)  ≈ 2.0615528128088303
    @test MkCell.score_sphere(bcc25*cellbcc)  ≈ 2.277608394786075
    @test MkCell.score_sphere(bcc28*cellbcc)  ≈ 2.23606797749979
    @test MkCell.score_sphere(bcc35*cellbcc)  ≈ 2.277608394786075
    @test MkCell.score_sphere(bcc48*cellbcc)  ≈ 2.50
    @test MkCell.score_sphere(bcc53*cellbcc)  ≈ 2.8613807855648994
    @test MkCell.score_sphere(bcc54*cellbcc)  ≈ 2.598076211353316
    @test MkCell.score_sphere(bcc65*cellbcc)  ≈ 3.031088913245535
    @test MkCell.score_sphere(bcc84*cellbcc)  ≈ 3.112474899497183
    @test MkCell.score_sphere(bcc91*cellbcc)  ≈ 3.112474899497183
    @test MkCell.score_sphere(bcc103*cellbcc) ≈ 3.344772040064913
    @test MkCell.score_sphere(bcc107*cellbcc) ≈ 3.418698582794336
    @test MkCell.score_sphere(bcc112*cellbcc) ≈ 3.418698582794336
    @test MkCell.score_sphere(bcc128*cellbcc) ≈ 3.4641016151377544
end;
@testset "hcp" begin
    @test MkCell.score_sphere(hcp1*cellhcp)   ≈ 0.6679616056830413
    @test MkCell.score_sphere(hcp3*cellhcp)   ≈ 0.9584314616427545
    @test MkCell.score_sphere(hcp6*cellhcp)   ≈ 1.074616185745714
    @test MkCell.score_sphere(hcp8*cellhcp)   ≈ 1.1224
    @test MkCell.score_sphere(hcp11*cellhcp)  ≈ 1.3650740590409982
    @test MkCell.score_sphere(hcp14*cellhcp)  ≈ 1.4759311049864985
    @test MkCell.score_sphere(hcp16*cellhcp)  ≈ 1.4490121692610682
    @test MkCell.score_sphere(hcp21*cellhcp)  ≈ 1.5538924372898746
    @test MkCell.score_sphere(hcp27*cellhcp)  ≈ 1.7448417081978145
    @test MkCell.score_sphere(hcp37*cellhcp)  ≈ 1.8962138873731165
    @test MkCell.score_sphere(hcp39*cellhcp)  ≈ 1.9775167626765309
    @test MkCell.score_sphere(hcp42*cellhcp)  ≈ 2.0169395793297
    @test MkCell.score_sphere(hcp53*cellhcp)  ≈ 2.167472294325043
    @test MkCell.score_sphere(hcp62*cellhcp)  ≈ 2.2968099877874093
    @test MkCell.score_sphere(hcp65*cellhcp)  ≈ 2.3420716783793503
    @test MkCell.score_sphere(hcp69*cellhcp)  ≈ 2.2968099877874093
    @test MkCell.score_sphere(hcp76*cellhcp)  ≈ 2.3974474509360992
    @test MkCell.score_sphere(hcp80*cellhcp)  ≈ 2.3974474509360992
    @test MkCell.score_sphere(hcp88*cellhcp)  ≈ 2.440843687470926
    @test MkCell.score_sphere(hcp90*cellhcp)  ≈ 2.446214087115026
    @test MkCell.score_sphere(hcp100*cellhcp) ≈ 2.5666337461092237
end;
end;


module Bickley

# The Bickley jet
Ψ(y, L) = - tanh(y) + 2y / L
U(y, L) = sech(y)^2 + 2 / L

# A sinusoidal tracer
C(y, L) = sin(2π * y / L)

# Slightly off-center vortical perturbations
ψ̃(x, y, ℓ, k) = exp(-(y + ℓ/10)^2 / 2ℓ^2) * cos(k * x) * cos(k * y)

# Vortical velocity fields (ũ, ṽ) = (-∂_y, +∂_x) ψ̃
ũ(x, y, ℓ, k) = + ψ̃(x, y, ℓ, k) * (k * tan(k * y) + y / ℓ^2)
ṽ(x, y, ℓ, k) = - ψ̃(x, y, ℓ, k) * k * tan(k * x)

end # module


# Gradient algorithm
function est_gradient!(dθ, θ, g, t; regressor=[0.0], output=0.0)
    w, y = regressor, output
    e = w'*θ - output
    dθ .= -g*w*e
    return θ
end

gradient_estimator(g) = (dθ, θ, p, t; kwargs...) -> est_gradient!(dθ, θ, g, t; kwargs...)


########################################################################################
#
# jFlow: a 3-D saturated-unsaturated groundwater flow model
#
# (1) Methodology is the integral finite difference method (IFDM)
# (2) Pressure head in a cell is defined at z of node point (i.e., if P = 0, S is typically = 0.5)
# (3) Unsaturated properties defined with Van Genuchten model, or as simple unconfined flow
#
########################################################################################

using SparseArrays

### data types

mutable struct Material
    index::Int64
    Kh::Float64                         # saturated hydraulic conductivity components
    Kz::Float64
    Ss::Float64                         # specific storage
    phi::Float64                        # porosity
    alpha::Float64                      # Van Genuchten model parameters
    N::Float64
    m::Float64
end

mutable struct Cell
    index::Int64                        # index tracking number for cell (for initial struct array setup)
    x::Float64                          # grid cell center coordinates
    y::Float64
    z::Float64                          # node point elevation
    vol::Float64                        # cell volume
    b::Float64                          # cell vertical thickness (==0 for VG unsat'd properties; >0 for Sy formulation)
    P::Float64                          # pressure head
    S::Float64                          # saturation
    mat::Material                       # corresponding material
    connects::Array{Int64, 1}           # list of associated connection indices
    neighbors::Array{Int64, 1}          # list of connected cells
end

mutable struct Connection
    cell1::Int64                        # connected cells (index numbers)
    cell2::Int64
    del1::Float64                       # respective distances from cell centerpoints to interface
    del2::Float64
    A::Float64                          # interfacial area of the connection
    zConnect::Bool                      # connection is designated as vertically-oriented
	KMean::Float64 						# mean saturated hydraulic conductivity across connection
end

mutable struct Params
    endTime::Float64                    # end of simulation time
    dt0::Float64                        # initial (and minimum) time step
    dtMax::Float64                      # maximum time step
    dhMax::Float64                      # maximum head change in any model cell, per time step
    f::Float64                          # time step multiplier
    c::Float64                          # Crank-Nicolson implicit solution weighting factor
	mesh::Bool 							# read externally generated mesh (true) or generate from input file (false)
end

include("jFlowIOModule.jl")             # various input and output functions included in separate module

### hydraulic parameterization functions (Van Genuchten or linear models for unsaturated conditions)

function kr(cell::Cell)::Float64
    # relative permeability as a function of water saturation
    if cell.b == 0
        kRel = cell.S^0.5 * (1.0 - (1.0 - cell.S^(1.0/cell.mat.m))^cell.mat.m)^2    # Van Genuchten formulation
    else
        kRel = cell.S           # for an unconfined aquifer condition
    end
    return kRel
end

function Sat(cell::Cell)::Float64
    # (effective) saturation as a function of pressure head
    if cell.b == 0.0
        # Van Genuchten formulation
        if cell.P < 0
            S = (1. + abs(cell.mat.alpha * cell.P)^cell.mat.N)^(-cell.mat.m)
        else
            S = 1.0
        end
    else
        # unconfined aquifer formulation
        if cell.P < cell.b/2
            S = maximum([cell.P/cell.b + 0.5, 0.0])
        else
            S = 1.0
        end
    end
    return S
end

function dSdP(cell::Cell)::Float64
    # storage term a function of change in effective saturation per change in capillary pressure head
    if cell.b == 0
        dS = -(cell.mat.alpha*abs(cell.P))^cell.mat.N * ((cell.mat.alpha*abs(cell.P))^cell.mat.N + 1.0)^(-cell.mat.m-1.0) * cell.mat.m * cell.mat.N/cell.P
    else
        dS = 1.0/cell.b
    end
    return dS * cell.mat.phi
end

function Storage(cell::Cell)::Float64
    # return appropriate storage term, depending on saturation state of cell
    if Sat(cell) < 1.0
        stor = dSdP(cell) * cell.vol
    else
        stor = cell.mat.Ss * cell.vol
    end
    return stor
end

### utility functions

function SortCell(tempCell::Array{Cell, 1})::Array{Cell, 1}
    # insertion sort approach to create ordered array of cell structs
    cell = [tempCell[1]]
    seq = [tempCell[1].index]
    for i = 2:length(tempCell)
        index = length(findall(seq.<tempCell[i].index)) + 1
        insert!(seq, index, tempCell[i].index)
        insert!(cell, index, tempCell[i])
    end
    println("Sorted cell struct array.")
    return cell
end

function MatchConnects(cell::Array{Cell, 1}, connect::Array{Connection, 1})::Array{Cell, 1}
    # populate cell connection lists
    for (i, cn) in enumerate(connect)
        push!(cell[cn.cell1].connects, i)
        push!(cell[cn.cell2].connects, i)
        push!(cell[cn.cell1].neighbors, cn.cell2)
        push!(cell[cn.cell2].neighbors, cn.cell1)
    end
    return cell
end

function FindStressPeriod(t::Float64, stressPeriods::Array{Float64, 1})::Int64
    # index number of stress period to which "t" belongs
    return sum((t.>=stressPeriods)*1)
end

function BlendedK(connect::Array{Connection, 1}, cell::Array{Cell, 1})::Array{Connection, 1}
    # weighted harmonic mean for saturated hydraulic conductivity heterogeneous cell interfaces
	for cn in connect
		if cn.zConnect == false
			cn.KMean = (cn.del1 + cn.del2)/(cn.del1/cell[cn.cell1].mat.Kh + cn.del2/cell[cn.cell2].mat.Kh)
		else
			cn.KMean = (cn.del1 + cn.del2)/(cn.del1/cell[cn.cell1].mat.Kz + cn.del2/cell[cn.cell2].mat.Kz)
		end
	end
    return connect
end

function EffectiveK(cn::Connection, cell::Array{Cell, 1})::Float64
    # combo weighted harmonic mean & arithmetic means for unsaturated cell interfaces
    krMean = (cn.del1*kr(cell[cn.cell1]) + cn.del2*kr(cell[cn.cell2]))/(cn.del1 + cn.del2)
    return krMean * cn.KMean
end

function TimeStep(t::Float64, dt::Float64, params::Params, tStops::Array{Float64, 1})
    # revise time step
    finFlag = false
    dtNew = minimum([dt*params.f, params.dtMax])
    for tStop in tStops
        if (t < tStop) && (t+dtNew > tStop)
            finFlag = (t+dtNew > tStops[end])       # indicates end-of-simulation-time has been reached
            dtNew = tStop - t
            break
        end
    end
    return dtNew, finFlag
end

function Vel(cell::Array{Cell, 1}, connect::Array{Connection, 1})
    # compute pore velocity vector arrays at cell node points
    vx = zeros(Float64, length(cell))
    vy = zeros(Float64, length(cell))
    vz = zeros(Float64, length(cell))
    xCompCell = zeros(Float64, length(cell))    # connection component projections along respective axes, per cell
    yCompCell = zeros(Float64, length(cell))
    zCompCell = zeros(Float64, length(cell))
    for cn in connect
        conduct = EffectiveK(cn, cell)        # effective hydraulic conductivity
        d = cn.del1+cn.del2
        xComp = (cell[cn.cell2].x - cell[cn.cell1].x)/d^2
        yComp = (cell[cn.cell2].y - cell[cn.cell1].y)/d^2
        zComp = (cell[cn.cell2].z - cell[cn.cell1].z)/d^2
        q = conduct * ((cell[cn.cell1].P+cell[cn.cell1].z)-(cell[cn.cell2].P+cell[cn.cell2].z)) / d
        vx[cn.cell1] += xComp * q/(cell[cn.cell1].mat.phi*cell[cn.cell1].S)
        vx[cn.cell2] += xComp * q/(cell[cn.cell2].mat.phi*cell[cn.cell2].S)
        vy[cn.cell1] += yComp * q/(cell[cn.cell1].mat.phi*cell[cn.cell1].S)
        vy[cn.cell2] += yComp * q/(cell[cn.cell2].mat.phi*cell[cn.cell2].S)
        vz[cn.cell1] += zComp * q/(cell[cn.cell1].mat.phi*cell[cn.cell1].S)
        vz[cn.cell2] += zComp * q/(cell[cn.cell2].mat.phi*cell[cn.cell2].S)
        xCompCell[cn.cell1] += abs(xComp)
        xCompCell[cn.cell2] += abs(xComp)
        yCompCell[cn.cell1] += abs(yComp)
        yCompCell[cn.cell2] += abs(yComp)
        zCompCell[cn.cell1] += abs(zComp)
        zCompCell[cn.cell2] += abs(zComp)
    end
    # normalize velocity components at each cell node by number of connections
    for i = 1:length(cell)
        vx[i] /= (xCompCell[i] + 1e-10)                 # 1e-10 factor added to prevent divide-by-zero errors
        vy[i] /= (yCompCell[i] + 1e-10)
        vz[i] /= (zCompCell[i] + 1e-10)
    end
    println("Computed pore velocity vectors.")
    return vx, vy, vz
end

### matrix operations functions

function AssembleMatrix(cell::Array{Cell, 1}, connect::Array{Connection, 1}, params::Params, dt::Float64, t::Float64, stressPeriods::Array{Float64, 1}, sourceCells::Array{Int64, 1}, source::Array{Float64, 2})
    # fill out the LHS of the equation matrix
    row_index = Int64[]                     # indexing system for sparse matrix
    col_index = Int64[]
    data = Float64[]
    b = Float64[]
    for (i, ce) in enumerate(cell)          # for each row
        diag = Storage(ce)/dt
        bSum = 0.
        for (j, nghbr) in enumerate(ce.neighbors)           # for each neighboring cell
            conduct = EffectiveK(connect[ce.connects[j]], cell) *
                connect[ce.connects[j]].A / (connect[ce.connects[j]].del1 + connect[ce.connects[j]].del2)
            push!(row_index, i)                                             # left-hand-side matrix
            push!(col_index, nghbr)
            push!(data, -params.c * conduct)
            diag += params.c * conduct
            bSum += conduct * ((cell[nghbr].P + cell[nghbr].z) - (ce.P + ce.z))         # right-hand-side (explicit) vector
        end
        push!(row_index, i)         # set diagonal term
        push!(col_index, i)
        push!(data, diag)
        push!(b, bSum)              # set explicit vector
    end
    # add source/sink terms to b[], as warranted
    j = FindStressPeriod(t, stressPeriods)
    for (i, cellIndex) in enumerate(sourceCells)
        b[cellIndex] += source[i, j]
    end
    return data, row_index, col_index, b
end

### main script

function jFlow(mode::Int64)

    # read and process input files
    params = ReadParams()                               # read model properties
    material = ReadMaterials()                          # read materials properties
    if params.mesh == true
        cell = ReadCells1(material)                     # read cell network from external grid generator
        connect = ReadGridConnects1()                   # read grid connections from externally-generated file
    else
        tempCell = ReadCells2(material)                 # read and assign cells (possibly out of sequence) from raw constraints
        cell = SortCell(tempCell)                       # use insertion sort approach to sequence cell struct array
        #cell = ReadCellMods(cell, material)            # modify specific cell properties, as specified
        connect = ReadGridConnects2()                        # read internal grid connections
        #connect = ReadSpecConnects(connect)                # append unique or boundary connections
    end
    cell = MatchConnects(cell, connect)                 # assign cell connection lists
	connect = BlendedK(connect, cell) 					# pre-calculated connection saturated K's
    stressPeriods, sourceCells, source = ReadSources()  # tabulate stress periods and source/sink cells
	drainCell, drainP = ReadDrains() 					# read cells designated as drains
    monitor = ReadMonitors()                            # note monitor locations (results tabulated for every time step)

    # create container arrays for monitoring points
    timeMon = Float64[]
    timeSeries = []
    for m in monitor
        push!(timeSeries, Float64[])
    end

    # summarize model setup
    WriteCells(cell, "CellSummary.csv", 1, drainCell, drainP)
    WriteConnections(connect)
    WriteSources(stressPeriods, sourceCells, source)

    if mode != 0            # if mode == 0, do not run model (just print out model setup and exit)

        # initialize model
        t = 0.0
        dt = params.dt0                     # first time step size
        tStops = copy(stressPeriods)        # times for which dt must be adjusted for "t" to match ...
        push!(tStops, params.endTime)
        finFlag = false
        convgFlag = false
        dP = zeros(Float64, length(cell))

        while finFlag==false

            while convgFlag == false

                # assemble matrix
                data, row_index, col_index, b = AssembleMatrix(cell, connect, params, dt, t, stressPeriods, sourceCells, source)
                A = sparse(row_index, col_index, data, length(cell), length(cell))

                # solve system of equations
                dP = \(A, b)

                # adjust cells with drains
				for (i, indx) in enumerate(drainCell)
					dP[indx] = minimum([dP[indx], drainP[i]-cell[indx].P])
				end

                # check convergence (maximum head change); adjust dt if needed
                if maximum(abs.(dP)) >= params.dhMax
                    dt /= params.f
                    @assert(dt > params.dt0)             # stop execution if dt falls below minimum
                else
                    convgFlag = true
                end

            end

            # update results
            t += dt
            for (i, ce) in enumerate(cell)
                ce.P += dP[i]
                ce.S = Sat(ce)
            end
            println("\t", "t = ", string(t))

            # write to monitor file(s)
            push!(timeMon, t)
            for (i, monCell) in enumerate(monitor)
                push!(timeSeries[i], cell[monCell].P)
            end

            # determine next time step
            dt, finFlag = TimeStep(t, dt, params, tStops)

            convgFlag = false           # reset head-change flag for next time step

        end

    end

    # write output
    WriteCells(cell, "FinalState.csv", 0, drainCell, drainP)	# write cell output file
    vx, vy, vz = Vel(cell, connect)                 			# compute and write velocity vectors to file
    WriteVectors(cell, vx, vy, vz)
    WriteTimeSeries(monitor, timeMon, timeSeries)   			# write monitor point time series to file

    println("Finished.")

end

### run script
jFlow(1)


@with_kw struct sparameters{R}
  #thermal constants
  b1::R = 0.243;
  b2::R = 0.393;
  b3::R = 1.534;
  #retention - VG model
  alpha::R = 5;
  n::R = 2;
  m::R = 1-1/n;
  theta_s::R = 0.4;
  theta_r::R = 0.01;
  Ks::R = 2/(24*60*60);
  #matric head top constrain
  h_max::R = -1e20;
  #additional parameters
  G_wT::R = 7; # gain factor [-] taken from Jiangbo et al. (2017)
  fc::R = 0.02; #taken from Jiangbo et al. (2017)
end

global soil_parameters=sparameters{Float64}()


module GaussianDistributions

# Gaussian
using LinearAlgebra, Random, Statistics
using Distributions
using LinearAlgebra: norm_sqr

import Random: rand, GLOBAL_RNG
import Statistics: mean, cov, var, std
import Distributions: pdf, logpdf, sqmahal, cdf, quantile
import LinearAlgebra: cholesky
import Base: size, iterate, length

export PSD, Gaussian

"""
    PSD{T}

Simple wrapper for the lower triangular Cholesky root of a positive (semi-)definite element `σ`.
"""
struct PSD{T}
    σ::T
    PSD(σ::T) where {T} = istril(σ) ? new{T}(σ) : throw(ArgumentError("Argument not lower triangular"))
end
cholesky(P::PSD) = (U=P.σ',)   # using a named tuple for now. FIXME? Also: TESTME

"""
Sum of the log of the diagonal elements. Second argument `d` is used
to handle `UniformScaling` and other linear operators whose dimensions
are determined by the dimension of the argument they work on.
"""
sumlogdiag(Σ::Float64, _) = log(Σ)
sumlogdiag(Σ, d) = sum(log.(diag(Σ)))
sumlogdiag(J::UniformScaling, d) = log(J.λ)*d

_logdet(Σ::PSD, d) = 2*sumlogdiag(Σ.σ, d)

_logdet(Σ, d) = logdet(Σ)
_logdet(J::UniformScaling, d) = log(J.λ) * d

_symmetric(Σ) = Symmetric(Σ)
_symmetric(J::UniformScaling) = J

"""
    Gaussian(μ, Σ) -> P

Gaussian distribution with mean `μ` and covariance `Σ`. Defines `rand(P)` and `(log-)pdf(P, x)`.
Designed to work with `Number`s, `UniformScaling`s, `StaticArrays` and `PSD`-matrices.

Implementation details: On `Σ` the functions `logdet`, `whiten` and `unwhiten`
(or `chol` as fallback for the latter two) are called.
"""
struct Gaussian{T,S}
    μ::T
    Σ::S
    Gaussian{T,S}(μ, Σ) where {T,S} = new(μ, Σ)
    Gaussian(μ::T, Σ::S) where {T,S} = new{T,S}(μ, Σ)
end

iterate(P::Gaussian) = P.μ, true
iterate(P::Gaussian, s) = s ? (P.Σ, false) : nothing
length(P::Gaussian) = 2

pair(u) = u[1], u[2]
pair(p::Gaussian) = p.μ, p.Σ

Base.:(==)(g1::Gaussian, g2::Gaussian) = g1.μ == g2.μ && g1.Σ == g2.Σ
Base.isapprox(g1::Gaussian, g2::Gaussian; kwargs...) =
    isapprox(g1.μ, g2.μ; kwargs...) && isapprox(g1.Σ, g2.Σ; kwargs...)
Gaussian() = Gaussian(0.0, 1.0)
mean(P::Gaussian) = P.μ
cov(P::Gaussian) = P.Σ
var(P::Gaussian{<:Number}) = P.Σ
std(P::Gaussian{<:Number}) = sqrt(var(P))
Base.convert(::Type{Gaussian{T, S}}, g::Gaussian) where {T, S} =
    Gaussian(convert(T, g.μ), convert(S, g.Σ))

dim(P::Gaussian) = length(P.μ)
whiten(Σ::PSD, z) = Σ.σ\z
whiten(Σ, z) = cholesky(Σ).U'\z
whiten(Σ::Number, z) = sqrt(Σ)\z
whiten(Σ::UniformScaling, z) = sqrt(Σ.λ)\z

unwhiten(Σ::PSD, z) = Σ.σ*z
unwhiten(Σ, z) = cholesky(Σ).U'*z
unwhiten(Σ::Number, z) = sqrt(Σ)*z
unwhiten(Σ::UniformScaling, z) = sqrt(Σ.λ)*z

sqmahal(P::Gaussian, x) = norm_sqr(whiten(P.Σ, x - P.μ))

rand(P::Gaussian) = rand(GLOBAL_RNG, P)
rand(RNG::AbstractRNG, P::Gaussian) = P.μ + unwhiten(P.Σ, randn(RNG, typeof(P.μ)))
rand(RNG::AbstractRNG, P::Gaussian{Vector{T}}) where T =
    P.μ + unwhiten(P.Σ, randn(RNG, T, length(P.μ)))
rand(RNG::AbstractRNG, P::Gaussian{<:Number}) = P.μ + sqrt(P.Σ)*randn(RNG, typeof(one(P.μ)))

logpdf(P::Gaussian, x) = -(sqmahal(P,x) + _logdet(P.Σ, dim(P)) + dim(P)*log(2pi))/2
pdf(P::Gaussian, x) = exp(logpdf(P::Gaussian, x))
cdf(P::Gaussian{Number}, x) = Distributions.normcdf(P.μ, sqrt(P.Σ), x)

Base.:+(g::Gaussian, vec) = Gaussian(g.μ + vec, g.Σ)
Base.:+(vec, g::Gaussian) = g + vec
Base.:-(g::Gaussian, vec) = g + (-vec)
Base.:*(M, g::Gaussian) = Gaussian(M * g.μ, M * g.Σ * M')
⊕(g1::Gaussian, g2::Gaussian) = Gaussian(g1.μ + g2.μ, g1.Σ + g2.Σ)
⊕(vec, g::Gaussian) = vec + g
⊕(g::Gaussian, vec) = g + vec
const independent_sum = ⊕

function rand_scalar(RNG::AbstractRNG, P::Gaussian{T}, dims) where {T}
    X = zeros(T, dims)
    for i in 1:length(X)
        X[i] = rand(RNG, P)
    end
    X
end

function rand_vector(RNG::AbstractRNG, P::Gaussian{Vector{T}}, dims::Union{Integer, NTuple}) where {T}
    X = zeros(T, dim(P), dims...)
    for i in 1:prod(dims)
        X[:, i] = rand(RNG, P)
    end
    X
end
rand(RNG::AbstractRNG, P::Gaussian, dim::Integer) = rand_scalar(RNG, P, dim)
rand(RNG::AbstractRNG, P::Gaussian, dims::Tuple{Vararg{Int64,N}} where N) = rand_scalar(RNG, P, dims)

rand(RNG::AbstractRNG, P::Gaussian{Vector{T}}, dim::Integer) where {T} = rand_vector(RNG, P, dim)
rand(RNG::AbstractRNG, P::Gaussian{Vector{T}}, dims::Tuple{Vararg{Int64,N}} where N) where {T} = rand_vector(RNG, P, dims)
rand(P::Gaussian, dims::Tuple{Vararg{Int64,N}} where N) = rand(GLOBAL_RNG, P, dims)
rand(P::Gaussian, dim::Integer) = rand(GLOBAL_RNG, P, dim)

"""
    logpdfnormal(x, Σ)

Logarithm of the probability density function of centered Gaussian with covariance Σ
"""
function logpdfnormal(x, Σ)

    S = cholesky(_symmetric(Σ)).U'

    d = length(x)
     -((norm(S\x))^2 + 2sumlogdiag(S,d) + d*log(2pi))/2
end
function logpdfnormal(x::Float64, Σ)
     -(x^2/Σ + log(Σ) + log(2pi))/2
end

"""
    conditional(P::Gaussian, A, B, xB)

Conditional distribution of `X[i for i in A]` given
`X[i for i in B] == xB` if ``X ~ P``.
"""
function conditional(P::Gaussian, A, B, xB)
    Z = P.Σ[A,B]*inv(P.Σ[B,B])
    Gaussian(P.μ[A] + Z*(xB - P.μ[B]), P.Σ[A,A] - Z*P.Σ[B,A])
end

"""
    correct(u, v, H) = u, yres, S

Joseph form correction step of a Kalman filter with `u = Gaussian(x, P)` state
and `v = Gaussian(y, R)` the observation with uncertainty `R`.
`H` is the observation operator. Returns corrected/conditional
distribution u, the residual and the innovation covariance.

See https://en.wikipedia.org/wiki/Kalman_filter#Update.
"""
function correct(u::T, v, H) where {T}
    x, Ppred = pair(u)
    y, R = pair(v)
    yres = y - H*x # innovation residual

    S = (H*Ppred*H' + R) # innovation covariance

    K = Ppred*H'/S # Kalman gain
    x = x + K*yres
    P = (I - K*H)*Ppred*(I - K*H)' + K*R*K' #  Joseph form

    T(x, P), yres, S
end

include("bivariate.jl")

end # module


using Test
using QuantumOptics
using Random, LinearAlgebra

@testset "phasespace" begin

Random.seed!(0)

D(op1::AbstractOperator, op2::AbstractOperator) = abs(tracedistance_nh(dense(op1), dense(op2)))

# Test quasi-probability functions
b = FockBasis(100)
alpha = complex(0.3, 0.7)
nfock = 3

X = [-2.1:1.5:2;]
Y = [0.13:1.4:3;]
psi_coherent = coherentstate(b, alpha)
rho_coherent = dm(psi_coherent)
psi_fock = fockstate(b, nfock)
rho_fock = dm(psi_fock)

Qpsi_coherent = qfunc(psi_coherent, X, Y)
Qrho_coherent = qfunc(rho_coherent, X, Y)
Qpsi_fock = qfunc(psi_fock, X, Y)
Qrho_fock = qfunc(rho_fock, X, Y)

Wpsi_coherent = wigner(psi_coherent, X, Y)
Wrho_coherent = wigner(rho_coherent, X, Y)
Wpsi_fock = wigner(psi_fock, X, Y)
Wrho_fock = wigner(rho_fock, X, Y)

laguerre3(x) = (-x^3+9x^2-18x+6)/6

for (i,x)=enumerate(X), (j,y)=enumerate(Y)
    beta = 1.0/sqrt(2)*complex(x, y)
    betastate = coherentstate(b, beta)

    q_coherent = 1/pi*exp(-abs2(alpha-beta))
    @test Qpsi_coherent[i, j] ≈ q_coherent
    @test Qrho_coherent[i, j] ≈ q_coherent
    @test qfunc(psi_coherent, beta) ≈ q_coherent
    @test qfunc(rho_coherent, beta) ≈ q_coherent
    @test qfunc(psi_coherent, x, y) ≈ q_coherent
    @test abs2(dagger(betastate) * psi_coherent)/pi ≈ q_coherent
    @test dagger(betastate) * rho_coherent * betastate/pi ≈ q_coherent

    q_fock = 1/pi*exp(-abs2(beta))*abs2(beta)^nfock/factorial(nfock)
    @test Qpsi_fock[i, j] ≈ q_fock
    @test Qrho_fock[i, j] ≈ q_fock
    @test qfunc(psi_fock, beta) ≈ q_fock
    @test qfunc(rho_fock, beta) ≈ q_fock
    @test abs2(dagger(betastate) * psi_fock)/pi ≈ q_fock
    @test dagger(betastate) * rho_fock * betastate/pi ≈ q_fock

    w_coherent = 1/pi*exp(-2*abs2(alpha-beta))
    @test Wpsi_coherent[i, j] ≈ w_coherent
    @test Wrho_coherent[i, j] ≈ w_coherent
    @test wigner(psi_coherent, beta) ≈ w_coherent
    @test wigner(rho_coherent, beta) ≈ w_coherent

    w_fock = 1/pi*(-1)^nfock*laguerre3(4*abs2(beta))*exp(-2*abs2(beta))
    @test Wpsi_fock[i, j] ≈ w_fock
    @test Wrho_fock[i, j] ≈ w_fock
    @test wigner(psi_fock, beta) ≈ w_fock
    @test wigner(rho_fock, beta) ≈ w_fock
end

# Test with offset
alpha = complex(4.3, 2.7)
nfock = 3

X = [0.1:0.1:4;] .+ real(alpha)
Y = [0.13:0.1:3;] .+ imag(alpha)
psi_coherent = coherentstate(b, alpha)
rho_coherent = dm(psi_coherent)
psi_fock = fockstate(b, nfock)
rho_fock = dm(psi_fock)

Qpsi_coherent = qfunc(psi_coherent, X, Y)
Qrho_coherent = qfunc(rho_coherent, X, Y)
Qpsi_fock = qfunc(psi_fock, X, Y)
Qrho_fock = qfunc(rho_fock, X, Y)

Wpsi_coherent = wigner(psi_coherent, X, Y)
Wrho_coherent = wigner(rho_coherent, X, Y)
Wpsi_fock = wigner(psi_fock, X, Y)
Wrho_fock = wigner(rho_fock, X, Y)

b_off = FockBasis(100,3)
psi_coherent_off = coherentstate(b_off, alpha)
rho_coherent_off = dm(psi_coherent_off)
psi_fock_off = fockstate(b_off, nfock)
rho_fock_off = dm(psi_fock)

@test isapprox(qfunc(psi_coherent, alpha), qfunc(psi_coherent_off, alpha))
@test isapprox(Qpsi_coherent, qfunc(psi_coherent_off, X, Y), atol=1e-6)
@test isapprox(Qrho_coherent, qfunc(rho_coherent_off, X, Y), atol=1e-6)
@test isapprox(qfunc(psi_fock, alpha), qfunc(psi_fock_off, alpha))
@test isapprox(Qpsi_fock, qfunc(psi_fock_off, X, Y), atol=1e-6)
@test isapprox(Qrho_fock, qfunc(rho_fock_off, X, Y), atol=1e-6)

@test isapprox(wigner(psi_coherent, alpha), wigner(psi_coherent_off, alpha), atol=1e-6)
@test isapprox(Wpsi_coherent, wigner(psi_coherent_off, X, Y), atol=1e-5, rtol=1e-5)
@test isapprox(Wrho_coherent, wigner(rho_coherent_off, X, Y), atol=1e-5, rtol=1e-5)
@test isapprox(wigner(psi_fock, alpha), wigner(psi_fock_off, alpha), atol=1e-5, rtol=1e-5)
@test isapprox(Wpsi_fock, wigner(psi_fock_off, X, Y), atol=1e-5, rtol=1e-5)
@test isapprox(Wrho_fock, wigner(rho_fock_off, X, Y), atol=1e-5, rtol=1e-5)

# Test qfunc with rand operators
b = FockBasis(50)
psi = randstate(b)
rho = randoperator(b)
X = [-2.1:1.5:2;]
Y = [-0.5:.8:3;]

Qpsi = qfunc(psi, X, Y)
Qrho = qfunc(rho, X, Y)
for (i,x)=enumerate(X), (j,y)=enumerate(Y)
    c = complex(x, y)/sqrt(2)
    state = coherentstate(b, c)
    q_rho = dagger(state) * rho * state/pi
    q_psi = abs2(dagger(state) *psi)/pi
    @test 1e-14 > abs(Qpsi[i,j] - q_psi)
    @test 1e-14 > abs(Qrho[i,j] - q_rho)
    @test 1e-14 > abs(qfunc(psi, c) - q_psi)
    @test 1e-14 > abs(qfunc(rho, c) - q_rho)
end

# Test SU(2) phasespace
b = SpinBasis(5)
theta = π*rand()
phi =2π*rand()
css = coherentspinstate(b,theta,phi)
dmcss = dm(css)
csssx = coherentspinstate(b,π/2,0)
dmcsssx = dm(csssx)
rs = randstate(b)
dmrs = dm(rs)
sx = expect(sigmax(b)/2,css); # eigenstate of jx operator
sy = expect(sigmay(b)/2,css); # eigenstate of jy
sz = expect(sigmaz(b)/2,css); # eigenstate of jz operator
ssq = sx^2 + sy^2 + sz^2

qsu2sx = qfuncsu2(csssx,theta,phi)
qsu2sxdm = qfuncsu2(dmcsssx,theta,phi)
res = 250
costhetam = Array{Float64}(undef,res,2*res)
for i = 1:res, j = 1:2*res
    costhetam[i,j] = sin(i*1pi/(res-1))
end
wsu2 = sum(wignersu2(rs,res).*costhetam)*(π/res)^2
wsu2dm = sum(wignersu2(dmrs,res).*costhetam)*(π/res)^2
qsu2 = sum(qfuncsu2(rs,res).*costhetam)*(π/res)^2
qsu2dm = sum(qfuncsu2(dmrs,res).*costhetam)*(π/res)^2

@test ssq ≈ float(b.spinnumber)^2

@test isapprox(qsu2sxdm, (2*float(b.spinnumber)+1)/(4pi)*(0.5*(sin(theta)cos(phi)+1))^(2*float(b.spinnumber)),atol=1e-2)
@test isapprox(qsu2sx, (2*float(b.spinnumber)+1)/(4pi)*(0.5*(sin(theta)cos(phi)+1))^(2*float(b.spinnumber)),atol=1e-2)
@test isapprox(qsu2, 1.0, atol=1e-2)
@test isapprox(qsu2dm, 1.0, atol=1e-2)

@test isapprox(wignersu2(csssx,π/2,0), (4*float(b.spinnumber)+1)/(4pi), atol=1e-2)
@test isapprox(wignersu2(dmcsssx,π/2,0), (4*float(b.spinnumber)+1)/(4pi),atol=1e-2)
@test isapprox(wsu2, 1.0, atol=1e-2)
@test isapprox(wsu2dm, 1.0, atol=1e-2)

# Test CSS for large spin number (#326)
b = SpinBasis(35)  # S=35 overflows `binomial` function, e.g. binomial(70, 26)
theta = π*rand()
phi =2π*rand()
css = coherentspinstate(b,theta,phi)
css0 = coherentspinstate(b,0,0)
css2 = exp(-0.5im * phi * dense(sigmaz(b))) * exp(-0.5im * theta * dense(sigmay(b))) * css0
@test norm(css - css2) < 1e-12
b_big = SpinBasis(1000)  # > 2000 overflows recursive binomial un-moderated with α^n
css_big = coherentspinstate(b_big,theta,phi)
@test abs(1.0 - norm(css_big)) < 1e-12
# we can go bigger than 10000 without overflowing (≈21000 is the limit), but
# at some point we lose more precision to floating-point errors than we'd
# normally want tolerate.
# @test abs(1 - norm(coherentspinstate(SpinBasis(10000),theta,phi))) < 1e-10

########### YLM test #############
res = 1000
int = 0
l = rand(4:33)
m = rand(0:l-1)
for i = 0:2pi/res:2pi, j = 0:pi/res:pi
    int += sin(j)*abs2(ylm(l,m,j,i))
end
t1 = abs(int*2*(pi/res)^2)
@test isapprox(t1, 1.00, atol=1e-2)

l1 = rand(33:40)
m1 = rand(8:30)
l2 = rand(77:80)
m2 = rand(0:10)
int = 0
for i = 0:2pi/res:2pi, j = 0:pi/res:pi
    int += sin(j)*ylm(l1,m1,j,i)*conj(ylm(l2,m2,j,i))
end
t2 = abs(int*2*(pi/res)^2)
@test isapprox(t2, 0, atol=1e-2)

end # testset


# Generate 3 Gaussian peak at position : 5, 10, 20 but transform true
# X into 1.1 * X + 0.2 before saving
#
using DelimitedFiles,Random
using NLS_Fit

Random.seed!(1234)

model = Gaussian_Peak() + Gaussian_Peak() + Gaussian_Peak()
θ1 = Float64[1,5,1]
θ2 = Float64[1.5,10,2]
θ3 = Float64[0.75,20,3]
θ = vcat(θ1,θ2,θ3)

X=Float64[1:0.25:30;]
n=length(X)
Y=eval_y(model,X,θ) + 0.1*(rand(n) .- 0.5)

@. X = 1.1*X + 0.2 # inverse map is: 0.91 * X - 0.18

rootDir = joinpath(dirname(pathof(NLS_Fit)), "..")
dataDir = joinpath(rootDir,"data")
dataFile = joinpath(dataDir,"recalibration.txt")
writedlm(dataFile,hcat(X,Y))