\chapter{Software Analysis}

In this chapter, general software development approaches and practices will be confronted with
requirements posed on the desired software library for recursive Bayesian estimation. After stating these
requirements, feasibility of various programming paradigms applied to our real-world problem is
discussed. Continues a comparison of suitable features of 3 chosen programming
languages: C++, MATLAB language and Python. Emphasis is put on the Python/Cython combination that was
chosen for implementation.

In whole chapter, the term \emph{user} refers to someone (a programmer) who uses the library in
order to implement higher-level functionality (such as simulation of dynamic systems).

\section{Requirements} \label{sec:Requirements}

Our intended audience is a broad scientific community interested in the field of the recursive
Bayesian estimation and decision-making. Keeping this in mind and in order to formalise expectations for
the desired library for Bayesian filtering, the following set of requirements was developed.

\noindent Functionality:
\begin{itemize}
	\item Framework for working with potentially conditional {\pdfs} should be implemented
		including support for basic operations such as product and chain rule. The chain rule
		implementation should be flexible in a way that for example
		\(p(a_t,b_t|a_{t-1},b_{t-1}) = p(a_t|a_{t-1},b_t)p(b_t|b_{t-1})\) product can be
		represented.
	\item Basic Bayesian filtering methods such as the Kalman and particle filter have to be present,
		plus at least one of more specialised algorithms --- a marginalized particle filter or
		non-linear Kalman filter variants.
\end{itemize}
General:
\begin{itemize}
	\item Up-to-date, complete and readable API\footnote{Application Programming Interface, a set of
		rules that define how a particular library is used.} documentation is required. Such
		documentation should be well understandable by someone that already understands mathematical
		background of the particular algorithm.
	\item High level of interoperability is needed; data input/output should be straightforward as
		well as using existing solutions for accompanying tasks such as visualising the results.
	\item The library should be platform-neutral and have to run on major server and workstation
		platforms, at least on Microsoft Windows and GNU/Linux.
	\item The library should be Free/Open-source software as it is believed by the authors that such
		licensing/development model results in software of greatest quality in long term. Framework
		used by the library should make it easy to adapt and extend the library for various needs.
\end{itemize}
Usability:
\begin{itemize}
	\item Initial barriers for installing and setting up the library should be lowest possible.
		For example a necessity to install third-party libraries from sources is considered
		infeasible.
	\item Implementation environment used for the library should allow for high programmer
		productivity; prototyping new solutions should be a quick and cheap (in terms of effort)
		operation. This requirement effectively biases towards higher-level programming
		languages.
\end{itemize}
Performance:
\begin{itemize}
	\item Computational overhead\footnote{excess computational costs not directly involved
		in solving particular problem; for example interpreter overhead.} should be kept reasonably
		low.
	\item Applications built atop of the library should be able to scale well on multi-processor
		systems. This can be achieved for example by thread-safety of critical library objects
		or by explicit parallelisation provided by the library.
\end{itemize}

It is evident that some of the requirements are antagonistic, most prominent example being demand
for \emph{low computational overhead} while still offering \emph{high programmer productivity} and
rapid prototyping. The task of finding tradeoffs between contradictory tendencies or developing smart
solutions that work around traditional limitations is left upon the implementations.

\section{Programming paradigms}

Many programming paradigms exist and each programming language usually suggests a particular paradigm,
though many languages let programmers choose from or combine multiple paradigms. This section
discusses how well could be three most prominent paradigms (procedural, object-oriented and
functional) applied to the software library for Bayesian filtering. Later on additional features of
implementation environments such as interpreted vs. compiled approach or argument passing convention
are evaluated.

\subsection{Procedural paradigm}

The procedural paradigm is the traditional approach that appeared along the first high-level programming languages.
The procedural programming can be viewed as a structured variant of imperative programming, where
programmer specifies steps (in form of orders) needed to reach desired program state.
Structured approach that emphasizes dividing the code into logical and self-contained
blocks (procedures, modules) is used to make the code more reusable, extensible and modular.
Today's most notable procedural languages include C and Fortran.

Most procedural languages are associated with very low overhead (performance of programs
compiled using optimising compiler tend to be very close to ideal programs written in
assembly code); mentioned languages are also spread and well-known in scientific computing.

On the other hand, while possible, it is considered an elaborate task by the author to write
a~modular and extensible library in these languages. Another disadvantage is that
usually only very basic building blocks are provided by the language --- structures like
lists and strings have to be supplied by the programmer or a third-party library. This only
adds to the fact that the procedural paradigm-oriented languages are commonly not easy to learn
and that programmer productivity associated with these languages may be much lower compared
to more high-level languages.

\subsection{Object-oriented paradigm} \label{sec:OOP}

The object-oriented paradigm extends the procedural approach with the idea of \emph{objects} ---
structures with procedures
(called \emph{methods}) and variables (called \emph{attributes}) bound to them. Other
feature frequently offered is \emph{polymorphism} (an extension to language's type
system that adds the notion of \emph{subtypes} and a rule that subtype of a given type can
be used everywhere where given type can be used) most often facilitated through a concept of
\emph{classes}, common models for sets of objects with same behaviour but different
payload; objects are then said to be \emph{instances} of classes. A subclass \emph{inherits}
methods and attributes from its superclass and can \emph{override} them or add its own.
\emph{Encapsulation}, a language mechanism to restrict access to certain object attributes and
methods, may be employed by the language to increase robustness by hiding implementation
details. In order to be considered object-oriented, statically typed languages
(p.~\pageref{desc:StaticTyping}) should provide \emph{dynamic dispatch}\footnote{a way of
calling methods where the exact method to call is resolved at runtime based on actual (dynamic)
object type (in contrast to static object type).}, en essential complement to polymorphism, for
certain or all object methods.

Notable examples of languages that support (although not exclusively) object-oriented
paradigm are statically typed C++, Java and dynamically typed (p.~\pageref{desc:DynamicTyping})
MATLAB language, Python, Smalltalk.

Object-oriented features typically have very small overhead compared to procedural code with
equal functionality, so additional complexity introduced is the only downside, in author's
opinion. We believe that these disadvantages are greatly outweighed by powerful features
that object-oriented languages provide (when utilised properly).

It was also determined that
the desired library for Bayesian filtering could benefit from many object-oriented techniques: {\pdf}
and its conditional variant could be easily modelled as classes with abstract methods that
would represent common operation such as evaluation in a given point or drawing random samples.
Classes representing particular {\pdfs} would then subclass abstract base classes and implement
appropriate methods while adding relevant attributes such as border points for uniform
distribution. This would allow for example to create generic form of particle filter
(p.~\pageref{sec:ParticleFilter}) that would accept any conditional {\pdf} as a parameter.
Bayesian filter itself can be abstracted into a class that would provide a method to compute
posterior {\pdf} from prior one taking observation as a parameter.

\subsection{Functional paradigm}
Fundamental idea of the functional programming is that
functions have no side effects --- their result does not change or depend on program state, only
on supplied parameters. A language where each function has mentioned attribute is called
\emph{purely functional} whereas the same adjective is applied to such functions in other
languages. This is often accompanied by a principle that all data are immutable (apart from
basic list-like container type) and that functions are so-called ``first-class citizens''
--- they can be passed to a function and returned. Placing a restriction of no side-effect on
functions allows compiler/interpreter to do various transformations: parallelisation of function
calls whose parameters don't depend on each other's results, skipping function calls where the
result is unused, caching return values for particular parameters.

Among languages specially designed for functional programming are: Haskell, Lisp dialects Scheme
and~Clojure, Erlang. Python supports many functional programming techniques\footnote{e.g. functions
as first-class citizens, closures, list comprehensions}.

While functional programming is popular subject of academic research, its use is much less
widespread compared to procedural and object-oriented paradigms. Additionally, in the author's
opinion, transition to functional programming requires significant change of programmer's
mindset. Combined with the fact that syntax of the mentioned functionally-oriented languages differs
significantly from many popular procedural or object-oriented languages, we believe that it would
be unsuitable decision for a library that aims for wide adoption.

\subsection{Other programming language considerations}

Apart from recently discussed general approaches to programming, we should note a few other
attributes of languages or their implementations that significantly affect software written using
them. The first distinction is based on type system of a language --- we may divide them into
2~major groups:
\begin{description}
	\item[statically typed languages] \hfill \phantomsection \label{desc:StaticTyping} \\
		bind object types to \emph{variables}; vast majority of type-checking is done at
		compile-time. This
		means that each variable can be assigned only values of given type (subject to
		polymorphism); most such languages require that variable (function parameter, object
		attribute) types are properly declared.
	\item[dynamically typed languages] \hfill \phantomsection \label{desc:DynamicTyping} \\
		bind object types to \emph{values}; vast majority of type-checking is done at runtime.
		Programmer can assign and reassign objects of arbitrary types to given variable. Variables
		(and object attributes) are usually declared by assignment.
\end{description}
We consider dynamically typed languages more convenient for programmers --- we're convinced that the
possibility of sensible variable reuse and lack of need to declare variable types lets the
programmer focus more on the actual task, especially during prototyping stage. This convenience however
comes with a~cost: dynamic typing imposes inevitable computing overhead as method calls and
attribute accesses must be resolved at runtime. Additionally, compiling a program written in statically
typed language can reveal many simple programming errors such as calling mistyped methods, even
in unreachable code-paths; this is not the case for dynamically-typed languages and we suggest
compensating this with more thorough test-suite (code coverage tools can greatly help with creating
proper test-suite, see \autoref{sec:PyBayesDocsTests} on page \pageref{sec:PyBayesDocsTests}).

Another related property is interpreted vs. compiled nature; we should emphasize that this property
refers to language \emph{implementation}, not directly to the language itself, e.g. C language
is commonly regarded as compiled one, several C interpreters however exist. We use the term
``language is compiled/interpreted'' to denote that principal implementation of that language is
compiled, respectively interpreted.
\begin{description}
	\item[compiled implementations] \hfill \\
		translate source code directly into machine code suitable for given target processor. Their
		advantage is zero interpreter overhead. Developers are required to install a
		compiler (and perhaps a build system) or an IDE\footnote{Integrated Development Environment}
		used by given project (library) to be able to modify it. Write-build-run-debug cycle is
		usually longer in comparison to interpreted implementations.
	\item[interpreted implementations] \hfill \\
		either directly execute commands in source code or, more frequently, translate source code
		into platform-independent \emph{intermediate representation} which is afterwards executed in
		a \emph{virtual machine}. We may allow the translate and execute steps to be separated so that
		Java and similar languages can be included. Advantages include usually shorter
		write-run-debug cycle that speeds up development and portable distribution options.
		Interpreted languages have been historically associated with considerable processing
		overhead, but \emph{just-in-time compilation}\footnote{interpreter feature that translates
		portions of bytecode into machine code at runtime.} along with \emph{adaptive
		optimisation}\footnote{a technique to use profiling data from recent past (collected perhaps
		when relevant portion of code was run in interpreted mode) to optimise just-in-time compiled
		code.} present in modern interpreters can minimise or even reverse interpreter
		overhead: Paul Buchheit have shown\footnote{\url{http://paulbuchheit.blogspot.com/2007/06/java-is-faster-than-c.html}}
		that second and onward iterations of fractal-generating Java program were actually
		5\%~faster than equivalent C program. We have reproduced the test with following results:
		Java program was 10\% slower (for second and subsequent iterations) than C program and 1600\%
		slower when just-in-time compilation was disabled. Complete test environment along with
		instructions how to reproduce it be found in the
		\href{http://github.com/strohel/PyBayes/tree/master/examples/benchmark_c_java}{\nolinkurl{examples/benchmark_c_java}}
		directory in the PyBayes source code repository.
\end{description}
There exists a historic link between statically typed and compiled languages, respectively
dynamically typed and interpreted languages. Java which is itself statically typed and it's major
implementation is interpreted and Erlang's (which is dynamically typed) compiled
HiPE\footnote{The High-Performance Erlang Project: \url{http://www.it.uu.se/research/group/hipe/}}
implementation are some examples of languages that break the rule. We believe that this historic
link is the source of a common misconception that interpreted languages are inherently slow. Our
findings (see also Python/Cython/C benchmark on p. \pageref{sec:CythonPerformace}) indicate that
the source of heavy overhead is likely to be the dynamic type system rather than overhead of modern
just-in-time interpreters. In accordance with these findings, we may conclude that choice of language
implementation type should rather be based on development and distribution convenience than on
expected performance.

Each programming language may support one or more following function call conventions that determine
how function parameters are passed:
\begin{description}
	\item[call-by-value convention] \hfill \\
		ensures that called function does not change variables passed as parameters from calling
		function by copying them at function call time. This provides clear semantics but incurs
		computational and memory overhead, especially when large data structures are used as
		parameters. As a form of optimisation, some language implementations may employ
		copy-on-write technique so that variables are copied only when they are mutated from within
		called function, thus saving space and time when some parameters are only read from.
	\item[call-by-reference convention] \hfill \\
		hands fully-privileged references to parameters to called function. These references can be
		used to modify or assign to parameters within called function and these changes are visible
		to calling function. This approach minimises function call overhead but may appear confusing
		to a programmer when local variable is changed ``behind her back" unexpectedly. On the other
		hand, call-by-reference allows for programming techniques impossible with call-by-value
		alone (e.g. a function that swaps two values).
	\item[call-by-object (call-by-sharing) convention] \hfill \\
		can be viewed as a compromise between call-by-value and call-by-reference: parameters are
		passed as references that can be used to modify referred objects (unless marked immutable),
		but cannot be used to assign to referred objects (or this assignment is invisible to calling
		function). When an object is marked as immutable,
		passing this object behaves like call-by-value call without copying overhead (in the calling
		function point of view). Java and Python use call-by-object as their sole function calling
		method\footnote{python case: \url{http://effbot.org/zone/call-by-object.htm}} and both mark
		certain elementary types (most prominently numbers and strings) as immutable. C's
		pointer-to-const and C++'s reference-to-const parameters can be viewed as call-by-object
		methods where referred objects are marked as immutable in called function scope.
\end{description}
We suggest that a language that supports at least one of call-by-reference or call-by-object
conventions is used for the desired recursive Bayesian estimation library; while call-by-value-only
languages can be simpler to implement, we are convinced that they impose unnecessary restrictions
on the library design and cause overhead in places where it could be avoided.

Last discussed aspect of programming languages relates to memory management:
\begin{description}
	\item[garbage-collected languages] \hfill \\
		provide memory management in the language itself. This fact considerably simplifies
		programming as programmer doesn't need to reclaim unused memory resources herself. Another advantage
		is that automatic memory management prevents most occurrences of several programming errors:
		memory leaks,\footnote{an error condition when a region of memory is no longer used, but not
		reclaimed.} dangling pointers\footnote{a pointer to an object that has been already destroyed;
		such pointers are highly error-prone.} and double-frees.\footnote{an error condition where a
		single region of memory is reclaimed twice; memory corruption frequently occurs in this
		case.} Two major
		approaches to garbage collection exist and both incur runtime computational or memory
		overhead. \emph{Tracing garbage collector} repeatedly scans program heap\footnote{an area of
		memory used for dynamic memory allocation.} memory for objects
		with no references to them, then reclaims memory used by these objects. Program performance
		may be substantially impacted while tracings garbage collector performs its scan; furthermore
		the moment when garbage collector fires may be unpredictable. \emph{Reference counting}
		memory management works by embedding an attribute, \emph{reference count}, to each object
		that could be allocated on heap and then using this attribute to track number of references
		to given object. When reference count falls to zero, the object can be destroyed. Reference
		counting adds small memory overhead per each object allocated and potentially significant
		computational overhead as reference counts have to be kept up-to-date. However, techniques
		exist that minimise this overhead, for example those mentioned in~\cite{LevPet:06}.
	\item[non garbage-collected languages] \hfill \\
		put the burden of memory management on shoulders of the programmer: she is responsible for
		correctly reclaiming resources when they are no longer in use. The advantages are clear:
		no overhead due to memory management, probably also smaller complexity of language
		implementation. However, as mentioned earlier, languages without automatic memory management
		make certain classes of programmer errors more likely to occur.
\end{description}
In our view, convenience of garbage-collected languages outweighs overhead they bring for a project
like a library for recursive Bayesian estimation targeting wide adoption. We also believe that automatic
memory management can simplify library design and its usage as there is no need to specify who is
responsible for destroying involved objects on the library side and no need to think about it at
the user side.

\section{C++}

C++ is regarded as one of the most popular programming languages today, along with Java and
C;\footnote{TIOBE Programming Community Index for July 2011:
\url{http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html}} it combines properties of
both low-level and high-level languages, sometimes being described as intermediate-level language.
C++ extensively supports both procedural and class-based object-oriented paradigm, forming a
multi-paradigm language; generic programming is implemented by means of \emph{templates}, which
allow classes and functions to operate on arbitrary data types while still being type-safe.
C++ is statically-typed, all major implementations are compiled, supports call-by-value (the
default), call-by-reference and a variant of call-by-object function call conventions. C++ lacks
implicit garbage collection for heap-allocated data --- the programmer must reclaim memory used
by those objects manually; use of \emph{smart pointers}\footnote{a template class that behaves like
a pointer through use of operator overloading but adds additional memory management features such as
reference counting} may although help with this task. C++ is almost 100\% compatible with the C
language in a way that most C programs compile and run fine then compiled as C++ programs. C++ also
makes it easy to use C libraries without a need to recompile them.~\cite{Str:00}

When used as an implementation language for the desired library for recursive Bayesian estimation, we
have identified potential advantages of the C++ language:
\begin{description}
	\item[low overhead] \hfill \\
		C++ was designed to incur minimal overhead possible. In all benchmarks we've seen (e.g. The
		Computer Language Benchmarks Game\footnote{\url{http://shootout.alioth.debian.org/}}), it is
		hard to outperform C++ by a significant margin (Fortran and assembly code would be
		candidates for that).
	\item[widespread] \hfill \\
		C/C++ code forms large part of the software ecosystem. Thanks to that, incredible number of
		both proprietary and free IDEs, debuggers,
		profilers and other related coding tools is available. This fact makes development more
		convenient.
	\item[libraries] \hfill \\
		Thanks to C++ popularity, several high-quality libraries for numerical calculations/computer
		algebra are available, many of them are free software or free to use. These are for example
		C interfaces to BLAS\footnote{Basic Linear Algebra Subprograms: \url{http://www.netlib.org/blas/}}
		and LAPACK\footnote{Linear Algebra PACKage: \url{http://www.netlib.org/lapack/}} (both low-level
		and fixed function), higher-level IT++\footnote{\url{http://itpp.sourceforge.net/}} built
		atop of BLAS/LAPACK or independent template-based library
		Eigen.\footnote{\url{http://eigen.tuxfamily.org/}} Additionally,
		OpenMP\footnote{The OpenMP API specification for parallel programming: \url{http://openmp.org/}}
		can be used to parallelise existing algorithms without rewriting them.
\end{description}
However, using C++ would, in our opinion, bring following major drawbacks:
\begin{description}
	\item[diversity] \hfill \\
		While there are many C/C++ libraries for specific tasks (such as data visualisation), it
		may prove difficult in our opinion to combine them freely as there are no \emph{de facto}
		standard data types for e.g. vectors and matrices --- many libraries use their own.
	\item[learning curve] \hfill \\
		C++ takes longer to learn and even when mastered, programmer productivity is subjectively
		lower compared to very high-level languages. We also fear that many members of out intended
		audience are simply unwilling to learn or use C++.
\end{description}
Moreover, discussion about statically-typed, compiled and non-garbage-collected languages from
previous section also apply. Due to this, we have decided not to use C++ if an alternative with
reasonable overhead is found.

Several object-oriented C++ libraries for recursive Bayesian estimation exist:
Bayes++\footnote{\url{http://bayesclasses.sourceforge.net/}}, BDM~\cite{BDM} and BFL~\cite{BFL}.
BDM library is later used to compare performance of Cython, C++ and MATLAB implementations of the
Kalman filter, see \autoref{sec:PyBayesPerformance} on page \pageref{sec:PyBayesPerformance}.

\section{MATLAB language}

MATLAB language is a very high-level language used exclusively by the
MATLAB\footnote{\url{http://www.mathworks.com/products/matlab/}} environment, a~proprietary platform
developed by MathWorks.\footnote{\url{http://www.mathworks.com/}} MATLAB language extensively
supports procedural programming paradigm and since version 7.6 (R2008a) class-based object oriented
paradigm is also fully supported.\footnote{\url{http://www.mathworks.com/products/matlab/whatsnew.html}}
MATLAB language is dynamically-typed, interpreted language with automatic memory management.

MATLAB language possesses, in our belief, following favourable attributes when used to implement the
desired library for Bayesian filtering:
\begin{description}
	\item[popularity among academia] \hfill \\
		While MATLAB language is not as widespread as C++ on the global scale, it is very popular in
		scientific community, our intended audience.
	\item[performance] \hfill \\
		MATLAB language is very well optimised for numerical computing.
	\item[wide range of extensions] \hfill \\
		High number of well integrated extension modules (toolboxes) is bundled with MATLAB or
		available from third parties. This makes associated tasks such as data visualisation
		particularly straightforward.
	\item[rapid development] \hfill \\
		Being a very high-level language, we expect programmer productivity in the MATLAB language being
		fairly high. MATLAB environment is itself a good IDE and its interactive shell fosters rapid
		prototyping.
\end{description}
Following disadvantages of the MATLAB language were identified:
\begin{description}
	\item[vendor lock-in] \hfill \\
		MATLAB is commercial software; free alternatives such as GNU
		Octave\footnote{\url{http://www.gnu.org/software/octave/}},
		Scilab\footnote{\url{http://www.scilab.org/}} or
		FreeMat\footnote{\url{http://freemat.sourceforge.net/}} exist, however all of them provide
		only limited compatibility with the MATLAB language. Developing for a non-standard proprietary
		platform always imposes risks of the vendor changing license or pricing policy etc.
	\item[problematic object model] \hfill \\
		We have identified in \autoref{sec:OOP} that object-oriented approach is important for
		a~well-designed and usable library for Bayesian filtering. Nonetheless MATLAB's
		implementation of object-oriented programming is viewed as problematic by many, including us. For
		example, function call parameter passing convention is determined by the object class/data
		type --- MATLAB distinguishes \emph{value classes} that have call-by-value semantics and
		\emph{handle classes} that have call-by-object semantics.\footnote{call-by-object semantics
		tested in version 7.11 (R2010b).} The resulting effect is that calling
		identical function with otherwise equivalent value and handle classes can yield very
		different behaviour.
	\item[hard-coded call-by-value semantics] \hfill \\
		2D array, a very central data-type of the MATLAB language, has call-by-value function call
		convention hard-coded; this results in potentially substantial function call overhead.
		Although current MATLAB versions try to minimise copying by employing copy-on-write
		technique\footnote{\url{http://blogs.mathworks.com/loren/2006/05/10/memory-management-for-functions-and-variables/}}
		or performing some operations in-place,\footnote{\url{http://blogs.mathworks.com/loren/2007/03/22/in-place-operations-on-data/}}
		our tests have shown that even combining these techniques doesn't eliminate unnecessary
		copying overhead which we believe is the main source of grave performance regression of
		object-oriented code with regards to imperative code; see \autoref{sec:PyBayesPerformance}
		on page \pageref{sec:PyBayesPerformance}.
\end{description}
We consider presented drawbacks significant and therefore decided not to use the MATLAB language for
the desired Bayesian filtering library. BDM library~\cite{BDM} contains both object oriented and
imperative implementation of the Kalman filter in the MATLAB language; these are compared with our
implementation in \autoref{sec:PyBayesPerformance}.

\section{Python}

Python\footnote{\url{http://www.python.org/}} is a very high level programming language designed for
outstanding code readability and high programmer productivity actively developed by the
Python Software Foundation.\footnote{\url{http://www.python.org/psf/}} Python extensively supports
procedural and class-based object-oriented programming paradigms and some features of the functional
programming. Python is dynamically-typed language with automatic memory management that exclusively employs
call-by-object function call parameter passing convention; elementary numeric types, strings and
tuples are immutable\footnote{\url{http://docs.python.org/reference/datamodel.html}} so that this
approach doesn't become inconvenient.

Principal Python
implementation, CPython, is written in C, is cross-platform and of interpreted type: it translates
Python code into bytecode which is subsequently executed in a virtual machine. Many alternative
implementations are available, to name a few: Jython\footnote{\url{http://www.jython.org/}} that
translates Python code into Java bytecode (itself written in Java),
IronPython\footnote{\url{http://ironpython.net/}} itself implemented on top of the .NET Framework,
just-in-time compiling PyPy\footnote{\url{http://pypy.org/}} written in Python itself or
Cython which is described in greater detail in the next section. All the
mentioned implementations qualify as free/open-source software.

Python language is bundled with a comprehensive standard library so that writing new projects is quick
from the beginning. Two major Python versions exists: Python 2, considered legacy and receiving only
bugfix updates, and Python 3, actively developed and endorsed version that brings a few incompatible
changes to the language syntax and to the standard library. Porting Python 2 code to version 3 is
however usually straightforward and can be automated to a great extent with tools bundled with
Python 3.

In our belief, Python shows following favourable attributes when used for the desired Bayesian
filtering library:
\begin{description}
	\item[development convenience, readability, rapid prototyping] \hfill \\
		Python developers claim that Python in an easy to learn, powerful programming language and
		our experience confirms their claims. Python code is easy to prototype, understand and
		modify in our opinion; prototyping is  with bundled interactive Python shell.
		While all these statements are subjective, they are shared among
		many.\footnote{\url{http://python.org/about/quotes/}} For example a statement \verb|x <= y <= z|
		has its mathematical meaning, which is unusual for programming languages.
	\item[NumPy, SciPy, Matplotlib] \hfill \\
		NumPy project\footnote{\url{http://www.numpy.org/}} is the de facto standard Python library for
		numeric computing; NumPy provides N-dimensional array type that is massively supported in
		very high number of projects. Parts of NumPy are written in C and Cython for speed.
		SciPy\footnote{\url{http://www.scipy.org/}} extends NumPy with more numerical routines.
		Matplotlib\footnote{\url{http://matplotlib.sourceforge.net/}} is powerful plotting library
		that natively supports SVG output. Combining these three and Python gives a very vital
		MATLAB alternative.
	\item[interoperability with C] \hfill \\
		CPython makes it possible to write modules\footnote{module in python sense is a code unit with
		its own namespace, normally each module corresponds to a .py file.} in
		C\footnote{\url{http://docs.python.org/extending/index.html}} (that are then called \emph{extension modules}).
		Cython makes it easy and convenient to write extension modules. Sadly, alternative
		implementations PyPy, Jython and IronPython don't currently fully support extension modules
		and are therefore ruled-out for our purposes because they in turn don't support
		NumPy.\footnote{PyPy and IronPython are nonetheless interesting for future consideration as
		both have NumPy support actively worked on.}
	\item[interoperability with MATLAB] \hfill \\
		SciPy contains procedures to load and save data in MATLAB .mat format; Matplotlib includes
		programming interface that resembles MATLAB's plotting procedures.
\end{description}
On the other hand, a few downsides exist:
\begin{description}
	\item[overhead] \hfill \\
		CPython implementation shows significant computational overhead, especially for numerical
		computations; CPython doesn't currently utilise any form of just-in-time compiling.
		NumPy is often used to trade off computational overhead for memory overhead:
		The Computer Language Benchmarks Game\footnote{\url{http://shootout.alioth.debian.org/}}
		contains an~example where a program heavily using NumPy is 20\x\ faster but consumes
		12\x\ more memory than a program that performed the same task and used solely the
		Python standard library. We have reproduced the benchmark with similar results; mentioned
		programs can me found in the
		\href{http://github.com/strohel/PyBayes/tree/master/examples/benchmark_py_numpy}{\nolinkurl{examples/benchmark_py_numpy}}
		directory in the PyBayes source code repository. We still consider this workaround suboptimal.
	\item[peculiar parallelisation] \hfill \\
		While Python natively supports threads and they are useful for tasks such as background I/O
		operations, Python threads don't scale on multiprocessor systems for CPU-bound processing;
		such code often runs at single-processor speed (when run in CPython). The reason behind that
		is that CPython employs a global-interpreter-lock (GIL) to to assure that only one thread
		executes Python bytecode at a time.\footnote{\url{http://docs.python.org/glossary.html}} This
		restriction can be worked around by using multiple python interpreters that communicate with
		each other; Python module \emph{multiprocessing} makes it almost as convenient as using
		threads.
\end{description}
Python compares favourably to other implementation environments presented before in our view; sole
major obstacle being excessive overhead of the CPython interpreter. It is discussed how this issue
can be solved using Cython in the next section.

We haven't found any Python library for recursive Bayesian estimation that would fulfil the
requirements presented at the beginning of this chapter.

\section{Cython}

Cython~\cite{BehBraCitDalSelSmi:11} is both an extension to the Python language and an implementation
of it (a compiler). Cython works by translating Cython modules (.pyx or .py files) into the C language which is
then compiled to form binary Python extension modules (dynamically loaded libraries --- .dll files
on Windows and .so files on UNIX platforms). Cython aims to be a strict superset of Python in a way
that a valid Python module is also a valid Cython module that behaves equally. Current development
snapshot of Cython virtually achieves this goal as it successfully passes 97.6\% of the Python 2.7.1
regression test suite. Additionally, interpreted Python modules and Cython-compiled modules can be
mixed and interchanged freely as Cython-compiled code can transparently interact with interpreted
Python code and vice-versa. However, Cython is not a replacement of CPython --- Cython-compiled
modules need to be executed by CPython because they make heavy use of CPython internals written in C
(C code emitted by the Cython compiler is largely composed of calls to functions from CPython C API);
Cython-compiled modules merely circumvent the virtual machine (bytecode interpreter) part of CPython,
thus virtually eliminating interpreter overhead. This is in fact probably the most serious
limitation of Cython --- it is tightly bound to one specific (although the most prevalent) Python
implementation, CPython.

Another key feature Cython supports is static typing of variables, function parameters and object
attributes. In addition, Cython allows statically typed variables to be native C types such as
\verb|int|, \verb|char *| or \verb|struct sockaddr| in addition to Python types; automatic
conversion on Python/C boundary is provided by Cython for C types with Python equivalents, C
\verb|double| is wrapped as Python \verb|float| and C \verb|long| is wrapped as Python
\verb|int| for example. The main purpose of static typing are impressive speed gains; an extreme
case exploited by our test case showed 65\x\ speed increase of typed Cython code compared
to untyped Cython code. In important thing to mention is that static typing is completely
\emph{voluntary} for the programmer; recent Cython versions also support experimental basic type
inference\footnote{a way to guess and prove type of certain variable using code analysis} (that
is guaranteed not to change semantics). Simply put, static typing prevents significant overhead
caused by highly dynamic nature of Python.

\subsection{Cython Features} \label{sec:CythonFeatures}

We continue by a brief technical description of some Cython extensions to Python and other features
so that Cython's benefits can be evaluated as a whole later. Unless noted otherwise, Cython version
0.14.1 is described here; please note that Cython is a rapidly evolving project and some of the
mentioned features or limitations may well change in future versions.
\begin{description}
	\item[cdef variables] \hfill \\
		Cython supports so-called \emph{cdef variables} in both global (module) scope and function
		(method) scope that can be typed; in addition to Python types, they can also have native
		C types. Cdef variables aren't visible to Python code, but some overhead is eliminated as they
		are, for example, not reference-counted.
	\item[cdef and cpdef functions] \hfill \\
		In addition to traditional Python functions (\verb|def function(x)| in code)
		that use rather expensive calling convention (e.g. all their positional arguments are packed
		into a tuple and all their keyword arguments are packed into a dictionary on each call)
		Cython supports so-called \emph{cdef functions} and \emph{cpdef functions}. Cdef functions
		(defined as \verb|cdef function(x)|, hence the name) use native C calling convention and
		are only callable from Cython code, with greatly reduced overhead; their return value can
		be typed (parameters can be typed even for traditional Python functions). Cpdef functions
		(defined as \verb|cpdef function(x)|) are same as cdef functions but in addition a Python
		wrapper around the C function with the same name is generated; such function is callable
		from Python (with overhead) and fast to call from Cython, cpdef functions therefore combine
		the benefits of both.
		Neither cdef or cpdef functions can be Python class methods, but see the next entry.
	\item[extension types (cdef classes)] \hfill \\
		In addition to traditional Python classes marked as \eqref{eq:PythonClass} in code that
		store their attributes and methods in a class dictionary, which leads to inefficient attribute
		lookups and method calls, Cython supports so-called \emph{cdef classes} (or extension types)
		that are defined using \eqref{eq:CdefClass} and use C structs to store their
		attributes and method table, a significantly faster approach than Python dictionary.
		\begin{align}
			&\mathtt{class~ClassName(SuperClass):} \label{eq:PythonClass} & ~ & ~ &\\
			&\mathtt{cdef~class~ClassName(SuperClass):} \label{eq:CdefClass}
		\end{align}
		Cdef
		classes can also have cdef and cpdef methods as their members; they can also contain cdef
		variables as attributes with optional additional modifiers \verb|public| (read-write from
		Python) or \verb|readonly| (read-only from Python). Cdef classes are, in contrast to similarly
		named cdef functions, visible to Python code where they appear as built-in types. Compared
		to traditional Python classes, cdef classes are however subject to 2 major limitations: all
		their attributes have to be statically declared (as opposed to dynamically created at
		runtime) plus multiple inheritance cannot be used. One can overcome both these limitations
		by subclassing them in Python, which is indeed possible.
	\item[interfacing C/C++ code] \hfill \\
		As described in \cite{BehBraSel:09}, Cython can be used to conveniently call C or
		C++\footnote{C code emitted by the Cython compiler is compilable also when treated as C++,
		thus allows interfacing with C++ code.} library functions. Suppose we want to call the
		\verb|sin()| function form the C standard library that is defined in \verb|math.h|. Following
		Cython module accomplishes that:
		\vspace{\parskip}
		\begin{Verbatim}[samepage=true,gobble=3,label=sin\_wrapper.pyx,frame=single]
			cdef extern from math.h:
			    double sin(double x)

			sin(0.123)  # call C function directly from Cython

			cpdef double sin_wrapper(double x):
			    return sin(x)
		\end{Verbatim}
		In the example above, the \verb|sin_wrapper()| function can be called from Python code.
		Cython is therefore an excellent tool to create Python wrappers around C/C++ libraries or
		use them directly.
	\item[NumPy support] \hfill \\
		As was noted above, many core parts of NumPy are written in C (or Cython), including the
		crucial data-type, N-dimensional array. Cython provides explicit support for NumPy
		data-types and core array-manipulation methods and functions allowing the programmer to
		take advantage of the speed gains of both.~\cite{Sel:09}

		On the other hand, NumPy support in Cython (or Cython support in NumPy) is far from complete,
		for example matrix multiplication, linear system solving and matrix decomposition functions
		are usually speeded up using C (and BLAS, LAPACK) internally, but to our knowledge it is
		presently impossible to call these functions from Cython without Python overhead (the overhead
		is however significant only for small matrix sizes).
	\item[pure Python mode] \hfill \\
		It is clear that in order to make use of all important features of Cython, one has to use
		syntax that is no longer valid Python code; this can be disadvantageous in many cases. To
		combat this, Cython offers so-called \emph{pure Python mode} where Cython-specific syntax
		is wrapped in Python constructs, e.g. \eqref{eq:CdefInt} can be alternatively formulated
		as \eqref{eq:CdefIntPure} --- such constructs are implemented by Cython
		shadow code to be no-ops when interpreted by Python and treated accordingly when compiled
		by Cython.
		\begin{align}
			&\mathtt{cdef~int~i} \label{eq:CdefInt} & ~ & ~ &\\
			&\mathtt{x~=~cython.declare(cython.int)} \label{eq:CdefIntPure}
		\end{align}
		Another variant of pure Python mode is to use so-called \emph{augmenting files}, .pxd files
		that accompany equally-named .py files. Such augmenting file can contain declarations of
		variables, functions and classes that appear in associated .py files and add Cython-specific
		features to them. For example, suppose that it is desired to have a function that is
		inexpensive to call from Cython but still providing full Python compatibility, one can write
		following Python module:
		\vspace*{4pt} % \parskip does not work here
		\begin{Verbatim}[samepage=true,gobble=3,label=module.py,frame=single]
			def f(x):
			    return x*x
		\end{Verbatim}
		And associated augmenting file:
		\vspace*{4pt} % \parskip does not work here
		\begin{Verbatim}[samepage=true,gobble=3,label=module.pxd,frame=single]
			cdef double f(double x)
		\end{Verbatim}
		It is forbidden to provide implementation in .pxd files. This alternative approach is
		advantageous to the first one because it doesn't require Cython to be installed on a target
		machine (on the other hand, some very special Cython features currently cannot be expressed
		in augmenting files). .pxd files also serve for sharing declarations between Cython modules
		in a way similar to C header (.h)
		files.\footnote{\url{http://docs.cython.org/src/userguide/sharing_declarations.html}}
	\item[parallelisation] \hfill \\
		One recent feature of current development snapshots of Cython is a native support for C-level
		parallelisation through OpenMP\footnote{The OpenMP API specification for parallel
		programming: \url{http://openmp.org/}} --- Cython adds new \verb|prange()| function that is
		similar to the Python \verb|range()| function except that loop statements are executed in
		parallel. Currently only loops whose statements can be called without holding the
		GIL\footnote{global-interpreter lock \url{http://docs.python.org/glossary.html}} can be
		efficiently parallelised. Our tests have shown that such parallel loops scale well with the
		number of processors, see \autoref{sec:CythonPerformace} on page \pageref{sec:CythonPerformace}.
	\item[Python 3 compatibility] \hfill \\
		Cython provides full compatibility with Python 3 in particularly robust way: Cython modules
		can be written in both Python 2 or Python 3 and C files that Cython produces can be compiled
		against both CPython 2 or CPython 3, independently from the source file
		version.\footnote{\url{http://wiki.cython.org/FAQ}}
		This effectively makes Cython a 2-way compatibility bridge between Python 2 and Python 3.
\end{description}
There are many areas where Cython can get better, however most of them are just missed optimisation
possibilities. We mention 2 major cases where Cython currently doesn't accept valid Python constructs:
the \emph{yield} statement\footnote{\url{http://docs.python.org/reference/simple_stmts.html}}
used to create generator functions is currently unsupported and
\emph{generator expressions}\footnote{\url{http://docs.python.org/reference/expressions.html}}
are supported only in special cases. Other minor inconveniences exist but we don't consider them
worth discussing here. Another important attribute of Cython is, in our belief, its active developer
community, for example we've discovered a bug related to Cython's pure Python mode capability that
was fixed upon providing a test-case.\footnote{\url{http://trac.cython.org/cython_trac/ticket/583}}

\subsection{Performance comparison with C and Python} \label{sec:CythonPerformace}

In order to evaluate performance of Cython, we've conducted a benchmark where equivalent Python,
Cython-compiled Python, Cython and C programs are compared. Inspired by the Cython tutorial, the
test program performs simple numerical integration of the function \(x^2\) from 0 to 3. Python
version of the test program is shown here, complete test environment along with instructions how
to reproduce can be found in the
\href{http://github.com/strohel/PyBayes/tree/master/examples/benchmark_c_cy_py}{\nolinkurl{examples/benchmark_c_cy_py}}
directory in the PyBayes source code repository.
\begin{Verbatim}[samepage=true,gobble=1,label=integrate\_python.py,frame=single]
	def f(x):
	    return x*x

	def integrate(a, b, N):
	    s = 0
	    dx = (b-a)/N
	    for i in xrange(N):
	        s += f(a + (i + 1./2.)*dx)*dx
	    return s
\end{Verbatim}
The test was performed on a 64-bit dual-core Intel Core i5-2520M CPU clocked at 2.50 Ghz with Intel Turbo
Boost and Hyper-threading enabled, giving the total of 4 logical processor cores (each physical core
being able to execute 2 CPU threads); operating system is Gentoo Linux compiled for the x86\_64
platform. Versions of relevant software packages are listed below:
\begin{description}
	\item[Python] 2.7.1
	\item[GNU C Compiler] 4.4.5; \verb|-O2| optimisation flag used when compiling C files
	\item[Cython] 0.14.1+ (git revision 0.14.1-1002-g53e4c10)
	\item[PyPy] 1.5.0-alpha0 with JIT compiler enabled
	\item[PyBayes] 0.3 (contains test program sources)
\end{description}
Program abbreviations used in benchmark results:

\noindent\begin{tabular}{rp{\textwidth-99pt}}
	\verb|c_omp|        & C version, for loop parallelised using OpenMP \\
	\verb|cy_typed_omp| & Cython version with all variables typed, parallelised using \verb|prange()| \\
	\verb|pypy|         & PyPy-executed Python version, single-threaded \\
	\verb|c|            & C version, single-threaded \\
	\verb|cython_typed| & Cython version with all variables typed, single-threaded \\
	\verb|cython|       & Cython-compiled Python version, single-threaded \\
	\verb|python|       & Python version, single-threaded \\
\end{tabular}

\begin{Verbatim}[label=typical benchmark run,frame=single]
Numerical integration from 0.0 to 3.0 of x^2 with 200000000 steps:

       c_omp: result = 9.0; real time = 0.446s; cpu time = 1.72s
cy_typed_omp: result = 9.0; real time = 0.447s; cpu time = 1.73s
        pypy: result = 9.0; real time = 0.851s; cpu time = 0.84s
           c: result = 9.0; real time = 1.597s; cpu time = 2.00s
cython_typed: result = 9.0; real time = 1.590s; cpu time = 1.58s
      cython: result = 9.0; real time = 33.26s; cpu time = 33.2s
      python: result = 9.0; real time = 95.05s; cpu time = 94.8s

Relative speedups:
      cython/python:       2.8571203030
cython_typed/cython:       20.9136660253
           c/cython_typed: 0.99693008347

cy_typed_omp/cython_typed: 3.55556994329
       c_omp/c:            3.57674225915

       c_omp/cy_typed_omp: 1.00186053812
\end{Verbatim}
The \verb|cpu time| quantity is not reliable in our belief and is mentioned only for illustration;
all measurements are based on the \verb|real time| quantity that measures wall-clock time needed to
perform the algorithm. The number of steps was chosen artificially high to get timings that are
easier to measure.
The benchmark had little variance in results across runs, the relative sample standard deviation
\eqref{eq:RelStdDev} was under 3\% for all measured quantities (run times) with \(N = 10\).
\begin{equation} \label{eq:RelStdDev}
	s_{\text{rel}} = \frac{1}{\overline{x}} \; \sqrt{\frac{1}{N-1} \sum_{i=1}^N (x_i - \overline{x})^2}
\end{equation}
The test produced a couple of interesting results: the same Python code was 2.9\x\ faster
when compiled by Cython; we are convinced that this is due to the Python interpreter overhead. Adding
static type declarations to the Cython code resulted in additional 21-fold speed-up giving the total
60\x\ increase in performance of a statically-typed Cython procedure compared to the dynamically
typed Python one, forming a very impressive result. Somewhat surprising results were obtained from
parallelisation tests --- parallelised Cython version was 3.5\x\ faster than equivalent
singe-threaded code on a system with only 2 physical processor cores, we speculate that this is due
to the Hyper-threading technology employed by the processor that reduced wasted CPU cycles (where
the CPU waits for memory fetches). Nonetheless this shows that Cython (and C) parallelisation techniques
using OpenMP scale very well. C and Cython versions of the algorithm performed virtually equally
in both singe-threaded and multi-threaded cases giving an indication that Cython performs a very good
job at optimising Python code in such simple cases. Another surprise was PyPy (operating on
unmodified Python code) performance --- it was considerably faster that both C and optimised Cython
code making it a very interesting option once it supports NumPy.

We should however note that these extreme Cython performance gains are specific for such simple and
highly numeric algorithms. We expect smaller benefits for more high-level code; Kalman filter tests
(\autoref{sec:PyBayesPerformance} on page \pageref{sec:PyBayesPerformance}) support our claims.

\subsection{Discussion}

Cython, in our view, fixes the last possible barrier (the CPython overhead) before Python can be used
for the desired library for Bayesian filtering. We consider especially important that optimisation
can be approached \emph{gradually} --- one can write code purely in Python and add static type
definitions only for performance critical parts (that show up as bottlenecks during profiling) and
only once the high performance is needed; Cython developers soundly discourage adding static types
everywhere. Second key feature of Cython, as identified by us and for
out purposes, is the pure Python mode. Employing Cython brings the disadvantages of compiled languages,
mainly a prolonged write-build-run-debug cycle. When the pure Python mode\footnote{specifically, the
variant where only augmenting files are used} is used, all these shortcomings
are effectively voided (as the code works also under plain Python) at the cost of very minor loss
of programming convenience. We expect that many potential library users don't have high performance
requirements, these could ignore Cython entirely and use the library as any other plain python
module.

Python/Cython combination was therefore chosen as the implementation environment for the desired
library for recursive Bayesian estimation; the library was named \emph{PyBayes} and is presented in
the next chapter.


\documentclass[preprint, onecolumn, amsmath, amssymb, aps]{revtex4-1}
\usepackage{graphicx}
\usepackage{dcolumn}
\usepackage{bm}

\usepackage{cancel}
\usepackage{titlesec}
\usepackage{listings}
\usepackage{xcolor}
\lstset
{
	language=Python,
	backgroundcolor=\color{black!5},
	basicstyle=\footnotesize,
}

\def\thesection{\arabic{section}}
\numberwithin{equation}{section}

\begin{document}

\title{The Transfer Matrix Method in Linear Dielectrics}

\author{Claudio Barros}

\affiliation{University at Buffalo, Amherst, New York 14260}

\date{\today}

\renewcommand{\abstractname}{\vspace{0\baselineskip}}

\begin{abstract}

The propagation of an electromagnetic wave through a series of dielectric media is explored through the use of the Transfer Matrix Method, which permits us to relate the electric fields between different layers of materials. As we shall see, we will be able to derive expressions for the reflected and transmitted power using nothing more than classical electrodynamics and some linear algebra. Our final goal will be to implement this method in a Python script and therefore model the phenomena of thin-film reflection and anti-reflection coatings.

\end{abstract}

\maketitle

\section{Introduction}

The Transfer Matrix Method (TMM) provides us with a powerful tool for calculating the propagation of light, or for that matter any wave-like phenomenon, through a series of optical elements. Most startling of all, the apparent complex nature of light propagation can be greatly simplified by casting these optical elements into matrices that transform a given initial beam of light into a final state based solely on the properties of the matrices. Now, these optical elements can represent a wide range of contraptions such as wave-plates, lenses, potentials, or films, which is the main subject of this paper. More specifically, we shall focus on the propagation of an electromagnetic wave through a series of dielectric thin-films of different thicknesses, and with different indices of refraction. Our starting point, as always, will be the Maxwell Equations coupled with their boundary conditions, after which it will be straightforward to arrive at expressions for our transfer matrices. The main purpose of this method is to eventually produce reflection and transmission plots for different layers of materials, and to model systems such as anti-reflecting coatings, among others. Next, we will use these tools to develop a program written in Python to verify our results and see how they compare with similar experiments performed in real-life.

\section{Electromagnetic Plane-Waves in Dielectric Media}
Let us begin with Maxwell's equations in their most general form:
\begin{equation}
\nabla \cdot \mathbf{D} = \rho_{f}
\end{equation}
\begin{equation}
\nabla \cdot \mathbf{B} = 0
\end{equation}
\begin{equation}\label{faraday}
\nabla \times \mathbf{E} = - \frac{\partial \mathbf{B}}{\partial t}
\end{equation}
\begin{equation}\label{ampere}
\nabla \times \mathbf{H} =  \frac{\partial \mathbf{D}}{\partial t} + \mathbf{J}_{f}
\end{equation}
\noindent
where $\rho_{f}$ and $\mathbf{J}_{f}$ are the free charge and current densities, respectively. In the next section we shall closely follow much of the conventions used in [\textbf{1}] chapter 7. In the absence of any sources, these two quantities vanish; since we are only considering wave propagation through dielectric media, we shall explicitly omit them for the remainder of our calculations. Furthermore, for isotropic linear media, we have the  relations: \\
\begin{equation}
\mathbf{D} = \epsilon \mathbf{E}
\end{equation}
\begin{equation}
\mathbf{H} = \frac{1}{\mu} \mathbf{B}
\end{equation}
\noindent
where $\epsilon = \epsilon_{0} (1 + \chi_{e})$ and $\mu = \mu_{0} (1 + \chi_{m})$ are the permittivity and permeability of the material. Next, let us assume an ansatz solution with a harmonic time dependence $e^{- i \omega t}$ and substitute this along with our previous assumptions into Eq.(\ref{faraday}) and Eq.(\ref{ampere}) to obtain:
\begin{equation}
\nabla \times \mathbf{E} - i \omega \mathbf{B} = 0
\end{equation}
\begin{equation}
\nabla \times \mathbf{B} + i \omega \mu \epsilon \mathbf{E} = 0
\end{equation}
\noindent
To obtain the Helmholtz wave equation, we simply take the divergence of these two equations and combine them to eliminate either $\mathbf{E}$ or $\mathbf{B}$. The result becomes:
\begin{equation}\label{helm1}
(\nabla^{2} + \mu \epsilon \omega^{2}) \mathbf{E} = 0
\end{equation}
\begin{equation}\label{helm2}
(\nabla^{2} + \mu \epsilon \omega^{2}) \mathbf{B} = 0
\end{equation}
\noindent
As usual, we assume plane-wave solutions of the form $e^{i( k \mathbf{n} \cdot \mathbf{r} - \omega t )}$ for some wave-vector $\mathbf{k} = k \mathbf{n}$, where $\mathbf{n}$ is a unit vector pointing in the direction of propagation. Substituting this plane-wave into Eq.(\ref{helm1}) results in the condition:
\begin{equation}
k^{2} = \mu \epsilon \omega^{2}
\end{equation}
\noindent
which is known as the dispersion relation. From the definition of the index of refraction $n = c / v$ ($n$ is not to be confused with the unit vector $\mathbf{n}$), we can derive an expression for the index of refraction that is entirely dependent on the properties of the material through which our wave is propagating:
\begin{equation}\label{index}
n = \sqrt{ \frac{ \mu \epsilon }{ \mu_{0} \epsilon_{0} } }
\end{equation}
\noindent
With all this in mind, the plane-wave solutions to Eq.(\ref{helm1}) and Eq.(\ref{helm2}) yield:
\begin{equation}
\mathbf{E}(\mathbf{r}, t) = \mathbf{E}_{0} e^{i( k \mathbf{n} \cdot \mathbf{r} - \omega t )}
\end{equation}
\begin{equation}
\mathbf{B}(\mathbf{r}, t) = \mathbf{B}_{0} e^{i( k \mathbf{n} \cdot \mathbf{r} - \omega t )}
\end{equation}
\noindent
Here the amplitudes $\mathbf{E}_{0}$ and $\mathbf{B}_{0}$ can be, in general complex; the real, physical fields are given by taking the real component of the above equations. One matter should be clarified at this point. We note that $\mathbf{n} = \mathbf{n}_{R} + i \mathbf{n}_{I}$, which demonstrates that $\mathbf{n}$ need not be a real vector. When this is taken into account, we obtain the additional term $e^{-k \mathbf{n_{I}} \cdot \mathbf{r}}$ in our plane-wave solutions, clearly exhibiting exponential growth or decay. Alternatively, this outcome can also be achieved by defining a complex index of refraction $n = \alpha + i \beta$ instead. In either case, these two situations are related to each other through their dispersion relation. For this paper we shall explicitly assume a complex index of refraction with a positive imaginary coefficient $\beta$, otherwise known as the extinction coefficient. From the additional term $e^{- \beta \mathbf{k} \cdot \mathbf{r} }$ it becomes apparent that this condition excludes exponential growth (i.e. stimulated emission) from consideration. \\

Two important consequences of the Maxwell equations allow us to simplify matters even furthers. From Gauss' laws for magnetic and electric fields, we see that:
\begin{equation}
\mathbf{n} \cdot \mathbf{E}_{0} = \mathbf{n} \cdot \mathbf{B}_{0} = 0
\end{equation}
\noindent
which simply states that electromagnetic waves are transverse in nature. Next, we can conclude from Faraday's law that the magnetic field can be formulated as:
\begin{equation}\label{B}
\mathbf{B}_{0} = \sqrt{\mu \epsilon} \mathbf{n} \times \mathbf{E}_{0}
\end{equation}
\noindent
Put together, these two equations demonstrate that the magnetic and electric fields are in phase and orthogonal to one another. It is for this reason that we can ignore $\mathbf{B}$ and focus solely on $\mathbf{E}$ for our calculations; Eq.(\ref{B}) provides the connection between the two. \\

\section{Reflection and Transmission at Dielectric Interfaces}

\begin{figure}[h]
\centering
\includegraphics[width=.75\linewidth]{1}
\caption{Propagation of a wave into a medium with a different index of refraction, which gives rise to reflected and refracted waves. The y-axis points into the page for this geometry [\textbf{1}].}
\label{fig:lat}
\end{figure}
Now that we have defined some of the more trivial aspects of our problem, let us derive the Fresnel equations for the situation at hand. Figure 1 demonstrates the geometry we shall be working with; a wave with wave-vector $\mathbf{k}$ is incident on a surface located at $z = 0$ at an angle $\theta_{i}$, which gives rise to a reflected wave $\mathbf{k}''$ at an angle $\theta_{r'}$ and a refracted wave $\mathbf{k}'$ at an angle $\theta_{r}$. Furthermore, we shall restrict our attention to two special cases of linearly polarized light. In the first of these, the electric field oscillates in a plane that is parallel to the plane of incidence, defined as the plane formed by all three wave rays (i.e. the z-x plane in the figure above). Conversely, the magnetic field oscillates in a plane that is orthogonal to the plane of incidence, as can be seen from Eq.(\ref{B}); such a wave is known as a Transverse Magnetic (TM) wave, or p-wave. When the electric field oscillates in a plane that is orthogonal to the plane of incidence, this wave is called a Transverse Electric (TE) wave, or s-wave. Note that we are intentionally leaving out more complicated behavior such as elliptically polarized light for the sake of brevity. \\

Using the superposition principle, we can imagine each of the two regions $\mu \epsilon$ and $\mu' \epsilon'$ as being composed of a combination of forward-moving and backward-moving waves. In the $\mu \epsilon$ region we have the following two waves:
\begin{equation}
\mathbf{E}(\mathbf{r}, t) = \mathbf{E}_{0} e^{i (\mathbf{k} \cdot \mathbf{r} - \omega t)}
\end{equation}
\begin{equation}
\mathbf{E}''(\mathbf{r}, t) = \mathbf{E}_{0}'' e^{i (\mathbf{k}'' \cdot \mathbf{r} - \omega t)}
\end{equation}
\noindent
whereas in the $\mu' \epsilon'$ region we have:
\begin{equation}
\mathbf{E}'(\mathbf{r}, t) = \mathbf{E}_{0}' e^{i (\mathbf{k}' \cdot \mathbf{r} - \omega t)}
\end{equation}
\noindent
Joining all three equations at the interface then gives:
\begin{equation}\label{superp}
\mathbf{E}_{0} e^{i (\mathbf{k} \cdot \mathbf{r} - \omega t)} + \mathbf{E}_{0}'' e^{i (\mathbf{k}'' \cdot \mathbf{r} - \omega t)} = \mathbf{E}_{0}' e^{i (\mathbf{k}' \cdot \mathbf{r} - \omega t)}
\end{equation}
\noindent
Since the frequency is decided at the source, it is the same for all three waves. This leads to the three relations:
\begin{equation}\label{k1}
|\mathbf{k}| = |\mathbf{k}''| = \omega \sqrt{\mu \epsilon}
\end{equation}
\begin{equation}\label{k2}
|\mathbf{k}'| = \omega \sqrt{\mu' \epsilon'}
\end{equation}
\noindent
From Eq.(\ref{superp}) and from the fact that the boundary conditions must hold for all $\mathbf{r}$ and $t$, we see that the arguments of the exponents must all be equal at $z = 0$. Notice how this is happens because the time-dependence is confined to a $e^{-i \omega t}$ harmonic. And so, at $z = 0$ we have the following conditions:
\begin{equation}
\mathbf{k} \cdot \mathbf{r} =  \mathbf{k}'' \cdot \mathbf{r} = \mathbf{k}' \cdot \mathbf{r}
\end{equation}
\noindent
This implies:
\begin{equation}
|k||r| \cos \theta_{i} = |k''||r| \cos \theta_{r'} \hspace{10mm} \rightarrow \hspace{10mm}\theta_{i} = \theta_{r'}
\end{equation}
\noindent
We have thus recovered the law of reflection, demonstrating the fact that all three wave rays lay in the plane of incidence. We may now eliminate the y-components of the scalar products given above and keep only the $k_{x}$ components so that $k_{x} = k_{x}'' = k_{x}'$. This gives:
\begin{equation}
k \sin \theta_{i} = k'' \sin \theta_{r'} = k' \sin \theta_{r}
\end{equation}
\noindent
so that:
\begin{equation}\label{snell}
k \sin \theta_{i} = k' \sin \theta_{r}
\end{equation}
\noindent
which is a re-statement of Snell's law, as can be seen by using Eq.(\ref{index}), Eq.(\ref{k1}), and Eq.(\ref{k2}). Together, the laws of reflection and refraction dictate the kinematic properties of wave propagation. The \textit{dynamic} properties are dictated by the interface, which we will now examine. The standard boundary conditions of the Maxwell equations state:
\begin{equation}
[ \epsilon (\mathbf{E}_{0} + \mathbf{E}_{0}'') - \epsilon' \mathbf{E}_{0}' ] \cdot \mathbf{n} = 0
\end{equation}
\begin{equation}
[ \mathbf{k} \times \mathbf{E}_{0} + \mathbf{k}'' \times \mathbf{E}_{0}'' - \mathbf{k}' \times \mathbf{E}_{0}' ] \cdot \mathbf{n} = 0
\end{equation}
\begin{equation}
(\mathbf{E}_{0} + \mathbf{E}_{0}'' - \mathbf{E}_{0}' ) \times \mathbf{n} = 0
\end{equation}
\begin{equation}
\left[ \frac{1}{\mu} ( \mathbf{k} \times \mathbf{E}_{0} + \mathbf{k}'' \times \mathbf{E}_{0}'' ) - \frac{1}{\mu'} ( \mathbf{k}' \times \mathbf{E}_{0}') \right] \times \mathbf{n} = 0
\end{equation}
\noindent
Notice how we have eliminated $\mathbf{B}$ through Eq.(\ref{B}). The first two of these equations state that the components of the electric and magnetic fields that are orthogonal to the interface experience a discontinuity and continuity at the interface, respectively; the latter two state that the parallel components of the electric and magnetic fields are continuous and discontinuous at the interface instead. Lets begin applying these boundary conditions to the case of a TM wave. The first condition yields:
\begin{equation}
\epsilon (E_{0} \sin \theta_{i} + E_{0}'' \sin \theta_{r'} ) = \epsilon' E_{0}' \sin \theta_{r}
\end{equation}
\noindent
Using Snell's law and the law of reflection then gives:
\begin{equation}
\sqrt{ \frac{\mu' \epsilon'}{\mu \epsilon} } (E_{0} + E_{0}'' ) = \frac{\epsilon'}{\epsilon} E_{o}'
\end{equation}
\noindent
This can be rearranged to obtain:
\begin{equation}\label{bc1}
\sqrt{ \frac{\epsilon}{\mu} } (E_{0} + E_{0}'' ) = \frac{\epsilon'}{\mu'} E_{o}'
\end{equation}
\noindent
The second boundary condition gives nothing since there are no components of the magnetic fields that are perpendicular to the interface. The third boundary condition imposes restrictions on the parallel components of the electric field and so:
\begin{equation}
-E_{0} \cos \theta_{i} + E_{0}'' \cos \theta_{r'} = -E_{0}' \cos \theta_{r}
\end{equation}
\noindent
Once again, the law of reflection simplifies this to:
\begin{equation}\label{bc2}
( E_{0} - E_{0}'' ) \cos \theta_{i} = E_{0}' \cos \theta_{r}
\end{equation}
\noindent
Lastly, the fourth boundary condition merely reproduces Eq.(\ref{bc1}). We now proceed to solve Eq.(\ref{bc1}) and Eq.(\ref{bc2}) for the ratios $E_{0}'' / E_{0}$ and $E_{0}' / E_{0}$, otherwise known as the reflection and transmission coefficients [\textbf{2}]:
\begin{equation}
r \equiv \frac{E_{0}''}{E_{0}} = \frac{ \frac{\mu}{\mu'} n'^{2} \cos \theta_{i} - n \sqrt{ n'^{2} - n^{2} \sin^{2} \theta_{i}}} {\frac{\mu}{\mu'} n'^{2} \cos \theta_{i} + n \sqrt{ n'^{2} - n^{2} \sin^{2} \theta_{i}}}
\end{equation}
\begin{equation}
t \equiv \frac{E_{0}'}{E_{0}} = \frac{2 n n' \cos \theta_{i}}{\frac{\mu}{\mu'} n'^{2} \cos \theta_{i} + n \sqrt{ n'^{2} - n^{2} \sin^{2} \theta_{i}}}
\end{equation}
\noindent
In this paper we will only consider non-magnetic materials for which $\mu = \mu_{0}$ which immediately yields $\mu / \mu' = 1$. Furthermore, the relation $n \sqrt{ n'^{2} - n^{2} \sin^{2} \theta_{i}} = n n' \cos \theta_{r}$ allows us to simplify these expressions to:
\begin{equation}\label{fresnel1}
r_{p} = \frac{ n_{f} \cos \theta_{i} - n_{i} \cos \theta_{f} }{ n_{f} \cos \theta_{i} + n_{i} \cos \theta_{f}}
\end{equation}
\begin{equation}
t_{p} = \frac{ 2 n_{i} \cos \theta_{i} }{ n_{f} \cos \theta_{i} + n_{i} \cos \theta_{f}}
\end{equation}
\noindent
In these expression we have re-labeled $n, n' \rightarrow n_{i}, n_{f}$ and $\theta_{i}, \theta_{r} \rightarrow \theta_{i}, \theta_{r}$ for clarity. The $p$ subscript denotes a p-wave (i.e. a TM wave). Similar calculations for an s-wave give the following relations:
\begin{equation}
r_{s} = \frac{ n_{i} \cos \theta_{i} - n_{f} \cos \theta_{f} }{ n_{i} \cos \theta_{i} + n_{f} \cos \theta_{f}}
\end{equation}
\begin{equation}\label{fresnel2}
t_{s} = \frac{ 2 n_{i} \cos \theta_{i} }{ n_{i} \cos \theta_{i} + n_{f} \cos \theta_{f}}
\end{equation}
\noindent
Equations (\ref{fresnel1} - \ref{fresnel2}) constitute the core of our theory, and we shall use them to derive the behavior of light as it propagates through multiple interfaces.

\section{The Transfer Matrix Method}

Having derived the Fresnel equations, our final step is to generalize them for the case where multiple dielectric interfaces are present. We imagine a scenario similar to Figure 1 but with $N$ layers of different thicknesses $d_{j}$ and indices of refraction $n_{j}$ stacked along the z-axis so that they lay parallel to the y-x plane. This means that the dielectric layers can be organized as follows:
\begin{align}\label{layers}
n(z) =& n_{0}, \hspace{10mm} z < z_{0}                  \nonumber \\
	  & n_{1}, \hspace{10mm} z_{0} < z < z_{1}           \nonumber \\
 	  & n_{2}, \hspace{10mm} z_{1} < z < z_{2}            \nonumber \\
 	  & \vdots                                             \nonumber \\
 	  & n_{N}, \hspace{10mm} z_{N - 1} < z                  \nonumber \\
\end{align}
\noindent
From this geometry, it is easy to see that the z-component of $\mathbf{k}$ in each layer becomes $k_{z j} = (2 \pi n_{j} / \lambda_{0}) \cos \theta_{j}$ since, as before, the wave rays are confined to the plane of incidence. Let us now consider the total electric field of a TE (s-wave) within each individual layer, as s-waves tend to satisfy simpler boundary conditions. The superposition principle allows us to once again decompose $\mathbf{E}(\mathbf{r}, t)$ into different components; one moving forward in the positive z-direction, and another moving backwards in the -z-direction [\textbf{3}][\textbf{4}]:
\begin{align}
E(\mathbf{r}, t)  &=  E_{F} e^{ i ( \mathbf{k}_{F} \cdot \mathbf{r} - \omega t) } + E_{B} e^{ i ( \mathbf{k}_{B} \cdot \mathbf{r} - \omega t) }          \nonumber \\
   	              &= E_{F} e^{ i ( k_{x} x + k_{zj} z - \omega t) } + E_{B} e^{ i ( k_{x} x - k_{zj} z - \omega t) }           \nonumber \\
             	  &=  E_{F} e^{ i k_{zj} z } + E_{B} e^{ -i  k_{zj} z }
\end{align}
\noindent
In the last line we absorbed $e^{ i k_{x} x }$ into the complex amplitudes and dropped the time dependence for clarity. We can recast this equation into a more illuminating form if we consider how the state of the light beam changes as it interacts with each layer $j$:
\begin{align}\label{system2}
E_{F}' = E_{F} e^{ i k_{zj} z }  \nonumber \\
E_{B}' = E_{B} e^{ -i  k_{zj} z }
\end{align}
\noindent
Interestingly enough, this resembles a two-state matrix equation of the form:
\begin{equation}
E' = T_{j} E
\end{equation}
\noindent
for some matrix $T_{j}$. Thus, as a wave propagates within a material of thickness $d_{j}$, the corresponding propagation matrix can be obtained through inspection of Eqs.(\ref{system2}):
\begin{equation}\label{Tj}
T_{j} =
\begin{pmatrix}
	\exp i k_{zj}d_{j} & 0 \\
	0 & \exp - i k_{zj}d_{j}
\end{pmatrix}  =
\begin{pmatrix}
	\exp i \Phi_{j} & 0 \\
	0 & \exp - i \Phi_{j}
\end{pmatrix}
\end{equation}
\noindent
where we have defined the phase angle as $\Phi_{j} = k_{zj}d_{j}$. As helpful as the $T_{j}$ matrix is, it does not provide the entire picture. We must now investigate how the state of a light wave changes as it passes \textit{through} an interface. The difference can be subtle, so the reader is advised to give is some thought. As was the case for wave reflection and refraction, boundary conditions will once again dictate the dynamics of our system. Since we are dealing with s-waves, the relevant boundary conditions are those for tangential $\mathbf{E}$ and $\mathbf{B}$, which we already acquired in the previous section:
\begin{equation}
E_{Fi} + E_{Bi} = E_{Fj} + E_{Bj} \nonumber
\end{equation}
\begin{equation}
n_{i} \cos \theta_{i} (E_{Fi} - E_{Bi}) = n_{j} \cos \theta_{j} (E_{Fj} - E_{Bj})
\end{equation}
\noindent
The difference in the indices between the left-hand and right-hand sides explicitly demonstrates the propagation of light through an interface. In matrix form these equations read:
\begin{equation}
\begin{pmatrix}
		1 & 1 \\
	n_{i} \cos \theta_{i} & - n_{i} \cos \theta_{i}
\end{pmatrix}
\begin{pmatrix}
	E_{Fi}  \\
    E_{Bi}
\end{pmatrix} =
\begin{pmatrix}
	1 & 1 \\
	n_{j} \cos \theta_{j} & - n_{j} \cos \theta_{j}
\end{pmatrix}
\begin{pmatrix}
	E_{Fj}  \\
	E_{Bj}
\end{pmatrix}
\end{equation}
\noindent
Rearranging this expression yields:
\begin{equation}
\begin{pmatrix}
		E_{Fj}  \\
		E_{Bj}
\end{pmatrix} =
\begin{pmatrix}
		1 & 1 \\
		n_{j} \cos \theta_{j} & - n_{j} \cos \theta_{j}
\end{pmatrix}^{-1}
\begin{pmatrix}
	1 & 1 \\
	n_{i} \cos \theta_{i} & - n_{i} \cos \theta_{i}
\end{pmatrix}
\begin{pmatrix}
		E_{Fi}  \\
		E_{Bi}
\end{pmatrix} \nonumber
\end{equation}
\begin{equation}
\begin{pmatrix}
		E_{Fj}  \\
		E_{Bj}
\end{pmatrix} =
	 T_{j}^{-1} T_{i}
\begin{pmatrix}
		E_{Fi}  \\
		E_{Bi}
\end{pmatrix} =
    T_{ji}
\begin{pmatrix}
	E_{Fi}  \\
	E_{Bi}
\end{pmatrix}
\end{equation}
\noindent
The transfer matrix $T_{ji}$ is now trivial to find. The end result is:
\begin{equation}\label{Tji}
T_{ji} = \frac{1}{t_{ji}}
\begin{pmatrix}
		1 & r_{ji} \\
		r_{ji} & 1
\end{pmatrix}
\end{equation}
\noindent
Evidently, the elements of $T_{ji}$ are nothing but the Fresnel coefficients we calculated in the previous section. Combining this interface matrix with the layer-propagation matrix Eq.(\ref{Tj}) gives the \textit{total} transfer matrix of our system $T$. For an arbitrary amount of layers, $T$ is obtained through ordinary matrix multiplication of each individual element:
\begin{equation}
T = T_{N(N-1)} T_{N-1} \hdots T_{32} \hspace{1mm} T_{2} \hspace{1mm} T_{21} \hspace{1mm} T_{1} \hspace{1mm} T_{10}
\end{equation}
\noindent
The order of these matrices is to be noted, as this configuration gives the total output from some initial input. For example, reversing this order result in the converse, which is to say:
\begin{equation}
\begin{pmatrix}
	E_{F}(z_{0})  \\
	E_{B}(z_{0})
\end{pmatrix} =
\begin{pmatrix}
	T_{11} & T_{12} \\
	T_{21} & T_{22}
\end{pmatrix}
\begin{pmatrix}
	E_{F}(z_{N - 1})  \\
	0
\end{pmatrix} \nonumber
\end{equation}
\noindent
Here we are relating the electric fields in the first layer to those in the last layer through a reversed $T$ matrix. Furthermore, we are also assuming that there are no incoming waves from the $N - 1$ layer so that the backwards-traveling wave is zero. We can solve this system of equations for the ratios of reflected-to-incident and transmitted-to-incident amplitudes; these are merely the reflecting and transmission amplitudes:
\begin{equation}
r = \frac{T_{21}}{T_{11}}  \hspace{20mm}   t = \frac{1}{T_{11}}
\end{equation}
\noindent
Our quantities of interest are the reflected and transmitted power, which are easily obtained from the Poynting vector. For convenience we simply state these well-know results [\textbf{1}]:
\begin{equation}\label{R}
R = |r|^{2}
\end{equation}
\begin{equation}
T_{p} = |t|^{2} \frac{  \mathrm{Re} ( n_{f} \cos \theta_{f}^{*} )  }{ \mathrm{Re} ( n_{i} \cos \theta_{i}^{*}  ) }
\end{equation}
\begin{equation}\label{T}
T_{s} = |t|^{2} \frac{  \mathrm{Re} ( n_{f} \cos \theta_{f} )  }{ \mathrm{Re} ( n_{i} \cos \theta_{i} ) }
\end{equation}
\noindent
The reflected power is apparently independent of polarization. Now that we have all the tools required, we can finally make progress and perform some trial computations.

\section{Numerical Implementations of the TMM in Python}

To put all the tools we developed in the previous sections, we carry out a computation in Python that showcases some of the more significant aspects of the TMM. The following code was adapted from Byrnes module on the TMM [\textbf{5}]. We start by defining a function that distinguishes whether a given wave is traveling forward or backwards in a given medium of complex reflectivity. To achieve this, we perform two \textbf{assert} statements that ensure that both parts of the index of refraction $n$ are positive. As we discussed before, a negative extinction coefficient would lead to exponential growth which we are avoiding. Now, the wave-number in a given layer $k_{z j} = (2 \pi n_{j} / \lambda_{0}) \cos \theta_{j}$ generally points in the direction of propagation in isotropic media, so we can use it to further refine the forward direction criteria. We thus create an \textbf{if} conditional that returns true if the extinction coefficient is above a flotation point value and false otherwise. Since decay points if the forward direction, a positive extinction coefficient is a good metric for this problem:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def theta_forward(n, theta):

	assert n.real >= 0
	assert n.imag >= 0

	n_cos_theta = n * np.cos(theta)
	if np.absolute(n_cos_theta.imag) > 100 * EPSILON:
		value = (n_cos_theta.imag > 0)
	else:
		value = (n_cos_theta.real > 0)

return value
\end{lstlisting}
\noindent
We then create a simple function that returns a refracted angle from an incident angle based on Snell's Law Eq.(\ref{snell}) and uses the previous function for forward propagation. If the angle fails to point in the forward direction, this function returns a backwards angle which is simply $(\pi - \theta)$:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def snells_law(n_1, n_2, theta_1):

	theta_2 = arcsin(n_1 * np.sin(theta_1) / n_2)
	if theta_forward(n_2, theta_2):
		return theta_2
	else:
		return np.pi - theta_2
\end{lstlisting}
\noindent
Moving onward, the next function defines a list of refracted angles within the dielectric structure based on the incident angle on the first layer. The \textbf{if not} conditionals assure that the wave propagates in the forward direction for the first layer \textbf{n\_list[0]} and for the last layer \textbf{n\_list[-1]} as well:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def snells_law_list(n_list, theta_incident):

	theta_refracted_list = arcsin(n_list[0]
				* np.sin(theta_incident) / n_list)
	if not theta_forward(n_list[0], theta_refracted_list[0]):
		theta_refracted_list[0] = np.pi - theta_refracted_list[0]
	if not theta_forward(n_list[-1], theta_refracted_list[-1]):
		theta_refracted_list[-1] = np.pi - theta_refracted_list[-1]
	return theta_refracted_list
\end{lstlisting}
\noindent
Now comes the part where we define the reflection amplitudes for s-waves and p-waves directly from the Fresnel equations Eqs.(\ref{fresnel1} - \ref{fresnel2}). The \textbf{if} loops take care of both situations:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def reflection_amplitude(polarization, n_i, n_f, theta_i, theta_f):

	if polarization == 's':
		return ((n_i * np.cos(theta_i) - n_f * np.cos(theta_f)) /
			(n_i * np.cos(theta_i) + n_f * np.cos(theta_f)))
	elif polarization == 'p':
		return ((n_f * np.cos(theta_i) - n_i * np.cos(theta_f)) /
			(n_f * np.cos(theta_i) + n_i * np.cos(theta_f)))
	else:
		raise ValueError("Polarization must be 's' or 'p'")

def amplitude_r(polarization, n_i, n_f, theta_i, theta_f):

	r = reflection_amplitude(polarization, n_i, n_f, theta_i, theta_f)
	return reflected_power(r)
\end{lstlisting}
\noindent
Similarly, the transmission amplitudes can be computed for s-waves and p-waves as well. The \textbf{amplitude\_r} and \textbf{amplitude\_t} functions serve to return values that can be fed into the computation of the reflected and transmitted power:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def transmission_amplitude(polarization, n_i, n_f, theta_i, theta_f):

	if polarization == 's':
		return ((2 * n_i * np.cos(theta_i)) /
			(n_i * np.cos(theta_i) + n_f * np.cos(theta_f)))
	elif polarization == 'p':
		return ((2 * n_i * np.cos(theta_i)) /
			(n_f * np.cos(theta_i) + n_i * np.cos(theta_f)))
	else:
		raise ValueError("Polarization must be 's' or 'p'")

def amplitude_t(polarization, n_i, n_f, theta_i, theta_f):

	t = transmission_amplitude(polarization, n_i, n_f, theta_i, theta_f)
	return transmitted_power(polarization, t, n_i, n_f, theta_i, theta_f)
\end{lstlisting}
\noindent
It is now straightforward to define functions that return the \textbf{reflected\_power} and the \textbf{transmitted\_power} functions based on Eqs.(\ref{R} - \ref{T}):
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def reflected_power(r):

	return np.absolute(r)**2

def transmitted_power(polarization, t, n_i, n_f, theta_i, theta_f):

	if polarization == 's':
		return np.absolute(t**2) * (((n_f * np.cos(theta_f)).real)
			/ (n_i * np.cos(theta_i)).real)
	elif polarization == 'p':
		return np.absolute(t**2)
			*(((n_f * np.conj(np.cos(theta_f))).real)
			/ (n_i * np.conj(np.cos(theta_i))).real)
	else:
		raise ValueError("Polarization must be 's' or 'p'")
\end{lstlisting}
\noindent
Lastly we define a 2 x 2 \textbf{matrix\_array} function which we will use as a template for our transfer matrices:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def matrix_array(a, b, c, d, dtype=float):

	template_array = np.empty((2,2), dtype=dtype)
	template_array[0,0] = a
	template_array[0,1] = b
	template_array[1,0] = c
	template_array[1,1] = d
	return template_array
\end{lstlisting}
\noindent
At long last, we can piece all the these functions together to build our TMM algorithm. Lets discuss some of its important features. The first block of code defines a series of tests meant to guide a user's inputs. For example, the first \textbf{if} conditional safeguards against the input of more than one incident wavelength or angle, while the second \textbf{if} conditional ensures that the dimensionality of the \textbf{n\_list} and \textbf{d\_list} arrays does not exceed one. Furthermore, it also returns an error if we have more indices of refraction values than dielectric layers and vice-versa. Next come a series of \textbf{assert} statements that verify that the first layer \textbf{d\_list[0]} and the second layer \textbf{d\_list[-1]} are of infinite thickness, and that our incident angle/index of refraction describe physical values:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
def transfer_matrix_method(polarization, n_list, d_list, theta_incident,
	wavelength):

	n_list = np.array(n_list)

	d_list = np.array(d_list, dtype=float)

	if ((hasattr(wavelength, 'size') and wavelength.size > 1)
		or (hasattr(theta_incident,'size') and theta_incident.size>1)):
	raise ValueError("Only 1 wavelength.......")
	if ((n_list.ndim != 1)
		or (d_list.ndim != 1)
		or (n_list.size != d_list.size)):
	raise ValueError("Problem with n_list or d_list!")

	assert d_list[0] == d_list[-1] == np.inf
	assert np.absolute((n_list[0]*np.sin(theta_incident)).imag)<100*EPSILON
	assert theta_forward(n_list[0], theta_incident)
\end{lstlisting}
\noindent
Once all of these checks have passed, we move on to the implementation of the TMM. The first four lines recast our relevant parameters as lists with values for each layer. Afterwards, we create arrays for our reflection and transmission amplitudes \textbf{r\_list} and \textbf{t\_list} and loop then through all layers. Finally, we create two matrix lists \textbf{Transfer1\_list} and \textbf{Transfer2\_list} which are precisely the $T_{j}$ and $T_{ji}$ matrices we defined in Eq.(\ref{Tj}) and Eq.(\ref{Tji}):
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
        num_layers = n_list.size

	theta_list = snells_law_list(n_list, theta_incident)

	k_list = 2 * np.pi * n_list * np.cos(theta_list) / wavelength

	phase_angle = k_list * d_list

	r_list = np.zeros((num_layers, num_layers), dtype=complex)
	t_list = np.zeros((num_layers, num_layers), dtype=complex)
	for i in range(num_layers-1):
		r_list[i,i+1] = reflection_amplitude(polarization, n_list[i],
   			        n_list[i+1],theta_list[i], theta_list[i+1])
		t_list[i,i+1]=transmission_amplitude(polarization, n_list[i],
		                n_list[i+1],theta_list[i], theta_list[i+1])

	Transfer1_list = np.zeros((num_layers, 2, 2), dtype=complex)
	for i in range(1, num_layers-1):
		Transfer1_list[i] = (1/t_list[i,i+1]) * np.dot(
			matrix_array(np.exp(-1j * phase_angle[i]),
			     0, 0, np.exp(1j * phase_angle[i]),dtype=complex),
			matrix_array(1, r_list[i,i+1], r_list[i,i+1],
			     1, dtype=complex))

	Transfer2_list = matrix_array(1, 0, 0, 1, dtype=complex)
	for i in range(1, num_layers-1):
		Transfer2_list = np.dot(Transfer2_list, Transfer1_list[i])

	Transfer2_list = np.dot(matrix_array(1,r_list[0,1], r_list[0,1], 1,
		dtype=complex)/t_list[0,1], Transfer2_list)
\end{lstlisting}
\noindent
Finally, we construct the reflection and transmission amplitudes/powers from our matrices and then end the algorithm by returning values for all variables. An extra function that is included is the \textbf{unpolarized} function which, as its name states, returns the reflected/transmitted powers for unpolarized light by averaging the s-waves with the p-waves:
\lstset{showspaces=false,showstringspaces=false}%
\begin{lstlisting}
	r = Transfer2_list[1,0]/Transfer2_list[0,0]
	t = 1/Transfer2_list[0,0]

	R = reflected_power(r)
	T = transmitted_power(polarization, t,
		n_list[0], n_list[-1], theta_incident, theta_list[-1])

	return {'r': r, 't': t, 'R': R, 'T': T,
	'k_list': k_list, 'theta_list': theta_list,
	'polarization': polarization, 'n_list': n_list, 'd_list': d_list,
	'theta_incident': theta_incident, 'wavelenght': wavelength}

def unpolarized(n_list, d_list, theta_incident, wavelength):

	s_data = transfer_matrix_method('s', n_list, d_list,
	 theta_incident, wavelength)
	p_data = transfer_matrix_method('p', n_list, d_list,
	 theta_incident, wavelength)
	R = (s_data['R'] + p_data['R']) / 2.
	T = (s_data['T'] + p_data['T']) / 2.
	return {'R': R, 'T': T}
\end{lstlisting}

\newpage

\section{Conclusion}

The transfer matrix method outlined in this paper is a deceptively simple yet versatile instrument that has proven to be indispensable in a diverse array of scientific fields. Although we focused on a narrow range of its applications, we nevertheless arrived at conclusions that seemed to be acquired almost out of sheer luck. Indeed it is quite surprising that this method works at all, given the complicated interactions that take place between light and matter. Please refer to the accompanying Jupyter Notebook for a more detailed overview of some of the computations that the TMM can perform.

\begin{thebibliography}{1}

\bibitem{1} J. Jackson, \textit{Classical Electrodynamics} (John Wiley \& Sons, Inc., 1999).

\bibitem{2} E. Hecht, \textit{Optics} (Pearson, 2002).

\bibitem{3} M. Born, and E. Wolf, \textit{Priciples of Optics} (Pergamon Press, 1986).

\bibitem{4} M. Claudia Troparevsky, Adrian S. Sabau, Andrew R. Lupini, and Zhenyu Zhang, "Transfer-matrix formalism for the calculation of optical response in multilayer systems: from coherent to incoherent interference," Opt. Express 18, 24715-24721 (2010)

\bibitem{5} https://pypi.org/project/tmm/

\end{thebibliography}

\end{document}

Tranfer Matrix Method Files/Final Writeup/Term Paper.tex


\section{Results}

\subsection{Data description after QC}
\label{sec:agv_data_description}
Related and unrelated male and female samples from 3 different African populations (table \ref{tab:short_summary}) were genotyped using the Illumina HumanOmni2.5 genotype chip array and sequenced on an Illumina HiSeq 2000 to different depths of coverage (table \ref{tab:samples}) and subsequently downsampled to a similar coverage of 4x (figure \ref{fig:coverage}). The \gls{QC} of the samples genotyped on the Illumina HumanOmni2.5-4 and 2.5-8 platforms is described elsewhere\cite{Gurdasani2015} and on page \pageref{subsec:chipQC}.
\input{tab/samples}
\input{fig/coverage}
Each of the two methods for obtaining genotypes yields a different number of non-monomorphic autosomal SNPs. Approximately 20\% of the SNPs from the Omni2.5 chip are monomorphic (table \ref{tab:chip_SNPs}).
\input{tab/chip_SNPs}
The majority of the sequenced \glspl{SNP} are already present in phase 1 of \gls{1000G}\cite{1000G2012}, but novel SNPs are present in all three populations (figure \ref{fig:intersection}).

\input{fig/intersection}

%\subsubsection{Comparison of variant calling and refinement methods}
%\subsubsection{Comparison of variant calling software}
%\subsubsection{Comparison of refinement software and reference panels}

\subsection{Comparison of whole genome sequencing and SNP array designs}
\label{subsec:result_downsampling_sensspec}
The sensitivity and specificity was evaluated for each of the study designs as described on page \pageref{subsec:sensspec}. The sensitivity of all study designs was greater than 0.95 for common variants (\gls{MAF} greater than 0.05) with respect to 8x for Ethiopians (figure \ref{fig:SN12f3}) and 4x sequence data for all populations (\ref{fig:SN12f4}). For variants with a \gls{MAF} less than 0.05 the sensitivity was lower. The sensitivity for singletons and novel variants relative to 1000G phase 1 was even lower, which is expected, since very few reads will support singletons. The sensitivity of the imputed SNP array data (Omni2.5) is comparable to that of the 0.5x coverage design. However, the disadvantage of the \gls{SNP} array design is the inability to acquire genotypes of novel variants, but this will ineviatiably become less relevant as the size and diversity of reference panels for imputation grow. All designs had high specificity relative to the higher coverage reference sequence data (8x figure \ref{fig:SN12f3} and 4x figure \ref{fig:SN12f4}), which indicates a low \gls{FPR} or identical false positives in the reference data.

\input{fig/SN12f3}
\input{fig/SN12f4}

\subsection{Data description after downsampling}
\label{subsec:result_downsampling}

As the coverage drops the ability to call rare \glspl{SNP} decreases (figure \ref{fig:downsampling_SNP_count} and table\ref{tab:downsampling_omni_intersection}). At lower coverage the genotype correlation between the calls from the SNP array and sequencing platforms is also reduced and especially for rare variants (figure \ref{fig:SN12f5}). Despite the correlations not being perfect for lower coverage designs for rare and common variants (figure \ref{fig:SN12f5}) the apparent sample size is still greater due to the greater sample size and in the case of common variants in particular compensates for inaccuracies in identifying true genotypes (figure \ref{fig:SN12f6} on page \pageref{subsec:result_apparent}).

\input{fig/downsampling_SNP_count}
\input{tab/downsampling_omni_intersection}
\input{fig/SN12f5}

\subsection{Apparent sample size}
\label{subsec:result_apparent}

By using the costs in table \ref{tab:costs} and equation \ref{eq:sample_size} and the calculated correlations we calculated the apparent sample size for different study designs at a fixed cost. When not taking into consideration computational costs the ultra low coverage study design is a good choice for capturing common variation at a low cost.

\input{fig/SN12f6} % apparent sample size