\subsection*{Philosophy of Homework}

Homework is the \textit{most} important part of this course.\footnote{In one student's \href{http://www.ratemyprofessors.com/ShowRatings.jsp?tid=1951051}{observation}, I give a \qu{mind-blowing homework} every week.} Success in Statistics and Mathematics courses comes from experience in working with and thinking about the concepts. It's kind of like weightlifting; you have to lift weights to build muscles. My job as an instructor is to provide assistance through your \href{http://en.wikipedia.org/wiki/Zone_of_proximal_development}{zone of proximal development}. With me, you can grow more than you can alone. To this effect, homework problems are color coded \ingreen{green} for easy, \inyellow{yellow} for harder, \inred{red} for challenging and \inpurple{purple} for extra credit. You need to know how to do all the greens by yourself. If you've been to class and took notes, they are a joke. Yellows and reds: feel free to work with others. Only do extra credits if you have already finished the assignment. The \qu{[Optional]} problems are for extra practice --- highly recommended for exam study.


\documentclass[14pt]{article}

\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{tikz}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{enumitem}
\usepackage{relsize}
\usepackage{scrextend}
\usepackage{graphicx}
\usepackage{listings}

\usetikzlibrary{automata,positioning}

%
% Assembly language code listing
%
\lstset{language=[x86masm]Assembler}

%
% Basic Document Settings
%

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\chead{\hmwkClass\ (\hmwkClassInstructor): \hmwkTitle}
\rhead{\firstxmark}
\lfoot{\lastxmark}
\cfoot{\thepage}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

%
% Create Problem Sections
%

\newcommand{\enterProblemHeader}[1]{
    \nobreak\extramarks{}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
}

\newcommand{\exitProblemHeader}[1]{
    \nobreak\extramarks{Problem \arabic{#1} (continued)}{Problem \arabic{#1} continued on next page\ldots}\nobreak{}
    \stepcounter{#1}
    \nobreak\extramarks{Problem \arabic{#1}}{}\nobreak{}
}

\setcounter{secnumdepth}{0}
\newcounter{partCounter}
\newcounter{homeworkProblemCounter}
\setcounter{homeworkProblemCounter}{1}
\nobreak\extramarks{Problem \arabic{homeworkProblemCounter}}{}\nobreak{}

%
% Homework Problem Environment
%
% This environment takes an optional argument. When given, it will adjust the
% problem counter. This is useful for when the problems given for your
% assignment aren't sequential. See the last 3 problems of this template for an
% example.
%
\newenvironment{homeworkProblem}[1][-1]{
    \ifnum#1>0
        \setcounter{homeworkProblemCounter}{#1}
    \fi
    \section{Problem \arabic{homeworkProblemCounter}}
    \setcounter{partCounter}{1}
    \enterProblemHeader{homeworkProblemCounter}
}{
    \exitProblemHeader{homeworkProblemCounter}
}

%
% Homework Details
%   - Title
%   - Due date
%   - Class
%   - Section/Time
%   - Instructor
%   - Author
%

\newcommand{\hmwkClass}{CS 219}
\newcommand{\hmwkTitle}{Homework\ \#10}
\newcommand{\hmwkDueDate}{November 23rd, 2016}
\newcommand{\hmwkDueTime}{4:00pm}
\newcommand{\hmwkClassInstructor}{Dr. Egbert}
\newcommand{\hmwkAuthorName}{Matthew J. Berger}

%
% Title Page
%

\title{
    \vspace{2in}
    \textmd{\textbf{\hmwkClass:\ \hmwkTitle}}\\
    \normalsize\vspace{0.1in}\small{Due\ on\ \hmwkDueDate\ at \hmwkDueTime}\\
    \vspace{0.1in}\large{\textit{\hmwkClassInstructor}}
    \vspace{3in}
}

\author{\textbf{\hmwkAuthorName}}
\date{}

\renewcommand{\part}[1]{\textbf{\large Part \Alph{partCounter}}\stepcounter{partCounter}\\}

%
% Various Helper Commands
%

% Useful for algorithms
\newcommand{\alg}[1]{\textsc{\bfseries \footnotesize #1}}

% Alias for the Solution section header
\newcommand{\solution}{\textbf{\large Solution:}}

% Alias for answers
\newcommand{\answer}{\textbf{\large Answer: }}

\begin{document}

\maketitle

\pagebreak

\begin{homeworkProblem}

1.) Write a program called SUB64 to subtract the 64-bit integer in memory locations 0x0150 and 0x154 from the 64-bit integer in
0x0160 and 0x0164. Store the result in memory location 0x0170 and 0x0174.\\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; SUB64
; Template by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
    mov eax,[0x0150]    ; lower 32 bits of first num
    mov edi,[0x0160]    ; lower 32 bits of second num
    mov edx,[0x0154]    ; lower 32 bits of first num
    mov esi,[0x0164]    ; lower 32 bits of second num
    sub %edi, %eax      ; Subtract lowest 32-bits, borrow reflected in carry flag
    sbb %esi, %edx      ; Subtract highest 32-bits, and the borrow if there was one
    mov [0x170],%esi    ; Copy high bits to memory
    mov [0x174],%edi    ; Copy low bits to memory
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
; N/A
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

2.) Write a program called COMBINE that combines the low-order nibbles of the four bytes in memory locations 0x0150 to 0x0153
into a single 16-bit word. The nibbles should be ordered low-to-high in the result beginning with the data from location 0x0150. Store
the result as 16-bits in memory location 0x0154. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; COMBINE
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BL,[0x0150l] ; Store value at 0x0150 in lower bits of reg BX
		MOV	BH,[0x0153h] ; Store value at 0x0154 in higher bits of reg B
		MOV CP,[0x0154]      ; Move offset address to base pointer
		MOV [CP],BX	     ; Write value of BX register to memory
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

3.) Write a program called FIND to find the larger of two signed bytes. Assume the two bytes are in memory locations 0x0150 and
0x0151. Store the larger of the two in memory location 0x0152.\\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; Find
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[0x150]
		MOV	CL,0		; Store larger of two the two bytes
LP1:	MOV	AL,[0x150+1]
		CMP	CL,AL		; Test if byte 1 > byte 2
		JA	LPC		; continue testing if not
		MOV	CL,AL		; otherwise we've found the larger byte
LPC:	DEC	BX
		JGE	LP1
		MOV	[0x152],CL      ; Store the largest byte
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

4.) Write a program called LSHIFT to shift logically the 32-bit contents of memory location 0x0150 left according to the 8-bit shift
count stored in memory location 0x0154 and store the results at memory address 0x0158. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; LSHIFT
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV BX, [0x0152] ; Store value at 0x0152 in register BX
		MOV CX, 0x0154   ; Move the address for the result into CX
		SHL BX, [0x0158] ; Left shift by the offset stored at 0x0158
		MOV [CX], BX     ; Move the value in BX to memory
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

5.) Write a program called FIND8 to find the largest unsigned 8-bit word in a list. The list begins at address 0x0154. The length of the
list is stored in an 8-bit variable at address 0x0150. Store the largest entry in memory location 0x0152. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; Find8
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
LP1:	MOV	AL,[BX+DLST]
		CMP	CL,AL			;TEST IF BL > AL
		JA	LPC				;CONTINUE IF NOT
		MOV	CL,AL			;ELSE STORE NEW MAX
LPC:	DEC	BX
		JGE	LP1
		MOV	[MAXV],CL
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

6.) Write a program called FIND32 to find the largest unsigned 32-bit word in a list. The list begins at address 0x0160. The length of
the list is stored in an 8-bit variable at address 0x0150. Store the largest entry in memory location 0x0154. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; Find32
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
LP1:	MOV	AL,[BX+DLST]
		CMP	CL,AL			;TEST IF BL > AL
		JA	LPC				;CONTINUE IF NOT
		MOV	CL,AL			;ELSE STORE NEW MAX
LPC:	DEC	BX
		JGE	LP1
		MOV	[MAXV],CL
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

7.) Write a program called SCAN to scan a list of unsigned bytes and find the smallest and largest entries in the list. The length of the
list is stored in a 16-bit variable at addresses 0x0152 and 0x0154. The list begins at address 0x0160. Store the smallest byte at address
0x0150 and the largest byte at address 0x0151. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; SCAN
; D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
LP1:	MOV	AL,[BX+DLST]
		CMP	CL,AL			;TEST IF BL > AL
		JA	LPC				;CONTINUE IF NOT
		MOV	CL,AL			;ELSE STORE NEW MAX
LPC:	DEC	BX
		JGE	LP1
		MOV	[MAXV],CL
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

8.) Write a program called COUNT to count the number of characters in a null-terminated ASCII string that are equal to a KEY. The
KEY is stored in memory location 0x0150. The string is stored in memory beginning at address 0x0160. Store the 8-bit count in
memory location 0x0154. (Assume the maximum count is 255.) \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; COUNT
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
LP1:	MOV	AL,[BX+DLST]
		CMP	CL,AL			;TEST IF BL > AL
		JA	LPC				;CONTINUE IF NOT
		MOV	CL,AL			;ELSE STORE NEW MAX
LPC:	DEC	BX
		JGE	LP1
		MOV	[MAXV],CL
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

9.) Write a program called ONES to determine the number of bits equal to one in a 32-bit variable. The 32-bit variable is in memory
location 0x0154. Store the 8-bit counter in memory location 0x0150. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; ONES
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
LP1:	MOV	AL,[BX+DLST]
		CMP	CL,AL			;TEST IF BL > AL
		JA	LPC				;CONTINUE IF NOT
		MOV	CL,AL			;ELSE STORE NEW MAX
LPC:	DEC	BX
		JGE	LP1
		MOV	[MAXV],CL
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

10.) Write a subroutine called STRLEN that determines the length of a null-terminated ASCII string. Pass the 16-bit start address of the
string to the subroutine in register BX. Return the length, excluding the null byte, in register CX. All registers (except CX) should
return to the calling program unchanged. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; STRLEN
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
LP1:	MOV	AL,[BX+DLST]
		CMP	CL,AL			;TEST IF BL > AL
		JA	LPC				;CONTINUE IF NOT
		MOV	CL,AL			;ELSE STORE NEW MAX
LPC:	DEC	BX
		JGE	LP1
		MOV	[MAXV],CL
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

11.) Write a subroutine called REPLACE that processes a null-terminated string of decimal characters and replaces leading zeros with
spaces. Pass the 32-bit address of the string to the subroutine in register BX.  \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; Replace
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
LP1:	MOV	AL,[BX+DLST]
		CMP	CL,AL			;TEST IF BL > AL
		JA	LPC				;CONTINUE IF NOT
		MOV	CL,AL			;ELSE STORE NEW MAX
LPC:	DEC	BX
		JGE	LP1
		MOV	[MAXV],CL
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\pagebreak

\begin{homeworkProblem}

12.) Write a program called UNPACK to convert the 16-bit BCD variable in memory locations 0x0150 and 0x0151 to four ASCII
characters with the high-order digit first, beginning in memory location 0x0154. \\

\solution

\begin{table}[h]
\begin{tabular}{|c|}
\hline
\begin{lstlisting}
; UNPACK
; Templated by D. Egbert ver 1.1 11/14/2016
org 100h
section .text
; beginning address of code = 0x0100
;*******************************************************
; put your code here
start:
		MOV	BX,[LSTL]
		MOV	CL,0			;USE CL FOR LARGEST VALUE
; end of your code
;*******************************************************
ILP:	JMP	ILP				;infinite loop
;
TIMES 50H -($-$$) DB 0
section .data
; beginning address of data = 0x0150
;*******************************************************
; put your data items here
LSTL:	DB	14
MAXV:	DB	0
DLST:	DB	254,5,25,250,100,150,30,200,253,15,23,46,73,175,0
; end of your data
;*******************************************************
\end{lstlisting} \\
\hline
\end{tabular}
\end{table}
\end{homeworkProblem}

\end{document}


\documentclass[12]{scrartcl}
\usepackage{amssymb,amsmath,gensymb,dsfont,calc,multicol,fullpage}
\makeatletter
\newcommand\Aboxed[1]{
   \@Aboxed#1\ENDDNE}
\def\@Aboxed#1&#2\ENDDNE{%
   &
   \settowidth\@tempdima{$\displaystyle#1{}$}
   \setlength\@tempdima{\@tempdima+\fboxsep+\fboxrule}
   \kern-\@tempdima
   \boxed{#1#2}
}
\makeatother

\begin{document}

\title{Homework 38, Section 7.7: 5, 6(b), 8(b), 9(b), 10}
\author{Alex Gordon}
\date{\today}
\maketitle
\section*{Homework}
\subsection*{5.}
The degree of the node must be greater than 2, because 1 would clearly not work. Because of this, this means that in order to use every vertex, we must have the same number of nodes in each part, meaning m and n need to be equal. Thus, as long as $m \geq 2$ then we can navigate any $k_{m,m}$
\subsection*{6. B)}
$a, d, c, e, b, a$ has the smallest weight at 24.
\subsection*{8. B)}
The greedy algorithm produces $a, e, b, d, c, a, $ with a weight of 29
\subsection*{9. B)}
The edge-greedy algorithm adds edges in the order $[a, e], [a, c ], [b ,d ], [b ,c ] $with a cycle $a, d, c, b, e, a$ with the weight of 24
\subsection*{10.}
The vertex $a$ leads to the cycle $a, d, b, c, a$ with the weight of 1000 while the cycle weights only 100.

\end{document}


\documentclass[5p,twocolumn,final]{elsarticle}

%\newcommand{\exerciseno}{1}
%\newcommand{\teamno}{0}

\makeatletter
\def\ps@pprintTitle{%
 \let\@oddhead\@empty
 \let\@evenhead\@empty
 %\def\@oddfoot{\emph{Algorithm Engineering -- Team \teamno{} -- Exercise \exerciseno{}\hfill}}%
 \let\@evenfoot\@oddfoot}
\makeatother
\usepackage{lipsum}
\usepackage{lineno,hyperref}
\usepackage{tikz,pgfplots,pgfplotstable,xcolor}
\usepackage{ifthen}
\modulolinenumbers[5]
\bibliographystyle{elsarticle-num}

\pgfplotsset{
	discard if not/.style 2 args={
		x filter/.code={
			\edef\tempa{\thisrow{#1}}
			\edef\tempb{#2}
			\ifx\tempa\tempb
			\else
				\def\pgfmathresult{inf}
			\fi
		}
	},
}
\pgfmathdeclarefunction{lg2}{1}{%
    \pgfmathparse{ln(#1)/ln(2)}%
}
\pgfmathdeclarefunction{lg10}{1}{%
    \pgfmathparse{ln(#1)/ln(10)}%
}

%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}
\begin{frontmatter}
	\title{Algorithm Engineering -- Exercise X}
	\author{Team Y: Grace Hopper, Ada Lovelace, and Joseph Weizenbaum}
%\begin{keyword}
%	Add keywords here
%\end{keyword}
\end{frontmatter}

\linenumbers
\section*{Remarks [Not to be contained in the final version]}
\begin{itemize}
	\item Write a bit to each of the following four sections (at most one page of text, including references!)
	\item Put the figures (at least one) on the subsequent pages.
	\item Use references when appropriate. E.\,g.\ always reference Skiena \cite{skiena1998} ;-)
\end{itemize}

\section{Implemented Features}
Introduce the method(s) employed in your submission to the exercise.
Ideas explained in the lecture can just be mentioned and do not need to be explained (again).

\section{Data Structures}
Tell us something about the data structures you used, and why they were a good (or not so good) choice.

\section{Highlights}
You have some freedom here.
Tell us about the most interesting observations you made during working on the assignment!

\section{Experiments}
You should do a few experiments on your submission, showing how well your solver performs.
For example (as a bare minimum), compare it to the solver of your previous submission.
Tell us here what experiments you made.
Use some figures to convince us of how great your submitted solver is.
See Figure~\ref{fig:example-diagrams} for two example diagrams.
The plots are built from the data supplied in \texttt{data.csv}.
For the top plot, the CSV needs to contain the running times of the two algorithms to compare.
For the bottom plot, the CSV needs to contain the running time of one algorithm, the graph size, and the solution size.
Thus, for the bottom plot you can easily use the CSV generated by the script \texttt{benchmark-fast.sh}, while for the top plot, you first have to merge the CSV files of two benchmark runs (one for each algorithm).
Observe that you can load multiple CSV files into the plots and, for example, give them different symbols.

% \section*{References}

\bibliography{bib}

%\clearpage
\onecolumn

\begin{figure}[t]
	\centering
	\def\maxValue{500} %
	\def\minValue{0.1} % choose such that all values are displayed
	\begin{tikzpicture}[scale=1]
		\begin{loglogaxis}[
					width=0.9\textwidth,
					height=0.55\textwidth,
					title={This is a plot for comparing running times},
					xlabel={Algorithm A [s]},
					ylabel={Algorithm B [s]},
					xmax=\maxValue,
					xmin=\minValue,
					ymax=\maxValue,
					ymin=\minValue,
					legend cell align=left,
					legend style = {
						at={(0.02, 0.98)}, % where is the legened located?
						anchor={north west}, % what corner of the legend box is specified in line above?
% 						font = \small  % if there is not enough space...
					}
				]

			% the data points from the csv files; here if one of the two entries in the csv-file is empty, then the value is replaced by 300:
			\addplot[
				only marks,color=black,mark=+,thick,
				x filter/.code={\ifthenelse{\equal{\pgfmathresult}{}}{\def\pgfmathresult{5.703782475}}{}}, % ln(300) = 5.703782475... ; needed due to logarithmic scale
				y filter/.code={\ifthenelse{\equal{\pgfmathresult}{}}{\def\pgfmathresult{5.703782475}}{}}  % ln(300) = 5.703782475... ; needed due to logarithmic scale
				] table [col sep=semicolon,x={Atime}, y={Btime}] {compare.csv};
			\addlegendentry{Test Instances}

			% helper lines indicating factor 1, 5, and 25:
			\addplot[color=black,domain=\minValue:\maxValue,samples=2] {x};
			\addlegendentry{Factor 1}
			\addplot[dashed,color=black!75,domain=\minValue:\maxValue,samples=2] {5*x};
			\addlegendentry{Factor 5}
			\addplot[dotted,color=black,domain=\minValue:\maxValue,samples=2] {25*x};
			\addlegendentry{Factor 25}

			\addplot +[color=red,solid,mark=none] coordinates {(\minValue, 300) (\maxValue, 300)};
			\addlegendentry{Timeout}

			% add plots without legend entry last:
			\addplot +[color=red,solid,mark=none] coordinates {(300, \minValue) (300, \maxValue)};
			\addplot[dashed,color=black!75,domain=\minValue:\maxValue,samples=2] {0.2*x};
			\addplot[dotted,color=black,domain=\minValue:\maxValue,samples=2] {0.04*x};

		\end{loglogaxis}
	\end{tikzpicture}

	\medskip

	\begin{tikzpicture}
		\begin{axis}[
				width=0.8\textwidth,
				height=0.6\textwidth,
				grid,
				xmode=log,
				ymode=log,
				title={A fancy plot with colors},
				xlabel={$n$},
				ylabel={$k$},
				legend cell align=left,
				legend style = {
					at={(0.02, 0.98)}, % where is the legened located?
					anchor={north west}, % what corner of the legend box is specified in line above?
					font = \small  % if there is not enough space...
				},
				colorbar,
				colorbar style={
					ytick={-1,0,1,2},
					ylabel=time in seconds,
					ylabel style={
						yshift=-7em
					},
					yticklabel={$10^{\pgfmathprintnumber{\tick}}$},
				},
			]
			\addplot[only marks,scatter src=explicit, scatter,mark=+,thick,discard if not={finished}{1}] table[col sep=semicolon,y={solsize},x={vertices},meta expr=lg10(\thisrow{time} + 0.01)] {results-2020-04-16-17-01-55-1-random.csv};
			\addlegendentry{Random}
			\addplot[only marks,scatter src=explicit, scatter,mark=o,thick,discard if not={finished}{1}] table[col sep=semicolon,y={solsize},x={vertices},meta expr=lg10(\thisrow{time} + 0.01)] {results-2020-04-16-17-01-55-2-social-networks.csv};
			\addlegendentry{Social Networks}
			\addplot[only marks,scatter src=explicit, scatter,mark=square,thick,discard if not={finished}{1}] table[col sep=semicolon,y={solsize},x={vertices},meta expr=lg10(\thisrow{time} + 0.01)] {results-2020-04-16-17-01-55-3-medium-sized.csv};
			\addlegendentry{Medium}
		\end{axis}
	\end{tikzpicture}
	\caption{
		\textbf{Top:} Comparison of the running times of two algorithms.
		The solid line denotes the values where both algorithms would have the same running time.
		Marks above the solid line indicate an instance where Algorithm A is faster, marks below the solid line indicate an instance where Algorithm B is faster.
		The dashed lines mark the factor 5 increase/decrease and the dotted lines mark the factor 20 increase/decrease.
		\textbf{Bottom:} An example for a heat map showing the running time depending on~$k$ and~$n$ at the same time.
		The marks indicate random instances (mark +), social networks (circles), and medium-sized graphs (squares).
		All scales are log-scales.
	}
	\label{fig:example-diagrams}
\end{figure}

\end{document}


\chapter{Introduction}

\section{A Problem}
Consider an image of a face. At least two \emph{systems} are at play in the image of a face, the face itself and the illumination. Both faces and illumination are complex and can vary greatly, but they are fundamentally acting independently of each other. Yet they combine to form the image.

A form of Deep Neural Networks, Deep Belief Networks, have achieved state of the art performance in facial recognition~\cite{huang2012learning,liu2014facial}, but this is only possible with a large amount of training data. This suggests that there is a disparity between the task and these networks, as the networks do not explicitly attempt to represent these \emph{systems} independently. This project takes steps toward representing complex \emph{systems/causes} separately with the primary task of decomposing images (more generally, vector based data).

Separating faces and illumination is too challenging for this project, as there is no way to verify/evaluate that the new approach is working. The concept of a face without illumination cannot be visualised. Instead I will start with the modest task of working with images where the source images being combined are known.

\section{Deep Belief Networks can achieve state of the art performance}
Deep Belief networks (DBNs) are powerful models that have proven to achieve state of the art performance in tasks such as image classification~\cite{5649922, Hinton:2006:FLA:1161603.1161605}, dimensionality reduction~\cite{noulas2008deep}, natural language recognition~\cite{Hinton:38131ww}, document classification~\cite{mcafee2008document} and semantic analysis~\cite{kim2012learning}.
Despite a DBN's expressiveness, there is no way to extract independent sources, the model instead learns how to represent the complex combination. This complex combination of sources is inherently richer than the individual sources acting alone. The DBN may learn features that correspond to each source during it's training process, however the architecture or training algorithm make no attempt to enforce this. Consider the face and illumination example, with 1~000 possible faces and 1~000 possible illuminations. If we only observe a combination of faces and illuminations there is 1~000~000 possible combinations we need to represent. It is more succinct to treat faces and illumination as independent \emph{systems}, which results in 2000 possible combinations.

DBNs are built of shallow networks called Restricted Boltzmann Machines (RBMs). Despite being building blocks for the DBN, RBMs can be used as models for practical tasks, for instance representing handwritten digits~\cite{fischer2014training}.

\section{A Proposed Solution and Contributions}

Frean and Marsland propose an alternative architecture to that of an RBM, the ORBM, which aims to encode two complex sources independently. Frean and Marsland also propose an algorithm to put this alternative architecture to use.

This project contributes:
\begin{enumerate}[$\mathcal{C}$1.]
  \item\label{item-c1} The first articulation of the proposed architecture, including the context needed to understand it.
  \item\label{item-c2} A Python implementation of the new architecture and algorithm.
  \item\label{item-c3} A suite of graded evaluations/tests that explore how the architecture and source separation algorithm work in practice.
\end{enumerate}
The evaluations will not address separating faces and illumination, instead the more general case of two images overlaying to form the data. The implications of this are compelling as applications could include removing noise from images or extracting images that are composed of many subjects. These are aspirational goals and this project will be performing tasks such as separating two handwritten digits composed on each other (see figure \ref{F:Composition-Example}).

\begin{figure}[h]
\begin{center}
  \includegraphics[width = 0.2\textwidth]{Assets/composition-example.png}
\caption{An example composition of a handwritten four and three, illustrating a non-trival task where the ground truth is known.}
\label{F:Composition-Example}
\end{center}
\end{figure}

%
% The shared views of four research groups, including the prolific Geoffrey Hinton, is that DBNs are favoured for use in speech recognition tasks over other deep learning approaches~\cite{Hinton:38131ww}. For instance \texttt{Stacked Denoising Autoencoders} and \texttt{Deep Convolutional Networks}.
%
%
% \subsection{Aspirational Examples as a motivation of the project}
%
% Now some thought experiments to highlight a fundamental weakness in DBNs.
%
% DBNs can capture rich non-linearity, making them ideal for tasks such as facial recognition, where pixels in the images change drastically depending on pose and illumination. These variations impede the use of classical feature based on geometry, such as Eigenfaces or Fischer-face~\cite{Lin:6100469ww}. There are at least two systems at play in an image of a face, the face itself that has a shape and colour, and the illumination which can vary greatly, casting shadows and highlighting parts of the face. Despite this, a DBN can achieve excellent classification performance on the UMIST dataset of 576 multi-viewed faced images of 20 individuals~\cite{Lin:6100469ww}.
%
% Consider another famous example that illustrates the idea of multiple sources acting independently to form some data, referred to as the \texttt{Cocktail Party Problem}. At a cocktail party many conversations are taking part at the same time, yet despite the cacophony, a partygoer is able to take part in their conversation, separating the sources.
%
% Both these examples exhibit independent sources arising and interacting in a non-linear way to generate a visible artifact, be it an image of a face or a audio recording. This project does not address these aspirational examples, they set the scene for it's motivation. Despite a DBN's expressiveness, there is no way to extract these sources, the model instead learning the how to represent the combination. The DBN might fully well learn features that correspond to each source during it's training process, however the architecture or training algorithm make no attempt to enforce this.
%
% % Deep Belief Networks are constructed by stacking Restricted Boltzmann Machines (RBMs). This proved a natural starting point for representing two sources. We worked in a shallow environment before attempting to stack RBMs.
% \subsection{Restricted Boltzmann Machines cannot separate sources either}
%
% DBNs are composed of a models with a shallow architecture, Restricted Boltzmann Machines (RBMs). RBMs provide a natural starting point for representing data causes by the combination of two sources.
%
% RBMs are two layer, fully connected, unsupervised neural networks. RBMs do not model causation, instead they model dependancies.
% % RBMs make a strong assumption that all features are dependant in the prior, inheriently modelling a single source.
%
%   % In the image domain, this equates to all pixels being assumed relevant to what we are trying to model. With enough examples RBMs can generalise, but there is no mechanism for modelling independant subjects.
%   Again using the example of images, an input image will map to a single representation. There lacks a mechanism for modeling  sources that are acting independantly, instead modeling inputs as a single source.
%
% % There is another structure, Belief/Bayesian Networks, and the parameterised version, the Sigmoid Belief Network, which makes the polar assumption where each feature is modelled by an independant cause. These are intractable to train in practice.
% \subsection{Sigmoid Belief Networks --- Intractably causal in practice}
% The Sigmoid Belief Network, the parameterized version of a Bayesain/Belief network appears as a natural choice for modelling independent sources in that it makes a polar assumption to the RBM; -- Warning Semicolon Use -- Every feature has an independant cause. The sigmoid belief networks assumption could capture data that has multiple sources, but this is intractable in practice.
%
% % Frean, Marsland propose a generative model that aims for a middle ground between these two existing models.
% \section{A proposed solution}
% \subsection{Trading tractibility for Source Separation}
% Frean and Marsland propose a generative model that aims to trade a small amount of the RBMs performance for richness, finding a middle ground between the Sigmoid Belief network and the Restricted Boltzmann Machine.
% Frean and Marsland also propose an algorithm to invert this proposed generative model, seperating the sources of an input.
%
% The new generative model, referred to onwards as an ORBM, uses an RBM to model each source and a sigmoid belief network to capture their combination to from data. This project explores the ORBM use for separating two causes.
%
% \begin{figure}[h]
% \begin{center}
%   \includegraphics[width = 0.5\textwidth]{Assets/ORBM_fig_1}
% \caption{A graphical representation showing the proposed generative model for capturing two causes, the ORBM.}
%
% \label{F:ORBM-fig-1}
% \end{center}
% \end{figure}
%
% Given the proposed model and algorithm, this project answers the following questions:
% \begin{itemize}
%   \item Can the ORBM represent two sources of composite data independent, that is can it be used to perform source separation?
%   \item Is the ORBMs two cause structure to rich to be tractable in practice?
% \end{itemize}
%
% \section{Results and Contribution}
%
% This project provides a suite of evalautions, in the form of tests of increasing complexity to answer the above questions. The aim being to incrementally verify the new generative model and inference algorithm.
%
% The tests give reasons for optimism, especially in the smaller cases. Challenges have been exposed in the evaluation that will need to be addressed going forward, to make the algorithm and model scalable.


\section{Fonts}

Fonts have these attributes: family, series, shape, size, and encoding. The first four attributes have some convenient high-level commands that are useful. These are the declarations and corresponding inline commands (defaults are out of the box defaults and can be overridden in the preamble): \begin{verbatim}

   % family
   \rmfamily \textrm{<text>}   % roman; default
   \sffamily \textsf{<text>}   % sans serif; (why not ss or ssf?)
   \ttfamily \texttt{<text>}   % monospace typewriter

   % series
   \mdseries \textmd{<text>}   % normal weight and expansion;
                               % default
   \bfseries \textbf{<text>}   % bold \end{verbatim}

\begin{verbatim}
   %%% shape
   % upright; \textnormal defaults to this
   \upshape \textup{<text>} \textnormal{<text>}

   % italics
   \itshape \textit{<text>} \emph{<text>} {\em ...}
   % \emph and \em toggle back and forth when nested
   % \itshape and \emph use italic space correction,
   % \textit does not

   % slanted
   \slshape \textsl{<text>}

   % small caps
   \scshape \textsc{<text>}

   % size declarations
   \normalsize % 10 pt by default; can be set to 11 or 12
   \large % 12 pt
   \Large % 14.4 pt
   \small % 9 pt
   \footnotesize % 8 pt
\end{verbatim}

The following text is an example of an unnamed environment. See the source code to understand how the commands are scoped:

\ind{\Large this text is Large {\slshape and now slanted}}

The preceding commands are actually built upon other lower level commands. These other commands have the form \verb2\font<attr>2, where \verb2<attr>2 is one of the  font attributes.\footnote{Pages 362-4 of Kopka and Daly cover these in detail.}

The encoding selects the lookup table for the font. In essence, this sets the fundamental character forms that are printed.

The family sets the font properties such as serif (r for roman), sans serif (s/ss), or equal spacing (tt for typewriter). There are various names for the families, with the letters r, s, and t often giving hints as to what the family looks like.

The series sets the weight and the width. The letter \q l\q\ for a weight means lighter than normal. The letter \q b\q\ means heavier than normal. The letter \q m\q\ means normal (think median). For \q l\q\ and \q b\q, these prefixes further modify the weight: s = semi (means less than \q l\q\ or \q b\q\ by themselves), e = extra, u = ultra. Width uses similar conventions, with \q m\q\ meaning normal, \q c\q\ meaning compressed, and \q x\q\ meaning expanded. Same modifiers as for weight apply.

The shape sets the angle or small caps. The letter \q n\q\ is normal, \q sl\q\ is slanted, \q it\q\ is italic, and \q  sc\q\ is small caps.\footnote{The book also shows a \q u\q\, which possibly means upright.}

The size sets the height of the letter \q x\q\ in points, along with the vertical separation between lines in points. There are 12 preset values the pitch can take out of the box.\footnote{Other values may be added, but the book doesn't mention how.}

If a font with all the specified attributes cannot be found, the tool issues a warning and says which attributes are used.

As an example:

\begin{verbatim}
   % defaults are OT1, cmr, mm, n, 10, and the default spacing
   % OT1 is a font class. cmr is Computer Modern Roman,
   % mm is probably median, n is upright, 10 is 10 pt.
   % use Cork encoding; this expands on Knuth's original table
   \fontencoding{T1}
   \fontfamily{cmss} % computer modern sans serif
   \fontseries{sbm}  % semibold normal width
   \fontshape{n}     % upright
   \fontsize{14.4}{16} % 14.4 pt height, 16 pt space between lines
   \selectfont % activates everything in preceding scope
\end{verbatim}

A more abbreviated command set is this: \begin{verbatim}
   \usefont{T1}{cmss}{sbm}{n}
   \fontsize{14.4}{16}
   \selectfont % remains in effect until next \selectfont command
\end{verbatim}


\section{Basic Electronic Components}

The following sections briefly introduce electronic components. Each of these pieces can be used to make complex, powerful circuits.\footnote{Most of these definitions come from Wikipedia, though they have been edited and augmented by the author in several places.}

\subsection*{Resistors}

A resistor limits the electrical current that flows through a circuit. Resistance is the restriction of current. In a resistor, the energy of the electrons that pass through the resistor are changed to heat and/or light. For example, in a light bulb there is a resistor made of tungsten which converts the electrons into light and a great deal of heat.

\begin{center}
  	\begin{circuitikz}
    	\draw (0,0)
      	to[battery] (0,2) %
     	to[R=$R_1$] (2,2)
     	to[short] (2,0) -- (0,0)
		;

     	\draw (0,0)
      	node[ground] {} % ground connection
		;
		\node[scale=0.7, thick ] at (-0.25,1.4) {$+$};
		\node[scale=0.7, thick ] at (-0.25,0.6) {$-$};

   \end{circuitikz}

{A very simple circuit. Electricity flows through the resistor (maybe it's a light bulb). The resistor controls how much current flows in the circuit, and heats up.}
\end{center}

\subsection*{Capacitors}

A capacitor (also called a \emph{condenser}, which is the older term) is an electronic device that stores electric energy (a ``charge''). It is similar to a battery, but is smaller, lightweight and charges up much quicker. Typically, capacitors hold much less energy than a battery, but can provide almost their entire charge to the circuit quickly. Capacitors are used in many electronic devices today, and can be made out of many different types of material. A capacitor-like effect can also result just from two conductors being close to each other, whether you want it to exist or not-- if you've ever been shocked by a door knob or metal refrigerator in the winter, you have been one plate of a capacitor, because you've been storing a charge!

\begin{center}
  	\begin{circuitikz}
    	\draw (0,0)
      	to[battery] (0,2) %
     	to[C=$C_1$] (2,2)
     	to[short] (2,0) -- (0,0)
		;

     	\draw (0,0)
      	node[ground] {} % ground connection
		;
		\node[scale=0.7, thick ] at (-0.25,1.4) {$+$};
		\node[scale=0.7, thick ] at (-0.25,0.6) {$-$};

		\vertionarray{0.75}{2}{\ionradius}{$+$}
		\vertionarray{1.25}{2}{\ionradius}{$-$}
   \end{circuitikz}

\medskip
\end{center}
{Open capacitors in a steady state. No current flows, but there is a charge.}

\subsection*{Diodes}

A diode is an electronic component with two electrodes (connectors). It allows electricity to go through it only in one direction.

Diodes can be used to convert alternating current to direct current (this circuit is called a diode bridge). They are often used in power supplies and can be used to decode AM (``amplitude modulation") radio signals (like in a crystal radio). Light-emitting diodes (LEDs) are a type of diode that produce light.

\begin{center}
  	\begin{circuitikz}
    	\draw (0,0)
      	to[battery] (0,2) %
     	to[R=$R_1$] (2,2)
		to[empty led](2,0)
     	to[short] (2,0) -- (0,0)
		;

		\draw(2.5, 1.5)
		node[right]{$D_1$};

     	\draw (0,0)
      	node[ground] {} % ground connection
		;
		\node[scale=0.7, thick ] at (-0.25,1.4) {$+$};
		\node[scale=0.7, thick ] at (-0.25,0.6) {$-$};

   \end{circuitikz}

\medskip
\end{center}

{A very simple circuit. Electricity flows through the resistor, that controls how much current flows into the LED. If the battery was put in backwards,  no electricity could flow in the opposite direction, so the LED would not light up.}

\subsection*{Inductors (Coils)}

An inductor, also called a coil or (rarely) a reactor, is a two-terminal electrical component that stores electrical energy in a magnetic field, when electric current is flowing through it. An inductor typically consists of an electric conductor, such as a wire, that is wound into a \emph{coil}. Sometimes inductors include a magnetic bar or ring, around which the wire is wound. Other times the wire is wound with only air in the middle (which still works, because the Earth has a magnetic field).

They are often used as electronic filters, to separate signals of different frequencies, and in combination with capacitors to make tuned circuits, used to tune radio and TV receivers.

\noindent The symbol for an inductor is a coil icon. Inductors are usually abbreviated with an L:

\bigskip

\begin{center}
  	\begin{circuitikz}
    	\draw (0,0)
      	to[battery] (0,2) %
     	to[L, l^=$L_1$] (2,2) % The inductor
		to[C, l^=$C_1$](2,0)
     	to[short] (2,0) -- (0,0)
		;

     	\draw (0,0)
      	node[ground] {} % ground connection
		;
		\node[scale=0.7, thick ] at (-0.25,1.4) {$+$};
		\node[scale=0.7, thick ] at (-0.25,0.6) {$-$};

   \end{circuitikz}

\medskip
\end{center}

A simple circuit with one coil and one capacitor. This sort of circuit is one way to make an \emph{oscillator}. This kind of circuit is found in radios.

\subsection*{Transistors}

A transistor is a \emph{semiconductor} (see below) device used to amplify or switch electronic signals and electrical power. It is composed of semiconductor material usually with at least three terminals for connection to an external circuit. A voltage or current applied to one pair of the transistor's terminals controls the current through another pair of terminals. Because the controlled (output) power can be higher than the controlling (input) power, a transistor can amplify a signal. Today, some transistors are packaged individually, but many more are found embedded in integrated circuits.

The transistor is the fundamental building block of modern electronic devices. It might be the most important -- or at least the most influential -- engineering discovery, ever.

Why are transistors important? They enable a lot of powerful new devices, automation, information analysis, and even artificial intelligence. They have enabled huge leaps in our ability to do new things, since 1947.

\bi

\+ They can act like switches -- with no moving parts, and that can be controlled with a tiny electrical signal

\+ They can take a small signal and turn it into a powerful signal; they can make sound louder, broadcast radio signals farther, or detect very tiny changes in other materials (thus, you can have electronic thermometers, or radio telescopes)

\+ When combined into logic gates (more on that below), they enable super-fast computations that have also changed how we do work, science, medicine, and almost every form of research

\ei

\subsection*{A Little History}

Many years ago, a ``computer'' meant ``the person who did the math problems." A few people invented adding machines like an abacus, and \emph{slide rules} (see Figure \ref{fig:sliderule}) made even difficult calculations very possible, but they still required a person to do the math. In the same way, people had learned about electricity, but couldn't do much with it except power light bulbs, street cars, and telegraphs. Talking to someone many miles away meant either using Morse code, or writing a letter.

\begin{figure}[!ht]
\begin{center}
\fbox{
\includegraphics[scale=0.10]{SlideRule-2.jpg}
}
\end{center}
\caption{A pocket-sized slide rule from the late 1960s. This example is one of many that were given as promotional gifts to people who worked on NASA's lunar missions.}
\label{fig:sliderule}
\end{figure}

The first discovery that led to the transistor was the \emph{vacuum tube diode}, which uses a hot wire in a glass enclosure with no air in it to push electrons to another conductor, just like any other diode. Unfortunately, vacuum tube diodes are big, fragile, hard to make, and produce a huge amount of heat. The \emph{vacuum tube triode}, which does everything a transistor does, came next. Since electrons are negatively charged, a slight negative charge on the third wire (between, but not touching, the hot-wire emitter and the collector) permitted people to control the flow of electrons from one side to the other -- it acted like a switch.

Vacuum tube triodes have all the same shortcomings as the vacuum diode, and have even more parts to fail. Despite their shortcomings, vacuum tubes changed the world in a \emph{huge} way: they enabled radios, televisions, radar, and the first digital computers. But these computers were the size of a small house, weighed 30 tons, and required powerful cooling.

In the 1940s, three scientists discovered that some substances could either behave like an insulator \emph{or} a conductor, depending on how much electricity or heat was applied to it. Being able to change a slab of glass or silicon from an insulator into a conductor meant they could do everything a vacuum tube could do, except much smaller, cheaper, and cooler. About nine years after they announced the discovery and proved that it works, the three men won the Nobel Prize. They had discovered a \emph{semiconductor}.

\stbox{6.0in}{There's a longer documentary about the transistor on YouTube {\color{webblue}\href{https://www.youtube.com/watch?v=jYxf3gYUZBM}{here}}.}

The world changed amazingly quickly when we started using transistors. Today, there are many different kinds of transistors. The computer in your house has many \emph{billions} of transistors inside it. So does your phone, your car, your microwave, your television...wow!

Some transistors are made especially for certain applications, like making audio louder, or pushing radio signals, or just for doing computations. These are all transistors, but they are different.

\subsection*{Electronics and Transistors}

Transistors are usually abbreviated with $Q$. There are several kinds of transistors, but they work mostly the same.

\begin{center}
\begin{circuitikz}
\draw
(1,1) node[npn](Q1){}
(0.5,1.5) node[left](){$Q_1$}
(Q1.B) node[left] {$Base$} %label
(Q1.C) node[right] {$Collector$} %label
(Q1.E) node[right] {$Emitter$} %label
;
\end{circuitikz}
\medskip

Most transistors have three pins. For this kind of transistor (called an ``NPN"):
\bi
\+ Electricity comes in the \emph{collector}
\+ Electricity applied to the \emph{base} opens up the flow of electricity to the \emph{emitter}
\+ Electricity, when able, flows out the emitter and towards a ground.
\+ Therefore, the base is like a switch. It turns the transistor from an insulator into a conductive wire.
\ei

\end{center}

\noindent Most computers use a transistor type called a ``FET" (\emph{Field Effect Transistor}), that is very efficient and does not require very much electricity to switch from ``on'' to ``off''. They are faster and run cooler than the older types of transistors. Somewhat confusingly, these transistors have different names for their pins, compared to the older (but still preferable for some applications) ``BJT" type transistors, seen above.

\begin{center}
\begin{circuitikz}
\draw
(1,1) node[nigfete,solderdot](Q2){} % an N-type IGFET, also called a MOSFET
(0.75,1.75) node[left](){$Q_2$}
(Q1.B) node[left](){$Gate$} %label
(Q1.C) node[right]() {$Drain$} %label
(Q1.E) node[right]() {$Source$} %label
;
\end{circuitikz}
\end{center}
\medskip

For questions of computers, we will treat transistors like a switch and nothing more. That's easy, right?

But how do transistors do all the cool stuff they do? Why are pocket calculators or iPhones possible? How do collections of electronic components enable us to do math, send text messages, or simulate a nuclear explosion without anything going kaboom?

At the absolute base of the entire electronics world, just above transistors, are \emph{logic gates}. They are the simplest building blocks of digital computers.

\stbox{6.0in}{A good additional introduction video is {\color{webblue}\href{http://ed.ted.com/lessons/how-transistors-work-gokul-j-krishnan}{here}} (\texttt{ed.ted.com}).}


\documentclass[epsfig,10pt,fullpage]{article}

\newcommand{\LabNum}{2}
\newcommand{\CommonDocsPath}{../../../../common/docs}
\input{\CommonDocsPath/preamble.tex}

\begin{document}

\centerline{\huge Computer Organization}
~\\
\centerline{\huge Laboratory Exercise \LabNum}
~\\
\centerline{\large Using Logic Instructions with the Nios\textsuperscript{\textregistered} II Processor}
~\\

Logic instructions are needed in many embedded applications.  Logic instructions are useful
for manipulation of bit strings and for dealing with data at the bit level where only a
few bits may be of special interest.  They are essential in dealing with input/output tasks.
In this exercise we will consider some typical uses.
We will use the Nios\textsuperscript{\textregistered} II processor in one of the pre-built computer systems that come with the Monitor Program.

\section*{Part I}
\addcontentsline{toc}{1}{Part I}
In this part you will implement a Nios II assembly language program that counts the longest
string of 1's in a word of data. For example, if the word of data is {\sf 0x103fe00f}, then
the required result is 9.

~\\
Perform the following:

\begin{enumerate}
\item Create a new folder to hold your Monitor Program project for this part. Create a
file called {\it part1.s}, and type the assembly language code shown in
Figure~\ref{fig:code} into this file.
This code uses an algorithm involving shift and AND operations to find the required
result---make sure that you understand how this works.

\item
Make a new Monitor Program project in the folder where you stored the {\it part1.s}
file. Depending on your DE-series board, use the corresponding computer system
listed in Table~\ref{tab:computer_systems}.

\begin{table}[H]
	\begin{center}
	\begin{tabular}{ l | l }
	\bf{Board} & \bf{Computer System} \\
	\hline
	\rule{0pt}{3ex}DE0-CV & DE0-CV Computer \\
	DE1-SoC & DE1-SoC Computer \\
	DE2-115 & DE2-115 Computer \\
	DE10-Lite & DE10-Lite Computer \\
	DE10-Standard & DE10-Standard Computer \\
	\end{tabular}
	\caption{DE-series board computer systems}
	\label{tab:computer_systems}
	\end{center}
\end{table}

\item
Compile and load the program. Fix any errors that you encounter (if you mistyped some of
the code). Once the program is loaded into memory in the computer, single step
through the code to see how the program works.
\end{enumerate}

\newpage
\section*{Part II}
\addcontentsline{toc}{2}{Part II}
Perform the following:
\begin{enumerate}
\item Make a new folder and make a copy of the file {\it part1.s} in that new folder. Give
the new file a name such as {\it part2.s}.
\item In the new file {\it part2.s}, take the code which calculates the
number of consecutive 1's and make it into a subroutine called ONES. Have the subroutine use
register r4 to receive the input data and register r2 for returning the result.
\item Add more words in memory starting from the label TEST\_NUM. You can add as many
words as you like, but include at least 10 words. To terminate the list include the word 0
at the end---check for this 0 entry in your main program to determine when all of the
items in the list have been processed.
\item In your main program, call the newly-created subroutine in a loop for every word of
data that you placed in memory. Keep track of the longest string of 1's in any of the words,
and have this result in register r10 when your program completes execution.
\item Make sure to use breakpoints or single-stepping in the Monitor Program to observe what
happens each time the ONES subroutine is called.
\end{enumerate}

\begin{figure}[H]
\begin{center}
\lstinputlisting[style=defaultNiosStyle]{../design_files/part1.s}
\end{center}
\caption{Assembly-language program that counts consecutive ones.}
\label{fig:code}
\end{figure}

\newpage
\section*{Part III}
\addcontentsline{toc}{3}{Part III}
One might be interested in the longest string of 0's, or even the longest string of
alternating 1's and 0's. For example, the binary number {\sf 101101010001} has a string of
6 alternating 1's and 0's.

~\\
 Write a new assembly language program that determines the following:
\begin{itemize}
\item Longest string of 1's in a word of data---put the result into register r10
\item Longest string of 0's in a word of data---put the result into register r11
\item Longest string of alternating 1's and 0's in a word of data---put the result into
register r12
(Hint: What happens when an n-bit number is XORed with an n-bit string of alternating 0's and 1's?)
\end{itemize}
Make each calculation in a separate subroutine called ONES, ZEROS, and ALTERNATE. Call
each of these subroutines in the loop that you wrote in Part III, and keep track of the
largest result for each calculation, from your list of data.

\section*{Part IV}
\addcontentsline{toc}{4}{Part IV}
In this part you are to extend your code from Part III so that the results produced are
shown on the 7-segment displays on your DE-series board. Display the longest string of 1's
(r10) on {\it HEX}$1-0$, the longest string of 0's (r11) on {\it HEX}$3-2$, and
the longest string of alternating 1's and 0's (r12) on {\it HEX}$5-4$.

~\\
Each result should be displayed as a two-digit decimal number. You may want to use the approach
discussed in Part IV of Exercise 1 to convert the numbers in registers r10, r11, and r12 from
binary to decimal.

~\\
The parallel ports in the computer systems that are connected to the 7-segment displays
are memory mapped devices.
Figure~\ref{fig:HEX} shows how the display segments are connected to the parallel ports
and the ports' respective memory mapped addresses.

~\\
To show each of the numbers from \red{0} to \red{9} it is necessary
to light up the appropriate display segments.
For example, to show \red{0} on {\it HEX}0 you have to turn on all
of the segments except for the middle one (segment 6). Hence, you would store the
bit-pattern $(00111111)_2$ into the address corresponding to the {\it HEX}$3-0$ parallel port to show this result.
A subroutine that produces such bit patterns in given in Figure~\ref{fig:code7}.

\begin{figure}[H]
	\begin{center}
	\includegraphics[scale=.75]{figures/figureHEX.pdf}
	\end{center}
	\caption{The parallel ports connected to the seven-segment displays, {\it HEX}$5-0$, of the DE-series board.}
\label{fig:HEX}
\end{figure}

An example of code that shows the contents of registers on the 7-segment displays is
illustrated in Figure~\ref{fig:codefrag}. Note that this code uses the DIVIDE subroutine that
was discussed in Part IV of Exercise 1.  The code in the figure shows only the steps needed
for register r10.  Extend the code to display all three result registers on the 7-segment
displays as described above.

\begin{figure}[H]
\begin{center}
\lstinputlisting[style=defaultNiosStyle, linerange=1-13]{../design_files/part4.s}
\end{center}
\caption{A subroutine that produces bit patterns for 7-segment displays.}
\label{fig:code7}
\end{figure}

\begin{figure}[H]
\begin{center}
\lstinputlisting[style=defaultNiosStyle, firstline=16]{../design_files/part4.s}
\end{center}
\caption{A code fragment for showing registers in decimal on 7-segment displays.}
\label{fig:codefrag}
\end{figure}

\input{\CommonDocsPath/copyright.tex}
\end{document}


\documentclass[a4paper,12pt]{article}
\usepackage[margin=2cm]{geometry}

\usepackage{comment}
\usepackage{longtable}
\usepackage{tabularx,booktabs}
\usepackage{listings}
\usepackage{graphicx}
\usepackage{hyperref}

\lstset{basicstyle=\ttfamily}
\newcommand{\py}{\lstinline}

% Math notations
\newcommand{\Z}{{\mathbb{Z}}}                     % set of integers
\input{src/moa-commands.tex}

\title{MoA formalism and arrays in software}
\author{Pearu Peterson and Hameer and Saul and Lenore and Travis\\
Quansight, Labs}

\begin{document}
\maketitle
\section{Introduction}

MoA - Mathematics of Arrays - defines a formalism (based on APL) to
describe multidimensional arrays and operations on these. On the other
hand, contemporary array processing software (NumPy, Xnd, Apache
Arrow, etc) implement multidimensional array container objects and
define UI to interact with the array objects. The aim of this document
is  to provide a mapping between MoA formalism and UI-s of widely used
array processing software.

\section{Basic definitions}

The MoA formalism is originally defined in ``A Mathematics of Arrays''
by Lenore Mullin, PhD thesis, 1988\cite{mul00}.  We use NumPy
\cite{travis0} user interface as a representative of a software
implementing UI for multidimensional arrays. For brevity, we assume
\verb+from numpy import *+.

\noindent
In general, \emph{an array is an object that has a mapping to get
  array items} \footnote{A mapping $f$ is a set of argument and value
  pairs,$(i,v)$, and is represented as $f(i)=v$. We consider only
  single-valued mappings.} . The arguments to the mapping are called
\emph{indices}, all valid indices define the so-called \emph{index
  set} of an array. The items can be arbitrary objects.
Two arrays are \emph{equal} iff their get items mappings are equal: their shapes and components are equivalent.
\emph{Element-wise operations} of two arrays are defined iff the index
sets of the two arrays are equal.
Arrays can be classified according to how the mappings are defined:
\begin{description}
\item[Multidimensional arrays:] The index set of
  an $N$-dimensional array is an $N$-dimensional hyperrectangle in
  $\Z^N$: $[0,\ldots,s_0-1]\times\cdots\times[0,\ldots,s_{N-1}-1]$
  where non-negative integers $(s_0,\ldots,s_{N-1})$ define the shape
  of an array.
\item[Ragged arrays:] The index set of an
  $N$-dimensional ragged array is a connected subset of an
  $N$-dimensional hyperrectangle in $\Z^N$. The concrete
  representation of the index set may have different forms.
\item[Sparse arrays:] The index set of an sparse array is a possibly
  disconnected subset of $N$-dimensional hyperrectangle in $\Z^N$ and
  is extended to the full hyperrectangle by defining the default
  value.
\end{description}

The MoA deals with multidimensional arrays only.  The NumPy implements
array object for multidimensional arrays only. There is one-to-one
relation between MoA formalism and NumPy UI.

\section{Optimization of Arrays in NumPy: Dense to Start}
We show how the inclusion of MoA and the Psi Calculus for dense arrays
reduces to a normal form. This is because the index compositions of the Psi Calculus, MoA's indexing, has, what Mathematicians call the {\em Church-Rosser} property, named after the Mathematicians who invented the Lambda  Calculus\footnote{The Church-Rosser Theorem states that reduction rules of the lambda calculus are confluent. As a consequence of the theorem, a term in the lambda calculus has at most one normal form, justifying reference to "the normal form" of a given normalizable term.}. This is a powerful concept in that programmers may design an algorithm differently, one may be efficient, another may not, even if the answer is the same.
\noindent
We show that there will not be performance issues for that choice because all algorithms using the MoA algebra will reduce to the same normal form. Another important concept is that it is possible to eliminate all intermediate arrays. We say possible, because we'll also discuss why we may {\em choose} to not materialize based on shape and size properties relative to access patterns and architectural components.
We'll first give a small example so the reader can follow through why the optimized version obtains the same results. Then, we'll discuss status of automation as well experiments that take our original and optimized versions through various sizes to illustrate our predictions.

\subsection{An Example}

In the following we demonstrate how to reduce a NumPy array expression
to its normal form using MoA.

{\bf Much editing to relate MoA notation}

Consider the following NumPy arrays
\begin{verbatim}
>>> A = np.arange(30).reshape((2,3,5))
>>> B = 2 + A
>>> A
array([[[ 0,  1,  2,  3,  4],
        [ 5,  6,  7,  8,  9],
        [10, 11, 12, 13, 14]],

       [[15, 16, 17, 18, 19],
        [20, 21, 22, 23, 24],
        [25, 26, 27, 28, 29]]])
>>> B
array([[[ 2,  3,  4,  5,  6],
        [ 7,  8,  9, 10, 11],
        [12, 13, 14, 15, 16]],

       [[17, 18, 19, 20, 21],
        [22, 23, 24, 25, 26],
        [27, 28, 29, 30, 31]]])
\end{verbatim}
that in MoA formalism are represented as
\begin{align}
  A^3 &\equiv \vccc235\reshape(\range30), \\
  B^3 &\equiv 2+A^3,
\end{align}
and the following array expression as an example case
\begin{verbatim}
>>> r = np.inner(A[1,0,:], np.outer(A[1,0,:], B[0,1,:])[2,:])
>>> r
13175
\end{verbatim}
that in MoA formalism reads
\begin{equation}
r \equiv (\vcc10\psi A^3) \innerprod+\times (\vc2\psi((\vcc10\psi A^3) \outerprod\times (\vcc01\psi B^3))).
\end{equation}
Note that when evaluating the array expression directly, the
computation involves 30 multiplication and 4 addition
operations. Using MoA, we reduce the above array expression to its
normal form where the number of operations is minimal.

First, the items of the left operand of the inner product are
\begin{align*}
  \getitem{(\vcc10\psi A^3)}{\vc i} = \getitem{A^3}{\vccc10i}
\end{align*}
The items of the right operand of the inner product are
\begin{align}
&\getitem{
  (\getitem{((\getitem{A^3}{\vcc10}) \outerprod\times (\getitem{B^3}{\vcc01}))}{\vc2})
}{\vc i} \\
&\equiv \getitem{((\getitem{A^3}{\vcc10}) \outerprod\times (\getitem{B^3}{\vcc01})))}{\vcc2i} \\
&\equiv \getitem{(\getitem{A^3}{\vcc10})}{\vc2} \times \getitem{(\getitem{B^3}{\vcc01})}{\vc i}\\
&\equiv \getitem{A^3}{\vccc102} \times \getitem{B^3}{\vccc01i}
\end{align}
As a result, we have
\begin{align*}
  r &\equiv ((\vcc10\psi A^3) \innerprod+\times  \getitem{B^3}{\vcc01})\times\getitem{A^3}{\vccc102}
\end{align*}
that in NumPy syntax reads
\begin{verbatim}
>>> r_nf = np.inner(A[1,0,:], B[0,1,:]) * A[1,0,2]
\end{verbatim}
which contains 6 multiplications and 4 additions.

%\subsection{A Small Example (TO %BE REMOVED)}
%Let's look at two NumPy expresssions. we  use small shapes to help the reader go through an expression to validate results. Large sizes are used in experiments to demonstrate conjectures.
%\begin{verbatim}
%>>> a=np.arange(30).reshape((2,3,5))
%>>> a
%array([[[ 0,  1,  2,  3,  4],
%        [ 5,  6,  7,  8,  9],
%        [10, 11, 12, 13, 14]],
%
%       [[15, 16, 17, 18, 19],
%       [20, 21, 22, 23, 24],
%       [25, 26, 27, 28, 29]]])
%        >>> b=a+2
%>>> b
%array([[[ 2,  3,  4,  5,  6],
%        [ 7,  8,  9, 10, 11],
%        [12, 13, 14, 15, 16]],

%       [[17, 18, 19, 20, 21],
%        [22, 23, 24, 25, 26],
%        [27, 28, 29, 30, 31]]])
%        >>> a.shape
%(2, 3, 5)
%>>> a[:,0,:]
%array([[ 0,  1,  2,  3,  4],
%       [15, 16, 17, 18, 19]])
%>>> a[:,0,:][1,]
%array([15, 16, 17, 18, 19])
%>>>
%>>>
%>>> A=a[:,0,:][1,]
%>>> B=b[:,1,:][0,]
%>>> np.multiply.outer(A,B)
%array([[105, 120, 135, 150, 165],
%       [112, 128, 144, 160, 176],
%       [119, 136, 153, 170, 187],
%       [126, 144, 162, 180, 198],
%       [133, 152, 171, 190, 209]])
%>>> C=np.multiply.outer(A,B)
%>>> A
%array([15, 16, 17, 18, 19])
%>>> B
%array([ 7,  8,  9, 10, 11])
%>>> C
%array([[105, 120, 135, 150, 165],
%       [112, 128, 144, 160, 176],
%       [119, 136, 153, 170, 187],
%       [126, 144, 162, 180, 198],
%       [133, 152, 171, 190, 209]])
%>>> C[2,:]
%array([119, 136, 153, 170, 187])
%>>> A * C[2,:]
%>>> np.inner(A,C[2,:])
%13175
%\end{verbatim}

%Let's reduce an expression to
%a Denotational Normal Form (DNF), a semantic normal form that is not yet %aware of memory layout or architectural components: vector registers, DMAs, %etc.
%We put all the expressions above into one that uses only a and b.
%Moreover, we use small sizes for the reader to follow examples. Our experiments
%use sizes from small to large to emphasize what sizes achieve the best speed-ups. We
%explain why this is true.
%np.inner((a[:,0,:][1,]),(np.multiply.outer((a[:,0,:][1,]),(b[:,1,:][1,])))[2,:])
%\begin{verbatim}
%To Reduce:

%np.inner((a[:,0,:][1,]),(np.multiply.outer((a[:,0,:][1,]),(b[:,1,:][0,])))[2%,:])

%\end{verbatim}

\subsection{Performance analysis}

Now let us benchmark the original form of this expression against it's reduced normal form.
For $n$ in ${\{1, 10, 20, ..., 200\}}$ where $\shape A \equiv \vccc2 3 5 \times n$ we measured the
execution of the original expression and the reduced expression.

\begin{figure}[h]
  \includegraphics[width=1\textwidth]{benchmarks/visualization-ratio.png}
  \caption{The ratio of the mean execution time of evaluating the reduced and the original expression decreases as the shape increases.}
  \label{fig:plot}
\end{figure}

\begin{figure}[h]
  \includegraphics[width=1\textwidth]{benchmarks/visualization-two-plots.png}
  \caption{The execution times for the original and the reduced expression.}
  \label{fig:plot2}
\end{figure}

As seen in Figure \ref{fig:plot}, the executing the reduced  form takes a third of the time of executing the original form when $n=1$ and a hundredth of the time when
~$n=80$. So as $n$ increases, so does the proportional time saving from reducing the expression.

To reproduce the benchmarks and for all the benchmark data as well as machine information please go to \href{https://github.com/Quansight-Labs/uarray-docs/tree/fcb6baf8bb7284ac7223de73a70cf47063df9e4b/benchmarks}{github.com/Quansight-Labs/uarray-docs}.

\section{Psi Reduction to a Denotational Normal Form (DNF) (TO BE REVISED)}
To $\psi$-R an array expression requires that we:
\begin{enumerate}
    \item We  prove that the shapes of both expressions are equivalent,
    \item We prove all indices based on shapes are equivalent.
\end{enumerate}

We equate the NumPy expressions to MoA expressions. Then we $\psi$-Reduce.
Each time we present a mathematical proof and want to become more mathematical in our explanation we use  {\em MoA} expressions to inform the reader we are doing mathematics without any mention of language. It is important to note that We can translate between the two notations easily  because they are semantically equivalent expressions. We use the {\em syntactic sugar} of Python and NumPy to  converse with the reader. We only change syntax for those interested in the mathematics and want to know more. Moreover, to show long proofs, it is often historic mathematically to use single variable names, and/or complex symbolic expressions.
\subsection{Steps to Automation}
We presently are working on a high-level array computing interface (library) that provides:
\begin{itemize}
    \item Automatic reduction of expressions.
    \item Multiple back ends for computation.
\end{itemize}
We have given an example of hand automation and build upon numerous prototypes to discuss our attempts
at automation easily accessible to contemporary applications, e.g. Tensorflow and PyTorch, that need it.
\subsection{What is this for?}
\begin{itemize}
    \item Reducing computation and memory from intermediate expressions.
    \item Express intent without worrying about optimization details.
    \item Writing array operations in an abstract way so that they can be executed efficiently on multiple types / more easily switch out implementation.
\end{itemize}
\subsubsection{Some Examples of Switching out Implementation.}
\begin{itemize}
    \item Demonstrate an expression with the NumPy backend
    \item Demonstrate how to write TensorFlow or PyTorch backend and then how to apply same expression with this backend.
    \item Show how this gets translated.
\end{itemize}
Overall goals are to create an array computation ecosystem in python work better together by making it more modular, thus making it possible to innovate faster at different levels without rewriting everything. Ideally we ant to automatically choose back-end based on size or nature of problem.
\newpage
{\bf Appendix}
\subsection{DNF (TO BE REVISED: see above.)}
We will use the same variable names from above except we annotate to indicate dimensionality, e.g. $\vec v$ for a vector and $A^2$ for a matrix, etc.
\begin{eqnarray}
A^3 \equiv \vccc235\reshape(\range30) \\
B^3 \equiv 2+A^3
\end{eqnarray}
$A^3,B^3$ are a and b above.
Let:
\begin{eqnarray}
\vec a \equiv (\vc1\psi( \vc0 \psi (\vccc102 \transpose A^3  ))) \\
\vec b \equiv (\vc0\psi( \vc1 \psi (\vccc102 \transpose B^3  )))\\
%{OP}^2 \equiv \vec a \;\; _{\bf outer}+\;\; \vec b \\
{OP}^2 \equiv \vec a \outerprod\times\vec b\\
%Result \equiv \vec a + _{\bf inner}\times (\vc2 \psi {OP}^2)\\
r \equiv \vec a \innerprod+\times(\vc2 \psi {OP}^2)\\
\end{eqnarray}
To find (1) shape, then (2) components.
\subsection{Get Shape}
\begin{eqnarray}
\shape (\vec a \innerprod+\times \ldots ) \equiv (\minus1 \drop (\shape \vec a) )\cat (1 \drop (\shape (\vc2 \psi {OP}^2))) \\
\equiv \minus1 \drop \Theta \cat 1 \drop \Theta
\end{eqnarray}
The shape of a vector is a one element vector, and 1 Drop a one element vector is the empty vector, $\Theta$.
Next, we get components based on shape.

$\forall \vec i s.t. 0  \leq^* \vec i <^* \Theta $ which says that the only valid index vector of an array with empty shape, i.e. a scalar is $\Theta$, the empty vector.
\begin{eqnarray}
\Theta \psi \vec a \innerprod+\times \vec b \equiv \reduce+ (\vec a \times (\vc2 \psi {OP}^2))
\end{eqnarray}
This is an identity, $\Theta \psi \xi \equiv \xi$ Now get shape of the multiplication.
\begin{eqnarray}
\shape ((\vec a \times (<2> \psi {OP}^2)) \equiv \shape \vec a \equiv  2 \drop (\shape < 1\;0\;2> \transpose A^3 )\equiv 2 \drop \vccc325 \equiv <5>
\end{eqnarray}
$\forall 0 \leq i <5 $
\begin{eqnarray}
\vc i \psi(\vec a \times \vc2\psi {OP}^2) \equiv \vc i \psi \vec a \times \vc i \psi \vc2 \psi {OP}^2 \\
\vc i \psi \vc1 \psi \vc0 \psi \vccc102 \transpose A^3 \equiv \vccc01i \psi \vccc102 A^3\\
\equiv \vccc10i \psi   A^3\\
\vc i \psi  \vc2 {OP}^2 \equiv \vcc2i \psi {OP}^2\\
\equiv \vcc2i \psi {\vec a \outerprod\times \vec b}
\end{eqnarray}
What is shape of $\vec a \outerprod\times \vec b $?

It is $(2\drop \shape \vccc102\transpose A^3) \cat (2\drop \shape \vccc102 \transpose B^3) \equiv \vcc55$

Therefore, substituting we get:
\[ (\vccc012 \psi A^3) \times \vcc01 \psi B^3\]
This gets the vector we need for the inner product.

Finally we sum the vector for all i.
\[ \reduce+ (\vcc10\psi A^3) \times(( \vccc102 \psi A^3)\times \vcc01 \psi B^3)\]
\begin{verbatim}
  sum((a[1,0,:] * (a[1,0,2]*b[0,1,:])))
\end{verbatim}

\bibliography{paper}
\bibliographystyle{plain}
\end{document}