% !TeX program = pdfLaTeX
\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{graphicx,psfrag,epsf}
\usepackage{enumerate}
\usepackage{natbib}
\usepackage{textcomp}
\usepackage[hyphens]{url} % not crucial - just used below for the URL
\usepackage{hyperref}
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}

%\pdfminorversion=4
% NOTE: To produce blinded version, replace "0" with "1" below.
\newcommand{\blind}{0}

% DON'T change margins - should be 1 inch all around.
\addtolength{\oddsidemargin}{-.5in}%
\addtolength{\evensidemargin}{-.5in}%
\addtolength{\textwidth}{1in}%
\addtolength{\textheight}{1.3in}%
\addtolength{\topmargin}{-.8in}%

%% load any required packages here

\usepackage{setspace}\doublespacing

\begin{document}

\def\spacingset#1{\renewcommand{\baselinestretch}%
{#1}\small\normalsize} \spacingset{1}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\if0\blind
{
  \title{\bf Kaggle-in-class Data Challenges Can Boost Student Learning}

  \author{
        Julia Polak \\
    Department of Statistics, University of Melbourne\\
     and \\     Dianne Cook \\
    Department of Econometrics and Business Statistics, Monash University\\
      }
  \maketitle
} \fi

\if1\blind
{
  \bigskip
  \bigskip
  \bigskip
  \begin{center}
    {\LARGE\bf Kaggle-in-class Data Challenges Can Boost Student Learning}
  \end{center}
  \medskip
} \fi

\bigskip
\begin{abstract}
Kaggle is a data modeling competition service, where participants
compete to build a model with lower predictive error than other
participants. Several years ago they released a simplified service that
is ideal for instructors to run competitions in a classroom setting.
This paper describes the results of an experiment to determine if
participating in a predictive modeling competition enhances learning.
The evidence suggests it does. In addition, students were surveyed to
examine if the competition improved engagement and interest in the
class.
\end{abstract}

\noindent%
{\it Keywords:} instructional technology, statistical modeling, data science, statistics education, data mining
\vfill

\newpage
\spacingset{1.45} % DON'T change the spacing!

\providecommand{\tightlist}{%
    \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}

\section{Introduction}\label{introduction}

Kaggle \citep{kaggle} is a platform for predictive modelling and
analytics competitions where participants compete to produce the best
predictive model for a given data set. It is well-known for its
competitions (e.g. \citet{darkmatter}), some of which come with rich
monetary prizes (e.g. \citet{heritage}). There are also learning
competitions \citep{kagglelearn}, designed to help novices hone their
data mining skills. Winners are typically expected to share their code,
and occasionally newly emerged algorithms are introduced to the broad
community, for example, deep neural networks \citep{HintonDahi12} and
XGBoost \citep{ChenXGBoost}.

In 2015, Kaggle InClass was introduced, as a self-service platform to
conduct competitions. These competitions can be private, limited to
members of a university course, and are easy to setup. This is an
opportunity for educators to provide a vehicle for students to
objectively test their learning of predictive modelling. As a
competition, with an independent clear performance metric, along with a
dynamic leader board, students can see how their model predictions
compare with the models produced by other students. Being able to make
multiple submissions over a several week time frame, enables them to try
out approaches to improve their models. This paper examines the
educational benefits of conducting predictive modeling competitions in
class on performance, engagement and interest.

In the past few years, the educational community started to collect
positive evidence on including competitions in the classroom. None of
these were data analysis competitions. \citet{VanNuland15} ran a
competition assessing anatomical knowledge, as part of an undergraduate
anatomy course. \citet{Calnon12} discusses robotics competitions as part
of computer science education. \citet{Canada15} discusses the
participation of students in externally run artificial intelligence
competitions. All of these studies found significant improvement in
student exam marks accredited to participation in competition.

Classroom competition is an example of active learning, which has been
shown to be pedagogically beneficial. \citet{Prince04} surveyed the
literature and found that all forms of active learning have positive
effect on the learning experience and student achievement. The magnitude
of the effect of different approaches, though, varies. What's more,
\citet{Freeman14} examined 158 studies published in about 50 STEM
educational journals. The authors found that student exam scores
increased by almost half a standard deviation through active learning.
Moreover, students in classes with traditional lecturing were 1.5 times
more likely to fail than their peers in classes with active learning.

A competition, like any other active learning method, that used for
assessment, has its advantages and disadvantages. It brings the `game'
feeling, increases the interest level among students and motivates for
higher performance (\citet{Shindler09}, p 105). However, it may have
negative influence if constructed poorly. Among the negative influences
are increased stress and anxiety, induced by fearing a low ranking,
failure or technology barriers. In addition, students may invest a
disproportionate amount of time and effort into competition. Despite
some received criticism, a properly set competition can benefit the
students greatly. The competition should be relatively short in duration
to avoid consuming undue energy. Students should be clear about the
rules and the goal. They should be properly rewarded and most important,
feel that they have reasonable chance to win or achieve high mark
\citep{Shindler09}.

\citet{Shelley08} raised the need for more quantitative and statistical
analysis of evidence in science education. This paper contributes to
this call by offering statistical analysis of the effects on learning of
classroom data competitions.

\section{Experimental setup}\label{experimental-setup}

\subsection{Data collection}\label{data-collection}

The experiment was conducted during Semester 2 2017. Data was collected
during two classes, one at the University of Melbourne (Computational
statistics and data mining ,MAST90083, denoted as CSDM), and one at
Monash University (Statistical thinking, ETC2420/5242, denoted as ST).

\subsection{Competition data}\label{competition-data}

Two data sets were compiled for the kaggle challenges: Melbourne
property auction prices and spam classification. The Melbourne auction
price data was collected by extracting information from real estate
auction reports (pdf) collected between Feb 2, 2013 and Dec 17, 2016.
The spam classification data was compiled by graduate students at Iowa
State University as part of a data mining class in 2009. Data was
compiled by monitoring and extracting information from their emails by
class members, over a period of a week, and manually tagging them as
spam or ham.

Both data sets were split into training and test sets, for the kaggle
challenge. Students had access to the true response variable only for
the training data. For the Melbourne housing data, students were
expected to predict price based on the property characteristics. For the
spam data, students were expected to build a classifier to predict
whether the email as spam or not.

Both data sets are challenging for prediction, with relatively high
error rates.

The training and the testing data sets of the Melbourne auction price
data were similar but not identical across the two institutions. Some of
the variables in the data set were simulated, e.g.~property land size
and house size. The simulated data was generated slightly differently
for different institutions. This was done deliberately to prevent
students passing answers from one institution to another.

\subsection{Participants}\label{participants}

Computational Statistics and Data Mining (CSDM) is designed for
postgraduate level students with math, statistics, information
technology or actuarial backgrounds. It covers modelling both continuous
(regression) and categorical (classification) response variables. The 63
students were randomized into one of two kaggle competitions, one
focused on regression (R) and the other classification (C). (One of the
63 students elected not to take part in the competition, and another
student didn't sit the exam, producing a final sample size of 61.) This
setup mimics randomized control trials, which is the gold standard, in
experiment design (\citet{Shelley09CH1}, ch.~1). Students built
prediction models and made submissions individually for 16 days, and
then were allowed to form groups to compete for another 7 days.

The reason for this strategy was first to motivate each of the students
to think about modelling and be actively engaged in the competitions
through individual submission. The lecturer allowed participants to
create groups towards the end of the competition to illustrate the
advantages of group work and ensemble models. Another reason for this
approach was the university policy, requiring a strategy to assign
students individually in group assignments. When the team members
develop the model together, it is quite difficult to accurately assess
the individual contribution of each student. The individual submissions
helped to encourage each student to engage in the modelling process. In
addition, it helped to assess the individual component of the final
score for the competition.

Table 1 compares the summary statistics for the two groups. We examine
the percentage correct overall on the final exam for the different
groups and the scores the students received for the second assignment.
The second assignment examined students knowledge about computational
methods, unrelated to the classification and regression methods. The two
groups statistics are similar.

\begin{table}[h]
\begin{center}
\begin{tabular}{l l | r | r | r r r r r}\hline
Score & Comp. group & N & Mean (SD)   & Min  &  Q1 & Median  &  Q3  & Max \\\hline
Exam   & Regression      & 30 &  66.9 (14.1)  &   36.0 &  57.5 &   69.2 &  77.5 &   88.5\\
       & Classification & 31 &  62.1 (17.2)  &   20.0  & 52.2  &  68.0   &  75.2 &  85.5\\\hline

2$^{nd}$ assignment & Regression      &    30  & 8.2 (2.2)  &  0.0  & 7.0 &  8.4   &  9.9  &   10.0 \\
                    & Classification  &    31  & 8.0 (3.0)  &  0.0  & 7.8  &  9.3   &  9.9  &   10.0 \\\hline

\end{tabular}
\caption{Computational statistics and data mining: Summary statistics of the exam score (out of 100) and the second assignment (out of 10) for the two competition groups.}
\end{center}
\label{tab:Melb_Group_Compare}
\end{table}

Statistical Thinking (ST), covers regression, but not classification,
and has a mix of undergraduate and postgraduate students. Only the 34
postgraduate (ST-PG) students were required to participate in the kaggle
competition, and competed in the regression (R) challenge. This was run
independently from the CSDM competition. The 145 undergraduate (ST-UG)
students were used as comparison for examining performance of the
postgraduate students. Although, it may be surprising at first glance,
the undergraduate students provide a reasonable comparison for the
graduate students. The class is taught to both cohorts simultaneously.
The entry requirements to the Bachelors of Commerce at Monash is high,
and these students have strong mathematics backgrounds. In the years
prior to this experiment, the undergraduate scores on the final exam are
indistinguishable from those of the graduate students. The competition
ran for one month. Students formed their own teams of 2-4 members to
compete.

Table 2 shows the summary statistics of the exam scores for the 34
postgraduate (ST-PG) students and for the 145 undergraduate (ST-UG)
students. The mean and the median scores of postgraduate students are a
bit lower than the corresponding scores of undergraduate students.
However, the interquartile range is similar. Interestingly, the highest
exam score was received by an undergraduate student.

\begin{table}[h]
\begin{center}
\begin{tabular}{l l | r | r | r r r r r}\hline
Score & Competition group & N & Mean (SD)   & Min  &  Q1 & Median  &  Q3  & Max \\\hline
Exam   & ST-PG      & 34  & 61.8 (12.2)  &  35.0 & 57.2  &   61.0  & 72.2  & 81.5 \\
       & ST-UG         & 145 & 63.0 (15.4)  &   0.0 & 56.0  &   64.0  & 73.0   & 90.0\\\hline
\end{tabular}
\caption{Statistical thinking: Summary statistics of the exam score (out of 100) for the two groups.}
\end{center}
\label{tab:Melb_Group_Compare}
\end{table}

Figure \ref{fig:Monash_quiz} displays the quiz scores taken by the
undergraduate (ST-UG) and postgraduate (ST-PG) students, as a beeswarm
plot \citep{beeswarm}. The distribution of quiz scores are similar,
except that undergraduates tend to have a longer lower tail.

\begin{figure}
\centering
\includegraphics{paper-kaggle_files/figure-latex/Monash_quiz-1.pdf}
\caption{\label{fig:Monash_quiz}Beeswarm plots of undergraduate (ST-UG)
and postgraduate (ST-PG) students quizzes during the semester. The
scores are very similar, although undergraduates have a longer lower
tail.}
\end{figure}

\section{Methodology}\label{methodology}

\subsection{Performance}\label{performance}

Better performance is equated to better understanding of the material,
as measured in the final exam. CSDM and ST included questions, with
several parts, on the final exam related to kaggle challenges. These
questions were identified prior to data analysis.

For all questions in the exam, difficulty and discrimination scores were
computed, using the mean and standard deviations. Of the questions
pre-identified as being relevant to the data challenges, only the parts
that corresponded to high level of difficulty and high discrimination
were included in the comparison of performance.

Scores for the relevant questions were summed, and converted into
percentage of the possible score. The total exam score was converted to
a percentage. One can expect, that on average, student's success rate
for each question will be about the same as the success rate in the
total exam. Understanding one topic better than another will result in
higher success rate for questions asking about the better understood
topic compared to the scores for other topics. For example, we would
expect from a student with a 70\% exam mark to get 70\% marks on each of
the questions in the exam, if she has similar knowledge level on all the
exam topics. If in some topic, say regression, the student has better
knowledge, she will perform better on the regression questions. Her
success rate on regression question will be higher than 70\%.
Consequently, her performance on some other questions should be below
70\% which is associated with lesser understanding of these topics.

Therefore, performance for each student was computed as the ratio of
these two numbers, percentage success in the regression (classification)
questions and percentage success in the total exam. A value of 1 would
indicate that the student's performance on that set of questions was
consistent with their overall exam performance, greater than 1 that they
performed better than expected, and lower than 1 meant less than
expected on that topic.

Using only the percentage of successes for each set of questions,
instead of the proposed ratio, will not differentiate between a better
performance and just a better student. Especially in the case of ST that
have a mixed population of master and undergraduate student.

The distribution of the performance scores by group is shown as a
boxplot. Focus is on the difference in median between the groups.
Permutation tests were conducted to examine difference in median scores
for students participating or not in a competition.

\subsection{Engagement}\label{engagement}

The students were allowed to submit at most one prediction per day,
while the competitions were open. The frequency of submissions, and the
accuracy (or error) of their predictions, made by individual students,
is recorded as a part of the kaggle system. To examine whether
engagement improved performance, scores on the questions related to the
competition normalised by total exam score (as computed in the
performance section) is examined in relation to frequency of submissions
during the competition. In addition, performance in the competition as
measured by accuracy or error, is also examined in relation to the
number of submissions. Scatterplots, correlation and linear models are
used to examine the associations.

\subsection{Interest}\label{interest}

Students in CSDM and ST-PG were invited to give feedback about the
course, in particular about the data competitions, before the final
exam. This information was voluntary, and students who completed the
questionnaire were rewarded with a coupon for a free coffee. The data
from this survey was viewed by the researchers after all course grades
had been reported. To reduce potential bias in students replies, we
emphasize this point as part of the instruction at the beginning of the
survey.

\section{Results}\label{results}

\subsection{Performance}\label{performance-1}

Figure \ref{fig:MAST90083} shows the data collected in CSDM. Performance
is plotted against type of question, separately for the competition they
completed. The difference in median scores indicates performance
improvement. Students who completed the classification competition
(left) performed relatively better on the classification questions than
the regression questions in the final exam. Conversely, students who
participated in the regression competition performed relatively better
on the regression questions.

\begin{figure}
\centering
\includegraphics{paper-kaggle_files/figure-latex/Melb-1.pdf}
\caption{\label{fig:MAST90083} Boxplots of performance on regression and
classification questions in the final exam, by type of data competition
completed in CSDM. Students generally performed better on the questions
corresponding to the competition they participated in.}
\end{figure}

\begin{table}[h]
\begin{center}
\begin{tabular}{|l|r|r|}\hline
Question Set & Median difference & Permutation $p$-value \\\hline
Classification & 0.250 & 0.033\\
Regression & 0.104 & 0.000\\\hline
\end{tabular}
\caption{Comparison of median difference in performance by competition group, for CSDM students, using permutation tests. Both sets of medians are significantly different, indicating improved scores for questions on the topic related to the Kaggle competition.}
\end{center}
\label{tab:Melb_Perm}
\end{table}

Table 3 shows the results of permutation testing of median difference
between the groups. Generally the results support the competition
improved performance. Students who participated in the kaggle challenge
for classification scored higher than those that did the regression
competition, on the classification problem. Using a permutation test,
this corresponds to a significant difference in medians. Similarly the
results show that students who did the regression challenge, performed
better on these exam questions.

Figure \ref{fig:ETC5242} shows the results for ST students. The boxplots
suggest that the students who participated in the challenge performed
relatively better than those that didn't on the regression question than
expected given their total exam performance.

Only the post-graduate students participated in the regression
competition, as their additional assessment requirement. Scores for the
question on regression (Q7a,b,c) in the final exam were compared with
the total exam score (RE). On these question parts, a, b, c, over all
the students all three were in the top 10 of difficulty, with students
scoring less than 70\%, on average. Parts b, c were in the top 10 for
discrimination, and part a was at rank 13.

\begin{figure}
\centering
\includegraphics{paper-kaggle_files/figure-latex/ETC5242-1.pdf}
\caption{\label{fig:ETC5242} Performance for regression question
relative to total exam score for students who did and didn't do the
regression data competition in Statistical thinking.}
\end{figure}

Based on the median, the students who participated in the kaggle
challenge scored 0.09 higher than those that didn't, a median of 1.01 in
comparison to 0.92. Using a permutation test, this corresponds to a
significant difference in medians, with \(p\)-value of 0.02.

\begin{figure}
\centering
\includegraphics{paper-kaggle_files/figure-latex/crossperformances-1.pdf}
\caption{\label{fig:crossperformances} Students performance in
classification and regression questions by competition type.}
\end{figure}

Figure \ref{fig:crossperformances} presents students' scores for
classification and regression questions. A score over 1 is considered as
outperform (relative to the expectation). Quarters one and three include
students that underperform or outperform on both types of questions,
respectively. In both cases the number of students that participated in
the classification competition is very close to the number of students
that participated in the regression competition (excluding a few
regression students on the border of score 1). Students in quarters two
and four outperform on one type of questions but not on the other type.
We can see that more regression students outperform on regression
questions than classification students (12 vs. 7). Similarly,
classification students do better on classification questions (11 vs.
3). This is another evidence towards positive influence of the data
competition on student's performances.

\subsection{Engagement}\label{engagement-1}

The number of submissions that a student made, may be an indicator of
performance on the exam questions related to the competition. A student
who is more engaged in the competition may learn more about the
material, and consequently perform better on the exam. Figure
\ref{fig:numsubmition} (top row) shows performance on the classification
and regression questions, respectively, against their frequency of
prediction submissions for the three student groups (CSDM classification
and regression, ST-PG regression) competitions. The relationship is weak
in all groups, and this mirrors insignificant results from a linear
model fit to both subsets. On the other hand, the predictive accuracy
improved with the number of submissions for the regression competitions.
There appears to be some nonlinearity present in these plots, suggesting
reduced returns. That is reasonable to expect. Also, some students
strategically make very poor initial predictions, to get a baseline on
error equivalent to guessing.

The competition performance relative to number of submissions is shown
in plots (d)-(f). Each point corresponds to one student, and accuracy or
error of the best predictions submitted is used. The regression
competition seemed to engage students more than the classification
challenge. Students submitted more predictions, and their models
improved with more submissions.

\begin{figure}
\centering
\includegraphics{paper-kaggle_files/figure-latex/numsubmition-1.pdf}
\caption{\label{fig:numsubmition} Scatterplots of the exam performance
(a-c) and competition performance (d-f) by number of prediction
submissions, for the three student groups. The relationships with exam
performance are weak. For the CSDM and ST-PG regression competitions, a
clear pattern is that predictions improved substantially with more
submissions. (House price in ST-PG were divided by 100,000, explaining
the difference in magnitude of error between two competitions.)}
\end{figure}

\subsection{Interest}\label{interest-1}

Figure \ref{fig:likert} shows the survey responses related to the kaggle
competition, for CSDM and ST-PG. The response rate for CSDM was 55\%,
with 34 of 61 students completing the survey. The response rate for
ST-PG was 50\%, 17 students out of 34 completed the survey.
Overwhelmingly, students reported that they found the competition
interesting and helpful for their learning in the course.

\begin{figure}
\centering
\includegraphics{paper-kaggle_files/figure-latex/likert-1.pdf}
\caption{\label{fig:likert}Summary of responses to survey of kaggle
competition participants. Overwhelmingly the response to the competition
was positive in both classes, especially the questions on enjoyment and
engagement in the class, and obtaining practical experience. (Table 4
lists the questions.)}
\end{figure}

\begin{table}[b!]
\begin{center}
\begin{tabular}{|l|p{15cm}|}\hline
Label & Question \\\hline
Q1 & I found the data competition is great fun.\\\hline
Q2 & Taking part in the data competition contributed a lot to my engagement with the subject.\\\hline
Q3 & Taking part in the data competition improved my confidence in my understanding of the covered material.\\\hline
Q4 & Taking part in the data competition improved my confidence in my success in the final exam.\\\hline
Q5 & Taking part in the data competition improved my confidence in my ability to use the acquired knowledge in practical applications.\\\hline
Q6 & I feel that the required time investment in the data competition was worthy. \\\hline
\end{tabular}
\end{center}
\caption{Questions asked in the survey of competition participants.}
\label{survey_Q}
\end{table}

After collecting the survey from the students we were told that the
questions about students engagement were positively worded. This may
have potentially led to some bias. An improved way could be to ask
directly about student's engagement, e.g. ``How would you rate your
level of engagement in this course?'' with set answer options of ``not
at all engaged'' -- up to ``extremely engaged'' with several choices in
between. Another improvement could be asking ST-UG students that didn't
take part in the competition about their level of engagement and compare
the answers with other students of ST-PG. We acknowledge that the
differences in the engagement levels may not necessarily be a result of
participation in the competition but it is still an interesting aspect.

The survey was not anonymous. However, the results became available to
the lectures only after all the grades were realised to the students.
This point was emphasized in the instructions to the students at the
beginning of the survey.

\section{Teachers' corner}\label{teachers-corner}

Creating a new competition is surprisingly easy. Information on setting
up a Kaggle InClass challenge is available on the service's web site
(\url{https://www.kaggle.com/about/inclass/overview}). There is a setup
wizard for step-by-step guidance on getting your competition underway.
We have created a short video illustrating the steps to establish a new
competition, available on the web
(\url{https://www.youtube.com/watch?v=tqbps4vq2Mc&t=32s}).

The kaggle service provides some data sets, primarily for student
self-learning. These are not suitable for use in a class challenge,
because all the data is available, and solutions are also provided. We
recommend providing your own data for the class challenge. Finding a
suitable data set for a competition can be a difficult task. The
criteria for a good data set are:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  the full set is not available to the students, to avoid plagiarism and
  use of unauthorized assistance.
\item
  the data is not too easy, or too hard, to model so that there is some
  discriminatory power in the results.
\item
  data should be relatively clean, to the point where the instructor has
  tested that a model can be fitted.
\item
  contains some challenges, that make standard off-the-shelf modeling
  less successful, like different variable types that need processing or
  transforming, some outliers, large number of variables.
\item
  if it is a classification challenge, it will work better with
  relatively balanced classes, because the overall accuracy is the
  easiest metric to use.
\end{enumerate}

The data sets used in our competitions can be shared with other
instructors by request.

Choosing the metric upon which to evaluate the model is another
decision. Our advice is to keep it simple, so you, and the students, can
understand the student scores. If you are running a regression
challenge, then the ``Root Mean Squared Error (RMSE)'' is a good choice.
If it is a classification challenge, then ``Categorization Accuracy'',
the percent of correct classifications, is reasonable.

The data needs to be split into training and testing sets. Kaggle will
then split your test set into two, a public set that is used to provide
ongoing scores to participants, and a private set, on which performance
is revealed only after the competition closes. If you have categorical
variables in the data set, you will want to make sure that all
categories are present in both training and test sets. The training set
will have both predictors and response, but the test set will have the
response variable removed. Each observation needs to be assigned an id,
because this will be needed to evaluate predictions. The solution file,
containing the id and the true response, is provided to the system for
evaluating submissions, and is kept private. A sample submission file
needs to be provided. Participants will submit their solutions in the
same format.

It is a good idea to build a basic model yourself on the training data,
and predict the test data. The performance of this model can be provided
to the participants as baseline to beat.

The competition needs to run without any intervention from the
instructor. Exception is, of course, an academic discussion motivated by
the competition between the teaching team and the students. For example,
a discussion about different models, their advantages and limitations.
The instructor can monitor students progress: the number of submissions,
student scores and even the uploaded data at any time. When the
competition ends the ``Leaderboard'' page provides a list of students
ordered by the final score. It also provides all the scores from all
past submissions (under ``Raw Data'' on ``Public Leaderboard'').

It may be recommended to limit students to one submission per day. It
encourages students to think about more efficient improvement of their
model before the next submission. It also prevents the student spending
too much time building and submitting models. Some students will become
so engaged in the competition that they might neglect their other
coursework. About halfway through the competition, students might be
allowed to form teams, to learn how averaging models can boost
performance.

In awarding course points to student effort, we typically align it to
performance. Participant ranks based on their performance on the private
part of the test data are recorded. Performance scores that are pretty
close to each other should be given the same rank, reflecting that there
may not be a significant difference between them. The best gets perhaps,
5 points, then a half a point drop until about 2.5 points, so that the
worst performing students still get 50\% for the task. Kaggle does not
allow you to download participants email addresses. All you see is their
kaggle name. Record the student names in kaggle to match with your class
records.

Along with the competition, students were expected to submit a report
that explained their modeling strategy, and what they had learned about
the data beyond the modeling. The overall score for this part of the
course was a combination of the mark for their report and their
performance in the challenge. In both courses this accounted for 10\% of
the final mark.

From an instructor perspective, its very rewarding watching the students
participate in the competition. It provides a truly objective way to
assess their ability to model in practice. Students are often motivated
to consult with the instructor about why their model is underperforming,
or what other approaches might produce better results.

\section{Discussion}\label{discussion}

This paper has described an experiment to examine the effectiveness of
data competitions on student learning, using Kaggle InClass as the
vehicle for conducting the competition. The primary finding, is that
participating in a data challenge competition, produces a statistically
significant improvement in the learning of the topic, although the
effect size is small. Secondarily, the competitions enhanced interest
and engagement in the course.

Because the experiment was conducted in the classroom setting as part of
the normal teaching of the courses, which imposed limitations on the
design. However, performance comparison was enabled in CSDM by a
randomized assignment of students to two topic groups, and in ST by
using a control group. In CDSM, the group sizes were relatively small,
approximately 30 students per group. For ST the control group was the
undergraduate students that took class. Using undergraduate students as
a control group for graduate students may be surprising. However, the
experience of teaching this subject over several years and some
statistical comparison of the two groups justifies the approach.
Undergraduate students performance in other tasks and exam questions,
not relevant to the competition, was equivalent to the postgraduate
students cohort.

\section{Future work}\label{future-work}

This work is one of a few quantitative analyses of data competition
influences on students' performance. More evidence needs to be collected
from other STEM courses to explore consistent positive influence.
Moreover, future investigation is required to understand the influence
of the different aspects of data competition implementation on the
magnitude of the performance improvement. For example, the competition
duration, availability and accessibility of additional material,
requirement of writing a final report or giving a short oral
presentation are elements worth investigating. Prior and post testing of
students might provide a better experimental design.

\section{Acknowledgments}\label{acknowledgments}

This project (title: Effect of Data Competition on Learning Experience)
has been approved by the Faculty of Science Human Ethics Advisory Group
University of AB (ID: 1749858.1 on September 4, 2017) and by CD
University Human Research Ethics Committee (ID: 9985 on August 24,
2017).

This document was produced in R \citep{R} with the package knitr
\citep{knitr}. Data cleaning was conducted using tidyr \citep{tidyr},
dplyr \citep{dplyr} and plots were made with ggplot2 \citep{ggplot2}.
The materials to reproduce the work are available at
\url{https://github.com/dicook/paper-quoll}.

\bibliographystyle{agsm}
\bibliography{bibliography.bib}

\end{document}


\documentclass[12pt]{article}

\include{preamble}

\title{Math 621 Fall 2017 \\ Midterm Examination Two}
\author{Professor Adam Kapelner}

\date{November 14, 2017}

\begin{document}
\maketitle

\noindent Full Name \line(1,0){410}% ~~~ Section (A, B or C)~ \line(1,0){30}

\thispagestyle{empty}

\section*{Code of Academic Integrity}

\footnotesize
Since the college is an academic community, its fundamental purpose is the pursuit of knowledge. Essential to the success of this educational mission is a commitment to the principles of academic integrity. Every member of the college community is responsible for upholding the highest standards of honesty at all times. Students, as members of the community, are also responsible for adhering to the principles and spirit of the following Code of Academic Integrity.

Activities that have the effect or intention of interfering with education, pursuit of knowledge, or fair evaluation of a student's performance are prohibited. Examples of such activities include but are not limited to the following definitions:

\paragraph{Cheating} Using or attempting to use unauthorized assistance, material, or study aids in examinations or other academic work or preventing, or attempting to prevent, another from using authorized assistance, material, or study aids. Example: using an \emph{unauthorized} cheat sheet in a quiz or exam, altering a graded exam and resubmitting it for a better grade, etc.
\\

\noindent I acknowledge and agree to uphold this Code of Academic Integrity. \\

\begin{center}
\line(1,0){250} ~~~ \line(1,0){100}\\
~~~~~~~~~~~~~~~~~~~~~signature~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ date
\end{center}

\normalsize

\section*{Instructions}

This exam is seventy five minutes and closed-book. You are allowed one 8.5'' $\times$ 11'' page (front and back) of a \qu{cheat sheet.} You may use a graphing calculator of your choice. Please read the questions carefully. If the question reads \qu{compute,} this means the solution will be a number otherwise you can leave the answer in choose, permutation, exponent, factorial or any other notation which could be resolved to a number with a computer. I advise you to skip problems marked \qu{[Extra Credit]} until you have finished the other questions on the exam, then loop back and plug in all the holes. I also advise you to use pencil. The exam is 100 points total plus extra credit. Partial credit will be granted for incomplete answers on most of the questions. \fbox{Box} in your final answers. Good luck!

\pagebreak

\problem Below are some theoretical questions.

\benum

%\subquestionwithpoints{6} Let $X \sim \exponential{\lambda}$ and $Y = \sqrt{X}$. Find $f_Y(y)$ and its support. If $Y$ is a brand name r.v., indicate it with the notation used in class. \spc{6}

\subquestionwithpoints{10} Let $X \sim \betanot{\alpha}{\beta}$ and $Y = 1 - X$. (i) Find $\support{Y}$. (ii) Find $f_Y(y)$. (iii) Simplify $f_Y(y)$. (iv) If $Y$ is a brand name r.v., indicate it with the notation used in class. \spc{10}

\subquestionwithpoints{10} Let $X \sim \logistic{0}{1}$ and $Y = e^X$ i.e. the log-logistic distribution analogous to the log-normal distribution. (i) Find $\support{Y}$. (ii) Find $f_Y(y)$. (iii) Simplify $f_Y(y)$. \spc{10}

\subquestionwithpoints{6} If $Y~|~X = x \sim \normnot{0}{\oneover{x}}$ and $X \sim \exponential{1}$, draw a tree diagram for $X,Y$.\spc{4}

\subquestionwithpoints{10} As given previously, if $Y~|~X = x \sim \normnot{0}{\oneover{x}}$ and $X \sim \exponential{1}$. (i) Find $\support{Y}$. (ii) Find $f_Y(y)$. (iii) Simplify $f_Y(y)$. (iv) If $Y$ is a brand name r.v., indicate it with the notation used in class. \spc{16}

\subquestionwithpoints{10} Let $X \sim \laplace{0}{1}$ and $Y = X\indic{X \geq 0}$. (i) Find $\support{Y}$. (ii) Find $f_Y(y)$. (iii) Simplify $f_Y(y)$. (iv) If $Y$ is a brand name r.v., indicate it with the notation used in class. \spc{10}

\subquestionwithpoints{10} Consider modeling human survival (with unit years) by $T \sim \weibull{\lambda}{k}$. Make up values of $k$ and $\lambda$ that are appropriate for this modeling challenge. Note that $\expe{T} = \oneover{\lambda}\Gammaf{1 + \oneover{k}}$. Explain why you chose these $k$ and $\lambda$ values. \spc{10}

\subquestionwithpoints{4} Why wouldn't $W \sim \gumbel{\mu}{\beta}$ be appropriate for modeling human survival (with unit years)? \spc{2}

\subquestionwithpoints{12} If $Z_1, Z_2 \iid \stdnormnot$ and $Y = \squared{Z_1 / Z_2}$. (i) Find $\support{Y}$. (ii) Find $f_Y(y)$. (iii) Simplify $f_Y(y)$. (iv) If $Y$ is a brand name r.v., indicate it with the notation used in class. \spc{5}

\subquestionwithpoints{12} Let $\Xoneton \iid \gammadist{1}{\beta}$ and $Y = \min{\Xoneton}$. (i) Find $\support{Y}$. (ii) Find $f_Y(y)$. (iii) Simplify $f_Y(y)$. (iv) If $Y$ is a brand name r.v., indicate it with the notation used in class. \spc{12}

\subquestionwithpoints{12} Let $X \sim \stduniform$ independent of $Y \sim \stduniform$ and $R = \frac{X}{Y}$. (i) Find $\support{R}$. (ii) Find $f_R(r)$. (iii) Simplify $f_R(r)$. (iv) If $R$ is a brand name r.v., indicate it with the notation used in class. \spc{11}

%\subquestionwithpoints{4} Evaluate $I_{\half}(2,2)$. \spc{3}

\eenum

\problem There are 5 class lectures left in Math 621. Write below about what kind of probability material \emph{you} want to learn. \ingray{[4 pt / 100 pts]}

\end{document}


\documentclass{tufte-handout}
\usepackage{amsmath}
\pagestyle{empty}
\usepackage[utf8]{inputenc}
\usepackage{mathpazo}
\usepackage{microtype}

\usepackage{tikz}
\usetikzlibrary{matrix}
\usetikzlibrary{chains}
\usetikzlibrary{decorations}

\title{Stable Matching}
\author{Thore Husfeldt}

\begin{document}

\maketitle

\subsection{Description}
Implement Gale–Shapley’s algorithm for stable matching and run it on several test inputs.

\begin{marginfigure}
\includegraphics[width=2in]{791px-Jealousy_and_Flirtation.jpg}
\caption{\emph{Jealousy and Flirtation} (1874) by Haynes King (1831-1904).
Source: Wikimedia Commons. Public domain.}
\end{marginfigure}

\subsection{Requirements}
Your programme has to take its input either from standard input (STDIN) or from a specified file.
It has to write to standard output (STDOUT).
Assuming all the files are in the right place and you wrote your code in Java, you must be able to run the programme from the command line with at least one of the following two commands:
\begin{verbatim}
  % java GS < sm-kt-p-4.in
  % java GS sm-kt-p-4.in
\end{verbatim}

\paragraph{Correctness}
The course data file directory contains a number of pairs of files called “{\tt sm-*-in.txt}” and “{\tt sm-*-out.txt}” that contain matching input and expected output.
Your solution must be consistent with these files.
(The ordering of your output makes no matter.)
Your code has to be as clear and crisp as possible.
Your solution must run in time $O(n^2)$, where $n$ is the number of women (and men).
In particular, this means that your solution must be able to handle the pairwise comparison of two men in a woman’s list of preferences in constant time.
Section [KT, 2.3] explains how to do this.

\section{Questions}

\begin{description}
  \item[What about coding paradigms?]
    I don’t much care.
    You can follow the book and make an array-based solution, where people are basically integers that are used as indices into various arrays.
    You can also try for an object-oriented approach where people are objects (or even make Person is an abstract class or an interface, implemented by Man and Woman), and the data structures use a standard collections package.
    Maybe there’s even a beatiful functional solution.
  \item[Style?]
    Your code should be as neat as you can make it.
    Don’t declare variables or fields or methods that you never use.
  \item[What about documentation?]
    I don’t much care.
    I’d much rather have your code be as short and sweet as possible, instead of full of comments.
    Especially, don’t waste time explaining the algorithm---the expected audience has read the book!
\end{description}

\section{Tips}

Algorithmically, this is supposed to be an easy exercise.
The algorithm is simple and spelt out in detail on p. 6 in the book, and many implementation issues are addressed in section 2.3.
However, what is probably going to take you a lot of time is reading the input, especially if you haven’t done that kind of thing before.
I suggest the following:
\begin{enumerate}
  \item First, hard-code the input, in a convenient data structure, while
you get the algorithm to work.
This will allow you to change your mind about how the data structures should look.
Only when everything seems to look OK, read (and parse!) your input from standard input.
\item If you do this in Java, learn {\tt java.util.Scanner}.
  You want something like {\tt Scanner SC = new Scanner(System.in)}.
\item When debugging, don’t throw away your test cases!
  Instead, make additional input–output file pairs.
\item Write another program that runs your Gale--Shapley implementation on all test files (both the ones I have made and those you may have produced during debugging) and checks them against the corresponding output files.
  This requires nontrivial file operations (opening matching filenames, reading directories, comparing files), and Java may not be the best language to do that in.
  (You could learn Perl or shell scripting.)
\item What annoyed me most about input are the indices.
  In my array-based solution, after changing my minds a million times, I settled for indexing the men by $\{0,\ldots,n-1.$ and the women $\{0,\ldots,n-1\}$.
    This made the parsing stage slightly more complicated, because the input file uses a different convention, but cleaned up the rest of my code a lot.
  \end{enumerate}
\subsection{Deliverables}

\begin{enumerate}
  \item The source code for your implementation
  \item A report in PDF.
  Use the report skeleton in the {\tt doc} directory.
  \end{enumerate}

\end{document}


\section{Related Work}
\label{sec:related-work}

To the best of our knowledge, the paper written by \citet{breiman1997arcing}  is one of the earliest in the GBDT field. It comprises the idea of viewing gradient boosting as the optimization algorithm minimizing certain cost function. This concept was subsequently developed by \citet{friedman2001greedy} who proposed to make the connection between stagewise model expansions and steepest-descent minimization and to consider boosting algorithms as iterative functional gradient descent algorithms. The intuition behind that work is that function minimization can be performed by repeatedly adding an imperfect model pointing to the direction of negative gradient to the existing one so as to correct the errors of the predecessors.

The gradient boosting is frequently used with decision trees as base functions. To determine the next base predictor, the enumeration of all possible splits on all the features has to be performed. The GBDT implementation provided by scikit-learn \cite{scikit-learn} employs exact greedy algorithm for choosing the predictor contributing the greatest score improvement. Since complete enumeration is computationally demanding, \citet{chen2016xgboost} propose an implementation called XGBoost which comprises an approximate algorithm for enumerating all possible splits in continuous features and several system design optimizations. These improvements allow to create a well-scalable model which outperforms the models existing by that time together with significant training process speed-up.

One of the most recent studies by \citet{DBLP:journals/corr/DorogushGGKPV17}, \cite{dorogushcatboost} proposes a new gradient-boosting library CatBoost which incorporates information contained in categorical features establishing a new strategy of computing the statistics for the category labels. Another crucial enhancement is a new way of
identifying and solving a bias problem in pointwise gradient estimates which is present in classical boosting algorithms and their existing implementations. Proposed implementation outperforms the existing libraries in terms of prediction quality on several benchmarks in classification tasks without any regards to regression problems which served us a motivation to perform a research in this particular area.


%% LyX 2.3.0 created this file.  For more info, see http://www.lyx.org/.
%% Do not edit unless you really know what you are doing.
\documentclass[ruled]{article}
\usepackage{courier}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage[letterpaper]{geometry}
\geometry{verbose}
\usepackage{color}
\usepackage{url}
\usepackage{algorithm2e}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage[unicode=true,
 bookmarks=false,
 breaklinks=false,pdfborder={0 0 1},backref=section,colorlinks=true]
 {hyperref}

\makeatletter

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% LyX specific LaTeX commands.
\providecommand{\LyX}{\texorpdfstring%
  {L\kern-.1667em\lower.25em\hbox{Y}\kern-.125emX\@}
  {LyX}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Textclass specific LaTeX commands.
\newcommand{\code}[1]{\texttt{#1}}

\@ifundefined{date}{}{\date{}}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% User specified LaTeX commands.
\definecolor{mygreen}{rgb}{0,0.6,0}
\definecolor{mygray}{rgb}{0.5,0.5,0.5}
\definecolor{mymauve}{rgb}{0.58,0,0.82}

\makeatother

\begin{document}
\global\long\def\reals{\mathbf{R}}
 \global\long\def\integers{\mathbf{Z}}
\global\long\def\naturals{\mathbf{N}}
 \global\long\def\rationals{\mathbf{Q}}
\global\long\def\ca{\mathcal{A}}
\global\long\def\cb{\mathcal{B}}
 \global\long\def\cc{\mathcal{C}}
 \global\long\def\cd{\mathcal{D}}
\global\long\def\ce{\mathcal{E}}
\global\long\def\cf{\mathcal{F}}
\global\long\def\cg{\mathcal{G}}
\global\long\def\ch{\mathcal{H}}
\global\long\def\ci{\mathcal{I}}
\global\long\def\cj{\mathcal{J}}
\global\long\def\ck{\mathcal{K}}
\global\long\def\cl{\mathcal{L}}
\global\long\def\cm{\mathcal{M}}
\global\long\def\cn{\mathcal{N}}
\global\long\def\co{\mathcal{O}}
\global\long\def\cp{\mathcal{P}}
\global\long\def\cq{\mathcal{Q}}
\global\long\def\calr{\mathcal{R}}
\global\long\def\cs{\mathcal{S}}
\global\long\def\ct{\mathcal{T}}
\global\long\def\cu{\mathcal{U}}
\global\long\def\cv{\mathcal{V}}
\global\long\def\cw{\mathcal{W}}
\global\long\def\cx{\mathcal{X}}
\global\long\def\cy{\mathcal{Y}}
\global\long\def\cz{\mathcal{Z}}
\global\long\def\ind#1{1(#1)}
\global\long\def\pr{\mathbb{P}}

\global\long\def\ex{\mathbb{E}}
\global\long\def\var{\textrm{Var}}
\global\long\def\cov{\textrm{Cov}}
\global\long\def\sgn{\textrm{sgn}}
\global\long\def\sign{\textrm{sign}}
\global\long\def\kl{\textrm{KL}}
\global\long\def\law{\mathcal{L}}
\global\long\def\eps{\varepsilon}
\global\long\def\convd{\stackrel{d}{\to}}
\global\long\def\eqd{\stackrel{d}{=}}
\global\long\def\del{\nabla}
\global\long\def\loss{\ell}
\global\long\def\tr{\operatorname{tr}}
\global\long\def\trace{\operatorname{trace}}
\global\long\def\diag{\text{diag}}
\global\long\def\rank{\text{rank}}
\global\long\def\linspan{\text{span}}
\global\long\def\proj{\text{Proj}}
\global\long\def\argmax{\operatornamewithlimits{arg\, max}}
\global\long\def\argmin{\operatornamewithlimits{arg\, min}}
\global\long\def\bfx{\mathbf{x}}
\global\long\def\bfy{\mathbf{y}}
\global\long\def\bfl{\mathbf{\lambda}}
\global\long\def\bfm{\mathbf{\mu}}
\global\long\def\calL{\mathcal{L}}
\global\long\def\vw{\boldsymbol{w}}
\global\long\def\vx{\boldsymbol{x}}
\global\long\def\vxi{\boldsymbol{\xi}}
\global\long\def\valpha{\boldsymbol{\alpha}}
\global\long\def\vbeta{\boldsymbol{\beta}}
\global\long\def\vsigma{\boldsymbol{\sigma}}
\global\long\def\vmu{\boldsymbol{\mu}}
\global\long\def\vtheta{\boldsymbol{\theta}}
\global\long\def\vd{\boldsymbol{d}}
\global\long\def\vs{\boldsymbol{s}}
\global\long\def\vt{\boldsymbol{t}}
\global\long\def\vh{\boldsymbol{h}}
\global\long\def\ve{\boldsymbol{e}}
\global\long\def\vf{\boldsymbol{f}}
\global\long\def\vg{\boldsymbol{g}}
\global\long\def\vz{\boldsymbol{z}}
\global\long\def\vk{\boldsymbol{k}}
\global\long\def\va{\boldsymbol{a}}
\global\long\def\vb{\boldsymbol{b}}
\global\long\def\vv{\boldsymbol{v}}
\global\long\def\vy{\boldsymbol{y}}

\title{Homework 6: Multiclass, Trees, and Gradient Boosting}

\maketitle
\textbf{Instructions}: Your answers to the questions below, including
plots and mathematical work, should be submitted as a single PDF file.
It's preferred that you write your answers using software that typesets
mathematics (e.g. \LaTeX , \LyX , or MathJax via iPython), though
if you need to you may scan handwritten work. You may find the \href{https://github.com/gpoore/minted}{minted}
package convenient for including source code in your \LaTeX{} document.
If you are using \LyX , then the \href{https://en.wikibooks.org/wiki/LaTeX/Source_Code_Listings}{listings}
package tends to work better.

\section{Reformulations of Multiclass Hinge Loss}

\subsection{Multiclass setting review}

Consider the multiclass output space $\cy=\left\{ 1,\ldots,k\right\} $.
Suppose we have a base hypothesis space \textbf{$\ch=\left\{ h:\cx\times\cy\to\reals\right\} $
}from which we select a compatibility score function. Then our final
multiclass hypothesis space is $\cf=\left\{ f(x)=\argmax_{y\in\cy}h(x,y)\mid h\in\ch\right\} $.
Since functions in $\cf$ map into $\cy$, our action space $\ca$
and output space $\cy$ are the same. Nevertheless, we will write
our class-sensitive loss function as $\Delta:\cy\times\ca\to\reals$,
even though $\cy=\ca$. We do this to indicate that the true class
goes in the first slot of the function, while the prediction (i.e.
the action) goes in the second slot. This is important because we
do not assume that $\Delta(y,y')=\Delta(y',y)$. It would not be unusual
to have this asymmetry in practice. For example, false alarms may
be much less costly than no alarm when indeed something is going wrong.

In the spirit of empirical risk minimization, we would like to find
$f\in\cf$ minimizing the empirical cost-sensitive loss:
\[
\min_{f\in\cf}\sum_{i=1}^{n}\Delta\left(y_{i},f(x_{i})\right),
\]
possibly with some regularization. But this is clearly intractable,
since we already know binary classification is intractable and that's
a special case of this formulation. In lecture we proposed an alternative,
tractable objective function: the multiclass SVM based on the convex
multiclass hinge loss.

\subsection{Two versions  of multiclass hinge loss (or generalized hinge loss)}

In lecture, we defined the \textbf{margin} of the compatibility score
function $h$ on the $i$th example $(x_{i},y_{i})$ for class $y$
as
\[
m_{i,y}(h)=h(x_{i},y_{i})-h(x_{i},y).
\]
We also gave a formulation of a multiclass SVM objective function,
where the loss on an individual example $\left(x_{i},y_{i}\right)$
was

\[
\ell_{1}(h,(x_{i},y_{i}))=\max_{y\in\cy-\left\{ y_{i}\right\} }\left(\max\left[0,\Delta(y_{i},y)-m_{i,y}(h)\right]\right).
\]
There's an alternative formulation, called the \textbf{generalized
hinge loss} in SSBD Section 17.2. There they define
\[
\ell_{2}(h,(x_{i,}y_{i}))=\max_{y\in\cy}\left[\Delta\left(y_{i},y\right)+h(x_{i},y)-h(x_{i},y_{i})\right].
\]

\begin{enumerate}
\item Show that if $\Delta(y,y)=0$ for all $y\in\cy$, then $\ell_{2}\left(h,\left(x_{i},y_{i}\right)\right)=\text{\ensuremath{\ell}}_{1}(h,\left(x_{i},y_{i}\right))$.
{[}Hint: Note that $\max_{y\in\cy}\phi(y)=\max\left(\phi(y_{i}),\max_{y\in\cy-\left\{ y_{i}\right\} }\phi(y_{i})\right)$.{]}

\item In the context of the generalized hinge loss, we've said that $\Delta(y_{i},y)$
is like the ``target margin'' between the score for true class $y_{i}$
and the score for class $y$. Suppose that for our compatibility function
$h$, all target margins are reached or exceeded on $x_{i}$. That
is
\[
m_{i,y}(h)=h(x_{i},y_{i})-h(x_{i},y)\ge\Delta(y_{i},y),
\]
for all $y\in\cy-\left\{ y_{i}\right\} $. Assume that $\Delta(y_{i},y)>0$
$\forall y\neq y_{i}$ and $\Delta(y_{i},y)=0$ for $y=y_{i}$. {]}
\begin{enumerate}
\item Show that under the conditions above, $\ell_{1}(h,(x_{i},y_{i}))=\ell_{2}(h,(x_{i},y_{i}))=0$.
\item Show that under the conditions above, we make the correct prediction
on $x_{i}$. That is, $f(x_{i})=\argmax_{y\in\cy}h(x_{i},y)=y_{i}$.
\end{enumerate}
\end{enumerate}

\section{SGD for Multiclass Linear SVM}

Suppose our output space and our action space are given as follows:
$\cy=\ca=\left\{ 1,\ldots,k\right\} $. Given a non-negative class-sensitive
loss function $\Delta:\cy\times\ca\to[0,\infty)$ and a class-sensitive
feature mapping $\Psi:\cx\times\cy\to\reals^{d}$. Our prediction
function $f:\cx\to\cy$ is given by
\[
f_{w}(x)=\argmax_{y\in\cy}\left\langle w,\Psi(x,y)\right\rangle
\]
For training data $(x_{1},y_{1}),\ldots,(x_{n},y_{n})\in\cx\times\cy$,
let $J(w)$ be the $\ell_{2}$-regularized empirical risk function
for the multiclass hinge loss. We can write this as
\[
J(w)=\lambda\|w\|^{2}+\frac{1}{n}\sum_{i=1}^{n}\max_{y\in\cy}\left[\Delta\left(y_{i},y\right)+\left\langle w,\Psi(x_{i},y)-\Psi(x_{i},y_{i})\right\rangle \right],
\]
for some $\lambda>0$.
\begin{enumerate}
\item {[}Optional{]} Show that $J(w)$ is a convex function of $w$. You
may use any of the rules about convex functions described in our \href{https://davidrosenberg.github.io/mlcourse/Notes/convex-optimization.pdf}{notes on Convex Optimization},
in previous assignments, or in the Boyd and Vandenberghe book, though
you should cite the general facts you are using. {[}Hint: If $f_{1},\ldots,f_{m}:\reals^{n}\to\reals$
are convex, then their pointwise maximum $f(x)=\max\left\{ f_{1}(x),\ldots,f_{m}(x)\right\} $
is also convex.{]}
\item Since $J(w)$ is convex, it has a subgradient at every point. Give
an expression for a subgradient of $J(w)$. You may use any standard
results about subgradients, including the result from an earlier homework
about subgradients of the pointwise maxima of functions. (Hint: It
may be helpful to refer to $\hat{y}_{i}=\argmax_{y\in\cy}\left[\Delta\left(y_{i},y\right)+\left\langle w,\Psi(x_{i},y)-\Psi(x_{i},y_{i})\right\rangle \right]$.)
\item Give an expression for the stochastic subgradient based on the point
$(x_{i},y_{i})$.
\item Give an expression for a minibatch subgradient, based on the points
$(x_{i},y_{i}),\ldots,\left(x_{i+m-1},y_{i+m-1}\right)$.
\end{enumerate}

\section{{[}Optional{]} Hinge Loss is a Special Case of Generalized Hinge
Loss}

Let $\cy=\left\{ -1,1\right\} $. Let $\Delta(y,\hat{y})=\ind{y\neq\hat{y}}.$
If $g(x)$ is the score function in our binary classification setting,
then define our compatibility function as
\begin{eqnarray*}
h(x,1) & = & g(x)/2\\
h(x,-1) & = & -g(x)/2.
\end{eqnarray*}
Show that for this choice of $h$, the multiclass hinge loss reduces
to hinge loss:
\[
\ell\left(h,\left(x,y\right)\right)=\max_{y'\in\cy}\left[\Delta\left(y,y')\right)+h(x,y')-h(x,y)\right]=\max\left\{ 0,1-yg(x)\right\}
\]

\section{Multiclass Classification - Implementation}

In this problem we will work on a simple three-class classification
example, similar to the one \href{https://davidrosenberg.github.io/mlcourse/Archive/2017Fall/Lectures/11b.multiclass.pdf\#page=10}{given in lecture}.
The data is generated and plotted for you in the skeleton code.

\subsection{One-vs-All (also known as One-vs-Rest)}

In this problem we will implement one-vs-all multiclass classification.
Our approach will assume we have a binary base classifier that returns
a score, and we will predict the class that has the highest score.
\begin{enumerate}
\item Complete the class OneVsAllClassifier in the skeleton code. Following
the OneVsAllClassifier code is a cell that extracts the results of
the fit and plots the decision region. Include these results in your
submission.
\end{enumerate}

\subsection{Multiclass SVM}

In this question, we will implement stochastic subgradient descent
for the linear multiclass SVM, as described in lecture and in this
problem set. We will use the class-sensitive feature mapping approach
with the ``multivector construction'', as described in our \href{https://davidrosenberg.github.io/mlcourse/Archive/2017Fall/Lectures/11b.multiclass.pdf\#page=28}{multiclass classification lecture}
and in SSBD Section 17.2.1.
\begin{enumerate}
\item Complete the skeleton code for multiclass SVM. Following the multiclass
SVM implementation, we have included another block of test code. Make
sure to include the results from these tests in your assignment, along
with your code.
\end{enumerate}

\section{{[}Optional{]} Audio Classification}

In this problem, we will work on the urban sound dataset \href{https://serv.cusp.nyu.edu/projects/urbansounddataset/urbansound8k.html}{URBANSOUND8K}
from the Center for Urban Science and Progress (CUSP) at NYU. (You
should download the data from that link.) We will first extract features
from raw audio data using the \href{https://github.com/librosa/librosa}{LibROSA}
package, and then we will train multiclass classifiers to classify
the sounds into 10 sound classes. URBANSOUND8K dataset contains 8732
labeled sound excerpts broken into 10 folds. Use folds 1 and 2 for
training, and folds 3 and 4 for validation.
\begin{enumerate}
\item In LibROSA, there are many functions for visualizing audio waves and
spectra, such as display.waveplot() and display.specshow(). Load a
random audio file from each class as a floating point time series
with librosa.load(), and plot their waves and \href{https://librosa.github.io/librosa/generated/librosa.display.specshow.html}{linear-frequency power spectrogram}.
If you are interested, you can also play the audio in the notebook
with functions display() and Audio() in IPython.display.
\item \href{https://en.wikipedia.org/wiki/Mel-frequency_cepstrum}{Mel-frequency cepstral coefficients (MFCC)}
are a commonly used feature for sound processing. We will use MFCC
and its first and second differences (like discrete derivatives) as
our features for classfication. First, use function feature.mfcc()
from LibROSA to extract MFCC features from each audio sample. (The
first MFCC coefficient is typically discarded in sound analysis, but
you do not need to. You can test whether this helps in the optional
problem below.) Next, use function feature.delta() to calculate the
first and second differences of MFCC. Finally, combine these features
and normalize each feature to zero mean and unit variance.
\item Train a linear multiclass SVM on your training set. Evaluate your
results on the validation set in terms of 0/1 error and generate a
confusion table. Compare the results to a one-vs-all classifier using
a binary linear SVM as the base classifier. For each model, may use
your code from the previous problem, or you may use another implementation
(e.g. from sklearn).
\item {[}More Optional{]} Compare results to any other multiclass classification
methods of your choice.
\item {[}More Optional{]} Try different feature sets and see how they affect
performance.
\end{enumerate}

\section{{[}Optional{]} Decision Tree Implementation}

In this problem we'll implement decision trees for both classification
and regression. The strategy will be to implement a generic class,
called \code{Decision\_Tree}, which we'll supply with the loss function
we want to use to make node splitting decisions, as well as the estimator
we'll use to come up with the prediction associated with each leaf
node. For classification, this prediction could be a vector of probabilities,
but for simplicity we'll just consider hard classifications here.
We'll work with the classification and regression data sets from previous
assignments.
\begin{enumerate}
\item {[}Optional{]} Complete the class \code{Decision\_Tree}, given in
the skeleton code. The intended implementation is as follows: Each
object of type \code{Decision\_Tree} represents a single node of
the tree. The depth of that node is represented by the variable self.depth,
with the root node having depth 0. The main job of the fit function
is to decide, given the data provided, how to split the node or whether
it should remain a leaf node. If the node will split, then the splitting
feature and splitting value are recorded, and the left and right subtrees
are fit on the relevant portions of the data. Thus tree-building is
a recursive procedure. We should have as many\code{Decision\_Tree}
objects as there are nodes in the tree. We will not implement pruning\textbf{
}here. Some additional details are given in the skeleton code.
\item {[}Optional{]} Complete either the \code{compute\_entropy} or \code{compute\_gini}
functions. Run the code provided that builds trees for the two-dimensional
classification data. Include the results. For debugging, you may want
to compare results with sklearn's decision tree. For visualization,
you'll need to install \code{graphviz}.
\item {[}Optional{]} Complete the function \code{mean\_absolute\_deviation\_around\_median}
(MAE). Use the code provided to fit the \code{Regression\_Tree} to
the krr dataset using both the MAE loss and median predictions. Include
the plots for the 6 fits.
\end{enumerate}

\section{Gradient Boosting Machines}

Recall the general gradient boosting algorithm\footnote{Besides the lecture slides, you can find an accessible discussion
of this approach in \url{http://www.saedsayad.com/docs/gbm2.pdf},
in one of the original references \url{http://statweb.stanford.edu/~jhf/ftp/trebst.pdf},
and in this review paper \url{http://web.stanford.edu/~hastie/Papers/buehlmann.pdf}. }, for a given loss function $\ell$ and a hypothesis space $\cf$
of regression functions (i.e. functions mapping from the input space
to $\reals$):
\begin{enumerate}
\item Initialize $f_{0}(x)=0$.
\item For $m=1$ to $M$:

\begin{enumerate}
\item Compute:
\[
{\bf g}_{m}=\left(\left.\frac{\partial}{\partial f(x_{j})}\sum_{i=1}^{n}\ell\left(y_{i},f(x_{i})\right)\right|_{f(x_{i})=f_{m-1}(x_{i}),\,i=1,\ldots,n}\right)_{j=1}^{n}
\]
\item Fit regression model to $-{\bf g}_{m}$:
\[
h_{m}=\argmin_{h\in\cf}\sum_{i=1}^{n}\left(\left(-{\bf g}_{m}\right)_{i}-h(x_{i})\right)^{2}.
\]
\item Choose fixed step size $\nu_{m}=\nu\in(0,1]$, or take
\[
\nu_{m}=\argmin_{\nu>0}\sum_{i=1}^{n}\ell\left(y_{i},f_{m-1}(x_{i})+\nu h_{m}(x_{i})\right).
\]
\item Take the step:
\[
f_{m}(x)=f_{m-1}(x)+\nu_{m}h_{m}(x)
\]
\end{enumerate}
\item Return $f_{M}$.
\end{enumerate}
In this problem we'll derive two special cases of the general gradient
boosting framework: $\ell_{2}$-Boosting and BinomialBoost.
\begin{enumerate}
\item Consider the regression framework, where $\cy=\reals$. Suppose our
loss function is given by
\[
\ell(\hat{y},y)=\frac{1}{2}\left(\hat{y}-y\right)^{2},
\]
and at the beginning of the $m$'th round of gradient boosting, we
have the function $f_{m-1}(x)$. Show that the $h_{m}$ chosen as
the next basis function is given by
\[
h_{m}=\argmin_{h\in\cf}\sum_{i=1}^{n}\left[\left(y_{i}-f_{m-1}(x_{i})\right)-h(x_{i})\right]^{2}.
\]
In other words, at each stage we find the base prediction function
$h_{m}\in\cf$ that is the best fit to the residuals from the previous
stage. {[}Hint: Once you understand what's going on, this is a pretty
easy problem.{]}
\item Now let's consider the classification framework, where $\cy=\left\{ -1,1\right\} $.
In lecture, we noted that AdaBoost corresponds to forward stagewise
additive modeling with the exponential loss, and that the exponential
loss is not very robust to outliers (i.e. outliers can have a large
effect on the final prediction function). Instead, let's consider
the logistic loss
\[
\ell(m)=\ln\left(1+e^{-m}\right),
\]
where $m=yf(x)$ is the margin. Similar to what we did in the $\ell_{2}$-Boosting
question, write an expression for $h_{m}$ as an argmin over $\cf$.
\end{enumerate}

\section{Gradient Boosting Implementation}

This method goes by many names, including gradient boosting machines
(GBM), generalized boosting models (GBM), AnyBoost, and gradient boosted
regression trees (GBRT), among others. Although one of the nice aspects
of gradient boosting is that it can be applied to any problem with
a subdifferentiable loss function, here we'll keep things simple and
consider the standard regression setting with square loss.
\begin{enumerate}
\item Complete the \code{gradient\_boosting} class. As the base regression
algorithm, you may use sklearn's regression tree. You should use
the square loss for the tree splitting rule and the mean function
for the leaf prediction rule. Run the code provided to build gradient
boosting models on the classification and regression data sets, and
include the plots generated. Note that we are using square loss to
fit the classification data, as well as the regression data.
\item {[}Optional{]} Repeat the previous runs on the classification data
set, but use a different classification loss, such as logistic loss
or hinge loss. Include the new code and plots of your results. Note
that you should still use the same regression tree settings for the
base regression algorithm.
\end{enumerate}

\end{document}


\documentclass[13pt,onlymath]{beamer}
\usefonttheme{serif}
\usepackage{graphicx,amsmath,amssymb,tikz,psfrag,epstopdf,fancyvrb}
\usepackage[lighttt]{lmodern}
%\usepackage{graphicx,psfrag}

\input defs.tex

%% formatting

\mode<presentation>
{
\usetheme{default}
}
\setbeamertemplate{navigation symbols}{}
\usecolortheme[rgb={0.13,0.28,0.59}]{structure}
\setbeamertemplate{itemize subitem}{--}
\setbeamertemplate{frametitle} {
    \begin{center}
      {\large\bf \insertframetitle}
    \end{center}
}

\newcommand\footlineon{
  \setbeamertemplate{footline} {
    \begin{beamercolorbox}[ht=2.5ex,dp=1.125ex,leftskip=.8cm,rightskip=.6cm]{structure}
      \footnotesize \insertsection
      \hfill
      {\insertframenumber}
    \end{beamercolorbox}
    \vskip 0.45cm
  }
}
\footlineon

\AtBeginSection[]
{
    \begin{frame}<beamer>
        \frametitle{Outline}
        \tableofcontents[currentsection,currentsubsection]
    \end{frame}
}

%% begin presentation

\title{\large \bfseries String Algorithms}

\author{Jaehyun Park\\[3ex]
CS 97SI\\
Stanford University}

\date{\today}

\begin{document}

\frame{
\thispagestyle{empty}
\titlepage
}

\section{String Matching Problem}

\begin{frame}{String Matching Problem}
\BIT
\item Given a text $T$ and a pattern $P$, find all occurrences of $P$ within $T$
\item Notations:
\BIT
\item $n$ and $m$: lengths of $P$ and $T$
\item $\Sigma$: set of alphabets (of constant size)
\item $P_i$: $i$th letter of $P$ (1-indexed)
\item $a$, $b$, $c$: single letters in $\Sigma$
\item $x$, $y$, $z$: strings
\EIT \EIT
\end{frame}

\begin{frame}{Example}
\BIT
\item $T = \mathtt{AGCATGCTGCAGTCATGCTTAGGCTA}$
\item $P = \mathtt{GCT}$
\item $P$ appears three times in $T$
\vfill
\item A naive method takes $O(mn)$ time
\BIT
\item Initiate string comparison at every starting point
\item Each comparison takes $O(m)$ time
\EIT
\vfill
\item We can do much better!
\EIT
\end{frame}

\section{Hash Table}

\begin{frame}{Hash Function}
\BIT
\item A function that takes a string and outputs a number
\item A good hash function has few collisions
\BIT
\item \ie, If $x\ne y$, $H(x) \ne H(y)$ with high probability
\EIT
\item An easy and powerful hash function is a polynomial mod some prime $p$
\BIT
\item Consider each letter as a number (ASCII value is fine)
\item $H(x_1 \ldots x_k) = x_1 a^{k-1} + x_2 a^{k-2} + \cdots + x_{k-1} a + x_k \pmod{p}$
\item How do we find $H(x_2 \ldots x_{k+1})$ from $H(x_1 \ldots x_k)$?
\EIT\EIT
\end{frame}

\begin{frame}{Hash Table}
\BIT
\item Main idea: preprocess $T$ to speedup queries
\BIT
\item Hash every substring of length $k$
\item $k$ is a small constant
\EIT
\vfill
\item For each query $P$, hash the first $k$ letters of $P$ to retrieve all the occurrences of it within $T$
\vfill
\item Don't forget to check collisions!
\EIT
\end{frame}

\begin{frame}{Hash Table}
\BIT
\item Pros:
\BIT
\item Easy to implement
\item Significant speedup in practice
\EIT
\vfill
\item Cons:
\BIT
\item Doesn't help the asymptotic efficiency
\BIT
\item Can still take $\Theta(nm)$ time if hashing is terrible or data is difficult
\EIT
\item A lot of memory consumption
\EIT\EIT
\end{frame}

\section{Knuth-Morris-Pratt (KMP) Algorithm}

\begin{frame}{Knuth-Morris-Pratt (KMP) Matcher}
\BIT
\item A linear time (!) algorithm that solves the string matching problem by preprocessing $P$ in $\Theta(m)$ time
\BIT
\item Main idea is to skip some comparisons by using the previous comparison result
\EIT
\item Uses an auxiliary array $\pi$ that is defined as the following:
\BIT
\item $\pi[i]$ is the largest integer smaller than $i$ such that $P_1 \ldots P_{\pi[i]}$ is a suffix of $P_1 \ldots P_i$
\EIT
\item ... It's better to see an example than the definition
\EIT
\end{frame}

\begin{frame}{$\pi$ Table Example (from CLRS)}
\begin{center}
\includegraphics[width=0.9\textwidth]{figures/pitable}
\end{center}
\BIT
\item $\pi[i]$ is the largest integer smaller than $i$ such that $P_1 \ldots P_{\pi[i]}$ is a suffix of $P_1 \ldots P_i$
\BIT
\item \eg, $\pi[6]=4$ since \texttt{abab} is a suffix of \texttt{ababab}
\item \eg, $\pi[9]=0$ since no prefix of length $\le 8$ ends with $c$
\EIT
\item Let's see why this is useful
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item $T=\mathtt{ABC\:\;ABCDAB\:\;ABCDABCDABDE}$
\item $P=\mathtt{ABCDABD}$
\item $\pi = (0,0,0,0,1,2,0)$
\item Start matching at the first position of $T$:
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp1}
\end{center}
\item Mismatch at the 4th letter of $P$!
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item We matched $k=3$ letters so far, and $\pi[k] = 0$
\BIT
\item Thus, there is no point in starting the comparison at $T_2$, $T_3$ (crucial observation)
\EIT
\item Shift $P$ by $k-\pi[k] = 3$ letters
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp2}
\end{center}
\item Mismatch at $T_4$ again!
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item We matched $k=0$ letters so far
\item Shift $P$ by $k-\pi[k] = 1$ letter (we define $\pi[0] = -1$)
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp3}
\end{center}
\item Mismatch at $T_{11}$!
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item $\pi[6]=2$ means $P_1 P_2$ is a suffix of $P_1 \ldots P_6$
\item Shift $P$ by $6 - \pi[6] = 4$ letters
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp4}
\end{center}
\item Again, no point in shifting $P$ by 1, 2, or 3 letters
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item Mismatch at $T_{11}$ again!
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp5}
\end{center}
\item Currently 2 letters are matched
\item Shift $P$ by $2-\pi[2] = 2$ letters
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item Mismatch at $T_{11}$ yet again!
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp6}
\end{center}
\item Currently no letters are matched
\item Shift $P$ by $0-\pi[0] = 1$ letter
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item Mismatch at $T_{18}$
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp7}
\end{center}
\item Currently 6 letters are matched
\item Shift $P$ by $6-\pi[6] = 4$ letters
\EIT
\end{frame}

\begin{frame}{Using the $\pi$ Table}
\BIT
\item Finally, there it is!
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/kmp8}
\end{center}
\item Currently all 7 letters are matched
\item After recording this match (at $T_{16} \ldots T_{22}$, we shift $P$ again in order to find other matches
\BIT
\item Shift by $7-\pi[7] = 7$ letters
\EIT\EIT
\end{frame}

\begin{frame}{Computing $\pi$}
\BIT
\item Observation 1: if $P_1\ldots P_{\pi[i]}$ is a suffix of $P_1 \ldots P_i$, then $P_1 \ldots P_{\pi[i]-1}$ is a suffix of $P_1 \ldots P_{i-1}$
\BIT
\item Well, obviously...
\EIT
\item Observation 2: all the prefixes of $P$ that are a suffix of $P_1 \ldots P_i$ can be obtained by recursively applying $\pi$ to $i$
\BIT
\item \eg, $P_1 \ldots P_{\pi[i]}$, $P_1 \ldots, P_{\pi[\pi[i]]}$, $P_1 \ldots, P_{\pi[\pi[\pi[i]]]}$ are all suffixes of $P_1 \ldots P_i$
\EIT \EIT
\end{frame}

\begin{frame}{Computing $\pi$}
\BIT
\item A non-obvious conclusion:
\BIT
\item First, let's write $\pi^{(k)}[i]$ as $\pi[\cdot]$ applied $k$ times to $i$
\item \eg, $\pi^{(2)}[i] = \pi[\pi[i]]$
\item $\pi[i]$ is equal to $\pi^{(k)}[i-1] + 1$, where $k$ is the smallest integer that satisfies $P_{\pi^{(k)}[i-1]+1} = P_i$
\BIT
\item If there is no such $k$, $\pi[i] = 0$
\EIT\EIT
\item Intuition: we look at all the prefixes of $P$ that are suffixes of $P_1 \ldots P_{i-1}$, and find the longest one whose next letter matches $P_i$
\EIT
\end{frame}

\begin{frame}[fragile]{Implementation}
\begin{Verbatim}[xleftmargin=25pt]
pi[0] = -1;
int k = -1;
for(int i = 1; i <= m; i++) {
  while(k >= 0 && P[k+1] != P[i])
    k = pi[k];
  pi[i] = ++k;
}
\end{Verbatim}
\end{frame}

\begin{frame}[fragile]{Pattern Matching Implementation}
\begin{Verbatim}[xleftmargin=25pt]
int k = 0;
for(int i = 1; i <= n; i++) {
  while(k >= 0 && P[k+1] != T[i])
    k = pi[k];
  k++;
  if(k == m) {
    // P matches T[i-m+1..i]
    k = pi[k];
  }
}
\end{Verbatim}
\end{frame}

\section{Suffix Trie}

\begin{frame}{Suffix Trie}
\BIT
\item Suffix trie of a string $T$ is a rooted tree that stores all the suffixes (thus all the substrings)
\item Each node corresponds to some substring of $T$
\item Each edge is associated with an alphabet
\item For each node that corresponds to $ax$, there is a special pointer called \emph{suffix link} that leads to the node corresponding to $x$
\item Surprisingly easy to implement!
\EIT
\end{frame}

\begin{frame}{Example}
\begin{center}
\includegraphics[width=0.7\textwidth]{figures/suftrie}

(Figure modified from Ukkonen's original paper)
\end{center}
\end{frame}

\begin{frame}{Incremental Construction}
\BIT
\item Given the suffix tree for $T_1 \ldots T_n$
\BIT
\item Then we append $T_{n+1} = a$ to $T$, creating necessary nodes
\EIT
\item Start at node $u$ corresponding to $T_1 \ldots T_n$
\BIT
\item Create an $a$-transition to a new node $v$
\EIT
\item Take the suffix link at $u$ to go to $u'$, corresponding to $T_2 \ldots T_n$
\BIT
\item Create an $a$-transition to a new node $v'$
\item Create a suffix link from $v$ to $v'$
\EIT\EIT
\end{frame}

\begin{frame}{Incremental Construction}
\BIT
\item Repeat the previous process:
\BIT
\item Take the suffix link at the current node
\item Make a new $a$-transition there
\item Create the suffix link from the previous node
\EIT
\item Stop if the node already has an $a$-transition
\BIT
\item Because from this point, all nodes that are reachable via suffix links already have an $a$-transition
\EIT\EIT
\end{frame}

\begin{frame}{Construction Example}
\begin{center}
\includegraphics[height=0.5\textheight]{figures/trie1}

Given the suffix trie for \texttt{aba}\\
We want to add a new letter \texttt{c}
\end{center}
\end{frame}

\begin{frame}{Construction Example}
\begin{center}
\includegraphics[height=0.5\textheight]{figures/trie2}
\end{center}
\end{frame}

\begin{frame}{Construction Example}
\begin{center}
\includegraphics[height=0.5\textheight]{figures/trie3}
\end{center}
\end{frame}

\begin{frame}{Construction Example}
\begin{center}
\includegraphics[height=0.5\textheight]{figures/trie4}
\end{center}
\end{frame}

\begin{frame}{Construction Example}
\begin{center}
\includegraphics[height=0.5\textheight]{figures/trie5}
\end{center}
\end{frame}

\begin{frame}{Construction Example}
\begin{center}
\includegraphics[height=0.5\textheight]{figures/trie6}
\end{center}
\end{frame}

\begin{frame}{Construction Example}
\BIT
\item Construction time is linear in the tree size
\item But the tree size can be quadratic in $n$
\BIT
\item \eg, $T = \mathtt{aa}\ldots\mathtt{abb}\ldots\mathtt{b}$
\EIT\EIT
\end{frame}

\begin{frame}{Construction Example}
\BIT
\item To find $P$, start at the root and keep following edges labeled with $P_1$, $P_2$, etc.
\vfill
\item Got stuck? Then $P$ doesn't exist in $T$
\EIT
\end{frame}

\section{Suffix Array}

\begin{frame}{Suffix Array}
\begin{center}
\includegraphics[width=0.8\textwidth]{figures/sufarray}
\end{center}
\end{frame}

\begin{frame}{Suffix Array}
\BIT
\item Memory usage is $O(n)$
\item Has the same computational power as suffix trie
\item Can be constructed in $O(n)$ time (!)
\BIT
\item But it's hard to implement
\EIT
\item There is an approachable $O(n \log^2 n)$ algorithm
\BIT
\item If you want to see how it works, read the paper on the course website
\item \url{http://cs97si.stanford.edu/suffix-array.pdf}
\EIT\EIT
\end{frame}

\begin{frame}{Notes on String Problems}
\BIT
\item Always be aware of the null-terminators
\item Simple hash works so well in many problems
\item If a problem involves rotations of some string, consider concatenating it with itself and see if it helps
\item Stanford team notebook has implementations of suffix arrays and the KMP matcher
\EIT
\end{frame}

\end{document}


\chapter{Introduction}
This project aims at creating a simple modle of our solar system using {\em MatLab.} While putting emphasis on the visuals, we kept the mathematical aspect simple.\\

Trajectories are not calculated and the orbits are perfect circles, due to the simple fact that the difference would be hardly noticeable. The sizes and distances are mostly to scale. We use a logarithmic scale for the sizes and a linear, though a very small scale, for the distances.
\pagebreak


% - review of background, in particular function spaces, regularity of functions
% Part 1: univariate approximation
% - spline approximation of smooth functions in 1D
% - polynomial and trigonometric approximation of analytic functions in 1D
% - linear best approximation
% - best n-term approximation (to be decided whether feasible)
% - multi-variate approximation by tensor products (low dimension)
% Part 2: Multi-variate approximation: cover a selection of the following:
% - mixed regularity, splines and sparse grids, Smolyak algorithm
% - radial basis functions and Gaussian processes
% - ridge functions and neural networks
% - compressed sensing and best n-term approximation
%
% Throughout the lecture each topic will cover (1) approximation rates, (2) algorithms, and (3) examples.

% Approximation Theory and Methods, M. J. D. Powell
% Approximation Theory and Approximation Practice, N. Trefethen
% A course in approximation theory, E.W.Cheney and W.A.Light
% Nonlinear approximation, R. DeVore  (Acta Numerica)

\documentclass[letterpaper, 11pt, reqno]{article}
\usepackage{graphicx, epsfig, psfrag}
\usepackage{amsfonts,amsmath,amssymb} % ,wasysym,amsbsy
\usepackage[left=0.5cm,right=5.5cm,top=3cm,bottom=3cm]{geometry}
% \usepackage[left=3cm,right=3cm,top=3cm,bottom=3cm]{geometry}
\usepackage{float}
\usepackage{hyperref}
\usepackage{theorem}
\usepackage{color}
\usepackage{subfigure}
\usepackage{mathrsfs}
\usepackage{bm}
\usepackage{bbm}
\usepackage{cleveref}
\usepackage{ams_proof}
\usepackage{showlabels}

% \usepackage{showkeys}
% \usepackage{minted}

\sloppy
\frenchspacing

\numberwithin{equation}{section}

\newcommand{\examnote}[1]{{\color{red} \it [\![ #1 ]\!]}}
\newcommand{\examrem}[1]{ \examnote{ removed non-examinable section }
}
% \newcommand{\examnote}[1]{}
\def\notexam{{\color{red} \bf (NE) }}

\newcommand{\alert}[1]{{\color{red} \it $\{$ #1 $\}$}}

\def\nbintro{{\tt [Notebook 01]}}
\def\nbtrig{{\tt [Notebook 02]}}
\def\nbpoly{{\tt [Notebook 03]}}
\def\nblsq{{\tt [Notebook 06]}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% THEOREM-STYLE-ENVIRONMENTS %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newcounter{theorem}
\numberwithin{theorem}{section}
\def\openthm#1#2{\refstepcounter{theorem}\bigskip
{\noindent\bf#1~\thetheorem\if#2!{. }\else{ (#2).}\fi}
\it}
\def\thmskip{}
\newenvironment{theorem}[1][!]{\openthm{Theorem}{#1}}{\thmskip}
\newenvironment{lemma}[1][!]{\openthm{Lemma}{#1}}{\thmskip}
\newenvironment{corollary}[1][!]{\openthm{Corollary}{#1}}{\thmskip}
\newenvironment{proposition}[1][!]{\openthm{Proposition}{#1}}{\thmskip}
\newenvironment{definition}[1][!]{\openthm{Definition}{#1}}{\thmskip}
\newenvironment{axiom}[1][!]{\openthm{Axiom}{#1}}{\thmskip}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% REMARK-STYLE-ENVIRONMENTS %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\def\openrem#1#2{\refstepcounter{theorem}\bigskip
{\noindent\it\bfseries#1~\thetheorem\if#2!{. }\else{ (#2). }\fi} }
\newenvironment{remark}[1][!]{\pushQED{\qed}
  \openrem{Remark}{#1}}{\popQED \medskip}
\newenvironment{example}[1][!]{\pushQED{\qed}
  \openrem{Example}{#1}}{\popQED \medskip}

\newcounter{exercise} \numberwithin{exercise}{section}
\def\openex#1#2{\refstepcounter{exercise}\medskip \noindent
  {\it\bfseries#1~\theexercise\if#2!{. }\else{ (#2). }\fi} }
\newenvironment{exercise}[1][!]{\pushQED{\qed}\openex{Exercise}{#1}}{\popQED
\medskip}

%%%%%%%%%%%%%%%%%%%%%%%%%
%    LISTS              %
%%%%%%%%%%%%%%%%%%%%%%%%%
\def\alist{\renewcommand{\labelenumi}{(\alph{enumi})}}
\def\ilist{\renewcommand{\labelenumi}{(\roman{enumi})}}
\def\deflist{\renewcommand{\labelenumi}{\arabic{enumi}.}}

\newenvironment{longilist} %
{\begin{list}{\hspace{1.4cm}{\it (\roman{enumi})}}{ %
      \usecounter{enumi} %
      \setlength{\leftmargin}{0mm} %
      \setlength{\rightmargin}{0mm} %
      \setlength{\listparindent}{\parindent}}
  } %
  {\end{list}} %

\input notation.tex

\begin{document}

\begin{center}
  \hfill \\[4cm]
  {\Large \bf Approximation Theory and Applications} \\[4mm]
  {\large Christoph Ortner} \\
  {\small \tt ortner@math.ubc.ca}  \\
  {\today}
\end{center}

\clearpage
\quad
\clearpage

\tableofcontents

\clearpage
\input{intro.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%% PART I %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\clearpage

\vspace{5cm}

{\huge Part I: Univariate Approximation}

\clearpage
\input{trig.tex}

\clearpage
\input{polys.tex}

\clearpage

\vspace{5cm}

{\huge Part II: Further Topics in Univariate Approximation}

\clearpage
\input{splines.tex}

\clearpage
\input{lsq.tex}

\clearpage
\input{nonlin.tex}

\clearpage
\vspace{5cm}

{\huge Part III: Approximation in High Dimension}

\clearpage
\input{sparse.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bibliographystyle{alpha}
\bibliography{apxthy}

\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%% PART II %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\clearpage

\vspace{5cm}

{\huge Part II: Topics in Multivariate Approximation}

\clearpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Tensor Products}

\section{Sparse Grids}

\section{Radial Basis Functions and GPs}

\section{Ridge Functions and Neural Networks}

% \begin{exercise}[Piecewise Smooth Functions] \label{exr:trig:pwsmooth}
%   In this exercise we will develop an $L^2$-error estimate for
%   piecewise smooth functions.
%   \begin{enumerate} \ilist
%     \item Let $g(x) := \sin(x/2)$, continued periodically. With a similar
%     argument as in the previous exercise prove that
%       $\|g - \Pi_N g \|_{L^2} \lesssim N^{-1/2}$.
%     %
%     \item Let $f \in C^{p-1}(\TT)$ with $f^(p-1)$ piecewise $C^{\infty}$
%     (technically, $C^{p,1}$ suffices) with finitely many points $\{x_j\}_{j=1}^J
%     \subset (-\pi, \pi]$ where $f^{(p)}$ jumps. Prove that there exist
%     $a_j \in \C$ such that
%     \[
%       f(x) - \sum_{j = 1}^J g(x - x_j) \in C^{p,1}(\TT).
%     \]
%     %
%     \item Deduce that
%     \[
%         \|f - \Pi_N f \|_{L^2} \lesssim N^{-p-1/2}. \qedhere
%     \]
%   \end{enumerate}
% \end{exercise}


%!TEX TS-program = xelatex
%!TEX encoding = UTF-8 Unicode

\documentclass[12pt]{article}
\usepackage{geometry}
\geometry{letterpaper}
\usepackage{fancyhdr}
\usepackage{extramarks}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amsfonts}
\usepackage{tikz}
\usepackage[plain]{algorithm}
\usepackage{algpseudocode}
\usepackage{caption}
\usepackage{booktabs}
\usepackage{graphics}
\usepackage{amsmath}
\usepackage{amsfonts}

\usepackage{xltxtra,fontspec,xunicode}
\usepackage[slantfont,boldfont]{xeCJK}
\setCJKmainfont{宋体}
\setmainfont{Optima}
\defaultfontfeatures{Mapping=tex-text}

\usepackage{xltxtra,fontspec,xunicode}
\usepackage[slantfont,boldfont]{xeCJK}
\setCJKmainfont{宋体}
\setmainfont{Optima}
\defaultfontfeatures{Mapping=tex-text}

\XeTeXlinebreaklocale “zh”
\XeTeXlinebreakskip = 0pt plus 1pt minus 0.1pt

\usepackage{listings}
\usepackage{color}
\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

\lstset{frame=tb,
  language=Java,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{dkgreen},
  stringstyle=\color{mauve},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

\topmargin=-0.45in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.5in
\textheight=9.0in
\headsep=0.25in

\linespread{1.1}

\pagestyle{fancy}
\lhead{\hmwkAuthorName}
\rhead{\hmwkClass}
\chead{\hmwkTitle}

\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}

\setlength\parindent{0pt}

% Homework Details

\newcommand{\hmwkTitle}{homework\ \#2}
\newcommand{\hmwkClass}{Deep Reinforcement Learning}
\newcommand{\hmwkAuthorName}{Tianxiao Hu}

\begin{document}
\pagebreak

\section{Problem 1}
(a)
\begin{equation}
\begin{aligned}
  & \nabla_\theta \mathbb{E}_{s_t, a_t \sim p(s_t, a_t)}[b_{(s_t)}]  \\
= & \nabla_\theta \mathbb{E}_{s_t, a_t}[\mathbb{E}_{a_t \sim \pi_\theta(a_t|s_t)}[b(s_t)]] \\
= & \mathbb{E} [\nabla_\theta \int \pi_\theta(a_t|s_t)b(s_t)da_t] \\
= & \mathbb{E}_{s_t \sim p(s_t)} [b(s_t) \cdot \int \nabla_\theta \pi_\theta(a_t|s_t)da_t] \\
= & \mathbb{E}_{s_t \sim p(s_t)}[b(s_t) \cdot \nabla_\theta \cdot \underbrace{\int \pi_\theta(a_t|s_t)da_t}_{\text{ = 1}}] \\
= & \mathbb{E}_{s_t \sim p(s_t)}[b(s_t) \cdot \nabla_\theta] \\
= & 0
\end{aligned}
\end{equation}

(b) \\
(1) Because this sequence is a markov decision process. Current state is only affected by last state. So conditioning on $(s_1, a_1, ..., a_{t^*-1}, s_{t^*})$ is equivalent to conditioning on $s_{t^*}$. \\
(2)
\begin{equation}
\begin{aligned}
  & \nabla_\theta \mathbb{E}_{\tau \sim \pi_\theta(\tau)}[b(s_t^*)] \\
= & \nabla_\theta \cdot \int p(a_{t^*}|s_{t^*} \cdot s_{t^*-1} \cdot a_{t^*-1} \cdot \cdots \cdot s_1) p(s_{t^* \cdot s_{t^*-1} \cdot \cdots \cdot s_1}) b(s_{t^*}) da_{t^*}ds_{t^*}\cdot \cdots \cdot ds_1 \\
= & \nabla_\theta[\int p_\theta(a_{t^*}|s_{t^*}) \cdot p(s_{t^*}| ... ) b(s_{t^*})ds_{t^*}da_t] \underbrace{[\int p(s_{t^*-1} \cdot \cdots \cdot s_1)ds_{t^*-1} \cdot \cdots \cdot ds_1]}_{\text{= 1}} \\
= & \underbrace{\nabla_\theta \int p_\theta(a_{t^*}|s_{t^*})da_{t^*}}_{\text{=0}} \cdot \mathbb{E}_{s_{t^*} \sim p(s_{t^*})}[b(s_{t^*})]) \cdot 1 \\
= & 0
\end{aligned}
\end{equation}

\section{Problem 4}

\begin{figure}[!h]
\centering
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=8cm]{Figure_1.png}
\caption{Learning curves for small batch experiments.}
\end{minipage}
\begin{minipage}[t]{0.48\textwidth}
\centering
\includegraphics[width=8cm]{Figure_2.png}
\caption{Learning curves for large batch experiments.}
\end{minipage}
\end{figure}

\textbf{Answers:}\\
\begin{enumerate}
\item Which gradient estimator has better performance without advantage-centering, the trajectory-centric one, or the one using reward-to-go?\\
The one using reward-to-go have a better performance. From the learning curves for small batch experiments, we can see the green curve(reward-to-go) has a high average return than the blue curve(trajectory-centric).
\item Did advantage centering help?\\
It helps. From the learning curves for small batch experiments, we can see the red curve(with advantage-centering) fluctuates less than the green curve(without advantage-centering).
\item Did the batch size make an impact?\\
Yes, by comparing the learning curves between small batch experiments and large batch experiments, we find large batch experiments converge more quickly.
\end{enumerate}

\section{Problem 5}

\begin{figure}[!h]
\centering
\includegraphics[width=3.8in]{Figure_3.png}
\caption{Learning curve with $b = 64$ and $lr = 0.006$. The policy gets to optimum at about iteration \#65.}
\end{figure}

\section{Problem 7}

\begin{figure}[!h]
\centering
\includegraphics[width=3.8in]{Figure_4.png}
\caption{Learning curve for LunarLander. The policy finally achieved an average return of around 180.}
\end{figure}

\section{Problem 8}
After a $3 \times 3$ grid search, the best parameter set is $b = 50000, r = 0.02$.

\textbf{Answer:} How did the batch size and learning rate affect the performance? \\
Large batch size will help the learning curve use less iterations to converge. Using a small learning rate can make sure not to miss any local minimum, but adjust the learning rate larger properly can help the performance improve more quickly.

\begin{figure}[!h]
\centering
\includegraphics[width=3.7in]{Figure_5.png}
\caption{Learning curve for HalfCheetah with different parameters.}
\end{figure}

\end{document}


% !TEX root = main.tex

\chapter{Sums of random variables}\label{chap:sums_of_rvs}

\input{06A_generating_functions}
\input{06B_law_of_large_numbers}
\input{06C_central_limit_theorem}


\chapter{{\bf pmatch}}\label{pmatch}

In this appendix we describe \scheme|pmatch|, a simple pattern
matcher written by Oleg Kiselyov.
Let us first consider a simple example of \scheme|pmatch|.

\schemedisplayspace
\begin{schemedisplay}
(define h
  (lambda (x y)
    (pmatch `(,x . ,y)
      (`(,a . ,b) (guard (number? a) (number? b)) (+ a b))
      (`(__ . ,c) (guard (number? c)) (* c c))
      (else (* x x)))))
\end{schemedisplay}

\noindent\scheme{(list (h 1 2) (h 'w 5) (h 6 'w))} $\Rightarrow$ \begin{schemeresponsebox}(3 25 36)\end{schemeresponsebox}

\wspace

\noindent
In this example, a dotted pair is matched against three
different kinds of patterns.

In the first pattern, the value of \scheme{x} is
lexically bound to \scheme{a} and the value of \scheme{y} is lexically bound to
\scheme{b}.  Before the pattern match succeeds, however, an
optional guard is run within the scope of \scheme{a} and \scheme{b}.
The guard succeeds only if \scheme{x} and \scheme{y} are numbers;
if so, then the sum of \scheme{x} and \scheme{y} is returned.

The second pattern matches against a pair,
provided that the optional guard succeeds.
If so, the value of \mbox{\scheme|y|} is lexically bound
to \scheme{c}, and the square of \scheme{y} is returned.

If the second pattern fails to match against \mbox{\scheme|`(,x . ,y)|},
because \scheme{y} is not a number, then the third and final clause is tried.
An \scheme{else} pattern matches against \emph{any} value, and never includes a guard.
In this case, the clause returns the square of \scheme{x}, which must be
a number in order to avoid an error at runtime.

Below is the definition of \scheme{pmatch}, which is implemented using
continuation-passing-style macros \cite{oai:CiteSeerPSU:392916}.

\newpage

%\schemedisplayspace
\begin{schemedisplay}
(define-syntax pmatch
  (syntax-rules (else guard)
    ((_ (op arg ...) cs ...)
     (let ((v (op arg ...)))
       (pmatch v cs ...)))
    ((_ v) (if #f #f))
    ((_ v (else e0 e ...)) (begin e0 e ...))
    ((_ v (pat (guard g ...) e0 e ...) cs ...)
     (let ((fk (lambda () (pmatch v cs ...))))
       (ppat v pat
         (if (and g ...) (begin e0 e ...) (fk))
         (fk))))
    ((_ v (pat e0 e ...) cs ...)
     (let ((fk (lambda () (pmatch v cs ...))))
       (ppat v pat (begin e0 e ...) (fk))))))

(define-syntax ppat
  (syntax-rules (__ quote unquote)
    ((_ v __ kt kf) kt)
    ((_ v () kt kf) (if (null? v) kt kf))
    ((_ v quotelit kt kf)
     (if (equal? v quotelit) kt kf))
    ((_ v (unquote var) kt kf) (let ((var v)) kt))
    ((_ v (x . y) kt kf)
     (if (pair? v)
       (let ((vx (car v)) (vy (cdr v)))
         (ppat vx x (ppat vy y kt kf) kf))
       kf))
    ((_ v lit kt kf) (if (equal? v quotelit) kt kf))))
\end{schemedisplay}

The first clause ensures that the expression whose value is to be
\scheme{pmatch}ed against is evaluated only once.  The second clause
returns an unspecified value if no other clause matches.

The remaining clauses represent the three types of patterns supported
by \scheme{pmatch}.  The first is the trivial \scheme{else} clause,
which matches against any datum, and which behaves identically to an
\scheme{else} clause in a \scheme{cond} expression.  The other two
clauses are identical, except that the first one includes a guard
containing one or more expressions---if the datum matches against the
pattern, the guard expressions are evaluated in left-to-right order.
If a guard expression evaluates to \scheme{#f}, then it is as if the
datum had failed to match against the pattern: the remaining guard
expressions are ignored, and the next clause is tried.
The expression \scheme{(fk)} is evaluated if the pattern it is
associated with fails to match, or if the pattern matches but the
guard fails.

\scheme{ppat} does the actual pattern matching over constants and
pairs. The wild-card pattern \scheme{__} matches against \emph{any} value%
\footnote{The \scheme|pmatch| presented in~\cite{alphamk} uses a single underscore ({\tt \_}) as the wild-card pattern. Here we use a double underscore ({\tt \_\_}) for compatibility with \RsixRS.};
the second pattern matches against the empty list;
the third pattern matches against a quoted value; and
the fourth pattern matches against \scheme{any} value,
and binds that value to a lexical variable with the specified identifier name.
The fifth pattern matches against a pair, tears it apart,
and recursively matches the \scheme{car} of the value against
the \scheme{car} of the pattern.  If that succeeds, the \scheme{cdr}
of the value is recursively matched against the \scheme{cdr} of the pattern.
(We use \scheme{let} to avoid building long \scheme{car/cdr} chains.)
The last pattern matches against constants, including symbols.

Here is the definition of \scheme{h} after expansion.
\schemedisplayspace
\begin{schemedisplay}
(define h
  (lambda (x y)
    (let ((v `(,x . ,y)))
      (let ((fk (lambda ()
                  (let ((fk (lambda () (* x x))))
                    (if (pair? v)
                      (let ((vx (car v)) (vy (cdr v)))
                        (let ((c vy))
                          (if (number? c) (* c c) (fk))))
                      (fk))))))
        (if (pair? v)
          (let ((vx (car v)) (vy (cdr v)))
            (let ((a vx))
              (let ((b vy))
                (if (and (number? a) (number? b))
                  (+ a b)
                  (fk)))))
          (fk))))))
\end{schemedisplay}

There are four kinds of improvements that should be resolved by the
compiler.  First, \scheme{vx} is not used in the top definition of
\scheme{fk}, so it should not get a binding. Second, the binding to
\scheme{a} and \scheme{b} should be parallel \scheme{let} bindings.
Third, where \scheme{c} is bound, could have been where \scheme{vy} is
bound, and where \scheme{a} and \scheme{b} are bound, could have been
where \scheme{vx} and \scheme{vy} are bound, respectively.  Fourth,
thunk creation is unnecessary where no guard is present.

The \scheme{mv-let} macro used in Chapter~\ref{akimplchapter} can be defined using \scheme{pmatch}.
\schemedisplayspace
\begin{schemedisplay}
(define-syntax mv-let
  (syntax-rules ()
    ((_ ((x ...) e) b0 b ...) (pmatch e ((,x ...) b0 b ...)))))
\end{schemedisplay}

\noindent\scheme|(mv-let ((x y z) (list 1 2 3)) (+ x y z))| $\Rightarrow$ \begin{schemeresponsebox}6\end{schemeresponsebox}