\documentclass[letterpaper,9pt,twoside,printwatermark=false]{pinp}

%% Some pieces required from the pandoc template
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}

% Use the lineno option to display guide line numbers if required.
% Note that the use of elements such as single-column equations
% may affect the guide line number alignment.

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

% The geometry package layout settings need to be set here...
\geometry{layoutsize={0.95588\paperwidth,0.98864\paperheight},%
          layouthoffset=0.02206\paperwidth,%
		  layoutvoffset=0.00568\paperheight}

\definecolor{pinpblue}{HTML}{185FAF}  % imagecolorpicker on blue for new R logo
\definecolor{pnasbluetext}{RGB}{101,0,0} %

\title{DALITE Q5 - Bootstrap, Tests of Significance, Binomial Distribution,
Inference for Means and Proportions. Solutions.}

\author[a]{EPIB607 - Inferential Statistics}

  \affil[a]{Fall 2019, McGill University}

\setcounter{secnumdepth}{5}

% Please give the surname of the lead author for the running footer
\leadauthor{Bhatnagar and Hanley}

% Keywords are not mandatory, but authors are strongly encouraged to provide them. If provided, please include two to five keywords, separated by the pipe symbol, e.g:
 \keywords{  Hypothesis testing |  Bootstrap |  t distribution |  One sample mean |  Normal calculations |  Confidence intervals |  Central Limit Theorem (CLT)  }

\begin{abstract}
This DALITE quiz will cover the bootstrap, an introduction to
significance testing, and inference for a single mean using the t
distribution.
\end{abstract}

\dates{This version was compiled on \today}

% initially we use doi so keep for backwards compatibility
\doifooter{\url{https://sahirbhatnagar.com/EPIB607/}}
% new name is doi_footer

\pinpfootercontents{DALITE Q5 due October 3, 2018 by 5pm}

\begin{document}

% Optional adjustment to line up main text (after abstract) of first page with line numbers, when using both lineno and twocolumn options.
% You should only change this length when you've finalised the article contents.
\verticaladjustment{-2pt}

\maketitle
\thispagestyle{firststyle}
\ifthenelse{\boolean{shortarticle}}{\ifthenelse{\boolean{singlecolumn}}{\abscontentformatted}{\abscontent}}{}

% If your first paragraph (i.e. with the \dropcap) contains a list environment (quote, quotation, theorem, definition, enumerate, itemize...), the line after the list may have some extra indentation. If this is the case, add \parshape=0 to the end of the list environment.

\hypertarget{hypothesis-tests-1}{%
\section{Hypothesis tests 1}\label{hypothesis-tests-1}}

The average human gestation time is 266 days from conception. A
researcher suspects that proper nutrition plays an important role and
that poor women with inadequate food intake would have shorter gestation
times even when given vitamin supplements. A random sample of 20 poor
women given vitamin supplements throughout the pregnancy has a mean
gestation time from conception of \(\bar{y}\)=256 days. The null
hypothesis for the researcher's test is

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \textbf{\(\textrm{H}_0\): \(\mu\) = 266 (Correct)}
\item
  \(\textrm{H}_0\): \(\mu\) = 256
\item
  \(\textrm{H}_0\): \(\mu\) \textless{} 266
\item
  \(\textrm{H}_0\): \(\bar{y}\) = 266
\end{enumerate}

\hypertarget{correct-rationales}{%
\subsection{Correct rationales}\label{correct-rationales}}

\begin{itemize}
\tightlist
\item
  The null hypothesis is usually a statement of ``no effect'' or ``no
  difference.''
\item
  The null hypothesis is the statement of no effect, so if there was no
  effect, the average gestation time would be the population mean.
\item
  The null hypothesis of this study is that the average gestation period
  of women with proper nutrition and poor women with inadequate food
  intake is the same that is 266 days. Hence, the population mean (all
  poor women) have the gestation period of 266 days.
\end{itemize}

\hypertarget{incorrect-rationales}{%
\subsection{Incorrect rationales}\label{incorrect-rationales}}

\begin{itemize}
\tightlist
\item
  The null hypothesis is ``the accepted fact'', which in this case is
  that poorer nutrion will yield results lower than the mean
  (\textless{}266) or H null: u\textless{}266
\item
  The researcher is testing that the hypothesis that poor women with
  inadequate food intake will have lower gestation time
\end{itemize}

\hypertarget{hypothesis-tests-2}{%
\section{Hypothesis tests 2}\label{hypothesis-tests-2}}

The average human gestation time is 266 days from conception. A
researcher suspects that proper nutrition plays an important role and
that poor women with inadequate food intake would have shorter gestation
times even when given vitamin supplements. A random sample of 20 poor
women given vitamin supplements throughout the pregnancy has a mean
gestation time from conception of \(\bar{y}\)=256 days. The researcher's
alternative hypothesis for the test is

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \(\textrm{H}_a\): \(\mu \neq 256\)
\item
  \textbf{\(\textrm{H}_a\): \(\mu < 266\) (Correct)}
\item
  \(\textrm{H}_a\): \(\mu < 256\)
\end{enumerate}

\hypertarget{correct-rationales-1}{%
\subsection{Correct rationales}\label{correct-rationales-1}}

\begin{itemize}
\tightlist
\item
  The alternative hypothesis is the point of view of the researchers. In
  this scenario, the researchers are suspecting that there will be a
  shorter gestation time for poor women in comparison to an average
  woman. The alternative hypothesis should be one-sided because the
  investigators only want to know if gestation time will be shorter and
  not if it is overall different (shorter or greater) from the average
  gestation time.
\item
  Researcher has specified a specific direction - that the sample would
  generate a mean gestation time less than 266. Therefore the
  alternative hypothesis is that \(\mu\) is less than 266.
\end{itemize}

\hypertarget{incorrect-rationales-1}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-1}}

\hypertarget{hypothesis-test-3}{%
\section{Hypothesis test 3}\label{hypothesis-test-3}}

The average human gestation time is 266 days from conception. A
researcher suspects that proper nutrition plays an important role and
that poor women with inadequate food intake would have shorter gestation
times even when given vitamin supplements. A random sample of 20 poor
women given vitamin supplements throughout the pregnancy has a mean
gestation time from conception of \(\bar{y}\)=256 days. Human gestation
times are approximately Normal with standard deviation \(\sigma\) = 16
days. The p-value for the researcher's test is (provide your calculation
in the rationale)

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  more than 0.1
\item
  \textbf{less than 0.01 (Correct)}
\item
  less than 0.001
\item
  less than 0.05
\item
  less than 0.025
\end{enumerate}

\hypertarget{correct-rationales-2}{%
\subsection{Correct rationales}\label{correct-rationales-2}}

\begin{itemize}
\tightlist
\item
  \(Z= (y - \mu)/ (\sigma / sqrt (n))\) = -2.795 \(\to\)
  \texttt{pnorm(-2.795,\ mean\ =\ 0\ ,\ sd=1\ )\ =\ 0.00026}
\item
  \texttt{mosaic::xpnorm(q=\ (256\ -\ 266)/(16/sqrt(20)))}
\item
  \texttt{stats::pnorm(q\ =\ 256,\ mean\ =\ 266,\ sd\ =\ 16/(sqrt(20)),\ lower.tail\ =\ T)}
\end{itemize}

\hypertarget{incorrect-rationales-2}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-2}}

\begin{itemize}
\tightlist
\item
  \(t* = (256-266) / (16 / \sqrt(20))\) = -2.795. degrees of freedom =
  n-1 = 19. p-value is between 0.01 and 0.005. This is a one tailed
  t-test so divide by 2 and p-value is between 0.005 and 0.0025.
  Therefore, less than 0.01.
\item
  \((256 - 266)/16 = -0.625 \to\) \texttt{pnorm(-0.625)} corresponds to
  a p value of 0.27
\item
  \(t = (\bar{y}-\mu)/(\sigma/\sqrt{n})\) then use \(t\) distribution
  with df=19
\end{itemize}

\hypertarget{hypothesis-tests-4}{%
\section{Hypothesis tests 4}\label{hypothesis-tests-4}}

Average human gestation time is 266 days, when counted from conception.
A hospital gives a 90\% confidence interval for the mean gestation time
from conception among its patients. That interval is 264 \(\pm\) 5 days.
Is the mean gestation time in that hospital significantly different from
266 days?

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \textbf{It is not significantly different at the 10\% level and
  therefore is also not significantly different at the 5\% level.
  (Correct)}
\item
  It is not significantly different at the 10\% level but might be
  significantly different at the 5\% level. c. It is significantly
  different at the 10\% level
\end{enumerate}

\hypertarget{correct-rationales-3}{%
\subsection{Correct rationales}\label{correct-rationales-3}}

\begin{itemize}
\tightlist
\item
  The average human gestation time falls within the interval of the
  hospital which is 259-269 days, therefore the mean gestation time is
  not significantly different at the 10\% level and since the 5\% level
  is even more specific, then it is not significantly different at the
  5\% level either.
\item
  266 is included in the 259-269 range, therefore the mean gestation
  time in the hospital is not significantly different from 266 days at
  the 10\% level. Looking at the significance at the 5\% level would
  make the confidence interval bigger, meaning there's not significant
  different there either
\end{itemize}

\hypertarget{incorrect-rationales-3}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-3}}

\begin{itemize}
\tightlist
\item
  Because 266 falls at the upper tail of the 90\% confidence interval,
  therefore it is significant different at 10\% significance level
\item
  It is significantly different at the 10\% level, since the CI does not
  contain the value zero.
\item
  The 90\% CI includes 266, but the 95\% CI may not include 266, in
  which case a value like 266 would only be observed less than or equal
  to 5\% of the time.
\item
  To be significantly different, p-value should be less than 5\%.
\item
  As 266 falls within 264+/- 5 days, it is not significant at the 10\%
  level, but it may not fall under this range because when the level of
  confidence is increased, the confidence interval becomes smaller.
\end{itemize}

\hypertarget{one-sample-mean}{%
\section{One sample mean}\label{one-sample-mean}}

We prefer the \(t\) procedures to the \(z\) procedures for inference
about a population mean because

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \(z\) can be used only for large samples
\item
  \textbf{\(z\) requires that you know the population standard deviation
  \(\sigma\) (Correct)}
\item
  \(z\) requires that you can regard your data as an SRS from the
  population
\end{enumerate}

\hypertarget{correct-rationales-4}{%
\subsection{Correct rationales}\label{correct-rationales-4}}

\begin{itemize}
\tightlist
\item
  The reason we can use z on large samples is because when we have large
  samples, we can use the sample sd as an estimate for the population
  sd. The core reason for using t is because we don't know the
  population sd and we can't estimate the sd with the small sample size.
\item
  because you don't know population sd, only sample sd for t procedures
\item
  Z test needs to have a known population standard deviation sigma and
  either a normal distribution or the sample size n is large. If the
  population standard deviation is unknown and the sample size is small
  then t test statistic can be used.
\end{itemize}

\hypertarget{incorrect-rationales-4}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-4}}

\begin{itemize}
\tightlist
\item
  df are smaller for t-test, and you do not need a SD value
\end{itemize}

\hypertarget{one-sample-mean-2}{%
\section{One sample mean 2}\label{one-sample-mean-2}}

Because \(t\) procedures are robust, the most important condition for
their safe use is that

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  the population standard deviation \(\sigma\) is known
\item
  the population distribution is exactly Normal
\item
  \textbf{the data can be regarded as an SRS from the population
  (Correct)}
\item
  the CLT hasn't kicked in yet
\item
  the sample size is small
\end{enumerate}

\hypertarget{correct-rationales-5}{%
\subsection{Correct rationales}\label{correct-rationales-5}}

\begin{itemize}
\tightlist
\item
  The point of the t test is that sigma is not known (rules out A) The
  CLT says that the population distribution doesn't matter (rules out B)
  We want the CLT to kick in (rules out D) The larger the sample size
  the better(rules out E).
\end{itemize}

\hypertarget{incorrect-rationales-5}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-5}}

\begin{itemize}
\tightlist
\item
  We should only use t procedures if the CLT is not applicable.
\item
  In order to use the T procedure comfortably, the sample size must be
  large and the population distribution must be normal.
\end{itemize}

\hypertarget{bootstrap}{%
\section{Bootstrap}\label{bootstrap}}

Which of the following statements \emph{best} describes the utility of
the bootstrap

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  The bootstrap frees us from the requirement of using simple formulas
  to derive confidence intervals
\item
  The bootstrap allows us to simulate a sampling distribution
\item
  \textbf{The bootstrap frees us from the assumption of a Gaussian
  sampling distribution for the mean (as per the CLT) (Correct)}
\item
  The bootstrap tells us if the sampling distribution is asymmetric
\end{enumerate}

\hypertarget{correct-rationales-6}{%
\subsection{Correct rationales}\label{correct-rationales-6}}

\begin{itemize}
\tightlist
\item
  Computer intensive methods can solve most problems without assuming
  that the data have a Gaussian distribution.
\item
  The bootstrap frees us from the assumption that the data conform to a
  bell-shaped normal distribution.
\end{itemize}

\hypertarget{incorrect-rationales-6}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-6}}

\begin{itemize}
\tightlist
\item
  due to the nature of bootstrap, you can resample things MANY times,
  which means that CLT will always kick in giving a gaussian
  distribution instead of assuming one exists
\item
  Bootstrap is useful in because it does not require the CLT and
  \textbf{the population} does not have to be normally distributed
\item
  The bootstrap doesn't rely on CLT, and gives us freedom from having to
  assume the normal distribution of \textbf{the population}.
\item
  The bootstrap allows us to derive estimates when often-used theories
  do not apply. This is especially useful when n is small (e.g.~CLT
  requires n \textgreater{} 30 to assume normality).
\end{itemize}

\hypertarget{binomial-distribution-1}{%
\section{Binomial Distribution 1}\label{binomial-distribution-1}}

In which of the following would Y not have a Binomial distribution?
Provide your justification in the rationale.

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \textbf{Y = Number, out of 60 occupants of 30 randomly chosen cars,
  wearing seatbelts. (Correct)}
\item
  Y = Number, out of 60 occupants of 60 randomly chosen cars, wearing
  seatbelts.
\item
  Y = Number, out of simple random sample of 100 individuals, that are
  left-handed.
\end{enumerate}

\hypertarget{correct-rationales-7}{%
\subsection{Correct rationales}\label{correct-rationales-7}}

\begin{itemize}
\tightlist
\item
  60 occupants from 30 cars would mean more than one occupant per car -
  whether each occupant is wearing a seatbelt or not isn't independent
  if more than one comes from the same car.
\item
  There is a chance that if someone in the car is not wearing a
  seatbelt, the other passenger in the car is not too. This means that
  it is not independent. \#idiotswhodon'twearseatbeltsdrivetogether
\item
  Not independent, idiots who don't wear seatbelts in cars drive
  together!!! VROOM VROOM
\item
  If there are more than one occupant in a car, they may influence each
  other to wear or not wear a seat belt.

  \begin{itemize}
  \tightlist
  \item
    Richard: ``hey Timmy, only losers wear seat belts, be cool like me
    and don't wear your seat belt''
  \item
    Timothy: ``oh snap, you're right Ricky, not wearing a seat belt is
    the fleekest''
  \item
    Richard and Timothy: ``we're going to live forever''
  \end{itemize}
\item
  The outcome does not fall into one of two categories as someone can be
  left-handed, right-handed, or ambidextrous.
\end{itemize}

\hypertarget{incorrect-rationales-7}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-7}}

\begin{itemize}
\tightlist
\item
  Each trial does not have an equal probability of success as there are
  more right-handed people than left-handed people.
\item
  Because in this case the observations are not independent. If the
  first volunteer selected is left-handed, the second one is more likely
  to be right-handed because there are more right-handed individuals
  than left-handed in the remaining sample.
\end{itemize}

\hypertarget{binomial-distribution-2}{%
\section{Binomial Distribution 2}\label{binomial-distribution-2}}

In which of the following would Y not have a Binomial distribution?
Provide your justification in the rationale.

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \textbf{You want to know what percent of married people believe that
  mothers of young children should not be employed outside the home. You
  plan to interview 50 people, and for the sake of convenience you
  decide to interview both the husband and the wife in 25 married
  couples. The random variable Y is the number among the 50 persons
  interviewed who think mothers should not be employed. (Correct)}
\item
  You observe the sex of the next 50 children born at a local hospital;
  Y is the number of girls among them.
\item
  Y = number of occasions, out of a randomly selected sample of 100
  occasions during the year, in which you were indoors.
\end{enumerate}

\hypertarget{correct-rationales-8}{%
\subsection{Correct rationales}\label{correct-rationales-8}}

\begin{itemize}
\tightlist
\item
  Husband's opinion may depend on wife's opinion, and vice versa -
  observations aren't independent.
\item
  One of the conditions for the binomial distribution is that the trials
  must be independent. In A, they are selecting 25 married couples in
  order to gather data on 50 people, but in this scenario either the
  husband or the wife could influence the other, so the trials are not
  independent.
\end{itemize}

\hypertarget{incorrect-rationales-8}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-8}}

\hypertarget{binomial-distribution-3}{%
\section{Binomial Distribution 3}\label{binomial-distribution-3}}

The U.S. National Center for Health Statistics reports that
approximately 12\% of emergency department visits result in hospital
admissions. Consider 20 randomly selected emergency department visits
and assume that visits to emergency departments are independent. What is
the approximate probability that at most 2 of the 20 visits would result
in hospital admissions?

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \textbf{0.5631 (Correct)}
\item
  0.2740
\item
  0.1344
\item
  0.2891
\item
  0.4369
\end{enumerate}

\hypertarget{correct-rationales-9}{%
\subsection{Correct rationales}\label{correct-rationales-9}}

\begin{itemize}
\tightlist
\item
  We have to callculate the probability at 0,1, and 2 and add them up:
  For 0: (1)(1)(0.88)\^{}20 For 1: 20(0.12)(0.88)\^{}19 For 2:
  190(0.12)\textsuperscript{2(0.88)}18 Adding these all up equals 0.5631
\item
  P(X\textless{} or=2) = P(X=2)+P(X=1)+P(X=0) = 0.5631
\item
  pbinom(2, 20, 0.12, lower.tail = TRUE)
\end{itemize}

\hypertarget{incorrect-rationales-9}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-9}}

\begin{itemize}
\tightlist
\item
  1- pbinom(2, size = 20, prob = 0.12)
\end{itemize}

\hypertarget{binomial-distribution-4}{%
\section{Binomial DIstribution 4}\label{binomial-distribution-4}}

The U.S. National Center for Health Statistics reports that
approximately 12\% of emergency department visits result in hospital
admissions. Consider 20 randomly selected emergency department visits
and assume that visits to emergency departments are independent. How
many hospital admissions do we expect, on average, in a random sample of
20 emergency department visits?

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  2
\item
  \textbf{2.4 (Correct)}
\item
  12
\item
  24
\item
  1.4533
\end{enumerate}

\hypertarget{correct-rationales-10}{%
\subsection{Correct rationales}\label{correct-rationales-10}}

\begin{itemize}
\tightlist
\item
  mean of a binomial distribution = np = 20*0.12 = 2.4
\item
  \(\mu\) = np = (20)(0.12) = 2.4 hospital admissions
\end{itemize}

\hypertarget{incorrect-rationales-10}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-10}}

\begin{itemize}
\tightlist
\item
  0.12 x 20 = 2.4 But we can't in a random sample get .4, so we choose
  2.
\end{itemize}

\hypertarget{binomial-distribution-5}{%
\section{Binomial Distribution 5}\label{binomial-distribution-5}}

The U.S. National Center for Health Statistics reports that
approximately 12\% of emergency department visits result in hospital
admissions. Consider 20 randomly selected emergency department visits
and assume that visits to emergency departments are independent. If Y is
the number of emergency department visits that result in hospital
admissions in random samples of 20 visits, what is approximately the
standard deviation of Y?

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \textbf{1.45 (Correct)}
\item
  2.11
\item
  4.46
\end{enumerate}

\hypertarget{correct-rationales-11}{%
\subsection{Correct rationales}\label{correct-rationales-11}}

\begin{itemize}
\tightlist
\item
  The standard deviation of U can be solved using the equation sigma =
  sqrt(np(1-p)) = sqrt(20\emph{.12}(1-0.12)) = 1.45
\end{itemize}

\hypertarget{incorrect-rationales-11}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-11}}

\hypertarget{binomial-distribution-6}{%
\section{Binomial Distribution 6}\label{binomial-distribution-6}}

According to the 2015 U.S. census update, approximately 13\% of
Americans are black. Let Y be the number of blacks in a random sample of
1500 Americans. What is the probability that the sample will contain 200
or more blacks?

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  pbinom(q = 200, size = 1500, prob = 0.13)
\item
  1-dbinom(x = 200, size = 1500, prob = 0.13)
\item
  dbinom(x = 200, size = 1500, prob = 0.13)
\item
  \texttt{pnorm(200, mean = 195, sd = 13.025, lower.tail = FALSE)}
  \textbf{(Correct)}
\item
  \texttt{1 - pbinom(q = 200, size = 1500, prob = 0.13, lower.tail = FALSE)}
\end{enumerate}

\hypertarget{correct-rationales-12}{%
\subsection{Correct rationales}\label{correct-rationales-12}}

\begin{itemize}
\item
  \begin{enumerate}
  \def\labelenumi{\alph{enumi})}
  \tightlist
  \item
    is not correct because it is looking at all values below 200 b) is
    not correct because it is looking at the probability of getting
    anything but 200 c) is not correct because it is looking at getting
    exactly 200 e) is not correct because it is looking at all values
    below \textbf{201} d) since sample size is large, we can use normal
    approximation for binomial distribution
  \end{enumerate}
\item
  In this case, we can use the normal approximation because n is large
  enough, n*p = 195 \textgreater{} 10, n(1-p) = 1305 \textgreater{} 10.
  Therefore, we can use pnorm function with q=200, lower.tail false. A
  doesn't specify a lower.tail=FALSE, so R gives lower.tail = TRUE by
  default. dbinom are wrong.
\end{itemize}

\hypertarget{incorrect-rationales-12}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-12}}

\begin{itemize}
\item
  because you are calculating the sample that will contain 200 or more,
  lower.tail is false.
\item
  \begin{enumerate}
  \def\labelenumi{\Alph{enumi})}
  \setcounter{enumi}{4}
  \tightlist
  \item
    would be right if it didn't have the 1 - before the code.
  \end{enumerate}
\item
  Enough ppl not to do a t-test
\end{itemize}

\hypertarget{one-sample-proportion-1}{%
\section{One Sample Proportion 1}\label{one-sample-proportion-1}}

A CDC report on secondhand smoke at home gives the following 95\%
confidence interval for the percent of California households that are
free of secondhand smoke: (90.8, 92.2). The correct interpretation for
this confidence interval is that we can be 95\% confident that

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  the proportion of households free of secondhand smoke in another
  sample of California households would be between 0.908 and 0.922
\item
  the population mean number of households in California that are free
  of secondhand smoke is between 90.8 and 92.2
\item
  \textbf{the true proportion of all California households that are free
  of secondhand smoke is between 0.908 and 0.922 (Correct)}
\end{enumerate}

\hypertarget{correct-rationales-13}{%
\subsection{Correct rationales}\label{correct-rationales-13}}

\begin{itemize}
\tightlist
\item
  The answer is C because the confidence interval looks at where the
  true population proportion will fall 95\% of the time.
\item
  A is not right because the CI is only for that trial, not for all
  trials. B is not right because they are not talking about a mean value
  but a proportion.
\end{itemize}

\hypertarget{incorrect-rationales-13}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-13}}

\hypertarget{one-sample-proportion-2}{%
\section{One Sample Proportion 2}\label{one-sample-proportion-2}}

A CDC report on secondhand smoke at home gives the following 95\%
confidence interval for the percent of California households that are
free of secondhand smoke: (90.8, 92.2). What is the margin of error for
this interval?

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  \textbf{0.007 (Correct)}
\item
  0.014
\item
  0.028
\end{enumerate}

\hypertarget{correct-rationales-14}{%
\subsection{Correct rationales}\label{correct-rationales-14}}

\begin{itemize}
\tightlist
\item
  The margin of error is half the width of the confidence interval.
\item
  The margin of error is half of the confidence interval. 92.2 - 90.8 /
  2 = 0.007
\end{itemize}

\hypertarget{incorrect-rationales-14}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-14}}

\hypertarget{one-sample-proportion-3}{%
\section{One Sample Proportion 3}\label{one-sample-proportion-3}}

How many observations must be recorded to estimate a population with
unknown proportion p to within +/- 0.02 with 95\% confidence?

\begin{enumerate}
\def\labelenumi{\alph{enumi}.}
\tightlist
\item
  n=25
\item
  n=1225
\item
  \textbf{n=2401 (Correct)}
\item
  n=2350
\item
  n=1691
\end{enumerate}

\hypertarget{correct-rationales-15}{%
\subsection{Correct rationales}\label{correct-rationales-15}}

\begin{itemize}
\tightlist
\item
  We would have to guess p in this case because it is not given. It is
  best to assume p is 0.5 when it is unknown because the margin of error
  is largest when p is 0.5 so by doing so we are giving a more
  conservation estimation for how many people we will need. We then use
  the formula n=(z/m)\^{}2(p)(1-p) where p=0.5, z=1.96, m=0.02.
\end{itemize}

\hypertarget{incorrect-rationales-15}{%
\subsection{Incorrect rationales}\label{incorrect-rationales-15}}

\begin{itemize}
\tightlist
\item
  1691 = (1.645/0.02)\^{}2 * (0.5 * (1-0.5))
\item
  Using the formula: n = (z*/m)(p)(1-p) = (1.96/0.02)(0.5)(1-0.5) = 24.5
  = 25. Here, test is conservative and p is set to 0.5.
\end{itemize}

%\showmatmethods

\bibliography{pinp}
\bibliographystyle{jss}

\end{document}


\chapter{Deep Learning Model}
\label{cp:dp}
After having tested that SPH-based method is a good strategy for this project, the next step is to choose one deep learning model for predicting contact grid images. The basic function of our model is to find the relationship between bodies images and contacts images. Since convolutional neural networks have been proved to be practical in similar tasks(image classification, objects recognition etc.), I though convolutional neural networks would be a good choice. Then, I will introduce some basic details about deep learning and convolutional neural networks.

This chapter consists of four sections.
\begin{itemize}
    \item \textbf{Section} \ref{dpintro}, a brief introduction to deep learning neural networks, including its history, development and current application.
    \item \textbf{Section} \ref{cnn}, decribing details of about Convolutional Neural Networks, including different types of layers and their functionas.
    \item \textbf{Section} \ref{training}, talking about the techqiues used in deep learning training process, which make training get convagence faster and more accurate.
\end{itemize}

\section{Introduction}
    \label{dpintro}
    As a subdiscipline of machine learning, deep learning attempts to perform data abstraction at a high level via multiple processing layers. It is an algorithm based on the data representation learning of machine learning. Observations (e.g., an image) can appear in various forms, such as a vector of specific pixel intensity, or more abstractly, a series of edges, regions of a certain shape, etc. Learning tasks can be completed more easily by applying some special representation methods (for example, facial recognition or facial expression recognition). The merit of deep learning lies in transforming manual acquisition into unsupervised or semi-supervised learning along with its efficient algorithms of hierarchical feature extraction \cite{schmidhuber2015deep}.

    Previous works on deep learning (or Cybernetics as called at that time) were first published during the 1940s to 1960s, which discusses biologically inspired models like Perceptron, Adaline, or Multi Mayer Perceptron \cite{rosenblatt2000probabilistic, schmidhuber2015deep}. Then, from the 1960s to 1980s backpropagation was invented and the second wave named Connectionism followed up \ cite {rumelhart1986learning}sn. This algorithm has been active up to now and is currently utilized for optimization of Deep Neural Networks. Convolutional Neural Networks (CNNs) is among the most outstanding contributions, which is designed to identify visual patterns of relative simplicity, such as handwritten characters \cite{lecun1995convolutional}. Finally, the appearance of more complex architectures in 2006 marks the arrival of deep learning’s modern era\cite{hinton2006fast,bengio2007greedy,huang2007unsupervised}. Breakthroughs in the processing of speech and natural language (2011) as well as image classification at ILSVRC\footnote{\url{http://www.image-net.org/challenges/LSVRC/}} the scientific competition (2012) have made deep learning a conqueror in many Machine Learning communities like Reddit and winner of challenges beyond conventional applications\footnote{\url{http://blog.kaggle.com/2014/04/18/winning-the-galaxy-challenge-with-convnets}}.

    Especially, deep learning has enormously impacted the field of computer vision by achieving unprecedented performance on tasks of image classification, objects detection, image segmentation, image captioning and so on in the past four years \cite{DBLP:journals/corr/GuWKMSSLWW15}. Such huge progress is primarily built upon the enlargement of computational resources such as frameworks (e.g.s TensorFlow\footnote{\url{https://www.tensorflow.org/}}), modern GPU implementations (e.g.  Cudnn\footnote{\url{https://developer.nvidia.com/cudnn}}), increasing amount of available annotated data, and participation in open source codes and share models based on community. Thanks to these advances, more audience is accessible to the expertise that is necessary for the training of modern convolutional networks. Higher accuracy can be achieved through training of larger and deeper architectures on greater datasets every year, while phenomenal outcomes have been exhibited by the already trained ones when they are transferred to smaller datasets or evaluated with varied visual tasks.

    The increasing popularity of deep learning in various research fields enables it the replacement of some conventional approachesg. The computer vision section gives many successful applications, such as image segmentation\cite{lecun2015deep},objection recognition\cite{he2016deep}.
     \begin{figure}[!ht]
        \centering
        \includegraphics[scale = 0.5]{Figures/Colored_neural_network}
        \caption{Visulization of one simple $3$-layers neural networks, including input layer, hidden layer and output layer, \textit{retrieved from Wikipedia}}
    \end{figure}

\section{Convolutional Neural Networks}
    \label{cnn}
    An input layer, an output layer, and multiple hidden layers make up the complete convolutional neural network. Normally, the hidden layers include convolutional layer, pooling layers, fully connected layers, and normalization layers.

    The process in neural networks is described classically as a convolution, while it is a cross-correlation rather than a convolution in terms of math. Only indices in the matrices are meaningful, which are thus assigned with weights.

    \subsection{Convolutions}
    An effective solution to such issue by utilizing the structure of image-encoded information is to assume that compared with the pixels at the opposite corners of the image, the spatially closer ones can construct some certain feature of interest cooperatively. Further, an important (smaller) feature in image label definition tends to possess equal importance wherever it is within the image.

    A convolution operator is entered. For a given two-dimensional image, I, and a small matrix, $\pmb{K}$ with size $h \times w$, (known as a convolution kernel), which is presumptively encoded with an extraction way of an interesting image feature, the convolved image, $\pmb{I} * \pmb{K}$, is computed by overlaying the kernel on image top  in every possible way and recording the sum of elementwise products between the image and the kernel:

    \begin{equation}
        (\pmb{I} * \pmb{K})_{xy} = \sum_{i=1}^{h}\sum_{j=1}^{w} \pmb{K}_{ij}\cdot \pmb{I}_{x+i-1, y+j-1}
    \end{equation}

    \begin{figure}[!h]
        \centering
        \includegraphics[scale=1.3]{Figures/convolve.png}
        \caption{One simple example of convolution.}
        \label{con}
    \end{figure}

    \subsection{Convolutional layers}
    Normal RGB images are represented by matrices containing color information in the form of Red-Gray-Blue color codes. An image therefore has size $h\times w \times d$, where $d$ is the number of channel of image, in normal RGB images, $d=3$. However, in the case of this thesis, channels consist of $[m, v_x, v_y, \omega, n_x]$, therefore in its case, $d=5$. Convolutional layers are essential layers in CNNs, producing feature maps from input images or lower level feature maps.

    Convolutional layers include a kernel (or filter). Let $K$ be a kernel with $x$ rows, $y$ columns and depth $d$. Then the kernel with size ($K_x \times K_y \times d$) works on a receptive field ($K_x \times K_Y$) on the image. The kernel height and width are smaller than the input image height and width. The kernel slides over (convolves with) the image, producing a feature map (Figure \ref{con}). Convolution is the sum of the element-wise multiplication of the kernel and the original image. Note that the depth d of the kernel is equal to the depth of its input. Therefore, it varies within the network. Usually, the depth of an image is the number of color channels, the three RGB channels. In this case, $d=5$.

    The kernel stride is a free parameter in convolutional layers which has to be defined before training. The stride is the number of pixels by which the kernel shifts at a time. A drawback of using convolutional layers is that it decreases the output map size. A larger stride will result in a smaller sized output. Equations \ref{out} show the relationship between output size $O$ and input size of an image $I$ after convolution with stride $s$ and kernel $K$. Furthermore, the feature map size decreases as the number of convolutional layers increases. Row output size $O_x$ and column output size $O_y$ of convolutional layers are determined as follows:
    \begin{equation}
        \begin{cases}
            O_x = \frac{I_x-K_x}{s} + 1 \\
            O_y = \frac{I_y - K_Y}{s} + 1 \\
        \end{cases}
        \label{out}
    \end{equation}
    As an example, an image of size ($32\times 32 \times 3$), a kernel of size ($3\times 3\times 3$) and a stride $s=1$ result in an activation map of size ($30 \times 30 \times 1$). Using additional $n$ kernels, the activation map becomes ($30\times30\times n$). So, additional kernels will increase the depth of the convolutional layer output. Online resources can be referred to for more animation examples with different types of convolution\footnote{\url{https://github.com/vdumoulin/conv_arithmetic}}.

    \begin{figure}[!h]
        \centering
        \includegraphics[scale = 0.3]{Figures/Conv_layers.png}
        \caption{Visualization of convolutions network, \textit{retrieved from Wikipedia}}
    \end{figure}

    \subsection{Activation Layer}
    Essentially, whether a neuron should be activated is decided by the activation functions. As a feature for the artificial neural networks of prime importance, they are in charge of distinguishing the relevant signals received by neuron from the irrelevant ones. Normally, we can express the general function as Equation \ref{ac}. And one mathematical model is shown in Figure \ref{action} to describe how activation function is.

    \begin{equation}
        y = f_{Activation}(\sum_{i}(w_i \cdot x_i) + bias)
        \label{ac}
    \end{equation}

    \begin{figure}
        \centering
        \includegraphics[scale=0.3]{Figures/action.pdf}
        \caption{Mathematical model for describing activation function}
        \label{action}
    \end{figure}

    By transforming the input signal, the nonlinear activation function can be obtained, which, in turn,  is subsequently sent to the next layer of neurons as an input. Rectified Linear Unit (ReLU) is considered as the activation function that enjoys the most pervasive application nowadays. More details are presented below.

    \subsubsection{Rectified Linear Unit}
    ReLU commonly appears in the mathematical form as follows,
    \begin{equation}
        y = max(0, x)
    \end{equation}
    Thanks to its linear non-saturating form (e.g. a factor of 6 in \cite{krizhevsky2012imagenet}), a great accelerating effect on converging the stochastic gradient descent were discovered for ReLU when compared with the $sigmoid$/$tanh$ functions. Since then, it has become a very favorable approach. Vanishing or exploding gradient can hardly affect it actually. Further, it is much more economical than the conventionals exponentials. Nonetheless, ReLU cannot cater for all datasets and architectures, for it may get rid of every piece of negative information.

    \subsection{Pooling Layer}
    Pooling layers are also known as downsampling layers. A commonly used pooling is max pooling (figure \ref{maxpooling}). The downsampled output is produced by taking the maximum input value within the kernel, resulting in the output with a decreased size. There are several other methods which are commonly used in neural networks, such as average pooling and L2-norm pooling. Average pooling used to be a common method, while it is quitting the historical arena these days due to the more practical mmaxpooling\cite{scherer2010evaluation}.

    There are two important arguments for implementing pooling layers,
    \begin{enumerate}
        \item Decreasing the number of weights.
        \item Decreasing the chance of overfitting the training data.
    \end{enumerate}
    \begin{figure}[!h]
        \centering
        \includegraphics[scale=0.5]{Figures/Max_pooling.pdf}
        \caption{Maxpooling with a ($2\times 2$) kernel and stride $s=2$. Maxpooling layers reduce spatial dimension of the input \cite{li2015convolutional}}
        \label{maxpooling}
    \end{figure}

    \subsection{Fully Connected Layer}
    Following the steps of convolutional and maxpooling layers, fully connected layer realizes the high-level reasoning in neural networks eventually. .As what a regular neural network shows, all activations in the above-mentioned layers are connected by the neurons at a fully connected layer. A matrix multiplication and the follow-up bias offset will then complete all the calculations of these activations.

    \subsection{Batch Normalization}
    \label{batchnor}
    Because of its assistance in faster convergence, this layer shoots to popularity rapidlyr\cite{ioffe2015batch}. By adding a normalization step of shifting inputs to zero-mean and unit variance, it achieves the comparability across features of inputs in every trainable layer. Consequently, a high learning rate is guaranteed during the network learning process.

    Besides, activation functions like TanH and Sigmoid are protected by it from getting stuck in saturation mode (e.g. gradient equal to $0$).

    \section{Training Method}
    \label{training}
    \subsection{Loss Function}
    The value of the loss function $L$ represents the difference between the training image after it has propagated through the network and desired annotated output image.

    Two assumptions are made about this loss function.
    \begin{enumerate}
        \item It is able to be defined as the average over the loss functions for individual training images, as the training often is carried out in batches.
        \item It should be able to be defined as a function of the network outputs.
    \end{enumerate}

    Below a brief overview is given of some widely used loss functions, where $f_{\theta}(x_i)$ are the neuron outputs and $y_i$ are the desired outputs.
    \subsubsection{Quadratic Cost Function}
    The Mean Squared Error (MSE) cost function is one of the simplest cost functions. Normally it will be used in estimation problems\cite{boureau2008sparse}.
    \begin{equation}
        L =\frac{1}{N}\sum_{n=1}^{N}(f_{\theta}(x_i) - y_i)^2
        \label{eq:mse}
    \end{equation}

    \subsection{Overfitting}
    Overfitting is a problem that arises in neural network training. When a model is overfitted to the training data, it loses its capability of generalization. The model has learned the training data, including noise, to such a great extent that it has failed to capture underlying general information. CNNs have a large number of weights to be trained, therefore overfitting can occur due to training too few training examples.

    \subsubsection{Regularization L2}
     As the first primary route to averting overfitting, the classical method of weight decay increased one more term to the cost function as a penalization for parameters in each dimension, so that the network is prevented from exact modeling of the training data and generalization of new examples is thus ensured.

    \begin{equation}
        Error(x, y) =  Loss(x, y) + \sum_{i}\theta_i
    \end{equation}
    where $\theta$ is with a vector containing all parameters in the network.

    \subsubsection{Data augmentation}
    The size of the training set can be enlarged by data augmentation in order to prohibit memorizing of the whole set by the model. A few types may be involved depending on the dataset. For instance, objects supposed to be constantly rotating,  such as galaxies or planktons, will fit different kinds of rotations to the original images.

    \subsubsection{Dropout}
    Dropout layers\cite{srivastava2014dropout} are a tool to prevent overfitting (Figure \ref{dropout}). In dropout, nodes and its connections are randomly dropped from the network. Dropout constrains the network adaptation to the training set, consequently it prevents that the weights are not too much fitted for this data. Training data will thus have reduced performance difference with validation date. Dropout layers are used during training only, not during validation or testing. Nowadays, dropout method has been the main method to prevent overfitting.

    \begin{figure}[!h]
        \centering
        \includegraphics[scale=0.7]{Figures/dropout.pdf}
        \caption{A nerual network structure before and after applying dropout}
        \label{dropout}
    \end{figure}

    \subsubsection{Early Stopping}
    \textbf{Early Stopping} is another normal way to prevent overfitting.
    It makes the training stop before it is to be overfitted by the model, which is practically useful during the training process of neural networks.

    \subsection{Stochastic Gradient Descent Variants}
    \label{sgd}
    In both Gradient Descent (GD) and Stochastic Gradient Descent (SGD) parameters are updated according to an update rule to minimize a loss function in an iterative manner. Computing the exact gradient using GD in large datasets is expensive (GD is deterministic), as this method runs through all training samples to perform a single update for one iteration step. In Stochastic Gradient Descent (or on-line Gradient Descent) an approximation of the true gradient is computed. This is done by using only one or a subset of training samples for a parameter update. When using a subset of training samples, this method is called mini-batch SGD.

    SGD is a method to minimize the loss function \(L(\theta)\)  parametrized by $\theta$. This is achieved by updating $\theta$ in the negative gradient direction of the loss function $\nabla_{\theta}L(\theta)$ with respect to the parameters, in order to decrease the loss function value. The learning rate $\eta$ determines the step size to get to the local or global minimum.
    \begin{equation}
        \theta_{n+1} = \theta_{n} - \eta \nabla_{\theta_{n}}L(f_{\theta_{n}}(x_i), y_i)
    \end{equation}

    \subsubsection{Mini-batch Stochastic Gradient Descent}
    This method performs an update for every mini-batch of $n$ training samples. Mini-batch SGD reduces the variance of the parameter updates. Larger mini-batches reduce the variance of SGD updates by taking the average of the gradients in the mini batch. This allows taking bigger step sizes. In the limit, if each batch contains one training sample, it is the same as regular SGD \cite{zhang2015deep}.

    \subsection{Learning Rate Scheduling in Gradient Optimization}
    \label{learning}
    There are serval variants of SGD available. Determining the appropriate learning rate, or step size, often is a complex problem. Applying too high learning rate cause suboptimal performance, while too low learning rate led to slow convergence. Learning rate scheduling is used as an extension of the SGD algorithm to improve performance. In learning rate scheduling, the learning rate is a decreasing function of the iteration number. Therefore, the first iterations have larger learning rate and consequently cause bigger parameter changes. Later iterations have similar learning rates, responsible for fine-tuning. I gave an overview of some gradient descent optimization algorithms,

    \subsubsection{Momentum}
    Momentum method is a method to speed up the SGD in the relevant direction.  A fraction $\gamma$ of the previous update is added to the current update. The mathematical details are defined in Equation \ref{mo}.
    \begin{equation}
        \begin{aligned}
            & v_n = \gamma v_{n-1} + \eta \nabla_{\theta}L(\theta) \\
            & \theta= \theta - v_{n}
        \end{aligned}
        \label{mo}
    \end{equation}

    \subsubsection{Nesterov Accelerated Gradient}
    The Momentum method does not take into the direction it is going in, while parameters’ next position can be approximated via the Nesterov Accelerated Gradient method. The update rule is given in Equation \ref{nag}.

    \begin{equation}
        \begin{aligned}
            & v_n = \gamma v_{n-1} + \eta \nabla_{\theta}L(\theta - \gamma v_{n-1}) \\
            & \theta= \theta - v_{n}
        \end{aligned}
        \label{nag}
    \end{equation}

    %\subsubsection{Adam}
    %The Adaptive Moment Estimation (Adam) optimizer \cite{DBLP:journals/corr/KingmaB14} determines an adaptive learning rate %for each parameter. Apart from the decaying average for past squared gradients $v_n$, an exponentially decaying average %is also maintained by Adam for the past gradients $m_n$. Estimates for the mean and the uncentered variance of the %gradients are Vectors $v_n$ and $m_n$, respectively, and both of them are biased towards zero. Bias-corrected estimates $\%hat{v}_t$ and $\hat{m}_t$ are computed for the update rule in Equation \ref{adam}.

    %\begin{equation}
    %    \theta_{n+1} = \theta_{n} - \frac{\eta}{\sqrt{\hat{v}_n} + \epsilon}\cdot \hat{m}_n
    %\end{equation}

    \subsection{Backpropagation}
     The CNN requires to adjust and update its kernel parameters, or weights, for the given training data. Backpropagation\cite{werbos1990backpropagation} is an efficient method for computing gradients required to perform gradient-based optimization of the weights in neural networks [19]. Minimizing the loss function (or error function), the weights are combined together on purpose for solving the problem of optimization. Herein, the gradient of error functions needs computing during each iteration, which requires continuity and differentiability of the loss functions for all iteration steps.

    The initial weights of an untrained CNN are randomly chosen. Consequently before training, the neural network cannot make meaningful predictions for network input, as there is no relation between an image and its labeled output yet. By exposing the network to a training data set, comprising images and their labeled outputs with correct classes, the weights are adjusted. Training is the adaptation of the weights in such way that the difference between desired output and network output is minimized, which means that the network is trained to find the right features required for classification. There are two computational phases in a neural network, the forward pass and the backward pass in which the weights are adapted.

    \subsubsection{Forward pass}
    An image is fed into a network. The first network layer outputs an activation map. Then, this activation map is the input to the first hidden layer, which computes another activation map. Using the values of this activation map as inputs to the second hidden layer, again another activation map is computed. Carrying out this process for every layer will eventually yield the network output.

    \subsubsection{Backward pass}
    In this phase, the weights are updated by backpropagation. One epoch of backpropagation consists of multiple parts, usually multiple epochs are carried out for a training image:
    \begin{enumerate}
        \item \textbf{Loss Function} In the forward pass, the inputs and desired outputs are presented. A pre-defined loss function $L$ is used to minimize the disparity when comparing the input and desired output. The goal is to adjust the weights so that the loss function value decreases, this is achieved by calculating the derivative with respect to the weights of the loss function.
        \item \textbf{Backward pass} During the backward pass, the weights that have contributed the most to the loss are chosen in order to adjust them so that the total loss decreases.
        \item \textbf{Weight update} In the final part all weights are updated in the negative direction of the loss function gradient.
    \end{enumerate}

    Hence, gradient computation for loss functions in terms the network weights is actually the core of backpropagation problem. Computing the partial derivative $\frac{\partial L}{\partial \omega}$ is essential (carried out in the backward pass) to minimize the loss function value. Stochastic Gradient Descent (SGD) is the most common way to optimize neural networks.

    \subsubsection{Backpropagation Example for a Multi-Layer Network}
    I describe the details of the backpropagation algorithm for one simple example in Figure \ref{bp}. The cost function $L$ is given below, $el$ is the error between the true output $d_l$ and network output $y_l$. The network output $y_l$ is computed in the forward pass and depends on outputs of the previous layer $v_j$ and the output layer weights $w^{o}_j$.
    \begin{figure}[!h]
        \centering
        \includegraphics{Figures/bp.pdf}
        \caption{The example for showing the details of backpropagation}
        \label{bp}
    \end{figure}
    Some mathesmatical equations we can get:
    \begin{equation}
        \begin{aligned}
            L &= (e_l)^2 \\
            e_l &= d_l -y_l \\
            y_l &= \sum_{j}w^{o}_j v_j
        \end{aligned}
        \label{me}
    \end{equation}

    The Jacobian is given by:
    \begin{equation}
        \frac{\partial L}{\partial w_{j}} = \frac{\partial L}{\partial e_l} \cdot \frac{\partial e_l}{\partial y_l} \cdot \frac{\partial y_l}{\partial w_{j}}
        \label{par_w}
    \end{equation}
    Then combining Equation \ref{me} and \ref{par_w}, we can caculate that,
    \begin{equation}
        \frac{\partial L}{\partial w^{o}_{j}} = -2v_j e_l
    \end{equation}

    Using SGD updating rules \textbf{Momentum} mentioned in section \ref{sgd} Equation \ref{mo}, the output weights are updated using:
    \begin{equation}
        w^{o}_{j}(n+1) = w^{o}_{j}(n) + \alpha(n)v_j e_j
    \end{equation}
    $\alpha(n)$ stands for the learning rate in $n$th-iteration, you can see more details in section \ref{sgd}.

    After having updated the output weights, the weights in the hidden layers can be updated. As it is a backward pass, first gradients of the output layers are computed, then the gradients of the hidden layers. The Jacobian is given as follows:

    \begin{equation}
        \frac{\partial L}{\partial w^{h}_{ij}} = \frac{\partial L}{\partial e_l} \cdot \frac{\partial e_l}{\partial y_l} \cdot \frac{\partial y_l}{\partial v_{j}} \cdot \frac{\partial v_j}{\partial w^{h}_{ij}}
        \label{par_w}
    \end{equation}

    Based on the networks shown in Figure \ref{bp}, we can define $v_j$ based on $w_ij$ and input $x_i$
    \begin{equation}
        v_j = \sum_{i=1}^{N_{input}}x_i\cdot w^{h}_{ij}
        \label{vj}
    \end{equation}

    Then combining Equation \ref{par_w} and \ref{vj}, we can calculate the Jacbian,
    \begin{equation}
        \frac{\partial L}{\partial w_{ij}^{h}} = -2e_l w_j^{o} x_i
    \end{equation}

    Which yields the update rule for the hidden layers:
    \begin{equation}
        w_{ij}^{h}(n+1) = w_{ij}^{h}(n)+ 2\alpha(n)e_l w_j^{o} x_i
    \end{equation}
    Finally the network is tested using a test dataset, this dataset contains data that differ from the ones in the training dataset. By increasing the amount of training data, the more training iterations are carried out, the better the weights are tuned.

\section{Summary}
Due to the great success which CNNs have achieved in computer vision and image related area. CNN will be a good choice for us to predict contact images($\pmb{G}_{\lambda}$). The main part of CNNs, convolutional layers, could extract the essential feature maps well from the original data. Essentially, we need to find the relationship between bodies image and contacts image. Due to CNNs' characteristics and successful using example, I would determine CNN as the deep learning model for this project.


\documentclass{memoir}
\usepackage{notestemplate}

%\logo{~/School-Work/Auxiliary-Files/resources/png/logo.png}
%\institute{Rice University}
%\faculty{Faculty of Whatever Sciences}
%\department{Department of Mathematics}
%\title{Class Notes}
%\subtitle{Based on MATH xxx}
%\author{\textit{Author}\\Gabriel \textsc{Gress}}
%\supervisor{Linus \textsc{Torvalds}}
%\context{Well, I was bored...}
%\date{\today}

\begin{document}

% \maketitle

% Notes taken on whenever

\section{Composite Field Extensions}
\label{sec:composite_field_extensions}

\begin{defn}[Composite Field]
	Let \(K_1\) and \(K_2\) be two subfields of a field \(K\). Then the \textbf{composite field of \(K_1\) and \(K_2\)}, denoted by \(K_1K_2\) is the smallest subfield of \(K\) containing both \(K_1\) and \(K_2\).
\end{defn}
The composite of any collection of subfields \(\left\{ K_i \right\} \) is defined similarly.\\\

\begin{prop}
	Let \(K_1\) and \(K_2\) be two finite extensions of \(F\) contained in \(K\). Then
	\begin{align*}
		[K_1K_2:F] \leq [K_1:F][K_2:F]
	\end{align*}
	with equality if and only if an \(F\)-vector space basis for \(K_1\) is linearly independent over \(K_2\) (or vice versa).
\end{prop}
If the \(F\)-vector space basis of \(K_1\) is \(\alpha_1,\ldots,\alpha_n\) and the \(F\)-vector space basis of \(K_2\) is \(\beta_1,\ldots,\beta_m\), then \(\left\{ \alpha_i \beta_j \right\}_{i,j = 1}^{n,m}\) is a \(F\)-vector span of \(K_1K_2\).
\begin{cor}
	If, furthermore, \([K_1:F] = n\) and \([K_2:F] = m\) with \(\textrm{gcd}(n,m) = 1\), then \([K_1K_2:F] = [K_1:F][K_2:F] = nm\).
\end{cor}

\begin{exmp}
	\begin{itemize}
		\item Consider \(K = \Q(\sqrt{2} )\Q(\sqrt[3]{2}) \). We have
			\begin{align*}
				\Q &\hookrightarrow^{2} \Q(\sqrt{2} ) \hookrightarrow^{3} \Q(\sqrt{2})\Q(\sqrt[3]{2}) = \Q(\sqrt[6]{2})\\
				\Q &\hookrightarrow^{3} \Q(\sqrt[3]{2}) \hookrightarrow^{2} \Q(\sqrt[6]{2})\\
				\Q &\hookrightarrow^{6} \Q(\sqrt[6]{2})
			\end{align*}
			where \(\hookrightarrow^{k}\) represents a degree \(k\) extension. % Fill in rest of field extensions here, and perhaps turn into diagrams
	\end{itemize}
\end{exmp}

\section{Splitting Fields}
\label{sec:splitting_fields}

Recall that for any field \(F\) and any polynomial \(f(x) \in F[x]\), there exists a field extension \(K\) over \(F\) that contains a root, say \(\alpha  \in K\), of \(f(x)\). In this case, \(f(x) = (x-\alpha )g(x)\) in \(K[x]\) as \(K[x]\) is a Euclidean domain.\\

Now we want a field extension \(K / F\) so that \(f(x) \in F[x]\) splits completely into linear factors in \(K[x]\).

\begin{defn}
	A field extension \(K\) of \(F\) is called a \textbf{splitting field for \(f(x) \in F[x]\)} if \(f(x) = \prod_{i} (x - \alpha_i) \) in \(K[x]\) and \(f(x)\) does NOT factor completely in \(K'[x]\) for any proper subfield \(K'\) of \(K\).
\end{defn}
\(f(x) \in K[x]\) splits completely if and only if \(K\) contains all roots of \(f(x)\).

\begin{exmp}
	\begin{itemize}
		\item The splitting field of \(x^2-2\) over \(\Q\) is \(\Q(\sqrt{2} )\)
		\item The splitting field of \((x^2-2)(x^2-3)\) over \(\Q\) is \(\Q(\sqrt{2} ,\sqrt{3} )\)
		\item The splitting field of \(x^3-2\) over \(\Q\) is NOT \(\Q(\sqrt[3]{2})\). The roots \(\sqrt[3]{2}\omega \) and \(\sqrt[3]{2}\omega^2 \) are in fact imaginary and hence are not in \(\Q(\sqrt[3]{2})\) (note that \(\omega \) represents the principal root of unity).
	\end{itemize}
\end{exmp}

\begin{thm}
	Splitting fields always exist. For any field \(F\), if \(f(x) \in F[x]\), then there exists a field extension \(K\) of \(F\) that is a splitting field for \(f(x)\).
\end{thm}
\end{document}

Abstract Algebra - Introductory/Algebra II/Notes/source/Lecture18 - CompositeSplittingFields.tex