# [ATM 623: Climate Modeling](../index.ipynb)
[Brian E. J. Rose](http://www.atmos.albany.edu/facstaff/brose/index.html), University at Albany
# Lecture 6: Elementary greenhouse models

### About these notes:

This document uses the interactive [`IPython notebook`](http://ipython.org/notebook.html) format (now also called [`Jupyter`](https://jupyter.org)). The notes can be accessed in several different ways:

- The interactive notebooks are hosted on `github` at https://github.com/brian-rose/ClimateModeling_courseware
- The latest versions can be viewed as static web pages [rendered on nbviewer](http://nbviewer.ipython.org/github/brian-rose/ClimateModeling_courseware/blob/master/index.ipynb)
- A complete snapshot of the notes as of May 2015 (end of spring semester) are [available on Brian's website](http://www.atmos.albany.edu/facstaff/brose/classes/ATM623_Spring2015/Notes/index.html).

Many of these notes make use of the `climlab` package, available at https://github.com/brian-rose/climlab

## Contents

1. [A single layer atmosphere](#section1)
2. [Introducing the two-layer leaky greenhouse](#section2)
3. [Tuning the leaky greenhouse model to observations](#section3)
4. [Level of emission](#section4)
5. [Radiative forcing in the 2-layer leaky greenhouse](#section5)
6. [Radiative equilibrium in the 2-layer leaky greenhouse](#section6)

____________

## 1. A single layer atmosphere
____________

We will make our first attempt at quantifying the greenhouse effect in the simplest possible greenhouse model: a single layer of atmosphere that is able to absorb and emit longwave radiation.

```python
from IPython.display import Image
Image('../images/MarshallPlumbFig2.6.png')
```

*Figure reproduced from Marshall and Plumb (2008): Atmosphere, Ocean, and Climate Dynamics*

### Assumptions

- Atmosphere is a single layer of air at temperature $T_a$
- Atmosphere is **completely transparent to shortwave** solar radiation.
- Atmosphere is **completely opaque to infrared** radiation
- Both surface and atmosphere emit radiation as **blackbodies**
- Atmosphere radiates **equally up and down** ($A\uparrow = A\downarrow = \sigma T_a^4$)
- There are no other heat transfer mechanisms

We can now use the concept of energy balance to ask what the temperature need to be in order to balance the energy budgets at the surface and the atmosphere, i.e. the **radiative equilibrium temperatures**.

### Energy balance at the surface

\begin{align}
\text{energy in} &= \text{energy out} \\
(1-\alpha) Q + \sigma T_a^4 &= \sigma T_s^4 \\
\end{align}

The presence of the atmosphere above means there is an additional source term: downwelling infrared radiation from the atmosphere.

We call this the **back radiation**.

### Energy balance for the atmosphere

\begin{align}
\text{energy in} &= \text{energy out} \\
\sigma T_s^4 &= A\uparrow + A\downarrow = 2 \sigma T_a^4 \\
\end{align}

which means that
$$ T_s = 2^\frac{1}{4} T_a \approx 1.2 T_a $$

So we have just determined that, in order to have a purely **radiative equilibrium**, we must have $T_s > T_a$.

The surface must be warmer than the atmosphere.

### Solve for the radiative equilibrium surface temperature

Now plug this into the surface equation to find

$$ \frac{1}{2} \sigma T_s^4 = (1-\alpha) Q  $$

and use the definition of the emission temperature $T_e$ to write

$$ (1-\alpha) Q = \sigma T_e^4 $$

*In fact, in this model, $T_e$ is identical to the atmospheric temperature $T_a$, since all the OLR originates from this layer.*

Solve for the surface temperature:
$$ T_s = 2^\frac{1}{4} T_e $$

Putting in observed numbers, $T_e = 255$ K gives a surface temperature of
$$T_s = 303 ~\text{K}$$

This model is one small step closer to reality: surface is warmer than atmosphere, emissions to space generated in the atmosphere, atmosphere heated from below and helping to keep surface warm.

BUT our model now overpredicts the surface temperature by about 15ºC (or K).

Ideas about why?

Basically we just need to read our list of assumptions above and realize that none of them are very good approximations:

- Atmosphere absorbs some solar radiation.
- Atmosphere is NOT a perfect absorber of longwave radiation
- Absorption and emission varies strongly with wavelength (atmosphere does not behave like a blackbody).
- Emissions are not determined by a single temperature $T_a$ but by the detailed vertical profile of air temperture.
- Energy is redistributed in the vertical by a variety of dynamical transport mechanisms (e.g. convection and boundary layer turbulence).

____________

## 2. Introducing the two-layer leaky greenhouse
____________

Let's generalize the above model just a little bit to build a slighly more realistic model of longwave radiative transfer.

We will address two shortcomings of our single-layer model:
1. No vertical structure
2. 100% longwave opacity

Relaxing these two assumptions gives us what turns out to be a very useful prototype model for **understanding how the greenhouse effect works**.

### Assumptions

- The atmosphere is **transparent to shortwave radiation** (still)
- Divide the atmosphere up into **two layers of equal mass** (the dividing line is thus at 500 hPa pressure level)
- Each layer **absorbs only a fraction $\epsilon$ ** of whatever longwave radiation is incident upon it.
- We will call the fraction $\epsilon$ the **absorptivity** of the layer.
- Assume $\epsilon$ is the same in each layer

Note that this last assumption is appropriate is the absorption is actually carried out by a gas that is **well-mixed** in the atmosphere.

Out of our two most important absorbers:

- CO$_2$ is well mixed
- H$_2$O is not (mostly confined to lower troposphere due to strong temperature dependence of the saturation vapor pressure).

But we will ignore this aspect of reality for now.

In order to build our model, we need to introduce one additional piece of physics known as **Kirchoff's Law**:
$$ \text{absorptivity} = \text{emissivity} $$

So if a layer of atmosphere at temperature $T$ absorbs a fraction $\epsilon$ of incident longwave radiation, it must emit
$$ \epsilon ~\sigma ~T^4 $$
both up and down.

### A sketch of the radiative fluxes in the 2-layer atmosphere

```python
Image('../images/2layerAtm_sketch.png', retina=True)
```

- Surface temperature is $T_s$
- Atm. temperatures are $T_0, T_1$ where $T_0$ is closest to the surface.
- absorptivity of atm layers is $\epsilon$
- Surface emission is $\sigma T_s^4$
- Atm emission is $\epsilon_0 \sigma T_0^4, \epsilon_1 \sigma T_1^4$ (up and down)
- Absorptivity = emissivity for atmospheric layers
- a fraction $(1-\epsilon)$ of the longwave beam is **transmitted** through each layer

### Longwave emissions
Let's denote the emissions from each layer as
\begin{align}
E_s &= \sigma T_s^4 \\
E_0 &= \epsilon \sigma T_0^4 \\
E_1 &= \epsilon \sigma T_1^4
\end{align}
recognizing that $E_0$ and $E_1$ contribute to **both** the upwelling and downwelling beams.

### Shortwave radiation
Since we have assumed the atmosphere is transparent to shortwave, the incident beam $Q$ passes unchanged from the top to the surface, where a fraction $\alpha$ is reflected upward out to space.

### Upwelling beam

Let $U$ be the upwelling flux of longwave radiation.

The upward flux from the surface to layer 0 is
$$ U_0 = E_s $$
(just the emission from the suface).

Following this beam upward, we can write the upward flux from layer 0 to layer 1 as the sum of the transmitted component that originated below layer 0 and the new emissions from layer 0:
$$ U_1 = (1-\epsilon) U_0 + E_0 $$

Continuing to follow the same beam, the upwelling flux above layer 1 is
$$ U_2 = (1-\epsilon) U_1 + E_1 $$

Since there is no more atmosphere above layer 1, this upwelling flux is our Outgoing Longwave Radiation for this model:

\begin{align}
OLR &= U_2 \\
  &= (1-\epsilon) \bigg((1-\epsilon) \Big( \sigma T_s^4 \Big) + \epsilon \sigma T_0^4 \bigg) + \epsilon T_1^4 \\
  &= (1-\epsilon)^2 \sigma T_s^4 + \epsilon(1-\epsilon) \sigma T_0^4 + \epsilon T_1^4
\end{align}

Notice that the three terms in the OLR represent the **contributions to the total OLR that originate from each of the three levels**.

### Downwelling beam

Let $D$ be the downwelling longwave beam. Since there is no longwave radiation coming in from space, we begin with
$$ D_2 = 0$$

Between layer 1 and layer 0 the beam contains emissions from layer 1:
$$ D_1 = E_1  = \epsilon \sigma T_1^4$$
( in general we can write $D_1 = (1-\epsilon)D_2 + E_1$ if we are dealing with a non-zero $D_2$)

Finally between layer 0 and the surface the beam contains a transmitted component and the emissions from layer 0:
$$ D_0 = (1-\epsilon) D_1 + E_0 = \epsilon(1-\epsilon) \sigma T_1^4 + \epsilon \sigma T_0^4$$

This $D_0$ is what we call the **back radiation**, i.e. the longwave radiation from the atmosphere to the surface.

____________

## 3. Tuning the leaky greenhouse model to observations
____________

In building our new model we have introduced exactly one parameter, the absorptivity $\epsilon$. We need to choose a value for $\epsilon$.

We will tune our model so that it **reproduces the observed global mean OLR** given **observed global mean temperatures**.

To get appropriate temperatures for $T_s, T_0, T_1$, let's revisit the [global, annual mean lapse rate plot from NCEP Reanalysis data](Lecture05 -- Radiation.ipynb) from the previous lecture.

### Temperatures
First, we set
$$T_s = 288 \text{ K}  $$

From the lapse rate plot, an average temperature for the layer between 1000 and 500 hPa is

$$ T_0 = 275 \text{ K}$$

Defining an average temperature for the layer between 500 and 0 hPa is more ambiguous because of the lapse rate reversal at the tropopause. We will choose

$$ T_1 = 230 \text{ K}$$

From the graph, this is approximately the observed global mean temperature at 275 hPa or about 10 km.

### OLR

From the [observed global energy budget](Lecture01 -- Planetary energy budget.ipynb) we set

$$ OLR = 239 \text{ W m}^{-2} $$

### Solving for $\epsilon$

We wrote down the expression for OLR as a function of temperatures and absorptivity in our model above. We just need to equate this to the observed value and solve a quadratic equation for $\epsilon$.

We will use the symbolic math packaged called `sympy` to help us out here.

```python
import sympy
sympy.init_printing()
epsilon, T_s, T_0, T_1, sigma = sympy.symbols('epsilon, T_s, T_0, T_1, sigma')

#  Define the contributions to OLR originating from each level
OLR_s = (1-epsilon)**2 *sigma*T_s**4
OLR_0 = epsilon*(1-epsilon)*sigma*T_0**4
OLR_1 = epsilon*sigma*T_1**4

OLR = OLR_s + OLR_0 + OLR_1

print 'The expression for OLR is'
OLR
```

Subsitute in the numerical values we are interested in:

```python
OLR2 = OLR.subs([(sigma, 5.67E-8), (T_s, 288.), (T_0, 275.), (T_1, 230.)])
OLR2
```

Now use the `sympy.solve` function to solve the quadratic equation for $\epsilon$:

```python
sympy.solve(OLR2 - 239., epsilon)
```

There are two roots, but the second one is unphysical since we must have $0 < \epsilon < 1$.

We conclude that our tuned value is

$$ \epsilon = 0.58$$

This is the absorptivity that guarantees that our model reproduces the observed OLR given the observed tempertures.

____________

## 4. Level of emission
____________

Even in this very simple greenhouse model, there is **no single level** at which the OLR is generated.

The three terms in our formula for OLR tell us the contributions from each level.

Let's make a row vector of these three terms:

```python
OLRterms = sympy.transpose(sympy.Matrix([OLR_s, OLR_0, OLR_1]))
OLRterms
```

Now evaluate these expressions for our tuned temperature and absorptivity:

```python
tuned = [(T_s, 288), (T_0, 275), (T_1, 230), (epsilon, 0.58), (sigma, 5.67E-8)]
OLRtuned = OLRterms.subs(tuned)
OLRtuned
```

So we are getting about 69 W m$^{-2}$ from the surface, 79 W m$^{-2}$ from layer 0, and 92 W m$^{-2}$ from the top layer.

In terms of fractional contributions to the total OLR, we have (limiting the output to two decimal places):

```python
sympy.N(OLRtuned / 239., 2)
```

Notice that the largest single contribution is coming from the top layer. This is in spite of that the emissions from this layer are weak, because it is so cold.

Comparing to observations, the actual contribution to OLR from the surface is about 22 W m$^{-2}$ (or about 9% of the total), not 69 W m$^{-2}$. So we certainly don't have all the details worked out yet!

As we will see later, to really understand what sets that observed 22 W m$^{-2}$, we will need to start thinking about the spectral dependence of the longwave absorptivity.

____________

## 5. Radiative forcing in the 2-layer leaky greenhouse
____________

Adding some extra greenhouse absorbers will mean that a greater fraction of incident longwave radiation is absorbed in each layer.

Thus **$\epsilon$ must increase** as we add greenhouse gases.

Suppose we have $\epsilon$ initially, and the absorptivity increases to $\epsilon_2 = \epsilon + \delta_\epsilon$.

Suppose further that this increase happens abruptly so that there is no time for the temperatures to respond to this change. **We hold the temperatures fixed** in the column and ask how the radiative fluxes change.

**Do you expect the OLR to increase or decrease?**

Let's use our two-layer leaky greenhouse model to investigate the answer.

The components of the OLR before the perturbation are

```python
OLRterms
```

After the perturbation we have

```python
delta_epsilon = sympy.symbols('delta_epsilon')
OLRterms_pert = OLRterms.subs(epsilon, epsilon+delta_epsilon)
OLRterms_pert
```

Let's take the difference

```python
deltaOLR = OLRterms_pert - OLRterms
deltaOLR
```

To make things simpler, we will neglect the terms in $\delta_\epsilon^2$. This is perfectly reasonably because we are dealing with **small perturbations** where $\delta_\epsilon << \epsilon$.

Telling `sympy` to set the quadratic terms to zero gives us

```python
deltaOLR_linear = sympy.expand(deltaOLR).subs(delta_epsilon**2, 0)
deltaOLR_linear
```

Recall that the three terms are the contributions to the OLR from the three different levels. In this case, the **changes** in those contributions after adding more absorbers.

Now let's divide through by $\delta_\epsilon$ to get the normalized change in OLR per unit change in absorptivity:

```python
deltaOLR_per_deltaepsilon = sympy.simplify(deltaOLR_linear / delta_epsilon)
deltaOLR_per_deltaepsilon
```

Now look at the **sign** of each term. Recall that $0 < \epsilon < 1$. **Which terms in the OLR go up and which go down?**

**THIS IS VERY IMPORTANT, SO STOP AND THINK ABOUT IT.**

The contribution from the **surface** must **decrease**, while the contribution from the **top layer** must **increase**.

**When we add absorbers, the average level of emission goes up!**

### "Radiative forcing" is the change in radiative flux at TOA after adding absorbers

In this model, only the longwave flux can change, so we define the radiative forcing as

$$ R = - \delta OLR $$

(with the minus sign so that $R$ is positive when the climate system is gaining extra energy).

We just worked out that whenever we add some extra absorbers, the emissions to space (on average) will originate from higher levels in the atmosphere.

What does this mean for OLR? Will it increase or decrease?

To get the answer, we just have to sum up the three contributions we wrote above:

```python
R = -sum(deltaOLR_per_deltaepsilon)
R
```

Is this a positive or negative number? The key point is this:

**It depends on the temperatures, i.e. on the lapse rate.**

### Greenhouse effect for an isothermal atmosphere

Stop and think about this question:

If the **surface and atmosphere are all at the same temperature**, does the OLR go up or down when $\epsilon$ increases (i.e. we add more absorbers)?

Understanding this question is key to understanding how the greenhouse effect works.

#### Let's solve the isothermal case

We will just set $T_s = T_0 = T_1$ in the above expression for the radiative forcing.

```python
R.subs([(T_0, T_s), (T_1, T_s)])
```

which then simplifies to

```python
sympy.simplify(R.subs([(T_0, T_s), (T_1, T_s)]))
```

#### The answer is zero

For an isothermal atmosphere, there is **no change** in OLR when we add extra greenhouse absorbers. Hence, no radiative forcing and no greenhouse effect.

Why?

The level of emission still must go up. But since the temperature at the upper level is the **same** as everywhere else, the emissions are exactly the same.

### The radiative forcing (change in OLR) depends on the lapse rate!

For a more realistic example of radiative forcing due to an increase in greenhouse absorbers, we can substitute in our tuned values for temperature and $\epsilon$.

We'll express the answer in W m$^{-2}$ for a 1% increase in $\epsilon$.

The three components of the OLR change are

```python
deltaOLR_per_deltaepsilon.subs(tuned) * 0.01
```

And the net radiative forcing is

```python
R.subs(tuned) * 0.01
```

So in our example, **the OLR decreases by 2.2 W m$^{-2}$**, or equivalently, the radiative forcing is +2.2 W m$^{-2}$.

What we have just calculated is this:

Given the observed lapse rates, a small increase in absorbers will cause a small decrease in OLR.

The greenhouse effect thus gets stronger, and energy will begin to accumulate in the system -- which will eventually cause temperatures to increase as the system adjusts to a new equilibrium.

____________

## 6. Radiative equilibrium in the 2-layer leaky greenhouse
____________

In the previous section we made no assumptions about the processes that actually set the temperatures. We used the model to calculate radiative fluxes, **given observed temperatures**. We stressed the importance of knowing the lapse rates in order to know how an increase in emission level would affect the OLR, and thus determine the radiative forcing.

A key question in climate dynamics is therefore this:

**What sets the lapse rate?**

It turns out that lots of different physical processes contribute to setting the lapse rate. Understanding how these processes acts together and how they change as the climate changes is one of the key reasons for which we need more complex climate models.

For now, we will use our prototype greenhouse model to do the most basic lapse rate calculation: the **radiative equilibrium temperature**.

We assume that

- the only exchange of energy between layers is longwave radiation
- equilibrium is achieved when the net radiative flux convergence in each layer is zero.

```python
E_s = sigma*T_s**4
E_0 = epsilon*sigma*T_0**4
E_1 = epsilon*sigma*T_1**4
E = sympy.Matrix([E_s, E_0, E_1])
E
```

#### The upwelling beam

```python
U = sympy.Matrix([E_s, (1-epsilon)*E_s + E_0, (1-epsilon)*((1-epsilon)*E_s + E_0) + E_1])
U
```

#### The downwelling beam

```python
fromspace = 0
D = sympy.Matrix([(1-epsilon)*((1-epsilon)*fromspace + E_1) + E_0, (1-epsilon)*fromspace + E_1, fromspace])
D
```

```python
# Net flux, positive up
F = sympy.simplify(U - D)
F
```

```python
# The absorption is then simply the flux convergence in each layer

# define a vector of absorbed radiation -- same size as emissions
A = E.copy()

#  absorbed radiation at surface
A[0] = F[0]
# Get the convergence
for n in range(2):
    A[n+1] = -(F[n+1]-F[n])

A = sympy.simplify(A)
A
```

```python
# Solve for radiative equilibrium by setting this equal to zero
T_e = sympy.symbols('T_e')
sympy.solve(A - sympy.Matrix([sigma*T_e**4, 0, 0]),[T_s**4, T_1**4, T_0**4])
```

The radiative equilibrium solution is thus

\begin{align}
T_s &= T_e \left( \frac{2+\epsilon}{2-\epsilon} \right)^{1/4} \\
T_1 &= T_e \left( \frac{1+\epsilon}{2-\epsilon} \right)^{1/4} \\
T_2 &= T_e \left( \frac{ 1}{2 - \epsilon} \right)^{1/4}
\end{align}

Plugging in $\epsilon = 0.58$ gives

\begin{align}
T_s &= 296 \text{ K} \\
T_0 &= 262 \text{ K} \\
T_1 &= 234 \text{ K} \\
\end{align}

Compare these to the values we derived from the observed lapse rates:
\begin{align}
T_s &= 288 \text{ K} \\
T_0 &= 275 \text{ K} \\
T_1 &= 230 \text{ K} \\
\end{align}

The **radiative equilibrium** solution is substantially **warmer at the surface** and **colder in the lower troposphere** than reality.

This is a very general feature of radiative equilibrium, and we will see it again very soon in this course.

### A follow-up assignment

You are now ready to tackle [Assignment 5](../Assignments/Assignment05 -- Radiative forcing in a grey radiation atmosphere.ipynb), where you are asked to extend this grey-gas analysis to many layers.

For more than a few layers, the analytical approach we used here is no longer very useful. You will code up a numerical solution to calculate OLR given temperatures and absorptivity, and look at how the lapse rate determines radiative forcing for a given increase in absorptivity.

[Back to ATM 623 notebook home](../index.ipynb)

____________
## Version information
____________

```python
%install_ext http://raw.github.com/jrjohansson/version_information/master/version_information.py
%load_ext version_information
%version_information numpy, sympy, climlab
```

    Installed version_information.py. To use it, type:
      %load_ext version_information

Software	Version

Python	2.7.9 64bit [GCC 4.2.1 (Apple Inc. build 5577)]

IPython	3.1.0

OS	Darwin 14.3.0 x86_64 i386 64bit

numpy	1.9.2

sympy	0.7.6

climlab	0.2.11

Thu May 14 16:02:40 2015 EDT

____________
## Credits

The author of this notebook is [Brian E. J. Rose](http://www.atmos.albany.edu/facstaff/brose/index.html), University at Albany.

It was developed in support of [ATM 623: Climate Modeling](http://www.atmos.albany.edu/facstaff/brose/classes/ATM623_Spring2015/), a graduate-level course in the [Department of Atmospheric and Envionmental Sciences](http://www.albany.edu/atmos/index.php), offered in Spring 2015.
____________

Lectures/Lecture06 -- Elementary greenhouse models.ipynb


# Lab 1.1: Introduction to Labs

```python
%matplotlib widget
import rad.css as css
import rad.example as ex
import rad.quiz as qz
from rad.const import c, k
from rad.radar import to_db, from_db, deg2rad, rad2deg
from math import sqrt, sin, asin, cos, acos, tan, atan2, pi, log, log10
css.add_custom_css()
```

Welcome class! To begin, there will be a short overview of the interactive tools that we will use throughout the course.

The backbones of our tutorials are called **Jupyter notebooks**[[1]](#ref_jupyter). A Jupyter notebook is a convenient way to weave together text, images, math formulae, and working, editable code. Behind the scenes of the notebook is a programming language of choice: MATLAB, Python, Julia, Java, C++, etc. For this course, we will use Python[[2]](#ref_python) as it is openly available. Do *not* worry if you have little or no experience with Python: the course will require very little coding, and we will cover what is needed.

## Appearance

Before doing anything, we should start by making the notebooks easy to read.

### Font Size
First, we can change the font size: to increase the font size, go to *Settings > JupyterLab Theme > Increase Content Font Size*. This can be done repeatedly until the text is comfortable to read.

### Hidden Code
To make the notebooks easier to read, most background code needed for examples and quizzes has been hidden by default; anytime you see a large ellipsis , there is code being suppressed (e.g., one of them is right below the title at the top of this notebook). To see what is going on in the background, we can click on the ellipsis to reveal it.

## Preparing Notebooks

The first thing we will want to do upon opening a new notebook is click the *Run All* button  at the top; it will ask if it is okay to restart, we can select *Restart*. This ensures that all examples and interactive code will be loaded and ready. Go ahead and do it for this notebook.

## Notebooks and Cells

Each notebook is made up of a list of **cells**; each cell can be either text or code. The cell you are reading right now is an example of a **text cell** (also called a *Markdown* cell; Markdown[[3]](#ref_markdown) is a quick way to format text). Below this, we will see our first example of a **code cell**:

```python
x = 1
print("The value of x is " + str(x) + ".")
```

The code cell above sets the value of a variable ``x`` to one and then prints its value. To run the code, first single-click on the cell to highlight it. Then, to **evaluate** the cell, you can either:

  * Click the *Run Cell*  button on the top bar of the notebook

  * Press *Shift+Enter*

After evaluating the cell, you can see the output printed below it. Also, in brackets to the left of the cell, you can see the evaluation number; if you evaluate the cell again, you can see the number go up. The evaluation numbers help you keep track of what you have run and in what order.

To **edit** any cell, you can double-click on it; this means you can edit any code you want and rerun it to see what changed. As an example, you can double-click the code cell above and change the value of ``x`` to any number you want and then re-evaluate.

To **add** a cell, single-click on an existing cell and then click the *Add Cell Below*  button. A new cell will be created below the one you selected. To change the type of cell (i.e., code, text), select the cell by single-clicking and then use the dropdown menu at the top.

**Note:** You can also edit and evaluate text cells. Evaluating a text cell turns it from raw text into a properly formatted web snippet. This can be a helpful way to add notes to your personal version of a notebook.

***

### Try It Out

Below is an empty code cell. Enter code to assign a value of 3 to a variable ``y`` and then print its value. Try evaluating the code to make sure it works. *Hint: You can copy and paste from the code cell above.*

```python
# Enter code here
```

***

## Interactive Plots

The primary tools for interaction in Jupyter notebooks are interactive plots with controls called *widgets*. To see how widgets work, we can first look at plotting a simple sine wave. There will be three variables to change: *Amplitude*, *Frequency*, and *Initial Phase* (we will discuss these more later). Try moving the sliders for each variable to see the plot adjust.

```python
ex.ex_1_1_1()
```

There are two useful features in the interactive plots: *zooming* and *live cursor data*. To zoom in on a particular part of the plot, you can use the *Zoom Box*  button. Try clicking it and drawing a box around a part of the plot. To drag the plot around, you can use the *Pan*  button. To return to the original plot view at any time, you can click the *Reset View*  button. The current location of the cursor inside the plot is continuously displayed below the plot; this is very helpful for finding values and locations of points on a curve or image.

## Quizzes

There will also be a number of quizzes to challenge you as the course progresses. These will most often be in the form of a blank text box and a submit button. For example, below see an example quiz that asks you to enter an answer of one.

***

### Question 1

Enter a value of one.

```python
qz.quiz_1_1_1()
```

***

**Note:** Unless otherwise stated, the tolerance for quiz answers will be within ±1% of the true answer.

## Calculator

For some of the quizzes, it will be helpful to have a calculator handy. If one is not available, no problem! We can use the Jupyter notebook for calculations. In most spots where calculations are necessary, a *Scratch Space* code cell is provided that can be used. As discussed above, we can also simply add a code cell anywhere using the *Add Cell Below*  button.

Most mathematical operations are as you would expect. For instance, to calculate the value of $2\sin(5 \pi /2) + 1$, we would use the following code cell:

```python
2*sin(5*pi/2) + 1
```

Here is a short list of the mathematical operators:

| Operation           | Symbol |
|---------------------|--------|
| Addition            | `+`    |
| Subtraction         | `-`    |
| Multiplication      | `*`    |
| Division            | `/`    |
| Exponent            | `**`   |
| Square root         | `sqrt` |
| Logarithm           | `log`  |
| Logarithm (Base 10) | `log10`|

Here is a list of trigonometric functions:

| Function       | Symbol    |
|----------------|-----------|
| Sine           | `sin`     |
| Arcsine        | `asin`    |
| Cosine         | `cos`     |
| Arccosine      | `acos`    |
| Tangent        | `tan`     |
| Arctangent     | `atan2`   |

Here are some handy constants:

| Constant       | Symbol |
|----------------|--------|
| $\pi$          | `pi`   |
| Speed of light | `c`   |

**Note:** All of these functions (along with useful definitions and formulae) are kept in the [Reference](Reference.ipynb) notebook.

***

### Question 2

Using the functions in the tables above, calculate the value of $\log_{10}\left(2.72^3\right) + \cos(4\pi/7)$:

```python
qz.quiz_1_1_2()
```

```python
# Scratch space
```

***

### Engineering Notation

For very large or small numbers, it can be convenient to write them in the form $a \times 10^b$, such as $0.000005 = 5 \times 10^{-6}$. These can be represented in notebook calculations using *engineering notation*. All this means is instead of writing $a \times 10^b$, you type `aEb`. For instance, if you wanted to use $5 \times 10^{-6}$, you type `5E-6`.

***

### Question 3

Calculate the value of:
$$\frac{7.76 \times 10^{4}}{5.1 \times 10^{-3}}$$

```python
qz.quiz_1_1_3()
```

```python
# Scratch space
```

***

**Note:** One of the easiest ways to satisfy the ±1% tolerance for quiz answers is to write answers in engineering notation with at least two decimal places defined.

There are a few preliminary topics to cover before we get to the radar course itself. In the following, we will review the basics of:

- *Decibels*, a useful unit for simplifying radar analysis
- *Sine waves*, the main structure of radar transmissions

## Review: Decibels

Many quantities dealing with radar systems can be extremely large or small. For instance, it is common to use megawatts of power for transmission ($10^6~\mathrm{W}$), and receive signals on the order of picowatts ($10^{-12}~\mathrm{W}$). Instead of dealing with that immense scale of numbers, the quantities are often converted to **decibels** (dB); this makes the values much easier to remember and/or manipulate without pen and paper. Here is a short list of example numbers and their corresponding decibel value:

To convert a number, $x$, to the value in decibels, $x_d$, we use the following formula:
$$ x_d = 10\log_{10}(x) $$
In code for calculations, the conversion to decibels looks like `x_d = 10*log10(x)`. Going back to the example values for radar, instead of using $10^6~\mathrm{W}$, we can write $60~\mathrm{dBW}$; likewise, we can talk about receiving signals with a power of $-120~\mathrm{dBW}$ instead of dealing with $10^{-12}~\mathrm{W}$.

**Note:** For this course, the original unit will be written at the end of $\mathrm{dB}$ to keep track of it; for instance, $\mathrm{dBW}$ means it is in decibels and the original unit was watts.

***

### Question 4

**(a)** What is $1.5 \times 10^5$ in decibels?

```python
qz.quiz_1_1_4a()
```

```python
# Scratch space
```

**(b)** What is $7.2 \times 10^{-7}$ in decibels?

```python
qz.quiz_1_1_4b()
```

```python
# Scratch space
```

***

To convert from the value in decibels, $x_d$, back to the original value, $x$, we use:

$$
\displaystyle x = 10^{x_d/10}
$$

In other words: you first divide the number in decibels by ten, then take ten to that power. In code, this would be written as `x = 10**(x_d/10)`. You can verify this from the examples above.

***

### Question 5

**(a)** What is $51.2~\mathrm{dBW}$ in watts (W)?

```python
qz.quiz_1_1_5a()
```

```python
# Scratch space
```

**(b)** What is $-20.1~\mathrm{dBW}$ in watts (W)?

```python
qz.quiz_1_1_5b()
```

```python
# Scratch space
```

***

To save time in calculations, we have added two functions that you can use in your notebook calculations:

- `to_db()`, convert from original units to decibels
- `from_db()`, convert from decibels to original units

For instance, to convert $567.2~\mathrm{W}$ to $\mathrm{dBW}$, we can write:

```python
to_db(567.2)
```

which gives us $27.5374~\mathrm{dBW}$. Now to convert back, we write:

```python
from_db(27.5374)
```

A helpful property of decibels is *turning multiplication into addition*. This is convenient as it can be cumbersome to multiply very large or small numbers together quickly where adding their equivalent values in decibels. In general,
$$
\begin{align}
z &= xy\\
z_{d} &= x_{d} + y_{d}\\
\end{align}
$$
and
$$
\begin{align}
z &= x/y\\
z_{d} &= x_{d} - y_{d}\\
\end{align}
$$

This means we can turn the following multiplication: $z = (4.4 \times 10^{3})(2.8 \times 10^{-4})$ into its decibel equivalent: $z_d = 36.4345 + (-35.5284)$, which is much easier to estimate quickly, i.e., it is going to be roughly $1~\mathrm{dB}$.

The last property of decibels we will cover is *converting exponents into scalars*. Because the decibel is calculated using a logarithm, the following is true:

$$
\begin{align}
x &= a^b\\
x_{d} &= b \cdot a_d
\end{align}
$$

So, if we want the a quantity $a^b$ in decibels, we simply find $a$ in decibels (i.e., $a_d$) and then multiply it by $b$. For example, the value $43^5$ in decibels can be calculated as $5 \cdot 10\log_{10}(43)$, or using code `5*to_db(43)`.

***

### Question 6

What is $3.7^5$ in decibels? (Hint: $3.7$ in decibels is $5.682~\mathrm{dB}$)

```python
qz.quiz_1_1_6()
```

```python
# Scratch space
```

***

## Review: Sine Waves

Sine waves form the main backbone for ranging sensor transmissions because they are easy to generate with hardware and are well understood. A sine wave as a function of time, $y(t)$, is described mathematically as

$$
y(t) = a\sin(2\pi f t + \phi)
$$

where
- $a$ is the **amplitude**, which decides the size of the wave
- $f$ is the **frequency** of the wave (in Hz), which dictates how quickly the sine wave varies as a function of time, i.e., how many sine wave cycles per second
- $\phi$ is the initial **phase** (in radians), which defines where the sine wave starts at time $t = 0$

Another important property is the **period**, $T$, which is the inverse of the frequency (i.e., $T = 1/f$) and the duration of a sine wave cycle. This can be measured by calculating the time from peak-to-peak or valley-to-valley.

By eye, you can see the period and the amplitude of an example sine wave below.

We can see one peak at $t = 1~\mathrm{s}$ and the next one at $t = 6~\mathrm{s}$, so the period of this sine wave is $T = 6 - 1= 5~\mathrm{s}$, since the frequency is the inverse of the period, this gives a frequency of $f = 1/5~\mathrm{Hz}$. The amplitude is the distance from the zero to the peak (or valley) of a wave; in this case, the amplitude is $a = 5$.

We can now revisit the interactive plot from above, showing how amplitude, phase, and frequency affect a sine wave. Try putting in the values we just calculated, i.e., $a = 5, f = 0.2~\mathrm{Hz}, \phi = 0~\mathrm{deg}$, and verifying that the sine wave matches the picture above.

```python
ex.ex_1_1_1()
```

***

### Question 7

From the following plot of a sine wave, estimate its amplitude, period, and frequency. *Hint: Remember you can zoom in, pan, and use your cursor to find values of the curve.*

```python
ex.ex_1_1_2()
```

**(a)** What is the amplitude?

```python
qz.quiz_1_1_7a()
```

```python
# Scratch space
```

**(b)** What is the period?

```python
qz.quiz_1_1_7b()
```

```python
# Scratch space
```

**(c)** What is the frequency?

```python
qz.quiz_1_1_7c()
```

```python
# Scratch space
```

***

## Summary

In this lab, we covered the basics of Jupyter notebooks, along with reviews of decibels and sine wave properties. The main operations for interacting with Jupyter notebooks are:
- Click *Run All* button  to prepare a newly-opened notebook
- Create a new cell using the *Add Cell Below*  button
- Evaluate a cell using the *Run Cell*  button
- Interact with plots using the *Zoom Box* , *Pan*, and *Reset View*  buttons

Decibels are a great way to take extremely large and small numbers and convert them to a scale that is much easier to remember and manipulate. Conversion to decibels is $x_d = 10\log_{10}(x)$ and conversion back is $x = 10^{x_d/10}$. You can convert multiplications $z = xy$ into additions $z_d = x_d + y_d$ and exponentiation $x = a^b$ to scaling $x_d = b \cdot a_d$.

Sine waves are very important to radar as they provide the basis for transmissions. A sine wave is defined by its amplitude, frequency, and initial phase. The frequency of a sine wave is the number of cycles that occurs within a seconds and its period is the time between cycles; the period is the inverse of the frequency.

## Footnotes

n/a

## References

[1] *Project Jupyter* [https://jupyter.org](https://jupyter.org)

[2] *Python* [https://python.org](https://python.org)

[3] *Markdown Guide* [https://www.markdownguide.org/](https://www.markdownguide.org/)


# ... and so we begin

## Critical information

# First steps

## Order of the day

* Learn to use Jupyter / iPython Notebook

* Get familiar with basic Python

* Start with Spyder, a traditional editor

* Fundamental Python-in-Science skills

# What is Jupyter
## (previously iPython Notebook)

An interactive Q&A-style Python prompt, with output in formatted text, images, graphs and more (and it even works with other languages too)

A bit like a cross between Mathematica and Wolfram Alpha, that runs in your browser, but you can save all your worksheets locally. We will explore this, as it is very useful for doing quick calculations, collaborating on research, as a whiteboard, nonlinear discussions where you can adjust graphs or calculations, as teaching tool (I hope), or simply storing your train of thought in computations and notes. This series of slides was prepared in Jupyter, which is why I can do this...

It lets you do things like...

```python
import datetime
print(datetime.date.today())
```

You want to add in .weekday()

## Lets you output LaTeX-style (formatted) maths

Example calculating the output of $ \int x^3 dx $:

```python
from sympy import *
init_printing()
x = Symbol("x")
integrate(x ** 3, x)
```

and just to prove I'm not making it up... (change 3 to 6 and Ctrl+Enter)

# How we're going to approach this

* Motivated by analysing scientific data

* Learning about basic debugging and standard bits of the language

* Using Etherpad for discussion, group notes and info

* For a more detailed intro to Python in science, definitely check out [swcarpentry.github.io/python-novice-inflammation/](swcarpentry.github.io/python-novice-inflammation/)

* Also, highly recommended, UCL's resources: [http://development.rc.ucl.ac.uk/training/introductory/](http://development.rc.ucl.ac.uk/training/introductory/)

The dataset used in this course is that generated by the Software Carpentry team, ensuring it has been well tested across a large number of sessions internationally. They make their resources freely available under a Creative Commons license, so you should definitely check it out. I should mention that, while it has been helpful for preparing these sessions, this course is endorsed by or affiliated with Software Carpentry.

# Today's tools

## *Highly* **technical** ***sticky*** notes!

* Stars say **I'm just working away over here** - put them up when you start

 - use them to show you're busy

* Arrows say **I've completed the task** - put them up when you finish

helps us see when most people are ready to move on

# Linux

You all have machines running Linux - common in scientific computing, easier to manage libraries/programs, easier for me to help you.

1. Click on the word `Activities` (*top-left*)
2. Click the Firefox button (1/2 way down on left)

 This isn't a Linux course, so we're going to keep it very basic, and stick to programming

# Etherpad
## Live questions, notes and comments

In Firefox, go to https://etherpad.mozilla.org and enter

**qub-python-course-23Yn9**

1. Enter your name at the top-right
2. In the chat window (bottom right) say, **hi**, **hello**, **bout ye** *<Return>*
3. In the big window, stick your name at the end, with some info

Use the name box - you can create a new pad, but you *will* be talking to yourself.
2 - , or anything else (polite, obviously)
3 - on a new line - suggested info is on that page
4 - don't forget to put up your arrow sticker when done
5 - you can even pick your favourite colour by clicking the little box next to your name

(Etherpad name: **qub-python-course-23Yn9**)
https://etherpad.mozilla.org

<br/>

# Jupyter

## Today's first Python tool

1. Press Alt-F2 and type **jupyter notebook** *<Return>*
2. When the new window appears, click on **Basic control structures** to open it

This is our complicated command for the day - the rest we get to do by point-and-click

note the 'y'

Python Course - 002a - And so we begin.ipynb


# Chapter 8: Tree-Based Methods
- **Chapter 8 from the book [An Introduction to Statistical Learning](https://www.statlearning.com/)**
- **By Gareth James, Daniela Witten, Trevor Hastie and Rob Tibshirani**
- **Pages from $332$ to $333$**
- **By [Mosta Ashour](https://www.linkedin.com/in/mosta-ashour/)**

**Exercises:**
- **[1.](#1)**
- **[2.](#2)**
- **[3.](#3)**
- **[4.](#4) [(a)](#4a) [(b)](#4b)**
- **[5.](#5)**
- **[6.](#6)**

#  8.4 Exercises
##  Conceptual

### $1.$ Draw an example (of your own invention) of a partition of two-dimensional feature space that could result from recursive binary splitting. Your example should contain at least six regions. Draw a decision tree corresponding to this partition. Be sure to label all aspects of your figures, including the regions $R_1, R_2, \dots,$ the cutpoints $t_1, t_2, \dots,$ and so forth.
### *Hint: Your result should look something like Figures $8.1$ and $8.2$.*

                     [x1 > t1]
                    |         |
            [x2 > t3]         [x2 < t5]
            |       |         |       |
      [x2>t2]       R4     [x1<t6]    [x2<t7]
      |     |              |     |    |     |
[x1>t4]     R3             R5    R6   R7    [x1<t8]
|     |                                     |     |
R1    R2                                    R8    R9

### $2.$ It is mentioned in Section $8.2.3$ that boosting using depth-one trees (or *stumps*) leads to an *additive* model: that is, a model of the form

$$f(X) = \sum_{j=1}^{p}{f_j(X_j)}$$

### Explain why this is the case. You can begin with $(8.12)$ in Algorithm $8.2$.

**Answer:**
>- Boosting has three tuning parameters which the third one is: the number $d$ of splits in each tree, which controls the complexity of the boosted ensemble.
>- When $d = 1$, each tree is a *stump*, consisting of a *single split*.
>- In this case, the boosted ensemble is fitting an additive model, since each term involves only a single variable.
>- With this setting the Boosting for Regression Trees algorithm becomes:
>  1. Set $\hat{f}(x) = 0$ and $r_i = y_i$ for all $i$ in training set.
>  2. For $b = 1, 2, ..., B,$ repeat:

>  $(a)$ Fit a tree $\hat{f^b}$ with **1** split to the training data $(X, r)$.

>  $(b)$ Update $\hat{f}$ by adding the new tree:

>  $$\hat{f}(x) \leftarrow \hat{f}(x) + \hat{f}^b(X_j)$$
 **Note:** it can't be shrunken because it is already as small as it can possibly be, also note that it is a function of a single variable.

>  $(c)$ Update the residuals,
>  $$r_i \leftarrow r_i - \hat{f}^b(X_j)$$
>  3. Output the boosted model,
$$\hat{f}(x) = \sum_{b=1}^{B}{\hat{f}^b(X_j)}$$
>- Because the model is using depth-one trees, each $\hat{f}^b$ generated in step $2$ is a function of a single feature $\hat{f}^b(X_j)$ and so the model output in step $3$ is a sum of functions of a given single variable which is presented as the additive model.

### 3. Consider the Gini index, classification error, and cross-entropy in a simple classification setting with two classes. Create a single plot that displays each of these quantities as a function of $\hat{p}_{m1}$. The $x$-axis should display $\hat{p}_{m1}$, ranging from $0$ to $1$, and the $y$-axis should display the value of the Gini index, classification error, and entropy.

### *Hint: In a setting with two classes, $\hat{p}_{m1} = 1 - \hat{p}_{m2}$. You could make this plot by hand, but it will be much easier to make in R.*

**Answer:**
- Recall these settings from the book page $312$. For $k= \{1,2\}$:

>- $\text{Classification Error}:$
>  $$ \begin{align}
E &= 1 - \mathop{max}_k(\hat{p}_{mk}) \tag{8.5}\\
  &= 1 - \mathop{max}_k\{\hat{p}_{m1}, \hat{p}_{m2}\}
\end{align}$$

>- $\text{Gini Index}:$
>  $$ \begin{align}
G &= \sum_{k=1}^K \hat{p}_{mk}(1 - \hat{p}_{mk}), \tag{8.6}\\
  &= \hat{p}_{m1}(1 - \hat{p}_{m1}) + \hat{p}_{m2}(1 - \hat{p}_{m2})
\end{align}$$

>- $\text{Entropy}:$
>  $$ \begin{align}
D &= - \sum_{k=1}^K \hat{p}_{mk} log \hat{p}_{mk}. \tag{8.7}\\
  &= - \hat{p}_{m1} log \hat{p}_{m1} - \hat{p}_{m2} log \hat{p}_{m2}
\end{align}$$

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

pm1 = np.arange(0.01, 1, 0.01)
pm2 = 1 - pm1

# Classification Error:
clf_err = 1 - np.maximum(pm1, pm2)

# Gini:
gini = (pm1 * (1 - pm1)) + (pm2 * (1 - pm2))

# Entropy:
entropy = - (pm1 * np.log(pm1)) - (pm2 * np.log(pm2))

# dataframe
df = pd.DataFrame({'Classification_Error': clf_err,
                   'Gini': gini,
                   'Entropy': entropy}).set_index(pm1)

plt.figure(figsize=(10, 8))
sns.lineplot(data=df);
```

### $4.$ This question relates to the plots in Figure $8.12$.

**$(a)$ Sketch the tree corresponding to the partition of the predictor space illustrated in the left-hand panel of Figure $8.12$. The numbers inside the boxes indicate the mean of $Y$ within each region.**
                 [X1 < 1]
                |        |
         [X2 < 1]        5
         |      |
[ X1 < 0 ]      15
|        |
3        [X2 < 0]
         |      |
         10     0

**$(b)$ Create a diagram similar to the left-hand panel of Figure $8.12$, using the tree illustrated in the right-hand panel of the same figure. You should divide up the predictor space into the correct regions, and indicate the mean for each region.**

```python
# dividing up the predictor space into regions
plt.figure(figsize=(8, 8))
plt.xlim([-1, 3])
plt.ylim([-1, 3])

plt.axhline(1)
plt.axhline(2)
plt.axvline(1, ymax=.5)
plt.axvline(0, ymin=.5, ymax=.75)

# plot the mean for each region
means = {'-1.08': (0, 0),
         '0.63': (2, 0),
         '-1.06': (-0.5, 1.5),
         '0.21': (1.5, 1.5),
         '2.49': (1, 2.5)}

for k, (p1, p2) in means.items():
    plt.text(p1, p2, k, fontsize='xx-large', ha='center');
```

### $5.$ Suppose we produce ten bootstrapped samples from a data set containing red and green classes. We then apply a classification tree to each bootstrapped sample and, for a specific value of $X$, produce $10$ estimates of $\text{P(Class is Red|X)}$:
 $$0.1, 0.15, 0.2, 0.2, 0.55, 0.6, 0.6, 0.65, 0.7, \text{and } 0.75.$$
 There are two common ways to combine these results together into a single class prediction. One is the majority vote approach discussed in this chapter. The second approach is to classify based on the average probability. In this example, what is the final classification under each of these two approaches?

```python
res = np.array([0.1, 0.15, 0.2, 0.2, 0.55, 0.6, 0.6, 0.65, 0.7, 0.75])

# Majority vote approach
majority = (0.5 < res).sum()

# The average Probability
average = np.mean(res)

print(f'Is red (by Majority): {majority > .5}, majority = {majority}')
print(f'Is red (by Average) : {average  > .5}, avg = {average}')
```

    Is red (by Majority): True, majority = 6
    Is red (by Average) : False, avg = 0.45

### $6.$ Provide a detailed explanation of the algorithm that is used to fit a regression tree.
**Answer:**
> The Regression Tree algorithm is shown as Algorithm  $8.1$  in the book. The algorithm is all about building a single tree for predictors, therefore we tend to select the best individual tree by applying **cost complexity pruning** and **K-fold cross-validation** to choose the best $\alpha$.
 It requires $4$ steps to build a Regression Tree, as shown in Algorithm $8.1$:

> $\text{Step 1:}$ Use **recursive binary splitting** to grow a large tree on the training data, stopping only when each terminal node has fewer than some minimum number of observations.
>- We take a top-down, greedy approach that is known as **recursive binary splitting**. The approach is *top-down* because it begins at the top of the tree (at which point all observations belong to a single region) and then successively splits the predictor space; each split is indicated via two new branches further down on the tree. It is *greedy* because at each step of the tree-building process, the best split is made at that particular step, rather than looking ahead and picking a split that will lead to a better tree in some future step.
>- We first select the predictor $X_j$ and the cutpoint $s$ such that splitting the predictor space into the regions $\{X|X_j < s\}$ and $\{X|X_j \geq s\}$ leads to the greatest possible
reduction in $\text{RSS}$, and then choose the predictor and cutpoint such that the resulting tree has the lowest $\text{RSS}$.
>- Next, we repeat the process, looking for the best predictor and best cutpoint in order to split the data further so as to minimize the $\text{RSS}$ within each of the resulting regions.
>- However, this time, instead of splitting the entire predictor space, we split one of the two previously identified regions.
>- We now have three regions. Again, we look to split one of these three regions further, so as to minimize the $\text{RSS}$. The process continues until a stopping criterion is reached; for instance, we may continue until no region contains more than five observations "as in Figure $8.3$".
>- Once the regions $R_1, \dots , R_J$ have been created, we predict the response for a given test observation using the mean of the training observations in the region to which that test observation belongs.
>- And finally, this process may produce good predictions on the training set, but is likely to **overfit** the data, leading to poor test set performance.

> $\text{Step 2:}$ Apply **cost complexity pruning** to the large tree in order to obtain a sequence of best subtrees, as a function of $\alpha$.
>- One possible strategy is that we can build the tree only so long as the decrease in the $\text{RSS}$ due to each split exceeds some (high) threshold.
>- Therefore, a better strategy is to grow a very large tree $T_0$, and then $\text{prune}$ it back in order to obtain a $\text{subtree}$.
>- To select a subtree with the lowest test error rate, we can estimate its test error using cross-validation or the validation set approach. But estimating the CV error for every possible subtree would be computationally expensive "Cumbersome".
>- So, Cost $\text{complexity pruning}$ also known as $\text{weakest link pruning}$ gives us a way to do just this. Rather than considering every possible subtree, we consider a sequence of trees indexed by a nonnegative tuning parameter $\alpha$.
>- For each value of α there corresponds a subtree $T \subset T_0$ is as small as possible such that
$$
\sum_{m=1}^{|T|} \sum_{i: x_i \in R_m} (y_i - \hat{y}_{Rm})^2 + \alpha |T|  \tag{8.4}
$$
>- Here the tuning parameter $\alpha$ controls a trade-off between the subtree's complexity and its fit to the training data.
>  - **Quick reminder:** (Equation $(8.4)$ is a similar $\text{cost + penalty}$ format as regularization methods as the **lasso** $(6.7)$ from Chapter $6$, in which a similar formulation was used in order to control the complexity of a linear model.)
>- When $\alpha$ = 0, then the subtree $T$ will simply equal to $T_0$, because then $(8.4)$ just measures the training error "cost".
>- As we start to increase $\alpha$, branches get pruned from the tree in a nested and predictable fashion, so obtaining the whole sequence of subtrees as a function of $\alpha$ is easy.

> $\text{Step 3:}$ We can select a value of $\alpha$ using a **validation set** or using **cross-validation**. We then return to the full data set and obtain the subtree corresponding to $\alpha$ as following.
>- Use **K-fold cross-validation** to choose $\alpha$. That is, divide the training observations into $K$ folds. For each $k = 1, \dots ,K$:
>  - $(a)$ Repeat Steps $1$ and $2$ on all but the $k$th fold of the training data.
>  - $(b)$ Evaluate the mean squared prediction error on the data in the left-out $k$th fold, as a function of $\alpha$. Average the results for each value of $\alpha$, and pick $\alpha$ to minimize the average error.

> $\text{Step 4:}$ Return the subtree from Step $2$ that corresponds to the chosen value of $\alpha$, re-fit a regression tree on the full dataset to obtain the subtree, and use this final form for prediction.

# Done!