# [ATM 623: Climate Modeling](../index.ipynb)

[Brian E. J. Rose](http://www.atmos.albany.edu/facstaff/brose/index.html), University at Albany

# Lecture 6: Elementary greenhouse models

### About these notes:

This document uses the interactive [`Jupyter notebook`](https://jupyter.org) format. The notes can be accessed in several different ways:

- The interactive notebooks are hosted on `github` at https://github.com/brian-rose/ClimateModeling_courseware
- The latest versions can be viewed as static web pages [rendered on nbviewer](http://nbviewer.ipython.org/github/brian-rose/ClimateModeling_courseware/blob/master/index.ipynb)
- A complete snapshot of the notes as of May 2017 (end of spring semester) are [available on Brian's website](http://www.atmos.albany.edu/facstaff/brose/classes/ATM623_Spring2017/Notes/index.html).

[Also here is a legacy version from 2015](http://www.atmos.albany.edu/facstaff/brose/classes/ATM623_Spring2015/Notes/index.html).

Many of these notes make use of the `climlab` package, available at https://github.com/brian-rose/climlab

```python
#  Ensure compatibility with Python 2 and 3
from __future__ import print_function, division
```

## Contents

1. [A single layer atmosphere](#section1)
2. [Introducing the two-layer grey gas model](#section2)
3. [Tuning the grey gas model to observations](#section3)
4. [Level of emission](#section4)
5. [Radiative forcing in the 2-layer grey gas model](#section5)
6. [Radiative equilibrium in the 2-layer grey gas model](#section6)
7. [Summary](#section7)

____________

## 1. A single layer atmosphere
____________

We will make our first attempt at quantifying the greenhouse effect in the simplest possible greenhouse model: a single layer of atmosphere that is able to absorb and emit longwave radiation.

```python
from IPython.display import Image
Image('../images/1layerAtm_sketch.png')
```

### Assumptions

- Atmosphere is a single layer of air at temperature $T_a$
- Atmosphere is **completely transparent to shortwave** solar radiation.
- The **surface** absorbs shortwave radiation $(1-\alpha) Q$
- Atmosphere is **completely opaque to infrared** radiation
- Both surface and atmosphere emit radiation as **blackbodies** ($\sigma T_s^4, \sigma T_a^4$)
- Atmosphere radiates **equally up and down** ($\sigma T_a^4$)
- There are no other heat transfer mechanisms

We can now use the concept of energy balance to ask what the temperature need to be in order to balance the energy budgets at the surface and the atmosphere, i.e. the **radiative equilibrium temperatures**.

### Energy balance at the surface

\begin{align}
\text{energy in} &= \text{energy out} \\
(1-\alpha) Q + \sigma T_a^4 &= \sigma T_s^4 \\
\end{align}

The presence of the atmosphere above means there is an additional source term: downwelling infrared radiation from the atmosphere.

We call this the **back radiation**.

### Energy balance for the atmosphere

\begin{align}
\text{energy in} &= \text{energy out} \\
\sigma T_s^4 &= A\uparrow + A\downarrow = 2 \sigma T_a^4 \\
\end{align}

which means that
$$ T_s = 2^\frac{1}{4} T_a \approx 1.2 T_a $$

So we have just determined that, in order to have a purely **radiative equilibrium**, we must have $T_s > T_a$.

*The surface must be warmer than the atmosphere.*

### Solve for the radiative equilibrium surface temperature

Now plug this into the surface equation to find

$$ \frac{1}{2} \sigma T_s^4 = (1-\alpha) Q  $$

and use the definition of the emission temperature $T_e$ to write

$$ (1-\alpha) Q = \sigma T_e^4 $$

*In fact, in this model, $T_e$ is identical to the atmospheric temperature $T_a$, since all the OLR originates from this layer.*

Solve for the surface temperature:
$$ T_s = 2^\frac{1}{4} T_e $$

Putting in observed numbers, $T_e = 255$ K gives a surface temperature of
$$T_s = 303 ~\text{K}$$

This model is one small step closer to reality: surface is warmer than atmosphere, emissions to space generated in the atmosphere, atmosphere heated from below and helping to keep surface warm.

BUT our model now overpredicts the surface temperature by about 15ºC (or K).

Ideas about why?

Basically we just need to read our **list of assumptions** above and realize that none of them are very good approximations:

- Atmosphere absorbs some solar radiation.
- Atmosphere is NOT a perfect absorber of longwave radiation
- Absorption and emission varies strongly with wavelength *(atmosphere does not behave like a blackbody)*.
- Emissions are not determined by a single temperature $T_a$ but by the detailed *vertical profile* of air temperture.
- Energy is redistributed in the vertical by a variety of dynamical transport mechanisms (e.g. convection and boundary layer turbulence).

____________

## 2. Introducing the two-layer grey gas model
____________

Let's generalize the above model just a little bit to build a slighly more realistic model of longwave radiative transfer.

We will address two shortcomings of our single-layer model:
1. No vertical structure
2. 100% longwave opacity

Relaxing these two assumptions gives us what turns out to be a very useful prototype model for **understanding how the greenhouse effect works**.

### Assumptions

- The atmosphere is **transparent to shortwave radiation** (still)
- Divide the atmosphere up into **two layers of equal mass** (the dividing line is thus at 500 hPa pressure level)
- Each layer **absorbs only a fraction $\epsilon$ ** of whatever longwave radiation is incident upon it.
- We will call the fraction $\epsilon$ the **absorptivity** of the layer.
- Assume $\epsilon$ is the same in each layer

This is called the **grey gas** model, where grey here means the emission and absorption have no spectral dependence.

We can think of this model informally as a "leaky greenhouse".

Note that the assumption that $\epsilon$ is the same in each layer is appropriate if the absorption is actually carried out by a gas that is **well-mixed** in the atmosphere.

Out of our two most important absorbers:

- CO$_2$ is well mixed
- H$_2$O is not (mostly confined to lower troposphere due to strong temperature dependence of the saturation vapor pressure).

But we will ignore this aspect of reality for now.

In order to build our model, we need to introduce one additional piece of physics known as **Kirchoff's Law**:

$$ \text{absorptivity} = \text{emissivity} $$

So if a layer of atmosphere at temperature $T$ absorbs a fraction $\epsilon$ of incident longwave radiation, it must emit

$$ \epsilon ~\sigma ~T^4 $$

both up and down.

### A sketch of the radiative fluxes in the 2-layer atmosphere

```python
Image('../images/2layerAtm_sketch.png')
```

- Surface temperature is $T_s$
- Atm. temperatures are $T_0, T_1$ where $T_0$ is closest to the surface.
- absorptivity of atm layers is $\epsilon$
- Surface emission is $\sigma T_s^4$
- Atm emission is $\epsilon \sigma T_0^4, \epsilon \sigma T_1^4$ (up and down)
- Absorptivity = emissivity for atmospheric layers
- a fraction $(1-\epsilon)$ of the longwave beam is **transmitted** through each layer

### A fun aside: symbolic math with the `sympy` package

This two-layer grey gas model is simple enough that we can work out all the details algebraically. There are three temperatures to keep track of $(T_s, T_0, T_1)$, so we will have 3x3 matrix equations.

We all know how to work these things out with pencil and paper. But it can be tedious and error-prone.

Symbolic math software lets us use the computer to automate a lot of tedious algebra.

The [sympy](http://www.sympy.org/en/index.html) package is a powerful open-source symbolic math library that is well-integrated into the scientific Python ecosystem.

```python
import sympy
#  Allow sympy to produce nice looking equations as output
sympy.init_printing()
#  Define some symbols for mathematical quantities
#  Assume all quantities are positive (which will help simplify some expressions)
epsilon, T_e, T_s, T_0, T_1, sigma = \
    sympy.symbols('epsilon, T_e, T_s, T_0, T_1, sigma', positive=True)
#  So far we have just defined some symbols, e.g.
T_s
```

```python
#  We have hard-coded the assumption that the temperature is positive
sympy.ask(T_s>0)
```

    True

### Longwave emissions

Let's denote the emissions from each layer as
\begin{align}
E_s &= \sigma T_s^4 \\
E_0 &= \epsilon \sigma T_0^4 \\
E_1 &= \epsilon \sigma T_1^4
\end{align}

recognizing that $E_0$ and $E_1$ contribute to **both** the upwelling and downwelling beams.

```python
#  Define these operations as sympy symbols
#  And display as a column vector:
E_s = sigma*T_s**4
E_0 = epsilon*sigma*T_0**4
E_1 = epsilon*sigma*T_1**4
E = sympy.Matrix([E_s, E_0, E_1])
E
```

### Shortwave radiation
Since we have assumed the atmosphere is transparent to shortwave, the incident beam $Q$ passes unchanged from the top to the surface, where a fraction $\alpha$ is reflected upward out to space.

```python
#  Define some new symbols for shortwave radiation
Q, alpha = sympy.symbols('Q, alpha', positive=True)
#  Create a dictionary to hold our numerical values
tuned = {}
tuned[Q] = 341.3  #  global mean insolation in W/m2
tuned[alpha] = 101.9/Q.subs(tuned)  #  observed planetary albedo
tuned[sigma] = 5.67E-8  #  Stefan-Boltzmann constant in W/m2/K4
tuned
#  Numerical value for emission temperature
#T_e.subs(tuned)
```

### Upwelling beam

Let $U$ be the upwelling flux of longwave radiation.

The upward flux from the surface to layer 0 is
$$ U_0 = E_s $$
(just the emission from the suface).

```python
U_0 = E_s
U_0
```

Following this beam upward, we can write the upward flux from layer 0 to layer 1 as the sum of the transmitted component that originated below layer 0 and the new emissions from layer 0:

$$ U_1 = (1-\epsilon) U_0 + E_0 $$

```python
U_1 = (1-epsilon)*U_0 + E_0
U_1
```

Continuing to follow the same beam, the upwelling flux above layer 1 is
$$ U_2 = (1-\epsilon) U_1 + E_1 $$

```python
U_2 = (1-epsilon) * U_1 + E_1
```

Since there is no more atmosphere above layer 1, this upwelling flux is our Outgoing Longwave Radiation for this model:

$$ OLR = U_2 $$

```python
U_2
```

The three terms in the above expression represent the **contributions to the total OLR that originate from each of the three levels**.

Let's code this up explicitly for future reference:

```python
#  Define the contributions to OLR originating from each level
OLR_s = (1-epsilon)**2 *sigma*T_s**4
OLR_0 = epsilon*(1-epsilon)*sigma*T_0**4
OLR_1 = epsilon*sigma*T_1**4

OLR = OLR_s + OLR_0 + OLR_1

print( 'The expression for OLR is')
OLR
```

### Downwelling beam

Let $D$ be the downwelling longwave beam. Since there is no longwave radiation coming in from space, we begin with

```python
fromspace = 0
D_2 = fromspace
```

Between layer 1 and layer 0 the beam contains emissions from layer 1:

$$ D_1 = (1-\epsilon)D_2 + E_1 = E_1 $$

```python
D_1 = (1-epsilon)*D_2 + E_1
D_1
```

Finally between layer 0 and the surface the beam contains a transmitted component and the emissions from layer 0:

$$ D_0 = (1-\epsilon) D_1 + E_0 = \epsilon(1-\epsilon) \sigma T_1^4 + \epsilon \sigma T_0^4$$

```python
D_0 = (1-epsilon)*D_1 + E_0
D_0
```

This $D_0$ is what we call the **back radiation**, i.e. the longwave radiation from the atmosphere to the surface.

____________

## 3. Tuning the grey gas model to observations
____________

In building our new model we have introduced exactly one parameter, the absorptivity $\epsilon$. We need to choose a value for $\epsilon$.

We will tune our model so that it **reproduces the observed global mean OLR** given **observed global mean temperatures**.

To get appropriate temperatures for $T_s, T_0, T_1$, let's revisit the [global, annual mean lapse rate plot from NCEP Reanalysis data](Lecture05 -- Radiation.ipynb) from the previous lecture.

### Temperatures

First, we set
$$T_s = 288 \text{ K}  $$

From the lapse rate plot, an average temperature for the layer between 1000 and 500 hPa is

$$ T_0 = 275 \text{ K}$$

Defining an average temperature for the layer between 500 and 0 hPa is more ambiguous because of the lapse rate reversal at the tropopause. We will choose

$$ T_1 = 230 \text{ K}$$

From the graph, this is approximately the observed global mean temperature at 275 hPa or about 10 km.

```python
#  add to our dictionary of values:
tuned[T_s] = 288.
tuned[T_0] = 275.
tuned[T_1] = 230.
tuned
```

### OLR

From the [observed global energy budget](Lecture01 -- Planetary energy budget.ipynb) we set

$$ OLR = 238.5 \text{ W m}^{-2} $$

### Solving for $\epsilon$

We wrote down the expression for OLR as a function of temperatures and absorptivity in our model above.

We just need to equate this to the observed value and solve a **quadratic equation** for $\epsilon$.

This is where the real power of the symbolic math toolkit comes in.

Subsitute in the numerical values we are interested in:

```python
# the .subs() method for a sympy symbol means
#  substitute values in the expression using the supplied dictionary
#  Here we use observed values of Ts, T0, T1
OLR2 = OLR.subs(tuned)
OLR2
```

We have a quadratic equation for $\epsilon$.

Now use the `sympy.solve` function to solve the quadratic:

```python
#  The sympy.solve method takes an expression equal to zero
#  So in this case we subtract the tuned value of OLR from our expression
eps_solution = sympy.solve(OLR2 - 238.5, epsilon)
eps_solution
```

There are two roots, but the second one is unphysical since we must have $0 < \epsilon < 1$.

Just for fun, here is a simple of example of *filtering a list* using powerful Python *list comprehension* syntax:

```python
# Give me only the roots that are between zero and 1!
list_result = [eps for eps in eps_solution if 0<eps<1]
print( list_result)
# The result is a list with a single element.
#  We need to slice the list to get just the number:
eps_tuned = list_result[0]
print( eps_tuned)
```

    [0.586041150248834]
    0.586041150248834

We conclude that our tuned value is

$$ \epsilon = 0.586 $$

This is the absorptivity that guarantees that our model reproduces the observed OLR given the observed tempertures.

```python
tuned[epsilon] = eps_tuned
tuned
```

____________

## 4. Level of emission
____________

Even in this very simple greenhouse model, there is **no single level** at which the OLR is generated.

The three terms in our formula for OLR tell us the contributions from each level.

```python
OLRterms = sympy.Matrix([OLR_s, OLR_0, OLR_1])
OLRterms
```

Now evaluate these expressions for our tuned temperature and absorptivity:

```python
OLRtuned = OLRterms.subs(tuned)
OLRtuned
```

So we are getting about 67 W m$^{-2}$ from the surface, 79 W m$^{-2}$ from layer 0, and 93 W m$^{-2}$ from the top layer.

In terms of fractional contributions to the total OLR, we have (limiting the output to two decimal places):

```python
sympy.N(OLRtuned / 239., 2)
```

Notice that the largest single contribution is coming from the top layer. This is in spite of the fact that the emissions from this layer are weak, because it is so cold.

Comparing to observations, the actual contribution to OLR from the surface is about 22 W m$^{-2}$ (or about 9% of the total), not 67 W m$^{-2}$. So we certainly don't have all the details worked out yet!

As we will see later, to really understand what sets that observed 22 W m$^{-2}$, we will need to start thinking about the spectral dependence of the longwave absorptivity.

____________

## 5. Radiative forcing in the 2-layer grey gas model
____________

Adding some extra greenhouse absorbers will mean that a greater fraction of incident longwave radiation is absorbed in each layer.

Thus **$\epsilon$ must increase** as we add greenhouse gases.

Suppose we have $\epsilon$ initially, and the absorptivity increases to $\epsilon_2 = \epsilon + \delta_\epsilon$.

Suppose further that this increase happens **abruptly** so that there is no time for the temperatures to respond to this change. **We hold the temperatures fixed** in the column and ask how the radiative fluxes change.

**Do you expect the OLR to increase or decrease?**

Let's use our two-layer leaky greenhouse model to investigate the answer.

The components of the OLR before the perturbation are

```python
OLRterms
```

After the perturbation we have

```python
delta_epsilon = sympy.symbols('delta_epsilon')
OLRterms_pert = OLRterms.subs(epsilon, epsilon+delta_epsilon)
OLRterms_pert
```

Let's take the difference

```python
deltaOLR = OLRterms_pert - OLRterms
deltaOLR
```

To make things simpler, we will neglect the terms in $\delta_\epsilon^2$. This is perfectly reasonably because we are dealing with **small perturbations** where $\delta_\epsilon << \epsilon$.

Telling `sympy` to set the quadratic terms to zero gives us

```python
deltaOLR_linear = sympy.expand(deltaOLR).subs(delta_epsilon**2, 0)
deltaOLR_linear
```

Recall that the three terms are the contributions to the OLR from the three different levels. In this case, the **changes** in those contributions after adding more absorbers.

Now let's divide through by $\delta_\epsilon$ to get the normalized change in OLR per unit change in absorptivity:

```python
deltaOLR_per_deltaepsilon = \
    sympy.simplify(deltaOLR_linear / delta_epsilon)
deltaOLR_per_deltaepsilon
```

Now look at the **sign** of each term. Recall that $0 < \epsilon < 1$. **Which terms in the OLR go up and which go down?**

**THIS IS VERY IMPORTANT, SO STOP AND THINK ABOUT IT.**

The contribution from the **surface** must **decrease**, while the contribution from the **top layer** must **increase**.

**When we add absorbers, the average level of emission goes up!**

### "Radiative forcing" is the change in radiative flux at TOA after adding absorbers

In this model, only the longwave flux can change, so we define the radiative forcing as

$$ R = - \delta OLR $$

(with the minus sign so that $R$ is positive when the climate system is gaining extra energy).

We just worked out that whenever we add some extra absorbers, the emissions to space (on average) will originate from higher levels in the atmosphere.

What does this mean for OLR? Will it increase or decrease?

To get the answer, we just have to sum up the three contributions we wrote above:

```python
R = -sum(deltaOLR_per_deltaepsilon)
R
```

Is this a positive or negative number? The key point is this:

**It depends on the temperatures, i.e. on the lapse rate.**

### Greenhouse effect for an isothermal atmosphere

Stop and think about this question:

If the **surface and atmosphere are all at the same temperature**, does the OLR go up or down when $\epsilon$ increases (i.e. we add more absorbers)?

Understanding this question is key to understanding how the greenhouse effect works.

#### Let's solve the isothermal case

We will just set $T_s = T_0 = T_1$ in the above expression for the radiative forcing.

```python
R.subs([(T_0, T_s), (T_1, T_s)])
```

which then simplifies to

```python
sympy.simplify(R.subs([(T_0, T_s), (T_1, T_s)]))
```

#### The answer is zero

For an isothermal atmosphere, there is **no change** in OLR when we add extra greenhouse absorbers. Hence, no radiative forcing and no greenhouse effect.

Why?

The level of emission still must go up. But since the temperature at the upper level is the **same** as everywhere else, the emissions are exactly the same.

### The radiative forcing (change in OLR) depends on the lapse rate!

For a more realistic example of radiative forcing due to an increase in greenhouse absorbers, we can substitute in our tuned values for temperature and $\epsilon$.

We'll express the answer in W m$^{-2}$ for a 1% increase in $\epsilon$.

The three components of the OLR change are

```python
deltaOLR_per_deltaepsilon.subs(tuned) * 0.01
```

And the net radiative forcing is

```python
R.subs(tuned) * 0.01
```

So in our example, **the OLR decreases by 2.2 W m$^{-2}$**, or equivalently, the radiative forcing is +2.2 W m$^{-2}$.

What we have just calculated is this:

*Given the observed lapse rates, a small increase in absorbers will cause a small decrease in OLR.*

The greenhouse effect thus gets stronger, and energy will begin to accumulate in the system -- which will eventually cause temperatures to increase as the system adjusts to a new equilibrium.

____________

## 6. Radiative equilibrium in the 2-layer grey gas model
____________

In the previous section we:

- made no assumptions about the processes that actually set the temperatures.
- used the model to calculate radiative fluxes, **given observed temperatures**.
- stressed the importance of knowing the lapse rates in order to know how an increase in emission level would affect the OLR, and thus determine the radiative forcing.

A key question in climate dynamics is therefore this:

**What sets the lapse rate?**

It turns out that lots of different physical processes contribute to setting the lapse rate.

Understanding how these processes acts together and how they change as the climate changes is one of the key reasons for which we need more complex climate models.

For now, we will use our prototype greenhouse model to do the most basic lapse rate calculation: the **radiative equilibrium temperature**.

We assume that

- the only exchange of energy between layers is longwave radiation
- equilibrium is achieved when the **net radiative flux convergence** in each layer is zero.

### Compute the radiative flux convergence

First, the **net upwelling flux** is just the difference between flux up and flux down:

```python
#  Upwelling and downwelling beams as matrices
U = sympy.Matrix([U_0, U_1, U_2])
D = sympy.Matrix([D_0, D_1, D_2])
# Net flux, positive up
F = U-D
F
```

####  Net absorption is the flux convergence in each layer

(difference between what's coming in the bottom and what's going out the top of each layer)

```python
# define a vector of absorbed radiation -- same size as emissions
A = E.copy()

#  absorbed radiation at surface
A[0] = F[0]
# Compute the convergence
for n in range(2):
    A[n+1] = -(F[n+1]-F[n])

A
```

#### Radiative equilibrium means net absorption is ZERO in the atmosphere

The only other heat source is the **shortwave heating** at the **surface**.

In matrix form, here is the system of equations to be solved:

```python
radeq = sympy.Equality(A, sympy.Matrix([(1-alpha)*Q, 0, 0]))
radeq
```

Just as we did for the 1-layer model, it is helpful to rewrite this system using the definition of the **emission temperture** $T_e$

$$ (1-\alpha) Q = \sigma T_e^4 $$

```python
radeq2 = radeq.subs([((1-alpha)*Q, sigma*T_e**4)])
radeq2
```

In this form we can see that we actually have a **linear system** of equations for a set of variables $T_s^4, T_0^4, T_1^4$.

We can solve this matrix problem to get these as functions of $T_e^4$.

```python
# Solve for radiative equilibrium
fourthpower = sympy.solve(radeq2, [T_s**4, T_1**4, T_0**4])
fourthpower
```

This produces a dictionary of solutions for the fourth power of the temperatures!

A little manipulation gets us the solutions for temperatures that we want:

```python
# need the symbolic fourth root operation
from sympy.simplify.simplify import nthroot

fourthpower_list = [fourthpower[key] for key in [T_s**4, T_0**4, T_1**4]]
solution = sympy.Matrix([nthroot(item,4) for item in fourthpower_list])
#  Display result as matrix equation!
T = sympy.Matrix([T_s, T_0, T_1])
sympy.Equality(T, solution)
```

In more familiar notation, the radiative equilibrium solution is thus

\begin{align}
T_s &= T_e \left( \frac{2+\epsilon}{2-\epsilon} \right)^{1/4} \\
T_0 &= T_e \left( \frac{1+\epsilon}{2-\epsilon} \right)^{1/4} \\
T_1 &= T_e \left( \frac{ 1}{2 - \epsilon} \right)^{1/4}
\end{align}

Plugging in the tuned value $\epsilon = 0.586$ gives

```python
Tsolution = solution.subs(tuned)
#  Display result as matrix equation!
sympy.Equality(T, Tsolution)
```

Now we just need to know the Earth's emission temperature $T_e$!

(Which we already know is about 255 K)

```python
# Here's how to calculate T_e from the observed values
sympy.solve(((1-alpha)*Q - sigma*T_e**4).subs(tuned), T_e)
```

```python
# Need to unpack the list
Te_value = sympy.solve(((1-alpha)*Q - sigma*T_e**4).subs(tuned), T_e)[0]
Te_value
```

#### Now we finally get our solution for radiative equilibrium

```python
#  Output 4 significant digits
Trad = sympy.N(Tsolution.subs([(T_e, Te_value)]), 4)
sympy.Equality(T, Trad)
```

Compare these to the values we derived from the **observed lapse rates**:

```python
sympy.Equality(T, T.subs(tuned))
```

The **radiative equilibrium** solution is substantially **warmer at the surface** and **colder in the lower troposphere** than reality.

This is a very general feature of radiative equilibrium, and we will see it again very soon in this course.

____________

## 7. Summary
____________

## Key physical lessons

- Putting a **layer of longwave absorbers** above the surface keeps the **surface substantially warmer**, because of the **backradiation** from the atmosphere (greenhouse effect).
- The **grey gas** model assumes that each layer absorbs and emits a fraction $\epsilon$ of its blackbody value, independent of wavelength.

- With **incomplete absorption** ($\epsilon < 1$), there are contributions to the OLR from every level and the surface (there is no single **level of emission**)
- Adding more absorbers means that **contributions to the OLR** from **upper levels** go **up**, while contributions from the surface go **down**.
- This upward shift in the weighting of different levels is what we mean when we say the **level of emission goes up**.

- The **radiative forcing** caused by an increase in absorbers **depends on the lapse rate**.
- For an **isothermal atmosphere** the radiative forcing is zero and there is **no greenhouse effect**
- The radiative forcing is positive for our atmosphere **because tropospheric temperatures tends to decrease with height**.
- Pure **radiative equilibrium** produces a **warm surface** and **cold lower troposphere**.
- This is unrealistic, and suggests that crucial heat transfer mechanisms are missing from our model.

### And on the Python side...

Did we need `sympy` to work all this out? No, of course not. We could have solved the 3x3 matrix problems by hand. But computer algebra can be very useful and save you a lot of time and error, so it's good to invest some effort into learning how to use it.

Hopefully these notes provide a useful starting point.

### A follow-up assignment

You are now ready to tackle [Assignment 5](../Assignments/Assignment05 -- Radiative forcing in a grey radiation atmosphere.ipynb), where you are asked to extend this grey-gas analysis to many layers.

For more than a few layers, the analytical approach we used here is no longer very useful. You will code up a numerical solution to calculate OLR given temperatures and absorptivity, and look at how the lapse rate determines radiative forcing for a given increase in absorptivity.

[Back to ATM 623 notebook home](../index.ipynb)

____________
## Version information
____________

```python
%load_ext version_information
%version_information sympy
```

    Loading extensions from ~/.ipython/extensions is deprecated. We recommend managing extensions like any other Python packages, in site-packages.

Software	Version

Python	3.6.2 64bit [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]

IPython	6.1.0

OS	Darwin 16.7.0 x86_64 i386 64bit

sympy	1.1.1

Wed Oct 11 13:31:13 2017 EDT

____________

## Credits

The author of this notebook is [Brian E. J. Rose](http://www.atmos.albany.edu/facstaff/brose/index.html), University at Albany.

It was developed in support of [ATM 623: Climate Modeling](http://www.atmos.albany.edu/facstaff/brose/classes/ATM623_Spring2015/), a graduate-level course in the [Department of Atmospheric and Envionmental Sciences](http://www.albany.edu/atmos/index.php)

Development of these notes and the [climlab software](https://github.com/brian-rose/climlab) is partially supported by the National Science Foundation under award AGS-1455071 to Brian Rose. Any opinions, findings, conclusions or recommendations expressed here are mine and do not necessarily reflect the views of the National Science Foundation.
____________

```python

```

Lectures/Lecture06 -- Elementary greenhouse models.ipynb


# Jupyter Notebook Tutorial
> A tutorial of core functionality of Jupyter Notebooks to have an enjoyable coding experience.

- toc: true
- badges: true
- comments: true
- author: Isaac Flath
- categories: [Jupyter,Getting Started]

# Top 3 uses:
1. Exploratory analysis, model creation, data science, any kind of coding that require lots of rapid experimentation and iteration.
1. Tutorials, guides, and blogs (like this one).  Because you have a great mix of text functionality with code, they work really well for tutorials and guides.  Rather than having static images, or code snippets that have to get updated each iteration, the code is part of the guide and it really simplifies the process.  Notebooks can be exported directly to html and be opened in any browser to give to people.  With the easy conversion to html, naturally it's easy to post them on a web page.
1.  Technical presentations of results.  You can have the actual code analysis done, with text explanations.  Excess code can be collapsed so that if someone asks really detailed questions you can expand and have every piece of detail.  Changes to the analysis are in the presentation so no need to save and put static images in other documents

# Cell Types

A cell can be 3 different types.  The most useful are code cells and markdown cells.

### Code Cells
    - Code cells run code  The next few cells are examples of code cells
    - While the most common application is Python, you can set up environments easily to use R, swift, and other languages within jupyter notebooks
### Markdown Cells
    - This cell is a markdown cell.  It is really nice for adding details and text explanations in where a code comment is not enough
    - They have all the normal markdown functionality, plus more.  For example, I can write any technical or mathy stuff using latex, or create html tables in markdown or html.
    - I can also make markdown tables.

##### Latex Formulas

$$\begin{bmatrix}w_1&w_2&w_3&w_4&w_5\\x_1&x_2&x_3&x_4&x_5\\y_1&y_2&y_3&y_4&y_5\\z_1&z_2&z_3&z_4&z_5\end{bmatrix}$$

$\begin{align}
\frac{dy}{du} &= f'(u) = e^u = e^{\sin(x^2)}, \\
\frac{du}{dv} &= g'(v) = \cos v = \cos(x^2), \\
\frac{dv}{dx} &= h'(x) = 2x.
\end{align}$

##### Markdown Table

| This | is   | a     | table | for   | demos |
|------|------|-------|-------|-------|-------|
| perc | 55%  | 22%   | 23%   | 12%   | 53%   |
| qty  | 23   | 19    | 150   | 9     | 92    |

```python
#collapse-hide
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

pd.options.display.max_columns = None
pd.options.display.max_rows = None

%matplotlib inline
```

# Running Code

Naturally you can run code cells and print to the Jupyter Notebook

```python
for x in range(0,5):
    print(x*10)
```

    0
    10
    20
    30
    40

# DataFrames

```python
iris = sns.load_dataset('iris')
iris[iris.petal_length > 6]
```

      sepal_length
      sepal_width
      petal_length
      petal_width
      species

      105
      7.6
      3.0
      6.6
      2.1
      virginica

      107
      7.3
      2.9
      6.3
      1.8
      virginica

      109
      7.2
      3.6
      6.1
      2.5
      virginica

      117
      7.7
      3.8
      6.7
      2.2
      virginica

      118
      7.7
      2.6
      6.9
      2.3
      virginica

      122
      7.7
      2.8
      6.7
      2.0
      virginica

      130
      7.4
      2.8
      6.1
      1.9
      virginica

      131
      7.9
      3.8
      6.4
      2.0
      virginica

      135
      7.7
      3.0
      6.1
      2.3
      virginica

# Plotting
Below we are going to make a few graphs to get the point accross.  Naturally, each graph can be accompanied with a markdown cell that gives context and explains the value of that graph.

### Line Chart

```python
# evenly sampled time at 200ms intervals
t = np.arange(0., 5., 0.2)

# red dashes, blue squares and green triangles
plt.plot(t, t, 'r--', t, t**2, 'bs', t, t**3, 'g^')
plt.show()
```

### Scatter Plot

Sometimes we will want to display a graph, but may not want all the code and details to be immediately visable.  In these examples we can create a scatter plot like below, but collapse the code cell.

This is great when you want to show a graph and explain it, but the details of how the graph was created aren't that important.

```python
#collapse-hide
data = {'a': np.arange(50),
        'c': np.random.randint(0, 50, 50),
        'd': np.random.randn(50)}
data['b'] = data['a'] + 10 * np.random.randn(50)
data['d'] = np.abs(data['d']) * 100

plt.scatter('a', 'b', c='c', s='d', data=data)
plt.xlabel('entry a')
plt.ylabel('entry b')
plt.show()
```

### Categorical Plot

We can create subplots to have multiple plots show up.  This can be especially helpful when showing lots of the same information, or showing how 2 different metrics are related or need to be analyzed together

```python
#collapse-hide
names = ['group_a', 'group_b', 'group_c']
values = [1, 10, 100]

plt.figure(figsize=(9, 3))

plt.subplot(131)
plt.bar(names, values)
plt.subplot(132)
plt.scatter(names, values)
plt.subplot(133)
plt.plot(names, values)
plt.suptitle('Categorical Plotting')
plt.show()
```

# Stack Traces

When you run into an error, by default jupyter notebooks give you whatever the error message is, but also the entire stack trace.

There is a debug functionality, but I find that these stack traces and jupyter cells work even better than a debugger.  I can break my code into as many cells as I want and run things interactively.  Here's a few examples of stack traces.

### Matrix Multiplication Good

Now we are going to show an example of errors where the stack trace isn't as simple.  Suppose we are trying to multiply 2 arrays together (matrix multiplication).

```python
a = np.array([
                [1,2,4],
                [3,4,5],
                [5,6,7]
            ])
b = np.array([
                [11,12,14],
                [31,14,15],
                [23,32,23]
            ])
a@b
```

    array([[165, 168, 136],
           [272, 252, 217],
           [402, 368, 321]])

### Matrix Multiplication Bad

Now if it errors because the columns from matrix a don't match the rows from matrix b, we will get an error as matrix multiplication is impossible with those matrices.  We see the same idea s the above for loop, stack trace with error and arrow pointing at the line that failed

```python
# here's another
a = np.array([
                [1,2,4],
                [3,4,5],
                [5,6,7]
            ])
b = np.array([
                [11,12,14],
                [31,14,15]
            ])
a@b
```

### Second Layer of Bad

But what if the line we call isn't what fails?  What if what I run works, but the function underneath fails?

In these example, you see the entire trace. It starts with are arrow at what you ran that errored.  It then shows an arrow that your code called that caused the error, so you can track all the way back to the source.  Here's how it shows a two step stack trace, but it can be as long as needed.

```python
def matmul(a,b):
    c = a@b
    return c
```

```python
matmul(a,b)
```

# Magic Commands

Magic commands are special commands for Juptyer Notebooks.  They give you incredible functionality and you will likley find the experience very frustrating without them.  A few that I use often are:
+ ? | put a question mark or 2 after a function or method to get the documentation.  ?? gives more detail than ?.  I can also use it to wild card search modules for functions.
+ shift tab | when you are writing something holding shift + tab will open a mini popup with the documentation for that thing.  It may be a funciton, method, or module.
+ ```%who``` or ```%whos``` or ```%who_ls``` | These are all variants that list the objects and variables.  I prefer %whos most of the time
+ ```%history``` | This allows you to look at the last pieces of code that you ran
+ $$ | wrapping latex code in dollar signs in a markdown cell renders latex code in markdown cells
+ ! | putting ! at the beginning of a line makes it run that in terminal.  For example ```!ls | grep .csv```
+ ```%time``` | I can use this to time the execution of things

```python
np.*array*?
```

    np.array
    np.array2string
    np.array_equal
    np.array_equiv
    np.array_repr
    np.array_split
    np.array_str
    np.asanyarray
    np.asarray
    np.asarray_chkfinite
    np.ascontiguousarray
    np.asfarray
    np.asfortranarray
    np.broadcast_arrays
    np.chararray
    np.compare_chararrays
    np.get_array_wrap
    np.ndarray
    np.numarray
    np.recarray

```python
np.array_equal??
```

    Signature: np.array_equal(a1, a2)
    Source:
    @array_function_dispatch(_array_equal_dispatcher)
    def array_equal(a1, a2):
        """
        True if two arrays have the same shape and elements, False otherwise.

        Parameters
        ----------
        a1, a2 : array_like
            Input arrays.

        Returns
        -------
        b : bool
            Returns True if the arrays are equal.

        See Also
        --------
        allclose: Returns True if two arrays are element-wise equal within a
                  tolerance.
        array_equiv: Returns True if input arrays are shape consistent and all
                     elements equal.

        Examples
        --------
        >>> np.array_equal([1, 2], [1, 2])
        True
        >>> np.array_equal(np.array([1, 2]), np.array([1, 2]))
        True
        >>> np.array_equal([1, 2], [1, 2, 3])
        False
        >>> np.array_equal([1, 2], [1, 4])
        False

        """
        try:
            a1, a2 = asarray(a1), asarray(a2)
        except Exception:
            return False
        if a1.shape != a2.shape:
            return False
        return bool(asarray(a1 == a2).all())
    File:      ~/opt/anaconda3/lib/python3.7/site-packages/numpy/core/numeric.py
    Type:      function

```python
a = np.array(np.random.rand(512,512))
b = np.array(np.random.rand(512,512))
%time for i in range(0,20): a@b
```

    CPU times: user 198 ms, sys: 2.68 ms, total: 200 ms
    Wall time: 34.8 ms

```python
%whos
```

    Variable   Type         Data/Info
    ---------------------------------
    a          ndarray      512x512: 262144 elems, type `float64`, 2097152 bytes (2.0 Mb)
    b          ndarray      512x512: 262144 elems, type `float64`, 2097152 bytes (2.0 Mb)
    data       dict         n=4
    i          int          19
    iris       DataFrame         sepal_length  sepal_<...>          1.8   virginica
    matmul     function     <function matmul at 0x1a169cc680>
    names      list         n=3
    np         module       <module 'numpy' from '/Us<...>kages/numpy/__init__.py'>
    pd         module       <module 'pandas' from '/U<...>ages/pandas/__init__.py'>
    plt        module       <module 'matplotlib.pyplo<...>es/matplotlib/pyplot.py'>
    sns        module       <module 'seaborn' from '/<...>ges/seaborn/__init__.py'>
    t          ndarray      25: 25 elems, type `float64`, 200 bytes
    values     list         n=3
    x          int          4

```python
%history -l 5
```

    matmul(a,b)
    np.*array*?
    np.array_equal??
    a = np.array(np.random.rand(512,512))
    b = np.array(np.random.rand(512,512))
    %time for i in range(0,20): a@b
    %whos

# Jupyter Extensions

There are many extensions to Jupyter Notebooks.  After all a jupyter notebook is just a JSON file, so you can read the JSON in and manipulate and transform things however you want!  There are many features, such as variable explorers, auto code timers, and more - but I find I that most are unneccesary.  About half the people I talk to don't use any, and the other half use several.

# NBDEV

NBdev is a jupyter extension/python library that allows you to do full development projects in Jupyter Notebooks.  There have been books and libraries written entirely in Jupyter notebooks, including testing frameworks and unit testing that goes with them.  A common misconception is that Jupyter notebooks cannot be used for that, though many people already have.

There are many features NBdev adds.  Here's a few.

Using notebooks written like this, nbdev can create and run any of the following with a single command:

+ Searchable, hyperlinked documentation; any word you surround in backticks will by automatically hyperlinked to the appropriate documentation
+ Cells in jupyter notebook marked with #export will be exported automatically to a python module
+ Python modules, following best practices such as automatically defining __ all __ (more details) with your exported functions, classes, and variables
+ Pip installers (uploaded to pypi for you)
+ Tests (defined directly in your notebooks, and run in parallel).
+ Navigate and edit your code in a standard text editor or IDE, and export any changes automatically back into your notebooks

I reccomend checking them out for more detail https://github.com/fastai/nbdev


# 一、马尔可夫性质

  马尔科夫性质——当前的状态只和上一时刻有关，在上一时刻之前的任何状态都和我无关。我们称其符合马尔可夫性质。

  具有马尔科夫性质的状态满足下面公式：

  $P(S_{t+1}|S_1,S_2,...,S_t) = P(S_{t+1}|S_t)$

  根据公式也就是说给定当前状态$S_t$，将来的状态与t时刻之前的状态已经没有关系。

# 二、马尔可夫链

  马尔可夫链是指具有马尔可夫性质的随机过程。在过程中，在给定当前信息的情况下，过去的信息状态对于预测将来状态是无关的。——可以称之为马尔科夫无后效性。

# 三、隐马尔可夫模型

  隐马尔可夫模型(Hidden Markov Model, HMM)是一种统计模型，在语音识别、行为识别、NLP、故障诊断等领域具有高效的性能。

  HMM是关于时序的概率模型，描述一个含有未知参数的马尔可夫链所生成的不可观测的状态随机序列，再由各个状态生成观测随机序列的过程。

  HMM是一个双重随机过程---具有一定状态的隐马尔可夫链和随机的观测序列。

  HMM随机生成的状态随机序列被称为状态序列；每个状态生成一个观测，由此产生的观测随机序列，被称为观测序列。

  举个例子来理解下hmm，我们使用输入法打字，敲出的每个字符就是观测序列，而实际我们想写的话就是隐藏序列。那不同公司的输入法都是在努力提高猜测咱们要写的话的能力。

## 1、定义

  对于HMM模型，首先我们假设Q是所有可能的隐藏状态的集合，V是所有可能的观测状态的集合，即：

$Q={q_1,q_2,...,q_N},V={v_1,v_2,...v_M}$

  其中，N是可能的隐藏状态数，M是所有的可能的观察状态数。

  对于一个长度为T的序列，I对应的状态序列, O是对应的观察序列，即：

$I={i_1,i_2,...,i_T},O={o_1,o_2,...o_T}$

  其中，任意一个隐藏状态$i_t∈Q$,任意一个观察状态$o_t∈V$

  HMM模型做了两个很重要的假设如下：

1. 齐次马尔科夫链假设。即任意时刻的隐藏状态只依赖于它前一个隐藏状态，这个我们在MCMC(二)马尔科夫链中有详细讲述。当然这样假设有点极端，因为很多时候我们的某一个隐藏状态不仅仅只依赖于前一个隐藏状态，可能是前两个或者是前三个。但是这样假设的好处就是模型简单，便于求解。如果在时刻t的隐藏状态是$i_t=q_i$,在时刻t+1的隐藏状态是$i_{t+1}=q_j$, 则从时刻t到时刻t+1的HMM状态转移概率$a_{ij}$可以表示为：

$a_{ij}=P(i_{t+1}=q_j|i_t=q_i)$

  这样$a_{ij}$可以组成马尔科夫链的状态转移矩阵A:

$A=[a_{ij}]_{N×N}$

2. 观测独立性假设。即任意时刻的观察状态只仅仅依赖于当前时刻的隐藏状态，这也是一个为了简化模型的假设。如果在时刻t的隐藏状态是$i_t=q_j$, 而对应的观察状态为$o_t=v_k$, 则该时刻观察状态$v_k$在隐藏状态$q_j$下生成的概率为$b_j(k)$,满足：

$b_j(k)=P(o_t=v_k|i_t=q_j)$

  这样$b_j(k)$可以组成观测状态生成的概率矩阵B:

$B=[b_j(k)]_{N×M}$

  除此之外，我们需要一组在时刻t=1的隐藏状态概率分布Π:

$Π=[π(i)]_N$  其中$π(i)=P(i_1=q_i)$

  一个HMM模型，可以由隐藏状态初始概率分布Π, 状态转移概率矩阵A和观测状态概率矩阵B决定。Π,A决定状态序列，B决定观测序列。因此，HMM模型可以由一个三元组λ表示如下：

$λ=(A,B,Π)$

## 2、hmm能解决的问题

1. 评估观察序列概率。即给定模型$λ=(A,B,Π)$和观测序列$O={o_1,o_2,...o_T}$，计算在模型$λ$下观测序列O出现的概率$P(O|λ)$。这个问题的求解需要用到前向后向算法。

2. 模型参数学习问题。即给定观测序列$O={o_1,o_2,...o_T}$，估计模型$λ=(A,B,Π)$的参数，使该模型下观测序列的条件概率$P(O|λ)$最大。这个问题的求解需要用到基于EM算法的鲍姆-韦尔奇算法。

3. 预测问题，也称为解码问题。即给定模型$λ=(A,B,Π)$和观测序列$O={o_1,o_2,...o_T}$，求给定观测序列条件下，最可能出现的对应的状态序列，这个问题的求解需要用到基于动态规划的维特比算法。【预测出的每一个隐状态都有N个值，每个值对应一个概率值；例如句子分词，那么就需要利用维特比算法找到整个句子的最大隐状态概率对应的隐状态路径。】

# 四、评估观察序列概率

## 1、求观测序列的概率

这个问题是这样的。我们已知HMM模型的参数𝜆=(𝐴,𝐵,Π)。其中𝐴是隐藏状态转移概率的矩阵，𝐵是观测状态生成概率的矩阵， Π是隐藏状态的初始概率分布。同时我们也已经得到了观测序列$O =\{o_1,o_2,...o_T\}$,现在我们要求观测序列𝑂在模型𝜆下出现的条件概率𝑃(𝑂|𝜆)。

乍一看，这个问题很简单。因为我们知道所有的隐藏状态之间的转移概率和所有从隐藏状态到观测状态生成概率，那么我们是可以暴力求解的。

我们可以列举出所有可能出现的长度为𝑇的隐藏序列$I = \{i_1,i_2,...,i_T\}$,分布求出这些隐藏序列与观测序列$O =\{o_1,o_2,...o_T\}$的联合概率分布𝑃(𝑂,𝐼|𝜆)，这样我们就可以很容易的求出边缘分布𝑃(𝑂|𝜆)了。

具体暴力求解的方法是这样的：首先，任意一个隐藏序列$I = \{i_1,i_2,...,i_T\}$出现的概率是：

$$P(I|\lambda) = \pi_{i_1} a_{i_{1}i_{2}} a_{i_{2}i_{3}}... a_{i_{T-1}\;\;i_T}$$

对于固定的状态序列$I = \{i_1,i_2,...,i_T\}$，我们要求的观察序列$O =\{o_1,o_2,...o_T\}$出现的概率是：

$$P(O|I, \lambda) = b_{i_1}(o_1)b_{i_2}(o_2)...b_{i_T}(o_T)$$

则𝑂和𝐼联合出现的概率是：

$$P(O,I|\lambda) = P(I|\lambda)P(O|I, \lambda) = \pi_{i_1}b_{i_1}(o_1)a_{i_1i_2}b_{i_2}(o_2)...a_{i_{T-1}\;\;i_T}b_{i_T}(o_T)$$

然后求边缘概率分布，即可得到观测序列𝑂在模型𝜆下出现的条件概率𝑃(𝑂|𝜆)，但是我们需要计算所有的T个隐状态、N种隐状态的自由组合，即$T^N$种组合的概率然后累加：

>边缘分布（Marginal Distribution）指在概率论和统计学的多维随机变量中，只包含其中部分变量的概率分布。\
>\
>**定义** \
>假设有一个和两个变量相关的概率分布：\
>$P(x,y)$\
>关于其中一个特定变量的边缘分布则为给定其他变量的条件概率分布：\
>$P(x)=\sum_yP(x,y)=\sum_yP(x|y)P(y)$ \
>解释：让y取所有的可能值，然后求出只有x变量的概率；举个例子：x为肤色（黑、白、黄）y为性别（男、女）那么$P(x)=P(x|男)+P(x|女)$ \
>参阅Wikipedia举例，下图中，X和Y遵从绿圈内所示的二元正态分布，红线和蓝线分别表示Y变量和X变量的边缘分布 \
>
>在这个边缘分布中，我们得到只关于一个变量的概率分布，而不再考虑另一变量的影响，实际上进行了降维操作。在实际应用中，例如人工神经网络的神经元互相关联，在计算它们各自的参数的时候，就会使用边缘分布计算得到某一特定神经元（变量）的值。

$$P(O|\lambda) = \sum\limits_{I}P(O,I|\lambda)  = \sum\limits_{i_1,i_2,...i_T}\pi_{i_1}b_{i_1}(o_1)a_{i_1i_2}b_{i_2}(o_2)...a_{i_{T-1}\;\;i_T}b_{i_T}(o_T)$$

虽然上述方法有效，但是如果我们的隐藏状态数𝑁非常多的那就麻烦了，此时我们预测状态有$N^T$种组合，算法的时间复杂度是$𝑂(𝑇𝑁^𝑇)$阶的。因此对于一些隐藏状态数极少的模型，我们可以用暴力求解法来得到观测序列出现的概率，但是如果隐藏状态多，则上述算法太耗时，我们需要寻找其他简洁的算法。

前向后向算法就是来帮助我们在较低的时间复杂度情况下求解这个问题

# 2、前向算法求HMM观测序列的概率

前向后向算法是前向算法和后向算法的统称，这两个算法都可以用来求HMM观测序列的概率。我们先来看看前向算法是如何求解这个问题的。

前向算法本质上属于动态规划的算法，也就是我们要通过找到局部状态递推的公式，这样一步步的从子问题的最优解拓展到整个问题的最优解。

在前向算法中，通过定义“前向概率”来定义动态规划的这个局部状态。什么是前向概率呢, 其实定义很简单：定义时刻𝑡时隐藏状态为$q_i$, 观测状态的序列为$𝑜_1,𝑜_2,...𝑜_𝑡$的概率为前向概率。记为：

$$\alpha_t(i) = P(o_1,o_2,...o_t, i_t =q_i | \lambda)\ \ \ \ 注意：字母是alpha不是a$$

既然是动态规划，我们就要递推了，现在我们假设我们已经找到了在时刻𝑡时各个隐藏状态的前向概率，现在我们需要递推出时刻𝑡+1时各个隐藏状态的前向概率。

从上图可以看出，我们可以基于时刻𝑡时各个隐藏状态的前向概率，再乘以对应的状态转移概率，即$\alpha_t(j)a_{ji}$就是在时刻𝑡观测到$o_1,o_2,...o_t$，并且时刻𝑡隐藏状态$𝑞_𝑗$, 时刻𝑡+1隐藏状态$𝑞_i$的概率。表示为：

$$P(o_1,o_2,...o_t,i_t=q_j,i_{t+1}=q_i|\lambda)=\alpha_t(j)a_{ji}$$

如果将想上面所有的线对应的概率求和，即$\sum\limits_{j=1}^N\alpha_t(j)a_{ji}$就是在时刻𝑡观测到$o_1,o_2,...o_t$，并且时刻𝑡+1隐藏状态$𝑞_i$的概率。表示为：

$$P(o_1,o_2,...o_t,i_{t+1}=q_i|\lambda)=\sum\limits_{j=1}^N\alpha_t(j)a_{ji}$$

继续一步，由于观测状态$O_{𝑡+1}$只依赖于𝑡+1时刻隐藏状态$𝑞_𝑖$, 这样$[\sum\limits_{j=1}^N\alpha_t(j)a_{ji}]b_i(o_{t+1})$就是在在时刻𝑡+1观测到$o_1,o_2,...o_t，o_{t+1}$，并且时刻𝑡+1隐藏状态𝑞𝑖的概率。表示为：

$$P(o_1,o_2,...o_t,o_{t+1},i_{t+1}=q_i|\lambda)=[\sum\limits_{j=1}^N\alpha_t(j)a_{ji}]b_i(o_{t+1})$$

而这个概率，恰恰就是时刻𝑡+1对应的隐藏状态𝑖的前向概率，这样我们得到了前向概率的递推关系式如下：

$$\alpha_{t+1}(i) = \Big[\sum\limits_{j=1}^N\alpha_t(j)a_{ji}\Big]b_i(o_{t+1})$$

我们的动态规划从时刻1开始，到时刻𝑇结束，由于$\alpha_T(i)$表示在时刻𝑇观测序列为$o_1,o_2,...o_T$，并且时刻𝑇隐藏状态$𝑞_𝑖$的概率，我们只要将所有隐藏状态对应的概率相加，即$\sum\limits_{i=1}^N\alpha_T(i)$就得到了在时刻𝑇观测序列为$o_1,o_2,...o_T$的概率。

下面总结下前向算法。

输入：HMM模型𝜆=(𝐴,𝐵,Π)，观测序列𝑂=($o_1,o_2,...o_T$)

输出：观测序列概率𝑃(𝑂|𝜆)

1) 计算时刻1的各个隐藏状态前向概率：

$$\alpha_1(i) = \pi_ib_i(o_1),\; i=1,2,...N$$

2) 递推时刻2,3,...𝑇时刻的前向概率（递推关系从时刻1往时刻T推，由t时刻的前向概率可以求出t+1时刻的前向概率）：

$$\alpha_{t+1}(i) = \Big[\sum\limits_{j=1}^N\alpha_t(j)a_{ji}\Big]b_i(o_{t+1}),\; i=1,2,...N$$

3) 计算最终结果：

$$P(O|\lambda) = \sum\limits_{i=1}^N\alpha_T(i)$$

从递推公式可以看出，我们的算法时间复杂度是$𝑂(𝑇𝑁^2)$，比暴力解法的时间复杂度$𝑂(𝑇𝑁^𝑇)$少了几个数量级。

## 3、HMM前向算法求解实例

我们的观察集合是:

$$𝑉={红，白}，𝑀=2$$

我们的状态集合是：

$$𝑄=\{盒子1，盒子2，盒子3\}，𝑁=3$$

而观察序列和状态序列的长度为3.

初始状态分布为：

$$\Pi = (0.2,0.4,0.4)^T$$

状态转移概率分布矩阵为：

$$A = \left( \begin{array} {ccc} 0.5 & 0.2 & 0.3 \\ 0.3 & 0.5 & 0.2 \\ 0.2 & 0.3 &0.5 \end{array} \right)$$

观测状态概率矩阵为：

$$B = \left( \begin{array} {ccc} 0.5 & 0.5 \\ 0.4 & 0.6 \\ 0.7 & 0.3 \end{array} \right)$$

球的颜色的观测序列:

$$O=\{红，白，红\}$$

首先计算时刻1三个状态的前向概率：

时刻1是红色球，隐藏状态是盒子1的概率为：

$$\alpha_1(1) = \pi_1b_1(o_1) = 0.2 \times 0.5 = 0.1$$

隐藏状态是盒子2的概率为：

$$\alpha_1(2) = \pi_2b_2(o_1) = 0.4 \times 0.4 = 0.16$$

隐藏状态是盒子3的概率为：

$$\alpha_1(3) = \pi_3b_3(o_1) = 0.4 \times 0.7 = 0.28$$

现在我们可以开始递推了，首先递推时刻2三个状态的前向概率：
时刻2是白色球，隐藏状态是盒子1的概率为：

$$\alpha_2(1) =  \Big[\sum\limits_{i=1}^3\alpha_1(i)a_{i1}\Big]b_1(o_2) = [0.1*0.5+0.16*0.3+0.28*0.2 ] \times 0.5 = 0.077$$

隐藏状态是盒子2的概率为：

$$\alpha_2(2) =  \Big[\sum\limits_{i=1}^3\alpha_1(i)a_{i2}\Big]b_2(o_2) = [0.1*0.2+0.16*0.5+0.28*0.3 ] \times 0.6 = 0.1104$$

隐藏状态是盒子3的概率为：

$$\alpha_2(3) =  \Big[\sum\limits_{i=1}^3\alpha_1(i)a_{i3}\Big]b_3(o_2) = [0.1*0.3+0.16*0.2+0.28*0.5 ] \times 0.3 = 0.0606$$

继续递推，现在我们递推时刻3三个状态的前向概率：
时刻3是红色球，隐藏状态是盒子1的概率为：

$$\alpha_3(1) =  \Big[\sum\limits_{i=1}^3\alpha_2(i)a_{i1}\Big]b_1(o_3) = [0.077*0.5+0.1104*0.3+0.0606*0.2 ] \times 0.5 = 0.04187$$

隐藏状态是盒子2的概率为：

$$\alpha_3(2) =  \Big[\sum\limits_{i=1}^3\alpha_2(i)a_{i2}\Big]b_2(o_3) = [0.077*0.2+0.1104*0.5+0.0606*0.3 ] \times 0.4 = 0.03551$$

隐藏状态是盒子3的概率为：

$$\alpha_3(3) =  \Big[\sum\limits_{i=1}^3\alpha_2(i)a_{i3}\Big]b_3(o_3) = [0.077*0.3+0.1104*0.2+0.0606*0.5 ] \times 0.7 = 0.05284$$

最终我们求出观测序列:$𝑂=\{红，白，红\}$的概率为：

$$P(O|\lambda) = \sum\limits_{i=1}^3\alpha_3(i) = 0.13022$$

## 4、后向算法求HMM观测序列的概率

后向算法和前向算法非常类似，都是用的动态规划，唯一的区别是选择的局部状态不同，后向算法用的是“后向概率”，那么后向概率是如何定义的呢？

定义时刻𝑡时隐藏状态为$𝑞_𝑖$, 从时刻𝑡+1到最后时刻𝑇的观测状态的序列为$𝑜_{𝑡+1},𝑜_{𝑡+2},...𝑜_𝑇$的概率为后向概率。记为：

$$\beta_t(i) = P(o_{t+1},o_{t+2},...o_T| i_t =q_i , \lambda)$$

后向概率的动态规划递推公式和前向概率是相反的。现在我们假设我们已经找到了在时刻𝑡+1时各个隐藏状态的后向概率$𝛽_{𝑡+1}(𝑗)$，表示为：

$$\beta_{t+1}(j) = P(o_{t+2},o_{t+3}...o_T| i_{t+1} =q_j , \lambda)$$

现在我们需要递推出时刻𝑡时各个隐藏状态的后向概率。如上图，我们可以计算出观测状态的序列为$o_{𝑡+2},o_{𝑡+3},...o_𝑇$， 𝑡时隐藏状态为$𝑞_𝑖$, 时刻𝑡+1隐藏状态为$𝑞_𝑗$的概率为$𝑎_{𝑖𝑗}𝛽_{𝑡+1}(𝑗)$, 表示为：

$$P(o_{𝑡+2},o_{𝑡+3},...o_𝑇|i_t=q_i,i_{t+1}=q_j,\lambda)=𝑎_{𝑖𝑗}𝛽_{𝑡+1}(𝑗)$$

接着可以得到观测状态的序列为$o_{𝑡+1},o_{𝑡+2},...o_𝑇$， 𝑡时隐藏状态为$𝑞_𝑖$, 时刻𝑡+1隐藏状态为$𝑞_𝑗$的概率为$a_{ij}b_j(o_{t+1})\beta_{t+1}(j)$, 表示为：

$$P(o_{𝑡+1},o_{𝑡+2},...o_𝑇|i_t=q_i,i_{t+1}=q_j,\lambda)=a_{ij}b_j(o_{t+1})\beta_{t+1}(j)$$

则把上面所有线对应的概率加起来，我们可以得到观测状态的序列为$o_{𝑡+1},o_{𝑡+2},...o_𝑇$， 𝑡时隐藏状态为$𝑞_𝑖$的概率为$\sum\limits_{j=1}^{N}a_{ij}b_j(o_{t+1})\beta_{t+1}(j)$，这个概率即为时刻𝑡的后向概率。表示为：

$$P(o_{𝑡+1},o_{𝑡+2},...o_𝑇|i_t=q_i,\lambda)=\sum\limits_{j=1}^{N}a_{ij}b_j(o_{t+1})\beta_{t+1}(j)$$

这样我们得到了后向概率的递推关系式如下：
$$\beta_{t}(i) = \sum\limits_{j=1}^{N}a_{ij}b_j(o_{t+1})\beta_{t+1}(j)$$
现在我们总结下后向算法的流程,注意下和前向算法的相同点和不同点：

输入：HMM模型𝜆=(𝐴,𝐵,Π)，观测序列$O=(o_1,o_2,...o_T)$

输出：观测序列概率𝑃(𝑂|𝜆)

1) 初始化时刻𝑇的各个隐藏状态后向概率：

$$\beta_T(i) = 1,\; i=1,2,...N$$

2) 递推时刻𝑇−1,𝑇−2,...1时刻的后向概率（递推关系从T时刻往T-1,T-2...1时刻推，利用t+1时刻的后向概率可以求t时刻的后向概率）：

$$\beta_{t}(i) = \sum\limits_{j=1}^{N}a_{ij}b_j(o_{t+1})\beta_{t+1}(j),\; i=1,2,...N$$

3) 计算最终结果：

$$P(O|\lambda) = \sum\limits_{i=1}^N\pi_ib_i(o_1)\beta_1(i)$$

此时我们的算法时间复杂度仍然是$𝑂(𝑇𝑁^2)$。

## 5、HMM常用概率的计算

利用前向概率和后向概率，我们可以计算出HMM中单个状态和两个状态的概率公式。

1.给定模型𝜆和观测序列𝑂,在时刻𝑡处于状态$𝑞_𝑖$的概率记为:

$$\gamma_t(i) = P(i_t = q_i | O,\lambda) = \frac{P(i_t = q_i ,O|\lambda)}{P(O|\lambda)}$$

那么$P(i_t = q_i ,O|\lambda)$怎么求呢？

我们先分析下这个概率求的是啥，

它的意思是:在给定HMM的参数λ，观测序列是$O(O_1,O_2,...O_T)$,在t时刻的状态为$q_i$的概率。

换句话理解就是:从1到t时刻，t时刻的状态为$q_i$,输出是$O(O_1,O_2,...O_T)$的概率P1；然后t时刻之后到T，在t时刻状态为$q_i$的基础上，输出序列是$o_{t+1},o_{t+2}..o_T$的概率P2，P1和P2相乘即为所求。

碰巧P1符合前向概率定义，P2符合后向概率定义，所以 $P1=\alpha_t(i) \ \ \  P2=\beta_t(i)$，则:

$$P(i_t = q_i ,O|\lambda) = P1*P2 = \alpha_t(i)\beta_t(i)$$

那么：

$$P(O|\lambda) = \sum_j^N \alpha_t(j)\beta_t(j) \ \ \ 注意：为啥P(O|\lambda)用这个表示，因为这样方便计算$$

所以：

$$\gamma_t(i) = \frac{ \alpha_t(i)\beta_t(i)}{\sum\limits_{j=1}^N \alpha_t(j)\beta_t(j)}$$

2.给定模型𝜆和观测序列𝑂,在时刻𝑡处于状态$𝑞_𝑖$，且时刻𝑡+1处于状态$𝑞_𝑗$的概率记为:

$$\xi_t(i,j) = P(i_t = q_i, i_{t+1}=q_j | O,\lambda) = \frac{ P(i_t = q_i, i_{t+1}=q_j , O|\lambda)}{P(O|\lambda)}$$

那么$P(i_t = q_i, i_{t+1}=q_j , O|\lambda)$怎么求呢？

我们先分析下这个概率求的是啥，

它的意思是：在给定HMM的参数下，观测序列是O，t时刻的状态是$q_i$，t+1时刻的状态是$q_j$的概率。

这个概率=$P(O_1,O_2,...O_t,i_t=q_i|λ)*P(q_i转移到t+1时刻的q_j)*P(o_{t+2},o_{t+3}..o_T|i_{t+1}=q_j, λ)*P(t+1时刻输出是O_{t+1})$

逐个解释下：

>$P(O_1,O_2,...O_t,i_t=q_i|λ)$:这不就是t时刻的前向概率$\alpha_t(i)$么\
>$P(q_i转移到t+1时刻的q_j)$:这不就是转移概率$a_{ij}$么\
>$P(o_{t+2},o_{t+3}..o_T|i_{t+1}=q_j, λ)$:这不就是t+1时刻的后向概率$\beta_{t+1}(i)$么\
>(注意:t+1时刻的后续输出序列是$o_{t+2},o_{t+3}..o_T$,即下标是从t+2开始，不是t+1开始，请仔细理解后向概率的定义)。\
>前面少了$O_{t+1}$这个时间点的观测序列，因此要补上$P(t+1时刻输出是O_{t+1})$,而这个不就是$b_{j}(O_{t+1})$么

所以：

$$P(i_t = q_i, i_{t+1}=q_j , O|\lambda) = \alpha_t(i)a_{ij}b_j(o_{t+1})\beta_{t+1}(j)$$

所以：

$$P(O|\lambda)=\sum\limits_{r=1}^N\sum\limits_{s=1}^N\alpha_t(r)a_{rs}b_s(o_{t+1})\beta_{t+1}(s)$$

所以：

$$\xi_t(i,j) = \frac{\alpha_t(i)a_{ij}b_j(o_{t+1})\beta_{t+1}(j)}{\sum\limits_{r=1}^N\sum\limits_{s=1}^N\alpha_t(r)a_{rs}b_s(o_{t+1})\beta_{t+1}(s)}$$

3.将$𝛾_𝑡(𝑖)$和$𝜉_𝑡(𝑖,𝑗)$在各个时刻𝑡求和，可以得到：

在观测序列𝑂下状态𝑖出现的期望值$\sum_{𝑡=1}^𝑇𝛾_𝑡(𝑖)$

在观测序列𝑂下由状态𝑖转移的期望值$\sum_{𝑡=1}^{𝑇-1}𝛾_𝑡(𝑖)$

在观测序列𝑂下由状态𝑖转移到状态𝑗的期望值$\sum_{𝑡=1}^{𝑇-1}𝜉_𝑡(𝑖,𝑗)$

# 五、模型参数学习问题

## 1、HMM模型参数求解概述

HMM模型参数求解根据已知的条件可以分为两种情况。

第一种情况较为简单，就是我们已知𝐷个长度为𝑇的观测序列和对应的隐藏状态序列，即${(𝑂_1,𝐼_1),(𝑂_2,𝐼_2),...(𝑂_𝐷,𝐼_𝐷)}$是已知的，此时我们可以很容易的用最大似然来求解模型参数。

假设样本从隐藏状态$𝑞_𝑖$转移到$𝑞_𝑗$的频率计数是$𝐴_{𝑖𝑗}$,那么状态转移矩阵求得为：

$$A = \Big[a_{ij}\Big], \;其中a_{ij} = \frac{A_{ij}}{\sum\limits_{s=1}^{N}A_{is}}$$

假设样本隐藏状态为$𝑞_𝑗$且观测状态为$𝑣_𝑘$的频率计数是$𝐵_{𝑗𝑘}$,那么观测状态概率矩阵为：

$$B= \Big[b_{j}(k)\Big], \;其中b_{j}(k) = \frac{B_{jk}}{\sum\limits_{s=1}^{M}B_{js}}$$

假设所有样本中初始隐藏状态为$𝑞_𝑖$的频率计数为$𝐶(𝑖)$,那么初始概率分布为：

$$\Pi = \pi(i) = \frac{C(i)}{\sum\limits_{s=1}^{N}C(s)}$$

可见第一种情况下求解模型还是很简单的。但是在很多时候，我们无法得到HMM样本观察序列对应的隐藏序列，只有𝐷个长度为𝑇的观测序列，即${(𝑂_1),(𝑂_2),...(𝑂_𝐷)}$是已知的，此时我们能不能求出合适的HMM模型参数呢？这就是我们的第二种情况，也是我们本文要讨论的重点。它的解法最常用的是鲍姆-韦尔奇算法，其实就是基于EM算法的求解，只不过鲍姆-韦尔奇算法出现的时代，EM算法还没有被抽象出来，所以我们本文还是说鲍姆-韦尔奇算法。

## 2、鲍姆-韦尔奇（Baum-Welch）算法原理

　鲍姆-韦尔奇算法原理既然使用的就是EM算法【（Expectation-Maximum）算法也称期望最大化算法】的原理，那么我们需要在E步求出联合分布𝑃(𝑂,𝐼|𝜆)基于条件概率$P(I|O,\overline{\lambda})$的期望，其中$\overline{\lambda}$为当前的模型参数，然后再M步最大化这个期望，得到更新的模型参数𝜆。接着不停的进行EM迭代，直到模型参数的值收敛为止。

首先来看看E步，当前模型参数为$\overline{\lambda}$, 联合分布𝑃(𝑂,𝐼|𝜆)基于条件概率$P(I|O,\overline{\lambda})$的期望表达式为：

$$L(\lambda, \overline{\lambda}) = \sum\limits_{I}P(I|O,\overline{\lambda})logP(O,I|\lambda)$$

在M步，我们极大化上式，然后得到更新后的模型参数如下：

$$\overline{\lambda} = arg\;\max_{\lambda}\sum\limits_{I}P(I|O,\overline{\lambda})logP(O,I|\lambda)$$

通过不断的E步和M步的迭代，直到$\overline{\lambda}$收敛。下面我们来看看鲍姆-韦尔奇算法的推导过程。

## 3、鲍姆-韦尔奇算法的推导

我们的训练数据为$\{(O_1, I_1), (O_2, I_2), ...(O_D, I_D)\}$，其中任意一个观测序列$O_d = \{o_1^{(d)}, o_2^{(d)}, ... o_T^{(d)}\}$,其对应的未知的隐藏状态序列表示为：$I_d = \{i_1^{(d)}, i_2^{(d)}, ... i_T^{(d)}\}$

首先看鲍姆-韦尔奇算法的E步，我们需要先计算联合分布$𝑃(𝑂,𝐼|\lambda)$的表达式如下：

$$P(O,I|\lambda) = \prod_{d=1}^D \pi_{i_1^{(d)}}\ b_{i_1^{(d)}}\ (o_1^{(d)})\ a_{i_1^{(d)}\ i_2^{(d)}}\ b_{i_2^{(d)}}\ (o_2^{(d)}) ... a_{i_{T-1}^{(d)} \ \ i_T^{(d)}}\ b_{i_T^{(d)}}\ \ (o_T^{(d)})$$

我们的E步得到的期望表达式为：

$$L(\lambda, \overline{\lambda}) = \sum\limits_{I}P(I|O,\overline{\lambda})logP(O,I|\lambda)$$

在M步我们要极大化上式。由于$P(I|O,\overline{\lambda}) = \frac{P(I,O|\overline{\lambda})}{P(O|\overline{\lambda})}$,而$𝑃(𝑂|\overline{\lambda})$是常数，因此我们要极大化的式子等价于：

$$\overline{\lambda} = arg\;\max_{\lambda}\sum\limits_{I}P(O,I|\overline{\lambda})logP(O,I|\lambda)$$

我们将上面$𝑃(𝑂,𝐼|𝜆)$的表达式带入我们的极大化式子，得到的表达式如下：

$$\overline{\lambda} = arg\;\max_{\lambda}\sum\limits_{d=1}^D\sum\limits_{I}P(O,I|\overline{\lambda})(log\pi_{i_1} + \sum\limits_{t=1}^{T-1}log\;a_{i_t,i_{t+1}} +  \sum\limits_{t=1}^Tlog b_{i_t}(o_t))$$

我们的隐藏模型参数$𝜆=(𝐴,𝐵,Π)$,因此下面我们只需要对上式分别对$𝐴,𝐵,Π$求导即可得到我们更新的模型参数$\overline{\lambda}$

首先我们看看对模型参数Π的求导。由于Π只在上式中括号里的第一部分出现，因此我们对于Π的极大化式子为：

$$\overline{\pi_i} = arg\;\max_{\pi_{i_1}} \sum\limits_{d=1}^D\sum\limits_{I}P(O,I|\overline{\lambda})log\pi_{i_1} = arg\;\max_{\pi_{i}} \sum\limits_{d=1}^D\sum\limits_{i=1}^NP(O,i_1^{(d)} =i|\overline{\lambda})log\pi_{i}$$

由于$𝜋_𝑖$还满足$\sum\limits_{i=1}^N\pi_i =1$，因此根据拉格朗日子乘法，我们得到𝜋𝑖要极大化的拉格朗日函数为：

$$arg\;\max_{\pi_{i}}\sum\limits_{d=1}^D\sum\limits_{i=1}^NP(O,i_1^{(d)} =i|\overline{\lambda})log\pi_{i} + \gamma(\sum\limits_{i=1}^N\pi_i -1)$$

其中，𝛾为拉格朗日系数。上式对$𝜋_𝑖$求偏导数并令结果为0， 我们得到：

$$\sum\limits_{d=1}^DP(O,i_1^{(d)} =i|\overline{\lambda}) + \gamma\pi_i = 0$$

令𝑖分别等于从1到𝑁，从上式可以得到𝑁个式子，对这𝑁个式子求和可得：

$$\sum\limits_{d=1}^DP(O|\overline{\lambda}) + \gamma = 0$$

从上两式消去𝛾,得到$𝜋_𝑖$的表达式为：

$$\pi_i =\frac{\sum\limits_{d=1}^DP(O,i_1^{(d)} =i|\overline{\lambda})}{\sum\limits_{d=1}^DP(O|\overline{\lambda})} = \frac{\sum\limits_{d=1}^DP(O,i_1^{(d)} =i|\overline{\lambda})}{DP(O|\overline{\lambda})} = \frac{\sum\limits_{d=1}^DP(i_1^{(d)} =i|O, \overline{\lambda})}{D} =  \frac{\sum\limits_{d=1}^DP(i_1^{(d)} =i|O^{(d)}, \overline{\lambda})}{D}$$

利用前向概率的定义可得：

$$P(i_1^{(d)} =i|O^{(d)}, \overline{\lambda}) = \gamma_1^{(d)}(i)$$

因此最终我们在M步$𝜋_𝑖$的迭代公式为：

$$\pi_i =  \frac{\sum\limits_{d=1}^D\gamma_1^{(d)}(i)}{D}$$

现在我们来看看𝐴的迭代公式求法。方法和Π的类似。由于𝐴只在最大化函数式中括号里的第二部分出现，而这部分式子可以整理为：

$$\sum\limits_{d=1}^D\sum\limits_{I}\sum\limits_{t=1}^{T-1}P(O,I|\overline{\lambda})log\;a_{i_t,i_{t+1}} = \sum\limits_{d=1}^D\sum\limits_{i=1}^N\sum\limits_{j=1}^N\sum\limits_{t=1}^{T-1}P(O,i_t^{(d)} = i, i_{t+1}^{(d)} = j|\overline{\lambda})log\;a_{ij}$$

由于$𝑎_{𝑖𝑗}$还满足$\sum\limits_{j=1}^Na_{ij} =1$。和求解𝜋𝑖类似，我们可以用拉格朗日子乘法并对$𝑎_{𝑖𝑗}$求导，并令结果为0，可以得到$𝑎_{𝑖𝑗}$的迭代表达式为：

$$a_{ij} = \frac{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T-1}P(O^{(d)}, i_t^{(d)} = i, i_{t+1}^{(d)} = j|\overline{\lambda})}{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T-1}P(O^{(d)}, i_t^{(d)} = i|\overline{\lambda})}$$

利用前向概率的定义和$𝜉𝑡(𝑖,𝑗)$的定义可得到在M步$𝑎_{𝑖𝑗}$的迭代公式为：

$$a_{ij} = \frac{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T-1}\xi_t^{(d)}(i,j)}{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T-1}\gamma_t^{(d)}(i)}$$

现在我们来看看𝐵的迭代公式求法。方法和Π的类似。由于𝐵只在最大化函数式中括号里的第三部分出现，而这部分式子可以整理为：

$$\sum\limits_{d=1}^D\sum\limits_{I}\sum\limits_{t=1}^{T}P(O,I|\overline{\lambda})log\;b_{i_t}(o_t) = \sum\limits_{d=1}^D\sum\limits_{j=1}^N\sum\limits_{t=1}^{T}P(O,i_t^{(d)} = j|\overline{\lambda})log\;b_{j}(o_t)$$

由于$𝑏_𝑗(𝑜_𝑡)$还满足$\sum_{𝑘=1}^𝑀𝑏_𝑗(𝑜_𝑡=𝑣_𝑘)=1$。和求解$𝜋_𝑖$类似，我们可以用拉格朗日子乘法并对$𝑏_𝑗(𝑘)$求导，并令结果为0，得到$𝑏_𝑗(𝑘)$的迭代表达式为：

$$b_{j}(k) = \frac{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T}P(O,i_t^{(d)} = j|\overline{\lambda})I(o_t^{(d)}=v_k)}{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T}P(O,i_t^{(d)} = j|\overline{\lambda})}$$

其中$𝐼(𝑜_t^{(𝑑)}=𝑣_𝑘)$当且仅当$𝑜_t^{(𝑑)}=𝑣_𝑘$时为1，否则为0. 利用前向概率的定义可得$𝑏_𝑗(𝑜_𝑡)$的最终表达式为：

$$b_{j}(k) = \frac{\sum\limits_{d=1}^D\sum\limits_{t=1, o_t^{(d)}=v_k}^{T}\gamma_t^{(d)}(j)}{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T}\gamma_t^{(d)}(j)}$$

有了$𝜋_𝑖,𝑎_{𝑖𝑗},𝑏_𝑗(𝑘)$的迭代公式，我们就可以迭代求解HMM模型参数了。

## 4、鲍姆-韦尔奇算法流程总结

概括总结下鲍姆-韦尔奇算法的流程。

**输入：** 𝐷个观测序列样本${(𝑂_1),(𝑂_2),...(𝑂_𝐷)}$

**输出：** HMM模型参数

1)随机初始化所有的$𝜋_𝑖,𝑎_{𝑖𝑗},𝑏_𝑗(𝑘)$

2)对于每个样本𝑑=1,2,...𝐷，用前向后向算法计算$\gamma_t^{(d)}(i)，\xi_t^{(d)}(i,j), t =1,2...T$

3)更新模型参数：

$$\pi_i =  \frac{\sum\limits_{d=1}^D\gamma_1^{(d)}(i)}{D}$$

$$a_{ij} = \frac{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T-1}\xi_t^{(d)}(i,j)}{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T-1}\gamma_t^{(d)}(i)}$$

$$b_{j}(k) = \frac{\sum\limits_{d=1}^D\sum\limits_{t=1, o_t^{(d)}=v_k}^{T}\gamma_t^{(d)}(j)}{\sum\limits_{d=1}^D\sum\limits_{t=1}^{T}\gamma_t^{(d)}(j)}$$

4)如果$𝜋_𝑖,𝑎_{𝑖𝑗,}𝑏_𝑗(𝑘)$的值已经收敛，则算法结束，否则回到第2）步继续迭代。

# 六、预测问题

## 1、HMM最可能隐藏状态序列求解概述

在HMM模型的解码问题中，给定模型$𝜆=(𝐴,𝐵,Π)$和观测序列$𝑂={𝑜_1,𝑜_2,...𝑜_𝑇}$，求给定观测序列O条件下，最可能出现的对应的状态序列$I^*= \{i_1^*,i_2^*,...i_T^*\}$,即$𝑃(𝐼^∗|𝑂)$要最大化。

一个可能的近似解法是求出观测序列𝑂在每个时刻𝑡最可能的隐藏状态$𝑖_𝑡^*$然后得到一个近似的隐藏状态序列$I^*= \{i_1^*,i_2^*,...i_T^*\}$。要这样近似求解不难，利用前向后向算法评估观察序列概率的定义：在给定模型𝜆和观测序列𝑂时，在时刻𝑡处于状态$𝑞_𝑖$的概率是$𝛾_𝑡(𝑖)$，这个概率可以通过HMM的前向算法与后向算法计算。这样我们有：

$$i_t^* = arg \max_{1 \leq i \leq N}[\gamma_t(i)], \; t =1,2,...T$$

近似算法很简单，但是却不能保证预测的状态序列是整体是最可能的状态序列，因为预测的状态序列中某些相邻的隐藏状态可能存在转移概率为0的情况【也就是这样求解没有考虑到转移概率】。

而维特比算法可以将HMM的状态序列作为一个整体来考虑，避免近似算法的问题【考虑转移概率】，下面我们来看看维特比算法进行HMM解码的方法。

## 2、维特比算法概述

维特比算法是一个通用的解码算法，是基于动态规划的求序列最短路径的方法。

既然是动态规划算法，那么就需要找到合适的局部状态，以及局部状态的递推公式。在HMM中，维特比算法定义了两个局部状态用于递推。

第一个局部状态是在时刻𝑡隐藏状态为𝑖所有可能的状态转移路径$i_1,i_2,...i_t$中的概率最大值。记为$𝛿_𝑡(𝑖)$:

$$\delta_t(i) = \max_{i_1,i_2,...i_{t-1}}\;P(i_t=i, i_1,i_2,...i_{t-1},o_t,o_{t-1},...o_1|\lambda),\; i =1,2,...N$$

由$𝛿_𝑡(𝑖)$的定义可以得到𝛿的递推表达式：

$$
\begin{align}
\delta_{t+1}(i) & =  \max_{i_1,i_2,...i_{t}}\;P(i_{t+1}=i, i_1,i_2,...i_{t},o_{t+1},o_{t},...o_1|\lambda) \tag{1}\\
& = \max_{1 \leq j \leq N}\;[\delta_t(j)a_{ji}]b_i(o_{t+1}) \tag{2}
\end{align}
$$

第二个局部状态由第一个局部状态递推得到。我们定义在时刻𝑡隐藏状态为𝑖的所有单个状态转移路径$(i_1,i_2,...,i_{t-1},i)$中概率最大的转移路径中第𝑡−1个节点的隐藏状态为$Ψ_𝑡(𝑖)$,其递推表达式可以表示为：

$$\Psi_t(i) = arg \; \max_{1 \leq j \leq N}\;[\delta_{t-1}(j)a_{ji}]$$

有了这两个局部状态，我们就可以从时刻0一直递推到时刻𝑇，然后利用Ψ𝑡(𝑖)记录的前一个最可能的状态节点回溯，直到找到最优的隐藏状态序列。

## 3、维特比算法流程总结

总结下维特比算法的流程：

**输入：** HMM模型𝜆=(𝐴,𝐵,Π)，观测序列$O=(o_1,o_2,...o_T)$

**输出：** 最有可能的隐藏状态序列$I^*= \{i_1^*,i_2^*,...i_T^*\}$

1）初始化局部状态【时刻1的各个隐藏状态前向概率和上一个时刻的所有最可能的隐状态】：

$$\delta_1(i) = \pi_ib_i(o_1),\;i=1,2...N$$

$$\Psi_1(i)=0,\;i=1,2...N$$

2)进行动态规划递推时刻𝑡=2,3,...𝑇时刻的局部状态：

$$\delta_{t}(i) = \max_{1 \leq j \leq N}\;[\delta_{t-1}(j)a_{ji}]b_i(0_{t}),\;i=1,2...N$$

$$\Psi_t(i) = arg \; \max_{1 \leq j \leq N}\;[\delta_{t-1}(j)a_{ji}],\;i=1,2...N$$

3)计算时刻𝑇最大的$𝛿_𝑇(𝑖)$,即为最可能隐藏状态序列出现的概率。计算时刻𝑇最大的$Ψ_𝑡(𝑖)$,即为时刻𝑇最可能的隐藏状态。

$$P* = \max_{1 \leq j \leq N}\delta_{T}(i)$$

$$i_T^* = arg \; \max_{1 \leq j \leq N}\;[\delta_{T}(i)]$$

4)利用局部状态Ψ(𝑖)开始回溯。对于𝑡=𝑇−1,𝑇−2,...,1：

$$i_t^* = \Psi_{t+1}(i_{t+1}^*)$$

最终得到最有可能的隐藏状态序列$I^*= \{i_1^*,i_2^*,...i_T^*\}$

## 4、HMM维特比算法求解实例

我们的观察集合是:

$$𝑉={红，白}，𝑀=2$$

我们的状态集合是：

$$𝑄=\{盒子1，盒子2，盒子3\}，𝑁=3$$

而观察序列和状态序列的长度为3.

初始状态分布为：

$$\Pi = (0.2,0.4,0.4)^T$$

状态转移概率分布矩阵为：

$$A = \left( \begin{array} {ccc} 0.5 & 0.2 & 0.3 \\ 0.3 & 0.5 & 0.2 \\ 0.2 & 0.3 &0.5 \end{array} \right)$$

观测状态概率矩阵为：

$$B = \left( \begin{array} {ccc} 0.5 & 0.5 \\ 0.4 & 0.6 \\ 0.7 & 0.3 \end{array} \right)$$

球的颜色的观测序列:

$$O=\{红，白，红\}$$

首先计算时刻1三个状态的前向概率：

首先需要得到三个隐藏状态在时刻1时对应的各自两个局部状态，此时观测状态为1：

$$𝛿_1(1)=𝜋_1𝑏_1(𝑜_1)=0.2×0.5=0.1$$
$$𝛿_1(2)=𝜋_2𝑏_2(𝑜_1)=0.4×0.4=0.16$$
$$𝛿_1(3)=𝜋_3𝑏_3(𝑜_1)=0.4×0.7=0.28$$
$$Ψ_1(1)=Ψ_1(2)=Ψ_1(3)=0$$

现在开始递推三个隐藏状态在时刻2时对应的各自两个局部状态，此时观测状态为2：

$$\delta_2(1) = \max_{1\leq j \leq 3}[\delta_1(j)a_{j1}]b_1(o_2) = \max_{1\leq j \leq 3}[0.1 \times 0.5, 0.16 \times 0.3, 0.28\times 0.2] \times 0.5 = 0.028$$

$$\Psi_2(1)=3$$

$$\delta_2(2) = \max_{1\leq j \leq 3}[\delta_1(j)a_{j2}]b_2(o_2) = \max_{1\leq j \leq 3}[0.1 \times 0.2, 0.16 \times 0.5, 0.28\times 0.3] \times 0.6 = 0.0504$$

$$\Psi_2(2)=2$$

$$\delta_2(3) = \max_{1\leq j \leq 3}[\delta_1(j)a_{j3}]b_3(o_2) = \max_{1\leq j \leq 3}[0.1 \times 0.3, 0.16 \times 0.2, 0.28\times 0.5] \times 0.3 = 0.042$$

$$\Psi_2(3)=3$$

继续递推三个隐藏状态在时刻3时对应的各自两个局部状态，此时观测状态为1：

$$\delta_3(1) = \max_{1\leq j \leq 3}[\delta_2(j)a_{j1}]b_1(o_3) = \max_{1\leq j \leq 3}[0.028 \times 0.5, 0.0504 \times 0.3, 0.042\times 0.2] \times 0.5 = 0.00756$$

$$\Psi_3(1)=2$$

$$\delta_3(2) = \max_{1\leq j \leq 3}[\delta_2(j)a_{j2}]b_2(o_3) = \max_{1\leq j \leq 3}[0.028  \times 0.2, 0.0504\times 0.5, 0.042\times 0.3] \times 0.4 = 0.01008$$

$$\Psi_3(2)=2$$

$$\delta_3(3) = \max_{1\leq j \leq 3}[\delta_2(j)a_{j3}]b_3(o_3) = \max_{1\leq j \leq 3}[0.028  \times 0.3, 0.0504 \times 0.2, 0.042\times 0.5] \times 0.7 = 0.0147$$

$$\Psi_3(3)=3$$

此时已经到最后的时刻，我们开始准备回溯。此时最大概率为$𝛿_3(3)$,从而得到$𝑖_3^*=3$

由于$Ψ_3(3)=3$,所以$𝑖_2^*=3$, 而又由于$Ψ_2(3)=3$,所以$𝑖_1^*=3$。从而得到最终的最可能的隐藏状态序列为：$(3,3,3)$

## 5、HMM模型维特比算法总结

维特比算法也是寻找序列最短路径的一个通用方法，和dijkstra算法有些类似，但是dijkstra算法并没有使用动态规划，而是贪心算法。同时维特比算法仅仅局限于求序列最短路径，而dijkstra算法是通用的求最短路径的方法。

```python

```