浅谈隐式马尔可夫模型 - A Brief Note of Hidden Markov Model (HMM)

Thanks B. H. Juang in Georgia Institute of Technology.
Thanks Wayne Ward in Carnegie Mellon University.

Introduction

Problem Formulation

Now we talk about Hidden Markov Model. Well, what is HMM used for? Consider the following problem:

Given an unknown observation: $O$, recognize it as one of $N$ classes with minimum probability of error.

So how to define the error and error probability?
Conditional Error: Given $O$, the risk associated with deciding that it is a class $i$ event:

$$R(S_{i} | O) = \sum_{j=1}^{N}e_{ij}P(S_{j} | O)$$ where $P(S_{j} | O)$ is the probability of that $O$ is a class $S_{j}$ event and $e_{ij}$ is the cost of classifying a class $j$ event as a class $i$ event. $e_{ij}>0, e_{ii}=0$.
Expected Error:
$$\mathcal{E}=\int R(S(O) | O)p(O)dO$$ where $S(O)$ is the decision made on $O$ based on a policy. Then the question can be considered as:

How should $S(O)$ be made to achieve minimum error probability? Or $P(S(O)|O)$ is maximized?

Bayes Decision Theory

If we institute the policy: $S(O) = S_{i} = \arg\max_{S_{j}}P(S_{j} | O)$ then $R(S(O)|O)=\min_{S_{j}}R(S_{j} | O)$. It is the so-called Maximum A Posteriori (MAP) decision. But how do we know $P(S_{j}|O), i=1,2,\dots,M$ for any $O$ ?

Markov Model

States : $S=\{S_{0}, S_{1},S_{2},\dots, S_{N}\}$
Transition probabilities : $P(q_{t}=S_{i} |q_{t-1}=S_{j})$
Markov Assumption:

$$P(q_{t}=S_{i} |q_{t-1}=S_{j}, q_{t-1}=S_{k},\dots)=P(q_{t}=S_{i} |q_{t-1}=S_{j})=a_{ji}, \quad a_{ji}\geq 0, \sum_{i=1}^{N}a_{ji}=1,\forall j$$

Hidden Markov Model

States: $S=\{S_{0}, S_{1},S_{2},\dots, S_{N}\}$
Transition probabilities : $P(q_{t}=S_{i} |q_{t-1}=S_{j})=a_{ji}$
Output probability distributions (at state $j$ for symbol $k$): $P(y_{t}=O_{k} | q_{t}=S_{j})=b_{j}(k, \lambda_{j})$ parameterized by $\lambda_{j}$ .

HMM Problems and Solutions

1. Evaluation: Given a model, compute probability of observation sequence.
2. Decoding: Find a state sequence which maximizes probability of the observation sequence.
3. Training: Adjust model parameters which maximizes probability of observed sequences.

Evaluation

Compute the probability of observation sequence $O=o_{1} o_{2} \dots o_{T}$ , given a HMM model parameter $\lambda$:

$$\begin{split} P(O|\lambda) & = \sum_{\forall Q}P(O, Q|\lambda),\qquad Q=q_{0}q_{1} q_{2}\dots q_{T} \\ & = \sum_{\forall Q} a_{q_{0}q_{1}}b_{q_{1}}(o_{1})\cdot a_{q_{1}q_{2}}b_{q_{2}}(o_{2})\cdots a_{q_{T-1}q_{T}}b_{q_{T}}(o_{T}) \end{split}$$
This is not practical since the number of paths is $O(N^{T})$. How to deal with it?

Forward Algorithm

$$\alpha_{t}(j)=P(o_{1}o_{2}\dots o_{t}, q_{t}=S_{j} | \lambda)$$
Compute $\alpha$ recursively:
$$\begin{gather} \alpha_{0}(j)=\left\{\begin{aligned}&1,\quad \text{if $S_j$ is the start state}\\ &0,\quad \text{otherwise}\end{aligned}\right.\\ \alpha_{t}(j)=\left[ \sum_{i=0}^{N}\alpha_{t-1}(i)a_{ij} \right]b_{j}(o_{t}),\quad t>0 \end{gather}$$ Then
$$P(O|\lambda)=\alpha_{T}(N)$$
Computation is $O(N^{2}T)$.

Backward Algorithm

$$\beta_{t}(i)=P(o_{t+1}o_{t+2}\dots o_{T}|q_{t}=S_{i}, \lambda)$$
Compute $\beta$ recursively:
$$\begin{gather} \beta_{T}(i)=\left\{\begin{aligned}&1,\quad \text{if $S_i$ is the end state}\\ &0,\quad \text{otherwise}\end{aligned}\right.\\ \beta_{t}(i)= \sum_{j=0}^{N}a_{ij} b_{j}(o_{t+1})\beta_{t+1}(j),\quad t<T \end{gather}$$ Then
$$P(O|\lambda)=\beta_{0}(0)$$
Computation is $O(N^{2}T)$.

Decoding

Find the state sequence $Q$ which maximizes $P(O, Q | \lambda)$.

Viterbi Algorithm

$$VP_{t}(i)=\max_{q_{0}q_{1}\dots q_{t-1}}P(o_{1}o_{2}\dots o_{t}, q_{t}=S_i|\lambda)$$
Compute $VP$ recursively:
$$VP_{t}(j)=\max_{i=0,1,\dots N}VP_{t-1}(i)a_{ij}b_{j}(o_{t})\quad t>0$$ Then
$$P(O, Q|\lambda)=VP_{T}(N)$$ Save each maximum for backtrace at end.

Training

For the sake of tuning $\lambda$ to maximize $P(O|\lambda)$, there is NO efficient algorithm for global optimum, nonetheless, efficient iterative algorithm capable of finding a local optimum exists.

Baum-Welch Reestimation

Define the probability of transiting from $S_{i}$ to $S_{j}$ at time $t$ given $O$ as1

$$\xi_{t}(i,j)=P(q_{t}=S_{i}, q_{t+1}=S_{j} |O, \lambda)=\frac{\alpha_{t}(i)a_{ij}b_{j}(O_{t+1})\beta_{t+1}(j)}{P(O|\lambda)}$$

Let

$$\begin{gather}\bar{a}_{ij}=\frac{\text{Expected num. of trans. from $S_{i}$ to $S_{j}$}}{\text{Expected num. of trans. from $S_{i}$}}=\frac{\sum_{t=0}^{T-1}\xi_{t}(i,j)}{\sum_{t=0}^{T-1}\sum_{j=0}^{N}\xi_{t}(i,j)} \\ \bar{b}_{j}(k)=\frac{\text{Expected num. of times in $S_{j}$ with symbol $k$}}{\text{Expected num. of times in $S_{j}$}}=\frac{\sum_{t:O_{t+1}=k}\sum_{i=0}^{N}\xi_{t}(i,j)}{\sum_{t=0}^{T-1}\sum_{i=0}^{N}\xi_{t}(i,j)} \end{gather}$$

Training Algorithm:

1. Initialize $\lambda = (A, B)$.
2. Compute $\alpha, \beta$ and $\xi$.
3. Estimate $\bar{\lambda}=(\bar{A}, \bar{B})$ from $\xi$.
4. Replace $\lambda$ with $\bar{\lambda}$.
5. If not converge, go to 2.

Reference

More details needed? Refer to :

1. “An Introduction to Hidden Markov Models”, by Rabiner and Juang.
2. “Hidden Markov Models: Continuous Speech Recognition”, by Kai-Fu Lee.

---

1. Forward-Backward Algorithm
   ↩