NLP basics: HMM

(Completed in the morning on the 6th)

Problem scenario-throwing unbalanced coins

Assuming that there are two coins A and B with uneven texture, the probability that coin A is positive is ${\mu }_1$, The probability that coin B is positive is ${\mu }_2$. Now there is a situation: Xiao Ming is tossing a coin. I don't know what coin (A or B) Xiao Ming throws. I can only see the observation sequence of the positive and negative results after the coin is thrown.
In this scenario, there are three uncertain elements: the probability that Xiaoming chooses to toss coin A or coin B when he first throws it; the transfer probability of which coin to toss next after Xiao Ming chooses a coin; Probability of the coin that is out of the coin.

The process of generating observation sequences from the above hidden variables is called a hidden Markov model. Hidden Horse has three basic problems:

Q1 Inference Problem

If the model parameters are known, how to reverse the values ​​of hidden variables based on the observed values? --> Viterbi algorithm

(Use the positive and negative of the coin to tell whether the coin is A or B)

Q2 The process of estimating parameters

Know the observed sequence value and estimate the parameters of the model; --> EM algorithm
(through the positive and negative sequence of the coin to derive the probability of the positive and negative sides of the coin A and B, and the probability of choosing whether the coin is A or B for the first time)

Q3: Forecast sequence

How to calculate the probability of an observation sequence when the model parameters are known?
(Find the probability of a certain positive and negative sequence of the coin)

Application scenario: part of speech tagging Pos

Question 1: Given model parameters, find the most suitable z

Option 1: Consider all possible values;

Option 2: FB Algorithm

Option 3: Viterbi

Problem 2: Inference Problem

Forward Algorithm

Backward Algorithm

Question 3: Estimate