HMM-5-连续观测-高斯混合模型(HMM-GMM)

5. 连续观测

有两个方法将连续观测序列和HMM结合.

将连续观测聚类为离散类别,这就对应到HMM离散观测序列. 然后按照HMM离散观测序列处理.
更改观测概率函数为密度函数,例如高斯概率密度, 这需要替换观测状态序列生成函数.

5.1 替换观测概率为高斯密度

在离散观测序列中,发射函数\( b_j(m) \)定义为:

\[ B = [b_j(m)] \quad where \quad b_j(m) \equiv P(O_t = v_m | q_t = S_j) \]

将其替换为高斯密度函数(也可以是其他分布概率密度函数, 但参数更新部分需要同步替换),可以让HMM实现连续观测.

\[\begin{align}
B = [b_j(O_t)] & \equiv P(O_t | q_t = S_j, \lambda) \sim \mathcal{N}(\mu_j, \sigma_j^2) \\
&= \frac{1}{\sqrt{2 \pi \sigma_j^2}} \exp \left[ \frac{-(O_t – \mu_j)^2}{2 \sigma_j^2} \right]
\end{align}\]

参数更新:

\[\begin{align}
\bar{\mu}_j &= \frac{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(i) O_t^k }{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(i)} \\
\bar{\sigma}_j^2 &= \frac{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(i) (O_t^k – \bar{\mu}_j)^2 }{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(i)}
\end{align}\]

5.2 高斯混合分布

考虑观测序列为\( L \)个高斯分布按某比例混合.

\[\begin{align}
B = [b_j(O_t)] & \equiv P(O_t | q_t = S_j, \lambda) = \sum_{l=1}^L C_{jl} \mathcal{N}(O_t, \mu_{jl}, \sigma_{jl}^2) \\
\text{s.t.} \\
& \sum_{l=1}^L C_{jl} = 1.0
\end{align}\]

参数更新需要考虑混合系数,通过混合密度期望比例更新.

\[\begin{align}
\gamma_t^k(j, l) &= \gamma_t^k(j) \frac{C_{jl} \mathcal{N}(O_t^k, \mu_{jl}, \sigma_{jl}^2)}{\sum_{l=1}^L C_{jl} \mathcal{N}(O_t^k, \mu_{jl}, \sigma_{jl}^2)} \\
\bar{C}_{jl} &= \frac{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(j,l)}{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \sum_{l=1}^L \gamma_t^k(j, l)} \\
\bar{\mu}_{jl} &= \frac{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(j,l) O_t^k }{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(j,l)} \\
\bar{\sigma}_{jl}^2 &= \frac{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(j,l) (O_t^k – \bar{\mu}_{jl})^2 }{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(j,l)}
\end{align}\]

5.3 多维高斯混合分布

多维相对单维只是替换了高斯函数,其余没有变化, \( d \)表示维度.

\[\begin{align}
\mathcal{N}(x, \mu, \Sigma) &= \frac{1}{ (2 \pi)^{\frac{d}{2}} |\Sigma|^{\frac{1}{2}} } \exp \left[ -\frac{1}{2} (x – \mu)^T \Sigma^{-1} (x – \mu) \right] \\
\gamma_t^k(j, l) &= \gamma_t^k(j) \frac{C_{jl} \mathcal{N}(O_t^k, \mu_{jl}, \Sigma_{jl})}{\sum_{l=1}^L C_{jl} \mathcal{N}(O_t^k, \mu_{jl}, \Sigma_{jl})} \\
\bar{\Sigma}_{jl} &= \frac{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(j,l) (O_t^k – \bar{\mu}_{jl}) (O_t^k – \bar{\mu}_{jl})^T }{\sum_{k=1}^K \sum_{t=1}^{T_{O^k}} \gamma_t^k(j,l)}
\end{align}\]

在多维分布计算中,连续观测序列的数据样本过少,样本各维度之间存在线形组合,会造成协方差矩阵奇异(共线/共面), 然后高斯函数计算失败. 因此样本数量需要远多于维度,且通过预处理消除线形组合维,如必要,可以有意添加噪声.

5.4 数值稳定

工程中使用\( \hat{\gamma}_t^k(j) \) 替换 \( \gamma_t^k(j) \), 否则可能因为序列过长导致计算结果超出机器精度容许范围.

5.5 代码试验

import numpy as np
from matplotlib import pyplot as plt
 
# multivaiance normal probability density function
def Multivar_Norm_PDF(x, mean, cov):
    dev = x - mean
    y = np.dot(dev, np.linalg.pinv(cov))  
    y = np.sum(y * dev, axis=0)  
    y = np.exp(  -0.5 * y )    
    y = (1.0 / ((2*np.pi)**(len(x) / 2) * np.linalg.det(cov) ** 0.5)) * y
    return y
 
# Gaussian mixture model
def GMM_PDF(x, coef, mean, cov):
    from scipy import stats
    v1 = sum([coef[l] * stats.multivariate_normal(mean[l], cov[l], True).pdf<

Shi Zhuolin's Blog

This guy is lazy, leaving nothing left.