[Filter] Normalized LMS adaptive filter

1. NLMS adaptive filter

1 Introduction

In the adaptive filtering algorithm, steady-state error and convergence speed are two of the most important performance indicators. For the traditional fixed-step adaptive filter, there is a great contradiction in satisfying the steady-state error and the convergence speed: a smaller step size can obtain a small steady-state error, but the convergence speed is relatively slow; on the other hand, If the step size is increased, the convergence speed of the filter is improved, but at the expense of a larger steady-state error.
For this reason, a variable step size adaptive filtering algorithm is proposed, that is, the normalized least mean square (Normalized Least Mean Square, NLMS) adaptive filtering algorithm . The filtering algorithm has a larger step size in the initial stage of filtering, and the convergence speed is faster; when the algorithm converges, the step size is reduced to ensure a higher convergence accuracy.

Do you have a question here? The NLMS algorithm is only normalized on the basis of the LMS algorithm. Why is it an LMS algorithm with variable step size? This question will be explained at the end of the algorithm formula derivation, please read it patiently.

2. Principle derivation

In the weight vector update formula of the LMS filter, $X\left(n\right)$ is a noisy signal, when$When\; X\left(n\right)$ is large, the LMS algorithm will have the problem of gradient noise amplification. In order to solve this problem, the normalized LMS algorithm can be used.
Let$W\left(n\right)$ sum$W(n+1)$ Respectively represent the$n$ times and$n+1$ , the design criterion of the normalized LMS filter can be expressed as the following constrained optimization problem:
Given an input vector$X\left(n\right)$ and desired output$d\left(n\right)$ , determine the updated weight vector as$W(n+1)$ so that the incremental vector$\delta W(n+1)=W(n+1)-$The Euclidean norm of$W$$($$n$$)$
$$min||\delta W(n+1)||The$$
above formula is limited by the constraints$W(n+1{)}^H\cdot X(n)=d(n)$ .
Convert the constrained optimization problem to an unconstrained optimization problem using the Lagrange multiplier method, and construct a real-valued quadratic cost function:
$$J\left(n\right)=||\delta W(n+1)|{|}^2+Re[l^{\ast }(d(n)-W(n+1{)}^H\cdot X(n))]$$
, where$\lambda$ is a complex Lagrangian multiplier,$\ast$ means complex conjugate,$Re$ represents the real part operation.
Further, there is:
$$J\left(n\right)=[W(n+1)-W(n){]}^H\cdot [W(n+1)-W(n)]+\lambda [d\left(n\right)-W(n+1{)}^H\cdot X(n)]$$
is such that the cost function$J\left(n\right)$ has a minimum value for$W(n+1)$最作求密：
$$\frac{\partial J(n)}{\partial W(n+1)}=2[W(n+1)-W(n)]-\lambda \cdot X\left(n\right)$$
令$\frac{\partial J(n)}{\partial W(n+1)}=0$，则$W(n+1)=W(n)+\frac{1}{2}\lambda \cdot X\left(n\right)$， 共代代入美国安全可得：
$$d(n)=[W(n)+\frac{1}{2}\lambda \cdot X(n){]}^H\cdot X(n)=W^H(n)\cdot X(n)+\frac{1}{2}\lambda \cdot X^H(n)\cdot X(n)=W^H(n)\cdot X(n)+\frac{1}{2}\lambda \cdot ||X(n)|{|}^2$$
又生$d(n)-W^H\left(n\right)\cdot X\left(n\right)=e\left(n\right)$ , substitute it into the above formula to get:
$$l=\frac{2e(n}{||X\left(n\right)|{|}^2}$$
Combining the above formulas, we can get:
$$\delta W(n+1)=W(n+1)-W(n)=\frac{1}{2}\lambda \cdot X(n)=\frac{X(n)}{||X(n)|{|}^2}\cdot e\left(n\right)$$
In order to control the incremental change of the tap weight vector from one iteration to the next without changing the direction of the vector, a positive real scalar factor μ$\mu$， 可得：
$$\mu \cdot \delta W(n+1)=\mu \cdot [W(n+1)-W(n)]=\mu \cdot \frac{1}{2}\lambda \cdot X(n)=\frac{\mu \cdot X(n)}{||X(n)|{|}^2}\cdot e\left(n\right)$$
finally obtains the cost functionJ ( n ) J(n)The optimal solution of$J$$($$n$$)$
$$W(n+1)=W(n)+\frac{}{||X(n)|{|}^2}X(n)\cdot e\left(n\right)$$
The above formula is the weight vector update formula of the normalized LMS, which shows the product vector$X\left(n\right)\cdot e\left(n\right)$ is relative to the tap input vector$The\; squared\; Euclidean\; norm\; of\; X\left(n\right)$ is normalized.
In order to guarantee$\frac{}{||X\left(n\right)|{|}^2}$The denominator of is not zero, let $||X(n)|{|}^2$ plus a constant$a(0<a\le 1)$ .
$$W(n+1)=W(n)+\frac{}{a+||X\left(n\right)|{|}^2}X(n)\cdot e\left(n\right)$$
If you understand that NLMS is a variable step size LMS algorithm:

把$\frac{}{\alpha +||X\left(n\right)|{|}^2}$As a whole, the step size $m^{\prime }=\frac{}{\alpha +||X\left(n\right)|{|}^2}$, when the weight vector is in the iterative update process, due to $X\left(n\right)$ is changing, so the step size$m^{\prime }$ will also be updated, that is, it is considered that during the working process of the NLMS filter, its step size will follow the input signal$X\left(n\right)$ changes.

The calculation steps of NLMS adaptive filter are summarized as follows:
$and\left(n\right)=W^T(n)X(n)$ --------------------------------------------------------------- (1)

$e(n)=d(n)-y\left(n\right)$ ---------------------------------------------- ------------------- (2)

$W(n+1)=W(n)+\frac{}{\alpha +||X\left(n\right)|{|}^2}X(n)\cdot e(n)$ ------------------------------- (3)

2. MATLAB simulation experiment

1. Filter effect

When filter order $L=20$ , step factor$m=When\; the\; value\; is\; 0.05$ , there is no obvious difference between the filtering effect of the LMS filter and the NLMS filter on the input signal with a signal-to-noise ratio of 15dB.
If the step size is increased to more than 0.06, the convergence speed of the LMS filter will be significantly faster, but the steady-state error will rise sharply, and the performance of the filter will deteriorate; while the NLMS filter still maintains good filtering performance.

clc;
clear;
close all;

%% 产生仿真信号
fs = 1000;                  % 采样频率
t = (0:1/fs:1-1/fs);        % 时间
f = 10;                     % 信号频率
x = sin(2*pi*f*t+pi/3);     % 原始信号
y = awgn(x,15,'measured');  % 添加高斯白噪声后的信号

%% NLMS自适应滤波器
L = 20;     % 滤波器阶数
Mu = 0.005;   % μ的范围为0到1
xn = y;     % 输入信号
dn = x;     % 期望信号
Alpha = 0.2;

[yn1, W1, en1] = LMS(xn,dn,L,Mu);
[yn2, W2, en2] = NLMS(xn, dn, L, Mu, Alpha);

%% 画图
figure;
subplot(3,1,1);plot(t,xn);xlabel('时间/s');ylabel('幅值');title('滤波器的输入信号');
subplot(3,1,2);plot(t,yn1);xlabel('时间/s');ylabel('幅值');title('LMS滤波器的输出信号');
subplot(3,1,3);plot(t,yn2);xlabel('时间/s');ylabel('幅值');title('NLMS滤波器的输出信号');
figure;
subplot(2,1,1);plot(en1);title('LMS滤波器的误差信号收敛情况');
subplot(2,1,2);plot(en2);title('NLMS滤波器的误差信号收敛情况');

function [yn, W, en] = NLMS(xn, dn, M, mu, alpha)

% input:
%       xn: 输入信号，大小为 1 x n
%       dn: 期望信号
%       M:  滤波器的阶数
%       mu: 收敛因子(步长)
%       alpha: 矫正值
%  output:
%         yn:   输出信号
%         W:    输出权系数
%         en:   输出误差信号

    [m,n] = size(xn);
    if m>1  % 如果输入信号为一列，则进行转置
        xn = xn';
    end
    if m>1 && n>1
        fprintf('输入信号有误！请检查输入信号是否为一行序列');
    end
    
    itr = n;    % 迭代次数等于输入信号的长度
    en = zeros(1,itr);
    W  = zeros(M,itr);    % 初始化权值矩阵，每一列代表一次迭代

    % 求最优权系数
    for i = M:itr                   % 第i次迭代
        x = xn(i:-1:i-M+1);         % 滤波器M个抽头的输入
        y = x*W(:,i-1);             % 滤波器的输出
        en(i) = dn(i) - y;          % 第i次迭代的误差
        W(:,i) = W(:,i-1) + (mu * x' * en(i)) / (alpha + norm(x)^2); % 滤波器权值计算的迭代式
        % norm(x)^2等效为x*x'
    end
    
    % 求输出序列  
    yn = inf * ones(size(xn));      % 初值设置为无穷大，画图的时候yn的前M个值不会绘图
    for k = M:n
        x = xn(k:-1:k-M+1);  
        yn(k) = x*W(:,end);  % 最终输出结果
    end
end

2. Effect of different parameters on filter performance

(1) Step size $When\; the\; step\; size\; of\; the\; \mu$
NLMS filter is 0.01, 0.1, 1, and 1.5, the convergence of the error signal is shown in the figure below. It can be seen from the figure that the step size$\mu$ affects the convergence speed and filtering accuracy, with$As\; \mu$ increases, the convergence speed becomes faster, but after convergence, the fluctuation amplitude near the optimal solution becomes larger, and the steady-state accuracy decreases accordingly;Decreasing the value of$\mu \; can\; guarantee\; the\; filtering\; accuracy,\; but\; the\; convergence\; time\; of\; the\; algorithm\; is\; greatly\; increased,\; which\; is\; not\; suitable\; for\; systems\; with\; high\; real-time\; requirements.$Therefore, it is necessary to compromise between the convergence speed and the filtering accuracy according to the specific situation. The appropriate parameters are also related to the specific characteristics of the signal to be processed, and multiple attempts are required to obtain the optimal value.

(2) Filter order$L$
in NLMS filter step size$m=Under\; the\; condition\; of\; 0.1$ , the filter order$When\; L$ is 10, 20, or 50, the corresponding error signal convergence is shown in the figure below. It can be seen from the figure that as the order of the filter increases, the filtering accuracy of the signal is significantly improved, but in the output signal of the filter, the amount of missing data also increases.

3. References

[1] Du Jianbang, He Jinyang, Zhuo Chao. Real-time filtering method of fiber optic gyroscope signal based on least mean square adaptive algorithm [J]. Chinese Journal of Inertial Technology, 2020, 28(06): 814-818+828. [2] Zhang
Hongmei , Han Wangang. A new variable step size LMS adaptive filtering algorithm research and its application [J]. Journal of Instrumentation, 2015, 36(08): 1822-1830. [3] He Zishu, Xia Wei, etc. Modern Digital
Signal Processing and its application[M]. Beijing: Tsinghua University Press, 2009: 156-157.