Efficient implementation of Level FHE & advanced algorithm compatible with Level FHE

references:

1. [CS05] Choi Y, Swartzlander E E. Parallel prefix adder design with matrix representation[C]//17th IEEE Symposium on Computer Arithmetic (ARITH’05). IEEE, 2005: 90-98.
2. [SV11] Smart N P, Vercauteren F. Fully homomorphic SIMD operations[J]. Designs, codes and cryptography, 2014, 71: 57-81.
3. [GHS12] Gentry C, Halevi S, Smart N P. Fully homomorphic encryption with polylog overhead[C]//Annual International Conference on the Theory and Applications of Cryptographic Techniques. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012: 465-482.
4. [GHS12] Gentry C, Halevi S, Smart N P. Homomorphic evaluation of the AES circuit[C]//Annual Cryptology Conference. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012: 850-867.
5. [HS13] Halevi S, Shoup V. Design and implementation of a homomorphic-encryption library[J]. IBM Research (Manuscript), 2013, 6(12-15): 8-36.
6. [ZQH+20] Zhang N, Qin Q, Hou Z, et al. Efficient comparison and addition for FHE with weighted computational complexity model[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020, 40(9): 1896-1908.

Carry-Lookahead Adder

Given two integers $A=a_{n-1}\cdots a_1{a}_0$，$B=b_{n-1}\cdots b_1{b}_0$, in order to calculate their sum, there are a variety of adders to choose from. We use the following logic symbols: AND gate $\cdot$ , OR gate$+$ , XOR gate$\oplus$ , where$\cdot$ has the highest priority.

Input a , b , cin ∈ { 0 , 1 } a,b,cin \in \{0,1\} of Full Adder (FA)a,b,cin∈{ 0,1}，输出 $sou\_\_cout\in \{0,1\}$，逻辑表达式为：
$$\begin{array}{rl}sout& =a\oplus b\oplus cin\\ cout& =a\cdot b+a\cdot cin+b\cdot cin\end{array}$$

Define signal $g=a\cdot b$（generate），$k=a+b$（kill），$p=a\oplus b$（propagate），那么
$$\begin{array}{rl}sout& =p\oplus cin\\ cout& =g+k\cdot cin\end{array}$$

Ripple-Carry Adder

Traveling wave carry adder (RCA): convert $n$ full adders are connected in series, input$A,B,c_0$，迭代计算，
$$\begin{array}{rl}{g}_i& =a_i\cdot b_i\\ k_i& =a_i+b_i\\ p_i& =a_i\oplus b_i\\ & \\ s_i& =p_i\oplus c_i\\ c_{i+1}& =g_i+k_i\cdot c_i\end{array}$$

The length of the critical path is the carry chain $c_1,\cdots ,c_{n-1},c_n$The number of iteration rounds $O\left(n\right)$

Carry-Lookahead Adder

Carry-ahead adder (CLA): Continuously expand the carry chain of RCA to obtain
$$\begin{array}{rl}{c}_{i+1}& =g_i+k_i\cdot (g_{i-1}+k_{i-1}\cdot (g_{i-2}+\cdots ))\\ & =g_i+\sum _{j=0}^{i-1}\left(\prod _{l=j+1}^i{k}_l\right)g_j+c_0\prod _{l=0}^i{k}_l\end{array}$$

So the carry signal $c_{i+1}$It can be directly based on $(g_i,k_i)$ signal is calculated without waiting for$c_i$Signal. The critical path length is only $O(\log n)$ , but the fan-in and fan-out coefficients are extremely high. In order to balance computational delay and circuit complexity, you can use [Blocked/Graded Carry Lookahead] ([Hardware Arithmetic Design] Hardware Adder Principle | Chapter 2: Carry Lookahead Adder, Blocked and Hierarchical Carry Lookahead Addition Device - Zhihu (zhihu.com)).

Parallel-Prefix Adder

Parallel Prefix Adder (PPA): In order to better understand CLA, it can be described in algebraic language $(g_i,k_i)$ signal merging process. We define the ordered pair$(g,$Binary operation ∘ \circon$k$$)$$\circ$ 如下：
$$(g_j,k_j)\circ (g_i,k_i):=(g_j+k_j{g}_i,k_i{k}_j)$$

Easy to verify, it is: associative, idempotent, and non-commutative.

Then the logical expression of CLA can be rewritten as:
$$\begin{array}{rl}(c_{i+1},k_0{k}_1\cdots k_i)& =(g_i+k_i\cdot (g_{i-1}+k_{i-1}\cdot (g_{i-2}+\cdots )),k_i\cdots k_1{k}_0)\\ & =(g_i,k_i)\circ (g_{i-1},k_{i-1})\circ \cdots \circ (g_0,k_0)\circ (c_0,1)\end{array}$$

Each signal $(g_i,k_i)$ and$(c_0,1)$ are all immediate. According to the associative law of binary operations, a family (g, k) (g,k)can be designed$(g,k)$ Parallel merging network of signals. Knowles proposed a class of depth-optimal parallel prefix adders. The circuit structure is:

Knowles Adder is limited to two well-known PPAs: Kogge-Stone Adder, Ladner-Fischer Adder . The LF structure requires infinite fan-out (suitable for FHE operations), while the KS structure requires infinite parallelism (number of multiplications). Brent-Kung Adder balances the fan-out coefficient and the number of computing units by increasing the number of level layers (multiplication depth), and there are some Hybrid Schemes that mix KS, LF, and BK structures.

We use Parallel-Prefix Graghs to simplify the representation of PPA, and each black dot represents a binary operation $\circ$ operation (two multiplications, one addition), the operation direction is from bottom to top. Knowles Adder with different fanouts:

The most suitable PPA for FHE operations is the Ladner-Fischer Adder: our soft implementation does not care about the fan-out coefficient, but focuses on the multiplication depth and number of multiplications.

HE library

Double-CRT

[HS13] designed the RLWE-BGV code implementation library HElib based on C++ NTL , using the SIMD technology of [SV11] and the Permuting/Routing technology of [GHS12].

For further acceleration, [HS13] converts the ring ${\mathbb{Z}}_q\left[x\right]/{\varphi }_m$The polynomials in$($$x$$)\; are\; expressed\; in\; double-CRT\; form.$Chain of moduli is used in BGV,$q_l={\prod }_{i=0}^l{p}_i$, in which $p_i$is a prime number of approximate size and satisfies $m|p_i-1$ makes${\mathbb{Z}}_{p_i}$mm exists in$mth$ primitive unit root$g_i$，于是 ${\varphi }_m(x)={\prod }_{j\in {\mathbb{Z}}_m^{\ast }}(x-g_i^j)(\mathrm{mod}{p}_i)$

Let’s first consider the modulus $q_l$Do CRT decomposition, abbreviation $R=Z\left[x\right]/{\varphi }_m\left(x\right)$ ,$R_p=R/(pR)$，
$$Z_{q_l}\left[x\right]/{\varphi }_m(x)\cong R/(q_{l}R)\cong R/(p_{0}R)\times R/\left(p_{1}R\right)\times \cdots \times R/\left(p_{l}R\right)$$

Then for each small ring $R_{p_i}\cong {\mathbb{Z}}_{p_i}\left[x\right]/{\varphi }_m(x)$ 继续做 CRT 分解，
$$Z_{p_i}\left[x\right]/{\varphi }_m(x)\cong Z_{p_i}[x]/(x-g_i^{j_1})\times \cdots \times Z_{p_i}\left[x\right]/(x-g_i^{j_{\varphi (m}}),j_t\in {\mathbb{Z}}_m^{\ast }$$

Assume $m=2^k$ makes${\varphi }_m(x)=x^{m/2}+1$ , thenZ m ∗ = { 1 , 3 , 5 , ⋯ , m − 1 } \mathbb Z_m^*=\{1,3,5,\cdots,m-1\}Zm∗​={ 1,3,5,⋯,m−1 } , you can useFFT/NTTto calculate quickly. For general$m$ (mixed radix-X), in$Z_m^{\ast }$The distribution of numbers in is irregular, and the conversion between coefficient representation and Double-CRT is more complicated.

To sum up, given the modulus $q_l$The ring element $a\in Z_{q_l}\left[x\right]/{\varphi }_m\left(x\right)$ , can be expressed in matrix form:$i$ 行是 $a(x)(\mathrm{mod}{p}_i)$ of the NTT domain,ii.Line$i$$j$ 列是元素 $a(g_i^j)\left(mod{p}_i\right)$ , shape$(l+1)\times \varphi (m)$
D o u b l e C R T l ( a ) = { a ( x ) ( m o d p i ) ( m o d x − ζ i j ) } 0 ≤ i ≤ l ,    j ∈ Z m ∗ = { a ( ζ i j ) ( m o d p i ) } 0 ≤ i ≤ l ,    j ∈ Z m ∗ \begin{aligned} DoubleCRT^l(a) &= \{a(x) \pmod{p_i}\pmod{x-\zeta_i^j}\}_{0\le i\le l,\,\, j\in \mathbb Z_m^*}\\ &= \{a(\zeta_i^j) \pmod{p_i}\}_{0\le i\le l,\,\, j\in \mathbb Z_m^*}\\ \end{aligned} DoubleCRTl(a)​={ a(x)(modpi​)(modx−gij​)}0≤i≤l,j∈Zm∗​​={ a ( gij​)(modpi​)}0≤i≤l,j∈Zm∗​​​

It is easy to verify that the addition/multiplication of ring elements is the component-wise addition/multiplication of the matrix ,
$$\begin{array}{rl}DoubleCR{T}^l(a+b)& =DoubleCR{T}^l(a)+DoubleCR{T}^l\left(a\right)\\ DoubleCR{T}^l(a\cdot b)& =DoubleCR{T}^l\left(a\right)\cdot DoubleCR{T}^l(a)\end{array}$$

Galois function $Gal\left(Q\right(\zeta )/Q)\cong {\mathbb{Z}}_m^{\ast }$The automorphism mapping in is
$$K_k(a(x)):=a(x^k)(\mathrm{mod}{\varphi }_m(x)),\forall k\in {\mathbb{Z}}_m^{\ast }$$

Because $m|p_i-1$ ,the function$\mathcal{G}a\mathcal{L}({\mathbb{Z}}_{p_i}(g)/Z_{p_i})$ are permutations between plaintext slots (Frob map$K_1\left(g\right)=g^{p_i}=\zeta$是恒健在线).Other mappingκ$K_k:g\mapsto g^k$ is equivalent to: converting the matrix$DoubleCR{T}^{}$The jjthof${}^l$$($$a$$)$$Column\; j$ , move to$jk(\mathrm{mod}m)$ column. So Routing is easy to implementon Double-CRT representation.

New Key Switching in CRT

Key switching is essentially about $W[s_1\to s_2]=En{c}_{s_2}(s_1;$Homomorphic multiplication operationof$e$$)$ :
$$\begin{gathered} c_2=c_{1}W=En{c}_{s_2}(c_1{s}_1;c_{1}e) \\ {c}_2{s}_2=c_{1}W{s}_2=c_1{s}_1 \end{gathered}$$

But the ciphertext $c_1$Norm as a scalar relative to modulus $q_l$If it is too large, the noise term after homomorphic operation will destroy the plaintext.

In BGV, in order to control the noise term of key switching, a binary decomposition scheme is used : given $s^{}$Ciphertextc ′ c'${}^{under\; \prime }$$c^{\prime }$，计算 $c^{\prime }={\sum }_i{2}^i{c}_i^{\prime }$，令
$$c^{\ast }=c_0^{\prime }|c_1^{\prime }|c_2^{\prime }|\cdots ,s^{\ast }=s^{\prime }|2s^{\prime }|4s^{\prime }|\cdots$$

Then calculate the matrix $W=W[s^{\ast }\to s]$ , which satisfies$s^{\ast }=W\cdot s$ , then calculate the ciphertext$c=c^{\ast }\cdot W$，
$$⟨c^{\prime },s^{\prime }⟩=⟨c^{\ast },s^{\ast }⟩=⟨c,s⟩$$

In order to calculate $c^{\prime }$ binary decomposition, first need to convert Double-CRT back to coefficient representation. Then,$\log q_l$Ci $c_i^{\prime }$Convert to Double-CRT respectively to perform the inner product operation of decryption. This costs $O(l\log q_l)$ length is$n=$FFT/NTT operation of$\varphi$$($$m$$)\; .$

[GHS12] used a different noise term control technique: temporarily boosting the modulus . Using the LSB encoding scheme, assume $⟨c^{\prime },s^{\prime }⟩=2e^{\prime }+a(\mathrm{mod}q)$ , for any odd number$p$ ,dimensions⟨
$$⟨c^{\prime },p{s}^{\prime }⟩=2p{e}^{\prime }+pa=2e^{\prime \prime }+a(modpq)$$

Among them, $e^{\prime \prime }=p{e}^{\prime }+a(p-1)/2$ is approximately the original noise$e^{\text{'}}$ pp$p$ times. Just perform a modulo switch and return to modulo$q$ , then the noise is basically still$e^{\prime }$ magnitude. Therefore, we choose a sufficiently large odd number$p\approx q\sqrt{m}$, use the transformation matrix $W=W[p{s}^{\prime }\to s](\mathrm{mod}The\; ciphertext\; of\; pq)$ serves as a key switching aid.

Although there is no need for $c^{\prime }$ has been binary decomposed, but it still needs to be calculated$c^{\prime }\left(modp\right)$ to form${\mathbb{Z}}_{pq}\left[x\right]/{\varphi }_m$Double-CRT representation of$($$x$$)\; .$However, because there is no need for dimension expansion$\log q_l$times, so only $O\left(l\right)$ FFT/NTT operations.

Note that because the modulus $pq\ge q^2$ , making the ratio of modulus/noise very large, so the dimension of the grid needs to be$n=\varphi \left(m\right)$ is expanded nearly twice to maintain safe strength. But the complexity of FFT/NTT is$O\left(n\log n\right)$ , so the calculation efficiency is still improved by about$\log q_l/2$ factor, and the storage overhead of public keys is also lower.

You can also use a mixture of base decomposition and modulus improvement: convert the ciphertext $c^{\prime }$ Decomposition为${\sum }_i{D}^i{c}_i^{\prime }$, where $D$ is a large radix such that$c_i^{\prime }$The number of is very small; then, for each $c_i^{\prime }$Adopt a plan to temporarily increase the modulus.

Modulus Switching in CRT

We hope to keep the ciphertext and secret key in Double-CRT representation except for some necessary coefficient representations to facilitate quick calculation. From the modulus $q_l={\prod }_{j=0}^l{p}_j$The cipher text under $c$ switches to modulus$q_{l-1}={\prod }_{j=0}^{l-1}{p}_j$The ciphertext c ′ c' under$c^{\prime }$ , maintain the decryption correctness$c^{\prime }s\equiv cs(\mathrm{mod}2)$ , and also requires "rounding error"$t=c^{\prime }-c/p_t$The norm of is very small.

Because $p_l$is an odd prime number, just calculate $c^{†}=p_l\cdot c^{\prime }$ , such that:$p_l|c^{†}$，$c^{†}\equiv c(\mathrm{mod}2)$ (sufficient but not necessary), and$c^{†}-$The norm of$c\; is\; very\; small.$Then output$c^{\prime }=c^{†}/p_l$That’s it. The mode switching algorithm under Double-CRT is:

1. From the last line of Double CRT, calculate $\bar{c}=c\left(mod{p}_l\right)$ , the value range is$[-p_l/2,p_l/2)$ , but note that$\stackrel{ˉBelongs}{c}$ to$R$ (available in$R_{q_l}$Medium simulation)
2. Will $\stackrel{Those\; coefficients\; of\; ˉ}{c}$ are odd numbers, by adding and subtracting$p_l$becomes an even number, the value range is $[-p_l,p_l)$ , get the small polynomial$\delta$ , meet$d\equiv c(\mathrm{mod}{p}_l)$ and$d\equiv 0(mod2)$
3. Default $Double\; CRT\; representation\; of\; \delta$ , and then calculate$c^{†}=c-d(\mathrm{mod}{q}_l)$ , which satisfies$p_l|c^{†}$
4. For each $c^{†}(mod{p}_j),j<l$ , by calculating$p_l^{-1}\cdot c^{†}\left(mod{p}_j\right)$ , obtain$c^{\prime }=c^{†}/p_l$Double-CRT representation

In the above process, step 1 cost $1$ INTT, step 3 cost$l$ NTT, totaling$O\left(l\right)$ length$n=$FFT/NTT operation of$\varphi$$($$m$$)\; .$

WCC

In high-level applications of Level FHE, the number of multiplications and the depth of multiplication can be used to measure the efficiency of the algorithm. However, in many works, people tend to only focus on a certain aspect rather than comprehensively considering it. Therefore, the advanced algorithms designed are not necessarily suitable for calculations on level FHE.

[ZQH+20] proposed the Weighted Computational Complexity (WCC) computing model. The complexity is defined as
$$WCC=\sum _{l=0}^L{W}_l\cdot N_l$$

Among them $N_l$It’s the llthThe number of multiplications in layer$l$$W_l$It’s the llthMultiplication cost of layer$l$$W_l=1,\forall l$ only focuses on the number of multiplications, if$W_l=0,l<L$ only focuses on the multiplication depth). According to [GHS12], the cost of key switching and mode switching under Double-CRT is$O\left(l\right)$ length$n=$NTT/FFT operation of$\varphi$$($$m$$)\; .$Therefore, ignoring the complicated details, roughly select the weights as$W_l=l+1,l\ge 0$ , which is appropriate and sufficient to guide us in designing advanced algorithms more suitable for level FHE.

WCC Analysis

[ZQH+20] analyzed the WCC complexity of integer comparison and integer addition.

Three comparison algorithms,

1. Linear Comparison (LinC)，线性比较
   $$co{m}_i=(a_i\oplus 1)\cdot b_i\oplus (a_i\oplus b_i\oplus 1)\cdot co{m}_{i-1}$$

   Final output $AS(A,B)=co{m}_{n-1}$, it can be calculated that $WCC=O(n^2)$

2. Iteration Comparison (IterC)，分治算法
   $$\begin{gathered} t_{i,1}=(a_i\oplus 1)\cdot b_i,z_{i,1}=a_i\oplus b_i\oplus 1 \\ {t}_{ij}=t_{i+h,j-h}+z_{i+h,j-h}\cdot t_{i,h},j>1,h=\lceil j/2\rceil \end{gathered}$$

   Final output $AS(A,B)=t_{0,n}$, its WCC general formula is more complicated and difficult to write

3. Logarithm Comparison (LogC), complete expansion
   $$\begin{gathered} d_i=(a_i\oplus 1)\cdot b_i,z_i=a_i\oplus b_i\oplus 1 \\ {c}_i=d_i\cdot \prod _{j=i+1}^{n-1}{z}_j \end{gathered}$$

   Final output $AS(A,B)={\sum }_{i=0}^{n-1}{c}_i$, use a binary tree to calculate $Z_{n-1}={\prod }_{j=1}^{n-1}{z}_j$, other $Z_{n-i}={\prod }_{j=i}^{n-1}{z}_j$is the product of some intermediate results of this binary tree (the total number of multiplications is $O\left(n\log n\right)$ But$O\left(n\right)$ ). Its complexity is$WCC=O(n{\log }^{2}n)$

Addition of two integers,

1. Ripple-Carry Adder, can calculate $WCC=O\left(n^2\right)$, the hidden constant is larger than LinC
2. Parallel-Prefix Adder, can calculate $WCC=O\left(n{\log }^{2}n\right)$ , the hidden constant is larger than LogC

Dot Multiplication

Similar to the binary operation of PPA, [ZQH+20] defines DotMC,
$$(P_1,G_1)\circ (P_0,G_0)=(P_1+G_1\cdot P_0,G_1\cdot G_0)$$

简记 $Add[(P,G)]=P+G$ is the merging operation of the final signal. Define the initial signal$p_i=(a_i\oplus 1)\cdot b_i,0\le i\le n-1$ ,$g_i=a_i\oplus b_i\oplus 1,1\le i\le n-1$，$g_0=0$，那么有：
$$\begin{array}{rl}AS(A,B)& =(a_{n-1}\oplus 1)\cdot b_{n-1}\oplus (a_{n-1}\oplus b_{n-1}\oplus 1)\cdot co{m}_{n-2}\\ & =Add\left[(p_{n-1},g_{n-1})\circ (p_{n-2},g_{n-2})\circ \cdots \circ (p_0,g_0)\right]\end{array}$$

Because the DotMC operation is combined, the binary tree method can be used to combine signals in parallel.

It can be calculated that $WCC=O\left(n\log n\right)$ , the number of multiplications is$O\left(n\right)$ , multiplication depth$O(\log n)$ . In comparison, the number of multiplications of LogC is$O\left(n\log n\right)$ , the computational complexity is$WCC=O\left(n{\log }^{2}n\right)$ . Reason: Similar tothe Horner Rule, there are many multiplications that are redundant.

CLA itself uses DotMC, but some of its DotMCs can delay calculations and shift calculations to lower levels, making $W_l$smaller.

The original CLA circuit is:

Move some DotMC (green dots),

The paper points out that when $n>At\; 16$ , the comparison of DotMC and OptCLA is more efficient, and with$The\; effect\; is\; better\; when\; n$ is increased.