[GAN] [Paper Notes] A Style-Based Generator Architecture for Generative Adversarial Networks

1. Thesis work

For thesis work, please refer to [ StyleGAN-Style-based Generative Adversarial Network ]

2.Code

Code reference styleGAN.py - facebookresearch/pytorch_GAN_zoo - GitHub1s

The simple structure of Trainer is as follows (some are omitted):

2.1 Mapping the network

2.1.1 Network structure

Composed of 8 MLPs, the output w is the same size as the input z (512×1)

class MappingLayer(nn.Module):
#dimInput：512 dimLatent：512 nLayers：8
    def __init__(self, dimIn, dimLatent, nLayers, leakyReluLeak=0.2):
        super(MappingLayer, self).__init__()
        self.FC = nn.ModuleList()

        inDim = dimIn
        for i in range(nLayers):
            self.FC.append(EqualizedLinear(inDim, dimLatent, lrMul=0.01, equalized=True, initBiasToZero=True))
            inDim = dimLatent

        self.activation = torch.nn.LeakyReLU(leakyReluLeak)

    def forward(self, x):
        for layer in self.FC:
            x = self.activation(layer(x))

2.1.2 Inputs to the mapping network

    def buildNoiseData(self, n_samples, inputLabels=None):
        inputLatent = torch.randn(
            n_samples, self.config.noiseVectorDim).to(self.device)
        return inputLatent, None
    #根据噪声维度生成随机输入


    inputLatent, targetRandCat = self.buildNoiseData(n_samples)

2.1.3 Truncation techniques

Using truncation techniques avoids sampling from extreme regions of W

Calculation method:

1. Calculate center of mass
2. Scale the deviation of a given w from the center as

        if self.training:
            self.mean_w = self.gamma_avg * self.mean_w + (1-self.gamma_avg) * mapping.mean(dim=0, keepdim=True)

        if self.phiTruncation < 1:#0.999
            mapping = self.mean_w + self.phiTruncation * (mapping - self.mean_w)

2.2 SECTION

(Although it was discovered in StyleGAN2 that AdaIN would introduce the "water droplet" problem)

• ①First, each feature map xi (feature map) is normalized independently . Each value in the feature map is subtracted from the mean of that feature map and then divided by the variance.
• ②A learnable affine transformation A (fully connected layer) converts w into the translation and scaling factor y of AdaIN in style = (ys,i,yb,i),
• ③ Then use the translation and scaling factors learned in style to perform scale and translation transformation on each feature map.

class AdaIN(nn.Module):

    def __init__(self, dimIn, dimOut, epsilon=1e-8):
        super(AdaIN, self).__init__()
        self.epsilon = epsilon
        self.styleModulator = EqualizedLinear(dimIn, 2*dimOut, equalized=True,
                                              initBiasToZero=True)
        self.dimOut = dimOut

    def forward(self, x, y):

        # x: N x C x W x H
        batchSize, nChannel, width, height = x.size()
        tmpX = x.view(batchSize, nChannel, -1)
        mux = tmpX.mean(dim=2).view(batchSize, nChannel, 1, 1)
        varx = torch.clamp((tmpX*tmpX).mean(dim=2).view(batchSize, nChannel, 1, 1) - mux*mux, min=0)
        varx = torch.rsqrt(varx + self.epsilon)
        x = (x - mux) * varx

        # Adapt style
        styleY = self.styleModulator(y)
        yA = styleY[:, : self.dimOut].view(batchSize, self.dimOut, 1, 1)
        yB = styleY[:, self.dimOut:].view(batchSize, self.dimOut, 1, 1)

        return yA * x + yB

2.3 synthesis network

2.3.1 Input constant

A randomly generated learnable parameter is input as the initial quantity

self.baseScale0 = nn.Parameter(torch.ones(1, dimMapping, 4, 4), requires_grad=True)

2.3.2 Random changes

 The architecture of StyleGAN generates images with some random (detail) changes in space by adding noise after each convolution.

            noiseMod = self.noiseModulators[nLayer]
            feature = Upscale2d(feature)
                        #卷积
            feature = group[0](feature) + noiseMod[0](torch.randn((batchSize, 1,
                                                      feature.size(2),
                                                      feature.size(3)), device=x.device))
            feature = self.activation(feature)
                       #AdaIN
            feature = group[1](feature, mapping)
                        #卷积
            feature = group[2](feature) + noiseMod[1](torch.randn((batchSize, 1,
                                                      feature.size(2),
                                                      feature.size(3)), device=x.device))
            feature = self.activation(feature)
                        #AdaIN
            feature = group[3](feature, mapping)

2.3.3 Noise generation

class NoiseMultiplier(nn.Module):

    def __init__(self):
        super(NoiseMultiplier, self).__init__()
        self.module = nn.Conv2d(1, 1, 1, bias=False)
        self.module.weight.data.fill_(0)

    def forward(self, x):

        return self.module(x)