AIMET API 文档（3）

1.1.4 模型验证实用程序

AIMET 提供了模型验证器实用程序来帮助检查 AIMET 功能是否可以应用于 Pytorch 模型。模型验证器当前检查以下条件：

没有模块被重用
操作具有与其关联的模块，并且未定义为功能（不包括一组已知操作）

在本节中，我们将介绍未通过验证检查的模型，并展示如何运行模型验证器，以及如何修复模型以使验证检查通过。

示例 1：具有重用模块的模型

我们从以下模型开始，其中包含两个共享同一模块实例的 relu 模块。

class ModelWithReusedNodes(torch.nn.Module):
    """ Model that reuses a relu module. Expects input of shape (1, 3, 32, 32) """

    def __init__(self):
        super(ModelWithReusedNodes, self).__init__()
        self.conv1 = torch.nn.Conv2d(3, 8, kernel_size=2, stride=2, padding=2, bias=False)
        self.bn1 = torch.nn.BatchNorm2d(8)
        self.relu1 = torch.nn.ReLU(inplace=True)
        self.linear = torch.nn.Linear(2592, 10)

    def forward(self, *inputs):
        x = self.conv1(inputs[0])
        x = self.relu1(x)
        x = self.bn1(x)
        x = self.relu1(x)
        x = x.view(x.size(0), -1)
        x = self.linear(x)
        return x

导入模型验证器：

from aimet_torch.model_validator.model_validator import ModelValidator

通过传入模型以及模型输入来在模型上运行模型验证器：

def validate_example_model():

    # Load the model to validate
    model = ModelWithReusedNodes()

    # Output of ModelValidator.validate_model will be True if model is valid, False otherwise
    ModelValidator.validate_model(model, model_input=torch.rand(1, 3, 32, 32))

对于模型上运行的每次验证检查，都会显示记录器打印：

Utils - INFO - Running validator check <function validate_for_reused_modules at 0x7f127685a598>

如果验证检查发现模型有任何问题，日志将包含有关如何解决模型的信息：

Utils - WARNING - The following modules are used more than once in the model: ['relu1']
AIMET features are not designed to work with reused modules. Please redefine your model to use distinct modules for
each instance.

最后，在验证结束时，将记录任何失败的检查：

Utils - INFO - The following validator checks failed:
Utils - INFO -     <function validate_for_reused_modules at 0x7f127685a598>

在这种情况下，validate_for_reused_modules 检查通知 relu1 模块在模型中被多次使用。我们通过为每种用途定义一个单独的 relu 实例来重写模型：

class ModelWithoutReusedNodes(torch.nn.Module):
    """ Model that is fixed to not reuse modules. Expects input of shape (1, 3, 32, 32) """

    def __init__(self):
        super(ModelWithoutReusedNodes, self).__init__()
        self.conv1 = torch.nn.Conv2d(3, 8, kernel_size=2, stride=2, padding=2, bias=False)
        self.bn1 = torch.nn.BatchNorm2d(8)
        self.relu1 = torch.nn.ReLU(inplace=True)
        self.relu2 = torch.nn.ReLU(inplace=True)
        self.linear = torch.nn.Linear(2592, 10)

    def forward(self, *inputs):
        x = self.conv1(inputs[0])
        x = self.relu1(x)
        x = self.bn1(x)
        x = self.relu2(x)
        x = x.view(x.size(0), -1)
        x = self.linear(x)
        return x

现在，重新运行模型验证器后，所有检查都通过了：

Utils - INFO - Running validator check <function validate_for_reused_modules at 0x7ff577373598>
Utils - INFO - Running validator check <function validate_for_missing_modules at 0x7ff5703eff28>
Utils - INFO - All validation checks passed.

示例 2：具有泛函的模型

我们从以下模型开始，该模型在前向传递中使用了一个torch线性功能层：

class ModelWithFunctionalLinear(torch.nn.Module):
    """ Model that uses a torch functional linear layer. Expects input of shape (1, 3, 32, 32) """

    def __init__(self):
        super(ModelWithFunctionalLinear, self).__init__()
        self.conv1 = torch.nn.Conv2d(3, 8, kernel_size=2, stride=2, padding=2, bias=False)
        self.bn1 = torch.nn.BatchNorm2d(8)
        self.relu1 = torch.nn.ReLU(inplace=True)
        self.relu2 = torch.nn.ReLU(inplace=True)

    def forward(self, *inputs):
        x = self.conv1(inputs[0])
        x = self.relu1(x)
        x = self.bn1(x)
        x = self.relu2(x)
        x = x.view(x.size(0), -1)
        x = F.linear(x, torch.randn(10, 2592))
        return x

运行模型验证器显示 validate_for_missing_modules 检查失败：

Utils - INFO - Running validator check <function validate_for_missing_modules at 0x7f9dd9bd90d0>
Utils - WARNING - Ops with missing modules: ['matmul_8']
This can be due to several reasons:
1. There is no mapping for the op in ConnectedGraph.op_type_map. Add a mapping for ConnectedGraph to recognize and
be able to map the op.
2. The op is defined as a functional in the forward function, instead of as a class module. Redefine the op as a
class module if possible. Else, check 3.
3. This op is one that cannot be defined as a class module, but has not been added to ConnectedGraph.functional_ops.
Add to continue.
Utils - INFO - The following validator checks failed:
Utils - INFO -      <function validate_for_missing_modules at 0x7f9dd9bd90d0>

检查已将 matmul_8 识别为缺少 pytorch 模块的操作。在这种情况下，这是由于日志中的原因#2，其中该层已被定义为前向函数中的函数。为了解决这个问题，我们通过将层定义为模块来重写模型。

class ModelWithoutFunctionalLinear(torch.nn.Module):
    """ Model that is fixed to use a linear module instead of functional. Expects input of shape (1, 3, 32, 32) """

    def __init__(self):
        super(ModelWithoutFunctionalLinear, self).__init__()
        self.conv1 = torch.nn.Conv2d(3, 8, kernel_size=2, stride=2, padding=2, bias=False)
        self.bn1 = torch.nn.BatchNorm2d(8)
        self.relu1 = torch.nn.ReLU(inplace=True)
        self.relu2 = torch.nn.ReLU(inplace=True)
        self.linear = torch.nn.Linear(2592, 10)
        with torch.no_grad():
            self.linear.weight = torch.nn.Parameter(torch.randn(10, 2592))

    def forward(self, *inputs):
        x = self.conv1(inputs[0])
        x = self.relu1(x)
        x = self.bn1(x)
        x = self.relu2(x)
        x = x.view(x.size(0), -1)
        x = self.linear(x)
        return x

1.1.5 AIMET PyTorch 定量分析器 API

AIMET PyTorch Quant Analyzer 分析 PyTorch 模型并指出模型中量化的敏感层。它检查模型对权重和激活量化的敏感性，执行每层敏感性和 MSE 分析。它还导出每层编码的最小和最大范围以及每层的统计直方图。

1.1.5.1 顶层API

class aimet_torch.quant_analyzer.QuantAnalyzer(model, dummy_input, forward_pass_callback, eval_callback, modules_to_ignore=None)[source]

QuantAnalyzer 工具提供

模型对权重和激活量化的敏感性
每层灵敏度分析
每层编码（最小 - 最大范围）

参数：

model (Module) – 用于分析量化的 FP32 模型。
dummy_input (Union[Tensor, Tuple]) – 将输入传递给模型。
forward_pass_callback (CallbackFunc) – 用于模型校准的回调函数，只需在模型上运行前向传递即可计算编码（增量/偏移）。此回调函数应使用代表性数据，并且应是整个训练/验证数据集（约 1000 个图像/样本）的子集。
eval_callback (CallbackFunc) – 用于确定模型性能的模型评估的回调函数。此回调函数预计返回标量值，表示针对整个测试/评估数据集评估的模型性能。
modules_to_ignore (Optional[List[Module]]) – 从分析中排除某些模块。

analyze(quant_scheme=<QuantScheme.post_training_tf_enhanced: 2>, default_param_bw=8, default_output_bw=8, config_file=None, results_dir=‘./tmp/’)[source]

分析模型的量化并通过执行来指出模型的敏感部分/热点

模型对量化的敏感性，
通过启用和禁用定量包装器来执行每层敏感性分析，
导出每层编码的最小 - 最大范围，
当量化方案为 TF 增强时导出每层统计直方图 (PDF)，
每层MSE分析

参数：

quant_scheme (QuantScheme) – 量化方案。支持的值为 QuantScheme.post_training_tf 或 QuantScheme.post_training_tf_enhanced。
default_param_bw (int) – 用于量化图层参数的默认位宽 (4-31)。
default_output_bw (int) – 用于量化层输入和输出的默认位宽 (4-31)。
config_file (Optional[str]) – 模型量化器的配置文件的路径。
results_dir (str) – 保存结果的目录。

enable_per_layer_mse_loss(unlabeled_dataset_iterable, num_batches)[source]

启用每层 MSE 损失分析。
参数：

unlabeled_dataset_iterable (Union[DataLoader[+T_co], Collection[+T_co]]) – 迭代未标记数据集的集合（即可使用 len 进行迭代）。该迭代产生的值预计能够直接传递给模型。
num_batches (int) – 批次数。建议使用大约 256 个样本/图像，因此如果数据加载器的批次大小为 64，则 4 个批次将导致 256 个样本/图像。

1.1.5.2 代码示例

所需引入

from typing import Any
import torch
from torchvision import models
from aimet_common.defs import QuantScheme
from aimet_torch.model_preparer import prepare_model
from aimet_torch.quant_analyzer import QuantAnalyzer, CallbackFunc

准备前向传递回调

# NOTE: In the actual use cases, the users should implement this part to serve
#       their own goals if necessary.
def forward_pass_callback(model: torch.nn.Module, _: Any = None) -> None:
    """
    NOTE: This is intended to be the user-defined model calibration function.
    AIMET requires the above signature. So if the user's calibration function does not
    match this signature, please create a simple wrapper around this callback function.

    A callback function for model calibration that simply runs forward passes on the model to
    compute encoding (delta/offset). This callback function should use representative data and should
    be subset of entire train/validation dataset (~1000 images/samples).

    :param model: PyTorch model.
    :param _: Argument(s) of this callback function. Up to the user to determine the type of this parameter.
    E.g. could be simply an integer representing the number of data samples to use. Or could be a tuple of
    parameters or an object representing something more complex.
    """
    # User action required
    # User should create data loader/iterable using representative dataset and simply run
    # forward passes on the model.

准备 eval 回调

# NOTE: In the actual use cases, the users should implement this part to serve
#       their own goals if necessary.
def eval_callback(model: torch.nn.Module, _: Any = None) -> float:
    """
    NOTE: This is intended to be the user-defined model evaluation function.
    AIMET requires the above signature. So if the user's calibration function does not
    match this signature, please create a simple wrapper around this callback function.

    A callback function for model evaluation that determines model performance. This callback function is
    expected to return scalar value representing the model performance evaluated against entire
    test/evaluation dataset.

    :param model: PyTorch model.
    :param _: Argument(s) of this callback function. Up to the user to determine the type of this parameter.
    E.g. could be simply an integer representing the number of data samples to use. Or could be a tuple of
    parameters or an object representing something more complex.
    :return: Scalar value representing the model performance.
    """
    # User action required
    # User should create data loader/iterable using entire test/evaluation dataset, perform forward passes on
    # the model and return single scalar value representing the model performance.
    return .8

准备模型

    model = models.resnet18(pretrained=True).cuda().eval()
    input_shape = (1, 3, 224, 224)
    dummy_input = torch.randn(*input_shape).cuda()
    prepared_model = prepare_model(model)

创建 QuantAnalyzer 对象

    quant_analyzer = QuantAnalyzer(model=prepared_model,
                                   dummy_input=dummy_input,
                                   forward_pass_callback=forward_pass_callback_fn,
                                   eval_callback=eval_callback_fn)
    # Approximately 256 images/samples are recommended for MSE loss analysis. So, if the dataloader
    # has batch_size of 64, then 4 number of batches leads to 256 images/samples.
    quant_analyzer.enable_per_layer_mse_loss(unlabeled_dataset_iterable=unlabeled_data_loader, num_batches=4)

运行定量分析器

    quant_analyzer.analyze(quant_scheme=QuantScheme.post_training_tf_enhanced,
                           default_param_bw=8,
                           default_output_bw=8,
                           config_file=None,
                           results_dir="./quant_analyzer_results/")