高通8775 QNN部署端到端头部位姿估计模型

2025-08-21

这个专栏将介绍如何使用高通的QNN SDK部署一个自己训练的端到端头部位姿估计模型。我使用的平台是高通的汽车级8775处理器，是面向下一代的舱驾融合平台。开发环境是Ubuntu 22.04。由于高通的部署工具链是闭源且门槛费高昂，参考的资料很少，所以本文将以自己的经验和理解为主，介绍如何使用QNN SDK部署一个端到端头部位姿估计模型，模型用于DMS/OMS等专用应用。

1- 环境安装

从2.3.4QNN SDK的手册来看:
使用HTP，是V73版本的架构，Hexagon SDK 版本为5.4.0

qnn_python环境

# 1-创建虚拟python环境, qnn2.3.4版本只支持这个python版本，请注意
conda create -n qnn234_env python==3.10.4
conda activate qnn234_env

cd /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/bin/
# 2-设置一下基础环境
source ./envsetup.sh
# 3- 准备安装相关的依赖
python "${QNN_SDK_ROOT}/bin/check-python-dependency"

这里要注意了，因为是在国内，所以说高通提供的安装脚本非常慢并且基本是失败的。这时候需要修改高通QNN_SDK中环境配置的python脚本${QNN_SDK_ROOT}/bin/check-python-dependency，为其添加清华源，保证快速安装。

#!/usr/bin/env python3
# =============================================================================
#
#  Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries.
#  All rights reserved.
#  Confidential and Proprietary - Qualcomm Technologies, Inc.
#
# =============================================================================

"""

Checks and installs Python dependencies for the QAIRT SDK
Currently supported: Python 3.8 and 3.10

"""

import argparse
import os
import pkg_resources
import re
import subprocess
import sys

from typing import Dict


class QAIRTEnvironmentError(Exception):

    def __str__(self):
        return f"[ERROR] {self.args[0]}"


class SupportedVersion:
    version: str
    overrides: Dict[str, str]

    def __init__(self, ver):
        self.version = ver
        self.overrides = {}


def get_ubuntu_version():
    if os.path.exists("/etc/os-release"):
        with open("/etc/os-release") as f:
            for line in f:
                if line.startswith("VERSION_ID="):
                    return line.strip().split("=")[1].strip('"')
    raise FileNotFoundError("/etc/os-release was not found.")


def normalize_package_name(pkg):
    return re.sub(r"[-_.]+", "-", pkg).lower()


def get_installed_versions(pkg_list, pip_list):
    installed_versions = []
    missing_val = "MISSING"
    missing_packages = []
    for package in pkg_list.keys():
        normalized = normalize_package_name(package)
        if normalized in pip_list:
            installed_versions.append(pip_list[normalized])
        else:
            installed_versions.append(missing_val)
            missing_packages.append(package)
    return installed_versions, missing_packages


def print_package_summary(pkg_list, installed_versions):
    headers = ["Package", "Recommended", "Installed"]
    columns = [pkg_list.keys(), pkg_list.values(), installed_versions]
    widths = [max(len(header), max([len(y) for y in column])) for header, column in zip(headers, columns)]
    print('\t'.join([f'{x.ljust(y)}' for x, y in zip(headers, widths)]))
    print('\t'.join(['~' * x for x in widths]))
    for row in zip(*columns):
        print('\t'.join([f'{x.ljust(y)}' for x, y in zip(row, widths)]))


def get_pkgs_to_install(pkg_list, installed_pkgs):
    """
    Gets list of packages to install. Packages that are already installed are skipped. If version for installed package
    does not match the expected package, warning statement is printed and package is still skipped
    :param pkg_list: list of packages to vet for installation
    :param installed_pkgs: packages already installed in the current environment
    :return: a dictionary of packages to install with pkg_name:pkg_version as key:value
    """
    to_be_installed_pkgs = {}
    for required_package_name in pkg_list.keys():
        if required_package_name in installed_pkgs.keys():
            recommended_package_version = pkg_list[required_package_name]
            installed_package_version = installed_pkgs[required_package_name]
            if recommended_package_version == installed_package_version:
                print(
                    f"{required_package_name} is already installed with tested version: {recommended_package_version}"
                )
            else:
                print(
                    f"WARNING: {required_package_name} installed version: {installed_package_version} "
                    f"does not match tested version: {recommended_package_version}"
                )

        else:
            to_be_installed_pkgs[required_package_name] = pkg_list[
                required_package_name
            ]

    return to_be_installed_pkgs


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("-s", "--skip_optional", required=False, action='store_true',
                        help="skip installing optional dependencies")

    args = parser.parse_args()

    version38 = SupportedVersion("3.8")
    version38.overrides['absl-py'] = '0.13.0'
    version38.overrides['attrs'] = '22.2.0'
    version38.overrides['decorator'] = '5.1.1'
    version38.overrides['invoke'] = '2.0.0'
    version38.overrides['joblib'] = '1.0.1'
    version38.overrides['jsonschema'] = '4.19.0'
    version38.overrides['lxml'] = '4.6.2'
    version38.overrides['mock'] = '5.1.0'
    version38.overrides['numpy'] = '1.23.5'
    version38.overrides['opencv-python'] = '4.5.2.52'
    version38.overrides['optuna'] = '3.3.0'
    version38.overrides['packaging'] = '21.0'
    version38.overrides['pandas'] = '2.0.2'
    version38.overrides['paramiko'] = '3.3.1'
    version38.overrides['pillow'] = '6.2.1'
    version38.overrides['plotly'] = '5.16.1'
    version38.overrides['psutil'] = '5.9.5'
    version38.overrides['pytest'] = '7.0.1'
    version38.overrides['pyyaml'] = '3.10'
    version38.overrides['scipy'] = '1.9.1'
    version38.overrides['tabulate'] = '0.8.5'
    version38.overrides['typing-extensions'] = '4.6.2'
    version38.overrides['xlsxwriter'] = '3.1.9'

    version310 = SupportedVersion("3.10")

    supported_python_versions = {
        version38.version: version38,
        version310.version: version310
    }

    if sys.platform.startswith('win'):
        version310.overrides["matplotlib"] = '3.5.0'
        version310.overrides['pandas'] = '2.0.2'

    current_python_version = f"{sys.version_info[0]}.{sys.version_info[1]}"

    # Verify the currently installed Python version
    try:
        # For Linux, we support specific Python versions depending on the host Ubuntu version
        if sys.platform.startswith('linux'):

            ubuntu_to_python_version = {
                "22.04": version310.version,
                "20.04": version38.version
            }

            ubuntu_version = get_ubuntu_version()
            if ubuntu_version not in ubuntu_to_python_version:
                raise QAIRTEnvironmentError(f"Unsupported Ubuntu version {ubuntu_version}.")

            if current_python_version != ubuntu_to_python_version[ubuntu_version]:
                raise QAIRTEnvironmentError(
                    f"Python {current_python_version} is unsupported on Ubuntu {ubuntu_version}."
                    f" Please use Python {ubuntu_to_python_version[ubuntu_version]} instead.")
        else:
            # Just verify that the Python version is one of the supported versions
            if current_python_version not in supported_python_versions:
                raise QAIRTEnvironmentError(f"Unsupported Python version {current_python_version}. "
                                            f"Please use one of: {', '.join(supported_python_versions.keys())}")

    except QAIRTEnvironmentError as e:
        print(e)
        exit()

    if ((('VIRTUAL_ENV' not in os.environ) and
         ('CONDA_DEFAULT_ENV' not in os.environ))):
        print(
            "Error: Script should only be execute in virtual environment, "
            "please check you had activated the python virtual environment"
        )
        exit()

    required_packages = {
        "safetensors": "0.4.3",
        "absl-py": "2.1.0",
        "aenum": "3.1.15",
        "attrs": "23.2.0",
        "dash": "2.12.1",
        "decorator": "4.4.2",
        "invoke": "1.7.3",
        "joblib": "1.4.0",
        "jsonschema": "4.19.0",
        "lxml": "5.2.1",
        "mako": "1.1.0",
        "matplotlib": "3.3.4",
        "mock": "3.0.5",
        "numpy": "1.26.4",
        "opencv-python": "4.5.4.58",
        "optuna": "3.3.0",
        "packaging": "24.0",
        "pandas": "2.0.1",
        "paramiko": "3.4.0",
        "pathlib2": "2.3.6",
        "pillow": "10.2.0",
        "plotly": "5.20.0",
        "protobuf": "3.19.6",
        "psutil": "5.6.4",
        "pydantic": "2.7.4",
        "pytest": "8.1.1",
        "pyyaml": "5.3",
        "rich": "13.9.4",
        "scikit-optimize": "0.9.0",
        "scipy": "1.10.1",
        "six": "1.16.0",
        "tabulate": "0.9.0",
        "typing-extensions": "4.10.0",
        "xlsxwriter": "1.2.2"
    }

    optional_packages = {
        "qairt-visualizer": "0.2.0"
    }

    for lib, version in supported_python_versions[current_python_version].overrides.items():
        required_packages[lib] = version

    installed_packages = {}
    for installed_package in pkg_resources.working_set:
        installed_package_name = installed_package.key
        if installed_package_name not in required_packages.keys():
            continue
        installed_packages[installed_package.key] = installed_package.version

    required_packages = dict(sorted(required_packages.items()))
    to_be_installed_packages = {}
    to_be_installed_packages.update(get_pkgs_to_install(required_packages, installed_packages))
    if not args.skip_optional:
        optional_packages = dict(sorted(optional_packages.items()))
        to_be_installed_packages.update(get_pkgs_to_install(optional_packages, installed_packages))

    if len(to_be_installed_packages) > 0:
        print(f"Python Modules missing: {', '.join(map(str, to_be_installed_packages.keys()))}")
        subprocess.check_call(
            [
                sys.executable,
                '-m',
                "pip",
                "install",
                "--upgrade",
                "pip"
            ],
            stdout=subprocess.DEVNULL,
        )

        print("Installing missing modules using pip3")

        for pkg_name, pkg_ver in to_be_installed_packages.items():
            print(f"Installing {pkg_name} version: {pkg_ver}")
            try:
            # 注意这里，添加清华源
                subprocess.check_call(
                    [sys.executable, '-m', "pip", "install", f"{pkg_name}=={pkg_ver}", "-i", "https://pypi.tuna.tsinghua.edu.cn/simple"],
                    stdout=subprocess.DEVNULL,
                )
            except subprocess.CalledProcessError:
                print(f"Error: Failed to install {pkg_name}:{pkg_ver}")
                if not args.skip_optional and pkg_name in optional_packages.keys():
                    print(f"To skip installing optional package {pkg_name}, pass --skip_optional.")

    pip_list = [x.split() for x in subprocess.check_output(
        [sys.executable, '-m', 'pip', 'list']).decode('utf-8').split('\n')[2:]]
    pip_list = {normalize_package_name(x[0]): x[1] for x in pip_list if x}

    print("\nSummary:")
    print("Required Packages:\n")
    installed_versions, required_packages_missing = get_installed_versions(required_packages, pip_list)
    print_package_summary(required_packages, installed_versions)

    print("\nOptional Packages:")
    if not args.skip_optional:
        print("To skip installing optional packages, pass --skip_optional\n")
    installed_versions, _ = get_installed_versions(optional_packages, pip_list)
    print_package_summary(optional_packages, installed_versions)

    if required_packages_missing:
        # Only throw runtime error if there are required packages missing
        raise RuntimeError("Missing required package(s): " + ", ".join(required_packages_missing))


if __name__ == "__main__":
    main()

对脚本修改完成后，执行，将会自动安装，如果遇到没办法安装的库，使用pip install xxxx 进行手动安装，我在安装的时候，matplotlib这个就遇到了不成功的问题，直接用的手动安装。脚本执行完成后，还需要安装一些额外的库，根据自己使用的框架和方法来选择就行，我使用的是pytorch和onnx，所以根据文档：

这里需要额外安装：

1
2
3

pip install onnx==1.16.1 -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install onnxruntime==1.18.0 -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install onnxsim== 0.4.36 -i https://pypi.tuna.tsinghua.edu.cn/simple

我训练模型和pth转onnx是在另一个环境上进行的，所以这里是没有安装torch的，我已经有了处理好的onnx模型。记得在导出模型的时候，将模型进行简化。

qnn_Android_NDK

由于我部署的目标系统是8775的安卓平台，所以需要NDK的支持。所以需要在Host开发机器上配置好NDK的环境。

# 下载对应的版本，参见QNN的文档：file:///home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/docs/QNN/general/setup/linux_setup.html
打开链接https://dl.google.com/android/repository/android-ndk-r26c-linux.zip

解压到想要存放的路径

激活环境和预备

# 环境激活
conda activate qnn234_env
export PATH=$PATH:/home/cty/hsae_data/qnn/pkg/android-ndk-r26c-linux/android-ndk-r26c
cd /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/bin/
source ./envsetup.sh
${QNN_SDK_ROOT}/bin/envcheck -a

python "${QNN_SDK_ROOT}/bin/check-python-dependency"

2-模型的转换、量化

非量化模型的步骤

首先将pth模型转化为ONNX模型，记得打开模型简化

1	qnn-onnx-converter -i '/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.onnx' -o '/home/cty/self_project/DMS/6DRepNet/model_file/no_quanti_qnn.cpp'

使用这个指令将模型转化为非量化版本的QNN模型，将会得到no_quanti_qnn.bin以及no_quanti_qnn.cpp还有一个json文件

量化模型的步骤

# 上面的是不量化的指令，如果要简单量化，需要使用额外的参数--input_list

qnn-onnx-converter -i '/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.onnx' \
				   -o '/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.cpp'\
				   --input_list "/home/cty/self_project/DMS/6DRepNet/model_file/quantification.txt"


qnn-onnx-converter \
  --input_network '/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.onnx' \
  --output_path '/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.cpp' \
  --input_list "/home/cty/self_project/DMS/6DRepNet/model_file/quantification.txt" \
  --weights_bitwidth 8 \
  --act_bitwidth 8 \
  --bias_bitwidth 32 \
  --act_quantizer_calibration min-max \
  --param_quantizer_calibration min-max \
  --act_quantizer_schema asymmetric \
  --param_quantizer_schema symmetric \
  --use_per_channel_quantization \
  --use_per_row_quantization

这里着重备忘一下如何制作量化所需要的数据，首先，这些数据是图像数据没错，要核对数据的尺寸，比如说这个网络的输入是3 x 244 x 244的图像，那么我就随机抽取测试集中的500张图像，并下采样到指定的224x224，然后拷贝到一个指定的位置，并将这些图像（一般是JPEG之类的压缩了的），归一化转化为bin文件，然后制作出相应的txt文件。

import os
import random
import cv2
import numpy as np
import shutil

# 图像所在目录
image_dir = '/home/cty/hsae_data/deep_learning_data/DMS/AFLW2000/'
# 目标目录
dest_dir = '/home/cty/self_project/DMS/6DRepNet/model_file/quantification_data/'

# 确保目标目录存在，如果不存在则创建
os.makedirs(dest_dir, exist_ok=True)

# 清空目标目录中的旧文件
for filename in os.listdir(dest_dir):
    file_path = os.path.join(dest_dir, filename)
    try:
        if os.path.isfile(file_path) or os.path.islink(file_path):
            os.unlink(file_path)
        elif os.path.isdir(file_path):
            shutil.rmtree(file_path)
    except Exception as e:
        print(f'Failed to delete {file_path}. Reason: {e}')

# 获取所有以 .jpg 结尾的文件
jpg_files = [os.path.join(image_dir, f) for f in os.listdir(image_dir) if f.endswith('.jpg')]

# 随机抽取 500 张图像
selected_files = random.sample(jpg_files, min(500, len(jpg_files)))

# 复制并下采样图像
for src_file in selected_files:
    # 构建目标文件路径
    dest_file = os.path.join(dest_dir, os.path.splitext(os.path.basename(src_file))[0] + '.bin')

    # 使用OpenCV读取并下采样图像
    img = cv2.imread(src_file)
    if img is not None:
        resized_img = cv2.resize(img, (224, 224))
        # 转换为 float32 并归一化到 [0, 1]
        resized_img = resized_img.astype(np.float32) / 255.0
        # 转换为 NHWC 格式（1, 224, 224, 3）
        resized_img = np.expand_dims(resized_img, axis=0)
        # 保存为 .bin 文件
        resized_img.tofile(dest_file)
    else:
        print(f"无法读取图像: {src_file}")

# 写入 quantification.txt
with open('/home/cty/self_project/DMS/6DRepNet/model_file/quantification.txt', 'w') as f:
    for file in os.listdir(dest_dir):
        if file.endswith('.bin'):
            f.write(os.path.join(dest_dir, file) + '\n')

print(f'已从 {len(jpg_files)} 张图像中随机选取 {len(selected_files)} 张进行下采样和二进制转换，并写入 quantification.txt')

执行之后，会得到量化后的模型，当然，也可以指定一些额外的参数来控制量化模型的精度，效果，大小，通道等等。

2025-07-24 15:23:37,314 - 270 - INFO - Saving QNN Model...
2025-07-24 15:23:37,334 - 270 - INFO - Model CPP saved at: /home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.cpp 
2025-07-24 15:23:37,335 - 270 - INFO - Model BIN saved at: /home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.bin 
2025-07-24 15:23:37,339 - 270 - INFO - Conversion complete!

量化细节

1. 指定量化输入数据

1	--input_list INPUT_LIST

用途: 指定用于量化校准的输入数据文件路径。
说明: 这是一个文本文件，每一行包含一个或多个二进制文件路径，每个文件包含一个输入数据（如图像、张量等）。

示例:

1	--input_list calibration_data.txt

其中 calibration_data.txt 内容如下：

1 2	/path/to/input1.bin /path/to/input2.bin

2. 设置权重和激活的量化位宽

1
2
3

--weights_bitwidth WEIGHTS_BITWIDTH
--act_bitwidth ACT_BITWIDTH
--bias_bitwidth BIAS_BITWIDTH

用途: 分别设置权重、激活值和偏置的量化位宽。
默认值:
- --weights_bitwidth: 8
- --act_bitwidth: 8
- --bias_bitwidth: 32
可选值:
- weights_bitwidth: 4 或 8
- act_bitwidth: 8 或 16
- bias_bitwidth: 8 或 32

示例:

1	--weights_bitwidth 8 --act_bitwidth 8 --bias_bitwidth 32

3. 激活值和权重的量化校准方法

1 2	--act_quantizer_calibration ACT_QUANTIZER_CALIBRATION --param_quantizer_calibration PARAM_QUANTIZER_CALIBRATION

用途: 设置激活值和参数（权重）的量化校准方法。
可选值:
- min-max（默认）
- sqnr
- entropy
- mse
- percentile

示例:

1	--act_quantizer_calibration min-max --param_quantizer_calibration min-max

4. 激活值和权重的量化模式

1 2	--act_quantizer_schema ACT_QUANTIZER_SCHEMA --param_quantizer_schema PARAM_QUANTIZER_SCHEMA

用途: 设置激活值和参数（权重）的量化模式（对称/非对称）。
可选值:
- asymmetric（默认）
- symmetric
- unsignedsymmetric

示例:

1	--act_quantizer_schema asymmetric --param_quantizer_schema symmetric

5. 启用逐通道量化

1	--use_per_channel_quantization

用途: 启用卷积层权重的逐通道量化，提高精度。
示例:
1
--use_per_channel_quantization

6. 启用逐行量化

1	--use_per_row_quantization

用途: 启用全连接层（Matmul）的逐行量化。
示例:
1
--use_per_row_quantization

7. 忽略模型自带的量化信息

1	--ignore_encodings

用途: 忽略模型中自带的量化编码信息，仅使用量化器生成的编码。
注意: 不能与 --quantization_overrides 同时使用。
示例:
1
--ignore_encodings

8. 使用量化覆盖文件

1	--quantization_overrides QUANTIZATION_OVERRIDES

用途: 指定一个 JSON 文件，其中包含自定义的量化参数，覆盖默认的量化行为。

示例:

1	--quantization_overrides quantization_overrides.json

9. 启用浮点回退

1	--float_fallback

用途: 当某些节点无法量化时，回退到浮点计算。
示例:
1
--float_fallback

10. 百分位校准值

1	--percentile_calibration_value PERCENTILE_CALIBRATION_VALUE

用途: 用于 percentile 校准方法的百分位值（90 到 100 之间）。
默认值: 99.99
示例:
1
--percentile_calibration_value 99.9

量化示例

假设你有一个 ONNX 模型 model.onnx，并且你已经准备好了校准数据 calibration_data.txt，你可以使用以下命令进行量化：

qnn-onnx-converter \
  --input_network model.onnx \
  --output_path quantized_model.qnn \
  --input_list calibration_data.txt \
  --weights_bitwidth 8 \
  --act_bitwidth 8 \
  --bias_bitwidth 32 \
  --act_quantizer_calibration min-max \
  --param_quantizer_calibration min-max \
  --act_quantizer_schema asymmetric \
  --param_quantizer_schema symmetric \
  --use_per_channel_quantization \
  --use_per_row_quantization

这个命令会使用 8 位量化权重和激活值，并启用逐通道和逐行量化，确保模型在保持精度的同时具备高效的推理能力。

4-部署至目标设备

为指定的目标设备编译模型so库

这里我使用的是高通的SA8775P汽车芯片平台，系统架构采用的QNX上虚拟出Android，所以是aarch64-android目标平台或者x86_64-linux-clang平台

export QNN_TARGET_ARCH="your-target-architecture-from-above"

export QNN_TARGET_ARCH="aarch64-android"
或者
export QNN_TARGET_ARCH="x86_64-linux-clang"

export QNN_TARGET_ARCH="aarch64-qnx"

我自己的主机是一个x86_64的Linux主机，在qnn的bin下面有qnn-model-lib-generator，执行，其中-c和-b是之前量化模型得到的目标文件，-o是输出的位置folder

[!NOTE] qnn-model-lib-generator
c - This indicates the path to the .cpp QNN model file.
b - This indicates the path to the .bin QNN model file. (b is optional, but at runtime, the .cpp file could fail if it needs the .bin file, so it is recommended).
o - The path to the output folder.
t - Indicate which architecture to build for.

NOTE：在转x86_64-linux-clang库的时候，会出现一些问题
我的电脑有环境问题：
export CPLUS_INCLUDE_PATH=/usr/include/c++/11:/usr/include/x86_64-linux-gnu/c++/11:$CPLUS_INCLUDE_PATH
export LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/11:$LIBRARY_PATH

python3 "${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-model-lib-generator" \
    -c "/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.cpp" \
    -b "/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.bin" \
    -o /home/cty/self_project/DMS/6DRepNet/model_file/ \
    -t ${QNN_TARGET_ARCH}

随后我们就得到了相应的so：libtest_qnn.so，这里编译HTP的so库是以HTP为后端的。

部署在指定的计算后端上

构建的模型和所有必要的文件传输到目标处理器，然后对其运行推理，这里我希望推理后端是DSP/HTP，也就是高通的计算芯片，他们命名很复杂，抛开这些，知道后端是DSP在做计算芯片即可。根据项目的需求，既可以是CPU，也可以是GPU，也可以是DSP
DSP 处理器需要量化模型而不是全精度模型。如果您没有量化模型，请按照 CNN 到 QNN 教程的第 2 步构建一个。
接着，要去找一下芯片对应支持的Hexagon架构和工具链

这里看起来大部分是手机的，对于汽车的平台，应该参照对应的SDK的手册。这个在一开始的环境配置里有写

部署在GPU上进行推理(QNX)

1-使用非量化版本的so

注意，这个地方需要QNX710系统的环境，一个是授权文件.qnx，一个是QNX环境

1	source /home/cty/hsae_data/qnx/qnx_710_envs/qnx_710/qnxsdp-env.sh

在之前我们使用量化版本的模型得到了so，现在要使用非量化的模型来，我命名为no_quanti_qnn

export QNN_TARGET_ARCH="aarch64-qnx"
# 选择对应的平台

qnn-onnx-converter -i '/home/cty/self_project/DMS/6DRepNet/model_file/test_qnn.onnx' -o '/home/cty/self_project/DMS/6DRepNet/model_file/no_quanti_qnn.cpp'

python3 "${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-model-lib-generator" \
    -c "/home/cty/self_project/DMS/6DRepNet/model_file/no_quanti_qnn.cpp" \
    -b "/home/cty/self_project/DMS/6DRepNet/model_file/no_quanti_qnn.bin" \
    -o /home/cty/self_project/DMS/6DRepNet/model_file/ \
    -t ${QNN_TARGET_ARCH}

于是得到了相关的libno_quanti_qnn.so

2-推送到目标设备上

adb devices

adb root && adb remount && adb shell
cd sdcard 
mkdir -p qnn_no_quanti_qnx_GPU
exit

# 如果是往QNX端推,要先推到Android这边，然后再推到QNX
# 我这里把相关的文件都放在Android端的：/sdcard/qnn_no_quanti_qnx_GPU
# 往Android传：
# 1- 模型文件本体和模型so文件
adb push /home/cty/self_project/DMS/6DRepNet/model_file/aarch64-android/libno_quanti_qnn.so /sdcard/qnn_no_qunti_qnx_GPU

# 2- 模型的执行部件和依赖库
adb push /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/bin/aarch64-qnx/qnn-net-run /sdcard/qnn_no_quanti_qnx_GPU
adb push /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/lib/aarch64-qnx/libQnnGpu.so /sdcard/qnn_no_quanti_qnx_GPU

# 3- 测试数据bin图像文件和对应的文件名字，注意txt里图像bin文件的路径要是对应QNX的路径
adb push /home/cty/self_project/DMS/6DRepNet/model_file/quantification_data/image00004.bin /sdcard/qnn_no_quanti_qnx_GPU
adb push /home/cty/self_project/DMS/6DRepNet/model_file/qnn_noquanti_data_GPU.txt /sdcard/qnn_no_quanti_qnx_GPU

# 4- 从Android往QNX传 第一个参数是QNX目标，第二个是Android目标
# 记得提前在QNX中创建好目录：mkdir -p /data/qnn_no_quanti_qnx_GPU
# 传输模型so和序列化bin
adb shell busybox ftpput 192.168.10.2 -u root /data/qnn_no_quanti_qnx_GPU/libno_quanti_qnn.so /sdcard/qnn_no_quanti_qnx_GPU/libno_quanti_qnn.so
# 传输依赖SDK
adb shell busybox ftpput 192.168.10.2 -u root /data/qnn_no_quanti_qnx_GPU/libQnnGpu.so /sdcard/qnn_no_quanti_qnx_GPU/libQnnGpu.so
adb shell busybox ftpput 192.168.10.2 -u root /data/qnn_no_quanti_qnx_GPU/qnn-net-run /sdcard/qnn_no_quanti_qnx_GPU/qnn-net-run

# 传输测试所使用的图像文件bin和txt
adb shell busybox ftpput 192.168.10.2 -u root /data/qnn_no_quanti_qnx_GPU/image00004.bin /sdcard/qnn_no_quanti_qnx_GPU/image00004.bin
adb shell busybox ftpput 192.168.10.2 -u root /data/qnn_no_quanti_qnx_GPU/qnn_noquanti_data_GPU.txt /sdcard/ty_test_qnn/qnn_noquanti_data_GPU.txt

chmod 777 /data/qnn_no_quanti_qnx_GPU/qnn-net-run

# 执行模型推理
./qnn-net-run \
   --model "./libno_quanti_qnn.so" \
   --input_list "./qnn_noquanti_data_GPU.txt" \
   --backend "./libQnnGpu.so" \
   --output "./output"
   
./qnn-net-run --model ./libno_quanti_qnn.so --input_list ./qnn_noquanti_data_GPU.txt  --backend ./libQnnGpu.so

3- 查看运行结果并可视化

运行完成后，会产生一个output文件夹，由于我只测试了一张图像，txt里只有一张图像，要看结果只能把文件从QNX里导出来到安卓再导出来到上位机

# 在安卓端运行:
mkdir -p /sdcard/qnn_no_quanti_qnx_GPU/results

busybox ftpget 192.168.10.2 -u root /sdcard/qnn_no_quanti_qnx_GPU/results/execution_metadata.yaml /data/qnn_no_quanti_qnx_GPU/output/execution_metadata.yaml 

busybox ftpget 192.168.10.2 -u root /sdcard/qnn_no_quanti_qnx_GPU/results/rotation_matrix.raw /data/qnn_no_quanti_qnx_GPU/output/Result_0/rotation_matrix.raw

# 在上位机运行
adb pull /sdcard/qnn_no_quanti_qnx_GPU/results /home/cty/self_project/DMS/6DRepNet/qnn_deploy/qnn_qnx_results_GPU

我这里使用的模型是一个很小的回归模型，通过写一个py脚本来解析raw的数据，并对其可视化，并使用上位机的推理结果来验证，可以看到以下的结果：

可以看到，QNN的执行结果和原始模型的推理结果还是存在一定的差距，但基本正确，这和很多因素有关系。

部署在HTP上进行推理(QNX)

1-序列化模型（HTP）

"$QNN_SDK_ROOT/bin/${QNN_TARGET_ARCH}/qnn-context-binary-generator" \
    --backend "${QNN_SDK_ROOT}/lib/${QNN_TARGET_ARCH}/libQnnHtp.so" \
    --model "/home/cty/self_project/DMS/6DRepNet/model_file/x86_64-linux-clang/libtest_qnn.so" \
    --binary_file "/home/cty/self_project/DMS/6DRepNet/model_file/x86_64-linux-clang/test_qnn.serialized"

将会得到类似以下的输出

# 输出如下：
qnn-context-binary-generator pid:127294
Starting stage: Graph Preparation Initializing
Completed stage: Graph Preparation Initializing (3209 us)
Starting stage: Graph Transformations and Optimizations
Completed stage: Graph Transformations and Optimizations (126726 us)
Starting stage: Graph Sequencing for Target
Completed stage: Graph Sequencing for Target (12516 us)
Starting stage: VTCM Allocation
Completed stage: VTCM Allocation (2280 us)
Starting stage: Parallelization Optimization
Completed stage: Parallelization Optimization (1974 us)
Starting stage: Finalizing Graph Sequence

====== DDR bandwidth summary ======
spill_bytes=0
fill_bytes=0
write_total_bytes=2048
read_total_bytes=2877440

Completed stage: Finalizing Graph Sequence (1442 us)
Starting stage: Completion
Completed stage: Completion (104 us)

2-推送相关的文件到目标设备上(QNX)

8775P的HTP(DSP)对应的Hexagon架构是V73，

+---------+----------------------------+--------------------+----------------+
   |   SoC   |   Verified Using Hexagon   | Architecture, file |  JSON Config   |
   |         | SDK version | Tool version |  directory to use  | "soc_id" value |
   +---------+-------------+--------------+--------------------+----------------+
   | SA8650, |    5.4.0    |    8.6.02    |        v73         |        52      |
   | SA8775  |             |              |                    |                |
   +---------+-------------+--------------+--------------------+----------------+

接着在目标设备上创建相应的一个目录用来放模型和库

adb devices

adb root && adb remount && adb shell

cd sdcard 
mkdir -p qnn_tmp
cd qnn_tmp

# 开个新窗口，用来传数据

#1-把模型so传过去
adb push /home/cty/self_project/DMS/6DRepNet/model_file/aarch64-android/libtest_qnn.so /sdcard/qnn_tmp


# 如果是往QNX端推,要先推到Android这边，然后再推到QNX
# 我这里把相关的文件都放在Android端的：/sdcard/ty_test_qnn
# 往Android传：
# 1- 模型文件本体和模型so文件
adb push /home/cty/self_project/DMS/6DRepNet/model_file/x86_64-linux-clang/libtest_qnn.so /sdcard/ty_test_qnn
adb push /home/cty/self_project/DMS/6DRepNet/model_file/x86_64-linux-clang/test_qnn.serialized.bin /sdcard/ty_test_qnn

# 2- 对应的SDK库和可执行文件
adb push /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/lib/aarch64-qnx/libQnnHtp.so /sdcard/ty_test_qnn

adb push /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/lib/aarch64-qnx/libQnnHtpV73Stub.so /sdcard/ty_test_qnn

adb push /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/bin/aarch64-qnx/qnn-net-run /sdcard/ty_test_qnn

adb push /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/bin/aarch64-qnx/qnn-profile-viewer /sdcard/ty_test_qnn

# 3- 测试数据bin图像文件和对应的文件名字，注意txt里图像bin文件的路径要是对应QNX的路径
adb push /home/cty/self_project/DMS/6DRepNet/model_file/quantification_data/image00004.bin /sdcard/ty_test_qnn

adb push /home/cty/self_project/DMS/6DRepNet/model_file/qnn_quantification_data.txt /sdcard/ty_test_qnn

# 4- 传输DSP库
adb push /home/cty/hsae_data/qnn/qnn_sdk_v2.34.0_auto_qnx/qaisw-v2.34.0.250424201103_119471-auto-qnx/lib/hexagon-v73/unsigned/libQnnHtpV73Skel.so /sdcard/ty_test_qnn


# 从Android往QNX传 第一个参数是QNX目标，第二个是Android目标
# 传输模型so和序列化bin
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/libtest_qnn.so /sdcard/ty_test_qnn/libtest_qnn.so
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/test_qnn.serialized.bin /sdcard/ty_test_qnn/test_qnn.serialized.bin
# 传输依赖SDK
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/libQnnHtp.so /sdcard/ty_test_qnn/libQnnHtp.so 
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/libQnnHtpV73Stub.so /sdcard/ty_test_qnn/libQnnHtpV73Stub.so
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/qnn-net-run /sdcard/ty_test_qnn/qnn-net-run
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/qnn-profile-viewer /sdcard/ty_test_qnn/qnn-profile-viewer
# 传输DSP库
adb shell busybox ftpput 192.168.10.2 -u root /mnt/etc/images/dsp/libQnnHtpV73Skel.so /sdcard/ty_test_qnn/libQnnHtpV73Skel.so
# 传输测试所使用的图像文件bin和txt
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/image00004.bin /sdcard/ty_test_qnn/image00004.bin
adb shell busybox ftpput 192.168.10.2 -u root /data/ty_test_qnn/qnn_quantification_data.txt /sdcard/ty_test_qnn/qnn_quantification_data.txt

chmod 777 /data/ty_test_qnn/qnn-net-run
chmod 777 /data/ty_test_qnn/qnn-profile-viewer
然后重启设备


# 从busybox进入QNX命令行环境 执行模型
cd /data/ty_test_qnn
export PATH=/data/ty_test_qnn:$PATH
export VENDOR_LIB=/data/ty_test_qnn
export HWINFO_LIB=/mnt/lib64/dll
export LD_LIBRARY_PATH=$HWINFO_LIB:$VENDOR_LIB:$LD_LIBRARY_PATH

# 根据设备类型设置 ADSP_LIBRARY_PATH
# 有 /dspfw 和 /dsplib 我的8775板子是有的
export ADSP_LIBRARY_PATH="/mnt/etc/images/dsp;$VENDOR_LIB"

# 无 /dspfw 和 /dsplib
export ADSP_LIBRARY_PATH="/mnt/etc/images/cdsp0;$VENDOR_LIB" # CDSP0
# 或
export ADSP_LIBRARY_PATH="/mnt/etc/images/cdsp1;$VENDOR_LIB" # CDSP1

# 禁用VTCM分区（根据你的设备选择cdsp0或cdsp1）：
echo 0 > /dev/pdbg/fastrpc/vtcm/cdsp0

# 运行模型
./qnn-net-run \
  --backend libQnnHtp.so \
  --input_list qnn_quantification_data.txt \
  --retrieve_context test_qnn.serialized.bin

3-查看运行结果并可视化

## 运行完成后，会产生一个output文件夹，由于我只测试了一张图像，txt里只有一张图像，要看结果只能把文件从QNX里导出来到安卓再导出来到上位机
busybox ftpget 192.168.10.2 -u root /sdcard/ty_test_qnn/results/execution_metadata.yaml /data/ty_test_qnn/output/execution_metadata.yaml


busybox ftpget 192.168.10.2 -u root /sdcard/ty_test_qnn/results/rotation_matrix.raw /data/ty_test_qnn/output/Result_0/rotation_matrix.raw

## 导出到上位机
adb pull /sdcard/ty_test_qnn/results /home/cty/self_project/DMS/6DRepNet/qnn_deploy/qnn_qnx_results_HTP

可视化方法和之前的一样，只不过是解析量化后的数据

QNN量化后的结果完全是错误的，量化损失的精度太多，基本让模型失效了，这个问题可以从后续分析，针对某些导致精度下降的层进行修改和修复。