快速体验Jetson Orin部署YOLOv11

一、参考资料Jetson Orin部署YOLOv11推理速度提升3倍的完整指南 - 知乎二、准备工作1. 硬件选型参考型号算力(TOPS)内存功耗价格适用场景Orin Nano 4GB204GB7~15W~$250单路摄像头、简单检测Orin Nano 8GB408GB7~15W~$350多路摄像头、中等模型Orin NX 8GB708GB10~25W~$600复杂检测、2~4路并行Orin NX 16GB10016GB10~25W~$900大模型、多路并行Orin AGX 32GB20032GB15~50W~$2000高端场景、多模型推荐工业单路检测选Orin Nano 8GB性价比最高。2. 系统安装刷JetPack# 在主机上安装 NVIDIA SDK Manager# 下载地址https://developer.nvidia.com/sdk-manager# 连接Orin进入recovery模式按住Recovery按钮按一下Reset# 打开SDK Manager选择# - Target Hardware: Jetson Orin Nano# - OS: JetPack 6.0 (基于Ubuntu 22.04 CUDA 12.2)# - 勾选: Jetson Runtime, CUDA, cuDNN, TensorRT, OpenCV# 等待刷机完成约20~30分钟刷机完成后首次启动Orin进入Ubuntu系统# 检查环境cat/etc/nv_tegra_release# 应该显示: R36 (release), REVISION: 3.1, GCID: 35697395nvidia-smi# 显示Jetson GPU信息nvcc--version# 显示CUDA 12.2系统优化# 开启MAXN模式最大性能模式sudonvpmodel-m0# 开启所有CPU核心满频sudojetson_clocks# 关闭图形界面节省约1GB内存和10W功耗sudosystemctl set-default multi-user.targetsudoreboot# 创建swap4GB内存容易不够用sudofallocate-l8G /var/swapfilesudochmod600/var/swapfilesudomkswap/var/swapfilesudoswapon/var/swapfileecho/var/swapfile none swap sw 0 0|sudotee-a/etc/fstab3. 安装Python环境Jetson上不能用pip直接装PyTorch需要用NVIDIA提供的预编译版本# 安装系统依赖sudoapt-getupdatesudoapt-getinstall-ypython3-pip libopenblas-base libopenmpi-dev libomp-dev# 创建虚拟环境python3-mvenv ~/yolo11_envsource~/yolo11_env/bin/activate# 安装PyTorchJetson专用版本pipinstall--no-cache https://developer.download.nvidia.com/compute/redist/jp/v61/pytorch/torch-2.3.0-cp310-cp310-linux_aarch64.whl# 安装torchvision需要从源码编译sudoapt-getinstall-ylibjpeg-dev zlib1g-dev libpython3-dev libavcodec-dev libavformat-dev libswscale-dev pipinstalltorchvision0.18.0# 安装Ultralyticspipinstallultralytics8.3.0# 验证python3-cimport torch; print(fPyTorch: {torch.__version__}); print(fCUDA: {torch.cuda.is_available()})# 输出: PyTorch: 2.3.0, CUDA: True三、Jetson Orin部署YOLOv111. 模型转换PyTorch → TensorRT1.1 导出ONNX# export_onnx.pyfrom ultralyticsimportYOLO modelYOLO(yolo11s.pt)# 先下载好权重# 导出ONNXmodel.export(formatonnx,imgsz640,opset17,simplifyTrue,dynamicFalse,# 固定输入尺寸TensorRT优化更好halfFalse# 先导出FP32的ONNX)print(导出完成yolo11s.onnx)1.2 转TensorRT引擎方式一用trtexec命令行转换推荐更稳定/usr/src/tensorrt/bin/trtexec\--onnxyolo11s.onnx\--saveEngineyolo11s.engine\--fp16\--workspace4096\--minShapesimages:1x3x640x640\--optShapesimages:1x3x640x640\--maxShapesimages:1x3x640x640# 方式二用Python API转换# pip install tensorrtpython3 export_tensorrt.py方式二用Python API转换pipinstalltensorrt python3 export_tensorrt.py# export_tensorrt.pyimporttensorrtastrtimportosdefbuild_engine(onnx_path,engine_path,fp16True):TRT_LOGGERtrt.Logger(trt.Logger.WARNING)buildertrt.Builder(TRT_LOGGER)configbuilder.create_builder_config()# 设置工作空间Jetson 8GB建议设2~4GBconfig.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE,430)iffp16:config.set_flag(trt.BuilderFlag.FP16)# 显存优化config.set_flag(trt.BuilderFlag.STRICT_TYPES)# 读取ONNXnetworkbuilder.create_network(1int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))parsertrt.OnnxParser(network,TRT_LOGGER)withopen(onnx_path,rb)asf:ifnotparser.parse(f.read()):forerrorinrange(parser.num_errors):print(f解析错误:{parser.get_error(error)})returnNone# 构建引擎print(正在构建TensorRT引擎可能需要5~15分钟...)engine_bytesbuilder.build_serialized_network(network,config)withopen(engine_path,wb)asf:f.write(engine_bytes)print(f引擎已保存到:{engine_path})# 显示引擎信息enginetrt.Runtime(TRT_LOGGER).deserialize_cuda_engine(engine_bytes)print(f输入绑定:{engine.get_binding_shape(0)})print(fFP16模式:{fp16})build_engine(yolo11s.onnx,yolo11s_fp16.engine,fp16True)1.3 用Ultralytics直接导出最简单# Ultralytics内置了TensorRT导出功能最推荐的方式yoloexportmodelyolo11s.ptformatengineimgsz640halfTruedevice0# 直接在Jetson上运行yolo predictmodelyolo11s.enginesourcecamera://02. 推理脚本2.1 高效推理TensorRT引擎# infer_jetson.pyimportcv2importtimeimportnumpyasnpfromultralyticsimportYOLOclassJetsonDetector: Jetson Orin上运行YOLOv11 TensorRT引擎 专为边缘设备优化 def__init__(self,engine_path,conf0.3,iou0.45):self.modelYOLO(engine_path,taskdetect)self.confconf self.iouiou# 预热首次推理会编译着色器需要预热dummynp.zeros((640,640,3),dtypenp.uint8)_self.model(dummy,verboseFalse)print(预热完成)defdetect(self,image,drawTrue):检测单张图片resultsself.model(image,confself.conf,iouself.iou,verboseFalse)detections[]annotatedimage.copy()forresultinresults:forboxinresult.boxes:cls_idint(box.cls[0])cls_nameself.model.names[cls_id]conf_scorefloat(box.conf[0])x1,y1,x2,y2map(int,box.xyxy[0])detections.append({class:cls_name,confidence:conf_score,bbox:[x1,y1,x2,y2]})ifdraw:cv2.rectangle(annotated,(x1,y1),(x2,y2),(0,255,0),2)cv2.putText(annotated,f{cls_name}{conf_score:.2f},(x1,y1-5),cv2.FONT_HERSHEY_SIMPLEX,0.5,(0,255,0),1)returndetections,annotateddefbenchmark(self,num_frames300):性能测试dummynp.random.randint(0,255,(640,480,3),dtypenp.uint8)# 预热for_inrange(30):self.model(dummy,verboseFalse)# 正式测试times[]for_inrange(num_frames):starttime.perf_counter()_self.model(dummy,verboseFalse)times.append((time.perf_counter()-start)*1000)timesnp.array(times)print(f Jetson 性能测试 ({num_frames}帧) )print(f平均延迟:{times.mean():.1f}ms)print(fP50延迟:{np.percentile(times,50):.1f}ms)print(fP99延迟:{np.percentile(times,99):.1f}ms)print(fFPS:{1000/times.mean():.1f})print(fGPU显存: 已使用TensorRT引擎显存占用见nvidia-smi)# 使用示例 if__name____main__:detectorJetsonDetector(yolo11s_fp16.engine,conf0.3)# 性能测试detector.benchmark(num_frames500)# 摄像头检测capcv2.VideoCapture(0)cap.set(cv2.CAP_PROP_FRAME_WIDTH,640)cap.set(cv2.CAP_PROP_FRAME_HEIGHT,480)whileTrue:ret,framecap.read()ifnotret:breakdets,annotateddetector.detect(frame)cv2.imshow(Jetson YOLOv11,annotated)ifcv2.waitKey(1)0xFFord(q):breakcap.release()cv2.destroyAllWindows()2.2 DeepStream部署专业方案DeepStream是NVIDIA官方的视频分析SDK能在Jetson上实现多路并行。安装DeepStream# 安装DeepStreamsudoapt-getinstall-ydeepstream-7.0创建配置文件# deepstream_config.txt简化版[application]enable-perf-measurement1perf-measurement-interval-sec5[stream0]urifile:///path/to/test_video.mp4[primary-gie]unique-id1engine-typenvinfer config-filedetect_config.txt运行DeepStreamdeepstream-app-cdeepstream_app_config.txtNoteDeepStream配置比较复杂入门建议先用Ultralytics推理熟练后再迁移到DeepStream。3. 速度优化对比在Jetson Orin Nano 8GB上的实测数据方案FPS延迟(ms)功耗(W)说明PyTorch FP32128315最慢不推荐PyTorch FP16185515半精度提升约50%ONNX Runtime FP16224512比PyTorch快一倍TensorRT FP16382610推荐方案提速3倍TensorRT INT8521910最快精度损失约1%TensorRT FP16 480p65158降低输入分辨率提速结论TensorRT FP16是最佳平衡点速度提升3倍精度几乎无损。4. 部署中的三个关键细节4.1 性能模式要手动开启Jetson出厂默认不是满血状态需要手动切换# 开启MAXN模式最大性能模式sudonvpmodel-m0# 开启所有CPU核心满频sudojetson_clocks4.2 视频解码要用硬件加速边缘设备通常接入USB摄像头或RTSP视频流。我们使用GStreamer管道做硬件解码把视频处理从CPU卸载到GPU避免CPU成为瓶颈。4.3 推理和显示要异步并行为了保证实时性三个线程各司其职线程1采集持续抓帧 → 放入队列线程2推理从队列取帧 → YOLO推理 → 输出结果线程3业务读取结果 → 画框显示 → 判断是否报警三个线程并行工作互不阻塞。四、FAQQ1TensorRT构建引擎时OOM方法一减小workspace大小。/usr/src/tensorrt/bin/trtexec--onnxyolo11s.onnx--saveEngineyolo11s.engine--fp16--workspace2048方法二或者使用更小的模型yoloexportmodelyolo11n.ptformatengineimgsz640halfTrueQ2推理速度不稳定时快时慢解决方法# 确保开启了MAXN模式和jetson_clockssudonvpmodel-m0sudojetson_clocks# 检查CPU调频cat/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor# 全部应该是performanceQ3USB摄像头花屏错误原因Jetson的USB3.0可能不稳定。解决方法修改/etc/extlinux.conf。sudovim/etc/extlinux.conf# 在APPEND行末尾添加:usbcore.autosuspend-1# 重启sudoreboot