CANN/asc-devkit Fixpipe性能测试

Fixpipe (L0C Egress) Performance Test Example【免费下载链接】asc-devkit本项目是CANN 推出的昇腾AI处理器专用的算子程序开发语言原生支持C和C标准规范主要由类库和语言扩展层构成提供多层级API满足多维场景算子开发诉求。项目地址: https://gitcode.com/cann/asc-devkitOverviewThis example tests the performance of the matrix computation egress path. It covers the data path that moves Cube computation results from the L0C Buffer to the L1 Buffer or Unified Buffer (UB).This is a non-functional performance test. It does not verify computation results and only collects the Fixpipe egress latency.Supported Products and CANN VersionsProductArchitecture CodeCANN VersionAscend 950PR/Ascend 950DTdav-3510 CANN 9.1.0Atlas A3 Training/Inference Seriesdav-2201 CANN 9.0.0Atlas A2 Training/Inference Seriesdav-2201 CANN 9.0.0Directory Structure├── fixpipe_perf │ ├── CMakeLists.txt // Build configuration file │ ├── fixpipe_perf.asc // Fixpipe egress performance test implementation and entry point │ ├── perf.sh // Performance test script │ ├── generate_roofline.py // Roofline generation script │ ├── README.md // Example documentationExample DescriptionThis example uses the runtime parameterSCENARIO_NUMto select different egress paths and data types. Matrix dimensions are passed at runtime through./demo SCENARIO_NUM M K N.The two egress paths use different interfaces:PathInterfaceHeader PathSupported ArchitectureL0C Buffer to L1 BufferDataCopybasic_api/kernel_operator_data_copy_intf.hdav-2201, dav-3510L0C Buffer to UBFixpipebasic_api/kernel_operator_fixpipe_intf.hdav-3510 onlyThe supported test scenarios vary by platform architecture:Atlas A3/A2 Training/Inference Platform ScenariosSCENARIO_NUMInput Data TypeData SourceExecution PathDescriptionTheoretical Bandwidth (Byte/cycle)Bandwidth Latency (cycle)1floatL0C BufferL0C Buffer - L1 BufferDataCopy egress with inline F322F16 conversion to half128202floatL0C BufferL0C Buffer - L1 BufferDataCopy egress with inline QF322B8_PRE quantization to int8_t6420Ascend 950PR/950DT Platform ScenariosSCENARIO_NUMInput Data TypeData SourceExecution PathDescriptionTheoretical Bandwidth (Byte/cycle)Bandwidth Latency (cycle)11floatL0C BufferL0C Buffer - L1 BufferDataCopy egress with inline F322F16 conversion to half1282612floatL0C BufferL0C Buffer - L1 BufferDataCopy egress with inline QF322B8_PRE quantization to int8_t642613floatL0C BufferL0C Buffer - UBFixpipe egress for float, non-dual-target mode1282614floatL0C BufferL0C Buffer - UBFixpipe egress for float, dual-target mode split along M dimension25626Theoretical bandwidth is calculated based on hardware egress parallelism: the egress unit processes 64 output elements per cycle, so theoretical bandwidth 64 × sizeof(destination data type) (Byte/cycle). Dual-target mode splits work across two sub-cores in parallel, effectively doubling the parallelism. The L0C Buffer to UB path is supported only on dav-3510 (scenarios 13 and 14). Scenario 14 splits the M×N matrix in the L0C Buffer along the M dimension into two halves and writes them simultaneously to the UB of two Vector cores.Build and RunRun the following steps from the root directory of this example to build and run it.Configure Environment VariablesConfigure environment variables based on the installation method of the CANN development kit on your system.source ${install_path}/cann/set_env.sh${install_path}is the CANN package installation directory. If no installation directory is specified, the default installation path is/usr/local/Ascend.Build the ExampleBuild for Atlas A3/A2 Training/Inference Platform (dav-2201):mkdir -p build cd build cmake -DCMAKE_ASC_ARCHITECTURESdav-2201 .. make -j cd ..Build for Ascend 950PR/950DT Platform (dav-3510):mkdir -p build cd build cmake -DCMAKE_ASC_ARCHITECTURESdav-3510 .. make -j cd ..Run the ExampleThe runtime parameter order isSCENARIO_NUM M K N:# Atlas A3/A2 Training/Inference Platform examples (scenarios 1, 2) ./build/demo 1 128 64 128 ./build/demo 2 128 64 128 # Ascend 950PR/950DT Platform examples (scenarios 11-14) ./build/demo 11 128 64 128 ./build/demo 12 128 64 128 ./build/demo 13 128 64 128 ./build/demo 14 128 64 128ParameterDescriptionSCENARIO_NUMTest scenario number. Use 1, 2 for Atlas A3/A2 Training/Inference Platform; use 11-14 for Ascend 950PR/950DT PlatformMNumber of matrix rowsKNumber of columns in matrix A (number of rows in matrix B)NNumber of matrix columnsMatrix dimensions must meet alignment requirements: M and N must be multiples of 16. For dual-target mode split along the M dimension, M must be a multiple of 2. For split along the N dimension, N must be a multiple of 32.Collecting Performance DataUse themsproftool to collect detailed performance data:msprof op build/demo 1 128 64 128 Themsproftool requires CANN Commercial or Community Edition. For details, refer to the msprof Tool Installation Guide.After the command completes, a folder namedOPPROF_{timestamp}_XXXis generated in the default directory. The performance data folder structure is as follows:├── dump # Raw performance data ├── ArithmeticUtilization.csv # Cube/vector instruction cycle ratio ├── L2Cache.csv # L2 Cache hit rate, which affects MTE2. Plan data movement logic carefully to increase hit rate ├── Memory.csv # UB, L1 Buffer, and main memory read/write bandwidth ├── MemoryL0.csv # L0A Buffer, L0B Buffer, and L0C Buffer read/write bandwidth ├── MemoryUB.csv # Vector and Scalar to UB read/write bandwidth ├── OpBasicInfo.csv # Operator basic information ├── PipeUtilization.csv # Computation unit and transfer unit latency and ratio ├── ResourceConflictRatio.csv # UB bank group, bank conflict, and resource conflict ratio └── visualize_data.bin # MindStudio Insight presentation fileThis example focuses on L0C Buffer egress performance data. View the specific performance data results as follows:cat ./OPPROF_*/PipeUtilization.csvKey metrics to monitor:MetricDescriptionaic_fixpipe_time(us)Latency of fixpipe-type instructions (L0C Buffer egress)Performance Test Scriptperf.shperforms batch building, runsmsprof op, extracts Fixpipe egress latency, and generates a CSV summary.# View help ./perf.sh --help # Test scenario 1, using dav-2201 by default ./perf.sh 1 # Test scenario 13, using dav-3510 by default ./perf.sh 13 # Explicitly specify the platform. The platform must match the scenario; otherwise, an error is reported ./perf.sh 1 dav-2201 ./perf.sh 13 dav-3510The script uses a built-in default shape sequence. Egress performance depends only on M and N. K is fixed at 64 to allow Mmad to produce L0C Buffer data beforehand. The sequence starts from a small M·N value and gradually increases to full capacity. The L0C Buffer size is 128 KB for dav-2201 and 256 KB for dav-3510. The saturation points differ, so the last entry differs between the two architectures:dav-2201 default shape sequence:Test_IDMKNM·NL0C Buffer Usage (float)11664162561 KB232643210244 KB3646464409616 KB4128641281638464 KB51286425632768128 KB (dav-2201 saturation)dav-3510 default shape sequence:Test_IDMKNM·NL0C Buffer Usage (float)11664162561 KB232643210244 KB3646464409616 KB4128641281638464 KB52566425665536256 KB (dav-3510 saturation)The L0C Buffer egress data volume is M × N × sizeof(destination type). K is used only to allow Mmad to produce the M×N result in the L0C Buffer beforehand and is not counted in the egress volume. Adjust K as needed.To test specific shapes, run./build/demo SCENARIO_NUM M K Ndirectly.After testing, results are saved toperf_data_${timestamp}_scenario${SCENARIO}/perf_result_scenario${SCENARIO}.csv. Rawmsprofdata is saved in thetest_${id}_${M}_${K}_${N}subdirectory under the same directory.Performance Metricsperf.shextractsaic_fixpipe_time(us)fromPipeUtilization.csvand calculates bandwidth based on the platform clock frequency and egress volume.The computed columns in the CSV are as follows:ColumnCalculationDescriptionAIC_FixPipe_Time(us)Extracted fromaic_fixpipe_time(us)inPipeUtilization.csvFixpipe egress latencyCycleTime(us) × Frequency(MHz)Cycle count converted based on platform clock frequencyBandwidth(GB/s)DataSize(bytes) / Time(us) / 1e3Data transfer bandwidthPerformance Metric Calculation MethodsTheaic_fixpipe_time(us)collected bymsprofinPipeUtilization.csvis the egress latency in microseconds.perf.shreads this time column and calculates the cycle count and measured bandwidth based on the platform clock frequency and egress data volume.Converting Time to CyclesThe clock frequency unit is MHz, which represents cycles per microsecond. No additional conversion is needed:Cycle Time(us) × Frequency(MHz)For example, the Atlas A3/A2 Training/Inference Platform has a clock frequency of 1800 MHz. Ifaic_fixpipe_time(us) 0.050000:Cycle 0.050000 × 1800 90.00 cyclesData Transfer Volume Calculationperf.shcalculates the egress volume based on the destination data type for each scenario. This value is used asDataSize(bytes)for bandwidth calculation:ScenarioData Volume CalculationData Type Size1, 11M × N × sizeof(half)2 bytes2, 12M × N × sizeof(int8_t)1 byte13, 14M × N × sizeof(float)4 bytesMeasured Bandwidth CalculationBandwidth is output in GB/s. SinceTime(us)is in microseconds,DataSize(bytes) / Time(us)yields MB/s. Dividing by1e3converts to GB/s:Bandwidth(GB/s) DataSize(bytes) / Time(us) / 1e3For example, scenario 11 with shape[128, 64, 128]and destination type half:DataSize 128 × 128 × 2 32768 bytes Time 0.050000 us Bandwidth 32768 / 0.050000 / 1e3 655.360 GB/sTheoretical Latency and Bandwidth UtilizationThe Theoretical Bandwidth (Byte/cycle) and Bandwidth Latency (cycle) in the scenario table can be used to estimate theoretical latency. Theoretical bandwidth is derived from egress parallelism (the egress unit processes 64 output elements per cycle, theoretical bandwidth 64 × sizeof(destination type), doubled for dual-target mode). The fixed latency represents the base startup overhead for a single transfer. The theoretical transfer time can be estimated as follows:TransferCycle DataSize(bytes) / TheoreticalBandwidth(Byte/cycle) TheoryCycle Latency(cycle) TransferCycle TheoryTime(us) TheoryCycle / Frequency(MHz) TheoryBandwidth(GB/s) DataSize(bytes) / TheoryTime(us) / 1e3The ratio of measured bandwidth to theoretical bandwidth can be used to evaluate bandwidth utilization:BandwidthUtilization MeasuredBandwidth(GB/s) / TheoryBandwidth(GB/s) × 100%The platform clock frequency is set automatically byperf.shbased on the scenario:PlatformArchitecture CodeClock FrequencyApplicable ScenariosAtlas A3/A2 Training/Inference Platformdav-22011800 MHz1, 2Ascend 950PR/950DT Platformdav-35101650 MHz11-14Roofline AnalysisThis example providesgenerate_roofline.py, which generates ASCII reports and charts from the CSV output produced byperf.sh.Python Package Dependenciesgenerate_roofline.pyuses Python standard libraries to read CSV and generate ASCII reports. To generate PNG/PDF charts, installmatplotlibandnumpy.python3 -m pip install --user matplotlib numpyIf these dependencies are not installed, the script still generates.txtASCII analysis reports but skips chart generation.# Automatically find the latest perf_data directory results python3 generate_roofline.py # Specify a CSV file python3 generate_roofline.py --csv perf_data_xxx_scenario11/perf_result_scenario11.csvThe script has built-in egress parallelism for each scenario. You do not need to manually specify peak bandwidth. The first-instruction overhead defaults to the scenario value (20 cycles for dav-2201, 26 cycles for dav-3510), and the clock frequency is automatically selected based on the scenario.Chart ExampleThe following is a Roofline chart example generated for scenario 1:NotesScenario numbers must match the platform: use scenarios 1, 2 for dav-2201 and scenarios 11-14 for dav-3510.perf.shvalidates the match and reports errors for mismatches.The L0C Buffer to UB path (scenarios 13, 14) is supported only on dav-3510 and uses theFixpipeinterface with the hybrid programming framework.Matrix dimensions must meet alignment requirements: M and N must be multiples of 16. For dual-target mode split along the M dimension, M must be a multiple of 2. For split along the N dimension, N must be a multiple of 32.This is a pure performance test and does not verify computation results. The kernel function does not initialize data in the L0A Buffer or L0B Buffer. It retains a minimal Mmad operation to produce L0C Buffer data beforehand.APipeBarrierPIPE_ALLis inserted between the Mmad pre-operation and the egress instruction to prevent pipeline overlap from causing inaccurateaic_fixpipe_timestatistics.【免费下载链接】asc-devkit本项目是CANN 推出的昇腾AI处理器专用的算子程序开发语言原生支持C和C标准规范主要由类库和语言扩展层构成提供多层级API满足多维场景算子开发诉求。项目地址: https://gitcode.com/cann/asc-devkit创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考