{"id":33026,"date":"2026-09-26T18:37:56","date_gmt":"2026-09-26T10:37:56","guid":{"rendered":"https:\/\/jdssl.top\/?p=33026"},"modified":"2026-09-26T20:28:38","modified_gmt":"2026-09-26T12:28:38","slug":"amd-r9700-32g","status":"publish","type":"post","link":"https:\/\/jdssl.top\/index.php\/2026\/09\/26\/amd-r9700-32g\/","title":{"rendered":"amd-r9700 32g,\u8dd1qwen3.8 27b\u6a21\u578b\uff0c\u8d85\u9ad8\u901f\u7ecf\u9a8c\u5206\u4eab\u3002Hermes\u8c03\u752860-80t\/s\uff0c\u666e\u901a\u4ee3\u7801\u8fbe140+t\/s"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

qwen 3.8 27b mxfp4\u6a21\u578b\uff1a<\/h4>

amd\u5b98\u65b9\u6a21\u578b\uff1ahttps:\/\/huggingface.co\/amd\/Qwen3.8-27B-Quark-AWQ-MXFP4\/tree\/main<\/a><\/p>

\u672a\u7ecf\u5ba1\u67e5mxfp4:https:\/\/huggingface.co\/just1moremodel\/Qwen3.8-27B-Uncensored-MXFP4-awq<\/a><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

\u4e00\u3001\u8fd9\u7bc7\u6587\u7ae0\u6700\u7ec8\u80fd\u5f97\u5230\u4ec0\u4e48<\/h3>

\u6211\u8fd9\u5957\u73af\u5883\u4e0d\u662f\u4e3a\u4e86\u8dd1\u4e00\u4e2a\u201c\u6f02\u4eae benchmark\u201d\u5c31\u7ed3\u675f\uff0c\u800c\u662f\u5b9e\u9645\u7ed9 Hermes \/ Agent\u3001\u4ee3\u7801\u4efb\u52a1\u3001\u957f\u4e0a\u4e0b\u6587\u9879\u76ee\u4f7f\u7528\u3002<\/p>

\u6700\u7ec8\u8c03\u4f18\u65b9\u5411\uff1a<\/p>

  • \u5355\u5361 Radeon AI PRO R9700 32GB\uff1b<\/li>
  • Qwen3.8-27B MXFP4\uff1b<\/li>
  • FP8 DFlash2 drafter\uff1b<\/li>
  • Radiance W4A8 \u4f18\u5316 kernel\uff1b<\/li>
  • TP=1\uff1b<\/li>
  • DFlash SPEC=7\uff1b<\/li>
  • \u5355 Agent \u957f\u4e0a\u4e0b\u6587\u7ea6 160K\uff1b<\/li>
  • Thinking \/ reasoning \u4fdd\u7559\uff1b<\/li>
  • Docker \u540e\u53f0\u8fd0\u884c\uff0cSSH \u5173\u95ed\u540e\u6a21\u578b\u7ee7\u7eed\u5de5\u4f5c\uff1b<\/li>
  • \u4ee3\u7801\u3001JSON\u3001\u7ed3\u6784\u5316\u8f93\u51fa\u901f\u5ea6\u660e\u663e\u9ad8\u4e8e\u666e\u901a\u804a\u5929\uff1b<\/li>
  • Hermes\u8c03\u7528\u901f\u5ea6\u8d85\u5feb\uff0c\u57fa\u672c\u65e0\u7b49\u5f85\uff0c\u5e76\u4e14\u901f\u5ea6\u8fbe\u5230\u60ca\u4eba\u768470+tokens\/s<\/li><\/ul>

    \u5148\u8bf4\u4e00\u4e2a\u975e\u5e38\u91cd\u8981\u7684\u7ed3\u8bba\uff1a<\/p>

    \u4e0d\u8981\u628a\u67d0\u4e00\u6b21\u8f93\u51fa\u663e\u793a 150\u3001170 tok\/s\uff0c\u7406\u89e3\u6210\u6240\u6709 Prompt \u90fd\u4f1a\u56fa\u5b9a\u8dd1\u5230\u8fd9\u4e2a\u901f\u5ea6\u3002<\/strong><\/p>

    DFlash2 \u7684 speculative decoding \u5f88\u5403\u201c\u9884\u6d4b\u63a5\u53d7\u7387\u201d\u3002\u540c\u4e00\u5f20 R9700\u3001\u540c\u4e00\u5957\u6a21\u578b\uff0c\u4ee3\u7801\u3001JSON\u3001\u804a\u5929\u3001\u6563\u6587\u7684 tokens\/s \u53ef\u4ee5\u5dee\u4e00\u500d\u3002<\/p>

    \u4e0a\u6e38\u5355\u5361 R9700 \u7684\u516c\u5f00 BetterBench \u6570\u636e\u4e5f\u6709\u7c7b\u4f3c\u73b0\u8c61\uff1a<\/p>
    \u7c7b\u578b<\/th>\u5355\u5361\u5b9e\u6d4b decode<\/th><\/tr><\/thead>
    code<\/td>145.2 tok\/s<\/td><\/tr>
    JSON<\/td>184.4 tok\/s<\/td><\/tr>
    math<\/td>168.6 tok\/s<\/td><\/tr>
    file edit<\/td>166.1 tok\/s<\/td><\/tr>
    reasoning<\/td>120.8 tok\/s<\/td><\/tr>
    chat<\/td>81.4 tok\/s<\/td><\/tr>
    prose<\/td>79.0 tok\/s<\/td><\/tr>
    \u7efc\u5408<\/td>137.7 tok\/s<\/td><\/tr><\/tbody><\/table>

    \u6240\u4ee5\u672c\u6587\u91cd\u70b9\u4e0d\u662f\u6559\u4f60\u8ffd\u4e00\u4e2a\u201c\u6700\u9ad8\u6570\u5b57\u201d\uff0c\u800c\u662f\uff1a<\/p>

    \u8ba9 R9700 \u771f\u6b63\u8d70\u5bf9\u4f18\u5316\u8def\u5f84\uff0c\u5e76\u6839\u636e\u81ea\u5df1\u7684\u7528\u9014\uff0c\u5728\u901f\u5ea6\u3001\u4e0a\u4e0b\u6587\u3001\u663e\u5b58\u548c\u7a33\u5b9a\u6027\u4e4b\u95f4\u627e\u5230\u5408\u9002\u7684 shape\u3002<\/strong><\/p><\/blockquote>

    \u4e8c\u3001\u6211\u7684\u5b9e\u6d4b\u786c\u4ef6\u73af\u5883<\/h4>

    \u8fd9\u7bc7\u6559\u7a0b\u57fa\u4e8e\u4e0b\u9762\u8fd9\u53f0\u673a\u5668\uff1a<\/p>

    CPU\uff1aIntel Core i5-12400\n\u5185\u5b58\uff1aDDR4 64GB\nPCIe\uff1aPCIe 5.0 x16\nGPU\uff1aRadeon AI PRO R9700 32GB\n\u529f\u8017\uff1a\u7ea6 210W\n\u7cfb\u7edf\uff1aUbuntu Desktop 26.04\n\u5bb9\u5668\uff1aDocker\n\u76ee\u6807\u6a21\u578b\uff1aQwen3.8-27B MXFP4\nDrafter\uff1aQwen3.8-27B-DFlash2-FP8\nvLLM\uff1aRadiance\n<\/code><\/pre>
    \u5f53\u524d\u4f7f\u7528\u955c\u50cf\uff1a<\/p>
    stilldeadcode\/vllm-radiance:0.9.3\n<\/code><\/pre>
    \u5f53\u524d\u4e0a\u6e38\u9879\u76ee\uff1a<\/p>
    https:\/\/github.com\/GGZ14\/vllm-mxfp4\n<\/code><\/pre>
    README \u5f53\u524d\u63a8\u8350 clone\uff1a<\/p>
    git clone https:\/\/codeberg.org\/ggz14\/radiance-vllm-mxfp4\n<\/code><\/pre>
    \u622a\u81f3\u672c\u6587\u6574\u7406\u65f6\uff0c\u4e0a\u6e38 README \u6807\u8bb0\uff1a<\/p>
    Repo version\uff1a0.12.0\nPinned image\uff1astilldeadcode\/vllm-radiance:0.9.3\n<\/code><\/pre>
    \u4e09\u3001\u4e3a\u4ec0\u4e48\u4e0d\u7528\u666e\u901a ROCm + \u5b98\u65b9 vLLM \u5c31\u7ed3\u675f<\/h4>
    R9700 \u5c5e\u4e8e RDNA4 \/ gfx1201\u3002<\/p>
    \u5b83\u5f53\u7136\u53ef\u4ee5\u8dd1 ROCm\uff0c\u4f46\u201c\u80fd\u8dd1\u201d\u4e0e\u201c\u8dd1\u5f97\u5feb\u201d\u4e0d\u662f\u4e00\u56de\u4e8b\u3002<\/p>
    Radiance \u8fd9\u5957\u9879\u76ee\u771f\u6b63\u6709\u4ef7\u503c\u7684\u5730\u65b9\uff0c\u662f\u9488\u5bf9 RDNA4 \u505a\u4e86\u5927\u91cf\u4e13\u7528\u4f18\u5316\uff0c\u5305\u62ec\uff1a<\/p>
    • MXFP4 W4A8 GEMM\uff1b<\/li>
    • R4D paged attention\uff1b<\/li>
    • GDN \u4f18\u5316\uff1b<\/li>
    • FP8 residual stream\uff1b<\/li>
    • INT2 draft head\uff1b<\/li>
    • DFlash2 speculative decoding\uff1b<\/li>
    • \u5355\u5361 TP=1 \u4e13\u7528 profile\uff1b<\/li>
    • KV cache pin\uff1b<\/li>
    • compile graph cache\uff1b<\/li>
    • cudagraph capture \u8c03\u6574\u3002<\/li><\/ul>
      \u7b80\u5355\u7406\u89e3\uff1a<\/p>
      \u666e\u901a vLLM\n\u2193\nROCm \u80fd\u8bc6\u522b R9700\n\nRadiance\n\u2193\n\u7ee7\u7eed\u628a R9700 \u7684\u6a21\u578b kernel\u3001attention\u3001draft\u3001KV\u3001graph \u8c03\u5230\u66f4\u9002\u5408\u8fd9\u5f20\u5361\n<\/code><\/pre>
      \u5982\u679c\u53ea\u662f\u628a\u6a21\u578b\u6210\u529f\u52a0\u8f7d\u8fdb\u663e\u5b58\uff0c\u4f46\u65e5\u5fd7\u91cc\u6ca1\u6709\u771f\u6b63\u4f7f\u7528 Radiance kernel\uff0c\u901f\u5ea6\u53ef\u80fd\u5dee\u5f88\u591a\u3002<\/p>
      \u56db\u3001\u5b89\u88c5\u524d\u5148\u68c0\u67e5\u4e3b\u673a<\/h4>
      \u81f3\u5c11\u786e\u8ba4\uff1a<\/p>
      ls -l \/dev\/kfd\nls -l \/dev\/dri\n<\/code><\/pre>
      \u4e24\u8005\u5e94\u8be5\u5b58\u5728\u3002<\/p>
      \u68c0\u67e5 Docker\uff1a<\/p>
      docker --version\ndocker ps\n<\/code><\/pre>
      \u5982\u679c\u666e\u901a\u7528\u6237\u6267\u884c Docker \u6ca1\u6743\u9650\uff1a<\/p>
      sudo usermod -aG docker $USER\n<\/code><\/pre>
      \u91cd\u65b0\u767b\u5f55\u540e\uff1a<\/p>
      docker ps\n<\/code><\/pre>
      \u78c1\u76d8\u5efa\u8bae\u81f3\u5c11\u9884\u7559\u51e0\u5341 GB\u3002\u4e0a\u6e38 Quick Start \u4f1a\u4e0b\u8f7d\u955c\u50cf\u3001\u6a21\u578b\u548c\u76f8\u5173\u6587\u4ef6\u3002<\/p>
      \u4e94\u3001\u7b2c\u4e00\u6b21\u5b89\u88c5\uff1a\u5efa\u8bae\u5148\u5b8c\u6574\u8dd1\u901a\u5b98\u65b9 Quick Start<\/h4>
      \u5982\u679c\u662f\u7b2c\u4e00\u6b21\u88c5\uff0c\u5148\u4e0d\u8981\u6025\u7740\u4e0a 160K\u3002<\/p>
      cd ~\ngit clone https:\/\/codeberg.org\/ggz14\/radiance-vllm-mxfp4\ncd radiance-vllm-mxfp4\n.\/docker-quickstart.sh\n<\/code><\/pre>
      \u5b83\u4f1a\u5b8c\u6210\uff1a<\/p>
      1. \u4e3b\u673a\u68c0\u67e5\uff1b<\/li>
      2. \u62c9\u53d6 Docker \u955c\u50cf\uff1b<\/li>
      3. \u51c6\u5907 checkpoint\uff1b<\/li>
      4. \u51c6\u5907\u76f8\u5173 kernel\uff1b<\/li>
      5. \u542f\u52a8\u670d\u52a1\uff1b<\/li>
      6. \u53d1\u9001\u6d4b\u8bd5\u8bf7\u6c42\u3002<\/li><\/ol>
        \u4e5f\u53ef\u4ee5\u62c6\u5f00\uff1a<\/p>
        .\/setup-mxfp4.sh\n.\/serve-mxfp4.sh\n<\/code><\/pre>
        \u9ed8\u8ba4 API\uff1a<\/p>
        http:\/\/127.0.0.1:8080\/v1\n<\/code><\/pre>
        \u6d4b\u8bd5\uff1a<\/p>
        curl http:\/\/127.0.0.1:8080\/v1\/chat\/completions \\\n  -H 'Content-Type: application\/json' \\\n  -d '{\n    \"model\":\"Qwen3.8\",\n    \"messages\":[\n      {\"role\":\"user\",\"content\":\"\u4f60\u597d\uff0c\u4ecb\u7ecd\u4e00\u4e0b\u4f60\u81ea\u5df1\"}\n    ]\n  }'\n<\/code><\/pre>
        \u516d\u3001\u5355\u5f20 R9700\uff1a\u4f18\u5148\u4f7f\u7528 serve-tp1.sh<\/h4>
        Radiance \u5df2\u7ecf\u63d0\u4f9b\u5355\u5361\u5165\u53e3\uff1a<\/p>
        .\/serve-tp1.sh\n<\/code><\/pre>
        \u4e0d\u8981\u628a\u53cc\u5361 TP=2 \u53c2\u6570\u751f\u642c\u786c\u5957\u5230\u5355\u5361\u3002<\/p>
        \u5148\u68c0\u6d4b\uff1a<\/p>
        .\/gpu-detect.sh\n<\/code><\/pre>
        \u6b63\u5e38\u5e94\u8be5\u770b\u5230\u7c7b\u4f3c\uff1a<\/p>
        AMD GPUs usable: 1 x Radeon AI PRO R9700\ntensor parallel: 1\nhardware sig: 1x7551-32624\n<\/code><\/pre>
        \u5982\u679c\u7cfb\u7edf\u91cc\u8fd8\u6709 AMD \u6838\u663e\uff0cRadiance \u4f1a\u6309\u663e\u5b58\u95e8\u69db\u6392\u9664\u5c0f\u663e\u5b58 iGPU\u3002<\/p>
        \u4e03\u3001Docker \u540e\u53f0\u542f\u52a8\uff1aSSH \u5173\u95ed\u540e\u7ee7\u7eed\u8fd0\u884c<\/h4>
        \u5b98\u65b9 launcher \u652f\u6301\uff1a<\/p>
        DETACH=1\n<\/code><\/pre>
        \u6240\u4ee5\u5355\u5361\u53ef\u4ee5\uff1a<\/p>
        DETACH=1 .\/serve-tp1.sh\n<\/code><\/pre>
        \u8fd9\u65f6 Docker \u4f7f\u7528 detached \u6a21\u5f0f\u542f\u52a8\u3002<\/p>
        \u542b\u4e49\uff1a<\/p>
        SSH \u65ad\u5f00\uff1a\u6a21\u578b\u7ee7\u7eed\u8fd0\u884c\n\u7ec8\u7aef\u5173\u95ed\uff1a\u6a21\u578b\u7ee7\u7eed\u8fd0\u884c\nCtrl+C \u9000\u51fa docker logs\uff1a\u6a21\u578b\u7ee7\u7eed\u8fd0\u884c\nUbuntu \u6574\u673a\u91cd\u542f\uff1a\u9ed8\u8ba4\u4e0d\u80fd\u7b49\u540c\u4e8e\u201c\u6a21\u578b\u4e00\u5b9a\u81ea\u52a8\u6062\u590d\u201d\n<\/code><\/pre>
        \u67e5\u770b\uff1a<\/p>
        docker ps\n<\/code><\/pre>
        \u65e5\u5fd7\uff1a<\/p>
        docker logs -f vllmmxfp4074\n<\/code><\/pre>
        \u505c\u6b62\uff1a<\/p>
        docker stop vllmmxfp4074\n<\/code><\/pre>
        \u5982\u679c\u4f60\u81ea\u5b9a\u4e49\u4e86\u5bb9\u5668\u540d\uff0c\u6bd4\u5982\uff1a<\/p>
        qwen38-mxfp4\n<\/code><\/pre>
        \u5219\uff1a<\/p>
        docker logs -f qwen38-mxfp4\ndocker stop qwen38-mxfp4\n<\/code><\/pre>
        \u516b\u3001\u6211\u7684\u5b9e\u9645\u76ee\u5f55\u7ed3\u6784<\/h4>
        \u6211\u7684 Ubuntu\uff1a<\/p>
        \/home\/syxj\/vllm\/radiance-vllm-mxfp4\n<\/code><\/pre>
        \u6a21\u578b\uff1a<\/p>
        \/run\/media\/syxj\/lexar\/vllm\/models\n<\/code><\/pre>
        \u542f\u52a8\u811a\u672c\uff1a<\/p>
        \/home\/syxj\/vllm\/\u542f\u505c\u811a\u672c\n<\/code><\/pre>
        Target\uff1a<\/p>
        \/run\/media\/syxj\/lexar\/vllm\/models\/Qwen3.8-27B-Uncensored-MXFP4-awq\n<\/code><\/pre>
        DFlash2\uff1a<\/p>
        \/run\/media\/syxj\/lexar\/vllm\/models\/Qwen3.8-27B-DFlash2-FP8\n<\/code><\/pre>
        \u8fd9\u91cc\u4f7f\u7528\u7684\u662f\u6211\u81ea\u5df1\u7684\u517c\u5bb9 MXFP4 target\u3002<\/p>
        \u5982\u679c\u53ea\u60f3\u590d\u73b0\u5b98\u65b9\u8def\u5f84\uff0c\u4f18\u5148\u4f7f\u7528 setup-mxfp4.sh<\/code> \u51c6\u5907\u7684\u5b98\u65b9 checkpoint\uff0c\u4e0d\u9700\u8981\u548c\u6211\u7684 SNAP \u540d\u79f0\u5b8c\u5168\u4e00\u6837\u3002<\/p>
        \u4e5d\u3001\u7b2c\u4e00\u4f18\u5148\u7ea7\uff1a\u786e\u8ba4 Radiance kernel \u771f\u6b63\u751f\u6548<\/h4>
        \u542f\u52a8\u4ee5\u540e\uff1a<\/p>
        docker logs qwen38-mxfp4 2>&1 | grep -E \\\n\"RadianceMxfp4W4A8LinearKernel|W4A8 fp8-WMMA|Using R4D backend|linear layers|FORCED ONTO AITER\"\n<\/code><\/pre>
        \u6211\u7684\u6b63\u5e38\u65e5\u5fd7\uff1a<\/p>
        [radiance.mxfp4] W4A8 fp8-WMMA GEMM ENABLED\nUsing RadianceMxfp4W4A8LinearKernel for MXFP4 GEMM\nUsing R4D backend\n[radiance.mxfp4] linear layers: 304\/304 on our kernel\n0 FORCED ONTO AITER\n<\/code><\/pre>
        \u6700\u5173\u952e\uff1a<\/p>
        304\/304 on our kernel\n0 FORCED ONTO AITER\n<\/code><\/pre>
        \u5982\u679c\u8fd9\u4e00\u5757\u4e0d\u5bf9\uff0c\u5148\u89e3\u51b3 kernel\uff0c\u518d\u8c03 MAXLEN \/ SPEC \/ \u529f\u8017\u3002<\/p>
        \u5341\u3001\u770b\u5230 native MXFP4 warning\uff0c\u4e0d\u8981\u53ea\u770b\u4e00\u884c\u5c31\u5224\u65ad\u5931\u8d25<\/h4>
        \u53ef\u80fd\u4f1a\u51fa\u73b0\uff1a<\/p>
        The current platform does not support native MXFP4\/MXFP6 computation.\nSimulated weight dequantization...\n<\/code><\/pre>
        \u4f46 Radiance \u540e\u9762\u4f1a\u5b89\u88c5\u81ea\u5df1\u7684 W4A8 kernel\u3002<\/p>
        \u7ee7\u7eed\u770b\uff1a<\/p>
        W4A8 fp8-WMMA GEMM ENABLED\nUsing RadianceMxfp4W4A8LinearKernel\n<\/code><\/pre>
        \u5982\u679c\u540e\u9762\u8fd9\u4e9b\u6b63\u5e38\uff0c\u8bf4\u660e Radiance \u5df2\u7ecf\u63a5\u7ba1\u3002<\/p>
        \u5341\u4e00\u3001DFlash2\uff1a\u5355\u6d41\u9ad8 Tokens\/s \u7684\u5173\u952e<\/h4>
        \u6211\u7684\u914d\u7f6e\uff1a<\/p>
        SPEC_METHOD=dflash\nSPEC=7\n<\/code><\/pre>
        Drafter\uff1a<\/p>
        Qwen3.8-27B-DFlash2-FP8\n<\/code><\/pre>
        \u68c0\u67e5\uff1a<\/p>
        docker logs qwen38-mxfp4 2>&1 | grep -E \\\n\"RADIANCE_DYNAMIC_DRAFT|INT2_DRAFT_HEAD|DFlash|running the draft eagerly\"\n<\/code><\/pre>
        \u6b63\u5e38\uff1a<\/p>
        RADIANCE_DYNAMIC_DRAFT=ON\nint2 draft head armed\n<\/code><\/pre>
        \u5982\u679c\u51fa\u73b0\uff1a<\/p>
        running the draft eagerly\n<\/code><\/pre>
        \u9700\u8981\u91cd\u70b9\u68c0\u67e5 DFlash graph\u3002<\/p>
        \u5341\u4e8c\u3001\u4e3a\u4ec0\u4e48\u540c\u4e00\u5f20\u5361\u6709\u65f6 150+\uff0c\u6709\u65f6\u53ea\u6709 80 tok\/s<\/h4>
        DFlash \u7684\u672c\u8d28\u4e0d\u662f\u628a\u6a21\u578b\u56fa\u5b9a\u53d8\u6210\u201c150 tok\/s\u201d\u3002<\/p>
        \u5b83\u662f\uff1a<\/p>
        drafter \u4e00\u6b21\u731c\u591a\u4e2a token\uff0ctarget \u4e00\u6b21\u9a8c\u8bc1\uff1b\u731c\u5bf9\u8d8a\u591a\uff0c\u4e00\u6b21 forward \u5b9e\u9645\u5f97\u5230\u7684 token \u8d8a\u591a\u3002<\/p><\/blockquote>
        \u4f8b\u5982\u540c\u6837\u7ea6 35ms\/update\uff1a<\/p>
        \u4ee3\u7801\uff1a\u4e00\u6b21\u63a5\u53d7 4.6 token\n\u804a\u5929\uff1a\u4e00\u6b21\u63a5\u53d7 2.7 token\n<\/code><\/pre>
        \u6700\u7ec8 tokens\/s \u4f1a\u5dee\u5f88\u591a\u3002<\/p>
        \u6240\u4ee5\uff1a<\/p>
        • HTML \/ JS \/ Python \u901a\u5e38\u66f4\u5feb\uff1b<\/li>
        • JSON \/ file edit \u901a\u5e38\u66f4\u5feb\uff1b<\/li>
        • \u81ea\u7531\u804a\u5929 \/ prose \u66f4\u6162\uff1b<\/li>
        • Hermes \u590d\u6742 Agent \u4efb\u52a1\u4ecb\u4e8e\u4e2d\u95f4\u3002<\/li><\/ul>
          \u8fd9\u662f speculative decoding \u7684\u6b63\u5e38\u7279\u5f81\u3002<\/p>
          \u5341\u4e09\u3001SPEC=7 \u8fd8\u662f SPEC=5<\/h4>
          \u6211\u4e3b\u8981\u4f7f\u7528\uff1a<\/p>
          SPEC=7\n<\/code><\/pre>
          \u56e0\u4e3a\u4e3b\u8981\u4efb\u52a1\u662f\uff1a<\/p>
          • \u4ee3\u7801\uff1b<\/li>
          • Agent\uff1b<\/li>
          • \u6587\u4ef6\u7f16\u8f91\uff1b<\/li>
          • HTML\uff1b<\/li>
          • JSON\uff1b<\/li>
          • \u5de5\u5177\u8c03\u7528\u3002<\/li><\/ul>
            \u5982\u679c\u4e3b\u8981\u662f\uff1a<\/p>
            • \u666e\u901a\u804a\u5929\uff1b<\/li>
            • \u957f prose\uff1b<\/li>
            • \u975e\u7ed3\u6784\u5316\u4e2d\u6587\u8f93\u51fa\uff1b<\/li><\/ul>
              \u5efa\u8bae\u81ea\u5df1 A\/B\uff1a<\/p>
              SPEC=5\nSPEC=7\n<\/code><\/pre>
              \u5176\u4ed6\u53c2\u6570\u5b8c\u5168\u4e0d\u52a8\u3002<\/p>
              \u5341\u56db\u3001\u5355 Agent \u7528\u6237\uff1aMAXSEQS \u4e0d\u8981\u76f2\u76ee\u8ffd\u5927<\/h4>
              \u539f\u6765\u7684\u957f\u4e0a\u4e0b\u6587\u914d\u7f6e\uff1a<\/p>
              MAXLEN=220000\nMAXSEQS=3\nCHUNK=2560\nSPEC=7\n<\/code><\/pre>
              \u540e\u6765\u6539\u6210\uff1a<\/p>
              MAXSEQS=1\n<\/code><\/pre>
              \u5bf9\u6211\u7684\u7528\u9014\u66f4\u5408\u7406\u3002<\/p>
              MAXSEQS \u4f1a\u5f71\u54cd\uff1a<\/p>
              • cudagraph capture\uff1b<\/li>
              • Mamba \/ GDN state\uff1b<\/li>
              • scheduler\uff1b<\/li>
              • KV \u4f59\u91cf\uff1b<\/li>
              • \u5e76\u53d1\u8bf7\u6c42\u80fd\u529b\u3002<\/li><\/ul>
                \u5982\u679c\u4f60\u7684\u573a\u666f\uff1a<\/p>
                \u4e00\u4e2a\u7528\u6237\n\u4e00\u4e2a Agent\n\u4e00\u5f20 R9700\n<\/code><\/pre>
                \u6ca1\u6709\u5fc5\u8981\u628a\u5f88\u591a\u663e\u5b58\u6d6a\u8d39\u5728\u5e76\u53d1\u4e0a\u3002<\/p>
                \u5341\u4e94\u3001CHUNK\uff1aPrefill \u548c\u663e\u5b58\u4e4b\u95f4\u7684\u4ea4\u6362<\/h4>
                \u5b98\u65b9\u5355\u5361\u504f\u541e\u5410\u7684\u9ed8\u8ba4\u65b9\u5411\uff1a<\/p>
                CHUNK=4096\n<\/code><\/pre>
                \u6211\u7684 160K \u957f\u4e0a\u4e0b\u6587\uff1a<\/p>
                CHUNK=2048\n<\/code><\/pre>
                \u5173\u7cfb\u5927\u81f4\u662f\uff1a<\/p>
                CHUNK \u5927\n\u2192 Prefill \u5feb\n\u2192 activation \u5cf0\u503c\u9ad8\n\u2192 KV \u5269\u4f59\u5c11\n\nCHUNK \u5c0f\n\u2192 Prefill \u7a0d\u4fdd\u5b88\n\u2192 activation \u5cf0\u503c\u4f4e\n\u2192 \u66f4\u5bb9\u6613\u7ed9\u957f\u4e0a\u4e0b\u6587\u7559 KV\n<\/code><\/pre>
                \u6240\u4ee5\uff1a<\/p>
                65K \/ 128K\uff0c\u91cd\u89c6 Prefill\uff1a\n\u4f18\u5148 4096\n\n160K\uff0c\u91cd\u89c6\u957f\u4e0a\u4e0b\u6587\uff1a\n\u53ef\u4ee5\u5148\u4ece 2048 \u8d77\n<\/code><\/pre>
                \u5341\u516d\u3001220K \u4e3a\u4ec0\u4e48\u4f1a\u5931\u8d25<\/h4>
                \u6211\u6700\u5f00\u59cb\u8bd5\u8fc7\uff1a<\/p>
                MAXLEN=220000\nMAXSEQS=3\nCHUNK=2560\nSPEC=7\nGPU_UTIL=0.98\nKV_MEM=0\n<\/code><\/pre>
                \u771f\u5b9e\u542f\u52a8\u65e5\u5fd7\uff1a<\/p>
                Available KV cache memory: 5.03 GiB\n\nTo serve at least one request with max seq len 220000:\n7.53 GiB KV cache is needed\n\nestimated maximum model length is 137280\n<\/code><\/pre>
                \u6240\u4ee5\u8fd9\u4e0d\u662f\u201c220K \u6bd4\u8f83\u6162\u201d\u3002<\/p>
                \u800c\u662f\uff1a<\/p>
                \u5728 KV_MEM=0 \u7684 profiling \u4e0b\uff0c220K \u6839\u672c\u8d77\u4e0d\u6765\u3002<\/strong><\/p><\/blockquote>
                \u5341\u4e03\u3001\u7406\u89e3 KV_MEM\uff1a0\u3001auto \u548c\u663e\u5f0f pin<\/h4>
                \u8fd9\u662f\u957f\u4e0a\u4e0b\u6587\u6700\u91cd\u8981\u7684\u53c2\u6570\u4e4b\u4e00\u3002<\/p>
                KV_MEM=0<\/h5>
                KV_MEM=0\n<\/code><\/pre>
                \u8868\u793a\u5f3a\u5236\u8ba9 vLLM profiling\u3002<\/p>
                \u4f18\u70b9\uff1a\u5b89\u5168\u3001\u4e0d\u5bb9\u6613\u628a\u663e\u5b58\u9876\u6b7b\u3002<\/p>
                \u7f3a\u70b9\uff1a\u4f1a\u4fdd\u5b88\u5730\u6263\u9664 transient activation peak\u3001cudagraph \u4f30\u7b97\u548c allocator margin\u3002<\/p>
                KV_MEM=auto<\/h5>
                KV_MEM=auto\n<\/code><\/pre>
                Radiance \u4f1a\u5148\u67e5 kv-profiles.tsv<\/code>\u3002\u5982\u679c\u5f53\u524d hardware signature + shape \u6709\u5df2\u6d4b pin \u5c31\u4f7f\u7528\uff0c\u5426\u5219\u56de\u9000 profiling\u3002<\/p>
                KV_MEM=<\/h5>
                \u4f8b\u5982\uff1a<\/p>
                KV_MEM=6100000000\n<\/code><\/pre>
                \u76f4\u63a5\u56fa\u5b9a KV cache bytes\u3002<\/p>
                \u5b83\u53ef\u4ee5\u62ff\u56de profiling \u7559\u4e0b\u7684 margin\uff0c\u4f46\u5fc5\u987b\u81ea\u5df1\u9a8c\u8bc1\u7a33\u5b9a\u6027\u3002<\/p>
                \u5341\u516b\u3001\u4e0d\u8981\u7167\u6284\u522b\u4eba\u7684 KV pin<\/h4>
                \u4e0a\u6e38\u516c\u5f00\u8fc7\u7c7b\u4f3c\u5355\u5361 R9700 profile\uff1a<\/p>
                1x7551-32624\nMAXSEQS=8\nCHUNK=4096\nMAXLEN=65536\nSPEC=dflash\nKV=6535819798 bytes\n<\/code><\/pre>
                \u8fd9\u53ea\u80fd\u8bf4\u660e\u5355 R9700 \u7684\u663e\u5f0f pin \u53ef\u4ee5\u9ad8\u4e8e\u4fdd\u5b88 profiling\u3002<\/p>
                \u5b83\u4e0d\u4ee3\u8868\u6240\u6709 R9700\u3001\u6240\u6709 MAXSEQS\u3001CHUNK\u3001MAXLEN \u90fd\u80fd\u76f4\u63a5\u4f7f\u7528\u540c\u4e00\u4e2a\u503c\u3002<\/p>
                KV margin \u8fd8\u53d7\uff1a<\/p>
                • activation peak\uff1b<\/li>
                • cudagraph capture set\uff1b<\/li>
                • allocator fragmentation\uff1b<\/li>
                • checkpoint\uff1b<\/li>
                • drafter\uff1b<\/li>
                • \u8f6f\u4ef6\u7248\u672c\uff1b<\/li><\/ul>
                  \u5f71\u54cd\u3002<\/p>
                  \u5341\u4e5d\u3001\u6b63\u786e\u505a\u6cd5\uff1a\u8fd0\u884c calibrate-kv.sh<\/h4>
                  Radiance \u5df2\u7ecf\u63d0\u4f9b\uff1a<\/p>
                  .\/calibrate-kv.sh\n<\/code><\/pre>
                  \u5148\u770b\u8ba1\u5212\uff1a<\/p>
                  .\/calibrate-kv.sh --dry-run\n<\/code><\/pre>
                  \u5feb\u901f\u53ea\u505a\u7b2c\u4e00\u9636\u6bb5\uff1a<\/p>
                  .\/calibrate-kv.sh --quick\n<\/code><\/pre>
                  \u6b63\u5f0f\u6821\u51c6\uff1a<\/p>
                  .\/calibrate-kv.sh\n<\/code><\/pre>
                  \u6821\u51c6\u811a\u672c\u4e0d\u4ec5\u770b\u201cKV \u5206\u914d\u6210\u529f\u201d\uff0c\u8fd8\u4f1a\u68c0\u67e5 health\u3001prefill \u548c decode\uff0c\u9010\u6b65\u628a pin \u63a8\u5230\u5931\u8d25\u8fb9\u754c\u518d\u56de\u9000\u3002<\/p>
                  \u4fee\u6539 MAXSEQS \u6216 CHUNK \u540e\uff0c\u5efa\u8bae\u91cd\u65b0\u6821\u51c6\u3002<\/p>
                  \u4e8c\u5341\u3001\u6211\u7684 160K \u957f\u4e0a\u4e0b\u6587 shape<\/h4>
                  \u76ee\u524d\u6211\u7684\u65b9\u5411\uff1a<\/p>
                  MAXLEN=160160\nMAXSEQS=1\nCHUNK=2048\nSPEC_METHOD=dflash\nSPEC=7\nGPU_UTIL=0.98\nRADIANCE_GDN_LAZY=0\n<\/code><\/pre>
                  \u4e3a\u4ec0\u4e48\u4e0d\u662f\u6574 160000\uff1f<\/p>
                  \u5f53\u524d TP=1 \u65e5\u5fd7\u663e\u793a attention block\uff1a<\/p>
                  880 tokens\n<\/code><\/pre>
                  \u6240\u4ee5\uff1a<\/p>
                  160160 = 880 \u00d7 182\n<\/code><\/pre>
                  \u5f53\u524d\u6211\u7684\u673a\u5668\u4f7f\u7528\u8fc7\uff1a<\/p>
                  KV_MEM=6100000000\n<\/code><\/pre>
                  \u5e76\u6210\u529f\u8fd0\u884c\u3002<\/p>
                  \u4f46\u8bf7\u6ce8\u610f\uff1a<\/p>
                  6100000000 \u662f\u6211\u7684\u673a\u5668\u4e0a\u7684\u5de5\u4f5c\u503c\uff0c\u4e0d\u662f\u6240\u6709 R9700 \u7684\u4e07\u80fd\u53c2\u6570\u3002\u6700\u7ec8\u4ee5\u81ea\u5df1 calibrate-kv.sh \u7684\u7ed3\u679c\u4e3a\u51c6\u3002<\/strong><\/p><\/blockquote>
                  \u4e8c\u5341\u4e00\u3001\u4e00\u4e2a\u9002\u5408\u5206\u4eab\u590d\u73b0\u7684 160K \u542f\u52a8 Wrapper<\/h4>
                  \u5047\u8bbe\uff1a<\/p>
                  \u9879\u76ee\uff1a\/home\/\u4f60\u7684\u7528\u6237\u540d\/vllm\/radiance-vllm-mxfp4\n\u6a21\u578b\uff1a\/mnt\/models\n<\/code><\/pre>
                  \u521b\u5efa\uff1a<\/p>
                  start-vllm-longctx.sh\n<\/code><\/pre>
                  \u5185\u5bb9\uff1a<\/p>
                  #!\/usr\/bin\/env bash\nset -euo pipefail\n\nBASE=\"\/home\/\u4f60\u7684\u7528\u6237\u540d\/vllm\/radiance-vllm-mxfp4\"\nMODELS=\"\/mnt\/models\"\nSNAP=\"$MODELS\/Qwen3.8-27B-Uncensored-MXFP4-awq\"\nDRAFTER=\"$MODELS\/Qwen3.8-27B-DFlash2-FP8\"\n\nexport RUNTIME=docker\nexport IMAGE=\"stilldeadcode\/vllm-radiance:0.9.3\"\nexport NAME=\"qwen38-mxfp4\"\nexport PORT=8081\n\nexport MODELS\nexport SNAP\nexport DRAFTER\n\nexport MAXLEN=160160\nexport MAXSEQS=1\nexport CHUNK=2048\n\nexport SPEC_METHOD=dflash\nexport SPEC=7\nexport GPU_UTIL=0.98\n\n# \u793a\u4f8b\u503c\uff1a\u6b63\u5f0f\u4f7f\u7528\u8bf7\u6362\u6210\u81ea\u5df1\u6821\u51c6\u7ed3\u679c\nexport KV_MEM=6100000000\n\n# \u591a\u8f6e\u804a\u5929\u4e0d\u8981\u6253\u5f00 lazy GDN\nexport RADIANCE_GDN_LAZY=0\n\n# \u957f\u4e0a\u4e0b\u6587 shape \u5355\u72ec cache\nexport CACHE=\"$HOME\/.radiance-cache-w4a8-093-longctx-160k-s1-c2048\"\n\n# Docker \u540e\u53f0\u8fd0\u884c\nexport DETACH=1\n\nexec \"$BASE\/serve-tp1.sh\"\n<\/code><\/pre>
                  \u6743\u9650\uff1a<\/p>
                  chmod +x start-vllm-longctx.sh\n<\/code><\/pre>
                  \u542f\u52a8\uff1a<\/p>
                  .\/start-vllm-longctx.sh\n<\/code><\/pre>
                  \u4e8c\u5341\u4e8c\u3001\u4e3a\u4ec0\u4e48 RADIANCE_GDN_LAZY=0<\/h4>
                  \u4e0a\u6e38\u5df2\u7ecf\u628a lazy GDN \u9ed8\u8ba4\u5173\u95ed\uff0c\u56e0\u4e3a\u591a\u8f6e\u804a\u5929\u4e2d\u51fa\u73b0\u8fc7\u91cd\u590d\u5faa\u73af\u3001\u7a7a\u56de\u590d\u548c\u72b6\u6001\u635f\u574f\u3002<\/p>
                  \u6240\u4ee5 Hermes \/ OpenAI-compatible \u591a\u8f6e\u804a\u5929\u5efa\u8bae\uff1a<\/p>
                  RADIANCE_GDN_LAZY=0\n<\/code><\/pre>
                  \u4e0d\u8981\u4e3a\u4e86\u7701\u4e00\u70b9\u72b6\u6001\u663e\u5b58\u4e3b\u52a8\u6539\u6210 1\u3002<\/p>
                  \u4e8c\u5341\u4e09\u3001Thinking \u6ca1\u6709\u56e0\u4e3a\u63d0\u901f\u88ab\u5173\u6389<\/h4>
                  \u6211\u7684\u914d\u7f6e\u6ca1\u6709\uff1a<\/p>
                  --disable-thinking\nenable_thinking=false\nthinking=false\n<\/code><\/pre>
                  \u65e5\u5fd7\u4ecd\u4f1a\u770b\u5230\uff1a<\/p>
                  reasoning_parser=qwen3\nchat_template=\/patches\/qwen-fixed-v22.3.jinja\n<\/code><\/pre>
                  \u5982\u679c\u51fa\u73b0\uff1a<\/p>
                  Model Runner V2 does not yet support the thinking_token_budget request parameter\n<\/code><\/pre>
                  \u5b83\u53ea\u662f\u8868\u793a V2 Runner \u4e0d\u652f\u6301\u201c\u6309\u8bf7\u6c42\u8bbe\u7f6e thinking token budget\u201d\uff0c\u4e0d\u7b49\u4e8e Thinking \u88ab\u5173\u95ed\u3002<\/p>
                  \u6211\u7684\u539f\u5219\uff1a<\/p>
                  \u63d0\u901f\u5f52\u63d0\u901f\uff0c\u4e0d\u9760\u5173\u95ed Thinking \u6765\u4f5c\u5f0a\u3002<\/strong><\/p><\/blockquote>
                  \u4e8c\u5341\u56db\u3001\u4e0d\u540c shape \u5efa\u8bae\u7528\u4e0d\u540c Compile Cache<\/h4>
                  \u4e0d\u8981\u8ba9\uff1a<\/p>
                  65K \/ 8 \/ 4096\n128K \/ 8 \/ 4096\n160K \/ 1 \/ 2048\n220K \/ 3 \/ 2560\n<\/code><\/pre>
                  \u5168\u90e8\u5171\u7528\u4e00\u4e2a cache\u3002<\/p>
                  \u5efa\u8bae\uff1a<\/p>
                  ~\/.radiance-cache-w4a8-093-tp1-throughput\n~\/.radiance-cache-w4a8-093-longctx-160k-s1-c2048\n<\/code><\/pre>
                  \u5982\u679c\u4fee\u6539\u8fc7 traced graph \u76f8\u5173\u914d\u7f6e\uff0c\u65e7 cache \u53ef\u80fd\u91cd\u653e\u65e7 graph\u3002<\/p>
                  \u4e8c\u5341\u4e94\u3001160K \u4ee5\u540e\u8fd8\u6709\u6ca1\u6709\u7a7a\u95f4<\/h4>
                  \u6709\uff0c\u4f46\u5efa\u8bae\u5c0f\u6b65\u63d0\u9ad8\uff1a<\/p>
                  160160\n\u2193\n166320\n\u2193\n176K \u5de6\u53f3\n\u2193\n180400\n\u2193\n\u6709\u4f59\u91cf\u518d\u8003\u8651 192K\n<\/code><\/pre>
                  \u6bcf\u5347\u4e00\u6863\uff1a<\/p>
                  1. MAXSEQS \u4e0d\u53d8\uff1b<\/li>
                  2. CHUNK \u4e0d\u53d8\uff1b<\/li>
                  3. SPEC \u4e0d\u53d8\uff1b<\/li>
                  4. \u91cd\u65b0\u6821\u51c6 KV\uff1b<\/li>
                  5. \u51b7\u542f\u52a8\uff1b<\/li>
                  6. \u56fa\u5b9a Prompt \u6d4b\u8bd5\uff1b<\/li>
                  7. \u518d\u8dd1\u771f\u5b9e Agent \u591a\u8f6e\u4efb\u52a1\u3002<\/li><\/ol>
                    \u4e0d\u8981\u4ece 160K \u76f4\u63a5\u8df3\u56de 220K\u3002<\/p>
                    \u4e8c\u5341\u516d\u3001GPU_UTIL \u4e0d\u5efa\u8bae\u7ee7\u7eed\u786c\u9876<\/h4>
                    \u5f53\u524d\uff1a<\/p>
                    GPU_UTIL=0.98\n<\/code><\/pre>
                    \u5df2\u7ecf\u6bd4\u8f83\u6fc0\u8fdb\u3002<\/p>
                    \u4e0d\u5efa\u8bae\u4e3a\u4e86\u51e0\u5341 K \u4e0a\u4e0b\u6587\u76f4\u63a5\u6539\uff1a<\/p>
                    GPU_UTIL=0.99\n<\/code><\/pre>
                    \u4f18\u5148\u7ea7\u5e94\u8be5\u662f\uff1a<\/p>
                    MAXSEQS\n\u2192 CHUNK\n\u2192 KV calibration\n\u2192 MAXLEN\n\u2192 \u6700\u540e\u624d\u8003\u8651 GPU_UTIL\n<\/code><\/pre>
                    \u4e8c\u5341\u4e03\u3001210W \u548c\u66f4\u9ad8\u529f\u8017\u600e\u4e48\u9009<\/h4>
                    \u4e0d\u8981\u7b80\u5355\u7406\u89e3\uff1a<\/p>
                    300W > 210W\uff0c\u6240\u4ee5\u6240\u6709\u4efb\u52a1\u90fd\u66f4\u5feb\n<\/code><\/pre>
                    \u66f4\u9ad8\u529f\u8017\u53ef\u80fd\u5bf9 prefill\u3001\u591a\u5e76\u53d1 aggregate throughput \u66f4\u6709\u5e2e\u52a9\uff0c\u4f46\u5355\u6d41 decode \u4e0d\u4e00\u5b9a\u540c\u6bd4\u4f8b\u63d0\u5347\u3002<\/p>
                    \u6211\u7684\u4e3b\u8981\u573a\u666f\u662f Hermes \/ Agent \/ \u5355\u7528\u6237 \/ \u957f\u65f6\u95f4\u8fd0\u884c\uff0c\u6240\u4ee5\u76ee\u524d\u66f4\u503e\u5411\uff1a<\/p>
                    210W + \u7a33\u5b9a\n<\/code><\/pre>
                    \u4e8c\u5341\u516b\u3001PCIe 3.0 x16 \u4f1a\u4e0d\u4f1a\u4e25\u91cd\u62d6\u6162<\/h4>
                    \u6211\u7684\u673a\u5668\u5c31\u662f PCIe 5.0 x16\u3002<\/p>
                    \u6a21\u578b\u5b8c\u6574\u9a7b\u7559\u663e\u5b58\u4ee5\u540e\uff0c\u5355\u6d41 decode \u7b2c\u4e00\u74f6\u9888\u901a\u5e38\u4e0d\u662f PCIe\uff0c\u800c\u66f4\u53ef\u80fd\u662f\uff1a<\/p>