AMD Radeon RX 7900 XT 20GB

Reported on GitHub by sterlp — posted 2026-09-22 Original report

Eval bug: ROCm error: an illegal memory access was encountered 0.4.1

Method (as reported): Reporter retained a HIP illegal-memory-access log during intermittent inference; no controlled completed throughput.

• Reported GPU setup 2 × RX 7900 XT
• GPU count 2
• Operating system Linux (Linux x86-64)
• Engine llama.cpp 0.4.1-dev build 10964; reported commit b29c606
• Backend HIP; ROCm 10 reported
• Model Qwen3.8-27B UD-Q5_K_XL
• Configured context 180,000 tokens
• Actual input input length not reported
• Full window tested no
• Reported context Configured 180K window is not measured occupied input or demonstrated full-window fit.
• Topology dual-split (Reported layer split with tensor split 0.55,0.45; active client count unreported.)
• Speculative decoding draft MTP max 3 reported

Not reported: CPU, system memory, conversion provider, artifact SHA-256 and driver version, actual occupied input and full-window behavior, active concurrent clients and observed server slots, physical residency and completed request throughput, cause of the intermittent HIP failure

• The issue is closed as duplicate of #20176 and remains bug-unconfirmed; this author report is not a universal HIP failure.
• The -np -1 argument is not an observed slot or concurrent-client count; a log slot ID is not controlled concurrency.
• This Linux HIP issue and the same author's Windows Vulkan stall issue #28744 are distinct reports, not independent replications.
• The local GGUF filename does not establish the conversion provider.
• No completed controlled dual-card rate, replica-versus-split comparison, or native-Windows result is supplied.

Setup guide (external)

Related link (external)

Reported on GitHub by sterlp — posted 2026-09-11 Original report

Dual RX 7900 XT Windows stream-stall issue

Method (as reported): Reporter describes one stalled streaming request and uncertain server/client cause; in-flight slot rate is not completed throughput.

• Reported GPU setup 2 × RX 7900 XT
• GPU count 2
• Operating system Windows (native Windows x86-64)
• Engine llama.cpp 0.4.0-dev build 10817; Clang 20.1.8
• Backend Vulkan
• Model Qwen3.8-27B Unsloth UD-Q5_K_XL (Unsloth)
• Configured context 150,000 tokens
• Actual input input length not reported
• Full window tested no
• Reported context Issue does not establish achieved prompt occupancy or completed work.
• Topology not reported — Actual device split and active client count unreported.
• Speculative decoding draft MTP max 4
• CPU Ryzen 7 7800X3D

Not reported: model file checksum, topology, system memory, driver version, exact GPU device selection/split and residency, number of simultaneous clients and active slots, whether the server or client caused the stalled stream, completed request performance

• Closed issue remains labeled bug-unconfirmed; the reporter was not certain the server caused the stall.
• The server alias says Qwen3.6, but the model path is Qwen3.8; do not use the alias as model identity.
• No numeric dual-GPU performance recommendation can be derived.

Setup guide (external)

Reported on Reddit by EqualCryptographer67 Original report

Qwen3.8-27B IQ2 single-response report on dual-XT host

Method (as reported): Median of nine IQ2 prompts; ROCm 2.46.0 is an author setup label, not the AMD driver or a verified engine build.

• Reported GPU setup RX 7900 XT 20GB; two installed on host
• GPU count not reported
• Operating system Windows (native Windows 11)
• Backend ROCm/HIP as author describes
• Driver AMD 32.0.31041.1004
• Model Qwen3.8-27B IQ2_XXS
• Actual input input length not reported
• Output 1,024 tokens
• Full window tested no
• Reported context Author says 8k context and roughly 700 input tokens; exact counts unreported.
• Simultaneous clients 1
• Topology not reported — Two cards installed; actual serving GPU count and card assignment unreported.
• Speculative decoding MTP3 as author describes
• CPU Ryzen 7 5800X3D
• Memory configuration 128 GB DDR4 reported; byte convention unreported
• Memory speed 3600 MT/s
• Board Gigabyte B550 EAGLE WIFI6
• PCIe XFX PCIe 4.0 x16; PowerColor chipset PCIe 3.0 x1
• Host notes XFX/PowerColor downclocked; both power limit -10%.

• 66.5 tok/s — generation speed (per request) · 1 client
  Author reports median across nine prompts, not a nine-request concurrent run.
  This row: quantisation: IQ2_XXS · speculative decoding: MTP3
• 59.9 tok/s — other (per request) · 1 client
  Author labels this including prompt processing; exact timing formula unreported; not generation TPS.
  This row: quantisation: IQ2_XXS · speculative decoding: MTP3

Not reported: engine, configured context, topology, system memory, Exact serving GPU count and card assignment, Conversion provider, artifact digest and engine build, Actual prompt occupancy, metric timing formula and sample spread

• Host has two GPUs, but a single-response result does not establish dual-GPU use.
• No matched task-quality evaluation supports preferring IQ2 or IQ4 from speed alone.
• The reduced clocks, power limits and chipset x1 path preclude transfer to a normal dual-card setup.
• One author and host; no FastLLM recipe for this setup.

Setup guide (external)

Reported on Reddit by EqualCryptographer67 Original report

Qwen3.8-27B IQ4 single-response report on dual-XT host

Method (as reported): Author's three IQ4 runs; ROCm 2.46.0 is an author setup label, not the AMD driver or a verified engine build.

• Reported GPU setup RX 7900 XT 20GB; two installed on host
• GPU count not reported
• Operating system Windows (native Windows 11)
• Backend ROCm/HIP as author describes
• Driver AMD 32.0.31041.1004
• Model Qwen3.8-27B IQ4_XS
• Actual input input length not reported
• Output 1,024 tokens
• Full window tested no
• Reported context Author says 8k context and roughly 700 input tokens; exact counts unreported.
• Simultaneous clients 1
• Topology not reported — Two cards installed; actual serving GPU count and card assignment unreported.
• Speculative decoding MTP3 as author describes
• CPU Ryzen 7 5800X3D
• Memory configuration 128 GB DDR4 reported; byte convention unreported
• Memory speed 3600 MT/s
• Board Gigabyte B550 EAGLE WIFI6
• PCIe XFX PCIe 4.0 x16; PowerColor chipset PCIe 3.0 x1
• Host notes XFX/PowerColor downclocked; both power limit -10%.

• 46.4 tok/s — generation speed (per request) · 1 client
  Author's single-response IQ4 figure; three runs, aggregation method unreported.
  This row: quantisation: IQ4_XS · speculative decoding: MTP3
• 43.1 tok/s — other (per request) · 1 client
  Author labels this including prompt processing; exact timing formula unreported; not generation TPS.
  This row: quantisation: IQ4_XS · speculative decoding: MTP3

Not reported: engine, configured context, topology, system memory, Exact serving GPU count and card assignment, Conversion provider, artifact digest and engine build, Actual prompt occupancy, metric timing formula and sample spread

• Host has two GPUs, but a single-response result does not establish dual-GPU use.
• No matched task-quality evaluation supports preferring IQ2 or IQ4 from speed alone.
• The reduced clocks, power limits and chipset x1 path preclude transfer to a normal dual-card setup.
• One author and host; no FastLLM recipe for this setup.

Setup guide (external)

Reported on Reddit by EqualCryptographer67 Original report

Qwen3.8-27B IQ4 two-copy C16 aggregate report

Method (as reported): Author's combined C16 throughput for two model copies; actual copy-to-card binding and repeated-sample method unreported.

• Reported GPU setup RX 7900 XT 20GB; two installed on host
• GPU count not reported
• Operating system Windows (native Windows 11)
• Driver AMD 32.0.31041.1004
• Model Qwen3.8-27B IQ4_XS
• Actual input input length not reported
• Reported context C16 request sizes, configured/occupied context and output counts are unreported.
• Simultaneous clients 16
• Topology not reported — Two separate model copies; process count and GPU-to-copy binding unreported.
• CPU Ryzen 7 5800X3D
• Memory configuration 128 GB DDR4 reported; byte convention unreported
• Memory speed 3600 MT/s
• Board Gigabyte B550 EAGLE WIFI6
• PCIe XFX PCIe 4.0 x16; PowerColor chipset PCIe 3.0 x1
• Host notes XFX/PowerColor downclocked; both power limit -10%.

• 103.5 tok/s — generation speed (aggregate across clients) · 16 clients
  Author reports combined generation throughput for two IQ4 copies at 16 concurrent requests; not per-response decode.
  This row: quantisation: IQ4_XS · topology: other · cards: not reported for this row

Not reported: engine, configured context, system memory, GPU-to-copy binding and actual serving GPU count, Replica process count, engine build and GGUF artifact digest, C16 request sizes, occupied contexts and output counts, C16 timing formula, repetitions and per-client latency/fairness, C16 request errors and task-quality checks

• The reported figure is combined capacity across concurrent requests, not one user's decode speed.
• The single-response IQ4 settings cannot be assigned to the C16 arm without evidence.
• Author says splitting one model was slower, but supplies no matched numeric split measurement or defined comparator.
• One author and host; no FastLLM recipe for this setup.

Setup guide (external)

Reported on GitHub by cadamcat · measured 2026-08-24 Original report

Dual Radeon vLLM: Qwen3.8 checkpoint campaign

Method (as reported): Random-prefix request ladder; decode first token to last; one stack per campaign.

HEAD seen at retrieval: 60c702076233 (not necessarily the commit the author measured)

• Reported GPU setup 2 × RX 7900 XT 20GB
• GPU count 2
• Operating system Linux (Proxmox VFIO Linux guest)
• Environment VM guest
• Engine vLLM 0.23 patched container
• Backend ROCm
• Model Qwen3.8-27B asymmetric AWQ int4 checkpoint
• Actual input input length not reported
• Reported context Author reports depth ladder with random prefixes, without a configured-context claim.
• Simultaneous clients 1
• Workers 1
• Topology dual-split (Tensor parallel, no P2P, VFIO guest)
• CPU Threadripper 1950X
• Board X399
• PCIe cross-die PCIe 3.0

• 12.3 tok/s — generation speed (per request) · API server · 1 client
  First-to-last decode at roughly 500-token depth; TTFT excluded
• 10.7 tok/s — generation speed (per request) · API server · 1 client
  First-to-last decode at roughly 32000-token depth; TTFT excluded

Not reported: system memory, Checkpoint digest, Exact configured context, Per-row variance, Native Windows result, Concurrent-client throughput

• This asymmetric checkpoint missed the native gfx1100 W4A16 kernel.
• Later vLLM and attention-patch results are separate software stacks.
• Same author and host as related dual-XT rows; not an independent replication.

Setup guide (external)

Reported on GitHub by cadamcat Original report

Dual Radeon: controlled Qwen3.8 split-KV attention test

Method (as reported): Two reverse-order A/B passes; fresh container per cell; same checkpoint; attention file only changed.

HEAD seen at retrieval: 60c702076233 (not necessarily the commit the author measured)

• Reported GPU setup 2 × RX 7900 XT 20GB
• GPU count 2
• Operating system Linux (Proxmox VFIO Linux guest)
• Environment VM guest
• Engine vLLM 0.27.1.dev
• Backend ROCm 10.0
• Model Qwen3.8-27B same asymmetric AWQ int4 checkpoint
• Actual input input length not reported
• Reported context Approximate reported depth rungs; exact occupied inputs not established here.
• Simultaneous clients 1
• Workers 1
• Topology dual-split (Tensor parallel, no P2P, VFIO guest)
• CPU Threadripper 1950X
• Board X399
• PCIe cross-die PCIe 3.0

• 35.2 tok/s — generation speed (per request) · API server · 1 client
  Compared with 3.83 tok/s (split-KV PR versus stock, first pass); reported difference: engine build
  First pass at reported 32768-token depth; same checkpoint and harness
• 37.05 tok/s — generation speed (per request) · API server · 1 client
  Compared with 3.81 tok/s (split-KV PR versus stock, reverse-order pass); reported difference: engine build
  Second pass at reported 32768-token depth; warm-up discarded
• 51.81 tok/s — generation speed (per request) · API server · 1 client
  Compared with 37.04 tok/s (split-KV PR versus stock, first pass); reported difference: engine build
  First pass at reported 1024-token depth; same checkpoint and harness
• 51.43 tok/s — generation speed (per request) · API server · 1 client
  Compared with 37.76 tok/s (split-KV PR versus stock, reverse-order pass); reported difference: engine build
  Second pass at reported 1024-token depth; warm-up discarded

Not reported: configured context, system memory, Checkpoint digest, Exact container digest in summary, Native Windows applicability, Multi-client behavior, Broader task quality

• The tested split-KV PR is unmerged and not a stock vLLM feature.
• Linux VFIO test is not a native Windows deployment result.
• An intermediate depth had two distinct performance modes and is not represented as one stable value.
• Related cadamcat rows are the same host and author, not independent replication.

Setup guide (external)

Related link (external)

Reported on Reddit by EqualCryptographer67 Original report

Flash-Next two-GPU Vulkan host-offload report

Method (as reported): Individual short runs; Vulkan 2.51.0 is an author setup label, not the AMD driver or a verified engine build.

• Reported GPU setup RX 7900 XT 20GB
• GPU count 2
• Operating system Windows (native Windows 11)
• Backend Vulkan as author describes
• Driver AMD 32.0.31041.1004
• Model Qwen3.8-Flash-Next Unsloth UD-IQ4_XS (Unsloth)
• Actual input input length not reported
• Full window tested no
• Reported context Author says 256k configured context, mostly empty; exact integer and occupancy unreported.
• Simultaneous clients 1
• Topology dual-split (Author reports Vulkan tensor split 1:1 on two cards.)
• Speculative decoding MTP off
• Offload experts-to-host (--n-cpu-moe 28; PLE on CPU; precise residency unverified)
• CPU Ryzen 7 5800X3D
• Memory configuration 128 GB DDR4 reported; byte convention unreported
• Memory speed 3600 MT/s
• Board Gigabyte B550 EAGLE WIFI6
• PCIe XFX PCIe 4.0 x16; PowerColor chipset PCIe 3.0 x1
• Host notes Eight CPU threads reported; cards downclocked, -10% power.

• 5.5–6.5 tok/s — generation speed (per request) · 1 client
  Author's range over individual short host-offloaded runs; configured window mostly empty.
  This row: quantisation: Unsloth UD-IQ4_XS · speculative decoding: MTP off · topology: dual-split
• 3.3–5.7 tok/s — other (per request) · 1 client
  Author's including-prompt-processing range; exact timing formula unreported; not generation TPS.
  This row: quantisation: Unsloth UD-IQ4_XS · speculative decoding: MTP off · topology: dual-split

Not reported: engine, configured context, system memory, Exact engine build and model digest, Occupied prompt and physical host/GPU model residency, Output count, repeated-sample spread and response correctness

• Around 93.7 GB is the author's reported model size, not a verified GGUF file length.
• Host offload is involved; this is not an all-GPU FastLLM recipe or an occupied 256k-window result.
• The single-GPU ROCm arm changes backend, context, KV and offload, so no controlled ratio is available.
• Flash-Next has separate terms and is not in the approved FastLLM automatic catalog.

Setup guide (external)

Reported on Reddit by EqualCryptographer67 Original report

Flash-Next one-GPU ROCm host-offload report

Method (as reported): Individual short warm run; ROCm 2.49.0 is an author setup label, not the AMD driver or a verified engine build.

• Reported GPU setup RX 7900 XT 20GB; one GPU used on two-card host
• GPU count 1
• Operating system Windows (native Windows 11)
• Backend ROCm/HIP as author describes
• Driver AMD 32.0.31041.1004
• Model Qwen3.8-Flash-Next Unsloth UD-IQ4_XS (Unsloth)
• Actual input input length not reported
• Full window tested no
• Reported context Author says 8k context; exact integer and occupied prompt unreported.
• Simultaneous clients 1
• Topology single (One GPU warm run on host with two installed cards.)
• Speculative decoding MTP off
• Offload experts-to-host (--n-cpu-moe 42; PLE on CPU; precise residency unverified)
• CPU Ryzen 7 5800X3D
• Memory configuration 128 GB DDR4 reported; byte convention unreported
• Memory speed 3600 MT/s
• Board Gigabyte B550 EAGLE WIFI6
• PCIe XFX PCIe 4.0 x16; PowerColor chipset PCIe 3.0 x1
• Host notes Eight CPU threads reported; which GPU used unreported.

• 8.6 tok/s — generation speed (per request) · 1 client
  Author's single warm host-offloaded short run; no full-window occupancy.
  This row: quantisation: Unsloth UD-IQ4_XS · speculative decoding: MTP off
• 5.7 tok/s — other (per request) · 1 client
  Author labels this including prompt processing; exact timing formula unreported; not generation TPS.
  This row: quantisation: Unsloth UD-IQ4_XS · speculative decoding: MTP off

Not reported: engine, configured context, system memory, Exact serving card, engine build and model digest, Occupied prompt and physical host/GPU model residency, Output count, repeated-sample spread and response correctness

• Around 93.7 GB is the author's reported model size, not a verified GGUF file length.
• Host offload is involved; this is not an all-GPU FastLLM recipe.
• The dual-GPU Vulkan arm changes backend, context, KV and offload, so no controlled ratio is available.
• Flash-Next has separate terms and is not in the approved FastLLM automatic catalog.

Setup guide (external)