Local AI inference on AMD graphics cards

Less integration work between your GPU and useful local AI

A local inference service lets applications use a model running on the same computer. Bitworks is bringing together existing inference engines, model formats, hardware checks and repeatable configurations so users spend less time assembling a working deployment. FastLLM is our reference name for that integration package, not a new inference engine or a replacement for the upstream projects it uses.

The intended workflow checks the hardware at each start, selects a suitable versioned model/configuration, verifies the components and serves a local OpenAI-compatible API. That simplified deployment experience is still in development. The card-and-model combinations here are candidate tests, not assertions that a particular system will load, fit or perform well.

Choose by usable memory and your workload

A card’s advertised VRAM is not the amount available to a model. Quantization changes stored weight size, while context, active requests, compute buffers and the desktop use memory too. A smaller model file can leave room for those needs; it does not establish answer quality or speed. Each card page names a specific memory variant and an exact model, quantization and context candidate so evidence can be checked against the same configuration.

The 20 GB RX 7900 XT, 24 GB RX 7900 XTX and 32 GB Radeon AI PRO R9700 are larger-memory candidates for Qwen3.8-27B. Qwen publishes the base model; each recipe identifies its separate third-party GGUF conversion. RX 9060 XT 8 GB and 16 GB variants have separate pages because they should not share a fit assumption. Evidence for one card does not automatically transfer to another.

Single-user response time and concurrent agents need different tests

A responsive chat, long-document analysis and several subagents working at once create different demands. Our benchmark methodology explains the planned 1/2/4/8-request tests, useful occupied context lengths, latency and failures, answer quality, host-system disclosure and source reproducibility. Every applicable catalog model belongs in that coverage, with missing or outside-policy cases identified.

Two cards can split one larger model or host independent copies for concurrent requests. Those are different configurations; neither guarantees faster responses for one user. More API clients are also not the same as more server processing slots. Host-memory offload is another separate experiment, not a full-GPU recipe.

Windows and Linux are separate assessments

Windows integration currently uses a supervised lab path with an AMD Vulkan driver and Microsoft Visual C++ x64 runtime prerequisites. It is not a released consumer installer, and candidate recipes have no approved card outcome or public performance result. Driver/prerequisite review remains separate from ordinary inference startup.

Linux has a source-checkout workflow for supervised assessment. Physical AMD/Linux serving has not been qualified by this project. Windows and Linux must be assessed independently, and neither has a public install button here.

Qwen3.8-Flash-Next has separate Qwen Community 1.0 terms, and Strata is a separate engine. Their host-offload work is research, not an approved integration recipe or Bitworks performance result.

Inspect the integration and benchmark source

The Bitworks AMD inference repository contains development source and benchmark tooling for review. It builds on upstream engines rather than creating a new one. Source publication does not qualify memory fit, operation placement, answer quality or performance; it also does not release a consumer installer. The integration-code license remains an owner decision, separate from the upstream component and model licenses.