Estimate — sign in for exact rates.

 Trusted GPU & AI infrastructure since 2015 — Huntsville, AL
AMD Inference

Local AI inference on AMD graphics cards

Learn how AMD hardware, existing inference engines, model choices and reproducible tests come together in practical local AI recipes.

Download FastLLM Preview 2026.11.12.2

FastLLM Preview 2026.11.12.2 is a signed Windows release for AMD Radeon graphics (a full release on GitHub, not a pre-release). It currently sets up the AMD Radeon RX 7900 XTX 24GB and AMD Radeon RX 7900 XT 20GB. It is not a qualified release, and its recipes remain candidates.

New in this release: Connect an app, a settable local port, an optional Thinking Off, an optional local chat page, model removal, a prompt-free run log and support bundle, and a model for 4 GB cards.

Get FastLLM Preview 2026.11.12.2 on GitHub

On the release page, download FastLLM-Setup-2026.11.12.2.exe. The publisher shows as Bitworks LLC.
SHA-256 08f37c9cce1421700f34f34a63c16045531762d582390d2591b9afdbf005b7b3

Test results for this release →

  • Microsoft Edge may say the file isn’t commonly downloaded while the publisher builds reputation.
  • Once installed, FastLLM shows an update banner when a newer version is available.
  • If you have 2026.10.21.12, FastLLM offers this update.

Use FastLLM from your apps

FastLLM runs an AI model on your Radeon card and lets other programs on the same computer talk to it. Apps that let you set a custom, OpenAI-style address may be able to connect; FastLLM has not tested each app. Everything stays on your computer: the address starts with 127.0.0.1, which only your own PC can reach.

What an app needs

Press Connect an app in the FastLLM window. It shows four things, each with a Copy button:

  • Base URL. Normally http://127.0.0.1:8080/v1. If you changed the port in Settings, it shows your port.
  • Model name. The name FastLLM is serving right now. Paste it into your app exactly.
  • Context limit. How much text one request can hold, counted in tokens. This depends on your card and the setup FastLLM picked. A request longer than the limit is refused with an error.
  • API key. FastLLM sets none, so any text works where an app insists on a key. (This is FastLLM's own fact, not something an app's documentation says.)

A plain word about coding agents

Coding agents such as Claude Code, Codex, Cline and Roo Code often send very long requests. If your setup's context limit is under 32,768 tokens, FastLLM tells you so in the window, and those long requests will be refused with an error. A request and its answer together must fit within the limit; anything longer is refused. If FastLLM is not running, the window says the limit is not known yet: "Coding agents often send requests longer than 32,768 tokens; check the limit after FastLLM starts."

Ready-made settings for popular apps

The window has a settings card to copy for each of these. Every card follows the app's own documentation, and the window links to it:

  • Any OpenAI-compatible app
  • Open WebUI
  • Continue
  • Cline and Roo Code
  • Claude Code
  • Codex

We have not tested how well each app behaves with a local model, and we make no promise about it. Claude Code in particular is designed for Anthropic's Claude models; results with a local model may vary.

What the server offers

AddressWhat it is
/v1/chat/completionsOpenAI-style chat
/v1/completionsOpenAI-style text completion
/v1/modelslist of models
/v1/messagesAnthropic-style
/v1/responsespartial at this engine version

"Partial at this engine version" means the engine behind FastLLM only implements part of the Responses format, so some Responses-based apps may not work fully. Codex uses this format. The Codex card follows the Codex documentation's local provider example (its base_url also ends in /v1, with no key setting); wire_api = "responses" is the setting the documentation's proxy and Azure examples use, and it is there because FastLLM serves the Responses API at /v1/responses.

Check that it answers

Check the API formats (in the Connect an app window) sends one short, fixed test message to the chat, messages and responses addresses and shows pass or fail for each (a pass means the reply has the expected structure; its text is never read), with the time it took. It runs only when you press it, needs FastLLM to be Ready, and keeps no text: FastLLM does not store your test message or the reply, only the pass/fail result and a few numbers.

Less integration work between your GPU and useful local AI

A local inference service lets applications use a model running on the same computer. Bitworks is bringing together existing inference engines, model formats, hardware checks and repeatable configurations so users spend less time assembling a working deployment. FastLLM is our reference name for that integration package, not a new inference engine or a replacement for the upstream projects it uses.

The intended workflow checks the hardware at each start, selects a suitable versioned model/configuration, verifies the components and serves a local OpenAI-compatible API. That simplified deployment experience is still in development. The card-and-model combinations here are candidate tests, not assertions that a particular system will load, fit or perform well.

Choose by usable memory and your workload

A card’s advertised VRAM is not the amount available to a model. Quantization changes stored weight size, while context, active requests, compute buffers and the desktop use memory too. A smaller model file can leave room for those needs; it does not establish answer quality or speed. Each card page names a specific memory variant and an exact model, quantization and context candidate so evidence can be checked against the same configuration.

The 20 GB RX 7900 XT, 24 GB RX 7900 XTX and 32 GB Radeon AI PRO R9700 are larger-memory candidates for Qwen3.8-27B. Qwen publishes the base model; each recipe identifies its separate third-party GGUF conversion. RX 9060 XT 8 GB and 16 GB variants have separate pages because they should not share a fit assumption. Evidence for one card does not automatically transfer to another.

Single-user response time and concurrent agents need different tests

A responsive chat, long-document analysis and several subagents working at once create different demands. Our benchmark methodology explains the planned 1/2/4/8-request tests, useful occupied context lengths, latency and failures, answer quality, host-system disclosure and source reproducibility. Every applicable catalog model belongs in that coverage, with missing or outside-policy cases identified.

Two cards can split one larger model or host independent copies for concurrent requests. Those are different configurations; neither guarantees faster responses for one user. More API clients are also not the same as more server processing slots. Host-memory offload is another separate experiment, not a full-GPU recipe.

Windows and Linux are separate assessments

Windows integration currently uses a supervised lab path with an AMD Vulkan driver and Microsoft Visual C++ x64 runtime prerequisites. It is not a released consumer installer, and candidate recipes have no approved card outcome or public performance result. Driver/prerequisite review remains separate from ordinary inference startup.

Linux has a source-checkout workflow for supervised assessment. Physical AMD/Linux serving has not been qualified by this project. Windows and Linux must be assessed independently, and neither has a public install button here.

Qwen3.8-Flash-Next has separate Qwen Community 1.0 terms, and Strata is a separate engine. Their host-offload work is research, not an approved integration recipe or Bitworks performance result.

Inspect the integration and benchmark source

The Bitworks AMD inference repository contains development source and benchmark tooling for review. It builds on upstream engines rather than creating a new one. Source publication does not qualify memory fit, operation placement, answer quality or performance; it also does not release a consumer installer. The integration-code license remains an owner decision, separate from the upstream component and model licenses.