REPRODUCIBLE LOCAL INFERENCE
Local LLM Systems Lab
Measured notes on GGUF artifacts, llama.cpp and AMD ROCm inference.
These numbers are measured on one host and one workload. They are not a universal model ranking or a prediction for visitor hardware.
Experimental platform
CPUAMD Ryzen AI MAX+ 395
GPURadeon 8060S, gfx1151
OSUbuntu 24.04.4 LTS
BackendHIP / ROCm 7.2
Runtimellama.cpp build 10626
ConcurrencyOne parallel slot
Experiment registry
Results are loaded from the versioned experiments.json dataset. Each record is a measured experiment with its host and runtime context, not a cross hardware ranking.
Selected measurements
| Model | Prompt tok/s | Generation tok/s | Prompt time | Generation time | SHA256 |
|---|
Validation controls
- Official Nemotron Q4_0 SHA256 was verified before diagnosis.
- A metadata only diagnostic copy passed CPU, partial HIP and full HIP loading.
- Bartowski Q4_K_M passed full HIP CLI generation and HTTP serving.
- Coding and structured tool calling smoke tests passed for the candidate.
- MTP was disabled and remains a separate unvalidated experiment.
Limitations
This is a narrow systems measurement, not a broad quality benchmark. It does not measure concurrent users, long context behavior, power use, thermal behavior, Vulkan performance or MTP throughput.