BENCORP GPU Ladder
Open coding-agent profiles, sorted by cost and proof.
This is a public notebook for model fit experiments: which open models run on which GPUs, what quality variables were preserved, what failed, and what a practical operator can reproduce.
The Shape
A useful local-agent profile is not just a model that loads. It has to preserve the quality variables that matter, clear a wall-clock speed floor, and survive enough task evidence to be worth using.
The ladder starts with the hardware people actually ask about: RTX 3090, A100, L40S or RTX 6000 Ada, then H100/H200 as the reference ceiling. The default floor is 5 client-observed wall tokens per second. Above that floor, quality wins.
Current Rungs
| GPU | Profile | Status | Read |
|---|---|---|---|
| RTX 3090 24GB | Ornith 1.0 35B Q5_K_S, 128K, q8 K / turbo2 V |
Runtime-proven. Quality evidence is useful but conflicting. | Report |
| RTX 3090 24GB | Qwen3.6 27B abliterated, likely Q5/Q6 lane | Next experiment. Stronger base-card coding metrics than the 35B-A3B sibling. | Plan |
| A100 40GB | Qwen3.6 27B higher-fidelity profile | Planned cost/intelligence comparison. | Pending |
| 4x A100 80GB | GLM-5.2 abliterated UD_Q3_K_M, 128K, q8 K / turbo2 V |
Profiling-proven CUDA control: 14.06 wall tok/s at 1k, 10.80 wall tok/s at 100k target. | Report |
| H100 / H200 | Qwen3.6 27B Q8 and larger candidates | Reference baseline exists internally; public report pending. | Pending |
What Counts As Proof
Fit
Exact model file, quant, context, KV cache format, placement, and observed VRAM.
Speed
Server decode tokens per second and wall tokens per second, especially at long-context decode.
Quality
Public benchmarks where available, plus agent-task results when the public harness does not answer the operator question.
Safety Boundary
These pages publish safe summaries: model links, runtime settings, costs, timings, benchmark summaries, and high-level conclusions. They do not publish private task traces, raw red-team artifacts, credentials, or sensitive model outputs.