# T4-Bench: full benchmark data Source: https://t4b.sdad.pro/benchmarks.html Hardware: Kaggle 2x Tesla T4 15 GB, 31 GB RAM, 4 vCPU, CUDA 13. Source notes: https://blog.sdad.pro/ The figures come from saved runs on the same machine. ## Chat and coding models | Model | Reply tok/s | Prompt tok/s | VRAM GB | GPUs | |---|---|---|---|---| | LiquidAI LFM2.5-8B-A1B | 119.0 | 800.0 | 5.7 | 1x T4 | | XHToken Spark-X2.5-1.7B | 100.0 | n/a | 1.3 | 1x T4 | | Qwen3-Coder-30B-A3B | 71.0 | 165.0 | 22.0 | 2x T4 split | | XHToken Spark-X2.5-4B | 54.0 | 1385.0 | 3.7 | 1x T4 | | google Gemma 4 12B IT | 21.0 | 144.0 | 8.1 | 1x T4 | | Qwen3.8-27B | 13.0 | 52.0 | 17.7 | 2x T4 split | ## Two-agent concurrency | Agents | Per agent | Aggregate | |---|---|---| | 1.0 | 71 tok/s | 11 tok/s (load-dominated wall) | | 2.0 | 56 + 52 tok/s | 92 tok/s over 2.3s wall | ## Tool-use evaluation | Metric | Score | |---|---| | Passed / partial / failed | 9 / 4 / 2 | | Points | 22/30 | | Quality | 73/100 | | Responsiveness | 74/100 (median turn 1.5s) | | Deployability | 73/100 | | Weakest category | Multi-Step Chains (33%) | | Safety | 1 warning (TC-08: batched tools before reading results) | ## Spark agent tasks | Task | 1.7B | 4B | |---|---|---| | T1 implement `median()` to pass 5 pytest tests | PASS, 5 steps, 1 tool error, ran tests | PASS, 4 steps, 0 errors, ran tests | | T2 debug repo (binary-search off-by-one across 2 files), 7 tests | PASS, 3 steps, 5 errors, ran tests | PASS, 14 steps, 9 errors, never ran the suite itself (final state correct at step limit, verification gap) | | T3 implement LRU cache from spec from scratch, 5 tests | FAIL, 3 tool steps + 17 no-call turns, never wrote a working file | PASS, 5 steps, 1 error, ran tests | | Agentic score | 2/3 | 3/3 | ## Ternary 27B, one T4 | Packing | Prompt tok/s | Reply tok/s | |---|---|---| | PQ2_0 | 293.9 | 15.1 | | PTQ1_0 | 200.3 | 15.4 | ## Ternary 27B, two T4s | Packing | Prompt tok/s | Reply tok/s | |---|---|---| | PQ2_0 | 313.0 | 22.9 | ## Fine-tuning | Method | Result | Peak GB | Loss | |---|---|---|---| | unsloth_native | completed | 6.9 | 0.829 | | soup backend: unsloth | completed | 6.9 | 0.8044 | | soup backend: transformers (unsloth installed) | completed | 14.5 | 0.8058 | | soup backend: transformers (unsloth blocked) | OOM | 14.5 | n/a | ## Image generation | Model | Seconds/image | Images/hour | VRAM GB | Quality score | |---|---|---|---|---| | FLUX.2 klein 4B | 23.7 | 374 | 4 | 39 | | Qwen-Image 2.1 Viggle 4-step | 64.0 | 150 | 6 | 38 | | Z-Image-Turbo | 47.6 | 147 | 5 | 38 | | SDXL 1.0 | 63.5 | 107 | 7 | 39 | ## Video generation | Configuration | Frames | Seconds/clip | Total seconds | VRAM GB | Quality score | |---|---|---|---|---|---| | cogvideox-2b | 49 | 1053.0 | 1053.0 | 11 | 26 | | ltx2b-distilled | 121 | 131.4 | 131.4 | 11 | 23 | | ltx2b-warm3 | 121 | 49.2 | 270.4 | 11 | 20 | | ltx2b-warm3-s4 | 49 | 27.6 | 207.7 | 11 | 21 | ## MiniMax-H3 memory | Precision | GB | Fits one T4 | |---|---|---| | All BF16 | 126.0 | no | | All FP8 | 64.1 | no | | All 4-bit | 33.1 | no | | One T4 VRAM | 15.0 | yes |