T4-BENCH

All benchmark data

Every number below is measured on the same Kaggle notebook: 2x Tesla T4 15 GB, 31 GB RAM, 4 vCPU, CUDA 13. Source notes: blog.sdad.pro.

Chat and coding models

ModelReply tok/sPrompt tok/sVRAM GBGPUs
LiquidAI LFM2.5-8B-A1B119.0800.05.71x T4
XHToken Spark-X2.5-1.7B100.0n/a1.31x T4
Qwen3-Coder-30B-A3B71.0165.022.02x T4 split
XHToken Spark-X2.5-4B54.01385.03.71x T4
google Gemma 4 12B IT21.0144.08.11x T4
Qwen3.8-27B13.052.017.72x T4 split

Two-agent concurrency

AgentsPer agentAggregate
1.071 tok/s11 tok/s (load-dominated wall)
2.056 + 52 tok/s92 tok/s over 2.3s wall

Tool-use evaluation

MetricScore
Passed / partial / failed9 / 4 / 2
Points22/30
Quality73/100
Responsiveness74/100 (median turn 1.5s)
Deployability73/100
Weakest categoryMulti-Step Chains (33%)
Safety1 warning (TC-08: batched tools before reading results)

Spark agent tasks

Task1.7B4B
T1 implement `median()` to pass 5 pytest testsPASS, 5 steps, 1 tool error, ran testsPASS, 4 steps, 0 errors, ran tests
T2 debug repo (binary-search off-by-one across 2 files), 7 testsPASS, 3 steps, 5 errors, ran testsPASS, 14 steps, 9 errors, never ran the suite itself (final state correct at step limit, verification gap)
T3 implement LRU cache from spec from scratch, 5 testsFAIL, 3 tool steps + 17 no-call turns, never wrote a working filePASS, 5 steps, 1 error, ran tests
Agentic score2/33/3

Ternary 27B, one T4

PackingPrompt tok/sReply tok/s
PQ2_0293.915.1
PTQ1_0200.315.4

Ternary 27B, two T4s

PackingPrompt tok/sReply tok/s
PQ2_0313.022.9

Fine-tuning

MethodResultPeak GBLoss
unsloth_nativecompleted6.90.829
soup backend: unslothcompleted6.90.8044
soup backend: transformers (unsloth installed)completed14.50.8058
soup backend: transformers (unsloth blocked)OOM14.5n/a

Image generation

ModelSeconds/imageImages/hourVRAM GBQuality score
FLUX.2 klein 4B23.7374439
Qwen-Image 2.1 Viggle 4-step64.0150638
Z-Image-Turbo47.6147538
SDXL 1.063.5107739

Video generation

ConfigurationFramesSeconds/clipTotal secondsVRAM GBQuality score
cogvideox-2b491053.01053.01126
ltx2b-distilled121131.4131.41123
ltx2b-warm312149.2270.41120
ltx2b-warm3-s44927.6207.71121

MiniMax-H3 memory

PrecisionGBFits one T4
All BF16126.0no
All FP864.1no
All 4-bit33.1no
One T4 VRAM15.0yes