https://kukie.tech/tags/kv-cache/Recent content in Kv-Cache on HistoryHugoko-krMon, 26 Oct 2026 02:00:00 +0900https://kukie.tech/posts/kukie-serving-01/Mon, 26 Oct 2026 02:00:00 +0900https://kukie.tech/posts/kukie-serving-01/Measuring how many concurrent requests one NVIDIA L40S serves within a latency SLA before building a GPU cluster — the SLA draft as TTFT and inter-token latency, a single g6e.xlarge with two stop guards, a measurement script split from the server so it can run as a Kubernetes Job, results for four open models, why MoE models outrun a dense 27B on memory bandwidth, KV cache FP8 for 20% more headroom, and two review fixes.