https://kukie.tech/tags/gpu/Recent content in Gpu on HistoryHugoko-krTue, 03 Nov 2026 10:00:00 +0900https://kukie.tech/posts/kukie-serving-03/Tue, 03 Nov 2026 10:00:00 +0900https://kukie.tech/posts/kukie-serving-03/Building kukie-oven, a Go operator that turns one ServedModel into a PVC, a download Job and a vLLM Deployment — the reconcile order, two server-side-apply field managers, telling someone else’s object from a leftover, Helm and GHCR packaging, an EBS CSI role limited by a tag boundary, and the first real run on an L40S: Ready in 14 minutes 11 seconds with no manual step.https://kukie.tech/posts/kukie-serving-02/Tue, 03 Nov 2026 09:00:00 +0900https://kukie.tech/posts/kukie-serving-02/Why an external LLM API plan became a Helm add-on for users’ GPU clusters — quality and cost numbers behind the pivot, three layers split by required permissions, a chart with per-layer switches and a preflight job, the measurement-to-429 loop, an honest comparison with KubeAI and LiteLLM, and what is built versus still planned.https://kukie.tech/posts/kukie-serving-01/Mon, 26 Oct 2026 02:00:00 +0900https://kukie.tech/posts/kukie-serving-01/Measuring how many concurrent requests one NVIDIA L40S serves within a latency SLA before building a GPU cluster — the SLA draft as TTFT and inter-token latency, a single g6e.xlarge with two stop guards, a measurement script split from the server so it can run as a Kubernetes Job, results for four open models, why MoE models outrun a dense 27B on memory bandwidth, KV cache FP8 for 20% more headroom, and two review fixes.https://kukie.tech/posts/triton-mlir/Tue, 10 Mar 2026 10:00:00 +0900https://kukie.tech/posts/triton-mlir/MLIR 기반 GPU 커널 컴파일러인 Triton이 뭔지, CUDA랑 뭐가 다른지, 내부 구조는 어떻게 생겼는지 정리https://kukie.tech/wiki/kv-cache/Mon, 01 Jan 0001 00:00:00 +0000https://kukie.tech/wiki/kv-cache/LLM 이 이미 읽은 토큰의 Key·Value 를 GPU 메모리에 저장해 두는 캐시. 요청마다 토큰 수만큼 자라며, 남은 GPU 메모리가 동시에 받을 수 있는 요청 수를 정한다. PagedAttention 과 FP8 저장으로 자리를 아낀다.https://kukie.tech/wiki/moe/Mon, 01 Jan 0001 00:00:00 +0000https://kukie.tech/wiki/moe/Mixture of Experts. 트랜스포머의 FFN 을 여러 전문가로 나누고 라우터가 토큰마다 몇 개만 골라 계산하는 구조. 전체 파라미터는 크지만 토큰 하나에 쓰는 활성 파라미터가 작아, 메모리 대역폭에 묶이는 decode 가 빠르다.