May provide support to management for the cost savings of running local LLMs. The paper also includes amortization costs for the GPUs. I was surprised by the findings and the short break even time with cloud api costs.
https://arxiv.org/abs/2601.09527
💬 Discussion r/LocalLLaMA (41 points, 24 commentaires)