How To Run Deepseek R1 671b Fully Locally On a $2000 EPYC Server (digitalspaceport.com)
from yogthos@lemmy.ml to technology@lemmy.ml on 01 Aug 2025 16:50
https://lemmy.ml/post/34006601

A note that this setup runs a 671B model in Q4 quantization at 3-4 TPS, running a Q8 would need something beefier. To run a 671B model in the original Q8 at 6-8 TPS you’d need a dual socket EPYC server motherboard with 768GB of RAM.

#technology