(C) Alec Muffett's DropSafe blog.
Author Name: Alec Muffett
This story was originally published on allecmuffett.com. [1]
License: CC-BY-SA 3.0.[2]


“Running DeepSeek on a small PC is the new ‘Running DOOM on a calculator’”

2025-01-31 14:10:44+00:00

Very true. It’s going to be interesting watching the world adapt to this.

No, not a distill, but a 2bit quantized version of the actual 671B model (102XXS), about 200GB large, running on a 14900K with 96GB DDR5 6800 and a single 3090 24GB (with 5 layers offloaded), and for the rest running off of PCle 4.0 SSD (Samsung 990 pro)

Although of limited actual usefulness, it’s just amazing that is actually works! With larger context it takes a couple of minutes just to process the prompt, token generation is actually reasonably fast.
[END]

[1] URL: https://alecmuffett.com/article/111081
[2] URL: https://creativecommons.org/licenses/by-sa/3.0/

DropSafe Blog via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/alecmuffett/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.