I built a private AI mini-cluster with Framework Desktop
Jeff Geerling
Aug 7, 2025 15:00
Create an account to hide videos, mark them watched, or save them to Don't Miss.
Create an account to hide videos, mark them watched, or save them to Don't Miss.
Create an account to hide videos, mark them watched, or save them to Don't Miss.
Chapters
Can we build a private AI cluster in a mini rack with Framework's Desktop motherboard?
Mentioned in this video:
- Framework Desktop: https://frame.work/desktop
- Level1Techs review of Framework Desktop: https://www.youtube.com/watch?v=L-xgMQ-7lW0
- Beowulf AI Cluster project: https://github.com/geerlingguy/beowulf-ai-cluster
- Blog post with all test results: https://www.jeffgeerling.com/blog/2025/i-clustered-four-framework-mainboards-test-huge-llms
- Other test results and benchmarks for the Framework Mainboard: https://github.com/geerlingguy/sbc-reviews/issues/80
- DeskPi RackMate T1 (Black - US): https://amzn.to/456iuXv
- DeskPi RackMate T1 (Black - EU): https://www.amazon.de/dp/B0FBFDZD4C?th=1
- DeskPi RackMate 2U Mini ITX Tray: https://frame.work/products/deskpi-rackmate-10-inch-2u-mini-itx-shelf
Framework sent the units used in this review, and DeskPi sent a mini rack and four rack trays to mount the Framework hardware. Neither company paid me, nor had any input into my testing or the production of the video. See my review video policies: https://github.com/geerlingguy/youtube?tab=readme-ov-file#review-videos
Support me on Patreon: https://www.patreon.com/geerlingguy
Sponsor me on GitHub: https://github.com/sponsors/geerlingguy
Merch: https://www.redshirtjeff.com
2nd Channel: https://www.youtube.com/@GeerlingEngineering
3rd Channel: https://www.youtube.com/@Level2Jeff
Contents:
Science & Technology
#homelab
#ai
#llm
#artificial
#intelligence
#chatgpt
#llama
#ollama
#dllama
#distributed
#cluster
#supercomputer
#private
#mini
#rack
#enterprise
#training
#model
#deskpi
#rackmount
#5g
#edge
#k8s
#kubernetes
#vllm
#rpc
#llama.cpp
#cpp
#mistral
#qwen
#performance
#benchmark
#comparison
#speed
#test
#testing
#setup
#how-to
#guide
#beowulf
#networking
#network
#thunderbolt
#inference
#tensor
#tensors
#loading
#memory
#ram
#vram
#gtt
#amd
#strix
#halo
#max
#395