Programmer manages to execute 2.8-trillion-parameter AI on basic hardware
A developer created a unique software solution that allows the Kimi K3 AI model, which features 2.8 trillion parameters, to operate on a regular PC equipped with only 8 GB of RAM and one CPU. Although the setup is functional, it is extremely slow, taking more than 30 seconds to produce just one token of text. Despite the minimal RAM usage, the system demands 1.7 TB of SSD space to store the model files.
by shortkt.com✓