wrkgn.
← All projects
Project record01 mention
Originally shared on Hacker News

32B Q4 LLM on a standard smartphone

This project runs a 32B-parameter quantized (Q4) large language model on a standard smartphone, streaming MoE weights from SSD/UFS storage and tiering memory across RAM/UFS/microSD on Android without root access.

Last shared
01 / Source history

Shared over time

Each entry reflects what was shared on that date. Details may have changed since then.

  1. 32B Q4 LLM on a standard smartphone

    This project runs a 32B-parameter quantized (Q4) large language model on a standard smartphone, streaming MoE weights from SSD/UFS storage and tiering memory across RAM/UFS/microSD on Android without root access.

    Posted by @nolon · Original comment ↗

    “I'm trying to run a 32B Q4 model on a standard smartphone. It's a pretty interesting project.”