Distributed operating system for ML resources
A distributed operating system for managing machine learning resources, coordinating the entire memory hierarchy, networking, and compute hierarchy. The project explores upcycling end-of-life NVIDIA GPUs (K80, M40, P100), offloading MoE expert calculations to MCUs with NPUs and possibly FPGAs, and managing a 10G LAN-attached network of older Xeon servers with GPUs and USB-attached MCUs as a machine learning fabric. It also investigates bypassing end-of-life CUDA driver support for older NVIDIA hardware using Rust/Vulkan and applying ideas from IBM's ACP/TPF operating system and Workload Manager, plus architectures like Freetoken (MoE caching/routing, Engram, KV-cache compression) and other efficient-modeling approaches.
Shared over time
Each entry reflects what was shared on that date. Details may have changed since then.
Distributed operating system for ML resources
A distributed operating system for managing machine learning resources, coordinating the entire memory hierarchy, networking, and compute hierarchy. The project explores upcycling end-of-life NVIDIA GPUs (K80, M40, P100), offloading MoE expert calculations to MCUs with NPUs and possibly FPGAs, and managing a 10G LAN-attached network of older Xeon servers with GPUs and USB-attached MCUs as a machine learning fabric. It also investigates bypassing end-of-life CUDA driver support for older NVIDIA hardware using Rust/Vulkan and applying ideas from IBM's ACP/TPF operating system and Workload Manager, plus architectures like Freetoken (MoE caching/routing, Engram, KV-cache compression) and other efficient-modeling approaches.
Posted by @softwarewright · Original comment ↗
“I am working on developing a distributed operating system for managing machine learning resources (the entire memory hierarchy, networking and compute hierarchy).”