A user deployed AI inference on a NAS without a discrete GPU, achieving 5 tokens per second. Despite initial skepticism about CPU-only performance, the setup proved functional for basic tasks. The NAS, already serving as a network-attached storage device, was repurposed with Ollama and Open WebUI to host AI models.

The experiment involved installing Ollama on the NAS and connecting it to Open WebUI. The user selected smaller 4B models, such as Qwen3 4B and Gemma 3 4B, to avoid overloading the hardware. This approach minimized resource strain while still enabling practical AI applications.

The NAS handled the AI workload at a rate of 5 tokens per second, which, while modest, was sufficient for lightweight tasks. The user mapped port 11434 for Ollama and port 3000 for Open WebUI, allowing remote access to the AI models. This setup demonstrated that even low-end hardware can support AI inference with the right configuration.

The results highlight the potential for repurposing existing hardware for AI tasks, reducing costs associated with dedicated AI infrastructure. However, the performance limitations of CPU-only inference may lead to higher latency and scalability challenges. Users must also consider vendor lock-in and governance issues when deploying AI on non-traditional platforms.

This experiment suggests that AI inference on NAS devices can be viable for specific use cases, particularly where cost and resource efficiency are priorities. While not suitable for high-performance applications, it opens new possibilities for deploying AI on existing hardware without significant investment.