After selling a crypto data company that relied heavily on cloud processing, the focus has shifted to building AI infrastructure that operates locally. This approach, using a NAS with an eGPU, prioritizes data privacy by ensuring information never leaves the local environment, even though it may not be cheaper or faster for large models. As AI technology evolves, a divide is anticipated between those who continue using cloud-based AI and a growing segment of users—such as developers and privacy-conscious individuals—who prefer running AI models on their own hardware. The current setup with Ollama on an RTX 4070 12GB demonstrates that mid-sized models are now practical for everyday use, highlighting the increasing viability of local-first AI. This matters because it addresses the growing demand for privacy and control over personal and sensitive data in AI applications.
