Google and Amazon launch joint assault on NVIDIA's AI dominance

Google has announced the open-source availability of TorchTPU, its version of PyTorch, marking a significant move to challenge Nvidia's extensive dominance in the AI industry. PyTorch, a popular open-source toolkit developed by Meta (formerly Facebook) in 2016, is widely used by AI app developers and startups for building, training, and running AI models, with estimates suggesting its use in 85% of AI applications. Historically, PyTorch's features were designed to work specifically with Nvidia's GPUs, which were the standard hardware for AI research at the time. In 2022, Meta transitioned PyTorch to the Linux Foundation, an organization that oversees numerous critical open-source projects.

TorchTPU enables AI applications to run on Google’s proprietary Tensor Processing Units (TPUs) and facilitates the effortless migration of existing PyTorch apps to TPUs and, consequently, to Google Cloud. Developers can achieve this with a simple configuration adjustment, eliminating the need for extensive code rewriting.

This announcement follows a similar initiative from Amazon, which recently released TorchNeuron, its own PyTorch variant. TorchNeuron allows AI applications to be ported to Amazon's homegrown AI chip, Trainium, and to AWS without any rewriting. Amazon is also actively working to further diminish Nvidia's influence on PyTorch by submitting code to the main PyTorch project designed to decouple various features from Nvidia's chips, thereby promoting broader hardware compatibility by default.

Google appears to be taking additional steps beyond Amazon's efforts. The company stated that it collaborated directly with key industry players such as Meta, Hugging Face, Inferact (the creator of vLLM), and RadixArk (the maker of SGLang) to ensure that TorchTPU's features and code libraries seamlessly integrate with these widely used AI development tools, mirroring the original PyTorch's functionality. Furthermore, Google has introduced a dedicated utility, the TPU agent suite, to assist developers in migrating their applications to Google's chips and cloud, including error resolution support. The search giant also updated its popular LiteRT tool, which supports AI execution on mobile phones, laptops, and embedded devices, to enhance its compatibility with PyTorch AI applications.

While these developments do not guarantee an immediate mass exodus of AI app developers from Nvidia's hardware, as they now have easier access to alternative chips and cloud platforms, both Amazon and Google are clearly making concerted efforts to dismantle Nvidia's long-standing competitive advantage in the AI hardware ecosystem.