Amazon is making a substantial investment in training a high-capacity large language model (LLM) with the ambition of competing with leading models developed by OpenAI and Alphabet, according to sources familiar with the matter.
The project, internally known as “Olympus,” features a model with 2 trillion parameters, potentially ranking it among the largest models currently in training. For comparison, OpenAI’s GPT-4, recognized as one of the top models, has 1 trillion parameters. The details of the project are not yet public, and those providing the information requested anonymity.
Rohit Prasad, former head of Alexa, is leading the team responsible for the ambitious AI model. As the head scientist of artificial general intelligence (AGI) at Amazon, Prasad reports directly to CEO Andy Jassy. He has brought in researchers who previously worked on Alexa AI and the Amazon science team to collaborate on training models, consolidating AI efforts across the company with dedicated resources.
While Amazon has already trained smaller models like Titan, the company is taking on the challenge of developing larger models to enhance its offerings, particularly on Amazon Web Services (AWS). Amazon believes that having proprietary models could increase the appeal of its services on AWS, meeting the demands of enterprise clients seeking access to high-performing models. The sources mentioned that there is no specific timeline for the release of the new model.
Large language models serve as the foundational technology for AI tools that learn from extensive datasets to generate responses resembling human language. However, training larger AI models is more costly due to the significant computing power required.
During an earnings call in April, Amazon executives expressed the company’s commitment to increasing investment in LLMs and generative AI while reducing investment in fulfillment and transportation within its retail business. Amazon declined to comment on the reported project.