Gil Press
DeepSeek spells the end of the dominance of Big Data and Big AI, not the end of Nvidia. Its focus on efficiency jump-starts the race for small AI models based on lean data, consuming slender computing resources. The probable impact of DeepSeek’s low-cost and free state-of-the-art AI model will be the reorientation of U.S. Big Tech away from relying exclusively on their “bigger is better” competitive orientation and the accelerated proliferation of AI startups focused on “small is beautiful.”
Most of the coverage of DeepSeek and all of Wall Street’s reaction focused on its claim of developing an AI model that performs as well as leading U.S. models at a fraction of the training cost. Beyond being “compute-efficient” and using a relatively small model (derived from larger ones), however, DeepSeek’s approach is data-efficient.
DeepSeek engineers collected and curated a training dataset consisting of “only” 800,000 examples (600,000 reasoning-related answers), demonstrating how to transform any large language model into a reasoning model. Anthropic’s Jack Clark called this “the most underhyped part of this [DeepSeek model]
No comments:
Post a Comment