Google DeepMind has developed a new method for training artificial intelligence that promises to significantly improve the efficiency of AI systems and reduce energy consumption in the AI field. This innovative technology could address growing concerns about the environmental impact of AI data centers.
JEST: A New Approach to AI Training
Google’s DeepMind research lab has unveiled an innovative method for training artificial intelligence models called JEST (Joint Example Selection), which could lead to dramatic changes in the field of AI. According to the published study, the new technology provides a 13-fold reduction in the number of training iterations and a 10-fold reduction in power consumption compared to existing methods, as reported by Tom’s Hardware. In other words, AI can be trained an order of magnitude faster and more efficiently.
JEST differs from traditional approaches in that it learns from whole batches of data rather than individual parts. JEST first builds a smaller AI model that evaluates the quality of data from sources and ranks packages by quality. It then compares its score with a set of lower quality data. JEST then determines which packages are most suitable for training, and then the larger model is trained based on the best data selected by the smaller model.
A key factor in the success of JEST is the use of high-quality, carefully curated datasets. This makes the method particularly demanding on initial information and may limit its use by amateurs and non-professional developers.
Addressing Environmental Concerns
Interestingly, the emergence of JEST coincided with growing concerns about the power consumption of AI systems. According to the researchers, AI workloads consumed about 4.3 GW of electricity in 2023, which is comparable to the annual consumption of Cyprus. Moreover, a single ChatGPT query consumes 10 times more energy than a Google search query.
Experts note that the new technology can be used in two ways: to reduce energy consumption while maintaining current performance, or to achieve maximum productivity at the same level of energy consumption. The choice of direction will depend on company priorities and market trends. We’ll keep you updated on how this unfolds.
The implementation of JEST could have a significant impact on the AI industry, given the high cost of training current models, notes NIX Solutions. For example, training costs for GPT-4 are estimated at $100 million, and future models may require even greater investments. Thus, the JEST method presented by Google DeepMind opens up fundamentally new opportunities for increasing efficiency and reducing costs in AI technology. The practical application of the method remains to be assessed.