By Julian Gaberle, 04/06/2021
Last month the Biden administration signed an Executive Order with the intention of reducing risks to financial stability and improving disclosure of climate related risks. A similar mandate was signed in the UK, where environmental and climate goals will now be explicitly considered by the Bank of England as part of monetary policy. Artificial Intelligence (AI) systems have been shown to have the potential to decouple economic growth from rising carbon emissions and environmental degradation; but what is the impact of the AI systems themselves on reaching a net-zero carbon economy?
AI systems have been used in a couple of ways to reduce carbon output. One method is to use AI in resource decoupling (reducing the resources used per unit of economic output), for example, by predicting crop yields or controlling heating and cooling of the built environment. Further, we have also seen impact decoupling. This decouples the economic output and environmental harm by using smarter planning systems. For example, AI systems have helped reduce economic emissions by forecasting energy usage, thus improving scheduling of renewable energy sources.
Figure 1: Demonstrating the increase in AI Model complexity over time. (Source: https://openai.com/)
Despite these advances, AI use cases have garnered significant notoriety for landmark developments in “state of the art” models such as GPT-3 (an AI model trained to understand language). The compute resources consumed to train a single model has been growing exponentially over the past decade (see Fig. 1) with a single training session of GPT-3 having an energy consumption equivalent to the yearly consumption of 126 Danish homes. Research and development of such models over multiple architectures and hyperparameters multiplies these costs by orders of magnitude, however, the true figure is hard to estimate due to lack of disclosure. Furthermore, such huge models require specialised data warehouses and vast arrays of compute chips, limiting the accessibility of such technologies to few tech companies.
At the same time, researchers are investigating more efficient models and are continuously improving existing ones (see Fig. 2). Using existing hardware, improvements can be made to data handling, parallel processing and code efficiency. From a hardware perspective, intense research is being done on custom processing chips (e.g., Cerebras’ wafer chips or Google’s TPUs) which are optimised for specific AI applications and thus reduce the resources required to develop and deploy a model.
Figure 2: Model efficiency over time. (Source: https://openai.com/)
Several factors influence the environmental impact of AI systems: the location of the compute resources used for development; the local energy supply; the size of the dataset; and the hardware used to train the models. Just 10 years ago, 79% of computing was done in smaller computer centres and company servers, but since then a dramatic shift to the cloud has taken place – by 2018 a paper in Science estimated that 89% of data centre computing was done in large data centres. These large data centres use tailored chips, high-density storage, so-called virtual-machine software, ultrafast networking and customized airflow systems. Furthermore, they can be built at more suitable sites for large compute clusters such as near lakes or in cooler regions for cheaper, more efficient cooling.
The intended carbon reduction (side-)effects of the migration from on-premise, small data centres (‘on-prem’) to larger, more efficient, cloud based solutions has so far had a relatively small net effect; having these resources available has allowed more, and more complex research to be done, increasing demand. If current development towards ever larger AI models continues, resource consumption of AI systems will outstrip any gains in efficiency.
How does Arabesque AI seek to solve these problems? As a business, one of our core values is Sustainability. This extends to our models as well. We have migrated to our Cloud partner Google Cloud Platform from a previously On-Prem system in order to reap efficiency benefits. We are working hard to increase the efficiency of our models; within our Research and Engineering teams, this is a key objective for 2021. While our models are exceptionally complex, they will only remain this way, and likely become more so. We feel there is more work we can do around efficiency of the compute processes we use, which will realise both environmental as well as obvious economic benefits. We encourage other Machine Learning and AI reliant businesses to focus on these goals as well.
Mwezi uliopita, utawala wa Biden ulitia saini Amri ya Utekelezaji kwa nia ya kupunguza hatari kwa utulivu wa kifedha na kuboresha ufunuo wa hatari zinazohusiana na hali ya hewa. Mkataba kama huo ulitiwa saini nchini Uingereza, ambapo malengo ya mazingira na hali ya hewa sasa yatazingatiwa wazi na Benki ya Uingereza kama sehemu ya sera ya fedha. "Mifumo ya akili bandia (AI) imeonyeshwa kuwa na uwezo wa ""kuunganisha"" ukuaji wa kiuchumi kutoka kwa ongezeko la uzalishaji wa kaboni na uharibifu wa mazingira, lakini ni nini athari ya mifumo ya AI yenyewe kufikia uchumi wa kaboni ya ""net zero""?" Mifumo ya AI imetumiwa kwa njia kadhaa kupunguza uzalishaji wa kaboni. Njia moja ni kutumia AI katika kuunganisha rasilimali (kupunguza rasilimali zinazotumiwa kwa kitengo cha pato la kiuchumi), kwa mfano, kwa kutabiri mavuno ya mazao au kudhibiti joto na baridi ya mazingira yaliyojengwa. Zaidi ya hayo, pia tumeona athari decoupling. Hii uncouples pato la kiuchumi na madhara ya mazingira kwa kutumia mifumo ya mipango ya smart. Kwa mfano, mifumo ya AI imesaidia kupunguza uzalishaji wa kiuchumi kwa kutabiri matumizi ya nishati, na hivyo kuboresha ratiba ya vyanzo vya nishati mbadala. Picha ya kwanza: Kuonyesha ongezeko la utata wa AI Model kwa muda. "Kupitia utafiti wa ""GPS,"" AI imeweza kuunda mifano ya teknolojia ya kisasa ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia ya teknolojia." Rasilimali za hesabu zinazotumiwa kufundisha mfano mmoja zimekuwa zikikua kwa kasi zaidi ya muongo uliopita (tazama Mchoro. "Kama ilivyoelezwa hapo juu, ""Mkutano mmoja wa mafunzo ya GPT-3 una matumizi ya nishati sawa na matumizi ya kila mwaka ya nyumba 126 za Denmark.""" Utafiti na maendeleo ya mifano kama hiyo juu ya usanifu mbalimbali na hyperparameters kuzidisha gharama hizi kwa amri ya ukubwa, hata hivyo, takwimu ya kweli ni vigumu kukadiria kutokana na ukosefu wa ufunuo. Zaidi ya hayo, mifano kubwa kama hiyo inahitaji maghala ya data maalumu na safu kubwa ya chips za kompyuta, ikipunguza upatikanaji wa teknolojia kama hizo kwa makampuni machache ya teknolojia. Wakati huo huo, watafiti ni kuchunguza mifano ufanisi zaidi na ni kuendelea kuboresha zilizopo (tazama Mchoro. 2) Kutumia vifaa vya sasa, maboresho yanaweza kufanywa kwa utunzaji wa data, usindikaji sambamba na ufanisi wa nambari. Kutoka kwa mtazamo wa vifaa, utafiti mkubwa unafanywa kwenye chips za usindikaji wa kawaida (kwa mfano, chips za wafer za Cerebras au Google's TPU) ambazo zimeboreshwa kwa matumizi maalum ya AI na hivyo kupunguza rasilimali zinazohitajika kuendeleza na kupeleka mfano. Picha ya pili: Ufanisi wa Mfano kwa Muda "Kama ilivyoelezwa hapo juu, ""Vitu kadhaa huathiri athari za mazingira za mifumo ya AI: eneo la rasilimali za kompyuta zinazotumiwa kwa maendeleo, usambazaji wa nishati ya ndani, ukubwa wa seti ya data, na vifaa vinavyotumiwa kufunza mifano.""" Miaka kumi tu iliyopita, asilimia 79 ya kompyuta ilitengenezwa katika vituo vidogo vya kompyuta na seva za kampuni, lakini tangu wakati huo mabadiliko makubwa ya wingu yamefanyika - kufikia 2018 karatasi katika Sayansi ilikadiria kwamba asilimia 89 ya kompyuta za kituo cha data zilitengenezwa katika vituo vikubwa vya data. Vituo hivi vikubwa vya data hutumia chips zilizoboreshwa, kuhifadhi kwa wiani wa juu, programu inayoitwa mashine ya virtual, mitandao ya haraka sana na mifumo ya mtiririko wa hewa iliyoboreshwa. Zaidi ya hayo, wanaweza kujengwa katika maeneo bora kwa ajili ya makundi makubwa ya kompyuta kama vile karibu na maziwa au katika mikoa baridi kwa ajili ya baridi ya bei nafuu, ufanisi zaidi. "Mapato ya kupunguza kaboni yaliyokusudiwa ya uhamisho kutoka kwa vituo vidogo vya data vya mahali pao hadi suluhisho kubwa, bora zaidi za wingu hadi sasa zimekuwa na athari ndogo ya jumla, kuwa na rasilimali hizi imeruhusu utafiti zaidi na tata zaidi kufanywa, na kuongezeka kwa mahitaji. """ Ikiwa maendeleo ya sasa kuelekea mifano kubwa zaidi ya AI itaendelea, matumizi ya rasilimali ya mifumo ya AI itazidi faida yoyote katika ufanisi. Jinsi gani Arabesque AI inatafuta kutatua matatizo haya? Kama biashara, moja ya maadili yetu ya msingi ni Uendelevu. Hii inaenea kwa mifano yetu pia. Tumehamia kwa mshirika wetu wa Cloud Google Cloud Platform kutoka kwa mfumo wa awali wa On-Prem ili kupata faida za ufanisi. """Tunafanya kazi kwa bidii kuongeza ufanisi wa mifano yetu, ndani ya timu zetu za Utafiti na Uhandisi, hii ni lengo kuu kwa 2021." Wakati mifano yetu ni ya kipekee tata, wao tu kubaki njia hii, na uwezekano wa kuwa zaidi hivyo. """Tunaona kuna kazi zaidi tunaweza kufanya kuhusu ufanisi wa michakato ya kompyuta tunayotumia, ambayo itatambua faida za mazingira na pia faida za kiuchumi." Tunahimiza biashara zingine za kujifunza mashine na biashara zinazotegemea AI kuzingatia malengo haya pia.