Cohere partners with Google Cloud to train large language models using dedicated hardware

[ad_1]

Google Cloud, Google’s cloud computing services platform, today announced a multi-year partnership with startup Cohere to “accelerate natural language processing (NLP) for businesses by making it more affordable.” As part of the partnership, Google Cloud says it will help Cohere build an IT infrastructure to power the Cohere API, allowing Cohere to train large language models on dedicated hardware.

The news comes a day after Cohere announced the general availability of its API, which allows customers to access optimized templates for a range of natural language applications, in some cases at a fraction of the cost of competing offerings. “Leading companies around the world are using artificial intelligence to radically transform their business processes and deliver more useful customer experiences,” said Google Cloud CEO Thomas Kurian in a statement. “Our work with Cohere will make it easier and cheaper for any organization to realize the possibilities of AI with powerful NLP services based on Google’s custom design. [hardware]. “

How Cohere Works

Based in Toronto, Canada, Cohere was founded in 2019 by a pedigreed team including Aidan Gomez, Ivan Zhang and Nick Frosst. Gomez, a former intern at Google Brain, co-authored the academic paper “Attention is All You Need,” which introduced the world to a fundamental architecture of the artificial intelligence model called Transformer. (Among other high-profile systems, OpenAI’s GPT-3 and Codex are based on the Transformer architecture.) Zhang, along with Gomez, is a contributor to FOR.ai, an open AI research collective involving scientists and engineers from the data. As for Frosst, he too, like Gomez, worked at Google Brain, publishing research on machine learning alongside Turing Award winner Geoffrey Hinton.

In a vote of confidence, even before launching its commercial service, Cohere raised $ 40 million from institutional venture capitalists, as well as Hinton, Google Cloud AI chief scientist Fei-Fei Li, co-director of the AI ​​lab of UC Berkeley Pieter Abbeel and former autonomous Uber chief guide Raquel Urtasun.

Unlike some of its competitors, Cohere offers two types of English NLP models, generation and representation, in the sizes Large, Medium and Small. Generation templates can complete tasks that involve generating text, such as writing product descriptions or extracting metadata from documents. In contrast, representational models are about understanding language, driving apps like semantic search, chatbots, and sentiment analysis.

To keep its technology relatively affordable, Cohere charges for font-based access based on the size of the model and the number of characters the apps use ($ 0.0025 to $ 0.12 per 10,000 characters per generation and $ 0.019 per 10,000 characters for representation). Only generation templates load on input and output fonts, while other templates load on output fonts. All perfected models, meanwhile – that is, models tailored to particular domains, industries or scenarios – are charged at twice the base model rate.

Great linguistic models

The partnership with Google Cloud will grant Cohere access to fourth generation dedicated tensor processing units (TPUs) running on Google Cloud instances. TPUs are custom chips developed specifically to accelerate AI training, powering products like Google Search, Google Photos, Google Translate, Google Assistant, Gmail, and Google Cloud AI API.

“The partnership will last until the end of 2024 with extension options in 2025 and 2026. Google Cloud and Cohere plan to collaborate on a go-to-market strategy,” Gomez told VentureBeat via email. “We met with a number of cloud service providers and felt that Google Cloud was best placed to meet our needs.”

Cohere’s decision to partner with Google Cloud reflects the logistical challenges of developing large language models. For example, Nvidia’s recently released Megatron 530B model was originally trained on 560 Nvidia DGX A100 servers, each of which house 8 80GB Nvidia A100 GPUs. Microsoft and Nvidia say they observed between 113 and 126 teraflops per second per GPU when training the Megatron 530B, which would push the cost of training into the millions of dollars. (A teraflop rating measures hardware performance, including GPUs.)

Inference, that is, running the trained model, is another challenge. On two of its expensive DGX SuperPod systems, Nvidia claims that inference (for example, the autocomplete of a sentence) with the Megatron 530B takes only half a second. But it can take more than a minute on a local CPU-based server. While cloud alternatives may be cheaper, they are not dramatically – one estimate puts the cost of running GPT-3 on a single Amazon Web Services instance at as little as $ 87,000 per year.

Consistent rival OpenAI trains its large language models on a Microsoft-hosted “AI supercomputer,” which invested over $ 1 billion in the company in 2020, including about $ 500 million in the form of Azure compute credits.

Accessible NLP

At Cohere, Google Cloud, which already offered a range of NLP services, is acquiring customers in a rapidly growing market during the pandemic. According to a 2021 survey by John Snow Labs and Gradient Flow, 60% of technology leaders indicated that their NLP budgets grew by at least 10% compared to 2020, while a third – 33% – said that its spending increased by more than 30%.

“We are dedicated to supporting companies, such as Cohere, through our advanced infrastructure offering to drive innovation in NLP,” Craig Wiley, director of product management at Google Cloud AI, told VentureBeat via email. . “Our goal is always to provide the best pipeline tools for NLP model developers. By bringing together the NLP experience of Cohere and Google Cloud, we will be able to deliver truly extraordinary results to customers. “

The global GNP market is projected to be worth $ 2.53 billion by 2027, up from $ 703 million in 2020. And if the current trend holds, a substantial portion of that spending will go to cloud infrastructure, to Google’s benefit. Cloud.

VentureBeat VentureBeat’s mission is to be a digital town center for technical decision makers to gain insights into transformative technology and transactions. Our site provides essential information on data technologies and strategies to guide you in driving your organizations. We invite you to become a member of our community, to access: up-to-date information on the topics of your interest our gated newsletters thought-leading content and discounted access to our award-winning events, such as Transform 2021: Learn more networking features and more again Become a member

[ad_2]