For instance, Meta Platforms Inc. says that by midsummer its AI Research SuperCluster system will house 16,000 Nvidia Corp. graphics processing units, a massive engine that Meta claims will be the fastest of its kind in the world. As well, the DeepMind Technologies lab, owned by Google parent Alphabet Inc., has announced the development of a new language model, commenting, “The high-school reading comprehension level approaches human-rater performance.”
While these companies take different tacks, both have the potential to catalyze tomorrow’s advances in drug discovery, new materials, remedies to climate change, closer analysis of military-drone footage and more.
And they are hardly the only ones: Microsoft Corp., Amazon.com Inc., Oracle Corp., International Business Machines Corp. and others are also in the AI marathon.
Consumers and investors, more focused on spasms in the stock market, may not be paying attention to projects not directly connected to lines of business or quarterly results. But research and development often hatch products that vault beyond a lab’s original aims. Xerox Holdings Corp.’s Parc lab, for instance, was a pioneer of personal computing. Researchers at Bell Labs, then part of what is now AT&T Inc., developed the transistor and prototyped an early cellphone.
The amounts of money being spent are massive. At Alphabet, R&D expenses rose to $31.562 billion last year from $27.573 billion in 2020; Meta spent $24.655 billion on R&D last year, rising from $18.447 billion the previous year, company filings show. Alphabet said R&D spending in 2021 amounted to 12.3% of revenue, while Meta reported it was 21% of its sales.
In contrast, S&P 500 companies spent an average 2.82% of revenue on R&D, according to Morningstar Inc., which calculated a revenue-weighted average for index constituents in each of their latest fiscal years.
Facebook parent Meta occupies the lane in the AI race dedicated to scaling data and computing power. Deep learning is a form of AI designed to mimic aspects of human neurons. It took off in 2010 when the ImageNet project proved it was possible to use two GPUs developed by Nvidia to train a large AI model to recognize labeled images, said Kyunghyun Cho, a co-founder of the startup Prescient Design and an associate professor of computer and data science at New York University.
Meta said its AI Research SuperCluster houses 6,080 Nvidia GPUs at present, a number that will rise to 16,000. The computing power will be used for language-processing and computer-vision research, with an eye toward creating the immersive world the company calls the metaverse, said Shubho Sengupta, a software engineer involved in the project.
The supercomputer will help Meta researchers build AI models that can work across hundreds of languages, analyze text, images and video together, and develop augmented-reality tools and rich, multidimensional experiences in the metaverse, the company said. The technology also will help it more easily identify harmful content, Meta said.
Alphabet’s DeepMind, which is based in London, employs a technique called reinforcement learning. In this approach, an algorithm learns through a process of trial and error.
Last month, DeepMind announced a new AI system called AlphaCode, which competes with human software developers. In December, DeepMind announced the development of a new language model that it says reduces scale without compromising performance. This comes on top of DeepMind’s claim that it has provided a leap in understanding protein structures, critical to drug discovery.
“It demonstrates that creative AI approaches can result in radical improvements in performance without having to do heavier number-crunching,” said Shuman Ghosemajumder, head of AI at F5 Inc. and a Google veteran.
It is too early to foresee what the commercial application of this research will be.
NYU’s Dr. Cho, whose startup Prescient was sold to Genentech Inc., a subsidiary of Roche, says big corporate labs represent much of the research in AI.
That work won’t drive corporate results over the short term. But longer term, it might.
“Such investment from top-tier tech companies who are heavily vested in AI technology will naturally encourage and compel other companies, both tech and non-tech, such as pharmaceuticals,” Dr. Cho said.
