The dispute, which stems from efforts to automate chip design, threatens to undermine the reputation of Google’s research in the academic community. It also could disrupt the flow of millions of dollars in government grants for research into AI and chips.
Google’s research unit has faced scrutiny since late 2020 after workers lodged open critiques about its handling of personnel complaints and publication practices.
The new episode emerged after the scientific journal Nature in June published “A graph placement methodology for fast chip design,” led by Google scientists Azalia Mirhoseini and Anna Goldie. They discovered that AI could complete a key step in the design process for chips, known as floorplanning, faster and better than an unspecified human expert, a subjective reference point.
But other Google colleagues in a paper that was anonymously posted online in March – “Stronger Baselines for Evaluating Deep Reinforcement Learning in Chip Placement” – found that two alternative approaches based on basic software outperform the AI. One beat it on a well-known test, and the other on a proprietary Google rubric.
Google declined to comment on the leaked draft, but two workers confirmed its authenticity.
Discover the stories of your interest
The company said it refused to publish Stronger Baselines because it did not meet its standards, and soon after fired Satrajit Chatterjee, a leading driver of the work. It declined to say why it fired him.
“It’s unfortunate that Google has taken this turn,” said Laurie Burgess, an attorney for Chatterjee. “It was always his goal to have transparency about the science, and he urged over the course of two years for Google to address this.”
Google researcher Goldie told the New York Times, which on Monday first reported the firing, that Chatterjee had harassed her and Mirhoseini for years by spreading misinformation about them.
Burgess denied the allegations, and added that Chatterjee did not leak Stronger Baselines.
Patrick Madden, an associate professor focused on chip design at Binghamton University who has read both papers, said he had never seen a paper before the one in Nature that lacked a good comparison point.
“It’s like a reference problem: Everyone gets the same jigsaw puzzle pieces and you can compare how close you come to getting everything right,” he said. “If they were to produce results on some standard benchmark and they were stellar, I would sing their praises.”
Google said the comparison to a human was more relevant and that software licensing issues had prevented it from mentioning tests.
Studies by big institutions such as Google in well-known journals can have an outsized influence on whether similar projects are funded in the industry. One Google researcher said the leaked paper had unfairly opened the door to questions about the credibility of any work published by the company.
After “Stronger Baselines” emerged online, Zoubin Ghahramani, a vice president at Google Research, wrote on Twitter last month that “Google stands by this work published in Nature on ML for Chip Design, which has been independently replicated, open-sourced, and used in production at Google.”
Nature, citing a UK public holiday, did not have immediate comment. Madden said he hoped Nature would revisit the publication, noting that peer reviewer notes show at least one asked for results on benchmarks.
“Somehow, that never happened,” he said.
