AI in brief A researcher is under fire for collecting course materials from lecturers without consent to train a chatbot, which he claimed could solve problems in assignments and exams for a computer science degree at MIT.
The controversial paper [PDF] described training and testing GPT-4 on a dataset of 4,500 problems obtained from different courses set by the elite university.
“We evaluate the ability of large language models to fulfill the graduation requirements for any MIT major in Mathematics and [Electrical Engineering and Computer Science]. Our results demonstrate that GPT-3.5 successfully solves a third of the entire MIT curriculum, while GPT-4, with prompt engineering, achieves a perfect solve rate on a test set excluding questions based on images,” the paper’s abstract claims.
The paper was slammed for questionable research methods – for example, GPT-4 already had access to solutions before it was tested, and graded answers incorrectly. And professor Armando Solar-Lezama, listed as a co-author on the paper, later revealed [PDF] that lead author Iddo Drori – a research affiliate visiting MIT at the time, and an academic at other institutions – had perhaps not obtained permission to collect the dataset in the first place.
Some lecturers confirmed to The Register that they had not shared their course material with Drori. Meanwhile, Solar-Lezema told us he only helped write the introduction of the paper, and didn’t know Drori was using the dataset to claim that GPT-4 was good enough to graduate with a computer science degree from MIT.
The paper has since been withdrawn from arXiv, and Drori is not at MIT today.
Generative AI raises antitrust concerns, says the Federal Trade Commission
The FTC is watching the generative AI hype closely to ensure that dominant players do not squash competition as the technology increasingly pervades society.
Officials warned that as the market matures, businesses might try to find ways to control the resources needed to develop models – like data, hardware, and talent.
They are paying close attention to cloud providers that might try to undercut rivals by offering their own AI services for lower prices or locking customers to their platforms. Meanwhile, Big Tech could consolidate power by snapping up startups to take advantage of new technologies and expertise.
The FTC noted that although the open source movement could counteract some of these concerns by making it easier for developers to build their own systems, the technology can still be misused – especially if the software is made generally available without safety guardrails in place.
“Generative AI is still evolving rapidly, but it already has the potential to transform many markets. Through vigorous law enforcement, the FTC strives to support a vibrant marketplace where new businesses can compete, researchers are free to move to the jobs where they can best advance the state of the technology, and entrepreneurs can continue to innovate,” it declared in a statement.
“As competition issues surrounding generative AI continue to develop, the Bureau of Competition, working closely with the Office of Technology, will use our full range of tools to identify and address unfair methods of competition.”
OpenAI office to open in London
OpenAI is launching its first international office in London.
The startup announced the move this week and said it hoped to attract more researchers and engineers to work on advancing AI alongside its workforce in the US.
“We see this expansion as an opportunity to attract world-class talent and drive innovation in AGI development and policy,” Sam Altman, CEO of OpenAI, said in a statement. “We’re excited about what the future holds and to see the contributions our London office will make towards building and deploying safe AI.”
The UK’s tech industry is growing, and has always been a leader in AI – particularly in research. London is also home to DeepMind, another top AI lab, and it makes sense for OpenAI to establish a presence there too. Headquartered in San Francisco, OpenAI reportedly employs 375 people full-time, according to a report in The Information. ®
