This post was originally published on here
What do we want from AI?
That was the question posed by Alex Pascal, executive director of the Berkman Klein Center for Internet & Society, at the outset of a panel discussion hosted by the center on how AI “can serve people and make our society better.”
The powerful, potentially game-changing technology has been in the headlines in recent months after high-profile reports of experimental AI agents going rogue and hacking into other systems. Industry leaders have warned that unchecked AI development poses dangers to humanity.
AI’s role, said Jeff Dunn, chief operating officer of the Digital Trust Council, is inherently double-edged.
“We want to cure diseases but in order to do that we need to use a tool that can create bioweapons,” he said. “You can’t just put a knife through the middle and hope for the best.”
Amit Goldenberg, associate professor at Harvard Business School and a member of its AI Institute, broke down the kinds of artificial intelligence now in use, and the range of the roles the technology can fill.
At its most basic, he said, “it’s a tool, like autocorrect, and it doesn’t have a personality. The other extreme is a full-fledged agent that has a name, that has a personality that you can interact with.”
“What are the type of interactions that will improve humans’ well-being and what are the type of interactions that will harm them?”
Amit Goldenberg
The “million-dollar question,” he said, is distinguishing where the different AI tools and models fall on the continuum and which ones are best suited for specific needs.
At the heart of this issue are two questions: “What are the type of interactions that will improve humans’ well-being and what are the type of interactions that will harm them?” Goldenberg said. “I work under the assumption that humans need humans.”
Avijit Ghosh, lead technical AI policy researcher at Hugging Face, said that we need more accurate means of evaluating AI systems in order to develop a greater sense of public trust in the technology. Hugging Face, a platform and library for AI developers, was hacked this summer by Open AI agents that went rogue. Ghosh said he couldn’t comment on the cyberattack on his company.
He noted that we are in an era of “benchmark maxing,” or posting more supposed benchmarks than necessary in order to boost public trust.
Dunn chimed in, saying that evaluations need to be done properly. “If they’re not validated or not audited, then it’s just a marketing stunt,” he said.
As a remedy, Ghosh recommended a system that measures both system behavior and its impact on people.
“It’s also not an unsolvable problem,” said Ghosh, who works on what current evaluation methods can and cannot capture. “We have decades of research in human-computer interactions.”
How precisely to conduct such evaluations was open for discussion.
Ghosh, for example, warned against anointing certain panels or experts to judge the value (or harm) of any AI agent.
Most regulations and evaluations currently come “from the top down,” he said. “I want the general public to have access because we don’t know what we don’t know.
“If we collectively, as a society, decide that these are the 20 organizations that are allowed to evaluate models and nobody else [can], then we are probably depriving ourselves of really interesting insights. Something interesting can come out of a bug report if the public were allowed to submit some.
“We need better ways for the general public to participate in the general reporting and assessment.”
AI companies, he said, could be incentivized to respond to such feedback. “Maybe in exchange, the government can say, ‘You will not have liability if you fix this reported problem within 60 days.’”
“If we can align the business with what humans would want, then I think we will be OK,” Dunn said.
