Last year, Zico Kolter and a team of researchers were attacking ChatGPT to expose flaws in it and similar models. Now, he's preparing to take up the helm as director of Carnegie Mellon's Machine Learning Department, and while optimistic about the pace of the field, he's still got his eye on safety.
"Whenever you have a massive new technology like this, there is always the possibility to use it in ways that benefit society and use it in ways that harm society," Kolter said. "Right now, generative AI techniques are tools that enable people to do certain things more efficiently and better. In that context, even just as a tool, there's massive ability to disrupt our current ecosystems."
CMU's Machine Learning Department was founded in 2006 and was the world's first academic department of its kind. As the sectors surrounding machine learning and artificial intelligence have rapidly expanded nationwide and in Pittsburgh in recent years, the university has been a leader in the field.
"Artificial intelligence is no longer an academic pursuit, it has become an industry pursuit," Kolter said. "AI has become kind of the new stand in for so much of the technology that [people] are building right now."
Kolter has worked for the university for 11 years and is currently an associate professor in the Computer Science Department. He and a team of researchers received nationwide media attention last year after publishing a paper detailing ways the team was able to circumvent safeguards in ChatGPT and other large language models. Beyond language models, Kolter is grappling with existential questions surrounding AI usage.
"The immediate disinformation, the possibility for autonomous systems to develop truly troubling behavior, down to actual, sort of real intelligence and the implications of this are all massive questions we need to grapple with," Kolter said. "We are not the only ones that should be having this conversation but we should absolutely have a strong voice in this conversation and be leading the field in the efforts to understand and discuss these things more."
Investment in artificial intelligence has risen substantially in recent years. Goldman Sachs forecasts global investment in AI to near $200 billion next year. But ability to secure funding is not a definitive metric of ability to secure success.
"I think it's a totally fair analogy to make between the current state of AI and the early internet, like in the late 90s," Kolter said. "It was very obvious it was going to be transformative but at the same time, was there a bubble? Yeah, of course there was. To be very clear, I'm not suggesting that it will play out in the same way here. It could be very different and its all speculative, but I think its a very apt analogy."
Because of possible future market trends, Kolter wants "to make Carnegie Mellon resilient to the whims of the funding particulars of the field."
"With these massively disruptive technologies there is going to be lots of hits and lots of misses when it comes to companies in the space," Kolter said. "My hope as chair of this department is that we educate and steer the field from a research and education standpoint that is resilient to whatever the short-term whims of the financial markets are."
