AI Vetting: Balancing Innovation and Risk

As AI evolves daily, so does UC’s approach to managing it. By balancing innovation with vigilance, the team’s ability to stay nimble empowers UC to confidently advance while protecting UC’s data, research, and people.

While the rapid rise of AI tools brings opportunities, it also introduces security, legal, and privacy risks. Across the UC system, a team of cybersecurity and digital risk experts helps leadership navigate these risks through a thorough risk analysis process.

The AI vetting process begins with an AI Security Risk Assessment (AI SRA), which builds on UC’s existing supplier review program. Privacy and legal assessments complete this three-part review. The AI SRA process involves in-depth conversations with vendors and a deep dive into each tool’s architecture to understand how it handles UC data and prevent its use in model training. Damian Luna, Cyber Risk Unit Manager, Office of the President, said, “We present leadership with the risks we uncover to help them make more informed choices about cutting-edge technology like AI.”

The team faces multiple challenges, including getting to the heart of what’s behind an AI product and managing vendors’ reluctance to reveal information while keeping pace with rapidly changing technology and client needs. “Oftentimes, vendors don’t want to share details about the inner workings of their product,” explained Eric Hull, Manager, IT Security, Office of the President. “But understanding those details is exactly what helps UC protect its data. We try to help people achieve what they want, but we need to do so in a way that’s safe for the institution. And that can take time and perseverance.”

One recent review is a perfect example of why this work is so critical. The team’s assessment of a popular AI transcription tool uncovered significant security and privacy concerns. The security, legal, and privacy teams recommended not using the tool in favor of the already-vetted supplier and product that provide similar services under the current license. The AI company later faced legal scrutiny over its data practices.