Four Privacy Considerations that Make AI Worth Bringing to Campus

By Pegah Parsi, Chief Privacy Officer, UCSD

Sometime in the last few years, AI went from novelty to infrastructure, and universities weren’t immune to the frenzy. Admissions uses it to draft outreach and screen applications. IT uses it to triage tickets. Chatbots handle event logistics. Meeting tools transcribe by default. Faculty use it to customize tutoring. None of this happened through a single centralized decision. It happened the way campus adoption always happens: quietly, unit by unit, one person at a time.

That speed is worth pausing on, not because privacy is a hurdle to clear, but because getting it right is what makes these tools worth trusting. Done well, privacy isn’t a brake on AI adoption; it’s what earns the confidence of students, parents, faculty, and staff.

Here are four privacy considerations worth building in from the start, the kind any IT staffer, privacy officer, or unit lead should be able to name before the next tool goes live.

1. “Anonymized” Doesn’t Mean What We Want It to Mean

Every campus sits on datasets that feel safe because someone called them “de-identified”: course evals, health logs, advising notes, research data. But de-identified was never the same thing as anonymous, and true anonymization is even harder when data meets AI systems.

Modern AI systems excel at combining fragments that mean little apart, but everything together. A harmless dataset (say, advising notes with names stripped out) becomes identifying the moment a system cross-references it against enrollment records, financial aid data, or public sources.

The Supreme Court described almost exactly this process in this summer’s Chatrie v. United States regarding geofence warrants. Officers received “anonymized” location data from Google, then narrowed it down step-by-step to an actual person. Justice Jackson’s concurrence didn’t mince words: anonymized data becomes identified awfully quickly once someone with access keeps narrowing the field. Swap “officer with a warrant” for “AI tool querying multiple databases,” and you have the same mechanism, just faster, quieter, and without much oversight.

To address this, units should focus on restraint rather than searching for better anonymization techniques. Before feeding data to an AI tool, don’t just ask if names were stripped. Ask the unit whether the tool needs this data at all (shout-out to data minimization!), and what else it can already access.

2. The Riskiest AI Tool Is the One Nobody Approved

If you want to find the biggest privacy hole in your unit’s AI use, don’t look at the tools IT approved. Look at the ones nobody did.

Staff and faculty everywhere paste student records, HR files, grant proposals, and disciplinary notes into personal ChatGPT, Claude, or Gemini accounts because it’s faster, and the tab is already open. It’s not malicious; it’s the digital equivalent of handing a sensitive file to a friendly stranger who promises to be careful.

Without an institutional agreement, you lack basic legal and policy protections: no data processing terms that comply with privacy laws and UC policy, no promise that inputs won’t train vendor models, no assurance that data won’t be sold, no right to deletion, no breach notification, and no audit trail.

An institutional contract doesn’t make AI risk-free, but it means someone kicked the tires on the vendor’s data practices.

The standard here ought to be straightforward: before personal data goes into any AI tool, verify whether UC has an enterprise agreement for it. Check with procurement to see if there is an agreement in place for the campus. If there is no agreement, there should be no personal data submitted, full stop, no matter how helpful the output would be.

3. When AI Learns Things About You That Nobody Told It

While re-identification connects existing data back to a person, inference is different: the AI generates new sensitive facts that were never recorded anywhere.

An AI notetaker doesn’t just transcribe words; it infers things like financial hardship, immigration stress, or mental health concerns purely from phrasing, then writes those inferences into a persistent summary. Nobody disclosed that information, and nobody consented to recording it. Yet there it is, attached to a person’s name without the institutional care normally applied to sensitive disclosures.

Most privacy laws struggle here (though hats off to the California Privacy Protection Agency for its rules classifying automated inference of traits as high-risk!). The laws have much less to say about information a model invents on the fly.

A good first step, even if it takes a minute, is for a human to read the summary before it becomes permanent, specifically looking for things the AI decided it knew, but nobody actually said. If it inferred something sensitive, that’s a call for a person to make, not a setting AI gets to decide for you.

4. The AI That Remembers Too Much, Across Too Many Rooms

Most major AI platforms recently added persistent memory, which is the ability to carry context across separate sessions. Most of us clicked “accept” without realizing it was there.

A person’s campus interactions span distinct contexts: an advising chat in fall, financial aid in winter, an IT ticket in spring, a recommendation letter draft in the summer. If an AI tool has memory enabled, details disclosed in one context, like a disability accommodation or family emergency, can resurface in a completely unrelated interaction.

Privacy principles require data to stay within the purpose for which they were collected. A persistent memory feature that quietly ignores that boundary defeats purpose limitation by design.

Campus IT leads need to start by asking a straightforward question: Do we know if our AI tools have persistent memory enabled, and is that memory siloed per use case or shared across all product interactions?

Steering, Not Stopping

A few other risks deserve watching, ones we are already more familiar with: algorithmic bias, transparency gaps where AI features default to “on,” and cross-border data processing for international scholars.

None of this is an argument for banning AI on campus, any more than we banned email in 2003. It’s an argument for pairing adoption with enterprise governance and literacy. Unit leads should ask four basic questions before a tool goes live:

What data leaves our control?
Do we have a contract?
Does the tool remember or infer things it shouldn’t?
Who’s accountable if it goes sideways?

If your campus has an AI governance structure or procurement review, bring these questions there. The tools aren’t going anywhere; the least we can do is understand what we’re feeding them.

Author