AI Proposes, Students Verify: A UCDC Model for Responsible Research with ChatGPT

By Dr. Tanya Golash-Boza, Professor of Sociology, UC Merced

The University of California Washington Center (UCDC) gives UC students the opportunity to spend a term in Washington, DC, where they take classes and complete internships. Students can also participate in research labs led by UC faculty.

These labs allow faculty members from across the UC system to advance their research while giving students direct experience with original sources and research methods. Students learn how researchers develop questions, evaluate evidence, and reach conclusions. They also contribute to research on issues being debated in the nation’s capital.

One UCDC research lab is now using ChatGPT to help analyze DC Council hearings about firearm violence and public safety. The project uses AI to organize a large amount of public information. Students then check its work, interpret the material, and make the analytical decisions.

Studying how evidence shapes public policy

I lead this research lab as a professor of sociology at UC Merced and executive director of UCDC. Our project examines how DC policymakers use evidence when making decisions about firearm-violence prevention and public safety.

Public hearings provide a detailed record of how government decisions are made. Elected officials, agency leaders, researchers, service providers, advocates, and residents use these hearings to define problems, debate solutions, justify spending, and evaluate public programs.

Our research team asked several questions: What kinds of evidence do public officials rely on? When do they cite research studies? How do they weigh academic research against agency statistics, professional expertise, community testimony, and lived experience? Does evidence shape decisions about budgets, contracts, staffing, program delivery, and accountability?

Governments across the country invest in programs described as “evidence-based,” but it is often difficult to see how research affects implementation. A study may influence how a program is designed, but that influence can disappear as the idea moves into a budget, contract, staffing plan, or reporting requirement.

Confronting a large body of material

Our first task was to identify DC Council hearings that addressed firearm violence, community violence intervention, youth safety, policing, victim services, or prevention funding.

Dhilena Wickramasinghe, a senior at UC Davis and a research intern at UCDC, reviewed the Council’s public archive and created an initial list of potentially relevant hearings. I reviewed the list and selected 20 hearings for analysis.

Each hearing lasts several hours. The Council’s automated transcripts are helpful, but they are often messy. Speakers may be misidentified. Words may be omitted, and specialized terms may be transcribed incorrectly.

Under a traditional workflow, a four-hour hearing could require approximately 24 hours of professional transcription and at least eight more hours for a researcher’s initial review. Across 22 hearings, the project could require several hundred hours before quality checks or substantive analysis began.

Using ChatGPT for the first pass

ChatGPT makes this initial stage more manageable. Once a transcript is available, we can use it to create a preliminary spreadsheet in about 15 to 30 minutes. The spreadsheet identifies passages that may be relevant and records who is speaking, what they said, and how their comments might relate to our research questions.

Researchers call this process “qualitative coding.” In plain language, qualitative coding means identifying an important passage and applying labels that describe what it is about. A passage might be coded as relating to program effectiveness, youth outcomes, funding, community trust, racial equity, or implementation barriers.

The AI-assisted review of the 20 hearings identified more than 1,000 potentially relevant passages. We treat these as suggestions, not findings. The next step is for students to verify them.

Students verify and interpret

Undergraduate research interns listen to the hearings themselves. They confirm who is speaking, compare the transcript with the audio, correct quotations, and add timestamps. They review each code ChatGPT suggested and decide whether to accept, revise, or reject it. They also add important testimony and exchanges that ChatGPT missed.

The human verification step matters because a passage cannot always be understood on its own. An agency official may cite a statistic that a Council member challenges several minutes later. A program may report enrolling 100 participants, but enrollment does not demonstrate that the program reduced violence or improved participants’ lives. A witness may say that “research shows” something without identifying a study or explaining its limitations.

ChatGPT can locate language that appears relevant. It cannot reliably determine whether a claim is credible, whether a statistic answers the question being asked, or whether a statement has been taken out of context. Those judgments remain with the students and the research team.

A second student independently reviews part of the material. I spot-check completed hearings and resolve difficult or disputed cases. Our spreadsheets preserve ChatGPT’s original suggestion alongside the student’s revision, so we can see how each decision was made.

Learning to use AI critically

The project gives students experience in public policy, qualitative research, and firearm-violence prevention. It also teaches them how to use AI without assuming that its output is accurate.

Students check AI-generated material against original sources. They learn to identify plausible-sounding errors, distinguish evidence from assertion, recognize missing context, and document their corrections. They are responsible for the work they approve.

AI does not eliminate student work. It allows students to spend less time searching transcripts and more time evaluating evidence and understanding policy debates. The work becomes less mechanical and more analytical.

This project is one example of what faculty-led research labs can do at UCDC. UC faculty can advance ongoing research while giving students direct experience with research and public-policy questions. In this case, ChatGPT allows us to review a much larger body of material, but it does not make the analytical decisions.

AI does the first analyses, then students verify, and the faculty lead remains responsible for the results.