NEWS: The Population Risk Model Redesign Project Team Wins the Silver AI Impact Award at the 2025 UC Tech Awards

Overview

UCLA Health’s Population Risk Model predicts which primary care patients are at the highest risk of preventable admissions or Emergency Department visits. This data is used by the Proactive Care Team to provide support and care for the patients. The Population Risk Model Redesign Project Team redesigned the model, enabling it to run monthly instead of quarterly, so the model prediction is based on current data and the impact of the intervention can be assessed more quickly.   

Additional enhancements were also made to the process to align with a Machine Learning Operations Framework and adherence to the organization’s Responsible AI Program. The team won the Silver AI Impact Award at the 2025 UC Tech Awards for this work.

The Challenge

CLA Health’s Population Risk Model was initially implemented in 2018. By identifying these at-risk individuals, the Proactive Care Team could provide the support and care the patients needed to stay healthy and out of the hospital. The model was initially designed to run on a quarterly frequency due to the amount of time and resources it took to run the model.   

  Since the model ran on a quarterly frequency, the model’s predicted risk score for patients may not be current, so there was a delay in identifying at-risk patients and performing the intervention. The Office of Health Informatics and Analytics (OHIA) Team was asked to design a process where the model could run on a monthly frequency and to find areas for automation since the quarterly process was time-consuming due to some manual steps.

The swim-lane diagram below shows the quarterly process and calls out the manual steps that needed to be addressed so the model could run more frequently. The details in the diagram are not as important as the comments on the right and left side of the diagram calling out the stops in the process for manual review and the manual tasks.  

The Approach

**Neda Xaymountry**, UCLA Health, commenting on the award on behalf of the Population Risk Model Redesign Project Team.

The following solutions were implemented to address the manual steps highlighted in the swim-lane diagram:

To address the two stops in the process for manual review, the OHIA team worked with the Office of Population Health and Accountable Care (OPHAC) team to create quality metrics and thresholds in the AI Platform (Databricks) to validate the data refresh and model output so manual review is only needed if critical metrics fail. If the metrics pass, the process can continue automatically.

To address a data scientist having to be involved with each model run, the OHIA team built a workflow pipeline in the Databricks environment where the features and model workflows are scheduled to run every month. Therefore, a data scientist only needs to get involved if model drift is detected or there’s a trigger to retrain the model.

To address the manual steps in the filtering logic of the model output, the OHIA team worked with the Population Health and Interoperability team to move the rules into the Electronic Health Record system so all the at-risk patients that are predicted by the model can be loaded into the system and then filtered further for specific business needs and resource constraints.

These solutions alone allowed the model to run on a monthly frequency, but the team wanted to build a scalable process that would allow the model to be retrained and could be repeated for other models in the future. Therefore, additional enhancements were made, such as:

Building the model features into an Enterprise Feature Store so the features can be reused by other models.

The model itself was also re-coded using the Databricks platform to adhere to UCLA Health’s Machine Learning Operations (MLOps) Framework guidelines.

Finally, the model was reviewed by the Health AI Council to evaluate the model in four AI risk domains: impact, appropriate data use, accountability, and bias. To prepare for the council’s evaluation, the team conducted a subgroup analysis for fairness/bias and provided a model facts label and answered the council’s questionnaire with detailed answers on the four risk domains.

The Impact

The redesigned model with a monthly frequency went live on July 2024 and has been running every month since then. With the new process, a smaller team is involved with the month-to-month operations of running the model, and the Proactive Care team gets a new batch of at-risk patients every month that they know is based on current data. Since the model is running more frequently, OPHAC can perform more analysis on whether the interventions are making an impact to key metrics like admissions rate, Emergency Department-only visits rate, and 30-day readmissions rate. They are currently working on a paper analyzing the intervention impact, and the results show the model is positively impacting these metrics.  

The model was reviewed by the Health AI Council in October 2024, and the model was rated as low risk in the four AI risk domains. The Council voted to continue the use of the model without any modifications.  

What’s Next

The solution that was built by the OHIA team for the Population Risk Model is now being utilized by other machine learning model projects that are in development.   

Meet the Award-Winning Team

Team Name: Population Risk Model Redesign Project
Award Category: AI Impact
Location: UCLA Health 

2025 Silver AI Impact Award Winners:

Shafi Alam – Data Engineer, Office of Health Informatics and Analytics
Branden Connolly – Lead Clinical Informatics Specialist, Office of Population Health and Accountable Care
Agapios Constantinides – Machine Learning Engineer, Office of Health Informatics and Analytics
Angel Ferreiro – Data Architect, Office of Health Informatics and Analytics
Mojtaba Heidarysafa – PhD, Data Scientist, Office of Health Informatics and Analytics 
Lehrick Leung – Senior Cogito Analytics Administrator, Office of Health Informatics and Analytics 
Jennifer Ng – Application Analyst, Population Health and Interoperability  
Joel Pantoja – Machine Learning Engineer, Office of Health Informatics and Analytics
Ajaya Pillai – Assistant Director, Office of Population Health and Accountable Care
Hanina Rosenstein – Program Manager, Office of Population Health and Accountable Care
Jason Wang PhD – Data Scientist, Office of Health Informatics and Analytics
Charles Wilson – Lead Data Engineer, Office of Health Informatics and Analytics
Neda Xaymountry – Senior Product Manager, Office of Health Informatics and Analytics