Streamlining Clinical Trial Recruitment: A Two-Stage Zero-Shot LLM Approach with Advanced Prompting

Identifying patient eligibility for clinical trials is a critical bottleneck hindering medical research progress because many clinical trials allow only small, specific patient cohorts to be included and require a certain number of participating patients to yield definitive results. Manually screening patients through unstructured medical records is time-consuming and expensive. This paper explores the potential of large language models (LLMs) enhanced with medical context to automate patient eligibility assessments for clinical trials. We first design a two-stage zero-shot LLM approach to analyze a patient’s medical history (presented as unstructured text) and to determine their eligibility for a specific trial. We use advanced prompting strategies to guide the LLM toward faster and more targeted matches between trials and eligible patients. Additionally, a two-stage retrieval pipeline pre-filters potential trials using efficient retrieval techniques, reducing the number of trials considered for each patient. This two-way matching substantially improves processing speed and cost-effectiveness for clinical trial recruitment. Our method holds promise for streamlining clinical trial patient recruitment to accelerate medical advances.