Optimizing RAG for Academic Advising: A Hybrid Routing and Metadata Filtering Approach for Enhanced Accuracy and Efficiency

Anuraj Sule, Abdul Rahman Mawlood Yunis
Proceedings of the The 39th Canadian Conference on Artificial Intelligence, PMLR 318:1145-1150, 2026.

Abstract

Academic advisors are essential to university life, yet they are often overwhelmed by the large number of student questions and the complexity of university policies. While Artificial Intelligence (AI) can help by searching through digital handbooks, current practices often struggle with two main problems: they are too slow and they often pull the wrong information. These systems typically search through thousands of documents for every single question, which not only wastes time but also creates "noise" where the AI gets confused by similar but irrelevant data. For an advisor, receiving the wrong policy information is a risk that cannot be ignored. This paper introduces a Hybrid Routing Layer designed to make AI a reliable assistant for professional advisors. Instead of a "brute-force" search that looks at everything, our system acts as an intelligent filter. It uses two main tools: first, a "Regex Router" that instantly finds specific items like course codes and second, a "Semantic Router" that understands the meaning behind policy questions. By narrowing the search area before the AI even begins to look for answers, we eliminate the noise that causes errors. We tested our system using a diverse set of real-world advising queries collected from the Wilfrid Laurier University (WLU) website. Our results show that this approach reduces the search space by 97%. It also makes the system 7x faster, cutting the wait time from 8.2 seconds down to just 1.3 seconds. Most importantly, it significantly improves the accuracy of the information provided to the advisor. This work provides a fast, accurate, and low-cost framework that allows advisors to support students with greater confidence and efficiency.

Cite this Paper

BibTeX
@InProceedings{pmlr-v318-sule26a, title = {Optimizing RAG for Academic Advising: A Hybrid Routing and Metadata Filtering Approach for Enhanced Accuracy and Efficiency}, author = {Sule, Anuraj and Yunis, Abdul Rahman Mawlood}, booktitle = {Proceedings of the The 39th Canadian Conference on Artificial Intelligence}, pages = {1145--1150}, year = {2026}, editor = {Bouzar-Benlabiod, Lydia and Leung, Carson}, volume = {318}, series = {Proceedings of Machine Learning Research}, month = {25--29 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v318/main/assets/sule26a/sule26a.pdf}, url = {https://proceedings.mlr.press/v318/sule26a.html}, abstract = {Academic advisors are essential to university life, yet they are often overwhelmed by the large number of student questions and the complexity of university policies. While Artificial Intelligence (AI) can help by searching through digital handbooks, current practices often struggle with two main problems: they are too slow and they often pull the wrong information. These systems typically search through thousands of documents for every single question, which not only wastes time but also creates "noise" where the AI gets confused by similar but irrelevant data. For an advisor, receiving the wrong policy information is a risk that cannot be ignored. This paper introduces a Hybrid Routing Layer designed to make AI a reliable assistant for professional advisors. Instead of a "brute-force" search that looks at everything, our system acts as an intelligent filter. It uses two main tools: first, a "Regex Router" that instantly finds specific items like course codes and second, a "Semantic Router" that understands the meaning behind policy questions. By narrowing the search area before the AI even begins to look for answers, we eliminate the noise that causes errors. We tested our system using a diverse set of real-world advising queries collected from the Wilfrid Laurier University (WLU) website. Our results show that this approach reduces the search space by 97%. It also makes the system 7x faster, cutting the wait time from 8.2 seconds down to just 1.3 seconds. Most importantly, it significantly improves the accuracy of the information provided to the advisor. This work provides a fast, accurate, and low-cost framework that allows advisors to support students with greater confidence and efficiency.} }
Endnote
%0 Conference Paper %T Optimizing RAG for Academic Advising: A Hybrid Routing and Metadata Filtering Approach for Enhanced Accuracy and Efficiency %A Anuraj Sule %A Abdul Rahman Mawlood Yunis %B Proceedings of the The 39th Canadian Conference on Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Lydia Bouzar-Benlabiod %E Carson Leung %F pmlr-v318-sule26a %I PMLR %P 1145--1150 %U https://proceedings.mlr.press/v318/sule26a.html %V 318 %X Academic advisors are essential to university life, yet they are often overwhelmed by the large number of student questions and the complexity of university policies. While Artificial Intelligence (AI) can help by searching through digital handbooks, current practices often struggle with two main problems: they are too slow and they often pull the wrong information. These systems typically search through thousands of documents for every single question, which not only wastes time but also creates "noise" where the AI gets confused by similar but irrelevant data. For an advisor, receiving the wrong policy information is a risk that cannot be ignored. This paper introduces a Hybrid Routing Layer designed to make AI a reliable assistant for professional advisors. Instead of a "brute-force" search that looks at everything, our system acts as an intelligent filter. It uses two main tools: first, a "Regex Router" that instantly finds specific items like course codes and second, a "Semantic Router" that understands the meaning behind policy questions. By narrowing the search area before the AI even begins to look for answers, we eliminate the noise that causes errors. We tested our system using a diverse set of real-world advising queries collected from the Wilfrid Laurier University (WLU) website. Our results show that this approach reduces the search space by 97%. It also makes the system 7x faster, cutting the wait time from 8.2 seconds down to just 1.3 seconds. Most importantly, it significantly improves the accuracy of the information provided to the advisor. This work provides a fast, accurate, and low-cost framework that allows advisors to support students with greater confidence and efficiency.
APA
Sule, A. & Yunis, A.R.M.. (2026). Optimizing RAG for Academic Advising: A Hybrid Routing and Metadata Filtering Approach for Enhanced Accuracy and Efficiency. Proceedings of the The 39th Canadian Conference on Artificial Intelligence, in Proceedings of Machine Learning Research 318:1145-1150 Available from https://proceedings.mlr.press/v318/sule26a.html.

Related Material