Introduction

Picture a large university that has put an AI assistant in front of its policy library. Behind it sit thousands of pages of academic regulations, course handbooks, scholarship terms, fee schedules and registry procedures. A student asks the assistant whether dropping a course this semester will cost them their scholarship. 

The assistant finds the course withdrawal policy, because it reads a lot like the question. It explains the withdrawal deadline and the W grade, and it sounds confident. What it misses is the clause in the scholarship terms that requires a minimum credit load each semester. That clause never mentions dropping a course. It talks about full-time enrolment and credit units. Depending on the student's circumstances, the tuition refund schedule and the prerequisite rules for next year's courses may also be relevant. 

This is a gap AI knowledge assistants can fall into. We think closing it takes more than a better graph database. It takes a better way to describe the connections that matter to the person asking. 

The Economics Behind the Consulting Pyramid

The analyst-to-partner career ladder, with its brutal "up-or-out" promotion policy, isn't a cultural quirk.

It's a rigorous economic design.

A landmark paper by Jonathan Levin and Steven Tadelis explains why knowledge-intensive firms organise themselves as partnerships and why training junior consultants is essential to creating leverage.

A partner who personally delivers every engagement is worth far less than one who leads a team of consultants serving multiple clients.

That leverage depends on continuously developing junior talent.

AI now performs much of the research and synthesis work that junior consultants traditionally learned through.

As that work disappears, training investment and leverage begin to separate.

Similar is not the same as related 

Basic vector-based retrieval-augmented generation (RAG) systems turn text into vectors and fetch the chunks that sit closest to the question. That works well when one passage holds the answer. It can struggle when the answer is spread across several documents that use different words. 

Research illustrates the challenge. The MultiHop-RAG benchmark from HKUST tested questions that need evidence from two to four documents and found that both retrieving the relevant evidence and reasoning across it remain difficult. These results highlight a limitation of the tested retrieval setups, rather than a limit shared by every RAG system. Sources: MultiHop-RAG benchmark from HKUST 

A university policy library can present a similar challenge: it is large and written by different offices using different terminology. 

Knowledge graphs connect the dots, if the graph means something 

Knowledge graphs tackle this by linking information across known connection points. Instead of hoping the scholarship clause lands near the question, the assistant can follow a link from the withdrawal policy to the scholarship terms. 

When it works, the gain can be substantial. In a Lettria benchmark reported by AWS, hybrid graph-and-vector retrieval answered 80% of questions correctly, compared with 50.83% for the vector-based RAG baseline. The evaluation covered financial reports, scientific papers, technical specifications and EU regulations, and answers were assessed by Lettria's in-house team. The figures describe that particular system and evaluation, rather than an expected gain for every implementation. Sources: benchmark AWS ran with its partner Lettria 

The catch is that storing information in a graph does not guarantee useful connections. A systematic comparison by Michigan State University and others found distinct strengths and weaknesses for RAG and GraphRAG across tasks. It also identified gaps in automatically extracted graph information. Building a graph can add substantial indexing cost, as research on the KET-RAG framework illustrates. Both performance and cost depend on the graph construction method and retrieval configuration. Sources: systematic comparison by Michigan State University and others; National University of Singapore 

Whether that work pays off depends partly on the ontology: the concepts, relationships and rules used to describe the domain. It also depends on extraction quality, retrieval design and how well the system is tested against real questions. 

Why entity-centred ontologies can miss the user's question 

Ontologies are not new. Academia and industry have built them for decades, long before large language models. An entity-centred ontology might describe relationships such as a student being enrolled in a course, a scholarship being awarded to a student, or a course having a prerequisite. 

That structure is useful for formal graph queries. It can help identify patterns in fraud analytics, or count how many scholarship holders are enrolled in a given course. But a graph built only around those entities may leave out the connection between withdrawing from a course and maintaining scholarship eligibility. A formal ontology could represent that connection; the challenge is identifying and modelling the relationships users actually need. 

Microsoft Research describes choosing the right entities and relationships as difficult and time-consuming, and notes that default extraction prompts can miss information important to a particular domain. This supports the case for tailoring graph construction to the intended use. Our proposal extends that idea by making the target audience's questions central to the process. Sources: Microsoft Research 

Semantic ontologies build the graph from the questions people ask 

At JustifyAI, we use the term semantic ontology to describe our proposed approach: a graph organised around the questions a particular audience asks. Rather than treating this as an entirely new category of ontology, we see it as a question-driven way to identify useful relationships. The target persona might be a student or a registry officer. 

The process is close to writing an FAQ for every relevant passage in the corpus. Here, passages or policy clauses form the graph's nodes, while links describe the questions or consequences that connect them. Of the withdrawal policy, a student might ask when the last day to drop a course is and what happens to their fees. Of the scholarship terms, they might ask what study load they must keep and what happens to their funding if they fall below it. Both passages contribute to answering a broader question: what happens if they reduce their study load this semester? That shared question provides a candidate link between them. Related questions can connect the refund schedule and prerequisite rules where those consequences apply. Generated questions and links must be checked against the source material, and answers should cite the relevant policy clauses. 

Related research supports the use of questions as retrieval handles. HopRAG, published in Findings of ACL 2025, connects passages through generated queries and uses graph traversal to find logically relevant evidence. Its authors report gains over a dense-retrieval baseline, including for a variant that does not use an LLM during traversal. Its mechanism is related to our proposal, although it is not identical to linking passages through shared persona-specific FAQs. Sources: HopRAG, presented at ACL 2025 

Plain-language relationship descriptions give an LLM additional context when deciding which connections to follow. A description such as “affects your funding if you reduce your study load” makes the reason for a link explicit. This approach prioritises language-rich relationships for LLM-guided retrieval, while retaining structured nodes and edges that a graph engine can store and query. 

A fine-tuned embedding model can help pick where to start 

An LLM exploring a graph still needs a starting point. An embedding model can provide that first retrieval step, and fine-tuning may improve its performance for the intended questions. 

The FAQ used to build the graph can also provide candidate training data. Each generated question paired with a supporting passage is an example of what this persona might ask and where relevant evidence lives. After checking those pairs, they can be used to fine-tune an embedding model that selects promising starting passages. The LLM then follows the relevant graph links. Using the same questions and corpus can align the two components, but their effectiveness still needs to be measured on held-out questions, including real user queries. 

Databricks has demonstrated domain-specific embedding fine-tuning using synthetic questions generated from source documents. Its results show that fine-tuning can improve retrieval on some tasks, while also noting that it may offer less value where baseline retrieval already performs well. This makes fine-tuning an option to evaluate, rather than an automatic requirement. Sources: Databricks 

The trade-off is maintaining persona-specific questions and links 

A registry officer may ask the same corpus different questions, such as which withdrawals trigger a scholarship review and who must be told. Their FAQ and useful connections may differ from a student's. One implementation could maintain a separate graph for each persona; another could use persona-specific views or links within a shared graph. Either way, adding audiences creates more material to validate and maintain. 

For relatively stable policy libraries, the upfront work can be spread across many queries. Changes to source documents still require updates to the affected questions and links. The trade-off depends on how frequently the corpus changes and how much value the additional connections provide. 

The embedding layer offers a similar choice: use a shared model, or maintain specialised models where evaluation shows a benefit. A separate model for every persona is a design option, rather than a requirement. Compact embedding models make experimentation practical. Google's EmbeddingGemma has 308 million parameters and can run in under 200MB of RAM with quantisation; that is an inference figure, not a fine-tuning memory requirement. In a separate financial-document tutorial, Philipp Schmid fine-tuned a BGE model on about 6,300 question–passage pairs in roughly three and a half minutes on a single GPU. Its NDCG@10 ranking score improved by 7.42% relative to the baseline. That result demonstrates feasibility for the tested setup, rather than the full cost of generating data, validating it and operating multiple models. Sources: EmbeddingGemma; fine-tuning guide by Philipp Schmid 

Three ingredients, one direction 

Put the pieces together and you get three ingredients to evaluate together: a question-driven semantic ontology shaped by the audience's needs, an embedding model that retrieves useful starting passages, and an LLM that uses the graph's relationship descriptions to explore relevant evidence. Fine-tuning the embedding model is an optional step where testing demonstrates a benefit. For the student worried about their scholarship, the goal is one grounded answer that covers the withdrawal deadline and credit-load rule, with refunds and effects on future courses included where relevant. 

For large organisational corpora, we believe this combination is a promising direction for knowledge graphs. The next step is to test whether it retrieves more complete evidence and produces better-grounded answers than existing retrieval approaches, using the questions people actually ask.