A model can output 82% with two decimal places. That does not mean the user should experience the result as an authoritative prediction.
In the UniPath capstone, we explored AI-assisted university ranking, semantic matching, course recommendations, academic-risk signals, and an admissions-likelihood component. The technical temptation was obvious: make the output feel smart by making it precise.
Precision can hide epistemic weakness
Our proof of concept used prepared/synthetic data. The admissions-likelihood component used logistic regression, but it was not trained and validated on a production-scale admissions dataset. Calling the resulting probability “your chance of admission” would have been a product-design mistake even if the code was correct.
The honest output needed context: what variables were included, where the data came from, which factors were missing, and how the user should treat the number.
Ranking has the same problem
A weighted university score can look objective while encoding subjective preferences about ROI, academic quality, satisfaction, location, industry outcomes, or extracurricular opportunities. The weights are not universal truths.
A better experience makes those priorities visible and lets the user understand why a school appears where it does.
Semantic similarity is useful—but it is not understanding
The prototype combined embeddings and cosine similarity with hard filters. That combination mattered. Embeddings can surface schools or courses that are semantically aligned with a student’s goals, but they should not override explicit constraints such as location, program availability, price, or eligibility.
Fallback behavior is part of responsible AI
If an AI recommendation cannot be generated, or the confidence is low, the product still needs a useful state. “No answer” should become a deterministic search/filter experience, an explanation of missing information, or a request for more input—not a fabricated recommendation.
Trust comes from legibility
For a high-stakes student decision, I would rather show a user a slightly less magical interface that explains its reasoning than a polished probability that suggests more certainty than the evidence supports.
That means surfacing drivers, allowing users to adjust preferences, separating hard constraints from soft matching, documenting data limitations, and making it easy to ignore or override the system.
What changed in how I think about AI products
I used to think of responsible AI mainly as a model-quality question: bias, accuracy, and privacy. I now think product semantics matter just as much. The interface decides whether an uncertain signal feels like advice, prediction, ranking, or fact.
An AI system earns trust when the user can understand what kind of claim it is making—and what kind of claim it is not.