Posts

Why I Built This: When “Good Search” Still Feels Wrong Most enterprise search stacks do a decent job at matching words. OpenSearch (and similar engines) commonly rely on lexical ranking families like TF‑IDF/BM25. These are strong baselines, but they tend to miss something important: user-specific intent. Three failure modes show up repeatedly in real systems: Ambiguity: the same term can mean different things to different users (or even the same user on different days). Intent drift: users’ interests change over time; yesterday’s context matters. One-size-fits-all ranking: a uniform ranking is applied across all users even though patterns differ by role, team, or experience level. A typical reaction is: “Let’s use embeddings” (dense retrieval, sentence transformers, vector databases, LTR models). That can work—but it also brings cost, latency, new infra, and sometimes reduced explainability. ...

Modern search systems increasingly rely on vector embeddings to find “similar” documents. This works remarkably well—until it doesn’t. If you have ever tried to apply off-the-shelf embeddings to a highly domain-specific corpus, you may have noticed a recurring problem: the results are linguistically reasonable but semantically wrong for your domain. This post explores why off-the-shelf embeddings often fail in the “real world” and how to build domain-adaptive embeddings that actually understand your business logic. ...

Introduction Modern search systems increasingly aim to support natural language (NL) interfaces over structured or semi-structured indices. While large language models (LLMs) are capable of translating free-form text into structured queries, deploying such systems reliably over strict query languages—such as Apache Lucene query syntax—remains challenging. Lucene’s query parser enforces precise syntactic rules. Even minor formatting mistakes (misplaced quotes, incorrect range expressions, missing operators) cause queries to fail outright. In practice, this makes naïve NL-to-query generation brittle and unsuitable for production systems. ...

Introduction Modern search platforms rarely rely on a single retrieval strategy. In practice, a production system often supports multiple search modes, such as: Keyword search for exact identifiers and error codes Semantic search for conceptual or explanatory queries Structured search where natural language is translated into a query DSL (e.g., Lucene) The challenge is not implementing these search engines individually — it is deciding, at query time, which engine to route a user query to. ...

📚 Series: Typo-Tolerant Autocomplete Part 1Autocomplete in Search : Design Constraints When You Can’t Reindex Part 2Using Tries for Prefix-Aware Typo Correction Part 3Bounding Typo Correction with Edit Distance Part 4A Query-Time Approach to Typo-Tolerant Autocomplete (current) ← Previous: Bounding Typo Correction with Edit Distance In the previous post, we looked at the constraints that make autocomplete difficult in real Lucene-based systems: large indices, high reindexing costs, tight latency budgets, and partial user input. ...

📚 Series: Typo-Tolerant Autocomplete Part 1Autocomplete in Search : Design Constraints When You Can’t Reindex Part 2Using Tries for Prefix-Aware Typo Correction Part 3Bounding Typo Correction with Edit Distance (current) Part 4A Query-Time Approach to Typo-Tolerant Autocomplete ← Previous: Using Tries for Prefix-Aware Typo CorrectionNext: A Query-Time Approach to Typo-Tolerant Autocomplete → Prefix-aware traversal narrows the search space, but it does not solve the entire problem. ...

📚 Series: Typo-Tolerant Autocomplete Part 1Autocomplete in Search : Design Constraints When You Can’t Reindex Part 2Using Tries for Prefix-Aware Typo Correction (current) Part 3Bounding Typo Correction with Edit Distance Part 4A Query-Time Approach to Typo-Tolerant Autocomplete ← Previous: Autocomplete in Search : Design Constraints When You Can’t ReindexNext: Bounding Typo Correction with Edit Distance → In the previous post, we looked at a query-time architecture for typo-tolerant autocomplete. That architecture depends on one core capability: efficiently reasoning about prefixes. ...

📚 Series: Typo-Tolerant Autocomplete Part 1Autocomplete in Search : Design Constraints When You Can’t Reindex (current) Part 2Using Tries for Prefix-Aware Typo Correction Part 3Bounding Typo Correction with Edit Distance Part 4A Query-Time Approach to Typo-Tolerant Autocomplete Next: Using Tries for Prefix-Aware Typo Correction → Autocomplete problems especially those involving the implementation of typo correction with autocomplete are rarely algorithmic in isolation. They are almost always constraint-driven. ...

This blog documents how I think about backend systems, search infrastructure, and engineering trade-offs in real-world production environments. The focus is not tutorials, but decisions.