Search

Why I Built This: When “Good Search” Still Feels Wrong Most enterprise search stacks do a decent job at matching words. OpenSearch (and similar engines) commonly rely on lexical ranking families like TF‑IDF/BM25. These are strong baselines, but they tend to miss something important: user-specific intent. Three failure modes show up repeatedly in real systems: Ambiguity: the same term can mean different things to different users (or even the same user on different days). Intent drift: users’ interests change over time; yesterday’s context matters. One-size-fits-all ranking: a uniform ranking is applied across all users even though patterns differ by role, team, or experience level. A typical reaction is: “Let’s use embeddings” (dense retrieval, sentence transformers, vector databases, LTR models). That can work—but it also brings cost, latency, new infra, and sometimes reduced explainability. ...

Introduction Modern search systems increasingly aim to support natural language (NL) interfaces over structured or semi-structured indices. While large language models (LLMs) are capable of translating free-form text into structured queries, deploying such systems reliably over strict query languages—such as Apache Lucene query syntax—remains challenging. Lucene’s query parser enforces precise syntactic rules. Even minor formatting mistakes (misplaced quotes, incorrect range expressions, missing operators) cause queries to fail outright. In practice, this makes naïve NL-to-query generation brittle and unsuitable for production systems. ...

Introduction Modern search platforms rarely rely on a single retrieval strategy. In practice, a production system often supports multiple search modes, such as: Keyword search for exact identifiers and error codes Semantic search for conceptual or explanatory queries Structured search where natural language is translated into a query DSL (e.g., Lucene) The challenge is not implementing these search engines individually — it is deciding, at query time, which engine to route a user query to. ...

📚 Series: Typo-Tolerant Autocomplete Part 1Autocomplete in Search : Design Constraints When You Can’t Reindex Part 2Using Tries for Prefix-Aware Typo Correction Part 3Bounding Typo Correction with Edit Distance Part 4A Query-Time Approach to Typo-Tolerant Autocomplete (current) ← Previous: Bounding Typo Correction with Edit Distance In the previous post, we looked at the constraints that make autocomplete difficult in real Lucene-based systems: large indices, high reindexing costs, tight latency budgets, and partial user input. ...

📚 Series: Typo-Tolerant Autocomplete Part 1Autocomplete in Search : Design Constraints When You Can’t Reindex (current) Part 2Using Tries for Prefix-Aware Typo Correction Part 3Bounding Typo Correction with Edit Distance Part 4A Query-Time Approach to Typo-Tolerant Autocomplete Next: Using Tries for Prefix-Aware Typo Correction → Autocomplete problems especially those involving the implementation of typo correction with autocomplete are rarely algorithmic in isolation. They are almost always constraint-driven. ...