From Website URL to Useful AI Support Answers: A Practical Training Workflow

<p>Adding a chat interface to a website is easy. Making it answer company-specific questions reliably is the real engineering and content problem.<br> A support assistant needs more than a capable language model. It needs a controlled set of sources, a retrieval workflow, a testing process, and a clear fallback when the available knowledge is insufficient.<br> This article describes a practical website-to-answer workflow for small and mid-sized products. It is based on lessons from building Oscar Chat, but the principles apply to any retrieval-based customer-support assistant.</p> <p>Start with source ownership</p> <p>Before crawling anything, define which pages the business considers authoritative.<br> A public website often contains several versions of the same information. There may be an old campaign landing page, a current pricing page, archived documentation, translated pages, and blog posts that describe features no longer available.<br> The crawler can collect all of them, but collection is not the same as truth.<br> A useful source policy answers three questions: which paths are allowed, which paths should be excluded, and which page wins when two sources conflict.<br> For a first implementation, stable pages are the safest source set: documentation, product pages, FAQs, shipping and return policies, service descriptions, and current pricing explanations.<br> Separate extraction from indexing</p> <p>A crawler retrieves pages. A knowledge pipeline turns those pages into usable evidence.<br> The extraction layer should remove repeated navigation, cookie banners, unrelated footer content, and other text that adds noise. It should preserve page titles, headings, meaningful paragraph boundaries, and the source URL.<br> The indexing layer can then divide the cleaned content into retrievable units. The exact chunking strategy depends on the page structure, but a useful chunk should retain enough context to make sense when retrieved independently.<br> A sentence such as “It is available for 30 days” is not useful without the heading or preceding description that explains what “it” means.<br> Metadata is equally important. At minimum, keep the page URL, page title, section heading, language, and crawl time. These fields help with debugging, filtering, and future refreshes.<br> Retrieval needs an uncertainty policy</p> <p>A common failure mode is allowing the assistant to answer whenever it can generate plausible text.<br> For customer support, plausible is not the standard. The answer should be supported by the retrieved source.<br> The system needs an uncertainty policy: if the available evidence is weak, conflicting, or missing, the assistant should say that it cannot confirm the answer and offer a human handoff or contact form.<br> This behavior is especially important for pricing, refunds, legal terms, medical information, financial services, and order-specific questions.<br> The best fallback is not a generic error. It should tell the visitor what can happen next: leave contact details, start live chat, open WhatsApp, or send the question to the appropriate team.<br> Build an evaluation set from real conversations</p> <p>Testing only a few ideal questions creates false confidence.<br> A practical evaluation set should include direct questions, informal questions, spelling mistakes, questions that combine two topics, unsupported questions, and questions where the correct response is a human handoff.<br> For example, a delivery policy might be tested with “How long does shipping take?”, “Will this arrive before Friday?”, “delivery netherlands?”, and “Can you guarantee arrival tomorrow?” These questions may refer to the same source but require different levels of certainty.<br> Review answers for factual accuracy, source relevance, completeness, tone, and appropriate refusal. Save failed questions and rerun them after every meaningful change to the content or retrieval workflow.<br> Treat refresh as part of the feature<