Enhancing Retrieval-Augmented Generation with XML Prompting

In my current role, I’ve been working on a Retrieval-Augmented Generation (RAG) system that provides safe, reliable personal finance guidance. With sensitive topics like budgeting, saving, and investing, outputs need to be accurate, actionable, and consistent. One change that helped the most is structuring prompts using XML.

Why XML Works Well

Formatting prompts with XML gives the Large Language Model (LLM) a clear, formal syntax and explicit instructions on how to structure its output. This works well with models like Claude 3 Haiku, which were trained on extensive XML data and handle this format reliably.

For example, a prompt might include distinct sections such as:

<role>You are a financial guide with extensive knowledge in personal finance and investments.</role>
<instructions>
    - Use only the information provided in the knowledge base to inform your responses.
    - Provide clear, concise, and personalized advice in response to the Question.
</instructions>
<examples>
    <example>
        <query>What’s a good retirement savings strategy for someone in their 20s?</query>
        <response>Start saving 12-14% of your income...</response>
    </example>
</examples>

This structured approach has a few benefits:

1. Clarity: Separating the parts of the prompt helps the model interpret each section as intended, which cuts down on errors.
2. Accuracy: With instructions, examples, and context marked out separately, the model generates responses that match the requirements more accurately.
3. Flexibility: XML tags make it easy to change parts of the prompt without rewriting the whole thing.
4. Parseability: Tagging both prompts and outputs makes it easy to extract specific parts of the response during post-processing.

Multi-Shot Prompting

Multi-shot prompting helps too. By including several examples in the <examples> section, the model recognizes patterns, adapts its tone, and produces consistent outputs.

For instance, giving examples of both simple and complex queries helps the model handle a wide range of user questions:

<examples>
    <example>
        <query>How much should I save for retirement in my 30s?</query>
        <response>By your 30s, aim to save at least 1-2 times your annual income...</response>
    </example>
    <example>
        <query>What’s the difference between a Roth IRA and a traditional IRA?</query>
        <response>A Roth IRA is funded with after-tax dollars, while a traditional IRA...</response>
    </example>
</examples>

Combined with XML structuring, this improves the model’s performance by giving it clear examples to emulate. Multi-shot prompting improves the model’s consistency and helps it generalize, while staying within the boundaries set by the <instructions> and <knowledge_base> sections.

The Impact on RAG Systems

In a RAG system, the LLM uses retrieved knowledge to generate responses. XML’s formal structure strengthens the alignment between the retrieved documents and the generated output. For instance:

• Consistency: XML keeps the response in a predefined format, which makes it easier to parse and integrate with downstream systems.
• Safety: With specific instructions (e.g., “Use only the information provided in the knowledge base”), the LLM stays away from unsupported or risky recommendations.
• Claude-Specific Benefits: The model’s familiarity with XML, combined with multi-shot examples, lets it produce predictable outputs that align with the goals of the RAG system.

Takeaway

Structuring prompts with XML is a simple technique that improves LLM performance, especially in sensitive applications like personal finance. Models like Claude 3 Haiku were trained on substantial XML data, so this approach plays to their strengths and produces more reliable, consistent outputs.

Paired with multi-shot prompting, structured XML prompts reduce ambiguity and help the model adapt to complex queries while staying precise. For anyone building a RAG system, or any application where output consistency matters, structured prompts are worth implementing.

For more detail on using XML tags to structure your prompts, see Anthropic’s documentation.