Improve clarity of AI ingestion documentation

ai-ingestion.mdx

@@ -12,40 +12,144 @@
   )
 }
 
-Documentation pages are content rich and can be a great source of information for LLMs for AI tools.
+## What is AI Ingestion?
 
-Today, you can use your documentation to fine-tune a chatbot, answer questions about your product, or use it to generate code.
+AI ingestion is the process of making your documentation accessible and consumable by Large Language Models (LLMs) and other AI tools. By properly formatting and exposing your documentation, you can:
 
-## Supported Shortcuts
+<CardGroup cols={2}>
+  <Card title="Enhanced AI Integration" icon="robot">
+    Enable AI tools to better understand and process your documentation content
+  </Card>
+  <Card title="Improved Accessibility" icon="universal-access">
+    Make your content more accessible to various AI-powered tools and platforms
+  </Card>
+  <Card title="Better User Experience" icon="users">
+    Allow users to interact with your documentation through AI-powered interfaces
+  </Card>
+  <Card title="Efficient Processing" icon="bolt">
+    Optimize how AI tools consume and analyze your documentation
+  </Card>
+</CardGroup>
 
-We support a variety of shortcuts and configurations to make it easier to use your documentation in AI tools.
+## Available Formats and Shortcuts
 
-### /llms.txt
+Mintlify automatically generates multiple formats of your documentation to support different AI ingestion needs. Here's a comprehensive guide to each format:
 
-An [industry standard](https://llmstxt.org) that helps general-purpose LLMs index more efficiently (e.g. a sitemap for AI).
+### 1. llms.txt Format
 
-Every documentation site automatically hosts a **/llms.txt** file at the root that lists all available pages in your documentation.
+<Info>
+  The llms.txt format follows the [industry standard](https://llmstxt.org) specification for helping LLMs efficiently index content, similar to how sitemaps work for search engines.
+</Info>
+
+Every documentation site automatically hosts a **/llms.txt** file at the root that provides a structured list of all available pages.
 
 <PreviewButton href="https://mintlify.com/docs/llms.txt">Open llms.txt for this site</PreviewButton>
 
-### /llms-full.txt
+Example structure of llms.txt:
+```txt
+https://your-docs.com/getting-started
+https://your-docs.com/api-reference
+https://your-docs.com/guides/authentication
+```
+
+### 2. llms-full.txt Format
 
-A markdown file of all your content is automatically available at **/llms-full.txt**.
+<Note>
+  This format is ideal when you need to provide complete documentation context to AI tools for comprehensive analysis or training.
+</Note>
 
-llms-full.txt contains the entire content of your docs, and is a great way to provide a complete context to AI tools.
+The **/llms-full.txt** endpoint provides a complete markdown version of your entire documentation in a single file.
 
 <PreviewButton href="https://mintlify.com/docs/llms-full.txt">Open llms-full.txt for this site</PreviewButton>
 
-### .md extension
+Use cases include:
+- Training custom AI models
+- Building documentation-aware chatbots
+- Creating comprehensive search indices
 
-Add **.md** to an individual docs page to see a markdown version of that page. Helps load individual pages into AI tools with better token efficiency & inference time.
+### 3. Markdown Version (.md)
 
-<PreviewButton href="https://mintlify.com/docs/quickstart.md">Open quickstart.md</PreviewButton>
+<Tip>
+  Adding `.md` to any page URL provides a markdown version of that specific page, optimized for AI processing.
+</Tip>
 
-### contextual menu
+Simply append **.md** to any documentation URL to access its markdown version:
 
-You can enable a [contextual menu](settings#param-contextual) to allow users to copy markdown source, view markdown source, or open ChatGPT with the page content.
+```plaintext
+https://your-docs.com/api-reference.md
+https://your-docs.com/getting-started.md
+```
 
-### ⌘ \+ c shortcut
+<PreviewButton href="https://mintlify.com/docs/quickstart.md">Open quickstart.md</PreviewButton>
+
+### 4. Quick Access Features
+
+#### Contextual Menu
+Enable the [contextual menu](settings#param-contextual) to provide users with quick actions:
+- Copy markdown source
+- View markdown source
+- Open content directly in ChatGPT
+
+#### Keyboard Shortcut
+Use **⌘ + c** (or **Ctrl + c** on Windows) to quickly copy the markdown source of any page.
+
+## Best Practices for AI Ingestion
+
+Follow these guidelines to maximize the effectiveness of AI ingestion:
+
+<AccordionGroup>
+  <Accordion title="Structure Your Content Well" icon="sitemap">
+    - Use clear headings and subheadings
+    - Maintain consistent formatting
+    - Include relevant metadata
+    - Use appropriate tags and categories
+  </Accordion>
+
+  <Accordion title="Optimize for AI Processing" icon="gears">
+    - Keep content concise and clear
+    - Use standard markdown formatting
+    - Avoid complex custom HTML
+    - Include code examples with proper language tags
+  </Accordion>
+
+  <Accordion title="Regular Maintenance" icon="clock-rotate-left">
+    - Keep content up to date
+    - Remove deprecated information
+    - Regularly verify external links
+    - Monitor AI tool feedback
+  </Accordion>
+</AccordionGroup>
+
+<Warning>
+  Avoid including sensitive information in your documentation as it will be accessible through these AI ingestion endpoints.
+</Warning>
+
+## Implementation Examples
+
+Here's how you might use these formats in different scenarios:
+
+<CodeGroup>
+```python Python
+import requests
+
+# Fetch markdown version of a specific page
+response = requests.get('https://your-docs.com/api-reference.md')
+markdown_content = response.text
+
+# Process with an AI tool
+ai_response = process_with_ai(markdown_content)
+```
+
+```javascript JavaScript
+// Fetch complete documentation
+async function fetchFullDocs() {
+  const response = await fetch('https://your-docs.com/llms-full.txt');
+  const content = await response.text();
+  return content;
+}
 
-Copy markdown source of any page using command \+ c.
+// Use in your AI application
+const docs = await fetchFullDocs();
+const aiResult = await processWithAI(docs);
+```
+</CodeGroup>

docs.json

@@ -24,7 +24,8 @@
               "index",
               "quickstart",
               "installation",
-              "editor"
+              "editor",
+              "ai-ingestion"
             ]
           },
           {