> ## Documentation Index
> Fetch the complete documentation index at: https://motherfuckingsideproject.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting

> Common issues when setting up AgentXP and how to fix them.

# Troubleshooting

> Common issues when setting up AgentXP and how to fix them.

## I get HTML instead of markdown

Your request is not being detected as an AI agent. Check two things:

1. **You're sending the right header.** Use `Accept: text/markdown` in your request:

```bash theme={null}
curl -H "Accept: text/markdown" http://localhost:3000/
```

2. **Your middleware is running.** Verify `middleware.ts` exists in the correct location — either your project root or `src/middleware.ts` if you use a `src/` directory. Next.js only loads middleware from these two paths.

If you're testing with a custom bot, add it to `customAgents` in your config. See [Custom agents](/docs/guides/custom-agents).

## Markdown output is empty or minimal

This happens when your page content is rendered client-side with JavaScript. AgentXP fetches the page HTML from the server, so content loaded via `useEffect`, `useState`, or client-side API calls won't appear in the output.

**Fix:** Make sure your pages use Server Components (the default in Next.js App Router) or static generation. Content must be present in the server-rendered HTML.

You can verify what the server returns by viewing your page's HTML source:

```bash theme={null}
curl http://localhost:3000/your-page | head -100
```

If the `<main>` section is mostly empty `<div>` tags with no text content, the page is client-rendered and AgentXP can't extract content from it.

## Cookie banners, chat widgets, or ads appear in the markdown

AgentXP strips common boilerplate elements (`nav`, `header`, `footer`, `aside`, `script`), but third-party widgets injected inside your `<main>` element may survive extraction.

**Fix:** Add their CSS selectors to `excludeSelectors`:

```typescript middleware.ts theme={null}
export default withAgentXP({
  transform: {
    excludeSelectors: [
      '.cookie-consent',
      '#intercom-container',
      '.crisp-client',
      '#drift-widget',
      '#hubspot-messages-iframe-container',
      '.chat-widget',
    ],
  },
})
```

See [Common widget selectors](/docs/reference/common-selectors) for a full reference.

## Content is cut off / truncated

AgentXP limits output to 8,000 tokens by default (roughly 32,000 characters). For long pages like documentation or legal content, increase the limit:

```typescript middleware.ts theme={null}
export default withAgentXP({
  transform: {
    maxTokens: 16000,
  },
})
```

Truncated output ends with a note indicating the content was cut. Increase `maxTokens` or use `contentSelector` to target only the main content area, which reduces output size by skipping sidebar and footer content.

## Middleware causes an infinite loop or timeout

This happened in versions before 0.1.1 when the middleware's internal fetch to your origin triggered itself recursively.

**Fix:** Update to the latest version. The middleware now sends an `x-agentxp-internal: 1` header on internal fetches and skips processing when it sees that header.

If you're on the latest version and still seeing loops, check that no other middleware or proxy is stripping the `x-agentxp-internal` header from internal requests.

## `/llms.txt` returns 404

The `/llms.txt` route is handled by the middleware, not by a file in your `public/` directory. If it returns 404, the middleware is not running for that path.

**Fix:** Check your middleware matcher. If you have a custom `config.matcher` in `middleware.ts`, make sure it includes `/llms.txt`:

```typescript middleware.ts theme={null}
export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
}
```

The default matcher (when you don't export one) catches all routes, including `/llms.txt`.

## Existing middleware stops working after adding AgentXP

Pass your existing middleware as the second argument to `withAgentXP()`:

```typescript middleware.ts theme={null}
import { withAgentXP } from '@reaganhsu/agentxp-next'
import { clerkMiddleware } from '@clerk/nextjs/server'

export default withAgentXP({}, clerkMiddleware())
```

AgentXP intercepts agent requests only. For regular browser requests, it calls your middleware unchanged. If you need to run your middleware first (for example, to set up auth context that AgentXP needs), reverse the order and call AgentXP from within your middleware instead.

## Wrong content is extracted

If AgentXP is extracting sidebar content, navigation text, or other non-main content, set `contentSelector` to target your main content area:

```typescript middleware.ts theme={null}
export default withAgentXP({
  transform: {
    contentSelector: 'main',
  },
})
```

Common selectors: `main`, `article`, `#content`, `.content`, `[role="main"]`. Pick the one that wraps your page's actual content in your HTML.

## Response is slow (> 2 seconds)

AgentXP fetches your page HTML internally before transforming it. If the response is slow:

1. **Your page itself is slow.** The internal fetch has a 5-second timeout. If your page takes 3 seconds to render, the agent response takes at least 3 seconds plus transformation time.
2. **The page is very large.** Pages with 50,000+ characters of HTML take longer to transform. Set `contentSelector` to reduce the amount of HTML processed.

Transformation alone takes under 50ms for typical pages. The bottleneck is almost always the origin page load time.