How do I check if I am blocking AI crawlers?

Open yoursite.com/robots.txt and look for the AI user-agents. If GPTBot, ClaudeBot, PerplexityBot, and OAI-SearchBot are disallowed or absent, you may be invisible to those engines.

How long until fixes show up in AI answers?

Once crawlers can read well-structured, credible content, new pages can be cited within weeks. Entity recognition and mentions build over months.

Why is my law firm not showing up in AI search?

By rankedLawyer · Updated July 13, 2026

Your firm is likely missing from AI search for one of five reasons: AI crawlers are blocked in robots.txt, your content renders only in the browser where crawlers cannot read it, you have no structured data, your answers are buried instead of front-loaded, or your firm is not a recognized entity with real mentions. Usually it is several of these at once.

The five most common causes

Blocked crawlers: robots.txt does not allow GPTBot, ClaudeBot, PerplexityBot, or OAI-SearchBot.
Client-only rendering: content loads via JavaScript, which AI crawlers do not execute.
No structured data: nothing tells the engine what your firm is.
Buried answers: your best content is below the fold instead of front-loaded.
No entity or mentions: the firm is not recognized or cited anywhere the AI trusts.

The one most firms overlook

AI crawlers do not run JavaScript. If your site renders its content client-side, the crawler sees a nearly empty page no matter how good the content looks in a browser. Server-side rendering is not optional for AI visibility.

How to diagnose it

Check robots.txt for the AI crawlers, view page source to confirm your content is in the raw HTML, look for a JSON-LD script tag, and ask ChatGPT or Perplexity for a lawyer in your city to see who they name. Each gap you find is a fixable cause.

Frequently asked

How do I check if I am blocking AI crawlers?: Open yoursite.com/robots.txt and look for the AI user-agents. If GPTBot, ClaudeBot, PerplexityBot, and OAI-SearchBot are disallowed or absent, you may be invisible to those engines.
How long until fixes show up in AI answers?: Once crawlers can read well-structured, credible content, new pages can be cited within weeks. Entity recognition and mentions build over months.