Structured Data for AI Crawlers: What Markup Does, and What It Cannot Do

Structured data does two entirely different jobs, and almost every guide on the subject blends them into one. The first job is cosmetic in the best sense: you add markup, and a search engine decorates your listing with review stars, a price, an event date, a recipe time, an expandable set of questions. The second job has nothing to do with appearance. It is telling a machine what your business is — that this name refers to this organisation, which sells these things, employs these people, and is the same entity as those profiles elsewhere. Same syntax, different purpose.

The distinction matters because the two have wildly different rules of engagement. The decorative job is governed by published specifications with eligibility criteria, testing tools that tell you whether you qualify, and documentation that changes when the rules do. You can verify your work. The comprehension job has no specification, no validator, no confirmation that anything read your markup, and no feedback of any kind. You are publishing into silence and reasoning about what happens next.

That asymmetry pulls attention in an unhelpful direction. Because rich results are measurable, people optimise for the narrow set of types that produce them and stop there. Because entity description is unmeasurable, it gets skipped. But the decorative types are a short list that may or may not include anything your business does, while the descriptive work applies to every business that exists and is the part that transfers to systems that were not built by search engines.

On format, JSON-LD is the sensible default and there is no serious argument left against it. It sits in its own block, it survives a redesign, and someone can edit it without touching a template. Its one real weakness is the flip side of its strength: because it is separable from the visible page, it drifts away from the visible page. The prose gets updated and the block does not, and nothing on screen reveals the discrepancy.

Which gives the first rule worth committing to. Markup must assert what the page asserts. A page saying one thing and a data block saying another is not a neutral inconsistency to be cleaned up later — it is a positive signal that your published facts are unreliable, and it is worse than having published no markup at all. The protection is to generate the structured data from the same source as the visible content rather than maintaining two copies of the truth by hand. Where that is impossible, validate on every deploy, because this class of error is invisible to everyone including you.

The part most sites skip entirely is connecting things up. It is common to find a site with a block on the homepage describing the organisation, a block on a service page describing that service, and a block on a bio page describing a person, with nothing linking them. Three islands. Assigning stable identifiers to your entities and referring to them by those identifiers is what turns islands into a graph — the service points at the organisation that provides it, the person points at the organisation that employs them, every page's data points back at the same identified thing rather than re-declaring a fresh copy of it. Machines are good at following that kind of reference and poor at guessing that three similar descriptions are one company.

The outward half of the same idea is declaring your other selves. A property exists for exactly this purpose: listing the URLs where the same entity appears — your professional profiles, your listings in the registers relevant to your field, your entry in public knowledge bases, your presence on the platforms your industry uses. This is you stating that these accounts are the same organisation as this website, which is otherwise something a system has to infer from a name and hope it inferred correctly.

Inference is where entity confusion actually starts, and markup only helps if the underlying facts are settled. Businesses generate ambiguity honestly and constantly. A legal name that differs from the name on the sign. A rebrand where both names remain in circulation. An acquisition that left two websites running. Multiple locations that each describe the company slightly differently. A holding company and an operating company with similar names. Each is perfectly normal commercially, and each hands a machine several plausible candidates with no reliable way to merge them. The remedy is administrative rather than technical: choose the canonical name, the canonical address format and the canonical one-line description, use them everywhere without variation, and reconcile the places that disagree. This is dull work and it routinely outperforms anything you could add to a template.

For choosing types, a more useful frame than a checklist is to ask which questions about your business you want answered without anyone having to interpret a sentence. Who are you and what is your legal identity. What do you sell, as discrete things rather than as prose. Where do you operate and who do you serve. Who are your named people and what are their actual qualifications. What do you charge, in what currency, and until when is that true. What questions do customers ask and what are the direct answers. Work backwards from those questions and the types select themselves; work forwards from a list of available types and you will implement whatever is easiest.

Two of those deserve emphasis because they are handled badly. Commercial data supports an availability status and a date beyond which a price should not be relied upon, and both fields exist because prices go stale and machines cannot tell. Using them costs nothing and gives a system a reason to distrust an old figure it found elsewhere. Separately, authorship and credentials are worth stating properly wherever your field is one where who says something changes how much it is worth — a named person with a real qualification, declared as a person rather than implied by a byline.

Now the limit, stated plainly. Structured data makes you legible. It does not make you credible, and it certainly does not make you correct. Anyone can declare anything in a data block; nothing validates the claim itself. That is exactly why systems weigh corroboration so heavily — because self-assertion is free and therefore nearly worthless on its own. Markup gets you unambiguously understood. Being preferred over the alternatives comes from other sources agreeing with you, which is slower, harder, and not something you can deploy.

That brings us to llms.txt, which needs a careful description. The idea is a plain text file at the root of your domain listing the pages you consider authoritative, in a simple format that is cheap for a language model to consume, with an expanded variant carrying the content itself. The appeal is obvious: rather than making a machine crawl and guess at your site's shape, you hand it a curated map.

The honest status is that it is a community proposal that gained visible traction, not a ratified standard, and adoption among the systems it addresses is uneven and shifting. There is a fair critique that it duplicates what a sitemap already does, and an awkward open question about what incentive any provider has to honour a file in which a site nominates its own most important pages. It may become a convention everyone supports. It may quietly disappear, as several confidently announced web standards have. Publishing one costs an afternoon and might help; building an AI visibility programme on it would be a mistake, and anyone presenting it as the centrepiece of the work is selling you the least substantial part.

Access sits underneath all of this and is worth a line even though it is not markup. Crawlers used by AI systems are not the ones you know from search, and they can be blocked in two independent places: the directives in your robots file, and the bot protection at your CDN or firewall, which frequently blocks them without anyone having decided to. Note also that the controls do not map cleanly onto intentions. Opting out of having your content used for training is a different question from whether an assistant can fetch your page to answer a question right now, the available directives do not separate those neatly, and the vocabulary differs between providers. Deciding deliberately is reasonable. Discovering the decision was made for you by an inherited configuration is not.

It helps to keep an explicit ledger of what is known here, because the field encourages confident claims. Established: search engines parse JSON-LD, rich results are real and documented, and consistent identity across sources demonstrably helps disambiguation in systems that have existed for years. Reasonable inference: text a machine can parse without interpretation is easier to retrieve and summarise accurately than text it must decode, and contradictions between your sources reduce the chance you are used. Speculative: any specific claim about how a particular model weights a particular property, whether llms.txt will matter, and every ranked list of markup types by impact that you will encounter, including the order suggested below.

On sequence, the order follows from cost and dependency rather than from any measured impact. Confirm the crawlers can reach you, since nothing downstream matters if they cannot. Describe the organisation and what it sells, since everything else attaches to that. Reconcile your identity across the places you appear, which is the tedious item that pays best. Add people and credentials where your field cares about them. Restructure content only on the pages that carry real commercial weight. Publish llms.txt last, on the grounds that it is inexpensive and unproven.

The reason to do the technical layer first is not that it is the most valuable part. It is that it is the part that gates everything else, the part that stays useful whichever way these systems evolve, and the part you can complete rather than merely improve. A business that has done all of it is still only understandable, not yet preferred. That is a real distinction and it is where the harder work starts.

Want this looked at for your own business?

Эта статья описывает подход. Страница услуги показывает объём работ, результаты и стоимость.