Meta's new terms reach past your Page to your website
If your shop has a Facebook Page or runs Instagram ads, Meta's updated terms say it may crawl your website and use what it finds to train its products, starting October 30. Robots.txt is still the lever you hold. Do you know what yours says?

From October 30, 2026, any business using Facebook, Instagram or Meta's ad tools accepts terms that let Meta crawl the websites tied to its products and use that content to train Meta's products. Meta says robots.txt still works and the goal is product data. Decide crawler by crawler, check what your robots.txt really says, look at what a free login unlocks, and keep dated copies.
What changes on October 30?
Meta's Commercial Terms pick up a new paragraph on October 30, 2026. It applies to anyone using Meta for business: a Facebook Page, Instagram ads, marketing messages, a product catalog, Ads Manager, Meta Business Suite.
The paragraph says you authorize Meta, "or any entity acting on our behalf," to use "web scrapers, bots, robots, spiders, crawlers, and user agents" and to "create test credentials" so it can crawl the websites "associated with your products or services," pull updates from them now and then, and use what it finds to "provide, develop, improve, research, train, and maintain" Meta's products. Nieman Lab reported it on October 15 after a small publisher spotted the notice in his Business Suite dashboard.
Agreement happens by carrying on. If you're still posting to your Page on October 31, you've accepted. There's no partial opt-out. The terms' alternative is to stop using Meta's products for business.
Meta's answer came today. Andy Stone, its vice president of communications, posted on X that the reports are "false" and that the update is "designed to make sure we're able to pull accurate information about products for sale on third-party websites when making on-platform ad and content recommendations, not about scraping news sites."
That may be exactly what Meta plans to do. The paragraph itself doesn't say so. PPC Land's line-by-line read of the October 30 text points out that it never mentions products, prices, or catalogs. So if you run a store, you're being asked for a wide permission and told only a narrow piece will be used. The piece you can still set yourself is in robots.txt.
Meta says robots.txt still works. Does it?
Meta told Nieman Lab that "content owners can still manage how their content is accessed by using robots.txt," and its crawler documentation (last updated May 21, 2026) explains how. The three legal experts Nieman Lab asked said the new contract gives Meta an opening to argue that it overrides blocking instructions. Meta says it won't. The terms don't mention robots.txt at all.
We can't settle a contract question for you, and we won't pretend to. The practical point is narrower. Robots.txt is the one signal Meta has publicly said it honors, and it's the one you control this afternoon.
Two caveats come straight from Meta's own page. The crawler that fetches pages because a user asked for them may bypass robots.txt. The link-preview crawler might bypass it when it runs security or integrity checks. Changes can also take up to 24 hours to land, because the crawlers cache your file. So don't leave it for the morning of the 30th.
And keep the old rule in mind: robots.txt is a request to well-behaved bots, not a lock. Anything that truly must not be read needs real access control, not a line in a text file.
Meta's side of this is now written into a contract. Your side is still a text file at /robots.txt, so make sure it says what you mean.
Which Meta crawler am I deciding about?
"Block Meta" sounds like one decision. It's really five. Meta lists five crawlers, each with its own user-agent name, and they do different jobs. Blocking all of them at once can cost you things you want, like preview cards on shared links or a chance to be cited in Meta AI answers.
The one that needs the most thought is meta-externalagent. Meta describes it as crawling for "training foundation AI models or improving products by indexing content directly." Same name, both jobs, so you can't let it index and refuse training. If you're fine with Meta indexing your product pages but would rather your original buying guides not end up as training data, robots.txt lets you split by folder for that one crawler. Allow /products/, say, and disallow /guides/. Meta's own example uses exactly that kind of per-folder rule.
For most shops our starting point would be: keep the link-preview crawler, leave the ads crawler open on product and collection pages, treat meta-webindexer as part of your AI search plan, and make the meta-externalagent call on purpose, folder by folder. If you're already working on how AI answers describe you, our note on whether AI answers point to your official page is the other half of that decision.
Decision guide
Meta's five crawlers, one decision each
| Consideration | What Meta says it does | If you disallow it |
|---|---|---|
| facebookexternalhit | Builds the title, description and thumbnail when someone shares your link. | Shared links may lose their preview card. Meta says integrity checks might still visit. |
| meta-externalads | Crawls to improve advertising and other business products. | Closest to Meta's stated product-data reason. Weigh it against the ads you run. |
| meta-externalagent | Crawls for training foundation AI models or indexing content for Meta products. | One name covers both jobs, so you can't allow indexing and refuse training. Split by folder if you need to. |
| meta-webindexer | Improves Meta AI search; Meta says allowing it helps it cite and link your content. | Less chance of being cited or linked in Meta AI answers. |
| meta-externalfetcher | Fetches a link because a user asked, including for agent tasks. | May bypass robots.txt anyway. Protect private pages with real permissions. |
What about pages behind a login?
The clause also lets Meta "create test credentials" to reach the covered sites. The terms don't explain what that means. James Grimmelmann of Cornell Law School told Nieman Lab it could technically let Meta "sign up as a fake user" to get past registration walls. Meta declined to say whether it would.
You don't need to settle that to act on it. List what sits behind sign-in on your site: wholesale price lists, member-only guides, B2B quote pages, files customers uploaded. Then ask one plain question about each. Is a free account the only thing standing between a stranger and this page?
If it is, that page is already open to anyone willing to type an email address. Meta is just the newest reason to notice. Pages that should stay private need a real permission step, like a wholesale account a person approves or a role that has to be granted. An email confirmation isn't that.
I sell products. Is there more to do than block or allow?
Yes, and for a store it's probably the part that matters most.
Take Meta at its word: it wants accurate product information from your site for ads and recommendations. PPC Land also reported on October 15 that Meta will turn on product browsing in existing ads on October 28, so a tap on an ad may open an in-app product page before the shopper ever reaches your site. If that's where things are going, whatever your product pages say is what people will see inside Meta's apps.
So read your own pages the way a crawler would. Is there a sale price still in the page code from a promotion that ended? An "in stock" flag on a variant that sold out? A delivery promise that was true in August? Compare your five best sellers' pages against your catalog feed and what the cart actually charges. When those three disagree, it's the customer who finds out. Our note on scoring catalog truth walks through that check.
Sequence of work
Two weeks, in order
- Oct 16
Meta says robots.txt still works. Open your live file and find out who writes it.
- This week
Read a week of logs for Meta's five user agents and the pages they hit.
- By Oct 27
Publish per-crawler, per-folder rules. Crawlers may cache the file for up to 24 hours.
- Oct 28
Product browsing in existing Meta ads switches on, as reported. Check your top product pages.
- Oct 29
Save dated copies of your robots.txt and Meta's terms.
- Oct 30
The updated Commercial Terms take effect. Continued business use counts as agreement.
What should we do before the 30th?
Here's the short list for whoever owns your website:
- Open the live file at yourdomain.com/robots.txt and find out who writes it. Your store platform, your CDN, or a plugin may be generating it without anyone on the team knowing.
- Pull a week of server or CDN logs and look for Meta's five user agents. Note which pages each one actually visits.
- Write the decision down, crawler by crawler and folder by folder, in sentences a non-developer can read. Then publish the rules.
- List anything a free account unlocks, and put a real permission step on what should stay private.
- Check your top product pages against your feed and your checkout.
- On October 29, save dated copies of your robots.txt and of Meta's terms page. If anyone argues later about what you allowed and when, you'll have it.
Leaving Meta altogether is a business decision, and for some shops it's a real one. If it isn't for you, this list is the next best thing.
Crawler rules, product pages that stay accurate, and account areas with real permissions are the kind of work we do in ecommerce and web application projects. If you open your robots.txt and can't tell what it's saying to Meta, send it over.
Related work: Howcast (publishing across platforms).
We build alongside the people doing the work.
Bring us the problem you’re thinking about.