What is Applebot-Extended ?
Applebot-Extended is a user-agent declared by Apple that functions as an extension of Applebot, Apple’s web crawler. According to Apple’s official documentation (https://support.apple.com/en-us/HT212614), this crawler is used to access content for training Apple’s foundational models used in generative AI features such as Apple’s virtual assistant and Spotlight search enhancements. It expands upon the standard Applebot’s indexing functionality by incorporating web data into broader AI training workflows.
Who is operating Applebot-Extended ?
Applebot-Extended is operated by Apple Inc., the multinational technology company headquartered in Cupertino, California. It is an internal part of Apple’s AI and Search technologies division. Further technical details and bot management instructions are available on Apple’s support site: https://support.apple.com/en-us/HT212614.
Why you should be interested in Applebot-Extended ?
As a website owner, you should be aware that Applebot-Extended goes beyond simple indexing for search results—it collects data potentially used for training generative AI models. This includes not only content parsing but also its use in downstream applications within Apple’s ecosystem. The crawler’s activity could influence bandwidth usage and data exposure. Apple has implemented an opt-out mechanism, allowing publishers to control inclusion in model training datasets.
How to block Applebot-Extended?
1. Robots.txt File:
To disallow content collection by Applebot-Extended specifically for AI training, add the following directive:
# block Applebot-Extended User-agent: Applebot-Extended Disallow: /
2. To block all Applebot activity (including search indexing), use:
# block Applebot User-agent: Applebot Disallow: /
More information on bot behavior and blocking policy can be found at: https://support.apple.com/en-us/HT212614
About the bot
Owner: Apple Inc.
Owner URL: apple.com
Bot URL: support.apple.com
Bot User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Applebot-Extended/1.0
Respects robots.txt: Yes
