Crawler Bot IP Addresses — Search Engines & AI Bots

Verify legitimate crawlers and AI bots by IP address

Fetch the latest official IP addresses for the major search engine and AI crawlers — Google, Bing, OpenAI (GPTBot, ChatGPT-User, SearchBot), Claude, and Ahrefs — in one place. IPv4 ranges are expanded to individual IP addresses for easy lookup, and IPv6 is provided as CIDR ranges. Click the button below to get the most recent list and export it to CSV.


Loading...

Get the latest officially published crawler & AI bot IP addresses

Verify legitimate search engine and AI crawlers by matching their requests against the official IP addresses published by each provider. The list covers traditional search engines (Google, Bing, DuckDuckGo), AI crawlers (OpenAI's GPTBot, ChatGPT-User, and SearchBot, Anthropic's Claude, Perplexity's PerplexityBot and Perplexity-User, and DuckDuckGo's DuckAssistBot), the Common Crawl bot (CCBot), and the Ahrefs SEO crawler.

This can help in several areas:

  • Enhanced Analytics: Differentiate between legitimate search engine crawlers and other types of traffic, ensuring accurate web analytics and insights.
  • Improved Security: Protect your site from malicious bots by distinguishing them from authentic crawlers, reducing the risk of DDoS attacks and other security threats.
  • Optimized Server Performance: Efficiently manage server load by prioritizing traffic from reputable search engines, ensuring faster and more reliable website performance.
  • AI Bot Verification: Confirm whether traffic claiming to be an AI crawler (GPTBot, ChatGPT-User, SearchBot, or Claude) actually originates from an officially published IP address.
  • Always Current: The list is fetched live from each provider every time you run the tool, so you always get the latest published ranges — re-run it whenever you do log file analysis or need to verify a crawler's IP.

What is User-agent Spoofing?

User Agent Spoofing is a technique used by web browsers and other software to disguise their identity. By altering the User Agent string, a browser can present itself as a different browser, version, or operating system. This can be used for various reasons, such as:

  • Privacy: Users may spoof their User Agent to prevent websites from tracking their browser or system specifics.
  • Compatibility: Some users may need to spoof their User Agent to access content or features optimized for different browsers or devices.
  • Testing and Development: Developers may use User Agent spoofing to test how their websites behave across different browsers and devices without switching hardware.

Log file analysis

By analyzing log files, we can see both the IP address and the User-agent string for each request. This way we can filter the legitimate ones from the non-legitimate.

Another tool you might be interested in is parsing user-agents in bulk. This provides an in-depth view and an interactive visualization on the details of the devices your users
use to access your site/application.

The IP addresses are obtained from these URLs:

  • Google: https://developers.google.com/search/apis/ipranges/googlebot.json
  • Bing: https://www.bing.com/toolbox/bingbot.json
  • Claude (Anthropic): https://claude.com/crawling/bots.json
  • OpenAI ChatGPT-User: https://openai.com/chatgpt-user.json
  • OpenAI SearchBot: https://openai.com/searchbot.json
  • OpenAI GPTBot: https://openai.com/gptbot.json
  • Ahrefs: https://api.ahrefs.com/v3/public/crawler-ip-ranges
  • DuckDuckGo (DuckDuckBot): https://duckduckgo.com/duckduckbot.json
  • DuckDuckGo (DuckAssistBot): https://duckduckgo.com/duckassistbot.json
  • Common Crawl (CCBot): https://index.commoncrawl.org/ccbot.json
  • Perplexity (PerplexityBot): https://www.perplexity.ai/perplexitybot.json
  • Perplexity (Perplexity-User): https://www.perplexity.ai/perplexity-user.json

IPv4 vs IPv6: how the list is built

Each provider publishes its IP ranges in CIDR notation (for example 66.249.64.0/27), where the number after the slash indicates how many leading bits are fixed for the network.

  • IPv4 ranges are expanded to individual addresses. Every address within each published range is listed on its own row, so you can match a log IP against a flat, static list using a simple lookup (such as Excel's VLOOKUP, a spreadsheet filter, or a WHERE ip IN (...) query) without needing CIDR containment logic.
  • IPv6 ranges are listed as CIDR ranges. A single IPv6 /64 range contains 2⁶⁴ addresses, which is impossible to enumerate, so IPv6 entries are kept in their original CIDR form. To match an IPv6 address, check whether it falls inside the range (CIDR containment) rather than looking for an exact match.

In the results, the cidr column always shows the originally published range, while the ip column is filled only for IPv4 (one row per address) and left empty for IPv6 ranges.

Here is an example of how the results look:

bot_name ip_version ip cidr
google 4 66.249.64.1 66.249.64.0/27
google 6 2001:4860:4801:10::/64
bing 4 157.55.39.5 157.55.39.0/24
openai_gptbot 4 20.171.206.10 20.171.206.0/24
ahrefs 4 5.39.1.225 5.39.1.224/27