How to whitelist MirrorWeb on Cloudflare

May 28, 20211 min read

Depending on your Cloudflare settings you may need to whitelist the MirrorWeb user agent to allow our crawlers to archive your site.

To whitelist MirrorWeb login to Cloudflare, click Firewall and then click Firewall Rules.

On the next screen click Create a Firewall rule.Give the firewall rule a name and select User Agent from Field dropdown, contains from the Operator dropdown and enter mirrorweb in the Value box. The pick Allow from the Choose an Action dropdown. Apply this rule by clicking Deploy.