Capture AI crawlers from your host or CDN.
Last reviewed 2026-10-19
AI crawlers never run your page's JavaScript. If your site runs on a host or CDN, its request logs or an edge Worker can report crawler visits to Virent without changing your application code.
Choose your host#
Each page has the same steps as Connect analytics → Hosting integrations in the dashboard, which fills in your site ID and domain. If you control the server that renders your pages, a framework guide is an alternative.
Logs or code#
| Log-based (Log Drains, Logpush, CloudFront, DataStream 2, Netlify, Google Cloud, Fastly) | Edge Worker or middleware | |
|---|---|---|
| Sees CDN-cached pages | Yes | Workers yes; middleware only sees requests that reach your app |
| Sees the final response status | Yes | Workers yes; middleware runs before the response |
| Changes to your site | None; the host sends logs | A Worker in front of the site, or a proxy.ts file |
| Plan requirements | Often an enterprise or paid plan on the host | Available on most plans |
See Runtimes and coverage for the details of each runtime.
Before you start#
- Your site ID and server secret (
vha_sk_…) are in Settings → Connection and in the Connect analytics dialog. - The secret is shown once, after Rotate + reveal. Rotating stops the old secret, so update existing installs. See Keys and secrets.
- Keep the secret in your host's secret store. Never put it in page code or commit it.
After setup#
Every page ends the same way: send a test crawler request to your site and watch the setup dialog, which shows "Virent is receiving AI crawler traffic" once the first request arrives. Only AI crawler requests are stored and count toward your plan; search indexers, SEO tools and monitoring are dropped on arrival.