Capture AI crawlers on Google Cloud.

For sites behind a global external Application Load Balancer. Request logs go through Cloud Logging and a Pub/Sub push subscription to Virent. The push URL carries a token: replace its value with your server secret (URL-encoded); the dashboard fills it in after Rotate + reveal.

You need your site ID and server secret from Settings → Connection (see Hosting integrations). In the dashboard, Connect analytics → Hosting integrations → Google Cloud shows these steps with your site ID, domain and secret filled in. Below, replace <your-site-id>, <paste-your-server-secret> and example.com with your own values.

Configure Google Cloud so example.com sends reliable AI crawler request telemetry to Virent.

Site ID: <your-site-id>
Use the full Virent server secret from the Connect dialog. Do not use the masked preview "vha_sk_••••" as the real value.

Use the current provider documentation: https://docs.cloud.google.com/load-balancing/docs/https/https-logging-monitoring

Steps:
1. Enable 100% load-balancer logs: Enable request logging on every backend service behind the global external Application Load Balancer and set the logging sample rate to 1.0 (100%).

2. Export matching request logs: Create a Cloud Logging sink to a Pub/Sub topic with this filter. Grant the sink writer identity permission to publish to the topic.

resource.type="http_load_balancer"
httpRequest.requestUrl:"example.com"

3. Create push delivery: Create a Pub/Sub push subscription using this HTTPS endpoint. The token is in the URL because Pub/Sub push subscriptions do not support arbitrary custom authorization headers; rotate the Virent secret if the URL is exposed.

https://virent.app/api/ingest/google-cloud?token=%3Cpaste-your-server-secret%3E

4. Activate and verify: Send an AI crawler request to example.com, confirm the subscription is acknowledged, and verify the event in Virent. Cloud Logging exports are usually delivered within seconds but are not synchronous.

Keep the secret server-side, preserve 100% request coverage where the provider supports sampling, and verify both normal site delivery and a crawler request after activation.

Steps#

1. 01

   Enable 100% load-balancer logs

   Enable request logging on every backend service behind the global external Application Load Balancer and set the logging sample rate to 1.0 (100%).

2. 02

   Export matching request logs

   Create a Cloud Logging sink to a Pub/Sub topic with this filter. Grant the sink writer identity permission to publish to the topic.

   Log Router sink filter

   resource.type="http_load_balancer"httpRequest.requestUrl:"example.com"

3. 03

   Create push delivery

   Create a Pub/Sub push subscription using this HTTPS endpoint. The token is in the URL because Pub/Sub push subscriptions do not support arbitrary custom authorization headers; rotate the Virent secret if the URL is exposed.

   Push endpoint

   https://virent.app/api/ingest/google-cloud?token=%3Cpaste-your-server-secret%3E

4. 04

   Activate and verify

   Send an AI crawler request to example.com, confirm the subscription is acknowledged, and verify the event in Virent. Cloud Logging exports are usually delivered within seconds but are not synchronous.

Check Google Cloud's current documentation if its screens differ from these steps.

Verify#

Send a test crawler request to your site. The setup dialog in Virent shows "Virent is receiving AI crawler traffic" once the first request arrives.