Cloud Logging provides a native log sink feature that lets you route logs directly to a BigQuery dataset without any custom code or additional infrastructure. By creating a filter in Cloud Logging to select only Compute Engine logs, then using the 'Create sink' option with BigQuery as the Sink Service and the platform-logs dataset as the Sink Destination, logs are streamed directly and continuously into BigQuery at no extra operational cost beyond BigQuery storage/query costs. This built-in mechanism avoids the added complexity and cost of building custom Pub/Sub-triggered Cloud Run functions, scheduled batch jobs, or relying on non-existent instance metadata keys to achieve the same result.
Community Discussion