Google Cloudintermediate

5 min read

title: Google Cloud Billing Export to BigQuery: Setup Guide & Dataset Requirements description: Enable Google Cloud billing export to BigQuery with step-by-step instructions. Includes dataset location requirements (US/EU only), service account setup, and troubleshooting for billing data analysis. difficulty: intermediate estimatedReadTime: 15 lastUpdated: January 2025 featured: true faqItems:

  • question: >- What is the best practice for selecting a Google Cloud project for billing exports? answer: >- It's advisable to create a dedicated Google Cloud project specifically for billing and FinOps tasks. This project should be linked to the Cloud Billing account from which you want to export data. By isolating billing operations in a single project, you simplify management and access control, ensuring that billing data remains organized and secure. When creating the project, ensure that it has billing enabled and that you configure it to use a clear dataset ID for easy identification.
  • question: What should I do if I encounter an invalid dataset region error? answer: >- An invalid dataset region error typically arises when the selected dataset location is unsupported. Ensure that your BigQuery dataset is in a valid region, such as US or EU, as these are required for billing exports. Avoid using multi-region settings unless necessary, and do not select customer-managed encryption keys (CMEK) unless you are certain they are compatible with billing data. If the issue persists, double-check your dataset configuration for any discrepancies.
  • question: >- How do I troubleshoot if no data appears in BigQuery after enabling billing export? answer: >- If you notice no data showing in BigQuery after enabling billing export, allow some time for the data to populate. It can take a few hours, with initial backfill potentially taking up to 5 days. Verify that the Cloud Billing account is correctly linked to the project and that the required APIs are enabled. Additionally, ensure you are checking the right dataset and that it is configured with a supported region. For resource-level data, such as GKE costs, ensure that GKE cost allocation is enabled to capture detailed information. heroImage: "https://images.unsplash.com/photo-1544717684-80f376b9dd98?w=1200&h=630&fit=crop"

Exporting your Google Cloud billing data to BigQuery allows you to analyze cost trends, generate reports, and create custom dashboards using powerful SQL queries. This guide walks you through the complete setup to enable billing exports for usage and pricing data.

Why Export Billing Data to BigQuery?

BigQuery provides a flexible way to:

  • Track costs over time
  • Break down costs by project, service, or label
  • Monitor usage at the resource level
  • Integrate with tools like Looker Studio

Note: BigQuery charges minimal fees for data storage and queries.

Step 1: Select or Create a Google Cloud Project

Choose a project that will own the BigQuery dataset used for billing exports.

🔒 Tip: It’s best practice to create a dedicated project for all your billing and FinOps tasks.

  1. Go to the Google Cloud project selector.
  2. Create a new project or select an existing one.
  3. Make sure the selected project is linked to the same Cloud Billing account you want to export data from.

Step 2: Enable Billing for the Project

Billing must be active on the selected project.

  • Navigate to Billing > Account Management in the Cloud Console.
  • Confirm the project is linked to your billing account.

Learn how to enable billing →

Step 3: Enable Required APIs

For usage and cost data:

No special API required beyond BigQuery.

For pricing data:

You must enable the BigQuery Data Transfer Service API.

  1. Go to the BigQuery Data Transfer Service API page.
  2. Select your project.
  3. Click Enable.

Step 4: Create a BigQuery Dataset

  1. Open the BigQuery console.
  2. In the Explorer panel, locate your project.
  3. Click Create Dataset.
  4. Configure:
    • Dataset ID: Use a clear name (e.g., cloud_billing_data).
    • Data location: Choose US or EU (multi-region recommended).
    • Table expiration: Leave disabled.
    • Encryption: Leave as Google-managed unless CMEK is required.

⚠️ Avoid unsupported dataset regions or customer-managed encryption keys unless you’re sure they are supported for billing export.


Step 5: Enable Cloud Billing Export

  1. Go to the Billing Export settings page.
  2. Select your billing account.
  3. For each data type, click Edit Settings:
    • Standard usage cost data – High-level cost breakdown
    • Detailed usage cost data – Cost breakdown per resource (e.g., VM, disk)
    • Pricing data – SKUs and list prices for your services
  4. Choose:
    • The project you created in step 1
    • The BigQuery dataset you created in step 4
  5. Click Save.

Service Accounts and Permissions

Google uses internal service accounts to write billing data to BigQuery. These are added automatically:

  • [email protected] (for usage data)
  • cloud-account-pricing@cloud-account-pricing.iam.gserviceaccount.com (for pricing data)

⚠️ Do not remove these service accounts from the dataset. Removing them will stop the export.

Export Timing and Retroactive Data

  • Usage data (multi-region datasets): Includes data from the start of the previous month.
  • Usage data (regional datasets): Only includes data from the day export is enabled.
  • Pricing data: Only starts from the date of export setup.

Troubleshooting and Notes

  • Invalid dataset region error: Ensure you’re using a supported dataset location (e.g., US or EU).
  • No data showing: It may take a few hours (up to 5 days for initial backfill) for data to appear.
  • Resource-level data: For GKE, enable GKE cost allocation to see detailed breakdowns.

Useful Links

Need help setting up or analyzing your Cloud Billing exports? Contact us for assistance with FinOps, dashboards, and optimization.

Need Professional Help?

Our team of experts can help you implement and configure these solutions for your organization.