How to Create a FinOps Project for Managing Google Cloud Billing

Implementing Financial Operations (FinOps) best practices in Google Cloud starts with creating a dedicated project to centralize billing exports, cost analysis tools, and financial dashboards. This guide walks you through setting up a FinOps project that serves as your organization's cloud financial management hub.

Prerequisites

Before you begin, ensure you have:

• Organization Administrator or Project Creator role in your Google Cloud organization
• Billing Account Administrator role on your Cloud Billing account
• Access to the Google Cloud Console or gcloud CLI installed
• Understanding of your organization's billing structure and requirements
• Basic knowledge of Google Cloud project hierarchy and IAM

Understanding FinOps Projects

A FinOps project is a centralized location for:

• Billing data exports: BigQuery datasets containing detailed cost and usage data
• Cost analysis tools: Looker Studio dashboards, scheduled queries, and reports
• Budget monitoring: Alerts, notifications, and cost anomaly detection
• Tagging and labeling: Centralized label management and enforcement
• Financial reporting: Custom views, aggregations, and executive summaries

Key benefits:

• Separates billing data from production workloads
• Provides dedicated access control for finance teams
• Centralizes all billing-related resources
• Simplifies audit and compliance reporting

Step-by-Step Guide

Phase 1: Create the FinOps Project

Using Google Cloud Console

1. Navigate to Project Creation

   • Open the Google Cloud Console
   • Click the project dropdown at the top of the page
   • Click NEW PROJECT

2. Configure Project Settings

   • Project name: Enter a descriptive name (e.g., "FinOps Hub", "Cloud Financial Management")
   • Project ID: Will be auto-generated or enter a custom ID (e.g., finops-billing-prod)
     • Must be globally unique across all Google Cloud
     • Cannot be changed after creation
     • Use lowercase letters, numbers, and hyphens only
   • Organization: Select your organization from the dropdown
   • Location: Select the appropriate folder or leave at organization root

3. Create the Project

   • Click CREATE
   • Wait for the project creation process to complete (typically 10-30 seconds)
   • Note the Project ID for later use

Using gcloud CLI

# Set your organization ID (find it with: gcloud organizations list)
ORG_ID="123456789012"

# Set billing account ID (find it with: gcloud billing accounts list)
BILLING_ACCOUNT_ID="ABCDEF-123456-ABCDEF"

# Create the FinOps project
gcloud projects create finops-billing-prod \
  --name="FinOps Hub" \
  --organization=$ORG_ID \
  --labels=purpose=finops,team=finance,environment=production

# Link the billing account
gcloud billing projects link finops-billing-prod \
  --billing-account=$BILLING_ACCOUNT_ID

# Set as current project
gcloud config set project finops-billing-prod

Phase 2: Enable Required APIs

Enable all APIs needed for billing analysis and financial management:

Using Console

1. Navigate to APIs & Services

   • In the Cloud Console, select your FinOps project
   • Click Navigation menu > APIs & Services > Library

2. Enable Required APIs

   • Search for and enable each of these APIs:
     • BigQuery API (for billing data storage)
     • BigQuery Data Transfer API (for scheduled queries)
     • Cloud Billing API (for programmatic billing access)
     • Cloud Resource Manager API (for project/resource metadata)
     • Compute Engine API (for VM pricing data)
     • Cloud Asset API (for resource inventory)

Using gcloud CLI

# Enable all required APIs in one command
gcloud services enable \
  bigquery.googleapis.com \
  bigquerydatatransfer.googleapis.com \
  cloudbilling.googleapis.com \
  cloudresourcemanager.googleapis.com \
  compute.googleapis.com \
  cloudasset.googleapis.com \
  --project=finops-billing-prod

Verify APIs are enabled:

gcloud services list --enabled --project=finops-billing-prod

Phase 3: Create BigQuery Dataset for Billing Exports

Using Console

1. Open BigQuery

   • Navigate to BigQuery in the Console
   • Ensure your FinOps project is selected

2. Create Dataset

   • Click the three-dot menu next to your project name
   • Select Create dataset
   • Dataset ID: billing_data
   • Data location: US (multi-region, required for billing exports)
   • Default table expiration: 730 days (2 years) - adjust based on retention policy
   • Click CREATE DATASET

Using gcloud CLI

# Create billing dataset with 2-year expiration
bq mk \
  --dataset \
  --location=US \
  --default_table_expiration=63072000 \
  --description="Cloud Billing export data for cost analysis" \
  finops-billing-prod:billing_data

Phase 4: Configure IAM Permissions

Set up appropriate access control for finance and engineering teams:

Finance Team Access

# Grant BigQuery Data Viewer to finance team
gcloud projects add-iam-policy-binding finops-billing-prod \
  --member="group:[email protected]" \
  --role="roles/bigquery.dataViewer"

# Grant Billing Account Viewer for read-only billing access
gcloud projects add-iam-policy-binding finops-billing-prod \
  --member="group:[email protected]" \
  --role="roles/billing.viewer"

FinOps Team Access (Full Control)

# Grant BigQuery Admin for data management
gcloud projects add-iam-policy-binding finops-billing-prod \
  --member="group:[email protected]" \
  --role="roles/bigquery.admin"

# Grant Billing Account Administrator for export configuration
gcloud billing accounts add-iam-policy-binding $BILLING_ACCOUNT_ID \
  --member="group:[email protected]" \
  --role="roles/billing.admin"

Engineering Team Access (Read-Only)

# Grant BigQuery Data Viewer for cost queries
gcloud projects add-iam-policy-binding finops-billing-prod \
  --member="group:[email protected]" \
  --role="roles/bigquery.dataViewer"

# Grant BigQuery Job User to run queries
gcloud projects add-iam-policy-binding finops-billing-prod \
  --member="group:[email protected]" \
  --role="roles/bigquery.jobUser"

Phase 5: Enable Billing Exports

Configure Cloud Billing to export data to your new FinOps project:

Using Console

1. Navigate to Billing

   • Click Navigation menu > Billing
   • Select your billing account

2. Configure BigQuery Export

   • Click Billing export in the left sidebar
   • Under "BigQuery export", click EDIT SETTINGS

3. Standard Export Configuration

   • Project: Select finops-billing-prod
   • Dataset: Select billing_data
   • Click SAVE

4. Detailed Export Configuration (Recommended)

   • Under "Detailed usage cost data", click EDIT SETTINGS
   • Project: Select finops-billing-prod
   • Dataset: Select billing_data (can use same dataset)
   • Click SAVE

5. Verify Export

   • Wait 24-48 hours for initial data to populate
   • Check BigQuery for new tables: gcp_billing_export_v1_XXXXXX_XXXXXX_XXXXXX

Using gcloud CLI

# Note: Billing export configuration must be done via Console or API
# The gcloud CLI does not directly support this operation
# Use the Console instructions above or the Cloud Billing API

Phase 6: Set Up Cost Monitoring and Alerts

Create Budget Alerts

1. Navigate to Budgets

   • In Cloud Console, go to Billing > Budgets & alerts
   • Click CREATE BUDGET

2. Configure Budget

   • Name: "Monthly Cloud Spend Budget"
   • Projects: Select all projects to monitor
   • Budget type: "Specified amount"
   • Target amount: Enter monthly budget (e.g., $10,000)
   • Threshold rules: Set alerts at 50%, 80%, 90%, 100%
   • Notifications: Add email addresses or Pub/Sub topics
   • Click FINISH

Create Anomaly Alerts (Optional)

# Create a Pub/Sub topic for cost anomalies
gcloud pubsub topics create billing-anomaly-alerts \
  --project=finops-billing-prod

# Subscribe email to the topic
gcloud pubsub subscriptions create email-alerts \
  --topic=billing-anomaly-alerts \
  --push-endpoint="https://your-webhook-endpoint.com/alerts"

Phase 7: Create Initial Cost Analysis Queries

Set up saved queries for common financial reports:

Monthly Cost by Project

-- Save this query in BigQuery for monthly reporting
SELECT
  invoice.month AS invoice_month,
  project.name AS project_name,
  service.description AS service_description,
  ROUND(SUM(cost), 2) AS total_cost,
  ROUND(SUM(usage.amount), 2) AS usage_amount,
  usage.unit AS usage_unit
FROM
  `finops-billing-prod.billing_data.gcp_billing_export_v1_*`
WHERE
  _TABLE_SUFFIX BETWEEN FORMAT_DATE('%Y%m01', DATE_SUB(CURRENT_DATE(), INTERVAL 1 MONTH))
    AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())
GROUP BY
  invoice_month,
  project_name,
  service_description,
  usage_unit
ORDER BY
  invoice_month DESC,
  total_cost DESC

Top 10 Cost Drivers

-- Identify highest cost resources
SELECT
  service.description AS service,
  sku.description AS sku,
  ROUND(SUM(cost), 2) AS total_cost,
  COUNT(*) AS line_items
FROM
  `finops-billing-prod.billing_data.gcp_billing_export_v1_*`
WHERE
  _TABLE_SUFFIX BETWEEN FORMAT_DATE('%Y%m01', CURRENT_DATE())
    AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())
GROUP BY
  service,
  sku
ORDER BY
  total_cost DESC
LIMIT 10

Phase 8: Document and Share

Create documentation for your FinOps project:

1. Create a README: Document project purpose, access procedures, and key resources
2. Share with stakeholders: Notify finance, engineering, and leadership teams
3. Schedule training: Conduct workshops on using BigQuery for cost analysis
4. Set up regular reviews: Schedule monthly cost review meetings

Best Practices

Project Organization

1. Use descriptive labels: Tag with purpose=finops, team=finance, environment=production
2. Separate environments: Consider separate FinOps projects for dev/test/prod
3. Document ownership: Clearly identify who owns and maintains the FinOps project
4. Version control: Store queries and dashboard definitions in Git

Access Control

1. Principle of least privilege: Grant minimum necessary permissions
2. Use groups: Assign permissions to groups, not individual users
3. Regular audits: Review IAM bindings quarterly
4. Service accounts: Use dedicated service accounts for automated processes

Cost Management

1. Set table expiration: Automatically delete old billing data based on retention policy
2. Use clustering: Cluster billing tables by project, service, or date for query efficiency
3. Monitor BigQuery costs: Set budgets for the FinOps project itself
4. Archive old data: Export to Cloud Storage for long-term retention at lower cost

Data Governance

1. Implement tagging strategy: Standardize labels across all projects for better cost attribution
2. Create data catalog: Document all datasets, tables, and their purposes
3. Set up data quality checks: Validate billing exports are complete and accurate
4. Enable audit logging: Track who accesses billing data and when

Reporting and Dashboards

1. Create executive summaries: Build high-level cost dashboards for leadership
2. Team-specific views: Provide filtered views for each engineering team
3. Trend analysis: Track cost trends over time to identify anomalies
4. Cost forecasting: Use historical data to predict future spending

Troubleshooting

Billing Data Not Appearing

Problem: No billing data in BigQuery after enabling exports

Solution:

• Wait 24-48 hours for initial data population
• Verify billing export is enabled: Console > Billing > Billing export
• Check that billing account is active and incurring charges
• Ensure dataset location is US multi-region
• Verify IAM permissions on the dataset

Permission Denied Errors

Problem: Users cannot access billing data

Solution:

• Grant roles/bigquery.dataViewer at project level
• Grant roles/bigquery.jobUser to allow query execution
• Verify users are authenticated with correct account
• Check that the FinOps project is linked to a billing account

API Not Enabled

Problem: "API not enabled" errors when running queries or exports

Solution:

• Enable BigQuery API: gcloud services enable bigquery.googleapis.com
• Enable Cloud Billing API: gcloud services enable cloudbilling.googleapis.com
• Wait 5-10 minutes for API enablement to propagate
• Retry the operation

Query Costs Too High

Problem: BigQuery queries consuming excessive budget

Solution:

• Use partitioned tables: Filter on _TABLE_SUFFIX for date ranges
• Add LIMIT clauses to exploratory queries
• Use clustering on frequently filtered columns
• Create materialized views for common aggregations
• Set custom query quotas per user/team

Dataset Location Mismatch

Problem: Cannot export billing data to dataset

Solution:

• Billing exports require US multi-region location
• Delete and recreate dataset with correct location
• Cannot change dataset location after creation
• Use bq show PROJECT:DATASET to verify location

Next Steps

After creating your FinOps project:

1. Configure detailed exports: Enable resource-level billing data
2. Build dashboards: Create Looker Studio reports for stakeholders
3. Implement tagging: Standardize labels across all projects
4. Set up automation: Create scheduled queries for regular reports
5. Enable GKE cost allocation: Track namespace-level Kubernetes costs
6. Integrate with tools: Connect to third-party FinOps platforms (CloudHealth, Spot.io, etc.)

Related Resources

• Create a BigQuery Dataset
• Enable Billing Exports to BigQuery
• Run Sample Billing Queries
• Understanding Billing Export Table Schemas
• Google Cloud FinOps Hub
• Google Cloud Cost Management Best Practices