How to Set Up Gemini CLI with Vertex AI for Enterprise

Enterprise organizations using Google Cloud can configure Gemini CLI to authenticate through Vertex AI, providing enhanced security, compliance controls, and centralized management. This guide covers the complete setup process for enterprise environments.

Why Use Vertex AI Instead of Consumer Authentication?

Vertex AI offers several advantages for enterprise deployments:

• Centralized billing: All usage is billed to your Google Cloud project
• IAM integration: Fine-grained access control through Google Cloud IAM
• Audit logging: Complete audit trails for compliance requirements
• Data residency: Control where your data is processed
• Organization policies: Enforce model restrictions across your organization
• Zero data retention: Enterprise data privacy guarantees
• Service accounts: Support for CI/CD pipelines and automated workflows

Prerequisites

Before configuring Vertex AI authentication, ensure you have:

• A Google Cloud project with billing enabled
• Google Cloud CLI (gcloud) installed and configured
• Gemini CLI installed (see How to Install Google Gemini CLI)
• Appropriate permissions to enable APIs and grant IAM roles
• Organization administrator access (for organization policy configuration)

Understanding Authentication Methods

Gemini CLI supports multiple authentication methods for Vertex AI:

Step 1: Set Up Your Google Cloud Project

Create or Select a Project

If you need a new project for Gemini CLI usage:

# Create a new project
gcloud projects create my-gemini-project --name="Gemini CLI Project"

# Set as active project
gcloud config set project my-gemini-project

Enable Required APIs

Enable the Vertex AI API in your project:

# Enable Vertex AI API
gcloud services enable aiplatform.googleapis.com

# Verify it is enabled
gcloud services list --enabled | grep aiplatform

For Google Workspace users who want to use Gemini Code Assist features, also enable:

# Enable Gemini for Google Cloud API (optional, for Code Assist)
gcloud services enable cloudaicompanion.googleapis.com

Step 2: Configure IAM Permissions

For Individual Users (ADC)

Grant the Vertex AI User role to developers who need access:

# Grant Vertex AI User role to a user
gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="user:[email protected]" \
    --role="roles/aiplatform.user"

For Service Accounts

Create a dedicated service account for Gemini CLI:

# Create service account
gcloud iam service-accounts create gemini-cli-sa \
    --display-name="Gemini CLI Service Account" \
    --description="Service account for Gemini CLI Vertex AI access"

# Grant Vertex AI User role
gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="serviceAccount:gemini-cli-sa@PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/aiplatform.user"

Role Comparison

Step 3: Authentication Setup

Method A: Application Default Credentials (Recommended for Developers)

ADC is the simplest method for developer workstations:

macOS/Linux:

# Authenticate with your Google account
gcloud auth application-default login

# Set required environment variables
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"

Windows (PowerShell):

# Authenticate with your Google account
gcloud auth application-default login

# Set required environment variables
$env:GOOGLE_CLOUD_PROJECT = "your-project-id"
$env:GOOGLE_CLOUD_LOCATION = "us-central1"

Important: If you previously set GOOGLE_API_KEY or GEMINI_API_KEY, unset them to use ADC:

# macOS/Linux
unset GOOGLE_API_KEY
unset GEMINI_API_KEY

# Windows PowerShell
Remove-Item Env:GOOGLE_API_KEY -ErrorAction SilentlyContinue
Remove-Item Env:GEMINI_API_KEY -ErrorAction SilentlyContinue

Method B: Service Account Key (Recommended for CI/CD)

For automated environments and CI/CD pipelines:

1. Create and download the service account key:

# Create key file
gcloud iam service-accounts keys create gemini-cli-key.json \
    --iam-account=gemini-cli-sa@PROJECT_ID.iam.gserviceaccount.com

2. Set environment variables:

macOS/Linux:

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/gemini-cli-key.json"
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"

Windows (PowerShell):

$env:GOOGLE_APPLICATION_CREDENTIALS = "C:\path\to\gemini-cli-key.json"
$env:GOOGLE_CLOUD_PROJECT = "your-project-id"
$env:GOOGLE_CLOUD_LOCATION = "us-central1"

Security Warning: Never commit service account key files to version control. Use secret management solutions like Google Secret Manager, HashiCorp Vault, or your CI/CD platform's secret storage.

Method C: Workload Identity (Recommended for Cloud Environments)

For workloads running on Google Cloud (GKE, Cloud Run, Compute Engine), use Workload Identity Federation for keyless authentication:

# The metadata server provides credentials automatically
# Just set the project and location
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"

Step 4: Configure Environment Variables

Required Variables

Optional Variables

Persisting Variables

Option 1: Shell Profile (macOS/Linux)

Add to ~/.bashrc, ~/.zshrc, or ~/.profile:

# Gemini CLI Vertex AI Configuration
export GOOGLE_CLOUD_PROJECT="your-project-id"
export GOOGLE_CLOUD_LOCATION="us-central1"

Reload your shell:

source ~/.zshrc  # or ~/.bashrc

Option 2: Shell Profile (Windows PowerShell)

Add to your PowerShell profile ($PROFILE):

# Gemini CLI Vertex AI Configuration
$env:GOOGLE_CLOUD_PROJECT = "your-project-id"
$env:GOOGLE_CLOUD_LOCATION = "us-central1"

Option 3: Gemini CLI .env File (All Platforms)

Create a .gemini/.env file in your home directory:

# Create directory
mkdir -p ~/.gemini

# Create .env file
cat << 'EOF' > ~/.gemini/.env
GOOGLE_CLOUD_PROJECT="your-project-id"
GOOGLE_CLOUD_LOCATION="us-central1"
EOF

Gemini CLI automatically loads variables from ~/.gemini/.env.

Option 4: Project-Specific .env File

For project-specific settings, create .gemini/.env in your project directory:

mkdir -p .gemini
echo 'GOOGLE_CLOUD_PROJECT="project-specific-id"' >> .gemini/.env
echo 'GOOGLE_CLOUD_LOCATION="europe-west1"' >> .gemini/.env

Step 5: Organization Policies (Enterprise Compliance)

Enterprise administrators can restrict which models are available using organization policies.

Restricting Model Access

Use the vertexai.allowedModels constraint to control model access:

# Create policy file
cat << 'EOF' > allowed-models-policy.yaml
name: projects/PROJECT_ID/policies/vertexai.allowedModels
spec:
  rules:
    - values:
        allowedValues:
          - publishers/google/models/gemini-2.0-flash
          - publishers/google/models/gemini-1.5-pro
          - publishers/google/models/gemini-1.5-flash
EOF

# Apply the policy
gcloud org-policies set-policy allowed-models-policy.yaml

Available Policy Actions

Data Residency and Privacy

Vertex AI provides enterprise data privacy guarantees:

• Zero data retention: Your prompts and responses are not stored or used for training
• Data residency: Choose regions that comply with your data residency requirements
• Audit logging: Enable Cloud Audit Logs for compliance tracking

Enable audit logging:

# Enable Data Access audit logs for Vertex AI
gcloud projects get-iam-policy PROJECT_ID --format=yaml > policy.yaml

# Edit policy.yaml to add audit config, then apply:
gcloud projects set-iam-policy PROJECT_ID policy.yaml

Step 6: Verify Configuration

Test Authentication

# Start Gemini CLI
gemini

# If authentication is successful, you will see the Gemini prompt
# Try a simple query
gemini "what project am I connected to?"

Verify Project and Location

# Check current gcloud configuration
gcloud config list

# Verify Vertex AI API is enabled
gcloud services list --enabled | grep aiplatform

# Test Vertex AI access directly
gcloud ai models list --region=us-central1

Common Verification Commands

# Check environment variables are set
echo $GOOGLE_CLOUD_PROJECT
echo $GOOGLE_CLOUD_LOCATION

# Windows PowerShell
echo $env:GOOGLE_CLOUD_PROJECT
echo $env:GOOGLE_CLOUD_LOCATION

Troubleshooting

"This account requires setting the GOOGLE_CLOUD_PROJECT"

Cause: Using a Google Workspace account without specifying a project.

Solution:

1. Set the GOOGLE_CLOUD_PROJECT environment variable
2. Ensure the Vertex AI API is enabled in that project
3. Verify you have the Vertex AI User role

"Permission denied" or "403 Forbidden"

Cause: Insufficient IAM permissions.

Solution:

# Grant Vertex AI User role
gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="user:[email protected]" \
    --role="roles/aiplatform.user"

"API not enabled" Error

Cause: Vertex AI API is not enabled in your project.

Solution:

gcloud services enable aiplatform.googleapis.com --project=PROJECT_ID

Service Account Key Not Found

Cause: GOOGLE_APPLICATION_CREDENTIALS points to wrong path.

Solution:

• Verify the file exists at the specified path
• Use an absolute path, not a relative path
• Check file permissions (readable by your user)

Region Not Supported

Cause: Specified region does not support Gemini models.

Solution: Use a supported region:

• us-central1 (recommended)
• us-east4
• europe-west1
• asia-southeast1
• global (for global endpoint)

Platform-Specific Notes

macOS

• Use ~/.zshrc for environment variables (zsh is default on modern macOS)
• Keychain Access can store gcloud credentials securely
• Apple Silicon Macs work with standard installation

Windows

• Use PowerShell for better compatibility
• Store service account keys in a secure location (not Desktop or Downloads)
• Consider using Windows Credential Manager for sensitive data
• WSL users can follow Linux instructions

Linux

• Use ~/.bashrc or ~/.profile for environment variables
• For headless servers, use service account authentication
• Systemd services should use environment files for credentials

Best Practices

Security

1. Rotate service account keys regularly (every 90 days)
2. Use Workload Identity when running on Google Cloud
3. Never commit credentials to version control
4. Apply least privilege - grant only necessary permissions
5. Enable audit logging for compliance tracking

Configuration Management

1. Use .gemini/.env files for project-specific settings
2. Document your configuration in team wikis
3. Use consistent regions across your organization
4. Test configuration in development before production

Cost Management

1. Monitor Vertex AI usage in Cloud Billing
2. Set budget alerts for unexpected usage spikes
3. Use appropriate model sizes - flash models for simple tasks
4. Consider quotas to prevent runaway costs

Next Steps

• Learn about organization policies for model access
• Configure VPC Service Controls for network security
• Set up Cloud Audit Logs for compliance
• Explore Gemini CLI advanced configuration

Related Resources

• Google Cloud Vertex AI Documentation
• Gemini CLI Authentication Guide
• Vertex AI IAM Roles
• Application Default Credentials