Secrets Management

HashiCorp Vault: Reading and Writing Secrets with CLI and API

Step-by-step guide to reading and writing secrets in HashiCorp Vault. Covers CLI commands, CURL API calls, JSON output, field selection, and response wrapping for secure DevOps.

By InventiveHQ Team

Managing secrets securely is fundamental to modern DevOps practices. HashiCorp Vault provides powerful CLI commands and REST API endpoints for reading and writing secrets. This guide covers all the essential operations you need for day-to-day secrets management.

Prerequisites

Before working with Vault secrets, ensure you have:

  • Vault CLI installed and configured
  • Authentication token with appropriate permissions
  • VAULT_ADDR environment variable set to your Vault server
# Set environment variable
export VAULT_ADDR='https://vault.example.com:8200'

# Verify connectivity
vault status

# Authenticate (if not already)
vault login

Understanding KV Secrets Engines

Vault's Key-Value (KV) secrets engine comes in two versions:

FeatureKV v1KV v2
VersioningNoYes
Soft deleteNoYes
MetadataNoYes
Check-and-setNoYes
CLI commandsvault read/writevault kv get/put

Most modern deployments use KV v2 for its versioning capabilities. This guide covers both versions.

Writing Secrets to Vault

Using Vault CLI (KV v2)

Write a secret with the vault kv put command:

# Write a single key-value pair
vault kv put secret/myapp password='mysecretpassword'

# Write multiple key-value pairs
vault kv put secret/myapp \
  username='admin' \
  password='mysecretpassword' \
  api_key='abc123xyz'

Using Vault CLI (KV v1)

For KV v1 secrets engines, use vault write:

vault write secret/myapp password='mysecretpassword'

Writing from Files

Write secrets from a JSON file:

# Create a JSON file with your secrets
cat > secrets.json << 'EOF'
{
  "username": "admin",
  "password": "mysecretpassword",
  "connection_string": "postgres://user:pass@host:5432/db"
}
EOF

# Write from file
vault kv put secret/database @secrets.json

# Clean up the file
rm secrets.json

Writing from Standard Input

For secure handling without files:

# Pipe from environment variable (no command history)
echo -n "$DB_PASSWORD" | vault kv put secret/database password=-

# Generate and store a random password
openssl rand -base64 32 | vault kv put secret/api-key value=-

Using CURL API

For API-based integration:

curl -X POST \
  -H "X-Vault-Token: $VAULT_TOKEN" \
  -d '{"data": {"username": "admin", "password": "secret"}}' \
  $VAULT_ADDR/v1/secret/data/myapp

Note: For KV v2, the API path includes /data/ after the mount point.

Reading Secrets from Vault

Basic Read Operations

# Read entire secret (KV v2)
vault kv get secret/myapp

# Read entire secret (KV v1)
vault read secret/myapp

Example output:

====== Secret Path ======
secret/data/myapp

======= Metadata =======
Key                Value
---                -----
created_time       2024-01-15T10:30:00.000000Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

====== Data ======
Key         Value
---         -----
password    mysecretpassword
username    admin

Reading Specific Fields

Extract a single field value:

# Get only the password field
vault kv get -field=password secret/myapp

# Use in scripts
DB_PASS=$(vault kv get -field=password secret/database)

JSON Output for Scripting

Get structured JSON output:

# Full JSON output
vault kv get -format=json secret/myapp

# Parse with jq
vault kv get -format=json secret/myapp | jq -r '.data.data.password'

Using CURL API

# KV v2 API read
curl -s \
  -H "X-Vault-Token: $VAULT_TOKEN" \
  $VAULT_ADDR/v1/secret/data/myapp | jq '.data.data'

Listing Secrets

List secret names at a path (not values):

# List secrets in a path (KV v2)
vault kv list secret/

# List secrets in a path (KV v1)
vault list secret/

# JSON output
vault kv list -format=json secret/

Deleting Secrets

Soft Delete (KV v2)

Soft delete preserves the data but marks it as deleted:

# Soft delete (can be recovered)
vault kv delete secret/myapp

# Delete specific version
vault kv delete -versions=2 secret/myapp
Advertisement

Permanent Deletion (KV v2)

Destroy permanently removes the data:

# Destroy specific versions permanently
vault kv destroy -versions=1,2,3 secret/myapp

# Destroy all versions and metadata
vault kv metadata delete secret/myapp

Delete (KV v1)

KV v1 deletion is always permanent:

vault delete secret/myapp

Response Wrapping for Secure Sharing

Response wrapping provides a secure method to share secrets without granting Vault access to recipients.

Creating a Wrapped Response

# Wrap a secret read with 5-minute TTL
vault kv get -wrap-ttl=5m secret/myapp

# Output includes a wrapping token
Key                              Value
---                              -----
wrapping_token                   hvs.CAESI...
wrapping_accessor               abc123...
wrapping_token_ttl              5m
wrapping_token_creation_time    2024-01-15T10:30:00.000Z
wrapping_token_creation_path    secret/data/myapp

Unwrapping the Secret

The recipient uses the one-time token:

vault unwrap hvs.CAESI...

After unwrapping, the token becomes invalid, ensuring one-time access.

Response Wrapping Best Practices

  • Use short TTLs (5-15 minutes) for sensitive secrets
  • Verify the wrapping token's creation path before unwrapping
  • Monitor for wrapped tokens that are never unwrapped (potential security concern)

Working with Secret Metadata (KV v2)

Reading Metadata

vault kv metadata get secret/myapp

Setting Custom Metadata

vault kv metadata put \
  -custom-metadata=environment=production \
  -custom-metadata=owner=platform-team \
  secret/myapp

Configuring Secret Settings

# Set max versions to keep
vault kv metadata put -max-versions=5 secret/myapp

# Require check-and-set
vault kv metadata put -cas-required=true secret/myapp

Best Practices

1. Never Store Secrets in Command History

# BAD: Password visible in history
vault kv put secret/app password=mypassword

# GOOD: Read from environment variable
vault kv put secret/app password="$PASSWORD"

# GOOD: Read from stdin
read -s PASSWORD && vault kv put secret/app password="$PASSWORD"

2. Use Field-Specific Reads in Scripts

# Efficient: Only retrieve what you need
DB_HOST=$(vault kv get -field=host secret/database)
DB_PASS=$(vault kv get -field=password secret/database)

3. Implement Proper Error Handling

#!/bin/bash
if ! SECRET=$(vault kv get -field=api_key secret/myapp 2>/dev/null); then
    echo "Error: Failed to retrieve secret" >&2
    exit 1
fi

4. Use Response Wrapping for Secret Distribution

Instead of copying secrets directly, wrap them:

# Generate wrapped token for deployment
WRAPPED_TOKEN=$(vault kv get -wrap-ttl=10m -format=json secret/deploy-key | jq -r '.wrap_info.token')
echo "Deploy with: vault unwrap $WRAPPED_TOKEN"

Troubleshooting

Permission Denied

Error: permission denied

Solution: Check your token's policies with vault token lookup and ensure they grant access to the secret path.

Path Not Found

No value found at secret/data/myapp

Solution: Verify the path exists with vault kv list secret/ and check the secrets engine version (v1 vs v2 use different paths).

Invalid Path

Error: Invalid path for a versioned K/V secrets engine

Solution: For KV v2, use vault kv commands instead of vault read/write.

Command Reference

OperationKV v2 CommandKV v1 Command
Writevault kv put secret/path key=valuevault write secret/path key=value
Readvault kv get secret/pathvault read secret/path
Listvault kv list secret/vault list secret/
Deletevault kv delete secret/pathvault delete secret/path
Destroyvault kv destroy -versions=N secret/pathN/A

Next Steps

For more advanced secrets management strategies, explore our complete HashiCorp Vault guide series.

Frequently Asked Questions

How do I write a secret to Vault?

Write a secret using 'vault kv put secret/myapp password=mysecret' for KV v2, or 'vault write secret/myapp password=mysecret' for KV v1. You can include multiple key-value pairs in a single command.

How do I read a secret from Vault?

Read a secret using 'vault kv get secret/myapp' for KV v2, or 'vault read secret/myapp' for KV v1. Add '-format=json' for JSON output or '-field=password' to retrieve a specific field.

What's the difference between KV v1 and KV v2?

KV v1 is a simple key-value store without versioning. KV v2 adds versioning, soft delete, metadata, and check-and-set operations. Use 'vault kv' commands for v2 and 'vault read/write' for v1.

How do I list all secrets in a path?

List secrets using 'vault kv list secret/' for KV v2, or 'vault list secret/' for KV v1. This shows secret names (keys) at that path, not the secret values themselves.

How do I delete a secret from Vault?

Delete a secret using 'vault kv delete secret/myapp' for KV v2 (soft delete), or 'vault delete secret/myapp' for KV v1. In KV v2, use 'vault kv destroy' for permanent deletion.

How do I read a specific field from a secret?

Use the -field flag: 'vault kv get -field=password secret/myapp'. This returns only the value without formatting, useful for scripts and automation.

What is response wrapping and when should I use it?

Response wrapping encrypts a secret and returns a one-time token instead. Use 'vault kv get -wrap-ttl=5m secret/myapp' to share secrets securely. The recipient unwraps with 'vault unwrap <token>'.

How do I write secrets from a JSON file?

Write from a JSON file using 'vault kv put secret/myapp @data.json' where data.json contains your key-value pairs. You can also pipe JSON: 'cat data.json | vault kv put secret/myapp -'.

How do I get secret output in JSON format?

Add '-format=json' to any read command: 'vault kv get -format=json secret/myapp'. This returns structured JSON with data, metadata, and other fields, useful for parsing with jq.

Can I write multiple key-value pairs in one command?

Yes, include multiple pairs: 'vault kv put secret/myapp username=admin password=secret api_key=abc123'. Each pair is stored as a separate field within the same secret.

hashicorpvaultsecrets managementdevopssecurity