Secret Manangment with Vault

Vault is similar to LastPass in that it is a tool for securely accessing secrets (API keys, passwords, certificates, etc.). Except it runs on your servers. Your typical system requires access to a various secrets: database credentials, API keys, service passwords, etc. Vault allows you to rotate keys, store keys securely, and audit key usage.

To get started with Vault, download and install the Vault binary from their website: https://www.vaultproject.io/

wget https://dl.bintray.com/mitchellh/vault/vault_0.1.0_linux_amd64.zip

Unzip it:

unzip vault_0.1.0_linux_amd64.zip

Move the binary somewhere useful:

mv vault /bin/

Since we're just testing Vault at the moment, we'll run it in development server mode. As the development server mode is insecure, do not use it in production. Start the Vault server with:

vault server -dev

Configure our environment to work with Vault:

export VAULT_ADDR='http://127.0.0.1:8200'

Also copy the keys and tokens that the process gives out:

echo 'a5fb8a821d9038b5d43a8bb1c71145c0df807f2e734ae4a5b8b046d9a461fd57' > vault.key
echo 'dc141c6d-e1ca-1c75-4fc2-9846671ab6fa' > vault.token

We can check the status of the vault with:

vault status

Let's write our first secret using the Vault cli:

vault write secret/hello value=world

Note that we write this as secret/[key]. This is required for arbitrary secrets. Also note that sending data via the CLI is not recommended since it will be logged in shell history.

You can read the secret with:

# Read the secret
vault read secret/hello

Get the results in JSON

vault read -format=json secret/hello

Get the value from the vault in usable form

vault read -format=json secret/hello | jq -c ‘.data.value’ | tr -d ‘"’

And deleting a secret:

vault delete secret/hello

Secrets are stored in vaults. You can have several vaults mounted, and you can check them with:

vault mounts

You can manipulate mounts like:

vault mount generic
vault unmount generic/

Authentication and Authorization

<p>
    You can create policies like this:
</p>
<p>
    <code>acl.hcl</code>:
</p>
<pre>

path “sys” { policy = “deny” }

path “secret” { policy = “write” }

path “secret/rootpass” { policy = “read” }

And add them to the vault like this:

vault policy-write [policy-name] acl.hcl

Test the policy by displaying it with:

vault policies [policy-name]

You then create a token that’s associated with that policy:

vault token-create -policy="[policy-name]"

Grab the outputted token and use it to auth with the vault:

vault auth [token]

Now you’re only able to access paths as defined in the policy.

But tokens are just one method of authentication, there are several others available. For servers / instance the App-ID auth backend makes a lot of sense.

Basic process for App-ID:

  1. Create a unique application id.
  2. Store the application id in configuration management (SaltStack, etc).
  3. Provision an instance.
  4. Configuration management states provide the instance with the app-id.
  5. A user-id is created for the instance (probably a hash of the instance-id).

For that last step - Salt’s reactor system would be handy, as it could register the instance when the minion joins the master by calling a Salt Module on a Vault node.

To create an app-id you would run this command:

vault write auth/app-id/map/app-id/app1 value=root display_name=Consul

Then when the client instance boots we get an instance id. On the vault servers we then need to authorize the instance:

vault write auth/app-id/map/user-id/i-287ffno8 value=app1 cidr_block=10.0.0.0/8

The above command could be called via a Salt Module when an Event fires into the Salt Reactor system. Otherwise, alternative solutions as necessary. If managing instance permissions like that is unappealing you can use tokens or user/pass type authorization models.

With the app-id and user-id set up, the vault client could authorize with combined token (app-id+user-id) and manipulate the vault.

Using Vault with Consul

If you’ve been following along with my posts, your probably starting to notice that a lot of services I’m writing about can fit together into a larger picture in an ever growing environment.

In this case, that larger picture is combining Vault with Consul. An example config (assuming you have Consul set up - see my previous post!):

config.hcl:

backend “consul” {
address = “consul.example.com:80”
path = “consul”
scheme = “https”
}

listener “tcp” { address = “[ip]:8200” tls_cert_file = “/path/to/cert” tls_key_file = “/path/to/key” }

statsite_addr = [ip]

Load this more production ready configuration with:

vault server -config=config.hcl

Assuming that you’ve never used the newly configured backend with vault you would now run:

vault init

When you initialize the backend grab the keys and root token and securely store them somewhere. You lose these, you lose all the secrets contained in that vault.

You need the keys to unseal the vault with:

vault unseal

Note: vault seal will seal the vault, encrypting it.

With the Consul configuration loaded and the vault unsealed it can now serve requests. See the HTTP API for more details on the API.