HashiCorp Vault

Generate Certificates with Hashicorp Vault
Hashicorp Vault Setup
Clustering HashiCorp Vault
HashiCorp Vault Cluster Unseal
Vault Wrapping Tokens
Vault Token Administration
Vault Single-Node Unseal
Vault Administrative Setup
Vault as Root CA
Vault as Intermediate CA
Handling Vault Node Restart

Generate Certificates with Hashicorp Vault

Here are steps to generate SSL certificates using HashiCorp Vault as an Intermediate CA.

NOTE: Be sure that you've setup a vault instance as an Intermediate CA.
See this page for how: Vault as Intermediate CA

For the latest intermediate CA url, see this page: Vault Services

Find the issuing role by navigating to Secrets/PKI/Roles.

Select the role, and click Generate Certificate:

Fill in the Common name as: router.ogsofttech.lan.

Set the TTL to 1 year (365 days).

Click Generate, to create the key and certificate, and you’ll see this:

Download the private key as: router.ogsofttech.lan-key.pem

Download the certiticate as: router.ogsofttech.lan-cert.pem

Download the CA chain as: router.ogsofttech.lan-cabundle.pem

NOTE: We are calling the downloaded CA chain file a “ca bundle”.
CA bundle is the standard naming convention for this file type.
Specifically, a cert is often concatenated with the CA bundle that signed it, to create a chain certificate file.

Now, you can copy the cert, ca bundle, and private key to the host, for usage.

If generating a pair for a linux host, you will need them as .crt and .key files.
Follow this: Converting PEM to crt and key

If generating a pair for an Nginx host, you will need

Hashicorp Vault Setup

Here are steps for setting up a secrets store using Hashicorp Vault, on Ubuntu 24.

References

Lots of steps were taken from here: https://developer.hashicorp.com/vault/tutorials/secrets-management/pki-engine

Server Setup

Before installing Vault, perform steps from this page, to setup the server: Ubuntu Host Setup

Since a typical vault cluster has limited internet visibility, it may be necessary to map in the local NTP server, to keep each node in sync.
See this page for how to setup each host/VM to use the local private NTP server: Ubuntu: Use Private NTP Server

Install Vault

Here are three lines to install vault.

First, we will setup the package source keys…

sudo wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

Now, add the package source entry…

echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list

Now, install vault…

sudo apt update && sudo apt install vault

Vault User

The installer created a user called, vault, and the service runs under it.
But, the default keys are restricted to root permissions.
So, we need to fix access to certificates.

For this, we will create a group to make it easier to control and test access to certs and keys.

sudo groupadd pki

And, we will add the vault user to the group:

sudo usermod -a -G pki vault

Clustered Config

NOTE: If you are standing up a single node vault, skip this section.

Work through the steps on this page, for clustering your vault servers: Clustering HashiCorp Vault

Vault Configuration

NOTE: This is for a single vault node. If you are standing up a cluster, skip this section.

Edit the config file at: /etc/vault.d/vault.hcl

Specifically, you want to set the tls cert/key paths to your own domain certs.

As well, you may want to limit the listener to just a single address, since the default is for all adapters on the host.

Once done, save the config update.

Continue Setup

Once installed and configured, we need to enable and start the vault services, with these lines:

sudo systemctl daemon-reload
sudo systemctl enable vault
sudo systemctl start vault
sudo systemctl status vault

You should be able to verify that the vault service is active by call this:

https://<host-ip>:8200/v1/sys/seal-status

NOTE: Use your VM’s fully-qualified name (or IP address), in the above.

If successful, the above call will return something like this:

{
  "type":"shamir",
  "initialized":false,
  "sealed":true,
  "t":0,
  "n":0,
  "progress":0,
  "nonce":"",
  "version":"1.17.6",
  "build_date":"2024-09-24T19:48:40Z",
  "migration":false,
  "recovery_seal":false,
  "storage_type":"file"
}

Cluster Unseal

If you're standing up a cluster, see this page: HashiCorp Vault Cluster Unseal

If you're standing up a single-node vault instance, see this page: Vault Single-Node Unseal

Administrative Setup

Once the vault is unsealed, you need to setup auditing and administrative policies.

See this page for how: Vault Administrative Setup

Creating Users (Access Tokens)

Once the vault cluster is setup, you need to establish some administrative users.

See this page for how to create admin and user tokens: Vault Token Administration

To protect tokens in transit, see this page for Response-Wrapped Tokens: Vault Wrapping Token

Login

Once your vault is unsealed, you can log into its UI.

Log into its UI, here: http://<host-ip>:8200/

You will be prompted to enter your root token, here:

If successful, you will see the root user’s dashboard:

Creating the First Secret Store

Now, you should create a secrets engine, to store things.
The one that makes the most sense, first, is to make a KV store.
You should be able to make a kv secrets store (actually, a kv-v2 store) from the dashboard:

The above will create the secrets store at the path: /kv
You can change this if needed.

Once created, you will see the new empty store:

Using Vault as a CA

Follow steps on this page, to setup a vault instance as a Root CA: Vault as Root CA

NOTE: You never want to directly sign certificates with a Root CA.
If you do, there is no way to revoke the root CA, without collapsing all PKI chains (by revoking the root CA certificate).
However. You can create an Intermediate CA that does signings.
And, that Intermediate CA can be revoked, without losing the root CA certificate.

So, we will create an Intermediate CA, to do actual signings.
And, the Intermediate CA's certificate will be signed by the Root CA, just before offlining the Root CA.

Once a root CA is setup, you can follow this page, to setup an Intermediate CA,
that will do actual signings: Vault as Intermediate CA

See this page for how to generate certificates: Generate Certificates with Hashicorp Vault

Clustering HashiCorp Vault

Here are special instructions for setting up a vault cluster.

NOTE: See the regular setup page for other details: Hashicorp Vault Setup

DNS Resolution

Since the vault services will communicate with eachother over TLS, they will need certificates.
And as such, the certs will include hostnames.

So, open the /etc/hosts file of each vault host, and add entries, at the bottom of the file, for each instance and API host.

Here's an example list of entries for a cluster:

192.168.75.10 vault02api
192.168.75.21 vault0201
192.168.75.22 vault0202
192.168.75.23 vault0203
192.168.75.24 vault0204
192.168.75.25 vault0205
192.168.75.26 vault0206

Filesystem Changes

Raft Folder

The Vault service will be running Raft. So, it will need a folder for the Raft backend.

NOTE: This may mean that the folder /opt/vault/data is obsolete.
But, we will not worry about that, for now.
The 'data' folder was created by the installer as the FS location for a storage = 'file' backend.

Create the raft folder with these:

sudo mkdir /opt/vault/raft

sudo chmod 700 /opt/vault/raft

sudo chown vault:vault /opt/vault/raft

TLS Folder

The installer already created the TLS folder, to store certificates.
It is at: /opt/vault/tls

We will leave it as is.

Config Folder

The installer created a config folder at: /etc/vault.d

We need to bolster its permissions, as it may contain seal stanzas.

Update permissions of the config folder with these:

sudo chmod 0750 /etc/vault.d

sudo chown root:vault /etc/vault.d

Firewall Rules

Update the local firewall rules for each vault host, to allow 8200 and 8201 access.

sudo ufw allow 8200

sudo ufw allow 8201

Certificates

We will create certificates for each vault instance, and put them in the tls folder at: /opt/vault/tls

Follow instructions, here, to generate certificates for each host: Generate Certificates with Hashicorp Vault

NOTE: Be sure to do the following:
- Set the common name to the fully qualified name: ex: vault0204.ogsofttech.lan.
- Set the expiry to two years (17520 hours).
- Set set the SAN IP to the address of the host: 192.168.75.24

Copy the CA certificate bundle (CA chain of issuer and root CA) into: /opt/vault/tls/ca.crt

NOTE: The ca.crt file should be the certificate bundle of issuer CA cert and root CA certificate.
These should be bundled (root + intermediate) in the ca.crt file, as a concatenated PEM.

Concatenate the vault service certificate with the issuer CA cert as a concatenated PEM.

Paste the certificate file (vault service + issuer CA cert) into: /opt/vault/tls/vault.crt

NOTE: The vault.crt file should include the leaf certificate (of the node) plus the signing intermediate.

Copy the vault service private key into: /opt/vault/tls/vault.key

Once key and certs are stored, we need to set permissions on the files, with these:

# Ownership: keep vault:vault
sudo chown vault:vault /opt/vault/tls/vault.crt /opt/vault/tls/vault.key /opt/vault/tls/ca.crt

# Permissions:
# private key: only vault access
sudo chmod 0600 /opt/vault/tls/vault.key
# server cert usually fine as world-readable
sudo chmod 0644 /opt/vault/tls/vault.crt
# CA cert usually fine as world-readable
sudo chmod 0644 /opt/vault/tls/ca.crt

Here's a quick sanity check, to verify the certificates on a host:

# Verify the server chain file against your trust bundle
openssl verify -CAfile /opt/vault/tls/ca.crt /opt/vault/tls/vault.crt

# See what the gateway/clients will see
openssl s_client -connect vault0204:8200 -showcerts -verify_return_error \
  -CAfile /opt/vault/tls/ca.crt </dev/null

Vault.HCL Changes

Now, we need to create a config file for each vault service (vault.hcl).

There will be some tailoring required for each host.
So, pay attention to the notes.

Open the vault config file at: /etc/vault.d/vault.hcl

sudo nano /etc/vault.d/vault.hcl

Below is a config file for a single node in a vault cluster.

NOTE: You will need to change the following lines for each host:
api_addr
cluster_addr
node_id

# Vault Configuration for Clustering.
# Created on 20250901

ui            = true
cluster_name  = "vaultcluster2"
# recommended with Integrated Storage
disable_mlock = true

# Advertise addresses (MUST be correct & reachable)
# Define the external API address that clients will use to communicate with this Vault node.
# set per node
api_addr     = "https://vault0204.ogsofttech.lan:8200"
# Define the internal address used by Vault nodes to communicate with this Vault node.
# Vault ignores the scheme of this URL, so it doesn't matter if http or https.
# set per node (node's intra-cluster IP:8201)
cluster_addr = "https://192.168.75.24:8201"

listener "tcp" {
  address                  = "0.0.0.0:8200"
  tls_disable              = 0
  tls_cert_file            = "/opt/vault/tls/vault.crt"
  tls_key_file             = "/opt/vault/tls/vault.key"
  tls_client_ca_file       = "/opt/vault/tls/ca.crt"
  # optional: require client certs from automation:
  # tls_require_and_verify_client_cert = "true"
}

# Updated to use raft backend (Integrated Storage).
storage "raft" {
  path    = "/opt/vault/raft"
# unique on each node  
  node_id = "vault0204"

  # Auto-join peers (preferred over manual joins)
  retry_join {
    leader_api_addr      = "https://vault0204.ogsofttech.lan:8200"
    leader_ca_cert_file  = "/opt/vault/tls/ca.crt"
  }
  retry_join {
    leader_api_addr      = "https://vault0205.ogsofttech.lan:8200"
    leader_ca_cert_file  = "/opt/vault/tls/ca.crt"
  }
  retry_join {
    leader_api_addr      = "https://vault0206.ogsofttech.lan:8200"
    leader_ca_cert_file  = "/opt/vault/tls/ca.crt"
  }
}

# Recommended health endpoint behavior for LBs
# (defaults are fine; LB should treat 200 as active, 429 as standby, 503 sealed)
# See /v1/sys/health docs for details.

Service File (Systemd Unit)

With folders fixed, certs and config defined, we need to configure the vault service for operation.

To do so, we need to identify the service file that the installer created, with this:

sudo systemctl cat vault.service

As of 20250803, the installer puts the systemd unit file, here:

/usr/lib/systemd/system/vault.service

We need to open it and update it:

sudo nano /usr/lib/systemd/system/vault.service

Once updated, save and close the systemd file.

Return to the generic vault setup page: https://wiki.galaxydump.com/link/434#bkmrk-continue-setup

HashiCorp Vault Cluster Unseal

These steps are for a new vault cluster that has been configured and started up, but is in an unsealed state.

For an existing cluster with unsealed nodes, see this page: Handling Vault Node Restart

Leader Initialization

The following will initialize a new vault cluster and return a set of unseal keys and a root token.

NOTE: The ca.crt file is privileged, You will need to run these commands as the root user.
Run the following to switch to the root user:

sudo -i

Go to the first node, and do these (as root):

# From an admin shell that can reach the VLAN:
export VAULT_ADDR="https://vault0204.ogsofttech.lan:8200"
export VAULT_CACERT="/opt/vault/tls/ca.crt"   # path on your admin box

# Initialize the cluster (choose your own shares/threshold)
vault operator init -key-shares=5 -key-threshold=3

NOTE: Use the fully-qualified hostname above, as it appears in the node's cert.

Once executed, the vault node will reply with 5 unseal keys and an initial root token.

Distribute each of these unseal keys to trusted admins, to store in offline password storage.

NOTE: Three (3) unseal keys are required to unseal the vault.

Use the initial root token to setup policies and auth.
Then, retire it.

Unseal the Leader

With the unseal keys from the initialized node (received above), we need to unseal its vault.

NOTE: We do this, while still as root, and on the same host that we got the keys from.

Now, unseal each node, by calling this command once each, for three of the five unseal keys:

NOTE: It will prompt you for the unseal key, each time you run it.

vault operator unseal

With the vault unsealed, we need to perform an initial login as root:

# Log in with the root token for initial setup tasks
vault login <root_token>

Once logged in, you can check the vault status with this:

vault status

If successful, you should see Initialized: true, Sealed: false, HA Enabled: true, and this node as leader.

The first node is online, and the cluster is up... sort of.

Each cluster member has auto-discovered a leader and established a RAFT quorum.

But, the other nodes are still not unsealed (since we did not configured auto-unseal).

Unseal Other Nodes

Similar to what you did, to unseal the first node, we will do the same to each member, below.

Switch to root on each node with:

sudo -i

Set exports for each node:

NOTE: Make sure that the vault_addr variable is pointing to the local node being unsealed, here.

export VAULT_ADDR="https://vault0205.ogsofttech.lan:8200";
export VAULT_CACERT="/opt/vault/tls/ca.crt"

NOTE: Use the fully-qualified hostname above, as it appears in the node's cert.

Now, unseal each node, by calling this command once each, for three of the five unseal keys:

NOTE: It will prompt you for the unseal key, each time you run it.

vault operator unseal

Check Status

Once you have initialized the leader node, and unsealed all nodes, we need to confirm that the cluster is good.

Run this:

NOTE: The vault_addr should be pointing to the leader, here.

# Set this if you are coming back to this page, and the environment value is not set...
export VAULT_ADDR="https://vault0204.ogsofttech.lan:8200";
# Run this to check status...
vault status

Confirm RAFT peers with this:

vault operator raft list-peers

NOTE: The above may only work on the current leader, because https.
We need to work through why this is, and solve it, so it can be run on any node.

When run, you will see something like this:

If healthy, you will see one node as leader, and the others as voting followers.

NOTE: Make sure each node you configured, is present.

There is a health check that can be performed, by calling this:

curl -s -H "X-Vault-Token: $VAULT_TOKEN" "$VAULT_ADDR/v1/sys/ha-status" | jq .

NOTE: Be sure that the VAULT_TOKEN and VAULT_ADDR environment variables are set.
Or, you can hardcode them with a minimal privilege user account.

When run, you will get a JSON list of nodes and their status.

Vault Wrapping Tokens

When creating access tokens for HashiCorp Vault, you always want to prevent them from falling into the wrong hands, or showing up as clear-text in command line history, logs and audit trains.

To reduce the chance of tokens being passed in the clear, you can create a new user token in a response-wrapped state.
This allows the token to be given to a user, over chat, or text, without much concern.

The response-wrapped token has a very short lifetime, and can only be redeemed once.

Here's how to create a response-wrapped token:

vault token create -role=admin -orphan -wrap-ttl=5m -format=json

The command response will include the wrapping token, like this:

{
  "request_id": "",
  "lease_id": "",
  "lease_duration": 0,
  "renewable": false,
  "data": null,
  "warnings": null,
  "wrap_info": {
    "token": "hvs.tH5Wn8bD3eJxvMq1iP7F",
    "accessor": "PNowpcQP0jJ9Jpz06o2oueYW",
    "ttl": 300,
    "creation_time": "2025-09-04T02:26:49.859843035Z",
    "creation_path": "auth/token/create/admin",
    "wrapped_accessor": "5KlfcQwbo05Ej37YOqBJnfHM"
  }
}

The wrap_info/token property is what you give to the user.

The user can then, redeem their access token by submitting the wrapping token, like this:

vault unwrap hvs.tH5Wn8bD3eJxvMq1iP7F

Vault will respond with the real token, and revoke further usage of the wrapping token.

NOTE: The wrapping token has an expiry. If exceeded, the user will need to request another.

Wrapped Token Benefits

Using wrapped tokens, prevents exposure in shell history or logs.

Delivery is safer, as you can drop the short-lived wrapper into a config management pipeline or paste in a chat.

The Vault audit log records that the wrapping token was created and who unwrapped it.

Vault Token Administration

Here are notes on access token administration.

Be sure that you've already setup an administrative policy in your vault cluster.
See this page for how: Vault Administrative Setup

Creating Admin Tokens

Once the admin policy exists, you can create administrative access tokens.

To do so, log into the leader note, with this:

# Use the root token...
vault login <root_token>

You can create a simple admin token that lasts 24 hours:

# Create an admin token that lasts 24 hours...
vault token create -policy=admin -ttl=24h

The above will issue short-lived admin tokens, so that the vault is more protected.
These can be minted by an automated script, that issues token as needed.

Traditional Admin

A more traditional administrative token needs to be longer-lived than a 24 hour token.

So, we will create a token role for admins.

This will create a periodic, orphan role so that admin tokens can be renewed forever (until revoked), and aren't tied to a parent token:

NOTE: You will need to be logged into the vault leader (vault login <token>).

vault write auth/token/roles/admin \
  allowed_policies="admin,default" \
  orphan=true \
  renewable=true \
  period=720h

The above token role, can create tokens that last 30 days, and can be renewed, without changing the token.

With the above admin role, you can create admin tokens of a more traditional lifetime, with this:

vault token create -role=admin -orphan -format=json

The above command will create admin tokens that can be given to your ops group, for longer-lived administrative access.

NOTE: Be sure to save the client_token field in a secure place, such as a private password manager.

Token Renewal

Before a token expires, you can renew it with this command:

VAULT_TOKEN=<admin_token> vault token renew

NOTE: The above can be automated, if the admin token is used by scripts or services.
(Automate with a small systemd timer/cron on a secure host or in your gateway.)

Creating Other Admins

You can use an admin token to create other admins, like this:

# Another admin token (same role)
vault login <admin_token>
vault token create -role=admin -orphan -format=json

Creating Tokens for other Groups

Once you have other roles established as policies, you can create tokens for users, in those roles/policies, like this:

vault token create -policy=kv-readers -ttl=8h

Response-Wrapped Tokens

See this page for Response Wrapped Tokens: Vault Wrapping Token

Vault Single-Node Unseal

Here are instructions on how to unseal a single-node vault cluster.

Vault CLI Comms

For the vault command to communicate with the running service, we must set an env variable for it:

export VAULT_ADDR=https://your_domain:8200

NOTE: You may have to set the above to http, if you haven’t given vault a valid ssl cert, yet.

Initialize the Vault

In order for the vault service to manage secrets, you must initialize its store.
To do this, you need to run the following:

vault operator init -key-shares=3 -key-threshold=2

You can change the key shares and threshold based on your risk profile.

The vault CLI will respond with something like this:

Unseal Key 1: eZcJeydRrqeSMZ1zTN+VVll9TFT2KvJy7VlnxUgtvuz5
Unseal Key 2: ntmqCKq8rgNnKT1YSLCjVmCCZBAA3NwUeqxIyRpYD4Wm
Unseal Key 3: 3FK1+Hsorh4p8/L9mki3VskaEU2eQhLqGOI/pJkTHMbx

Initial Root Token: s.hY0ieybfDqCadz7JpL88uO3x

Save these values in a secure location, as this is the only time you will receive them.

Now, you can run the vault status command, and see it has changed to Initialized=true.

Run this to get vault status:

vault status

Now, you need to unseal the vault, so it can be used.

Notice the vault status showed an unseal progress of 0/2.
This means, that at least two more unseal tokens must be submitted for it to be available for access.

Enter enough tokens to unlock your vault with this command, and paste in one of the tokens when prompted:

vault operator unseal

Once the unseal threshold has been met, the Sealed state will become ‘false’.

Your vault is available for access.

Vault Administrative Setup

Once you have a vault instance or cluster unsealed, you can setup auditing, and administrative policies, with these instructions.

See this page for how to setup a vault instance or cluster: Hashicorp Vault Setup

Audit Logging

Create a folder for capturing audit logs on each node (leader and followers):

sudo mkdir -p /var/log/vault
sudo chown vault:vault /var/log/vault
sudo chmod 0750 /var/log/vault

Tell the leader node to store audit logs in the created folder:

vault audit enable file file_path=/var/log/vault/audit.log

Admin Policy

WARNING: The initial root token has broad privilege, and bypasses all policy checks, being meant for bootstrapping and emergencies.
You need to create an actual admin policy, so that proper security and auditing can occur.

Here, we will setup an admin policy, so that we can stop using the initial root token.

Doing so, gives us many benefits:

Users created under the admin policy share the same privileges as the root user.
We can issue short-lived admin tokens.
We can audit all admin actions (audit logs will show policy=admin and the user, not just root).
We can later, reduced privileges, instead of everyone having anonymous root access.

NOTE: We only have to create the admin policy once.
We can do it on the current leader.

Create a file called: admin.hcl:

sudo nano /etc/vault.d/admin.hcl

NOTE: It doesn't matter where we generate the admin.hcl file, as the policy write command will pull it into the vault.
But for simplicity, we will create it on the leader node, so we have easy tracking of what setup steps have been done.

Give the admin policy file, this content:

# --- token management (mint/lookup/renew/revoke/roles) ---
path "auth/token/create"         { capabilities = ["update"] }
path "auth/token/create/*"       { capabilities = ["update"] }
path "auth/token/lookup"         { capabilities = ["read", "update"] }
path "auth/token/lookup-self"    { capabilities = ["read"] }
path "auth/token/renew"          { capabilities = ["update"] }
path "auth/token/renew-self"     { capabilities = ["update"] }
path "auth/token/revoke"         { capabilities = ["update"] }
path "auth/token/revoke-self"    { capabilities = ["update"] }
path "auth/token/roles"          { capabilities = ["list", "read"] }
path "auth/token/roles/*"        { capabilities = ["create","read","update","delete","list"] }

# --- policies (so admins can maintain policies without root) ---
path "sys/policies/acl"          { capabilities = ["list"] }
path "sys/policies/acl/*"        { capabilities = ["create","read","update","delete","list"] }

# --- core admin knobs (optional but typical) ---
path "sys/audit"                 { capabilities = ["read"] }
path "sys/audit/*"               { capabilities = ["create","read","update","delete","sudo"] }
path "sys/auth"                  { capabilities = ["read"] }
path "sys/auth/*"                { capabilities = ["create","read","update","delete","sudo"] }
path "sys/mounts"                { capabilities = ["read"] }
path "sys/mounts/*"              { capabilities = ["create","read","update","delete","sudo"] }

# --- give admin wide access to secrets during bootstrap (tighten later) ---
path "*"                         { capabilities = ["create","read","update","delete","list","sudo"] }

Once created, save and close.

Here are some notes about the above policy:

The top block allows an admin to perform all token management actions.
The middle block allows admins to manipulate policies.
The third block allows admins to perform other root actions.
The bottom block (path "*") allows access to all paths in the vault (like root).
- capabilities = create/read/update/delete/list - These allow full browsing and editing.
- capabilities = sudo - allows doing system-level actions like:
  - enabling audit devices,
  - enabling auth methods,
  - mounting new secrets engines,
  - etc.

So, the above admin policy is just like root, but allows us to have named users as admins.

Enable the admin policy with this:

vault policy write admin admin.hcl

Once written, the policy is stored in RAFT, and replicated to all nodes.

We can confirm the policy with this:

vault policy read admin

Now, any tokens with the admin policy, will function with the above administrative privileges.

Creating Admin Tokens

Once the admin policy exists, you can create administrative access tokens.

See this page for how to create admin and user tokens: Vault Token Administration

To protect tokens in transit, see this page for Response-Wrapped Tokens: Vault Wrapping Token

Vault as Root CA

Here are steps you can follow to setup a vault instance as a Root CA.

NOTE: This page assumes that you have created a single-node vault instance to serve as your Root CA.
See this page for how to do that: Hashicorp Vault Setup

NOTE: These steps will create a root CA with one signing key.
You should create an intermediate CA, as well, that will perform the actual signing of certs.
This will allow you to offline this root CA, once the intermediate CA is up.

Starting the PKI Engine

NOTE: From here down, is steps for creating a Root CA.
If you are setting up an Intermediate CA, skip to ‘Configure Intermediate CA’.

For Vault to serve as a root CA, you have to add the PKI secrets engine.

To do this, enable the pki secrets engine:

For a Root CA, set the Max lease time to as long as possible, as the root CA will be offlined after generation.

Once initialized, it will look like this:

For a root CA, click Configure PKI to begin setup.

Choose the Generate Root option.

Set Type to Internal.

Set your Common name. Usually, this is a domain with a private TLD suffix.

Set the issuer name.

Set a TTL that is maxed out: 87600 hours.

Scroll down and fill in the issuer URLs, matching the origin for your root CA host:

When finished, click Done, to generate the root CA key and certificate.

You will see a page like this:

Copy out the root CA certificate, and save it to a file, named: ogsofttech.lan_ca.crt

Install it on all machines of the local network, so they will recognized the signed SSL certs of hosts.

Current Intranet Root CA Certificate

The current root CA for the local intranet can be found in the secure share at this path:

\SecureShare git\oga\ogsofttech.lan\rootCA

NOTE: This certificate should be installed on any host that will consume services signed by the root ca.

See this page for how to install it on an Ubuntu host: How to Add Root CAs to Ubuntu

Root CA Rotation

To make things easier, when it comes time to rotate your root CA key, add a role, now.

Click PKI.

Click Roles.

Click Create Role.

Give the new role a name that you will recognize as the root CA rotation role: CA_rotation_role

Leave the rest of the fields empty, and click Create.

Now, you have a working root CA key/cert for your network.

We will use it to sign the certificate of an Intermediate CA, that will do all the actual work.

And, we will offline the root CA.

Vault as Intermediate CA

Here are instructions for setting up a vault instance as an Intermediate CA.

NOTE: This page assumes that you have created a single-node vault instance to serve as your Root CA.
See this page for how to do that: Hashicorp Vault Setup

NOTE: Always create the Intermediate CA as a separate instance from your Root CA.
This allows you to offline the Root CA, to uphold the integrity of your root CA certificate.

To properly sign certificates, you will want to generate an Intermediate CA that will do all the actual issuing.
And, you will offline the root CA, after the Intermediate CA is usable.

See this page if you need to setup a vault instance as a Root CA: Vault as Root CA

Configure Intermediate CA

For Vault to serve as an Intermediate CA, you have to add the PKI secrets engine.

To do this, enable the pki secrets engine:

For an Intermediate CA, set the Max lease time to 43800 hours, and enable the engine.

Once initialized, it will look like this:

For an Intermediate CA, click Configure PKI to begin setup.

Choose the Generate intermediate CSR option, and fill in the blanks.

Set Type to Internal.

Set your Common name to your domain with a suffix that it’s an Intermediate CA, like this:

ogsofttech.lan Int CA 01

Click Generate, to make the CSR.

Paste the CSR onto the clipboard, and save it as a file: pki_intermediate.csr.

Open the web UI for your root CA, and navigate to Secrets/PKI/Issuers, and click Sign Intermediate.

Paste the CSR content into the CSR field.

Set the domain name as the common name: ogsofttech.lan

Select pem_bundle from the Format dropdown:

Click Sign, to sign the CSR from the Intermediate CA.

Once signed, you’ll see this:

It contains the signed certificate for the Intermediate CA, the cert of the issuing CA, and the CA chain cert.

NOTE: Save all three in a known place, as this is the only time these will be available.

Download the certificate as: ogsofttech.lan-intca01.cert.pem

Open the UI for the Intermediate CA, and navigate to the Import a CA:

NOTE: You get to this, by clicking on the Issuers tab, and clicking Import.

Upload the certificate that you saved to: ogsofttech.lan-intca01.cert.pem

Click Import Issuer.

The Issuers tab will show two entries, one for the root, and one for the Int CA:

Issuing Role

Before you can issue certificates to hosts and such, you need to create an issuer role.

To do this, we need to identify the issuer’s Guid.
We can get this by clicking the terminal button, and running this:

read -field=default pki/config/issuers

Note the Guid that was returned.

Now, go to Secrets/PKI/Roles, and create a new role.

Give the role a name in this format: ogsofttech-dot-lan

Unclick the Use Default Issuer, and select the Guid that matches what we found above.

Set the TTL to 43800 hours.

At the bottom of the form…

In the Allowed Domains field, add our domain: ogsofttech.lan.

Check the “Allow Subdomains” checkbox.

And, click Create, to make our issuing role.

Now, you have a role that can create and sign certificates.

If not already done, you can offline your root CA, as it needs to be kept safe, and only needed to create a new Intermediate CA.

See this page for how to generate certificates: Generate Certificates with Hashicorp Vault

Handling Vault Node Restart

Each time a Vault node restarts, you will have to unseal it, so that it can participate in the cluster.
Here are steps to do that.

Switch to root on the node with:

sudo -i

Set exports for the node:

NOTE: Make sure that the vault_addr variable is pointing to the local node being unsealed, here.

export VAULT_ADDR="https://vault0205.ogsofttech.lan:8200";
export VAULT_CACERT="/opt/vault/tls/ca.crt"

NOTE: Use the fully-qualified hostname above, as it appears in the node's cert.

Now, unseal each node, by calling this command once each, for three of the five unseal keys:

NOTE: It will prompt you for the unseal key, each time you run it.

vault operator unseal

Once unsealed, you can verify cluster membership with this:

vault operator raft list-peers

When run, you will see something like this:

If healthy, you will see one node as leader, and the others as voting followers.

NOTE: Make sure each node you configured, is present.

HashiCorp Vault

Generate Certificates with Hashicorp Vault

Hashicorp Vault Setup

References

Server Setup

Install Vault

Vault User

Clustered Config

Vault Configuration

Continue Setup

Cluster Unseal

Administrative Setup

Creating Users (Access Tokens)

Login

Creating the First Secret Store

Using Vault as a CA

Clustering HashiCorp Vault

DNS Resolution

Filesystem Changes

Raft Folder

Firewall Rules

Certificates

Vault.HCL Changes

Service File (Systemd Unit)

HashiCorp Vault Cluster Unseal

Leader Initialization

Unseal the Leader

Initial Root Login

Unseal Other Nodes

Check Status

Vault Wrapping Tokens

Wrapped Token Benefits

Vault Token Administration

Creating Admin Tokens

Traditional Admin

Token Renewal

Creating Other Admins

Creating Tokens for other Groups

Response-Wrapped Tokens

Vault Single-Node Unseal

Vault CLI Comms

Initialize the Vault

Vault Administrative Setup

Audit Logging

Admin Policy

Creating Admin Tokens

Vault as Root CA

Starting the PKI Engine

Current Intranet Root CA Certificate

Root CA Rotation

Vault as Intermediate CA

Configure Intermediate CA

Issuing Role

Handling Vault Node Restart