GPG Key Repo

Purpose: Manage gpg keys for:

• SOPS

Key Management

• Role: New User: new key to be added; can be a new employee being added for first time, existing employee getting access to a new repo, key rotation, etc
• Role: Existing User: user who already has access to the appropriate project
  • Definition: List of all users: verify/.sops.yaml
• Keys Repository: This Git Repository (communication-keys), manages public keys and configuration in Project Repositories
• Project Repository: Git Repository for each Project which contains SOPS-Encrypted secrets, e.g. for GitOps Deployments using Helm Files

Playbook for Configuring Access for New Users

Overview:

1. In the Keys Repo (communication-keys):
   1. New User creates,adds GPG Key
   2. Existing User configures groups
   3. Existing User configures verification SOPS Config
   4. New User installs SOPS
   5. New User verifies SOPS installation using verification SOPS Config
   6. Status: New User has working key, working sops installation. Caveat: No access to secrets in other repos yet
2. In the Project Repo
   1. Existing User adds New User Key to SOPS Config, Secrets Files
   2. New User verifies access
   3. Status: New User has access to SOPS-encrypted secrets within Project Repo

1a. Onboarding: [New User]: create and add a gpg key

1. Clone this repository
2. Create a branch titled add_pubkey_[firstname]-[lastname].
   • CLI: e.g. git branch add_pubkey_Max-Musterman
   • Note: no strict naming convention for the branch, it's strictly a Human-in-the-Loop process
3. Follow steps 1-13 at the following link: https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key
   • CAVEAT: step 14 is not necessary, as it is specific to a GitHub account
4. add ONLY the PUBLIC part of your gpg key!!! to your branch
   • file format: <email>@netgo.de.gpg.pub
   • Example: max.musterman@netgo.de.gpg.pub
5. git: commit the new file, push
6. gitlab: open a MergeRequest
7. Hand-Off: Assign the MR to an Existing User in your Team to have your key added.
   • Hint: Look up all Existing Users in the comments at: verify/.sops.yaml

1b. Onboarding: [Existing User|New User]: Add new user to groups

Prerequisite: Determine the groups to which access is needed, e.g. a specific repository. If uncertain, ask a Team Member for help!

Technical Instructions - please forgive the complexity

Create a symlink from the group-directory back to the keyfile

1. cd groups/<project_name>
   • Example: cd groups/devnso-adp-argocd
   • Explanation: Access for each repo is tracked using the ./groups/ directory; each sub-directory represents a "group" (Note: some "groups" are also "roles", e.g. admin)
   • Explanation: Most of the groups correspond directly to git repository names, aka "project name"
2. ln -s ../../<path_to_key.gpg.pub>
   • Example: ln -s ../../max.musterman@netgo.de.gpg.pub

2. Onboarding: [Existing User]: Configure sops config

Context: This repo stores the keys used to encrypt secrets in other repos; these "consumer" repos each contain a sops config .sops.yaml which manages access to the encrypted files (e.g. secrets.yaml)

For verification purposes, this repo also contains a sample .sops.yaml to which every key in the repo is added. This allows both Existing Users to instantly verify the new key, and New Users to verify that their sops installation works correctly.

Update Verification SOPS Config

1. Checkout the New User's branch titled add_pubkey_[firstname]-[lastname].
   • CLI: e.g. git branch add_pubkey_Max-Musterman
2. Run ./verify/usr_confirm_keycfg.sh
3. git: commit the new file, push
4. Status: New User has working SOPS Configuration
5. Hand-Off: Proceed to next steps; Instruct New User to verify access to secrets

Update Project SOPS Config

The following commands explain how to update the .sops.yaml for a repository.

Note: For a worked-through example, see next section.

1. Prerequisite
   1. Obtain Project Repo
      • CLI:: git clone <repo_url>
2. Create Branch
   • CLI: git checkout -b add_pubkey_[firstname]-[lastname] origin/main
3. Configure Project Repo for New User
   1. List available groups:
      • CLI: ${PATH_TO_COMMUNICATION_KEYS_REPO}/bin/update_sops.sh --list_groups
        • Reminder: Group Name usually corresponds to Repository Name
   2. Update sops config AND all secrets files:
      • CLI: ${PATH_TO_COMMUNICATION_KEYS_REPO}/bin/update_sops.sh -g <group_name> $(find . -name secrets.yaml)
4. Commit the changes, Create Change Request (PR/MR)
   1. git: commit the changes to .sops.yaml and secrets files (secrets.yaml) files
      • CLI: git add .sops.yaml $(find . -name secrets.yaml)
        • CAVEAT: check for other changes with git status ! The files do not always follow consistent conventions.
      • CLI: git commit -m "adds <firstname>.<lastname> to sops config"
   2. git: push branch
      • CLI: git push -u origin add_pubkey_[firstname]-[lastname]
   3. gitlab: open a MergeRequest, Review, Merge
   4. Status: New User has access to SOPS-Encrypted Secrets within Project Repo
   5. Hand-Off: Proceed to next steps; Instruct New User to verify access to secrets

Example

Prerequisite Obtain Repo

# E.g. update sops config for DevNSO
% git clone git@git.dev-at.de:cloud-solutions/nso/devnso-adp-argocd.git
% cd devnso-adp-argocd/

Create Branch

## OPINIONATED GIT - use preferred method
git checkout -b add_pubkey_max-musterman origin/main

Configure Project Repo for New User

# List available groups
# Output:
% ${PATH_TO_COMMUNICATION_KEYS_REPO}/bin/update_sops.sh --list_groups
# INFO: listing groups
admin
automation
devnso-adp-argocd

# For a given group, update sops config AND all secrets files - New Users cannot add themselves!
# Output:
% ${PATH_TO_COMMUNICATION_KEYS_REPO}/bin/update_sops.sh -g devnso-adp-argocd $(find . -name secrets.yaml)
# RUN: generate SOPS config
# RUN: gpg --import *.gpg.pub
# RUN: sops updatekeys ./loki/loki/secrets.yaml
...
# SUCCESS: all users with keys in this dir should have functional keys

Commit the changes, Create Change Request (PR/MR)

# git: commit the changes to `.sops.yaml` and secrets files (`secrets.yaml`) files
% git add .sops.yaml $(find . -name secrets.yaml)
# **CAVEAT**: check for other changes with `git status` ! The files do not always follow consistent conventions.
# output:
% git status
...
no changes added to commit (use "git add" and/or "git commit -a")

% git commit -m "adds max.musterman to sops config"

git push -u origin add_pubkey_max-musterman

Now proceed to with the remaining steps, i.e. gitlab: open a MergeRequest, Review, Merge

3. Onboarding: [New User] Configure SOPS

SOPS is used for encrypting secrets, e.g. credentials for various systems

Install

1. Install Sops

https://github.com/getsops/sops

Note:

• MacOS: If desired, one can also use brew to install sops: brew install sops; although this is not officially maintained, the formula is essentially the same as the official installation instructions

2. Configure

Add the following to your ~/.bashrc or ~/.zshrc - but not to your ~/.profile as it must be set per session:

# Enable interactive passphrase prompt for SOPS
export GPG_TTY=$(tty)

3. Verify

Run the following command to verify local SOPS installation and key configuration:

# Follow the interactive prompts:
./verify/usr_confirm_keycfg.sh

Usage

Decrypt and Display Secrets in Terminal:

GPG_TTY=$(tty) sops secrets.yaml

Note: The GPG_TTY is necessary to have the password prompt appear. src: https://www.varokas.com/secrets-in-code-with-mozilla-sops/

Note: secrets.yaml is just an example; the file can have any name

4. Offboarding: [Existing User]: Archive Expired Keys (EOL)

To mark a key as expired:

1. move it to the archive/ dir
2. for each group, update the project repo
3. remove the key from the group

1. This repo: archive

# archive key - DO NOT delete - need this for auditing
git mv ${keyname} "archive/${keyname}_$(date '+%Y-%m-%d').archive"
# remove from verification sops
./verify/usr_confirm_keycfg.sh

2. For each group / repo:

Prerequisite: Local copy of each repo corresponding to a group

# list all groups to which the key is registered
find groups/ -name ${keyname}

# For each group, update sops config in that repo
# Example:
% cd devnso-adp-argocd
% ${PATH_TO_COMMUNICATION_KEYS_REPO}/bin/update_sops.sh -g devnso-adp-argocd
# now git commit, push, etc

3. This repo: update groups

# remove from groups
find groups -name ${keyname} | xargs git rm

Advanced

Reference: Commands for gpg keys

import gpg keys

gpg --import /path/to/keys/*.gpg.pub

list imported gpg keys

gpg --list-keys --keyid-format=long

SOPS Example - Manual

The steps in the following example can be run locally in order to:

• create a sample secrets file
• encrypt the file
• decrypt the file

If these steps work, sops is configured correctly - on your machine ;-)

#!/usr/bin/env bash
set -ueo pipefail
# demo: create a file with a mock secret, src: https://bash-org-archive.com/?244321
# PREREQUISITE: valid sops config, i.e. .sops.yaml - Note: most repos already have one
# further reading: https://github.com/getsops/sops?tab=readme-ov-file#using-sops-yaml-conf-to-select-kms-pgp-and-age-for-new-files
yq -n '.demo.credentials.secret = "hunter2"' > secrets.yaml
# encrypt
sops -e -i secrets.yaml

# decript, print to console
sops -d secrets.yaml

Contributing

Tests: ./verify/test.sh

Caveat: requires working SOPS config,pgp key, etc