Protect your secrets with BlackBox

Safely store secrets in version control

What?

TLDR: Stores encrypted secrets in a private repository and provides tools to manage access for multiple users (including automated ones).

Note: This is a technique used frequently in software engineering, but is definitely applicable everywhere!

Developers often maintain secret API keys, certificates, database passwords, and other information that would be dangerous if exposed publicly. These files are often stored in plain text in a private repository, and we just hope nobody gains access.

With a vault tool, these files are encrypted using a system of public and private keys. These keys are used to sign messages that can be shared safely and reliably decoded by the intended recipients.

It also simplifies the process of managing access to these secrets with asymmetric encryption. This means each user can have their own pair of private and public keys to access the secrets.

Why?

Obviously, you don't want to store secret things in plain view. When building applications, you will inevitably need to maintain a password, SSL certificate, API key, or some other form of information that should not be exposed. In some cases, you might even need to share these with other people.

Tools like Blackbox make this process simple and safe, and can protect you in some common misfortunes.

Getting Started

The instructions assume you are initializing a new repository.

For instructions on how to use an existing one, you should consult the README of that repository.

⚠ Create a temporary backup of any existing files you might be working with. With just a few typos or missteps, It is possible to lose access to these files!

1. Generate GPG Key

The first thing you will need is a GPG key. You can generate a key using a few methods:

📄 Github Docs (Windows, Mac OS, Linux)
🔐 GPG Suite (Mac OS)

Windows Users

Using GPG on Windows might take a few additional steps. There are a few ways about this, but a common one is to use Cygwin.

I believe there are instructions for Cygwin at the beginning of this tutorial:
nstalling rTorrent on Windows

You can then install the gnupg and git packages with either a package manager like Chocolatey or apt-cyg

2. Connect GPG Key

Add your GPG key to your Git provider. On Windows machines, you will need to use a bash-compatible terminal, like Cygwin.

Code:

gpg --list-secret-keys --key-id-format LONG

    /home/username/.gnupg/pubring.kbx
    -------------------------------
    sec   rsa2048/3804BB82D39DC0E3 2020-05-22 [SC]
          409B6B1796C275462A1703113804BB82D39DC0E3
    uid                 [ultimate] Your Name <emailAddress>
    ssb   rsa2048/85BB5679C71866D7 2020-05-22 [E]

We need to export the key to use it in Github. Using your key ID (e.g. 3804BB82D39DC0E3), run this command:

Code:

    gpg --armor --export 3804BB82D39DC0E3

You should see a long string of random characters printed out in a block. This is your public key. Copy it to your clipboard.

In Github, go to Settings > SSH and GPG > New GPG key. Paste your key into the text box and save it.

3. Install Blackbox

Depending on your system, you can also install Blackbox in a number of ways:

Mac OSX

Code:

    brew install blackbox

Linux

Code:

    cd /opt
    git clone https://github.com/StackExchange/blackbox.git
    cd blackbox
    sudo make manual-install

     # Restart your terminal to use the binaries

📄 Official Installation Instructions for other methods.

4. Initialize Repository

If you are starting from scratch, you will need to create a git repository:

Code:

# From the directory your secrets repository will be

mkdir my-secrets && cd my-secrets
git init

    Initialized empty Git repository in /home/username/my-secrets/.git/

5. Initialize Blackbox

If you are starting from a repository without Blackbox, you will need to initialize a Blackbox system:

Code:

blackbox_initialize

Enable blackbox for this git repo? (yes/no) yes
VCS_TYPE: git

NEXT STEP: You need to manually check these in:
    git commit -m'INITIALIZE BLACKBOX' .blackbox /home/username/my-secrets/.gitignore

Blackbox will generate a few files. You should commit these now.

Code:

git commit -am "initialize blackbox"

    [master (root-commit) abc375] initialize blackbox
     3 files changed, 3 insertions(+)
     create mode 100644 .blackbox/blackbox-admins.txt
     create mode 100644 .blackbox/blackbox-files.txt
     create mode 100644 .gitignore

Now add your GPG key to Blackbox. You can identify your key the e-mail address you assigned to it, or the GPG key ID.

Code:

blackbox_addadmin <emailAddress>

    gpg: keybox '/home/username/my-secrets/.blackbox/pubring.kbx' created
    gpg: /home/username/my-secrets/.blackbox/trustdb.gpg: trustdb created
    gpg: key 3804BB82D39DC0E3: public key "Your Name <emailAddress>" imported
    gpg: Total number processed: 1
    gpg:               imported: 1

Commit these changes as well.

Code:

git commit -am "++ admin: 409B6B1796C275462A1703113804BB82D39DC0E3"

    [master f7e82f0] New admin: 409B6B1796C275462A1703113804BB82D39DC0E3
     3 files changed, 1 insertion(+)
     create mode 100644 .blackbox/pubring.kbx
     create mode 100644 .blackbox/trustdb.gpg

Encrypt a File

1. First, make a fake secret file to play with:

Code:

    echo 'APP_SECRET=secretPa$$w0rd!' >> .env

Now, register the file:

Code:

blackbox_register_new_file .env

    ========== PLAINFILE .env
    ========== ENCRYPTED .env.gpg
    ========== Importing keychain: START
    gpg: Total number processed: 1
    gpg:              unchanged: 1
    ========== Importing keychain: DONE
    ========== Encrypting: .env
    ========== Encrypting: DONE
    ========== Adding file to list.
    ========== CREATED: .env.gpg
    ========== UPDATING REPO:

    NOTE: "already tracked!" messages are safe to ignore.

    [master eaad108] registered in blackbox: .env
     3 files changed, 2 insertions(+)
     create mode 100644 .env.gpg

    ========== UPDATING VCS: DONE
    Local repo updated.  Please push when ready.

git push

2. You should now see a new file by the same name,
but appended with the file extension [code single].gpg[/code]. If you open the file, its contents should be encrypted.

Accessing your File

Before diving into these methods, you should be aware of how to safely clean up after yourself:

Code:

blackbox_shred_all_files

========== FILES BEING SHREDDED:
    SHRED: .env
========== DONE.

This should safely remove any decrypted files while still maintaining their encrypted originals.

There are a few methods exposed for reading your encrypted files, each with a specific use case in mind.

Quickly log the file's contents

This method will quickly dump the files contents into your terminal and then clean up after itself.

Code:

blackbox_cat .env

========== PLAINFILE ".env"
========== Importing keychain: START
gpg: Total number processed: 1
gpg:              unchanged: 1
========== Importing keychain: DONE
========== EXTRACTING .env

APP_SECRET=secretPa$$w0rd!

Quickly edit a file's contents

When you need to make a quick edit or addition to a file from your terminal, you can use the edit command. This will open the file in your default editor, and re-encrypt the file after you exit.

Code:

blackbox_edit .env

========== PLAINFILE ".env"
========== ENCRYPTED ".env.gpg"
========== PLAINFILE ".env"
========== Importing keychain: START

## your file will open in $EDITOR 

========== Importing keychain: DONE
========== EXTRACTING .env
========== PLAINFILE ".env"
========== ENCRYPTED ".env.gpg"
========== Encrypting: .env
========== Encrypting: DONE
========== UPDATED ".env.gpg"

Likely next step:
    git commit -m".env.gpg updated" ".env.gpg"

There will sometimes be a number of unchanged files referenced in the output here. You can validate your changes were made by checking [code single]git status[/code]

Edit with cleanup

You can also perform a similar action to the open-ended edit above, but with some cleanup functions included:

Code:

blackbox_edit_start .env

========== PLAINFILE ".env"
========== Importing keychain: START
gpg: Total number processed: 1
gpg:              unchanged: 1
========== Importing keychain: DONE
========== EXTRACTING .env

ls -la

...
-rw-r--r-- 1 user   group   28 Jul 25 11:43 .env

Once you're done with your edits (however you might have completed them), you can simply run the following command to encrypt your changes.

Code:

blackbox_edit_end .env

========== PLAINFILE ".env"
========== ENCRYPTED ".env.gpg"
========== Encrypting: .env
========== Encrypting: DONE
========== UPDATED ".env.gpg"
Likely next step:
    git commit -m".env.gpg updated" ".env.gpg"

Decrypt the file for external processes

This method is useful for when you might need to make changes in the file through an external process, or maybe you just need it open for a while.

⚠️ Remember to shred or clear your decrypted files when you're done!

Code:

blackbox_decrypt_file .env

========== PLAINFILE ".env"
========== Importing keychain: START
gpg: Total number processed: 1
gpg:              unchanged: 1
========== Importing keychain: DONE
========== EXTRACTING .env

ls -la

...
-rw-r--r-- 1 username group   71 Jul 25 11:22 .env

Extending your use cases

With a bit of creativity and bash, you can extend these into even more useful operations.

One quick favorite of mine is to output a file's contents directly into the clipboard:

Code:

 # output the contents into your clipboard
blackbox_cat | xclip -selection c

 # windows or cygwin (I think)
blackbox_cat > /dev/clipboard

Conclusion

These are just a few of the most basic uses of Blackbox. It also offers commands for managing your registered keys or adding new users. You can also compare versions of files or see what has changed across all files. There's even some helpers for running batch commands from an automated system.

It should also be mentioned there are alternatives to Blackbox. Not all alternatives are created equally, and some might fit your scenario better than others. I've tested a few of them for my own use cases, but will leave you to form your own opinions on them.

Ultimately, after about a month of testing a handful of configurations, I have landed on the following:

• Bitwarden for password management
• Blackbox for vaulting
• Dashlane for "lower tier" passwords
• Automated ZSH scripts and environments to load configurations from each of the above

If you have any questions, or if there are any errors, please feel free to reach out to me and I will do my best to help you.

Remember to play around with files you don't care about first!

---

Edits:

7/24/2020
Added note about use cases outside of development.
Fixed coloring of terminal output in one code block